Create large pageindex regions memory-efficiently

[stephenrkell]

Large bigallocs create large runs of the same 2-byte value in the pageindex. When we fill multiple system pages with this repeating 2-byte value, we should able to use CoW techniques to avoid spending physical memory unnecessarily. I think I have some recipes that can do this, using splice() on Linux.

This might allow us to relax our practice of not indexing the pageindex itself within the pageindex (relevant to #16) -- or, more generally, of excluding very large mappings from the pageindex.

[stephenrkell]

Update: annoyingly, it seems these splice-based recipes don't quite work, and one that would have worked, using remap_file_pages, no longer does following the replacement of that (deprecated) system call with a less efficient emulation.

We still want something like this, though. It's possible some other recipes would work, or we can use SIGBUS to fill these areas lazily like we do for the zero-filled pageindex.

[stephenrkell]

I think SIGBUS is the way to go. We keep a list of lazily filled areas. The pageindex's own pageindex area becomes one of these, as is any that would otherwise be too large according to BIGGEST_SANE_USER_ALLOC or whatever our current test is. When we populate parts of these areas, on handling the signal, we don't need to chop out that range from our list, as long as we don't change the contents from what is implied by that entry. So maybe we need to get a write fault later, if we modify that part of the pageindex? Or just get it right in memset_bigalloc and be sure that we don't use any other function to update pageindex.

(Should we be protecting pageindex and big_allocations from user writes? Ideally yes, of course. This comes back to the "supercalls" concept, i.e. that calls into liballocs should cross a privilege level. We can't do a perfect job given only Intel hardware, but we can do some hardening... possibly with memory protection keys? The rough idea would be to wrpkru on entry to liballocs, to set the write key, and again on the way out to clear it, while also doing some libsystrap-style "paranoid scanning" for wrpkru in executable mappings so that user code does not update that register, at least not without our vetting the update.)

As with all signal handling, it'd be nice if our use of SIGBUS did not interfere with the guest program's use of the same signal. I had some ideas in a relevant libsystrap issue, but I feel there are multiple recipes needed for signal sharing: handler dispatch chaining (e.g. for guest SIGILLs), address-space demuxing (for SIGSEGV and SIGBUS), and possibly others. For my blog I've been working on a post about signals, to get straight in my head the (many) different cases/kinds of signal handling, which I should finish before attempt the design work here.