Use a more cache-friendly representation for nested vectors

[eira-fransham]

This is especially an issue for Vec<Bytes>. If we're iterating over a jagged array of u8s we want to have as little indirection as possible. I wrote up a quick proof of concept that displays the qualities we'd need - immutable inner lists and a mutable outer one. It's 100% safe code even though I could get away with some unsafety to skip bounds checks - I don't make any use of the trustworthiness of JaggedArray's elements - and possibly wastes space because of the use of SmallVec. It might be better to use a ExtendFromSlice bound now that it's landed in smallvec. Worth a discussion, but I think that the increased cache- and size-efficiency (only a maximum of 2 wasted capacity allocations instead of 1 per element) is worth the hassle of moving it over.

I looked for a jagged array crate but couldn't find one. There's this but the entire thing is immutable rather than just the inner elements, so it's more like Box<[Box<[T]>]> instead of Vec<Box<[T]>>.

Honestly an easier method still would be to use an arena using the "Allocators Take III" API (I don't know what else to call it for now) but that's not going to be usable for some time, let alone stable.

[debris]

Closing this issue as it is very broad. If still relevant, please reopen it and define the scope of changes - describe which structures should be updated