Consider using iframes for rustdoc HTML files

[jyn514]

One of the things #671 (comment), #668, servo/html5ever#414 show is that the live server's performance is very heavily tied to the html parsing (outside of builds). It would be better to instead serve the rustdoc HTML inside of an iframe, but that would require a lot of UI testing we don't currently have.

Related issues: #595, #167

cc @Mark-Simulacrum, @pietroalbini

[Mark-Simulacrum]

An alternative to iframes or similar is to instead pre-run HTML parsing on rustdoc output and store the header and body and classes separately, such that we could then template them in quickly.

[jyn514]

That doesn't help with the existing HTML content, though. Also it would triple the number of files we store on S3, which would have some overhead.

[Mark-Simulacrum]

Well, we could store new files as such (which would over time migrate us over to the new format). With regards to S3, there's no reason we can't pack in theory, for example storing the HTML like: u64 length of head, head raw, body raw

[jyn514]

Ok, that sounds feasible at least. The other things we would get from inline frames though are

• less network time since we don't have to download the docs ourselves, only link to the s3 bucket
• less processing time since we don't have to do string formatting on files up to 45 MB
• clear separation of rustdoc JS from docs.rs JS (which allows us to make a more strict CSP)

[pietroalbini]

We should not use iframes:

• Search engines don't recognize the content inside iframes as part of the website's content. This would reduce docs.rs's visibility in search engines a ton.
• We can't serve files from S3 directly, S3 downloads from the internet are way more expensive than our CloudFront setup.
• Providing a good UX when clicking links in iframes is not easy.

If we want to remove parsing, Mark's solution is more feasible.