Very large performance regression from 0.22 to 0.24

[jyn514]

(I'm aware 0.25.1 is the latest version, if this is fixed on master let me know and I can close this. 0.25 removed RcDom so it would take some set up on my end to switch)

Docs.rs uses html5ever to parse HTML files generated by rustdoc. We recently updated from 0.22 to 0.24 and docs.rs started using so much CPU that all requests turned into 500 errors. The 0.24 version was deployed from 20:04 to 20:20 in the following diagram:

See also rust-lang/docs.rs#669 (and many other PRs linked from there, we had to bisect in production).

Do you know what could have caused the slowdown? I saw that no new dependencies were introduced so it must have been something internal to html5ever. I can try to reproduce this on a single file if you're not sure, but right now it's preventing us from upgrading.

[jdm]

Can you share an example document that shows a measurable performance difference when parsing it?

[jdm]

My only guess as to what could have changed is 81ea581; none of the rest of the commits look like they should have an impact on parsing performance.

[jyn514]

I can try, it might take me a while.

[jyn514]

It can't have been 81ea581 because that's only part of 0.25, not 0.24 (according to github's tag parsing)

[jyn514]

Ok I managed to reproduce with the docs.rs code, although the slowdown doesn't seem to be enough to explain the CPU spike above.

With 0.22:

Benchmarking parse regex html: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 51.2s or reduce sample count to 20.
parse regex html        time:   [10.066 ms 10.095 ms 10.123 ms]                              
Found 19 outliers among 100 measurements (19.00%)
  19 (19.00%) high mild

With 0.24:

parse regex html        time:   [13.598 ms 13.630 ms 13.665 ms]                              
                        change: [+35.060% +35.431% +35.778%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

The code under test: https://github.com/rust-lang/docs.rs/blob/5c389cabe8629483f00261dfb455dd842dc05940/src/utils/html.rs

Benchmark code:

extern crate criterion;
extern crate cratesfyi;

use criterion::{black_box, criterion_group, criterion_main, Criterion};
use cratesfyi::utils::extract_head_and_body;

pub fn criterion_benchmark(c: &mut Criterion) {
    let html = std::fs::read_to_string("benches/struct.CaptureMatches.html").unwrap();
    c.bench_function("parse regex html", |b| b.iter(|| extract_head_and_body(black_box(&html))));
}

criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);

HTML file:
capture_matches.zip

Let me know if you want me to try and reproduce with html5ever directly.

[jyn514]

Benchmarks with 0.25 and kuchiki still show a big slowdown, but not as bad as 0.24: rust-lang/docs.rs#671 (comment)

[jdm]

We probably won't be able to investigate this until sometime next week due to deadlines, unfortunately.

[jyn514]

Just checking on the status of this, you mentioned you might have time to look at it this week? No problem if not, this isn't urgent for us.