gh-135953: Add flamegraph reporter to sampling profiler

[lkollar]

Add --flamegraph option to the sampling profiler to produce a self-contained HTML report.

• Issue: Expose internal stack introspection APIs as a statistical runtime analysis tool #135953

[AA-Turner]

I don't think we should reference external hosts/CDNs like this, previous JS hosts have ceased operations & we shouldn't make an implicit dependency between them (plus, security implications of random downloads).

Is it possible to take a leaf out of flamegraph.pl's book and produce a file that doesn't need external resources? From memory, the produced SVG there 'just works' with any SVG renderer.

[lkollar]

Fair point. An SVG wouldn't be able to replicate the functionality though as this is a dynamic viewer which lets users explore the flame graph interactively. An alternative option would be to vendor the JS libraries and bundle them into the output when we generate the HTML.

[AA-Turner]

I don't know enough about the trade-offs between a viewer implemented in SVG vs JS to usefully comment on what implementation language is chosen. Vendoring the referenced JS files here would make sense to me, but we should be aware of the trade-offs -- the standard library becomes responsible for the security content of that JS, etc.

A different option might be to put a minimal 'viewer' in the stdlib and have fancier options as PyPI packages. That would keep the maintence burden to the core team lower, as this PR does introduce a fairly large chunk of not-Python (HTML/JS/CSS) into the stdlib.

[lkollar]

I understand the concern about vendoring JS files, but I think the self contained nature of the generated HTML is a huge strength. It means no extra tool is needed, you can just open it in a browser. For example, you can perform a profiling session on a headless production server, copy the HTML to another machine, and immediately open and explore it without installing anything else, or having to take other steps. This also makes it very easy to share results with others.

If we remove the JS files and try to keep it more minimal, then we lose the interactivity and users will need to take extra steps to get this functionality. For flamegraphs this makes the result less useful, since without the ability to filter or drill down into frames it becomes much harder to analyse and find bottlenecks.

So I see the maintenance and security concerns with vendoring, but in my opinion there is a clear benefit here. Being able to get an interactive standalone performance report directly from a Python program with no dependencies and no extra setup is a very powerful feature. For this reason, I'd argue for vendoring these JS files over other alternatives.

[lkollar]

The Linux perf tool does something very similar: https://github.com/torvalds/linux/blob/b320789d6883cc00ac78ce83bccbfe7ed58afcf0/tools/perf/scripts/python/flamegraph.py#L36-L37

I definitely agree that vendoring is better than linking to external hosts as perf does, as it will ensure that reports will work forever even if the CDN doesn't exist anymore, and it also makes it possible to use the reports in environments without internet access. We just need to add a build step where we embed these in the HTML.

[AA-Turner]

Thanks for this reasoning and the thoughtful responses. I agree that JS is definitely the better choice for a nicer UI & more features, etc. Pragmatically, it probably makes sense to move forward with this PR, switching to vendored JS sources.

The reason I'm pulling on this thread though is that vendoring a large chunk of JS code is something new for us, and I want to make sure we've properly considered the ramifications, especially over a 5+ year support window, given how quickly various JS libraries can move and update.

Other vendored code in the repo is, as far as I am aware, either fairly stable C code that's not exposed at the Python level, or the ensurepip wheels that are vouched for by a long history and core developer overlap with the pip project. Indeed, we're trying to reduce vendoring in some places (eg we removed the setuptools wheel, libmpdec is slated for de-vendoring).

It might make sense to start a Discourse discussion about what our views are to vendoring code like this, though I don't know if I'm overblowing the issue of 'different programming language' too much here.

A

[pablogsal]

Thanks for the thoughtful consideration, @AA-Turner. I appreciate you taking the time to think through the long-term implications.

I'd like to offer a slightly different perspective on the risk profile here. The JS libraries we're vendoring (d3.js and the flamegraph components) are quite mature and stable for this use case. They're widely used including in similar high profile places like the Linux kernel's own perf tooling. The code we're including is essentially "done" and doesn't need regular updates or maintenance like a typical dependency. The only pourpose is to show the flame graph. Also there is no backwards compatibility here as long as it shows the results and we could easily swap it for something in the future if we need to.

More importantly, these are purely client-side visualization libraries that generate static HTML/JS files with no network connectivity or external dependencies once created. They're essentially self-contained data presentation tools, similar to how we might include CSS for formatting. They just happen to be written in JS for interactivity.

Given that this pattern is already established in tools like Linux and other profilers (which has been stable for years), and considering the significant user experience benefits of having these work out-of-the-box without external dependencies, I think the risk-benefit calculation here is quite favorable. The security and maintenance overhead is minimal since these files are essentially static assets.

While I understand the desire to be thorough about new patterns, I suspect a broader Discourse discussion might be more overhead than this particular case needs and will likely not yield any focused discussion. If you feel strongly about this I am happy to open a discussion about it but I would recommend vendoring it for now and addressing any issues if they actually materialize. We have plenty of time to discover problems here. I will also check with the security developer in resident in case he sees something I am missing.

how does this sound?