PVP breakage in Cabal releases

[ffaf1]

Describe the bug
As a dev team, we are failing in spotting PVP breakage.

To Reproduce
Last Cabal release (3.12.1.0) was a minor one (that is from from 3.12.0.0 to 3.12.1.0).
This means no PVP breaking changes should have been introduced.

Alas, at least two of those breaking changes are present in the release, see PRs #9766 and #9878. Both those PRs introduce a new constructor, hence the backport actually changed the API for Cabal.

Expected behavior
We don't let breaking changes slip in minor cabal releases, be it with the help of a checklist or automated testing or anything else (suggestion welcome).

[Kleidukos]

Can we use the tool GHC devs use to export the public interface of a module (library?) and have golden tests in CI?

[Kleidukos]

I'm thinking of this: https://gitlab.haskell.org/ghc/ghc/-/tree/master/utils/dump-decls?ref_type=heads

[ffaf1]

I will add that we need to decide what to do now, too, for 3.12.2.0. Revert the changes? Leave them there since most likely people have alreaded updated their code?

I opened a ticked specific for 3.12.1.0, #10161

[ffaf1]

For 3.12.1.0, go to #10161

[geekosaur]

I mentioned in #10161 that I believe this means we rushed 3.12.0.0 out too soon. I believe this because of all the stuff, including these two PRs, that we squeezed into 3.12.1.0 instead.

[Bodigrim]

Can we use the tool GHC devs use to export the public interface of a module (library?) and have golden tests in CI?

You can use https://github.com/phadej/cabal-extras/tree/master/cabal-diff.

[Bodigrim]

I mentioned in #10161 that I believe this means we rushed 3.12.0.0 out too soon.

It's imperative that a version of Cabal-syntax / Cabal, precisely matching one used as a boot library in GHC 9.10.1, gets released; no good would come from delaying it. If I may, I'd suggest releasing cabal-install simultaneously with Cabal next time, even if it feels unpolished and rough.

[Mikolaj]

I mentioned in #10161 that I believe this means we rushed 3.12.0.0 out too soon. I believe this because of all the stuff, including these two PRs, that we squeezed into 3.12.1.0 instead.

Thank you for the explanation of your reasoning. However, note that when the decision to release 3.12.0.0 was made at a Thursday meeting and process of the release was started by cutting the 3.12 branch, there was, by definition, zero pending backports (we cut release branches from current master, not from any older, tested version). We were 100% ready in terms of backports when we started the 3.12.0.0 release tasks. But, definitely, let's do a post-mortem at some later point and let's use the time to refine our inklings into well-thought arguments and recommendations.

[ulysses4ever]

@Bodigrim, thank you for all your input, it's very helpful.

I'd suggest releasing cabal-install simultaneously with Cabal next time, even if it feels unpolished and rough.

Sadly, the reasoning behind not releasing the tool at that time had a different ground: we were short-handed at that moment and no one had cycles to put the effort into it. At least, that's my recollection, and others should feel free to correct me if I'm wrong.

[Bodigrim]

Sadly, the reasoning behind not releasing the tool at that time had a different ground: we were short-handed at that moment and no one had cycles to put the effort into it.

The lesson I learned from maintaining other boot libraries is to keep HEAD always release-ready. Because depending on GHC release cycles or security vulnerabilities uncovered you might be pushed to make a release on a very short notice.

[Kleidukos]

The lesson I learned from maintaining other boot libraries is to keep HEAD always release-ready. Because depending on GHC release cycles or security vulnerabilities uncovered you might be pushed to make a release on a very short notice.

I will definitely take this advice for Haddock as well, thank you. :)