API breaks are not for free

[darosior]

The rust-bitcoin project has been pretty reckless with API breaks. This trend started about a couple of years ago. API breaks introduce costs and risks to downstream projects. Some of these projects are used in production. I believe as Bitcoin developers we should strive to minimize the risk to end users' money. Here i will argue most API breaks (sometimes even justified on vague "safety" rationale) since this trend started have effectively be, on balance, a net-negative to the safety of end users' funds.

First of all let me be clear that i do not claim API breaks should never happen. There may be good reasons for them. My point is there should be good reasons for them to happen.

Also, please appreciate how there is a dispersed cost of the code churn to all users of the library, whereas people actively following developments may be less impacted (and more inclined to take advantage of the benefits brought up by the changes). As such, readers of this issue (who are part of the latter group) may overestimate the benefits and underestimate the costs. For this reason i also don't think my projects are those most impacted. The large majority of impacted downstream maintainers would not bother working on a structured and justified writeup advocating for less API breaks. They would just patch the breaks mechanically (and sometimes incorrectly) or just not upgrade at all.

You are well aware of the potential gains from changes which necessitate an API break. Let me layout the costs. I can think of 3 main categories:

1. Code churn increases the likelihood of introducing bugs. The rust-bitcoin crate is a low level library core to a lot of Bitcoin projects written in Rust. Breaks of its API induce large refactorings in downstream projects. Refactorings, especially large ones, and especially of core parts of the software, risk introducing behaviour discrepancies with unintended consequences.
2. Opportunity costs. Besides the risk of defects, such large refactorings introduce maintenance burden: both for the developer writing the code and the one(s) reviewing it. This time could be affected to developments with higher impacts on the safety/security of end users than refactoring half the codebase because a core type was made "safer" by upstream and can't be used the same way anymore.
3. Overly complexified API for "safety" purpose gets bypassed. Related to the second point around costs, if the refactoring is too involved the user might aim for the simplest patch that compiles (thereby introducing bugs, again) or just opt out of using the types altogether.

These have led to the introduction of bugs and people not upgrading anymore: 72% of rust-bitcoin downloads at crates.io this month are for releases older than 5 versions back, and only 12% are for any of the last 4 releases. Of course people not upgrading means they don't benefit from bug fixes, which is another safety cost to take into account.

To substantiate my claim, here is a list of some past refactorings for which i believe the cost of the API break outweighed its benefits.

First of all a few instances of "let's break the API because this is more idiomatic / feels nicer":

• Implemented unsized Script #1155
• Remove ToHex #1531
• Re-name hash inner/byte methods #1577
• Rename from_slice methods to decode #1621
• Improve SighashCache API #1625
• Rename network module and move/clean up types #1854
• Rename PartiallySignedTransaction to Psbt #1938
• Make Payload and AddressEncoding private #1979
• Implement CompressedPublicKey #2277
• Rename txid to compute_txid #2366
• Default to forward for tagged hashes #2707
• Rename Siphash::as_u64 to Siphash::to_u64 #3119

Another interesting instance is #1489, which introduced massive refactorings for downstream projects and ended up creating at least a couple bugs i'm aware of because of points 1) and 3) above. Which is ironic, as the change was motivated by "safety" reasons.

Let me finish by mentioning this frustration has also been expressed by several developers in the industry. I hope they'll chime in and further support our case by sharing their own experience.

Could the rust-bitcoin project become more conservative with regard to API breaks? Thank you all for your work on maintaining and bettering this great library, i hope you'll take my feedback into consideration.

[stevenroose]

I think most contributors here know that I wholly stand with Antoine on this one. I have openly complained on a few specific occasions, but mostly done so in private conversations, as I wasn't sure whether I was just being the annoying oddball. I'm happy there are more voices confirming this frustration.

[Kixunil]

We had complaints a few times already but the issue is I don't really see a good solution other than to continue in what we already do. Nobody so far suggested an alternative that's clearly better. Including yourself. We want to stabilize at least the core types but we don't want to stabilize garbage API like people tend to do in other languages. Should we just work on new API completely separately and one day replace it in a huge rewrite everything event? That's kinda what's happening right now anyway.

Also note that there are currently only three active contributors with the worst possible combination of timezones possible, so any changes are really slow. My common response to "when stable" is "when will you help us make it stable?".

Another interesting instance is #1489, which introduced massive refactorings for downstream projects and ended up creating at least a couple bugs

I'd love to see what those bugs were. I've seen it catch bugs in production so unless we compare them it's hard to determine what is worse.

Regarding your list:

#1155 was definitely the right choice since some applications need the performance to parse whole chain etc. The API is million times better now.

#1531 was indeed an imperfect decision. The base idea was right but we should've just convert the method to an inherent. However I still maintain the position that in reality pretty much nobody should call the method since it allocates. Display is the right thing to use. If you had a horrible blow up when rewriting your code was likely slow.

#1577 improved the readability of the code significantly. Maybe we should've made the deprecation period longer. Sadly I'm still not sure if the names were good and maybe we should've removed the slice methods and name the array methods *_bytes.
This is related to Rust's previous poor support for arrays (especially > 32 elements) which got fixed only recently.

#1621 is pretty much the same kind as #1577

#1625 addressed a real issue in a real application and at the same time was really non-disruptive for all sensibly-implemented types. Every reference type from std worked without change plus you got to use Transaction on top of it. That's clearly a net benefit. Or are you suggesting that my application should have horrible code just so 0.1% of people don't need to write impl Borrow<Transaction> for MyNiecheType { fn borrow(&self) -> &Transaction { &**self } }?

#1854 is same kind as #1577 although I'd agree that the benefit is even smaller.

#1938 do you really enjoy typing out that long-ass name when we have a perfectly sensible, widely-used and widely-understood abbreviation? Though we should've kept the old name as type alias (maybe deprecated).

#1979 ironically that was a step to stop doing breaking changes in the future. If we didn't do it we would have to break it eventually for a different reason.

#2277 was extremely helpful in removing useless checks, error handling or panics all over the code. If your code was written correctly you should've just type-bubbled after upgrade. If it didn't work it uncovered design problems with the code.

#2366 same kind as #1577 though maybe the deprecation period should've been even longer.

#2707 was sensible but poorly handled. We should've first ensured that all code on upgrade will work correctly.

#3199 is in line with #1577 and we need to put back the deprecated method.

[apoelstra]

@darosior it's unfortunate you weren't able to come to the meetup in Nashville. We did discuss this at some length on the first day. One outcome was that BDK and LDK intend to just "freeze" on rust-bitcoin 0.32 so that they don't have to watch the churn in real-time, treating the next several versions as though we were just rewriting the API in private, even as we publish things on crates.io.

Meanwhile for our part, for the next long period we will be doing our changes in bitcoin-primitives and bitcoin-addresses and I'm not sure when we are going to release another version of rust-bitcoin. But when we do, the essential types will be reexports from primitives 1.0 which will be the same for all future rust-bitcoin releases (that's the dream anyway). So future "unnecessary changes" will at least be ones that you can take up at your leisure, not things that you'll be forced to do/not do just because bdk does.

Relatedly, the reason so many recent downloads of rust-bitcoin are for old reasons are because most users consume rust-bitcoin through bdk or ldk (or through fedimint, which is even further behind).

Regarding your specific issues, as @Kixunil has said a couple of them were really important to us (e.g. the Script/ScriptBuf thing and the Address thing) and if we had a limited breakage budget we still would've done those.

One in particular we did very poorly -- the ToHex removal. For our specific types there was a replacement (Display) on day one, but we failed to communicate that well and anyway there was no replacement for Vec::<u8>::to_hex (now there is the similarly-named as_hex and the front page of docs.rs/hex-conservative shows how to use it, but OK). For rust-elements I wound up copying the old hex module out of bitcoin_hashes. Examples of things I couldn't do were having my own Tweak type which wrapped a 32-byte array with some constraints. It wasn't a wrapper of a hash or any other rust-bitcoin type. It was a bare 32-byte array and we used to have a method to hex-serialize this because this is the most obvious fundamental thing a systems programming language should be able to do but the rustlib developers told me it was "too niche" and then we didn't. I'm sorry about that.

Then as for the pure renames -- as_u64 to to_u64 and such -- we should have a much much longer deprecation cycle for these. Like, "4 versions/2 years" unless we find ourselves ready to launch 1.0 and these deprecations are the only things stopping us.

[darosior]

Thanks all for the constructive replies.

Nobody so far suggested an alternative that's clearly better. Including yourself.

Well, i did. Sorry if i wasn't clear. Starting from the postulate that any change should be judged on a cost-benefit basis, i have argued that most of those changes (or the way they were done anyway) did more harm than good.

So the suggestion is: keep the current API if it's a lesser evil than changing it.

We want to stabilize at least the core types but we don't want to stabilize garbage API like people tend to do in other languages.

I wouldn't call the API from two years ago "garbage". Rust-bitcoin was already a pretty good library which did the job correctly. Sure the API perhaps wasn't the neatest or most idiomatic, but i'm not sure it's what users of this library cared most about. I'd venture hypothesizing they'd value correctness, robustness and minimal code churn over idiomatic APIs (to_ vs as_, txid vs compute_txid, Script vs ScriptBuf for the owned type, etc..).

Again don't get me wrong. I am well aware there is always a tradeoff between improving the library and minimizing code churn and API breaks. I'm just trying to argue the cursor was put at one extreme of the spectrum, which i believe isn't ideal.

My common response to "when stable" is "when will you help us make it stable?".

This starts from the perspective that rust-bitcoin is a new, unused, library. This does not match reality. The question isn't "when stable", but "could you stop making it unstable?". It was fairly stable before all this started. It was already used in production then, and is even more now. You can't just cover your eyes and act as if your decisions didn't have direct consequences on numerous downstream projects.

#1155 was definitely the right choice since some applications need the performance to parse whole chain etc. The API is million times better now.

This did not require breaking the API. What broke it is a rename, which was only done for stylistic reasons.

Besides, "make the API better" can't be the only thing you value. This is the whole point of this discussion.

So future "unnecessary changes" will at least be ones that you can take up at your leisure, not things that you'll be forced to do/not do just because bdk does.

I'm not sure we can assume most users of rust-bitcoin will be through BDK. Liana for one does not use BDK (as of now).

Regarding your specific issues, as @Kixunil has said a couple of them were really important to us (e.g. the Script/ScriptBuf thing and the Address thing) and if we had a limited breakage budget we still would've done those.

@Kixunil did not address my point about the API break. I do agree having an unsized Script type is useful, of course! I'm not arguing against that. The existing Script type was already owned, it could have been kept this way and the newly introduced unsized type could have been named differently. Instead it was chosen to rename the existing type into ScriptBuf and introduce the unsized type as Script just because "it matches Path/PathBuf".

My point was: if you take into consideration this change will break every single user of the library out there, "let's rename one of our most core types" should require a pretty strong rationale to be considered. And "this way it matches Path/PathBuf" strikes me as absolutely not meeting this bar.

[stevenroose]

This starts from the perspective that rust-bitcoin is a new, unused, library. This does not match reality. The question isn't "when stable", but "could you stop making it unstable?". It was fairly stable before all this started. It was already used in production then, and is even more now. You can't just cover your eyes and act as if your decisions didn't have direct consequences on numerous downstream projects.

I think this is the main point I have been trying to make for a long time. Somehow this "stable API" ideal is used as a motivation to actually make the API less stable. I've used rust-bitcoin in production since 2016 and the API was really quite stable until maybe 2019 or so. (This obviously also means that I'm biased towards keeping existing API, because I know it by heart and any change means I have to go back to rustdocs.)

I'm in the same boat of absolutely loving the beauty of a well-designed Rust API, but if there is already an existing API that works and is being used by people, making it "better" should come with rough agreement from its users that it is actually an improvement. And the statistics show that users are not agreeing. The argument that "most of these old version are through BDK and LDK", is exactly showcasing the issue. If even entire "at the forefront of Rust bitcoin development" projects like BDK and LDK are believing it's not worthwhile to update their rust-bitcoin version (foregoing bug fixes and improvements), I feel there is clearly a problem. That's not mentioning all private projects of which I know numerous ones that stopped caring about rust-bitcoin updates, presumably because the cost to update outweighs the perceived benefit.

I typed a bunch of snarky comments in response to some of the comments above on specific items. I'll refrain from sending them, but instead drop some really simple things that could have been done differently that would have had a way lower impact on API, just to show that many breaking changes were not needed to achieve the desired improvements:

• /// Compute the txid of this [Transaction]. Note that this is not a getter and the txid is computed on every invocation.
• BorrowedScript
• impl<T: DisplayHex> ToHex for T

Some other things are just not really worth the effort, sure some names are suboptimal, but few people really care: to_ vs as_, Inner vs Bytes, PartiallySignedTransaction vs Psbt. Often a type alias with uber-long deprecation period (really no harm in just keeping a deprecated alias around forever) is enough.

[stevenroose]

Ultimately, repeating Antoine's response to

My common response to "when stable" is "when will you help us make it stable?".

People use rust-bitcoin. They are using the API as-is and as long as they don't complain, they are probably pretty content with it. Reasoning "wouldn't it be nice if instead..." cannot be sufficient motivation for breaking something that already works. "I've had problem X in my project and I believe a change Y would have avoided this or improved my ability to avoid/solve this problem" is better.

[yancyribbens]

The argument that "most of these old version are through BDK and LDK", is exactly showcasing the issue

And that's the tip of the ice burg I would say. Products that use BDK and LDK downstream which sometimes also depend on rust-bitcoin are way behind in keeping up to date, even though in some cases they are startups that only recently wrote their products (like within the last few years). I'm not unsympathetic to how a startup is just trying to get a product to market let alone trying to keep up with each nitty api change. Although, I would also say, if people want stability in an API and language, Rust is definitely not the choice.

I have openly complained on a few specific occasions, but mostly done so in private conversations

While I appreciate the conversation here, It's things like this that I am most frustrated with. There seems to be a lot of back room talk and old boys club agreements and to me that's not helpful. I'm glad to see a constructive conversation on this so everyone is on the same page.

[Kixunil]

Starting from the postulate that any change should be judged on a cost-benefit basis, i have argued that most of those changes (or the way they were done anyway) did more harm than good.

To do this, we either need some objective way to measure it, which I don't know about (LMK if you do) or rely on our intuition which we already do and is unlikely to change until someone replaces us.

Rust-bitcoin was already a pretty good library which did the job correctly.

Maybe some code "smells" more to me but I saw a huge amount of problems. Types being wrong, invariants unprotected, From panicking, ..., and most importantly, the Taproot API was a horrible mess. In one project just updating bitcoin version led to deletion of >400 lines of code that was either reimplementing the same things or worked around the limitations. This is a big deal.

correctness, robustness and minimal code churn over idiomatic APIs

Those often go against each-other. Idiomatic APIs are idiomatic because Rust developers already figured out they prevent footguns. Though renames are arguably an extreme case.

I'm just trying to argue the cursor was put at one extreme of the spectrum, which i believe isn't ideal.

The reason we did that is specifically to stabilize - stop doing it ASAP. This is pretty much exactly how Rust itself was developed and I believe the choice they made was right. BTW have you considered the possibility that Rust is not the right language for you?

The question isn't "when stable", but "could you stop making it unstable?".

Nope, that was the literal question I got. If people want a much worse API they can just use the old version and ask us to backport important fixes which we already do anyway.

It was already used in production

If your "production" is not accounting for added resources needed to upgrade a library that is clearly unstable (the leading 0) then you need to fix your decision process.

This did not require breaking the API. What broke it is a rename, which was only done for stylistic reasons.

Even without rename we would need to break API since deref coercions don't apply in some rare cases. (But OK, the fallout wouldn't be that bad.) But you can't claim it's just "stylistic" if it's literally consistent with std with the clear intention to make it easier to understand what it does. Further, this change was surprisingly easy to perform with simple find&replace tools. I did it myself, just s/Script/ScriptBuf/g on all files did most of the job with just a few adjustments followed by changing &ScriptBuf to &Script. (Also automated.)

Besides, "make the API better" can't be the only thing you value.

Well, it pretty much is right now because the project is 0.* and the same thing worked well for Rust so why not repeat it?

I'm not sure we can assume most users of rust-bitcoin will be through BDK. Liana for one does not use BDK (as of now).

Doesn't matter, the point is the same.

@Kixunil did not address my point about the API break.

I thought your main point was "this is too much" so I asked you to suggest a more concrete change. If your point is different please state it more clearly.

it could have been kept this way and the newly introduced unsized type could have been named differently

I did actually think about it but couldn't find a satisfactory name. Now I'm thinking maybe Scr would be a good one from your perspective but every newbie would kick me in the ass for picking such name. Also nobody objected to it. Maybe in the spirit of "help us make it stable" you could watch the repo and suggest things like better names?

And "this way it matches Path/PathBuf" strikes me as absolutely not meeting this bar.

Making names consistent with something people already know and understand seems good enough to me if the alternatives are not great. But you can also use bitcoin::ScriptBuf as Script; if you hate the renames so much.

Somehow this "stable API" ideal is used as a motivation to actually make the API less stable.

The API is equally unstable, we just rush breakages that would happen anyway more.

API was really quite stable until maybe 2019 or so

Well, the API had a ton of problems that few people saw. It had a bunch of C++-isms and other things that actually caused bugs, confusion or performance issues. It even had blatantly wrong stuff like panicking From.

making it "better" should come with rough agreement from its users that it is actually an improvement

How if almost no users communicate? Note that there are also users who like our changes and they don't need to speak up for something that's good for them so the conversation will always be biased towards whoever is dissatisfied.

The argument that "most of these old version are through BDK and LDK", is exactly showcasing the issue.

But it also showcases that simply staying on an older version is not too big of a deal?

• BorrowedScript

That would be a completely wrong name for the what is currently called Script and if you're suggesting BorrowedScript<'a> then that would be multiple times worse because we'd have to duplicate everything for & and &mut. The only reasonable longer name that I can think of is ScriptSlice, which being longer disincentivizes people from using it instead of owned Script.

• impl<T: DisplayHex> ToHex for T

Nah, we should've just had inherent methods. But if you're doing a ton of to_hex calls your code is not great anyway.

Often a type alias with uber-long deprecation period (really no harm in just keeping a deprecated alias around forever) is enough.

Yes, I do agree our deprecation period should be longer - until 1.0 or if we have to remove stuff for other reason. Note that there's a problem in Rust itself that #[deprecated] does nothing on pub use items. I recommend you to help the Rust devs fix that if you care about it.

They are using the API as-is and as long as they don't complain, they are probably pretty content with it.

I'm among the people who use it. I'm not content and I complain a lot. And I'm actually putting in the work to make the changes.

"I've had problem X in my project and I believe a change Y would have avoided this or improved my ability to avoid/solve this problem" is better.

That's the exact thing I'm doing.

I'm not unsympathetic to how a startup is just trying to get a product to market let alone trying to keep up with each nitty api change.

I work for a startup. Upgrading the library is not really that big deal. The inefficiencies are elsewhere.

Although, I would also say, if people want stability in an API and language, Rust is definitely not the choice.

Exactly, at least not now.

There seems to be a lot of back room talk and old boys club agreements and to me that's not helpful.

The irony is this includes me - people often refer to some API problems behind my back and never show me the code they find problematic. And often when I ask for a specific snippets/links I get nothing. So how am I even supposed to know what's actually a problem? I might be the biggest advocate for "clean up the API while we can" approach, yet nobody pings me about specific issues. 🤷‍♂️

Anyway, this whole issue is ultimately a struggle between various people wanting different tradeoffs. In the extreme, if we stop our work and stop doing breaking changes we will satisfy people that are complaining now but will piss off people who are not complaining now. We have currently no mechanism to decide which is more valuable. But hopefully we can support more people if we stick with the plan and release primitives 1.0, slowly progressing from there.

[apoelstra]

Broadly I agree with you @Kixunil but I don't think it's fair to say "0.x means unstable and therefore you just need to deal with the instability if you use this". In some sense this is true, in that the people directly sponsoring the project aren't directing us to be more conservative and nobody else is owed anything...but in practice it's not reasonable for people to develop their own entire rust-bitcoin library.

Aside from this library being the center of gravity and having multiple Rust and Bitcoin experts reviewing and contributing to it, Rust itself forces network effects by its use of strong types and making types incompatible if they come from different (versions of) libraries. So if you want to use Rust in production on Bitcoin software you are more-or-less forced to use this library. And if we're making gratuitous changes or even good changes with bad tradeoffs ... people have a right to complain about this and to be upset with us.

It's also not fair to say "maybe Rust isn't for you". Yes, the crates.io ecosystem is full of crates that are permanently 0.x, who hide behind this to justify constant API breaks (while refusing to go to 1.0 because it'd take away their ability to do this). But this is clearly a problem with the crates.io ecosystem, not with users who hope for something better. To the extent possible we should push against the temptation to do this.

In retrospect I think a lot of the pain would have been avoided if we had started working on stabilizing primitives and addresses years ago but for a long time I was dead-set on having everything in one giant crate to reduce the apparent size of users' dependency trees. Which in turn made every upgrade a minor instance of libc-pocalypse.

Though I guess no matter what the ScriptBuf thing, the Address<Unchecked> and the hex thing were going to be a giant PITA. (I think push-based encoding may be similar, but as long as we retain the deserialize/serialize functions and the hex API it might be okay.)

[tcharding]

Thanks @darosior for raising the issue, hard conversations are definitely worth having. Opinions are always going to differ and when there are tradeoffs involved each individual dev/project may not see the same cost/benefit. The whole 'Rust kind of forces everyone to be on the same version' is also a real thing.

It might not be obvious but we (@apoelstra, @Kixunil, and I) are attempting to do the best we can for many projects/users going forward for a long time. If I've understood you (and @stevenroose) I believe some of the differences in opinion come from how long you guys have been using the library. It may interest you to know that I am the only person paid to work on this library, @Kixunil does it because he wants to consume the library, @apoelstra because he started it.

One other thing possibly worth mentioning, I started working at the time that you mention starting to experience pain so some of it is my fault. The 0.29 -> 0.30 upgrade was extremely invasive, and I take full responsibility for that, I was new and changed a lot of things in one go. There was no excuse for that excessive upgrade pain, and we are sorry. I believe that the subsequent upgrades have been simple, if not trivial.

FTR there are no gentleman's clubs. All dev happens here in the clear, some people message me on Signal occasionally if they don't want to say it in public and I usually mention it online. Andrew and I chat on the phone once a month. I occasionally send a group email to all the listed maintainers if I want to be really sure they saw something but since starting to use discussions I have not felt the need to do so as often.

Finally, we had an in-person meet in Nashville last month, everyone in this thread was, I believe, invited to come. It was invite-only but the invite condition was "send this invite to anyone you know who writes Rust Bitcoin software". There will be another event next year, we are hoping to do it on an approximately annual basis. Understandably some folk don't like traveling to the US but at this stage the next one is in the US again. And because all good posts should include a random quote "America, the land of the free. Whoever told you that is your enemy" - Rage Against The Machine.

[Kixunil]

Rust itself forces network effects by its use of strong types and making types incompatible

I don't think this is really a property of the language. If this were e.g. Python and we changed something the code would just get silently broken in an equivalent situation. Thanks to this being Rust we can do it much more safely since we get the compiler explain what is broken and also it enforces visibility rules.

So if you want to use Rust in production on Bitcoin software you are more-or-less forced to use this library.

Yeah, that's true. On one hand it sucks for the users, on the other it incentivizes a single ecosystem sharing the types (although they are not really the same right now since 0.32::Transaction != 0.33::Transaction).

while refusing to go to 1.0 because it'd take away their ability to do this

I'm not sure if this is the case. It might be for some people but I think most are just unsure what the ideal API should look like. Also as we've seen first hand with our Taproot API, you don't learn what's right API until you start actually using it.

In retrospect I think a lot of the pain would have been avoided if we had started working on stabilizing primitives and addresses years ago but for a long time I was dead-set on having everything in one giant crate to reduce the apparent size of users' dependency trees. Which in turn made every upgrade a minor instance of libc-pocalypse.

Yes, that's true. I think this is also a lesson for us that breaking Rust's conventions can get us burned significantly, so we should be very careful about that. If the choice is breaking Rust's convention vs people having to get used to how Rust does thing we should strongly prefer the latter - they'll need to get used to stuff sooner or later anyway because of some other crate. Also things like "my dependency tree contains more than one bitcoin line" are basically sticking head into the sand. The reality is you either need to get all code reviewed or you need to get a signature from someone you trust. The irony is that with smaller crates the review is much easier.

Though I guess no matter what the ScriptBuf thing, the Address<Unchecked> and the hex thing were going to be a giant PITA.

Exactly. Our APIs suddenly started returning/accepting unsized types and it's also good to just not use &ScriptBuf so people might want to change those anyway.

I think push-based encoding may be similar, but as long as we retain the deserialize/serialize functions and the hex API it might be okay.

It definitely will for people using the current consensus encoding traits but we keep the module and backwards-compatibility. Maybe we can make the forwarding macro public too.

Also tagged scripts will be about as bad as the change to unsized and I don't think there's anything we can reasonably do about that. We could have Script<Untagged> and allow users to cast it to other script types but they should rather update their types, not do weird casts.

The whole 'Rust kind of forces everyone to be on the same version' is also a real thing.

Only for entities (Clean Architecture term). And sadly, this crate mixed entities with non-entities. We need to separate things that change often from those that change rarely.

The 0.29 -> 0.30 upgrade was extremely invasive, and I take full responsibility for that

It was going to be massive either way, you didn't change that much. We also had had a cool upgrade guide which counteracted some of the pain. Maybe more things should've been deprecated rather than removed/renamed but note also that #[deprecated] on use is broken and that made some deprecations impossible. To be clear my point is don't beat yourself up too much even if you somehow made situation worse.

FTR there are no gentleman's clubs.

There is sometimes feedback from downstream user to maintainer though.

Understandably some folk don't like traveling to the US but at this stage the next one is in the US again.

There is also BTC Prague dev/hack/day which while being more broadly scoped allows for having rust-bitcoin workshops. If someone makes interesting Open Source Bitcoin software I might be able to get you an invite.

[elsirion]

I'll stay out of the stability <> perfect API discussion, I've been on both sides, wouldn't want to stabilize annoying APIs either and have probably cause my fair share of upgrade pain to people in the past. So I'll just describe the upgrade problems and possible solutions I can see from my Fedimint perspective:

The main problem with rust-bitcoin upgrades is that we have to rip out and replace the entire ecosystem at once (bitcoin, miniscript, lightning, bitcoin_hashes, and some smaller crates). For a project with over 100k LOC that's merging ~4 PRs a day when everyone is online that's not feasible to naively do in one go. From our side there were two options for the last migration:

1. Write a script that can auto-fix nearly all the incompatibilities, so a PR can be prepared in under an hour while we coordinate stopping merging stuff
2. Going crate-by-crate and adding conversion functions between the old and new rust-bitcoin type versions

Last time we went with the latter, it wasn't pretty but it worked and I doubt the first option would have been much better without a more semantically aware version of sed.

I'm excited to see how primitives will reduce that problem, it sounds like a great idea!

If you have to break primitives you could import the previous version of the crate and implement bidirectional conversion between all old and current versions of types to make upgrading easier. That's what tokio did pre-1.0 iirc and can likely be automated with proc macros. Although I totally see that this could be a lot of work and if I don't do it I don't have grounds to complain (and we'll just manually impl conversions for the types relevant to Fedimint I guess).

Staying on older rust-bitcoin versions hasn't been too much of a problem for us so far (currently on 0.30.2), we'll likely upgrade again when we need some new functionality from LDK or if there is some critical issue. Lmk if that causes problems to the rust-bitcoin project though, from my side just skipping a few versions and having to deal with the fallout less often seems preferable.

Rust-bitcoin is already great and fit for our purposes, I respect your drive to make it even better (I'd do mostly the same as a maintainer) and we'll deal with the incompatibilities that arise somehow.

[yancyribbens]

@darosior while we are on the topic, maybe you want to weigh in on #2585, see #3025. I've been asking to revert this change and maybe your feedback would help.

FTR there are no gentleman's clubs

This gave me a little chuckle. After this comment, the items all make me think of an old boys club, including invite only meetings in Nashville. I heard rumor of a meeting Nashville, although I'm not sure I'd ever call it a straight invite. Not that I'm mad though, I'd prefer to spend my time doing things I enjoy (like making contributions here). Although, I do get mad when people start giving my work a thumbs down without apparent reason after what I imagine to be closed door discussions in Nashville.