-
Notifications
You must be signed in to change notification settings - Fork 62
Add clickhouse-cluster-config to omdb blueprint output #6968
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
de8cef1 to
caa075b
Compare
Print clickhouse cluster config related tables for `blueprint show` and `blueprint diff` omdb commands. A bunch of the complexity and duplication here arises from the fact that we are diffing not only between blueprints that have identical structures, but between a blueprints and collections that have drastically different contents. This is useful, but we probably should consider a separating the two types of diffs and reworking all the blueprint diff logic to use some sort of semantic diff between types such as https://github.com/distil/diffus as recommended by @sunshowers. This still needs some manual testing, and potentially some expectorate based testing depending upon what those tests currently do.
caa075b to
d147f4c
Compare
|
Fixes ##6941 |
|
Expectorate tests added. Ready for review. |
jgallagher
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the long wait on this one! LGTM
nexus/types/src/deployment.rs
Outdated
| let Some(config) = &self.blueprint.clickhouse_cluster_config else { | ||
| return None; | ||
| }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tiny nit - I think this could be
| let Some(config) = &self.blueprint.clickhouse_cluster_config else { | |
| return None; | |
| }; | |
| let config = self.blueprint.clickhouse_cluster_config.as_ref()?; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Silly me. Thanks! Fixed in 81e85f5
nexus/types/src/deployment.rs
Outdated
| diff_table.keepers, | ||
| // Safety: The call to | ||
| // `ClickhouseClusterConfigDiffTables::single_blueprint_table` | ||
| // always returns all tables. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't looked at ClickhouseClusterConfigDiffTables yet, but leaving a note mostly for myself for when I get to it: could it return a different type so we don't have to unwrap here?
Edit: followed up in #6968 (comment)
| /// A table specific to a sled resource, such as a zone or disk. | ||
| // | ||
| // TODO: Rename to `BpTable` since it is also used for blueprint wide tables | ||
| // like those relating to `ClickhouseClusterConfig`? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd vote yes on this, I think? Nothing in this table is sled-specific, AFAICT; it's more like BpGenerationTable or something.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd vote yes on this, I think? Nothing in this table is sled-specific, AFAICT; it's more like
BpGenerationTableor something.
Done in f0ae7db
I thought this was going to be a larger change. I liked BpGenerationTable, but went with the shorter BpTable because of the ripple of changes to other type names.
| .. | ||
| }, | ||
| None, | ||
| ) => Some(ClickhouseClusterConfigDiffTables::removed_from_collection(before)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
W.r.t. my question earlier about the unwrap; I think the situation is:
- The only method that doesn't fill in
ClickhouseClusterConfigDiffTables::serversisremoved_from_collection() make_clickhouse_cluster_config_diff_tables()can therefore also return aserversvalue ofNone
I guess we could make a second diff table type where servers is not optional, return it from all the methods except this one and removed_from_collection(), and add a From impl to convert it into ClickhouseClusterConfigDiffTables? Doesn't seem terrible, but does seem like a fair bit of busywork to avoid a single .unwrap(). I'd probably do it, but if you'd rather not that's fine with me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the suggestion @jgallagher. I was unhappy with this unwrap as well, especially since the SAFETY message would become invalid if someone changed a method far away from that message.
Fix made in 81e85f5
Print clickhouse cluster config related tables for
blueprint showandblueprint diffomdb commands.A bunch of the complexity and duplication here arises from the fact
that we are diffing not only between blueprints that have identical
structures, but between blueprints and collections that have
drastically different contents. This is useful, but we probably should
consider separating the two types of diffs and reworking all the
blueprint diff logic to use some sort of semantic diff between types
such as https://github.com/distil/diffus as recommended by @sunshowers.
In order to make the
cluster_secretUUID generation deterministic fortests I had use an rng seed in the
BlueprintBuilder. This required movingcreation of the initial
ClickhouseClusterConfigand it's wrappingClickhouseAllocatorfromBlueprintBuilder::new_based_ontoBlueprintBuilder::build.Fixes #6941