Skip to content

Conversation

PIG208
Copy link
Member

@PIG208 PIG208 commented Aug 22, 2024

Blocked by #267, but these are migrations that can happen in parallel. I imagine that there won't be much overhead rebasing this kind of changes.

Fixes: #268

@PIG208 PIG208 force-pushed the server-5 branch 4 times, most recently from de9d000 to 6984723 Compare August 23, 2024 20:40
@PIG208 PIG208 marked this pull request as ready for review August 23, 2024 21:11
@PIG208
Copy link
Member Author

PIG208 commented Aug 23, 2024

The PR itself is ready, but we won't be ready to merge it until we refuse to connect to older servers. Going through some reviews will still be helpful, though.

@PIG208 PIG208 added the maintainer review PR ready for review by Zulip maintainers label Aug 23, 2024
Copy link
Collaborator

@chrisbobbe chrisbobbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for going ahead with this! This will be good to have ready to merge once #267 is done (that's on my plate right now). Small comments below.

// bool isAdmin; // obsoleted by [role]; ignore
// bool isGuest; // obsoleted by [role]; ignore
bool? isBillingAdmin; // TODO(server-5)
bool isBillingAdmin;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

api: Mark isBillingAdmin as requried, relying on server 5+, FL 73+

While `realm_user` update events also have this field added at the same
feature level, it remains optional.

See "Feature level 73" from Zulip API changelog:

   https://zulip.com/api/changelog

Signed-off-by: Zixuan James Li <[email protected]>

I would just leave out the sentence about how isBillingAdmin remains optional in RealmUserUpdateEvent. That's the normal, expected thing to happen; it matches all the other person fields, in that they might be null if they didn't change in the event.

Also nit in summary line: 'required"

Comment on lines -237 to -447
// This field is absent in `realm_users` and `realm_non_active_users`,
// which contain no system bots; it's present in `cross_realm_bots`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

api: Drop _readIsSystemBot, relying on server 5+, FL 83+

This comment still applies, right? The change made in FL 83 is just that is_cross_realm_bot stops being a name we have to care about. The comment can go above the defaultValue: false, and it seems like we should keep the test that confirms that false is chosen if is_system_bot is absent.

}
if (topic != null || prevTopic != null) {
// Both are present if the topic was edited
assert(topic != null && prevTopic != null);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition in this assert might happen because of unexpected data from the server. To validate data from the server, we'll want code that runs in production, and reserve asserts to check our own invariants that we maintain within the bounds of the app. Could replace the assert and comment with:

        // Crunch-shell validation: both are present if the topic was edited
        topic as String;
        prevTopic as String;

@PIG208
Copy link
Member Author

PIG208 commented Aug 27, 2024

Thanks! Updated the PR.

@chrisbobbe
Copy link
Collaborator

Thanks! LGTM except bump on this part of #904 (comment) :

it seems like we should keep the test that confirms that false is chosen if is_system_bot is absent.

@PIG208
Copy link
Member Author

PIG208 commented Aug 29, 2024

Got that test back. Thanks for the review!

@chrisbobbe
Copy link
Collaborator

Great! Marking for Greg's review once he's back from vacation.

@chrisbobbe chrisbobbe added integration review Added by maintainers when PR may be ready for integration and removed maintainer review PR ready for review by Zulip maintainers labels Aug 29, 2024
@gnprice
Copy link
Member

gnprice commented Sep 7, 2024

BTW there's generally no need to push a revision when it's a pure rebase, with no merge conflicts or other changes. The update causes notifications, and doesn't really save effort later — there'll typically be another rebase anyway just before merge, and git rebase is a very easy step to do at merge time when there's no conflict to resolve.

(As it happens I'm just about to take a look at this PR, but it wasn't because of the notification from the revision just now; it was from scanning the list of PRs marked for my review.)

Copy link
Member

@gnprice gnprice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @PIG208 for taking care of these, and thanks @chrisbobbe for the previous reviews!

Generally this looks great. Small comments below.

Comment on lines 208 to 228
// bool isAdmin; // obsoleted by [role]; ignore
// bool isGuest; // obsoleted by [role]; ignore
bool? isBillingAdmin; // TODO(server-5)
bool isBillingAdmin;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit in commit message:

api: Mark isBillingAdmin as required, relying on server 5+, FL 73+

See "Feature level 73" from Zulip API changelog:

   https://zulip.com/api/changelog

Signed-off-by: Zixuan James Li <[email protected]>

In the summary line, let's say User.isBillingAdmin rather than just isBillingAdmin. Similarly for fields in later commits.

That helps the reader place the field in the relevant context within the overall model.

In this case it also disambiguates things: there's an isBillingAdmin field over on RealmUserUpdateEvent too, and as a previous revision noted (as discussed in a previous review above), that one remains optional.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think the context info is pretty helpful. While the commit summaries get longer, it is justified for this kind of migration changes.

// This field is absent in `realm_users` and `realm_non_active_users`,
// which contain no system bots; it's present in `cross_realm_bots`.
return (json[key] as bool?)
?? (json['is_cross_realm_bot'] as bool?) // TODO(server-5): renamed to `is_system_bot`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit in commit message:

api: Drop _readIsSystemBot, relying on server 5+, FL 83+

Can focus the summary on what changed in the API, rather than on the method name which is an implementation detail of our (now-deleted) code:

api: Drop is_cross_realm_bot fallback, relying on server 5+, FL 83+

Comment on lines -37 to +39
final int zulipFeatureLevel;
final String zulipVersion;
final String? zulipMergeBase; // TODO(server-5)
final String zulipMergeBase;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes in the GetServerSettingsResult type are a bit sticky because this is where we learn the version of a server in the first place. For #267 we'll want to give a reasonable error message when the server is too old, and that'll probably include saying what version the server actually is (as well as the minimum version we support).

Ultimately we do want to keep modernizing this type, like the others. But there might be some wrinkles around how we handle it. I guess the bottom line is that the details of these particular changes might shift once we have an implementation of #267 in hand. (/cc @chrisbobbe)

Comment on lines 337 to 588
final int selfUserId;

final UserSettings? userSettings; // TODO(server-5)
final UserSettings userSettings;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should get a corresponding update in PerAccountStoreChecks.

(Similarly in other commits.)

final Uri realm;
final String email;
final int? userId; // TODO(server-5) new in FL 108
final int userId;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is one that especially benefits from identifying the class:

login: Mark userId as required, relying on server 5+, FL 108+.

so e.g.:

api: Mark WebAuthPayload.userId as required, relying on server 5+, FL 108+

Otherwise "userId" could be all sorts of things.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bump / nit: use api: for prefix, like for other API changes

(in this revision it's:

login: Mark WebAuthPayload.userId as required, relying on server 5+, FL 108+.

)

userId = int.tryParse(userIdStr, radix: 10);
if (userId == null) throw const FormatException();
}
int? userId = int.tryParse(userIdStr, radix: 10);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: since this is now initialized just once, make it final:

Suggested change
int? userId = int.tryParse(userIdStr, radix: 10);
final userId = int.tryParse(userIdStr, radix: 10);

Comment on lines 707 to 930
final int? userId; // TODO(server-5)
final bool? renderingOnly; // TODO(server-5)
final int userId;
final bool renderingOnly;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this summary line seems to cover only one of the three changes here:

api: Mark renderingOnly as required, relying on server 5+, FL 114+

Can say "fields on UpdateMessageEvent" to skip listing all three of them in the crowded space of the summary line.

Message origMessage, {
int? userId = -1, // null means null; default is [selfUser.userId]
bool? renderingOnly = false,
int userId = -1, // default is [selfUser.userId]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

Suggested change
int userId = -1, // default is [selfUser.userId]
int? userId, // default is [selfUser.userId]

That seems more conventional. The old code was odder because it had to deal with null being a valid value for the underlying field, and so not available to signify "use the default".

///
/// Gives null if the server reports that the message doesn't exist.
// TODO(server-5) Simplify this away; just use getMessage.
Future<Message?> getMessageCompat(ApiConnection connection, {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit in commit message:

See also: 631f4d68c58a84dff66a6a9a80322584f0d60152

Instead say:

See also commit 631f4d68c58a84dff66a6a9a80322584f0d60152.

That way it's clear this hex blob refers to a commit (implicitly one in this same repo).

PIG208 added a commit to PIG208/zulip-flutter that referenced this pull request Sep 10, 2024
Marked as TODO as this might change in the future.
See zulip#904 (comment).

See "Feature level 88" from Zulip API changelog:

   https://zulip.com/api/changelog

Signed-off-by: Zixuan James Li <[email protected]>
PIG208 added a commit to PIG208/zulip-flutter that referenced this pull request Sep 10, 2024
Marked as TODO as this might change in the future.
See zulip#904 (comment).

See "Feature level 88" from Zulip API changelog:

   https://zulip.com/api/changelog

Signed-off-by: Zixuan James Li <[email protected]>
PIG208 added a commit to PIG208/zulip-flutter that referenced this pull request Sep 10, 2024
… required, relying on server 5+, FL 116+

See zulip#904 (comment).

See "Feature level 116" from Zulip API changelog:

   https://zulip.com/api/changelog

Signed-off-by: Zixuan James Li <[email protected]>
@PIG208
Copy link
Member Author

PIG208 commented Sep 10, 2024

Thanks for the review! Added some TODOs commits for GetServerSettingsResult changes, making them easy to drop, and addressed the issues from the review. This revision should be fairly readable with range-diff.

@PIG208 PIG208 requested review from gnprice and removed request for gnprice September 10, 2024 18:26
Copy link
Member

@gnprice gnprice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the revisions! Just a few comments below.

zulipFeatureLevel: (json['zulip_feature_level'] as num).toInt(),
zulipVersion: json['zulip_version'] as String,
zulipMergeBase: json['zulip_merge_base'] as String?,
zulipMergeBase: json['zulip_merge_base'] as String,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem to have a corresponding change in realm.dart.

Presumably this came about as you made some edits and missed rerunning build_runner for one of them, or something like that. I am puzzled that our CI didn't complain, though — tools/check build_runner should have failed because this doesn't match what a freshly-generated file should say. So that would be worth trying to debug.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For future reference, this was fixed in #947.

Comment on lines 37 to 40
// TODO: Modernize this once we get to #267
final String? zulipMergeBase;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should keep a TODO(server-5) comment until we do actually modernize it. That way it naturally shows up in any future round of grepping for opportunities to modernize our API usage.

final Uri realm;
final String email;
final int? userId; // TODO(server-5) new in FL 108
final int userId;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bump / nit: use api: for prefix, like for other API changes

(in this revision it's:

login: Mark WebAuthPayload.userId as required, relying on server 5+, FL 108+.

)

Comment on lines -713 to +935

final List<MessageFlag> flags;
final int? editTimestamp; // TODO(server-5)
final int editTimestamp;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this doesn't seem accurate:

Fields including `userId` and `editTimestamp` are optional and act
as alternatives to `renderingOnly` for older server versions.

since those fields are no longer optional.

I think the commit message can skip explaining the fallback that previously existed for renderingOnly. It's directly relevant only when looking at the past; and the details are in this commit's diff.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, "were" would be more accurate here. Leaving this out makes sense.

Comment on lines 271 to 343
class UnreadDmSnapshot {
@JsonKey(readValue: _readOtherUserId)
final int otherUserId;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

api: Remove sender_id fallback for UnreadMessagesSnapshot.otherUserId, relying on server 5+, FL 119+

that's not the class this field is on 🙂

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good find! I should learn to not trust the diffs heading and be more careful from now on 😓

diff --git a/lib/api/model/initial_snapshot.dart b/lib/api/model/initial_snapshot.dart
index 565d89fe..6be29e78 100644
--- a/lib/api/model/initial_snapshot.dart
+++ b/lib/api/model/initial_snapshot.dart
@@ -269,15 +269,9 @@ class UnreadMessagesSnapshot {
 /// An item in [UnreadMessagesSnapshot.dms].
 @JsonSerializable(fieldRename: FieldRename.snake)
 class UnreadDmSnapshot {
-  @JsonKey(readValue: _readOtherUserId)
   final int otherUserId;
   final List<int> unreadMessageIds;
 
-  // TODO(server-5): Simplify away.
-  static dynamic _readOtherUserId(Map<dynamic, dynamic> json, String key) {
-    return json[key] ?? json['sender_id'];
-  }
-

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah, the diff-hunk headers can be misleading like that sometimes 🙂

@gnprice gnprice marked this pull request as draft September 16, 2024 23:48
chrisbobbe added a commit to chrisbobbe/zulip-flutter that referenced this pull request Aug 4, 2025
There's a planned change to make zulipMergeBase required in
GetServerSettingsResult (see zulip#904). When we do that, we'd otherwise
just treat a missing zulipMergeBase on an ancient server the same
way we treat any malformed response. Better to check if the server
is an ancient one we don't support, just like when the /register
response is malformed.
final int zulipFeatureLevel;
final String zulipVersion;
final String? zulipMergeBase; // TODO(server-5)
final String zulipMergeBase;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I've thought more about this, and while we don't rely on zulipMergeBase to decide if a server is ancient, it would break things to require it without also checking to see if the server is too old. Sent #1783 for that, which we should merge before coming back to this.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

api: Mark GetServerSettingsResult.zulipMergeBase as required

See "Feature level 88" from Zulip API changelog:

   https://zulip.com/api/changelog

This does not interfere with our handling of
disallow-connecting-to-ancient-servers, which does not rely on
GetServerSettingsResult.

In this commit message, let's just remove that last paragraph, or reword to explain that we do use GetServerSettingsResult, but it's OK because of #1783.

chrisbobbe added a commit to chrisbobbe/zulip-flutter that referenced this pull request Aug 6, 2025
There's a planned change to make zulipMergeBase required in
GetServerSettingsResult (see zulip#904). When we do that, we'd otherwise
just treat a missing zulipMergeBase on an ancient server the same
way we treat any malformed response. Better to check if the server
is an ancient one we don't support, just like when the /register
response is malformed.
chrisbobbe added a commit to chrisbobbe/zulip-flutter that referenced this pull request Aug 7, 2025
There's a planned change to make zulipMergeBase required in
GetServerSettingsResult (see zulip#904). When we do that, we'd otherwise
just treat a missing zulipMergeBase on an ancient server the same
way we treat any malformed response. Better to check if the server
is an ancient one we don't support, just like when the /register
response is malformed.
chrisbobbe added a commit to chrisbobbe/zulip-flutter that referenced this pull request Aug 7, 2025
There's a planned change to make zulipMergeBase required in
GetServerSettingsResult (see zulip#904). When we do that, we'd otherwise
just treat a missing zulipMergeBase on an ancient server the same
way we treat any malformed response. Better to check if the server
is an ancient one we don't support, just like when the /register
response is malformed.
Copy link
Collaborator

@chrisbobbe chrisbobbe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks mostly good, just a few commit-message comments, which we can fix up when merging. Marking for Greg's review.

final int zulipFeatureLevel;
final String zulipVersion;
final String? zulipMergeBase; // TODO(server-5)
final String zulipMergeBase;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

api: Mark GetServerSettingsResult.zulipMergeBase as required

See "Feature level 88" from Zulip API changelog:

   https://zulip.com/api/changelog

This does not interfere with our handling of
disallow-connecting-to-ancient-servers, which does not rely on
GetServerSettingsResult.

In this commit message, let's just remove that last paragraph, or reword to explain that we do use GetServerSettingsResult, but it's OK because of #1783.

final int zulipFeatureLevel;
final String zulipVersion;
final String? zulipMergeBase; // TODO(server-5)
final String zulipMergeBase;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

api: Mark InitialSnapshot.zulipMergeBase as required, relying on server 5+, FL 88+

See "Feature level 88" from Zulip API changelog:

   https://zulip.com/api/changelog

This does not interfere with our handling of
disallow-connecting-to-ancient-servers, which does not rely on
InitialSnapshot.

Similarly remove misleading last paragraph at the end; disallow-connecting-to-ancient-servers does rely on InitialSnapshot, in the sense that that's one of the places we learn if the server is ancient or not.

@chrisbobbe chrisbobbe removed the maintainer review PR ready for review by Zulip maintainers label Aug 7, 2025
@chrisbobbe chrisbobbe requested a review from gnprice August 7, 2025 02:45
PIG208 and others added 16 commits August 13, 2025 18:49
While is_billing_admin has become required since feature level 73, it then
got removed at feature level 363.

See "Feature level 73" and "Feature level 363":

   https://zulip.com/api/changelog
We weren't using this; and since it's deprecated, we don't expect
to ever use it in the future.

Because this doesn't appear in the current API docs (except in a
"Changes" note saying how it used to be there), it doesn't need
even a commented-out tombstone, either, like we have for isGuest
and some others.

(This change is NFC on all well-behaved servers; it's not quite NFC
only in that if a server now sends this field but with a value of
an unexpected type, the old code would reject that as malformed and
the new code won't notice or care.)
…n server 5+, FL 83+

See "Feature level 83" from Zulip API changelog:

   https://zulip.com/api/changelog
We'll shortly be making the corresponding fields non-nullable on
GetServerSettingsResult and InitialSnapshot.  That will potentially
make it look tempting to make this one non-nullable too; so explain
why this should be treated differently.
See "Feature level 88" from Zulip API changelog:

   https://zulip.com/api/changelog

This is one of the few places (along with registerQueue) where it's
important that we smoothly handle attempting to connect to an
ancient server where this field might indeed be missing.  Happily,
we do handle those since commit 733bb88 (zulip#1783).
…er 5+, FL 88+

See "Feature level 88" from Zulip API changelog:

   https://zulip.com/api/changelog

This is one of the few places (along with getServerSettings) where
it's important that we smoothly handle attempting to connect to an
ancient server where this field might indeed be missing.  Happily,
we've already arranged to do that even when the JSON doesn't match
our expectations: see the call site on UpdateMachine of the
constructor ZulipVersionData.fromMalformedServerResponseException.
This was done at the API boundary in commit e6f4576.  But the
resulting simplifications can be propagated a bit further.

See "Feature level 89" from Zulip API changelog:

   https://zulip.com/api/changelog
It is true that userId will always be present after feature level 114,
but it can still be null (as of July 2025).

So supporting Zulip 5.0+ only does not require changing the type of
UpdateMessageEvent.userId.

See CZO discussion:
  https://chat.zulip.org/#narrow/channel/412-api-documentation/topic/.60user_id.60.20in.20.60update_message.60.20event/near/2215174

API documentation:
  https://zulip.com/api/get-events#update_message
…relying on server 5+, FL 115+.

See "Feature level 115" from Zulip API changelog:

  https://zulip.com/api/changelog

See also:

  zulip/zulip#18067
…relying on server 5+, FL 115+

See "Feature level 115" from Zulip API changelog:

  https://zulip.com/api/changelog
…ired, relying on server 5+, FL 116+

See "Feature level 116" from Zulip API changelog:

   https://zulip.com/api/changelog
…, relying on server 5+, FL 118+

"topic" is no longer optional when the topic was edited, we also don't
need to expect "prev_subject" as opposed to "prev_topic".

See "Feature level 118" from Zulip API changelog:

   https://zulip.com/api/changelog

See also:

   https://zulip.com/api/get-messages#response
See "Feature level 120" from Zulip API changelog:

   https://zulip.com/api/changelog

See also commit 631f4d6, which introduced this wrapper.

Fixes: zulip#268
@gnprice
Copy link
Member

gnprice commented Aug 14, 2025

Thanks for the review (and for #1783)!

Made those commit-message edits and rebased. I also added a related comment in a small new commit:
506b87a server_support [nfc]: Explain why zulipMergeBase should remain nullable

I'll let CI run on this, and then I plan to merge.

@gnprice gnprice merged commit 36cfe99 into zulip:main Aug 14, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

integration review Added by maintainers when PR may be ready for integration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Require Zulip Server 5

3 participants