Skip to content

Conversation

@bnaecker
Copy link
Collaborator

This adds more observability into the network protocol used to talk to ClickHouse, in an attempt to help resolve #8595

This adds more observability into the network protocol used to talk to
ClickHouse, in an attempt to help resolve #8595
.await;
let elapsed = now.elapsed();
probes::sql__query__done!(|| (&id));
probes::sql__query__done!(|| id.to_string());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's interesting; maybe we should figure out how to to_string() things that implement it rather than doing serde (which is--I assume--why you're doing this).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that'd be sweet, especially for types like this where there's a canonical form or just one piece of data. The {"ok": "xxxx-yyyy-zzzz"} serialization form is annoying in that case.

break Err(Error::UnexpectedPacket(kind));
}
ServerPacket::Data(block) => {
probes::data__packet__received!(|| {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm confused; were we getting this twice or something?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We were not. I removed this probe, subsuming it in the new packet-received probe that has the entire packet structure in it. This one is duplicative now.

));
Err(Error::UnexpectedPacket(packet.kind()))
}
Some(Err(e)) => Err(e),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we interested in probes for this condition?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The error condition? I think we get the invalid-packet probe firing in those cases, such as here. Does that seem like it covers it?

@bnaecker bnaecker enabled auto-merge (squash) July 17, 2025 19:48
@bnaecker bnaecker merged commit a45cd1b into main Jul 17, 2025
18 checks passed
@bnaecker bnaecker deleted the improved-clickhouse-probes branch July 17, 2025 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Oximeter frequently hit SQL query timeout errors on rack2

3 participants