Skip to content

Conversation

@trinity-1686a
Copy link
Contributor

Description

add the node_id to emitted spans, making traces easier to interpret

How was this PR tested?

spin a one node cluster, make it trace to itself, and verify the new field is shown in jaeger

}
}()
.await
.unwrap_or_else(|| "unknown".to_string());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the default value contain a random number in it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it won't be useful because the 2 situations where this is emitted should be:

  • the command is not one that behave like a node at all
  • we couldn't read the config file, and assuming that stays true, the node will crash shortly

but there is little harm at doing it, so why not

// Except for race conditions, an error will get logged the 2nd time the config is read,
// inside the command. In case of race condition, we just don't know the node id for tracing
// purpose
let node_id = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I suggest we do simpler... but less correct.
Just use the QW_NODE_ID env variable here.

It won't capture people using the node_id config value, and it will break if we change the helm chart, but it has the merit of not adding the extra complexity in this PR (loading the config twice, etc.)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(your call though)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i prefer reading the config: not all deployments of quickwit uses an helm chart
i'd argue that our configuration loading process is complexe, but well encapsulated, so loading the config twice doesn't harm code maintainability

@guilload
Copy link
Member

This is a recurring pattern and issue. We want to emit some logs / metrics / traces containing info from the node config but it is loaded after setting up, sometimes immutable (hello metrics crate), logs / metrics / traces exporters.

Can we consider:

  1. start the process
  2. init logs / metrics / traces exporters with sensible defaults
  3. load node config
  4. overwrite logs / metrics / traces exporters
    ?

I know it sucks to have to do this refactor when we just wanted to add a new tag but that would make the codebase healthier.

@guilload
Copy link
Member

We can also ensure the trace is set up with EnvResourceDetector and rely on OTEL_RESOURCE_ATTRIBUTES to achieve what we want for what we're doing at the kennel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants