Basic event logging for observability

[bboreham]

Log-like API to generate events, plus simple implementation dumping to stderr.

Only wired into ingester for now.

You would turn this on via command-line parameter -event.sample-rate=1000, for example, then one in every 1000 "events" will appear in your log file, and you can extract them for analysis in your favourite tool.

[csmarchbanks]

Couple questions: Why this instead of logging on a jaeger trace?

Would it be worthwhile to have the event log function look for a jaeger span and add it there as well?

[bboreham]

Mostly because I have no idea how to use Jaeger to do statistical analysis across multiple events. If you can point me at some explanation that'd be great.

Looking for a span is a good idea in the general case, but these particular ones are timer-based - currently there is no parent span, and if we added one it would just collect a random set of events on each tick.

[bboreham]

Correction, adding to the flush queue is timer based; pulling from the flush queue is when-available. There's no structure, which is where I typically see the real power of Jaeger.

[csmarchbanks]

Thanks for the explanation!