feat(otel): Add support for storing run spans and log data in Clickhouse #2567
Conversation
|
WalkthroughThis change introduces a ClickHouse-backed event repository and infrastructure: new repository class, singleton wiring, feature-flagged store resolution, and per-store routing across services, presenters, routes, and the OTLP exporter. It propagates taskEventStore through run/attempt paths, adds trace pub/sub, retries in the dynamic flush scheduler, and removes legacy taskEvent utilities. Environment adds ClickHouse and resource monitor settings; a ResourceMonitor singleton is added and conditionally started. ClickHouse client gains fast query/builder APIs with new migrations for task events and usage MV tables. Caches switch to a local createMemoryStore with eviction options. Docker updates the ClickHouse image and config. Estimated code review effort🎯 5 (Critical) | ⏱️ ~180 minutes Pre-merge checks and finishing touches❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 22
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (8)
apps/webapp/app/routes/otel.v1.logs.ts (1)
7-16: Don’t compare content-type exactly; handle charset parametersExact equality will reject common headers like
application/json; charset=utf-8. Use startsWith/normalization.Apply:
- const contentType = request.headers.get("content-type"); + const contentType = request.headers.get("content-type")?.toLowerCase() ?? ""; - if (contentType === "application/json") { + if (contentType.startsWith("application/json")) { const body = await request.json(); - const exportResponse = await otlpExporter.exportLogs(body as ExportLogsServiceRequest); + const exportResponse = await otlpExporter.exportLogs(body as ExportLogsServiceRequest); return json(exportResponse, { status: 200 }); - } else if (contentType === "application/x-protobuf") { + } else if (contentType.startsWith("application/x-protobuf")) { const buffer = await request.arrayBuffer();Also applies to: 25-30
apps/webapp/app/routes/otel.v1.traces.ts (1)
7-16: Handle charset in content-typeAccept
application/json; charset=utf-8and similar for protobuf.- const contentType = request.headers.get("content-type"); + const contentType = request.headers.get("content-type")?.toLowerCase() ?? ""; - if (contentType === "application/json") { + if (contentType.startsWith("application/json")) { const body = await request.json(); const exportResponse = await otlpExporter.exportTraces(body as ExportTraceServiceRequest); return json(exportResponse, { status: 200 }); - } else if (contentType === "application/x-protobuf") { + } else if (contentType.startsWith("application/x-protobuf")) { const buffer = await request.arrayBuffer();Also applies to: 25-30
apps/webapp/app/routes/api.v1.runs.$runId.trace.ts (1)
41-47: Align getTraceDetailedSummary call with its signature
eventRepository.types.ts defines getTraceDetailedSummary(storeTable, environmentId, traceId, options?), but here you’re passing run.createdAt and run.completedAt as extra positional args. Remove those parameters or update the repository interface/implementation to accept from/to.
Location: apps/webapp/app/routes/api.v1.runs.$runId.trace.ts:41-47apps/webapp/app/v3/marqs/sharedQueueConsumer.server.ts (1)
2188-2206: Don't clobber existing OTEL resource attributesSetting
OTEL_RESOURCE_ATTRIBUTESto a JSON string overwrites any value defined in the environment and drops all previously configured resource attributes. For example, a user-providedservice.name=foowould disappear once this branch runs, which will break downstream telemetry. Please append the$trigger.taskEventStoreattribute to the existing value (or initialise it if absent) using the standardkey=valuecomma-separated format instead of replacing it.- if (taskEventStore) { - const resourceAttributes = JSON.stringify({ - [SemanticInternalAttributes.TASK_EVENT_STORE]: taskEventStore, - }); - - variables.push(...[{ key: "OTEL_RESOURCE_ATTRIBUTES", value: resourceAttributes }]); - } + if (taskEventStore) { + const entry = `${SemanticInternalAttributes.TASK_EVENT_STORE}=${taskEventStore}`; + const existingIndex = variables.findIndex( + (variable) => variable.key === "OTEL_RESOURCE_ATTRIBUTES" + ); + + if (existingIndex >= 0) { + const existingValue = variables[existingIndex].value ?? ""; + const separator = existingValue.length > 0 ? "," : ""; + variables[existingIndex] = { + key: "OTEL_RESOURCE_ATTRIBUTES", + value: `${existingValue}${separator}${entry}`, + }; + } else { + variables.push({ key: "OTEL_RESOURCE_ATTRIBUTES", value: entry }); + } + }apps/webapp/app/runEngine/concerns/traceEvents.server.ts (1)
109-121: Record idempotency message via the resolved repository, not the defaultThis writes the debug message to the default Postgres repository even when the trace is routed to ClickHouse, fragmenting traces across stores.
Apply this diff:
- await this.eventRepository.recordEvent( + await repository.recordEvent( `There's an existing run for idempotencyKey: ${idempotencyKey}`, { taskSlug: request.taskId, environment: request.environment, attributes: { runId: existingRun.friendlyId, }, context: request.options?.traceContext, parentId: event.spanId, } );apps/webapp/app/v3/otlpExporter.server.ts (3)
126-133: Fix trace count logging (counts scopes, not spans).current totalSpans sums scopeSpans length; should sum inner spans.
- logger.debug("Exporting traces", { - resourceSpans: request.resourceSpans.length, - totalSpans: request.resourceSpans.reduce( - (acc, resourceSpan) => acc + resourceSpan.scopeSpans.length, - 0 - ), - }); + logger.debug("Exporting traces", { + resourceSpans: request.resourceSpans.length, + totalSpans: request.resourceSpans.reduce( + (acc, resourceSpan) => + acc + + resourceSpan.scopeSpans.reduce((sum, scope) => sum + scope.spans.length, 0), + 0 + ), + });
792-799: Whitelist OTLP exporter env vars in env.server.ts
- Define
OTLP_EXPORTER_VERBOSEandSERVER_OTEL_SPAN_ATTRIBUTE_VALUE_LENGTH_LIMITin theenvexport (apps/webapp/app/env.server.ts)- Update the OTLPExporter instantiation in apps/webapp/app/v3/otlpExporter.server.ts to use
env.*instead ofprocess.env.*
248-249: Emit logs with LOG kind
DownstreamisLogEventonly recognizesTaskEventKind.LOG, so emitting logs as"INTERNAL"prevents them from being classified as log events. In apps/webapp/app/v3/otlpExporter.server.ts, replacekind: "INTERNAL" as constwith
kind: TaskEventKind.LOG
🧹 Nitpick comments (34)
packages/core/src/v3/utils/flattenAttributes.ts (1)
262-268: Precompute filtered key lookups to avoid repeated scans
filteredKeys?.includes(key)walks the array on every iteration. When unflattening large attribute maps this balloons to O(n·m). Converting the list to aSetonce keeps the loop O(n) without changing behavior.const result: Record<string, unknown> = {}; - for (const [key, value] of Object.entries(obj)) { - if (filteredKeys?.includes(key)) { + const filteredKeySet = filteredKeys ? new Set(filteredKeys) : undefined; + + for (const [key, value] of Object.entries(obj)) { + if (filteredKeySet?.has(key)) { continue; }TaskEvent_Property_Analysis.md (1)
9-31: Fix table section rows to satisfy MD056markdownlint (MD056) is firing because these section rows only declare a single cell even though the header defines four columns. GitHub will render them, but our lint pipeline keeps failing. Pad the rows out to four cells (and repeat for the other section dividers in the doc).
-| **Core Identity & Structure** | +| **Core Identity & Structure** | | | |Based on markdownlint-cli2
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsx (1)
305-307: Avoid unsafe type assertions for spanOverrides; narrow the type once.Replace the local cast at usage with a typed alias near destructuring to keep types sound and remove the
asat callsite.Apply:
- const { events, duration, rootSpanStatus, rootStartedAt, queuedDuration, overridesBySpanId } = - trace; + const { + events, + duration, + rootSpanStatus, + rootStartedAt, + queuedDuration, + overridesBySpanId: rawOverridesBySpanId, + } = trace; + const overridesBySpanId = rawOverridesBySpanId as Record<string, SpanOverride> | undefined; @@ - const spanOverrides = selectedSpanId ? overridesBySpanId?.[selectedSpanId] : undefined; + const spanOverrides = selectedSpanId ? overridesBySpanId?.[selectedSpanId] : undefined; @@ - spanOverrides={spanOverrides as SpanOverride | undefined} + spanOverrides={spanOverrides}Also applies to: 328-329, 378-379
apps/webapp/app/presenters/v3/RunStreamPresenter.server.ts (1)
44-45: Confirm eventEmitter lifecycle is per-subscription to avoid cross-subscriber listener removal.
eventEmitter.removeAllListeners()assumes an instance scoped to this subscription. If shared, scope removals to the "message" event or return an isolated emitter fromsubscribeToTrace.apps/webapp/app/routes/resources.runs.$runParam.logs.download.ts (2)
70-77: Prefer Web Streams and CompressionStream over Node streams for isomorphism.Remix routes should avoid Node-specific streams/zlib. Use ReadableStream and pipeThrough(new CompressionStream("gzip")) for portability.
As per coding guidelines
-import { createGzip } from "zlib"; -import { Readable } from "stream"; +// no Node-specific stream imports @@ - // Create a Readable stream from the runEvents array - const readable = new Readable({ - read() { - runEvents.forEach((event) => { - try { - if (!user.admin && event.kind === TaskEventKind.LOG) { - // Only return debug logs for admins - return; - } - this.push(formatRunEvent(event) + "\n"); - } catch {} - }); - this.push(null); // End of stream - }, - }); - - // Create a gzip transform stream - const gzip = createGzip(); - - // Pipe the readable stream into the gzip stream - const compressedStream = readable.pipe(gzip); - - // Return the response with the compressed stream - return new Response(compressedStream as any, { + const stream = new ReadableStream({ + start(controller) { + for (const event of runEvents) { + try { + if (!user.admin && event.kind === TaskEventKind.LOG) continue; + controller.enqueue(new TextEncoder().encode(formatRunEvent(event) + "\n")); + } catch {} + } + controller.close(); + }, + }); + + const body = + typeof CompressionStream !== "undefined" + ? stream.pipeThrough(new CompressionStream("gzip")) + : stream; // fallback: uncompressed if CompressionStream not available + + return new Response(body, { status: 200, headers: { "Content-Type": "application/octet-stream", "Content-Disposition": `attachment; filename="${parsedParams.runParam}.log"`, - "Content-Encoding": "gzip", + ...(typeof CompressionStream !== "undefined" ? { "Content-Encoding": "gzip" } : {}), }, });
125-127: De-duplicate util by reusing shared helper.Import
getDateFromNanosecondsfrom~/v3/eventRepository/common.serverinstead of redefining it here.Apply in this file:
+import { getDateFromNanoseconds } from "~/v3/eventRepository/common.server"; @@ -function getDateFromNanoseconds(nanoseconds: bigint) { - return new Date(Number(nanoseconds) / 1_000_000); -}Note: If you adopt the precision fix, update the shared helper accordingly in a follow-up.
apps/webapp/app/routes/otel.v1.logs.ts (1)
22-24: Set protobuf response Content-TypeReturn header helps OTLP clients decode properly.
- return new Response(ExportLogsServiceResponse.encode(exportResponse).finish(), { - status: 200, - }); + return new Response(ExportLogsServiceResponse.encode(exportResponse).finish(), { + status: 200, + headers: { "Content-Type": "application/x-protobuf" }, + });apps/webapp/app/v3/services/cancelTaskRunV1.server.ts (1)
115-119: Include store in error log contextAdding the store eases debugging across mixed repositories.
if (cancelRunEventError) { logger.error("[CancelTaskRunServiceV1] Failed to cancel run event", { error: cancelRunEventError, runId: cancelledTaskRun.id, + taskEventStore: cancelledTaskRun.taskEventStore, }); }apps/webapp/app/routes/otel.v1.traces.ts (1)
22-24: Set protobuf response Content-TypeMirror OTLP expectations for binary responses.
- return new Response(ExportTraceServiceResponse.encode(exportResponse).finish(), { - status: 200, - }); + return new Response(ExportTraceServiceResponse.encode(exportResponse).finish(), { + status: 200, + headers: { "Content-Type": "application/x-protobuf" }, + });internal-packages/clickhouse/src/client/types.ts (2)
93-98: Document and narrow the unsafe insert APIinsertUnsafe bypasses schema validation; add JSDoc warning and prefer Record<string, unknown> over any.
- insertUnsafe<TRecord extends Record<string, any>>(req: { + insertUnsafe<TRecord extends Record<string, unknown>>(req: { name: string; table: string; settings?: ClickHouseSettings; }): ClickhouseInsertFunction<TRecord>;
4-4: Use type-only import for ClickHouseSettingsAvoids unnecessary runtime import.
-import { ClickHouseSettings } from "@clickhouse/client"; +import type { ClickHouseSettings } from "@clickhouse/client";internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts (1)
780-805: Propagating taskEventStore in emitted events is consistent; add it to runAttemptStarted tooFor symmetry and consumer simplicity, include taskEventStore on runAttemptStarted as well.
this.$.eventBus.emit("runAttemptStarted", { time: new Date(), run: { id: updatedRun.id, status: updatedRun.status, createdAt: updatedRun.createdAt, updatedAt: updatedRun.updatedAt, attemptNumber: nextAttemptNumber, baseCostInCents: updatedRun.baseCostInCents, executedAt: updatedRun.executedAt ?? undefined, + taskEventStore: updatedRun.taskEventStore, }, organization: { id: updatedRun.runtimeEnvironment.organizationId, }, project: { id: updatedRun.projectId, }, environment: { id: updatedRun.runtimeEnvironment.id, }, });Also applies to: 900-913, 978-1001, 1418-1442, 1560-1583
apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts (1)
22-45: Prefer logger over bare console outputWe generally steer clear of
console.login the webapp and lean on our logger utilities so messages flow through the configured transports. Could we swap the raw console call forlogger.info(or similar) on the ClickHouse instance? Same refactor would help if we ever need to adjust log levels across environments.internal-packages/clickhouse/schema/007_add_task_events_v1.sql (2)
47-47: Consider reordering the primary key for typical trace queriesIf most queries filter by environment_id and trace_id (optionally with a time window), consider ORDER BY (environment_id, trace_id, start_time). Using toUnixTimestamp(start_time) in ORDER BY can reduce data‑skipping for time‑range scans and loses sub-second ordering. Evaluate query patterns before changing.
48-49: TTL setting may retain expired rows longer than expectedSETTINGS ttl_only_drop_parts = 1 drops whole parts only. If rows within a part expire at different times, expired rows may persist until the entire part qualifies, impacting retention and storage. If you need row‑level TTL guarantees, remove this setting (default) to allow merges to purge expired rows.
Would you like me to propose a version with row‑level TTL and merge tuning?
apps/webapp/app/v3/eventRepository/index.server.ts (1)
20-30: Store naming consistencygetEventRepository returns store = "clickhouse" or the actual table name ("taskEvent" | "taskEventPartitioned"). Ensure downstream consumers (callbacks, presenters) expect this mixed namespace ("backend" vs "table"). If they need a backend discriminator, return both backend ("postgres" | "clickhouse") and table ("taskEvent" | "taskEventPartitioned").
I can draft a minimal change returning { repository, backend: "postgres" | "clickhouse", storeTable?: "taskEvent" | "taskEventPartitioned" }.
apps/webapp/app/runEngine/concerns/traceEvents.server.ts (1)
14-22: Remove unused ClickHouse DI to simplify concernsclickhouseEventRepository is no longer used after introducing getEventRepository. Drop the field and constructor arg to avoid confusion and keep DI minimal.
Apply this diff:
- private readonly clickhouseEventRepository: ClickhouseEventRepository; + // no-op - constructor( - eventRepository: EventRepository, - clickhouseEventRepository: ClickhouseEventRepository - ) { + constructor(eventRepository: EventRepository) { this.eventRepository = eventRepository; - this.clickhouseEventRepository = clickhouseEventRepository; }You can also remove the unused imports of ClickhouseEventRepository, flags, FEATURE_FLAG, env, and getTaskEventStore if not needed elsewhere.
apps/webapp/app/env.server.ts (3)
1123-1125: Constrain rollout percent to a sane 0–100 range (int).Prevents invalid percentages and accidental over-rollout.
- EVENT_REPOSITORY_CLICKHOUSE_ROLLOUT_PERCENT: z.coerce.number().optional(), + EVENT_REPOSITORY_CLICKHOUSE_ROLLOUT_PERCENT: z.coerce.number().int().min(0).max(100).optional(),
1121-1122: Add lower bounds to batch size and flush interval.Avoids zero/negative values that would stall batching.
- EVENTS_CLICKHOUSE_BATCH_SIZE: z.coerce.number().int().default(1000), - EVENTS_CLICKHOUSE_FLUSH_INTERVAL_MS: z.coerce.number().int().default(1000), + EVENTS_CLICKHOUSE_BATCH_SIZE: z.coerce.number().int().min(1).default(1000), + EVENTS_CLICKHOUSE_FLUSH_INTERVAL_MS: z.coerce.number().int().min(1).default(1000),
1112-1116: Ensure EVENTS_CLICKHOUSE_URL is required when ClickHouse is used.If EVENT_REPOSITORY_DEFAULT_STORE is "clickhouse" or rollout > 0, this must be present; today it’s optional. Either enforce in schema (discriminated union/refine) or validate at repository initialization.
Would you like me to add a schema refine that asserts EVENTS_CLICKHOUSE_URL is defined when ClickHouse is active?
apps/webapp/app/v3/services/tracePubSub.server.ts (2)
48-53: Await Redis quit to ensure clean shutdown.Avoids dangling sockets/listeners on unsubscribe.
const unsubscribe = async () => { await redis.unsubscribe(channel); - redis.quit(); + await redis.quit(); this._subscriberCount--; };
10-16: Consider removing or exposing _subscriberCount.Currently unused; either remove to reduce noise or expose a getter for metrics.
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (4)
136-139: insertManyImmediate should bypass batching.Immediate variant currently just queues normally. If callers rely on sync-ish persistence, write directly and publish.
- async insertManyImmediate(events: CreateEventInput[]): Promise<void> { - this.insertMany(events); - } + async insertManyImmediate(events: CreateEventInput[]): Promise<void> { + const records = events.flatMap((e) => this.createEventToTaskEventV1Input(e)); + const [err] = await this._clickhouse.taskEvents.insert(records); + if (err) throw err; + await this.#publishToRedis(records); + }
118-121: Avoid logging full event payloads at debug level.Attributes/metadata may contain PII; logging entire batches is risky and noisy.
- logger.debug("ClickhouseEventRepository.flushBatch", { - events, - }); + logger.debug("ClickhouseEventRepository.flushBatch", { + eventCount: events.length, + sampleTraceId: events[0]?.trace_id, + });
158-161: TTL is hard-coded to 30 days in many places.Make retention configurable (env or config param) and pass via ClickhouseEventRepositoryConfig.
Based on learnings
Also applies to: 228-231, 256-259, 288-291, 314-317, 448-450, 586-589, 610-612, 658-661, 699-701, 745-747, 791-793, 835-837
1312-1324: Harden metadata parsing.A single malformed JSON string would throw and break the whole request. Catch and return undefined.
- #parseMetadata(metadata: string): Record<string, unknown> | undefined { + #parseMetadata(metadata: string): Record<string, unknown> | undefined { if (!metadata) { return undefined; } - const parsed = JSON.parse(metadata); + let parsed: unknown; + try { + parsed = JSON.parse(metadata); + } catch { + return undefined; + }apps/webapp/app/v3/eventRepository/common.server.ts (3)
37-41: Simplify JS Date duration calc.Unreachable typeof check; keep it simple.
-export function calculateDurationFromStartJsDate(startTime: Date, endTime: Date = new Date()) { - const $endtime = typeof endTime === "string" ? new Date(endTime) : endTime; - - return ($endtime.getTime() - startTime.getTime()) * 1_000_000; -} +export function calculateDurationFromStartJsDate(startTime: Date, endTime: Date = new Date()) { + return (endTime.getTime() - startTime.getTime()) * 1_000_000; +}
102-131: Ensure error “message” is a string for CUSTOM/STRING errors.error.raw may not be a string. Coerce safely to avoid downstream type issues.
- case "CUSTOM_ERROR": { - return { - type: "Error", - message: error.raw, - }; - } + case "CUSTOM_ERROR": { + return { + type: "Error", + message: typeof error.raw === "string" ? error.raw : JSON.stringify(error.raw), + }; + } @@ - case "STRING_ERROR": { - return { - type: "Error", - message: error.raw, - }; - } + case "STRING_ERROR": { + return { + type: "Error", + message: typeof error.raw === "string" ? error.raw : String(error.raw), + }; + }Please confirm TaskRunError["raw"] is string in all cases; if not, the above prevents non-string assignment.
158-166: Simplify empty-object check.Small nit for readability.
-export function isEmptyObject(obj: object) { - for (var prop in obj) { - if (Object.prototype.hasOwnProperty.call(obj, prop)) { - return false; - } - } - - return true; -} +export function isEmptyObject(obj: object) { + return Object.keys(obj).length === 0; +}apps/webapp/app/v3/otlpExporter.server.ts (4)
161-168: Clarify log message (it says “Skipping” but allows through).Message is misleading.
- logger.debug("Skipping resource span without trigger attribute", { + logger.debug("Allowing resource span without trigger attribute", { attributes: resourceSpan.resource?.attributes, spans: resourceSpan.scopeSpans.flatMap((scopeSpan) => scopeSpan.spans), });
36-41: Depend on interfaces, not concrete repos.Constructor types should be IEventRepository to decouple from implementations.
- constructor( - private readonly _eventRepository: EventRepository, - private readonly _clickhouseEventRepository: ClickhouseEventRepository, + constructor( + private readonly _eventRepository: IEventRepository, + private readonly _clickhouseEventRepository: IEventRepository, private readonly _verbose: boolean, private readonly _spanAttributeValueLengthLimit: number ) {
764-768: Support Uint8Array for bytes values.OTLP bytes may arrive as Uint8Array, not Buffer.
-function isBytesValue(value: AnyValue | undefined): value is { bytesValue: Buffer } { +function isBytesValue(value: AnyValue | undefined): value is { bytesValue: Buffer | Uint8Array } { if (!value) return false; - return Buffer.isBuffer(value.bytesValue); + return Buffer.isBuffer(value.bytesValue) || value.bytesValue instanceof Uint8Array; }
770-777: Avoid unnecessary Buffer copy in binaryToHex.Use Buffer’s toString directly when possible.
-function binaryToHex(buffer: Buffer | string | undefined): string | undefined { +function binaryToHex(buffer: Buffer | Uint8Array | string | undefined): string | undefined { if (!buffer) return undefined; if (typeof buffer === "string") return buffer; - return Buffer.from(Array.from(buffer)).toString("hex"); + return Buffer.isBuffer(buffer) + ? buffer.toString("hex") + : Buffer.from(buffer).toString("hex"); }apps/webapp/app/v3/eventRepository/eventRepository.types.ts (1)
222-223: Prefer unknown over any for metadata.Improves type safety at call sites.
- metadata: any; // Used by SpanPresenter for entity processing + metadata: unknown; // Used by SpanPresenter for entity processing
...app/routes/engine.v1.dev.runs.$runFriendlyId.snapshots.$snapshotFriendlyId.attempts.start.ts
Show resolved
Hide resolved
...anizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsx
Outdated
Show resolved
Hide resolved
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
♻️ Duplicate comments (3)
apps/webapp/app/v3/eventRepository/originalRunIdCache.server.ts (1)
24-42: Redis-backed store must remain optionalWe still create the Redis store even when no connection details are configured, so in dev/test it falls back to 127.0.0.1:6379 and every lookup rejects. Please restore the in-memory store as the baseline and only push a
RedisCacheStorewhen the options include a real host/port/socket/url (keeping the keyPrefix/TLS flags). Otherwise this cache breaks anywhere Redis isn’t provisioned.apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (1)
1719-1762: Timezone handling is incorrect in DateTime64→Date conversionInputs that carry a
Zor±HH:MMsuffix still take the fast path, so we drop the offset and end up shifting times by hours. That breaks trace timelines whenever ClickHouse emits offsets. Please skip the fast path when a timezone indicator is present and apply the parsed offset before constructing theDate.export function convertClickhouseDateTime64ToJsDate(date: string): Date { - // Fast path for common format: "2025-09-23 12:32:46.130262875" or "2025-09-23 12:32:46" - // This avoids the expensive regex for the common case - if (date.length >= 19 && date[4] === "-" && date[7] === "-" && date[10] === " ") { + const tzIndex = + date.indexOf("Z", 19) !== -1 + ? date.indexOf("Z", 19) + : Math.max(date.indexOf("+", 19), date.indexOf("-", 19)); + const hasTimezone = tzIndex !== -1; + + // Fast path for common format without timezone: "2025-09-23 12:32:46.130262875" or "2025-09-23 12:32:46" + if ( + !hasTimezone && + date.length >= 19 && + date[4] === "-" && + date[7] === "-" && + date[10] === " " + ) { @@ - const fraction = m[7] ?? ""; // up to 9 digits + const fraction = m[7] ?? ""; // up to 9 digits + const sign = m[8] as "+" | "-" | undefined; + const offH = m[9] ? Number(m[9]) : 0; + const offM = m[10] ? Number(m[10]) : 0; @@ - return new Date(Date.UTC(year, month - 1, day, hour, minute, second, msPart)); + let msEpoch = Date.UTC(year, month - 1, day, hour, minute, second, msPart); + + if (sign) { + const offsetMinutesSigned = (sign === "+" ? 1 : -1) * (offH * 60 + offM); + msEpoch -= offsetMinutesSigned * 60_000; + } + + return new Date(msEpoch); }apps/webapp/app/v3/otlpExporter.server.ts (1)
256-260: Stop hard-codingenvironmentType.This still forces every log/span into
DEVELOPMENT, so prod/test traces sent through ClickHouse will be misclassified and downstream filters/analytics stay wrong. Please derive the value from the resource/log attributes (e.g.SemanticInternalAttributes.ENVIRONMENT_TYPE) or repository/env config, and only fall back to"DEVELOPMENT"as a last resort. Apply the same fix inconvertSpansToCreateableEvents, too.
🧹 Nitpick comments (3)
apps/webapp/app/services/authorizationRateLimitMiddleware.server.ts (1)
149-152: Consider usingcreateMemoryStorefor consistency.Unlike other files in this PR (billingCache.ts, fairQueueSelectionStrategy.ts) that adopt the
createMemoryStorefactory, this file still manually instantiatesMemoryStorewith a custom frequency of0.001(vs. the factory's default0.01).If the lower frequency is intentional for rate limiting workloads, consider documenting why. Otherwise, using
createMemoryStore(limiterCache?.maxItems ?? 1000)would align with the standardization effort and reduce maintenance.Option 1: Use factory with default frequency (if 0.001 is not critical):
- const memory = new MemoryStore({ - persistentMap: new Map(), - unstableEvictOnSet: { frequency: 0.001, maxItems: limiterCache?.maxItems ?? 1000 }, - }); + const memory = createMemoryStore(limiterCache?.maxItems ?? 1000);Option 2: Use factory with custom frequency (if 0.001 is required):
- const memory = new MemoryStore({ - persistentMap: new Map(), - unstableEvictOnSet: { frequency: 0.001, maxItems: limiterCache?.maxItems ?? 1000 }, - }); + const memory = createMemoryStore(limiterCache?.maxItems ?? 1000, 0.001);internal-packages/cache/src/stores/memory.ts (1)
5-13: Consider adding parameter validation.The factory accepts
maxItemsandfrequencywithout validation. Consider adding guards for edge cases like non-positive values to prevent runtime surprises.export function createMemoryStore(maxItems: number, frequency: number = 0.01) { + if (maxItems <= 0) { + throw new Error("maxItems must be positive"); + } + if (frequency < 0 || frequency > 1) { + throw new Error("frequency must be between 0 and 1"); + } return new MemoryStore({ persistentMap: new Map(), unstableEvictOnSet: { frequency, maxItems, }, }); }apps/webapp/app/services/resourceMonitor.server.ts (1)
6-11: Consider enriching the ResourceMonitor configuration.The current configuration uses an empty
ctxandverbose: false. For better observability:
- ctx: Could include app metadata (e.g.,
{ service: "webapp", env: env.NODE_ENV }).- verbose: Set to
falsefor production but could be toggled via environment variable for debugging.- processName: Consider setting to identify logs (e.g.,
"trigger-webapp").Example enhancement:
+import { env } from "~/env.server"; + function initializeResourceMonitor() { return new ResourceMonitor({ - ctx: {}, - verbose: false, + ctx: { service: "webapp", env: env.NODE_ENV }, + verbose: env.RESOURCE_MONITOR_VERBOSE === "true", + processName: "trigger-webapp", }); }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yamlreferences/hello-world/src/trigger/telemetry.tsis excluded by!references/**references/hello-world/trigger.config.tsis excluded by!references/**
📒 Files selected for processing (38)
apps/webapp/app/entry.server.tsx(2 hunks)apps/webapp/app/env.server.ts(2 hunks)apps/webapp/app/eventLoopMonitor.server.ts(1 hunks)apps/webapp/app/presenters/v3/SpanPresenter.server.ts(9 hunks)apps/webapp/app/routes/otel.v1.logs.ts(1 hunks)apps/webapp/app/routes/otel.v1.traces.ts(1 hunks)apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsx(6 hunks)apps/webapp/app/services/authorizationRateLimitMiddleware.server.ts(2 hunks)apps/webapp/app/services/platform.v3.server.ts(1 hunks)apps/webapp/app/services/realtimeClient.server.ts(1 hunks)apps/webapp/app/services/requestIdempotency.server.ts(1 hunks)apps/webapp/app/services/resourceMonitor.server.ts(1 hunks)apps/webapp/app/v3/dynamicFlushScheduler.server.ts(2 hunks)apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts(1 hunks)apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts(1 hunks)apps/webapp/app/v3/eventRepository/originalRunIdCache.server.ts(1 hunks)apps/webapp/app/v3/marqs/fairDequeuingStrategy.server.ts(1 hunks)apps/webapp/app/v3/otlpExporter.server.ts(12 hunks)apps/webapp/app/v3/services/tracePubSub.server.ts(1 hunks)apps/webapp/app/v3/services/worker/workerGroupTokenService.server.ts(5 hunks)docker/docker-compose.yml(3 hunks)internal-packages/cache/package.json(1 hunks)internal-packages/cache/src/index.ts(1 hunks)internal-packages/cache/src/stores/memory.ts(1 hunks)internal-packages/clickhouse/package.json(1 hunks)internal-packages/clickhouse/src/client/client.ts(4 hunks)internal-packages/clickhouse/src/client/noop.ts(4 hunks)internal-packages/clickhouse/src/client/queryBuilder.ts(2 hunks)internal-packages/clickhouse/src/client/types.ts(5 hunks)internal-packages/clickhouse/src/taskEvents.ts(1 hunks)internal-packages/run-engine/src/engine/billingCache.ts(2 hunks)internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts(4 hunks)internal-packages/run-engine/src/run-queue/fairQueueSelectionStrategy.ts(2 hunks)packages/cli-v3/package.json(1 hunks)packages/cli-v3/src/commands/dev.ts(1 hunks)packages/cli-v3/src/rules/manifest.ts(1 hunks)packages/core/src/v3/serverOnly/index.ts(1 hunks)packages/core/src/v3/serverOnly/resourceMonitor.ts(1 hunks)
✅ Files skipped from review due to trivial changes (2)
- packages/cli-v3/package.json
- apps/webapp/app/eventLoopMonitor.server.ts
🚧 Files skipped from review as they are similar to previous changes (6)
- apps/webapp/app/routes/otel.v1.logs.ts
- apps/webapp/app/v3/services/worker/workerGroupTokenService.server.ts
- apps/webapp/app/v3/services/tracePubSub.server.ts
- internal-packages/clickhouse/src/client/noop.ts
- internal-packages/clickhouse/src/taskEvents.ts
- apps/webapp/app/routes/otel.v1.traces.ts
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
internal-packages/cache/src/stores/memory.tsinternal-packages/run-engine/src/engine/billingCache.tspackages/core/src/v3/serverOnly/index.tsapps/webapp/app/v3/eventRepository/originalRunIdCache.server.tsinternal-packages/run-engine/src/run-queue/fairQueueSelectionStrategy.tspackages/core/src/v3/serverOnly/resourceMonitor.tspackages/cli-v3/src/commands/dev.tsinternal-packages/run-engine/src/engine/systems/runAttemptSystem.tsapps/webapp/app/services/resourceMonitor.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsinternal-packages/clickhouse/src/client/queryBuilder.tsinternal-packages/clickhouse/src/client/types.tsapps/webapp/app/entry.server.tsxapps/webapp/app/v3/otlpExporter.server.tsapps/webapp/app/services/authorizationRateLimitMiddleware.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsinternal-packages/cache/src/index.tsapps/webapp/app/services/platform.v3.server.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsxapps/webapp/app/services/requestIdempotency.server.tsapps/webapp/app/v3/marqs/fairDequeuingStrategy.server.tsapps/webapp/app/services/realtimeClient.server.tsinternal-packages/clickhouse/src/client/client.tsapps/webapp/app/v3/dynamicFlushScheduler.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.tspackages/cli-v3/src/rules/manifest.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
packages/core/src/v3/serverOnly/index.tsapps/webapp/app/v3/eventRepository/originalRunIdCache.server.tspackages/core/src/v3/serverOnly/resourceMonitor.tsapps/webapp/app/services/resourceMonitor.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/entry.server.tsxapps/webapp/app/v3/otlpExporter.server.tsapps/webapp/app/services/authorizationRateLimitMiddleware.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/services/platform.v3.server.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsxapps/webapp/app/services/requestIdempotency.server.tsapps/webapp/app/v3/marqs/fairDequeuingStrategy.server.tsapps/webapp/app/services/realtimeClient.server.tsapps/webapp/app/v3/dynamicFlushScheduler.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
When importing from @trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from @trigger.dev/core’s package.json
Files:
apps/webapp/app/v3/eventRepository/originalRunIdCache.server.tsapps/webapp/app/services/resourceMonitor.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/entry.server.tsxapps/webapp/app/v3/otlpExporter.server.tsapps/webapp/app/services/authorizationRateLimitMiddleware.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/services/platform.v3.server.tsapps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsxapps/webapp/app/services/requestIdempotency.server.tsapps/webapp/app/v3/marqs/fairDequeuingStrategy.server.tsapps/webapp/app/services/realtimeClient.server.tsapps/webapp/app/v3/dynamicFlushScheduler.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.ts
{apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Files:
apps/webapp/app/v3/eventRepository/originalRunIdCache.server.tsapps/webapp/app/services/resourceMonitor.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/entry.server.tsxapps/webapp/app/v3/otlpExporter.server.tsapps/webapp/app/services/authorizationRateLimitMiddleware.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/services/platform.v3.server.tsapps/webapp/app/services/requestIdempotency.server.tsapps/webapp/app/v3/marqs/fairDequeuingStrategy.server.tsapps/webapp/app/services/realtimeClient.server.tsapps/webapp/app/v3/dynamicFlushScheduler.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.ts
apps/webapp/app/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Modules intended for test consumption under apps/webapp/app/**/*.ts must not read environment variables; accept configuration via options instead
Files:
apps/webapp/app/v3/eventRepository/originalRunIdCache.server.tsapps/webapp/app/services/resourceMonitor.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/v3/otlpExporter.server.tsapps/webapp/app/services/authorizationRateLimitMiddleware.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/services/platform.v3.server.tsapps/webapp/app/services/requestIdempotency.server.tsapps/webapp/app/v3/marqs/fairDequeuingStrategy.server.tsapps/webapp/app/services/realtimeClient.server.tsapps/webapp/app/v3/dynamicFlushScheduler.server.tsapps/webapp/app/env.server.tsapps/webapp/app/presenters/v3/SpanPresenter.server.ts
🧬 Code graph analysis (19)
internal-packages/cache/src/stores/memory.ts (1)
internal-packages/cache/src/index.ts (2)
createMemoryStore(10-10)MemoryStore(10-10)
internal-packages/run-engine/src/engine/billingCache.ts (2)
internal-packages/cache/src/index.ts (1)
createMemoryStore(10-10)internal-packages/cache/src/stores/memory.ts (1)
createMemoryStore(5-13)
apps/webapp/app/v3/eventRepository/originalRunIdCache.server.ts (2)
internal-packages/redis/src/index.ts (1)
RedisOptions(4-4)internal-packages/cache/src/index.ts (4)
DefaultStatefulContext(3-3)RedisCacheStore(9-9)createCache(2-2)Namespace(4-4)
internal-packages/run-engine/src/run-queue/fairQueueSelectionStrategy.ts (2)
internal-packages/cache/src/index.ts (1)
createMemoryStore(10-10)internal-packages/cache/src/stores/memory.ts (1)
createMemoryStore(5-13)
packages/cli-v3/src/commands/dev.ts (1)
packages/cli-v3/src/commands/install-rules.ts (1)
initiateRulesInstallWizard(161-229)
internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts (1)
internal-packages/cache/src/stores/memory.ts (1)
createMemoryStore(5-13)
apps/webapp/app/services/resourceMonitor.server.ts (1)
packages/core/src/v3/serverOnly/resourceMonitor.ts (1)
ResourceMonitor(103-697)
apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts (2)
apps/webapp/app/env.server.ts (1)
env(1201-1201)internal-packages/clickhouse/src/index.ts (1)
ClickHouse(61-172)
internal-packages/clickhouse/src/client/queryBuilder.ts (1)
internal-packages/clickhouse/src/client/types.ts (3)
ColumnExpression(24-27)ClickhouseReader(29-135)ClickhouseQueryFunction(8-14)
internal-packages/clickhouse/src/client/types.ts (1)
internal-packages/clickhouse/src/client/queryBuilder.ts (1)
ClickhouseQueryFastBuilder(95-189)
apps/webapp/app/entry.server.tsx (2)
apps/webapp/app/env.server.ts (1)
env(1201-1201)apps/webapp/app/services/resourceMonitor.server.ts (1)
resourceMonitor(4-4)
apps/webapp/app/v3/otlpExporter.server.ts (6)
apps/webapp/app/v3/eventRepository/eventRepository.server.ts (3)
EventRepository(71-1470)events(1465-1469)eventRepository(1472-1472)apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (3)
ClickhouseEventRepository(76-1652)span(1634-1651)events(124-127)apps/webapp/app/v3/eventRepository/eventRepository.types.ts (2)
CreateEventInput(24-58)IEventRepository(311-416)apps/webapp/app/env.server.ts (1)
env(1201-1201)apps/webapp/app/v3/eventRepository/common.server.ts (1)
generateSpanId(71-73)apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts (1)
clickhouseEventRepository(6-9)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (8)
internal-packages/clickhouse/src/index.ts (1)
ClickHouse(61-172)apps/webapp/app/v3/eventRepository/eventRepository.types.ts (14)
IEventRepository(311-416)CreateEventInput(24-58)TraceEventOptions(95-106)EventBuilder(108-114)TraceAttributes(86-91)CompleteableTaskRun(67-80)TraceSummary(272-276)SpanSummary(246-263)SpanOverride(265-270)SpanDetail(184-223)SpanSummaryCommon(229-244)TraceDetailedSummary(298-301)SpanDetailedSummary(278-296)RunPreparedEvent(180-182)apps/webapp/app/v3/dynamicFlushScheduler.server.ts (1)
DynamicFlushScheduler(21-423)internal-packages/clickhouse/src/taskEvents.ts (8)
TaskEventV1Input(5-22)TaskEventV1Input(24-24)TaskEventSummaryV1Result(40-50)TaskEventSummaryV1Result(52-52)TaskEventDetailsV1Result(111-121)TaskEventDetailsV1Result(123-123)TaskEventDetailedSummaryV1Result(73-84)TaskEventDetailedSummaryV1Result(86-86)apps/webapp/app/v3/services/tracePubSub.server.ts (1)
tracePubSub(62-62)apps/webapp/app/v3/eventRepository/common.server.ts (10)
parseEventsField(88-100)convertDateToNanoseconds(43-45)removePrivateProperties(134-156)extractContextFromCarrier(8-21)getNowInNanoseconds(23-25)calculateDurationFromStart(31-35)generateTraceId(67-69)generateDeterministicSpanId(51-63)createExceptionPropertiesFromError(102-131)calculateDurationFromStartJsDate(37-41)apps/webapp/app/v3/eventRepository/originalRunIdCache.server.ts (1)
originalRunIdCache(62-75)apps/webapp/app/v3/taskEventStore.server.ts (1)
TaskEventStoreTable(45-45)
apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsx (3)
apps/webapp/app/v3/eventRepository/eventRepository.types.ts (1)
SpanOverride(265-270)apps/webapp/app/presenters/v3/SpanPresenter.server.ts (4)
Span(24-24)run(559-565)run(567-632)run(634-636)apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (1)
span(1634-1651)
apps/webapp/app/services/requestIdempotency.server.ts (2)
internal-packages/cache/src/index.ts (1)
MemoryStore(10-10)internal-packages/cache/src/stores/memory.ts (1)
MemoryStore(3-3)
apps/webapp/app/services/realtimeClient.server.ts (2)
internal-packages/cache/src/index.ts (1)
MemoryStore(10-10)internal-packages/cache/src/stores/memory.ts (1)
MemoryStore(3-3)
internal-packages/clickhouse/src/client/client.ts (6)
internal-packages/clickhouse/src/client/types.ts (4)
ColumnExpression(24-27)ClickhouseQueryFunction(8-14)ClickhouseQueryBuilderFastFunction(20-22)ClickhouseInsertFunction(137-143)internal-packages/tracing/src/index.ts (1)
startSpan(49-79)packages/core/src/v3/tryCatch.ts (1)
tryCatch(8-15)internal-packages/clickhouse/src/client/errors.ts (2)
QueryError(33-42)InsertError(24-32)internal-packages/clickhouse/src/client/queryBuilder.ts (1)
ClickhouseQueryFastBuilder(95-189)packages/core/src/v3/workers/taskExecutor.ts (1)
result(1252-1299)
apps/webapp/app/v3/dynamicFlushScheduler.server.ts (2)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (1)
flushId(104-122)apps/webapp/app/services/runsReplicationService.server.ts (4)
flushId(491-585)T(588-623)attempt(655-665)error(626-653)
apps/webapp/app/presenters/v3/SpanPresenter.server.ts (4)
apps/webapp/app/v3/eventRepository/eventRepository.types.ts (2)
SpanDetail(184-223)IEventRepository(311-416)apps/webapp/app/v3/eventRepository/eventRepository.server.ts (1)
eventRepository(1472-1472)apps/webapp/app/v3/eventRepository/index.server.ts (1)
resolveEventRepositoryForStore(10-18)apps/webapp/app/v3/taskEventStore.server.ts (2)
getTaskEventStoreTableForRun(47-51)TaskEventStoreTable(45-45)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (28)
packages/core/src/v3/serverOnly/index.ts (1)
10-10: Re-export looks goodBringing
resourceMonitorinto the public surface here makes the new utility accessible without touching existing consumers.apps/webapp/app/services/platform.v3.server.ts (1)
47-53: LGTM: Probabilistic cache eviction policy added.The
unstableEvictOnSetconfiguration with 1% frequency and 1000 max items prevents unbounded memory growth for the platform cache while maintaining performance. This is appropriate given the cache stores limits and usage data with 5-10 minute TTLs.apps/webapp/app/services/requestIdempotency.server.ts (1)
36-42: LGTM: Conservative eviction policy for idempotency cache.The
unstableEvictOnSetconfiguration uses a more conservative frequency (0.001 vs 0.01 in platform cache), which is appropriate for request idempotency where false evictions could lead to duplicate processing. The 1000 item limit with 24-hour TTL provides good protection against memory growth.apps/webapp/app/entry.server.tsx (2)
243-243: LGTM: Resource monitor service import.Import follows the established pattern for other monitoring services in this file.
255-257: LGTM: Resource monitor singleton and monitoring behavior verified
Confirmed thesingletonhelper usesglobalThis.__trigger_singletonsto instantiateresourceMonitorexactly once andstartMonitoringclears any existing interval before setting a new one. No duplicate timers or observers will be registered in the current setup, and a 1 s interval is acceptable for resource telemetry without further changes.apps/webapp/app/services/realtimeClient.server.ts (1)
86-89: LGTM: Cache eviction policy for realtime filters.The
unstableEvictOnSetconfiguration with 1% frequency and 1000 max items is appropriate for the createdAt filter cache. The eviction policy prevents unbounded memory growth while the 1-2 week TTL accommodates long-lived Electric shape subscriptions.internal-packages/clickhouse/package.json (1)
9-9: Approve: Bump @clickhouse/client to ^1.12.1
Minor version update with no breaking changes and no security advisories found via GitHub API.internal-packages/run-engine/src/engine/billingCache.ts (1)
56-56: LGTM! Standardized memory store creation.The change replaces direct
MemoryStoreinstantiation with thecreateMemoryStore(1000)factory, ensuring consistent eviction policy (1000 items, 0.01 frequency) across the billing cache.internal-packages/run-engine/src/run-queue/fairQueueSelectionStrategy.ts (1)
110-110: LGTM! Consistent memory store initialization.The factory pattern is applied consistently, replacing direct instantiation with
createMemoryStore(1000)for the concurrency limit cache.internal-packages/cache/src/stores/memory.ts (1)
5-13: LGTM! Clean factory implementation.The factory provides a consistent interface for creating memory stores with eviction policies across the codebase.
apps/webapp/app/services/resourceMonitor.server.ts (1)
6-11: LGTM! Singleton pattern applied correctly.The resource monitor is properly initialized as a singleton, ensuring a single instance across the webapp.
internal-packages/run-engine/src/engine/systems/runAttemptSystem.ts (8)
3-3: LGTM!Correct import for the new factory-based memory store API.
129-129: LGTM!The factory call
createMemoryStore(5000, 0.001)correctly replaces the previous direct instantiation, with a max of 5000 items and a 0.1% eviction frequency.
745-790: LGTM!The
taskEventStorefield is correctly propagated in therunSucceededevent payload, maintaining consistency with the data model.
889-910: LGTM!The
taskEventStorefield is correctly included in therunAttemptFailedevent when a force requeue occurs.
978-1000: LGTM!The
taskEventStorefield is correctly included in therunRetryScheduledevent payload.
1326-1425: LGTM!The
taskEventStorefield is correctly selected and propagated in therunCancelledevent.
1525-1566: LGTM!The
taskEventStorefield is correctly selected and propagated in therunFailedevent.
447-447: taskEventStore column exists in schema and migrations
FoundtaskEventStore String @default("taskEvent")ininternal-packages/database/prisma/schema.prisma:677and a correspondingCOLUMN "taskEventStore"addition in migration20250212075957_add_task_event_store_to_task_run/migration.sql. Resolving.apps/webapp/app/routes/resources.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam.spans.$spanParam/route.tsx (5)
82-82: LGTM!Correct import of the
SpanOverridetype from the event repository.
124-129: LGTM!The
spanOverridesparameter is correctly threaded through the component hierarchy fromSpanViewtoSpanBody.Also applies to: 180-187, 194-199
213-213: LGTM!Applying span overrides before rendering ensures the UI reflects the corrected state.
258-288: LGTM! Override logic is correct.The
applySpanOverridesfunction properly handles all override cases:
- Status flags are mutually exclusive (cancelled > error)
- Duration uses
typeof !== "undefined"to allow zero values- Events are correctly merged by concatenation
The fix from the past review comment (using
typeof !== "undefined"instead of a falsy check) has been correctly implemented at line 275.
1057-1057: LGTM!Using
run.taskVersioninstead oflockedToVersionaligns with the data model changes in this PR.apps/webapp/app/v3/dynamicFlushScheduler.server.ts (4)
2-2: LGTM!The
tryCatchimport is used for safe error handling in the flush logic.
201-247: LGTM! Retry logic correctly extracted.The refactored
tryFlushnested function cleanly encapsulates the retry logic with exponential backoff (500ms delay between attempts, max 3 attempts). Error tracking and metrics are properly maintained.
249-255: LGTM!The
tryCatchwrapper provides safe error handling, ensuring that one failed batch doesn't crash the entire scheduler.
261-266: LGTM! Shutdown deadlock fixed.The condition
this.consecutiveFlushFailures < 3 || this.isShuttingDowncorrectly allows recursive flushing during shutdown (to drain remaining batches) while still protecting against runaway recursion on repeated failures in normal operation. This addresses the past review concern.Based on learnings.
apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
internal-packages/clickhouse/schema/008_add_task_events_v1_mvs.sql(1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (4)
internal-packages/clickhouse/schema/008_add_task_events_v1_mvs.sql (4)
2-13: LGTM! Minute-level aggregation table is well-structured.The
SummingMergeTreeengine is appropriate for aggregating event counts, and the ordering key aligns with the materialized view's GROUP BY clause. The 8-day TTL is reasonable for high-granularity minute-level data.
15-26: LGTM! Hour-level aggregation table mirrors the minute-level design.The 400-day TTL provides over 13 months of retention for hourly aggregates, which is appropriate for longer-term analysis. The structure is consistent with the minute-level table.
28-37: LGTM! Minute-level materialized view correctly aggregates from source events.The materialized view properly transforms raw task events into per-minute aggregates using
toStartOfMinute(start_time)for bucketing andcount()for event counting. The GROUP BY matches the target table's ordering key.
39-48: LGTM! Hour-level materialized view correctly chains from minute-level table.The view correctly sources from the minute-level table (not the MV), which is the proper approach in ClickHouse. Using
sum(event_count)aggregates the pre-summed minute counts, andtoStartOfHour(bucket_start)buckets them into hourly intervals.
internal-packages/clickhouse/schema/008_add_task_events_v1_mvs.sql
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts (1)
32-35: Honor the EVENTS_CLICKHOUSE_COMPRESSION_REQUEST flagRequest compression is hardcoded to
true, so operators cannot disable it even when the new env flag is set. Please derive the boolean fromenv.EVENTS_CLICKHOUSE_COMPRESSION_REQUEST(e.g. treat"1","true"case-insensitive as true, otherwise false) instead of forcing compression on.Apply this diff:
- compression: { - request: true, - }, + compression: { + request: env.EVENTS_CLICKHOUSE_COMPRESSION_REQUEST === "1", + },apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (1)
179-181: Fix bigint comparison in span event filter
BigInt(event.startTime)throws whenevent.startTimeis already a bigint (which it is), so this filter will explode the first time it runs. Compare against the bigint directly instead of re-wrapping it.- return records.filter((r) => BigInt(r.start_time) > BigInt(event.startTime)); + return records.filter((r) => BigInt(r.start_time) > event.startTime);
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
apps/webapp/app/env.server.ts(2 hunks)apps/webapp/app/presenters/v3/RunPresenter.server.ts(4 hunks)apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsx(5 hunks)apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts(1 hunks)apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts(1 hunks)apps/webapp/app/v3/eventRepository/eventRepository.server.ts(27 hunks)apps/webapp/app/v3/eventRepository/eventRepository.types.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
apps/webapp/app/presenters/v3/RunPresenter.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsxapps/webapp/app/v3/eventRepository/eventRepository.types.tsapps/webapp/app/env.server.tsapps/webapp/app/v3/eventRepository/eventRepository.server.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
apps/webapp/app/presenters/v3/RunPresenter.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsxapps/webapp/app/v3/eventRepository/eventRepository.types.tsapps/webapp/app/env.server.tsapps/webapp/app/v3/eventRepository/eventRepository.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
When importing from @trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from @trigger.dev/core’s package.json
Files:
apps/webapp/app/presenters/v3/RunPresenter.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsxapps/webapp/app/v3/eventRepository/eventRepository.types.tsapps/webapp/app/env.server.tsapps/webapp/app/v3/eventRepository/eventRepository.server.ts
{apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Files:
apps/webapp/app/presenters/v3/RunPresenter.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/env.server.tsapps/webapp/app/v3/eventRepository/eventRepository.server.ts
apps/webapp/app/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Modules intended for test consumption under apps/webapp/app/**/*.ts must not read environment variables; accept configuration via options instead
Files:
apps/webapp/app/presenters/v3/RunPresenter.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.tsapps/webapp/app/v3/eventRepository/eventRepository.types.tsapps/webapp/app/env.server.tsapps/webapp/app/v3/eventRepository/eventRepository.server.ts
🧬 Code graph analysis (7)
apps/webapp/app/presenters/v3/RunPresenter.server.ts (4)
apps/webapp/app/env.server.ts (1)
env(1202-1202)apps/webapp/app/v3/eventRepository/eventRepository.server.ts (1)
eventRepository(1476-1476)apps/webapp/app/v3/eventRepository/index.server.ts (1)
resolveEventRepositoryForStore(10-18)apps/webapp/app/v3/eventRepository/eventRepository.types.ts (1)
SpanSummary(246-263)
apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts (3)
apps/webapp/app/env.server.ts (1)
env(1202-1202)internal-packages/clickhouse/src/index.ts (1)
ClickHouse(61-172)apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (1)
ClickhouseEventRepository(77-1657)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (8)
internal-packages/clickhouse/src/index.ts (1)
ClickHouse(61-172)apps/webapp/app/v3/eventRepository/eventRepository.types.ts (13)
IEventRepository(311-417)CreateEventInput(24-58)TraceEventOptions(95-106)EventBuilder(108-114)TraceAttributes(86-91)CompleteableTaskRun(67-80)TraceSummary(272-276)SpanSummary(246-263)SpanOverride(265-270)SpanDetail(184-223)SpanSummaryCommon(229-244)TraceDetailedSummary(298-301)SpanDetailedSummary(278-296)apps/webapp/app/v3/dynamicFlushScheduler.server.ts (1)
DynamicFlushScheduler(21-423)internal-packages/clickhouse/src/taskEvents.ts (8)
TaskEventV1Input(5-22)TaskEventV1Input(24-24)TaskEventSummaryV1Result(40-50)TaskEventSummaryV1Result(52-52)TaskEventDetailsV1Result(111-121)TaskEventDetailsV1Result(123-123)TaskEventDetailedSummaryV1Result(73-84)TaskEventDetailedSummaryV1Result(86-86)apps/webapp/app/v3/services/tracePubSub.server.ts (1)
tracePubSub(62-62)apps/webapp/app/v3/eventRepository/common.server.ts (10)
parseEventsField(88-100)convertDateToNanoseconds(43-45)removePrivateProperties(134-156)extractContextFromCarrier(8-21)getNowInNanoseconds(23-25)calculateDurationFromStart(31-35)generateDeterministicSpanId(51-63)generateSpanId(71-73)createExceptionPropertiesFromError(102-131)calculateDurationFromStartJsDate(37-41)packages/core/src/v3/isomorphic/traceContext.ts (1)
serializeTraceparent(27-29)apps/webapp/app/v3/eventRepository/originalRunIdCache.server.ts (1)
originalRunIdCache(62-75)
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsx (1)
apps/webapp/app/v3/eventRepository/eventRepository.types.ts (1)
SpanOverride(265-270)
apps/webapp/app/v3/eventRepository/eventRepository.types.ts (3)
apps/webapp/app/v3/taskEventStore.server.ts (2)
DetailedTraceEvent(25-43)TaskEventStoreTable(45-45)apps/webapp/app/v3/eventRepository/eventRepository.server.ts (4)
events(1469-1473)storeTable(854-956)storeTable(958-1005)storeTable(1007-1068)apps/webapp/app/v3/otlpExporter.server.ts (1)
events(115-121)
apps/webapp/app/env.server.ts (1)
apps/webapp/app/v3/services/bulk/BulkActionV2.server.ts (1)
process(87-357)
apps/webapp/app/v3/eventRepository/eventRepository.server.ts (7)
apps/webapp/app/v3/eventRepository/eventRepository.types.ts (3)
CreateEventInput(24-58)CompleteableTaskRun(67-80)SpanDetail(184-223)apps/webapp/app/v3/dynamicFlushScheduler.server.ts (1)
DynamicFlushScheduler(21-423)apps/webapp/app/v3/taskEventStore.server.ts (2)
TaskEventStore(57-306)TaskEventStoreTable(45-45)apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (6)
events(129-132)spanId(1018-1110)spanId(1240-1326)spanId(1470-1552)span(1639-1656)flushId(109-127)apps/webapp/app/v3/eventRepository/common.server.ts (4)
convertDateToNanoseconds(43-45)generateTraceId(67-69)generateDeterministicSpanId(51-63)generateSpanId(71-73)apps/webapp/app/v3/eventRepository/originalRunIdCache.server.ts (1)
originalRunIdCache(62-75)apps/webapp/app/v3/services/tracePubSub.server.ts (1)
tracePubSub(62-62)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (15)
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsx (5)
100-100: LGTM!Import correctly structured for the SpanOverride type usage.
175-175: LGTM!Correctly sources
maximumLiveReloadingSettingfrom the presenter result, enabling per-repository configuration.
305-306: LGTM!Correctly destructures
overridesBySpanIdfrom the trace data for per-span override support.
328-329: LGTM!Correctly computes per-span overrides using optional chaining. The logic handles both missing
overridesBySpanIdand missing entries gracefully.
378-378: LGTM!Correctly passes
spanOverridesto theSpanViewcomponent. The type assertion ensures type safety at the call site.apps/webapp/app/presenters/v3/RunPresenter.server.ts (8)
6-10: LGTM!Imports correctly structured for the new repository resolution and fallback trace logic.
50-54: LGTM!Added fields (
taskIdentifier,parentSpanId) are correctly included in the Prisma select and used in the fallback trace logic.
144-144: LGTM!Correctly provides fallback
maximumLiveReloadingSettingfrom env when logs are not available.
148-149: LGTM!Correctly resolves the event repository based on the run's store configuration, enabling per-store event handling.
151-158: LGTM!Correctly passes
run.runtimeEnvironment.idtogetTraceSummary, enabling environment-scoped trace retrieval.
160-194: LGTM!Fallback trace logic correctly synthesizes a minimal
SpanSummaryfrom run metadata when trace data is unavailable. The status derivation forisError,isPartial, andisCancelledappropriately maps run statuses to span states, providing graceful degradation.
261-261: LGTM!Correctly propagates
overridesBySpanIdfrom the trace summary to enable per-span overrides in the UI.
263-263: LGTM!Correctly sources
maximumLiveReloadingSettingfrom the resolved event repository, enabling per-store configuration.apps/webapp/app/env.server.ts (2)
495-495: LGTM!New
RESOURCE_MONITOR_ENABLEDflag correctly structured with safe default (disabled).
1113-1128: LGTM!Comprehensive Clickhouse event repository configuration correctly structured with:
- Fallback to existing
CLICKHOUSE_URLfor reuse- Safe defaults for batch/flush operations
- Proper enum constraints for store selection
- Optional rollout percentage for gradual migration
Note:
EVENTS_CLICKHOUSE_MAX_LIVE_RELOADING_SETTING(2000) is intentionally higher thanMAXIMUM_LIVE_RELOADING_EVENTS(1000), reflecting Clickhouse's ability to handle larger event volumes.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (2)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (2)
158-158: Past concern: Guard attributes against undefined.A previous review flagged that
event.propertiesmay be undefined. Looking at line 354,createEventToTaskEventV1InputAttributesacceptsAttributes(non-optional). IfCreateEventInput.propertiescan beundefined, passevent.properties ?? {}or update the helper signature to acceptAttributes | undefined.Based on learnings from past review comments.
178-180: Past concern: Redundant BigInt conversion.
event.startTimeis already abigint, so wrapping it inBigInt()is unnecessary. The comparison should be:- return records.filter((r) => BigInt(r.start_time) > BigInt(event.startTime)); + return records.filter((r) => BigInt(r.start_time) > event.startTime);Based on learnings from past review comments.
🧹 Nitpick comments (5)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (5)
487-502: Improve type safety in setAttribute merging.The
setAttributeimplementation uses@ts-ignoretwice (lines 496, 499) to bypass type checking when merging attributes. Consider defining a helper function with proper types or using a type assertion that's more specific than@ts-ignoreto maintain better type safety.
1724-1743: Fast path ignores timezone offsets.The fast path optimization (lines 1724-1743) only handles the common format
"2025-09-23 12:32:46.130262875"but doesn't check for or handle timezone offsets likeZ,+HH:MM, or-HH:MM. If ClickHouse returns dates with offsets in this format (unlikely but possible), the fast path will produce incorrect results. Either:
- Add a check to fall through to regex if offset characters are detected (e.g., check for
Z,+,-after position 19), or- Document that the fast path assumes no timezone suffix.
// Fast path for common format: "2025-09-23 12:32:46.130262875" or "2025-09-23 12:32:46" // This avoids the expensive regex for the common case - if (date.length >= 19 && date[4] === "-" && date[7] === "-" && date[10] === " ") { + if (date.length >= 19 && date[4] === "-" && date[7] === "-" && date[10] === " " && + !date.includes("Z") && !date.includes("+") && !date.includes("-", 11)) {Based on learnings from past review comments.
1112-1118: Consider adding error handling for JSON.parse.
#parseAttributescallsJSON.parsewithout error handling (line 1117). Ifattributes_textcontains malformed JSON (due to corruption or bugs), this will throw and crash the query. Consider wrapping in try-catch and returning{}on parse failure, logging the error for debugging.#parseAttributes(attributes_text: string): Record<string, unknown> { if (!attributes_text) { return {}; } - return JSON.parse(attributes_text) as Record<string, unknown>; + try { + return JSON.parse(attributes_text) as Record<string, unknown>; + } catch (e) { + logger.error("Failed to parse attributes_text", { attributes_text, error: e }); + return {}; + } }
1328-1354: Consider adding error handling for JSON.parse in #parseMetadata.Similar to
#parseAttributes,#parseMetadatacallsJSON.parse(line 1342) without error handling. Malformed metadata could crash queries. Wrap in try-catch and returnundefinedon parse failure.#parseMetadata( metadata: string, cache: Map<string, Record<string, unknown>> ): Record<string, unknown> | undefined { if (!metadata) { return undefined; } const cached = cache.get(metadata); if (cached) { return cached; } - const parsed = JSON.parse(metadata); + let parsed; + try { + parsed = JSON.parse(metadata); + } catch (e) { + logger.error("Failed to parse metadata", { metadata, error: e }); + return undefined; + } if (typeof parsed !== "object" || parsed === null) { return undefined; } const result = parsed as Record<string, unknown>; cache.set(metadata, result); return result; }
88-102: Load shedding is disabled but isDroppableEvent is defined.
loadSheddingEnabledis set tofalse(line 97), which means theisDroppableEventfunction (lines 98-101) will never be called. If load shedding is not needed, consider removing theisDroppableEventconfiguration to simplify the code. If it's intended for future use, add a comment explaining the plan.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
When importing from @trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from @trigger.dev/core’s package.json
Files:
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts
{apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Files:
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts
apps/webapp/app/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Modules intended for test consumption under apps/webapp/app/**/*.ts must not read environment variables; accept configuration via options instead
Files:
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts
🧬 Code graph analysis (1)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (7)
internal-packages/clickhouse/src/index.ts (1)
ClickHouse(61-172)apps/webapp/app/v3/eventRepository/eventRepository.types.ts (9)
IEventRepository(311-417)CreateEventInput(24-58)TraceEventOptions(95-106)EventBuilder(108-114)TraceAttributes(86-91)CompleteableTaskRun(67-80)TraceSummary(272-276)SpanDetail(184-223)TraceDetailedSummary(298-301)apps/webapp/app/v3/dynamicFlushScheduler.server.ts (1)
DynamicFlushScheduler(21-423)internal-packages/clickhouse/src/taskEvents.ts (8)
TaskEventV1Input(5-22)TaskEventV1Input(24-24)TaskEventSummaryV1Result(40-50)TaskEventSummaryV1Result(52-52)TaskEventDetailsV1Result(111-121)TaskEventDetailsV1Result(123-123)TaskEventDetailedSummaryV1Result(73-84)TaskEventDetailedSummaryV1Result(86-86)apps/webapp/app/v3/services/tracePubSub.server.ts (1)
tracePubSub(62-62)apps/webapp/app/v3/eventRepository/common.server.ts (8)
parseEventsField(88-100)convertDateToNanoseconds(43-45)removePrivateProperties(134-156)extractContextFromCarrier(8-21)getNowInNanoseconds(23-25)calculateDurationFromStart(31-35)createExceptionPropertiesFromError(102-131)calculateDurationFromStartJsDate(37-41)apps/webapp/app/v3/eventRepository/originalRunIdCache.server.ts (1)
originalRunIdCache(62-75)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (2)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (2)
354-370: Attributes helper already handles undefined correctly.The helper
createEventToTaskEventV1InputAttributescorrectly handles the case whereremovePrivatePropertiesreturnsundefined(lines 357-359), so the previous concern about undefined attributes appears to have been addressed. However, the signature still accepts non-optionalAttributes. Consider updating the signature toattributes?: Attributesfor clarity.
109-127: Ignore Redis publish concern –#publishToRedisis only invoked after theinsertErrorcheck and never runs on failed inserts.Likely an incorrect or invalid review comment.
ericallam
force-pushed
the
feat/clickhouse-otel
branch
from
91572d8 to
2515fd5
Compare
There was a problem hiding this comment.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (8)
packages/core/src/v3/utils/flattenAttributes.ts (1)
265-312: Filtering misses nested keysWith flattened keys like
"body.message", the check only matches exact strings, so passingfilteredKeys = ["body"]still hydrates every descendant. That defeats the intent to drop a whole top-level namespace. Please gate on the first path segment (after array index normalization) before recursing.- if (filteredKeys?.includes(key)) { - continue; - } - - const parts = key.split(".").reduce( + const parts = key.split(".").reduce( (acc, part) => { if (part.startsWith("[") && part.endsWith("]")) { // Handle array indices more precisely const match = part.match(/^\[(\d+)\]$/); if (match && match[1]) { acc.push(parseInt(match[1])); } else { // Remove brackets for non-numeric array keys acc.push(part.slice(1, -1)); } } else { acc.push(part); } return acc; }, [] as (string | number)[] ); + const topLevel = parts[0]; + if ( + filteredKeys && + topLevel !== undefined && + filteredKeys.includes(String(topLevel)) + ) { + continue; + } + let current: any = result;apps/webapp/app/v3/eventRepository/eventRepository.server.ts (2)
164-420: Verify hard-codedenvironmentType: "DEVELOPMENT".All
complete*RunEventmethods hard-codeenvironmentType: "DEVELOPMENT"(lines 178, 222, 258, 303, 352, 399). This will incorrectly classify production run events as development events.The
CompleteableTaskRuntype includes anisTestfield (line 79 in eventRepository.types.ts), but there's noenvironmentTypefield. Verify:
- Should
environmentTypebe derived from the run's actual environment (production vs. development)?- Should
CompleteableTaskRuninclude anenvironmentTypefield?- Is this hard-coded value intentional for a specific use case?
#!/bin/bash # Check if CompleteableTaskRun should include environmentType echo "=== Checking CompleteableTaskRun definition ===" ast-grep --pattern $'export type CompleteableTaskRun = $$$' echo "=== Checking TaskRun model for environmentType field ===" rg -nP --type=prisma -C5 'model TaskRun' echo "=== Checking if environment type is available from runtimeEnvironment ===" rg -nP --type=ts -C3 'runtimeEnvironment.*type' -g '!node_modules' | head -30
739-818: Verifykindfield inSpanDetailreturn type.Line 800 adds a
kindfield to the returned object:kind: spanEvent.kind,However, the
SpanDetailtype definition (lines 183-222 in eventRepository.types.ts) doesn't include akindfield. This will cause a TypeScript compilation error or type mismatch.Verify and apply one of these fixes:
Option 1: Add
kindtoSpanDetailtype definition:In
eventRepository.types.ts, update theSpanDetailtype:export type SpanDetail = { // Core Identity & Structure spanId: string; parentId: string | null; message: string; // Status & State isError: boolean; isPartial: boolean; isCancelled: boolean; level: TaskEventLevel; + kind: TaskEventKind;Option 2: Remove
kindfrom the return value if it's not needed:return { // Core Identity & Structure spanId: spanEvent.spanId, parentId: spanEvent.parentId, message: spanEvent.message, // Status & State isError: span.data.isError, isPartial: span.data.isPartial, isCancelled: span.data.isCancelled, level: spanEvent.level, - kind: spanEvent.kind,apps/webapp/app/v3/marqs/sharedQueueConsumer.server.ts (1)
1-2223: Unify OTEL_RESOURCE_ATTRIBUTES assignment to JSON.stringify in packages/python/src/index.ts
Replace the manual template‐string concatenation of OTEL_RESOURCE_ATTRIBUTES withJSON.stringifyof an object. For example, at each occurrence around lines 35, 95, 157, 207, 254, and 299 inpackages/python/src/index.ts, change fromOTEL_RESOURCE_ATTRIBUTES: `${SemanticInternalAttributes.EXECUTION_ENVIRONMENT}=trigger,${...}`to
OTEL_RESOURCE_ATTRIBUTES: JSON.stringify({ [SemanticInternalAttributes.EXECUTION_ENVIRONMENT]: "trigger", ...taskContext.attributes })apps/webapp/app/v3/runEngineHandlers.server.ts (1)
51-58: Use the DB taskRun’s store when resolving the repositoryIn
runSucceededwe fetchtaskRun(includingtaskEventStore) but still resolve the repository withrun.taskEventStore. The enginerunpayload may omit or stale this field, which would drop us back to the default store and mis-route completion events. UsetaskRun.taskEventStoreinstead so we honor the persisted store.- const eventRepository = resolveEventRepositoryForStore(run.taskEventStore); + const eventRepository = resolveEventRepositoryForStore(taskRun.taskEventStore);apps/webapp/app/presenters/v3/RunPresenter.server.ts (1)
140-146: Unify maximumLiveReloadingSetting source (env vs repository).Early-return uses env.MAXIMUM_LIVE_RELOADING_EVENTS while the success path uses eventRepository.maximumLiveReloadingSetting. This can diverge per store. Resolve via repository in both paths.
- return { - run: runData, - trace: undefined, - maximumLiveReloadingSetting: env.MAXIMUM_LIVE_RELOADING_EVENTS, - }; + return { + run: runData, + trace: undefined, + maximumLiveReloadingSetting: resolveEventRepositoryForStore(run.taskEventStore) + .maximumLiveReloadingSetting, + };internal-packages/clickhouse/src/client/noop.ts (1)
38-52: Interface mismatch: query() missing name and settings.NoopClient implements ClickhouseReader but omits required fields. This breaks type compatibility and call sites (e.g., builder.execute).
- public query<TIn extends z.ZodSchema<any>, TOut extends z.ZodSchema<any>>(req: { - query: string; - params?: TIn; - schema: TOut; - }): (params: z.input<TIn>) => Promise<Result<z.output<TOut>[], QueryError>> { + public query<TIn extends z.ZodSchema<any>, TOut extends z.ZodSchema<any>>(req: { + name: string; + query: string; + params?: TIn; + schema: TOut; + settings?: ClickHouseSettings; + }): (params: z.input<TIn>) => Promise<Result<z.output<TOut>[], QueryError>> { return async (params: z.input<TIn>) => { const validParams = req.params?.safeParse(params);apps/webapp/app/runEngine/concerns/traceEvents.server.ts (1)
110-121: Use the resolved repository for recordEvent to keep writes in the selected store.Calling this.eventRepository here can split writes across stores when ClickHouse is selected. Use repository.recordEvent captured above so the log lands with the span in the same store.
- await this.eventRepository.recordEvent( + await repository.recordEvent( `There's an existing run for idempotencyKey: ${idempotencyKey}`, { taskSlug: request.taskId, environment: request.environment, attributes: { runId: existingRun.friendlyId, }, context: request.options?.traceContext, parentId: event.spanId, } );
♻️ Duplicate comments (8)
packages/cli-v3/src/commands/dev.ts (1)
149-154: Add error handling for the rules install wizard.The error from
initiateRulesInstallWizardis still being silently swallowed. The MCP install pattern (lines 130-140) captures and logs errors; this code should follow the same approach to inform users when rules installation fails.Apply this diff to add error handling:
- await tryCatch( - initiateRulesInstallWizard({ - manifestPath: options.rulesInstallManifestPath, - branch: options.rulesInstallBranch, - }) - ); + const [rulesError] = await tryCatch( + initiateRulesInstallWizard({ + manifestPath: options.rulesInstallManifestPath, + branch: options.rulesInstallBranch, + }) + ); + + if (rulesError) { + log.error(`Failed to install agent rules: ${rulesError.message}`); + }apps/webapp/app/routes/engine.v1.dev.runs.$runFriendlyId.snapshots.$snapshotFriendlyId.attempts.start.ts (1)
99-110: Fix OTEL_RESOURCE_ATTRIBUTES format and merge behavior (spec requires key=value, not JSON)Currently sets OTEL_RESOURCE_ATTRIBUTES to a JSON string and overwrites any existing value. Build a comma‑separated key=value string, merge with existing OTEL_RESOURCE_ATTRIBUTES, and keep the JSON copy in TRIGGER_OTEL_RESOURCE_ATTRIBUTES for internal use. This was flagged earlier and remains unresolved here.
- if (taskEventStore) { - const resourceAttributes = JSON.stringify({ - [SemanticInternalAttributes.TASK_EVENT_STORE]: taskEventStore, - }); - - variables.push( - ...[ - { key: "OTEL_RESOURCE_ATTRIBUTES", value: resourceAttributes }, - { key: "TRIGGER_OTEL_RESOURCE_ATTRIBUTES", value: resourceAttributes }, - ] - ); - } + if (taskEventStore) { + const otelAttr = `${SemanticInternalAttributes.TASK_EVENT_STORE}=${taskEventStore}`; + const existing = variables.find((v) => v.key === "OTEL_RESOURCE_ATTRIBUTES")?.value; + const merged = + existing + ? (existing.includes(`${SemanticInternalAttributes.TASK_EVENT_STORE}=`) ? existing : `${existing},${otelAttr}`) + : otelAttr; + + variables.push({ key: "OTEL_RESOURCE_ATTRIBUTES", value: merged }); + + const triggerResourceAttributes = JSON.stringify({ + [SemanticInternalAttributes.TASK_EVENT_STORE]: taskEventStore, + }); + variables.push({ key: "TRIGGER_OTEL_RESOURCE_ATTRIBUTES", value: triggerResourceAttributes }); + }apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts (1)
24-36: Honor the compression toggle for ClickHouse requests
compression.requestis still hard-coded totrue, so the newEVENTS_CLICKHOUSE_COMPRESSION_REQUESTflag never disables request compression. That makes the operator control ineffective. Please derive the boolean from the env value (treating undefined as the current default oftrue) before constructing the client.- const clickhouse = new ClickHouse({ + const shouldCompressRequests = + env.EVENTS_CLICKHOUSE_COMPRESSION_REQUEST === undefined + ? true + : ["1", "true", "TRUE"].includes( + String(env.EVENTS_CLICKHOUSE_COMPRESSION_REQUEST) + ); + + const clickhouse = new ClickHouse({ url: url.toString(), name: "task-events", keepAlive: { enabled: env.EVENTS_CLICKHOUSE_KEEP_ALIVE_ENABLED === "1", idleSocketTtl: env.EVENTS_CLICKHOUSE_KEEP_ALIVE_IDLE_SOCKET_TTL_MS, }, logLevel: env.EVENTS_CLICKHOUSE_LOG_LEVEL, compression: { - request: true, + request: shouldCompressRequests, }, maxOpenConnections: env.EVENTS_CLICKHOUSE_MAX_OPEN_CONNECTIONS, });apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (3)
354-370: Make attributes helper null‑safe and align callers.Accept undefined and return {} consistently; avoids accidental crashes if a caller ever passes undefined.
- private createEventToTaskEventV1InputAttributes(attributes: Attributes): Record<string, unknown> { - const publicAttributes = removePrivateProperties(attributes); + private createEventToTaskEventV1InputAttributes( + attributes?: Attributes + ): Record<string, unknown> { + const publicAttributes = removePrivateProperties(attributes);
179-181: Fix bigint comparison when filtering span events.event.startTime is already a bigint; don’t wrap it in BigInt(). Compare the record value (cast) to the bigint directly.
- return records.filter((r) => BigInt(r.start_time) > BigInt(event.startTime)); + return records.filter((r) => BigInt(r.start_time) > event.startTime);
1724-1768: DateTime64 → JS Date ignores timezone offsets (fast-path applies when it shouldn’t; fallback doesn’t adjust).When strings have Z or ±HH:MM, the fast path still runs and the fallback path drops the offset. This shifts times in the UI.
Apply both fixes:
- Only take the fast path when there is no trailing offset/Z.
- In the fallback, parse and subtract the offset minutes (mirror the nanoseconds converter).
export function convertClickhouseDateTime64ToJsDate(date: string): Date { - // Fast path for common format: "2025-09-23 12:32:46.130262875" or "2025-09-23 12:32:46" - // This avoids the expensive regex for the common case - if (date.length >= 19 && date[4] === "-" && date[7] === "-" && date[10] === " ") { + // Fast path for common format without timezone suffix: + // "YYYY-MM-DD HH:MM:SS(.fff...)" with no trailing Z or ±HH:MM + const hasTzSuffix = date.endsWith("Z") || /[+-]\d{2}:?\d{2}$/.test(date); + if (!hasTzSuffix && date.length >= 19 && date[4] === "-" && date[7] === "-" && date[10] === " ") { const year = Number(date.substring(0, 4)); const month = Number(date.substring(5, 7)); const day = Number(date.substring(8, 10)); const hour = Number(date.substring(11, 13)); const minute = Number(date.substring(14, 16)); const second = Number(date.substring(17, 19)); // Parse fractional seconds if present let ms = 0; if (date.length > 20 && date[19] === ".") { // Take first 3 digits after decimal (milliseconds), pad if shorter const fracStr = date.substring(20, Math.min(23, date.length)); ms = Number(fracStr.padEnd(3, "0")); } return new Date(Date.UTC(year, month - 1, day, hour, minute, second, ms)); } // Fallback to regex for other formats (T separator, timezone offsets, etc.) const s = date.trim(); const m = CLICKHOUSE_DATETIME_REGEX.exec(s); if (!m) { throw new Error(`Invalid ClickHouse DateTime64 string: "${date}"`); } const year = Number(m[1]); const month = Number(m[2]); // 1-12 const day = Number(m[3]); // 1-31 const hour = Number(m[4]); const minute = Number(m[5]); const second = Number(m[6]); const fraction = m[7] ?? ""; // up to 9 digits + const sign = m[8] as "+" | "-" | undefined; + const offH = m[9] ? Number(m[9]) : 0; + const offM = m[10] ? Number(m[10]) : 0; // Convert fractional seconds to exactly 9 digits (nanoseconds within the second). const nsWithinSecond = Number(fraction.padEnd(9, "0")); // 0..999_999_999 // Split into millisecond part (for Date) const msPart = Math.trunc(nsWithinSecond / 1_000_000); // 0..999 - return new Date(Date.UTC(year, month - 1, day, hour, minute, second, msPart)); + let msEpoch = Date.UTC(year, month - 1, day, hour, minute, second, msPart); + if (sign) { + const offsetMinutesSigned = (sign === "+" ? 1 : -1) * (offH * 60 + offM); + msEpoch -= offsetMinutesSigned * 60_000; + } + return new Date(msEpoch); }packages/core/src/v3/serverOnly/resourceMonitor.ts (1)
304-307: Only skip our own PID when scanning.Re-using the
<= process.pidguard still drops every target that started before this monitor, so metrics stay empty. Please revert to skipping onlyprocess.pid(as noted earlier) in both the /proc scan and ps fallback.- if (processPid <= process.pid) { + if (processPid === process.pid) { continue; } … - // Ignore processes that have a lower PID than our own PID - return pid > process.pid; + return pid !== process.pid;Also applies to: 369-371
apps/webapp/app/v3/otlpExporter.server.ts (1)
259-260: Stop hard-codingenvironmentType.
environmentTyperemains forced to"DEVELOPMENT", so prod/test data lands in the wrong bucket. Please derive it from the OTLP resource/log attributes (e.g.,SemanticInternalAttributes.ENVIRONMENT_TYPE) with a final fallback, and keep both logs & spans in sync.Also applies to: 347-349
🧹 Nitpick comments (25)
apps/webapp/app/v3/services/expireEnqueuedRun.server.ts (1)
81-82: Move eventRepository instantiation inside the ttl conditional block.The
eventRepositoryis only used whenrun.ttlis truthy (line 83). Instantiating it unconditionally at line 81 performs unnecessary work whenrun.ttlis falsy.Apply this diff to defer instantiation:
- const eventRepository = resolveEventRepositoryForStore(run.taskEventStore); - if (run.ttl) { + const eventRepository = resolveEventRepositoryForStore(run.taskEventStore); + const [completeExpiredRunEventError] = await tryCatch(apps/webapp/app/eventLoopMonitor.server.ts (1)
68-69: Consider using structured logging for consistency and observability.The new console.log statement will not be captured by structured log aggregation systems. Since the file already imports
logger(line 7) and uses it for structured logging elsewhere (line 110), consider usinglogger.warn()orlogger.info()here for consistency.Apply this diff to use structured logging:
- console.log("🥸 Event loop blocked", { asyncType: cached.type, time }); + logger.warn("event_loop_blocked", { asyncType: cached.type, time });This change would align better with the PR's observability goals (storing log data in ClickHouse) and ensure the event-loop-blocked events are captured in your logging infrastructure.
apps/webapp/app/routes/engine.v1.dev.runs.$runFriendlyId.snapshots.$snapshotFriendlyId.attempts.start.ts (1)
2-2: Use type‑only import for MachinePresetAvoid pulling the Zod schema value at runtime; import the type only.
-import { MachinePreset, SemanticInternalAttributes } from "@trigger.dev/core/v3"; +import type { MachinePreset } from "@trigger.dev/core/v3"; +import { SemanticInternalAttributes } from "@trigger.dev/core/v3";apps/webapp/app/v3/marqs/sharedQueueConsumer.server.ts (2)
25-26: Import is fine; consider type‑only import for MachinePreset in this fileReduce runtime bundle by importing MachinePreset as a type.
-import { - AckCallbackResult, - MachinePreset, +import { + AckCallbackResult, V3ProdTaskRunExecution, V3ProdTaskRunExecutionPayload, TaskRunError, TaskRunErrorCodes, TaskRunExecution, TaskRunExecutionLazyAttemptPayload, TaskRunExecutionResult, TaskRunFailedExecutionResult, TaskRunSuccessfulExecutionResult, parsePacket, serverWebsocketMessages, SemanticInternalAttributes, } from "@trigger.dev/core/v3"; +import type { MachinePreset } from "@trigger.dev/core/v3";
2187-2219: De‑duplicate this OTEL attribute injection with a shared helperBoth this file and the route construct the same env vars. Extract a small utility (e.g., addOtelResourceAttributes(vars, attrs)) to centralize kv/JSON building and merging.
apps/webapp/app/v3/services/triggerTask.server.ts (1)
97-101: LGTM: per-store tracing is correctly DI’dPassing both repositories into DefaultTraceEventsConcern aligns with store-aware routing. To avoid per-call instantiation, consider memoizing the concern as a class field.
export class TriggerTaskService extends WithRunEngine { + private readonly traceEventConcern = new DefaultTraceEventsConcern( + eventRepository, + clickhouseEventRepository + ); public async call( @@ private async callV2( @@ - const traceEventConcern = new DefaultTraceEventsConcern( - eventRepository, - clickhouseEventRepository - ); + const traceEventConcern = this.traceEventConcern;apps/webapp/app/v3/services/triggerTaskV1.server.ts (1)
729-773: Minor: prefer logger over console for consistencyOptional: replace console.log with logger to keep structured logs and consistent levels.
- console.log("Failed to get queue name: No task found", { + logger.debug("Failed to get queue name: No task found", { @@ - console.log("Failed to get queue name: Invalid queue config", { + logger.debug("Failed to get queue name: Invalid queue config", {apps/webapp/app/v3/eventRepository/eventRepository.types.ts (2)
222-222: Consider stronger typing for metadata.The
metadatafield is typed asany, which eliminates type safety. Consider usingunknownor a more specific type to maintain type safety while allowing flexibility.Apply this diff to improve type safety:
- metadata: any; // Used by SpanPresenter for entity processing + metadata: unknown; // Used by SpanPresenter for entity processingThen add proper type guards or validation where
metadatais accessed.
311-417: Consider using type instead of interface.The coding guidelines indicate "We usually use types over interfaces" for TypeScript. While
interfaceworks here, consider refactoring to a type for consistency.Apply this diff to align with coding guidelines:
-export interface IEventRepository { +export type IEventRepository = { maximumLiveReloadingSetting: number; // Event insertion methods insertMany(events: CreateEventInput[]): Promise<void>; ... -} +};As per coding guidelines.
apps/webapp/app/routes/_app.orgs.$organizationSlug.projects.$projectParam.env.$envParam.runs.$runParam/route.tsx (1)
378-378: Remove redundant type assertion.The type assertion
as SpanOverride | undefinedis redundant sincespanOverridesis already inferred asSpanOverride | undefinedfrom line 328.Apply this diff:
- spanOverrides={spanOverrides as SpanOverride | undefined} + spanOverrides={spanOverrides}apps/webapp/app/presenters/v3/RunPresenter.server.ts (2)
10-11: env import becomes redundant after unifying max setting.If you adopt the change above, you can drop the env import.
- import { env } from "~/env.server"; + // env import no longer neededAlso applies to: 148-149
258-264: Harden overridesBySpanId default.Fallback traceSummary lacks overridesBySpanId; default to {} to keep shape consistent.
- overridesBySpanId: traceSummary.overridesBySpanId, + overridesBySpanId: traceSummary.overridesBySpanId ?? {},apps/webapp/app/env.server.ts (1)
1114-1130: Tighten ClickHouse env validation and rollout bounds.
- Make EVENTS_CLICKHOUSE_URL required after fallback (consistent with V4_* pattern).
- Use BoolEnv for compression request.
- Bound rollout percent to 0–100.
- EVENTS_CLICKHOUSE_URL: z - .string() - .optional() - .transform((v) => v ?? process.env.CLICKHOUSE_URL), + EVENTS_CLICKHOUSE_URL: z + .string() + .optional() + .transform((v) => v ?? process.env.CLICKHOUSE_URL) + .pipe(z.string().min(1)), - EVENTS_CLICKHOUSE_COMPRESSION_REQUEST: z.string().default("1"), + EVENTS_CLICKHOUSE_COMPRESSION_REQUEST: BoolEnv.default(true), - EVENT_REPOSITORY_CLICKHOUSE_ROLLOUT_PERCENT: z.coerce.number().optional(), + EVENT_REPOSITORY_CLICKHOUSE_ROLLOUT_PERCENT: z.coerce.number().int().min(0).max(100).optional(),internal-packages/clickhouse/src/client/noop.ts (1)
54-63: queryFast columns type should accept ColumnExpression.Match the interface and the fast builder.
- public queryFast<TOut extends Record<string, any>, TParams extends Record<string, any>>(req: { + public queryFast<TOut extends Record<string, any>, TParams extends Record<string, any>>(req: { name: string; query: string; - columns: string[]; + columns: Array<string | ColumnExpression>; settings?: ClickHouseSettings; }): (params: TParams) => Promise<Result<TOut[], QueryError>> {internal-packages/clickhouse/src/client/queryBuilder.ts (1)
129-134: Type nit: narrow whereIf condition to boolean.Small clarity/readability win.
- whereIf(condition: any, clause: string, params?: QueryParams): this { + whereIf(condition: boolean, clause: string, params?: QueryParams): this {apps/webapp/app/runEngine/concerns/traceEvents.server.ts (1)
16-22: Remove unused ClickhouseEventRepository dependency from this class.It isn’t referenced after introducing #getEventRepository; leaving it adds noise and DI burden.
- private readonly clickhouseEventRepository: ClickhouseEventRepository; ... - constructor( - eventRepository: EventRepository, - clickhouseEventRepository: ClickhouseEventRepository - ) { + constructor(eventRepository: EventRepository) { this.eventRepository = eventRepository; - this.clickhouseEventRepository = clickhouseEventRepository; }If other constructors still pass the second arg, update call sites accordingly.
internal-packages/clickhouse/src/taskEvents.ts (2)
26-38: Consider typed insert with the existing schema for safer dev-time validation.You already define TaskEventV1Input; using insert(...) provides shape validation while keeping your JSON settings.
-export function insertTaskEvents(ch: ClickhouseWriter, settings?: ClickHouseSettings) { - return ch.insertUnsafe<TaskEventV1Input>({ +export function insertTaskEvents(ch: ClickhouseWriter, settings?: ClickHouseSettings) { + return ch.insert(TaskEventV1Input)({ name: "insertTaskEvents", table: "trigger_dev.task_events_v1", settings: { enable_json_type: 1, type_json_skip_duplicated_paths: 1, input_format_json_throw_on_bad_escape_sequence: 0, input_format_json_use_string_type_for_ambiguous_paths_in_named_tuples_inference_from_objects: 1, ...settings, }, }); }If perf is a concern, keep insertUnsafe; otherwise the typed path can catch regressions early.
55-70: Align operation name with the function (“getTraceSummary”) for clearer tracing/metrics.Purely cosmetic but helps observability.
- name: "getTraceEvents", + name: "getTraceSummary",internal-packages/clickhouse/src/client/types.ts (1)
4-7: Use type-only imports to avoid unnecessary runtime dependencies and potential cycles.-import { ClickHouseSettings } from "@clickhouse/client"; +import type { ClickHouseSettings } from "@clickhouse/client"; ... -import { ClickhouseQueryBuilder, ClickhouseQueryFastBuilder } from "./queryBuilder.js"; +import type { ClickhouseQueryBuilder, ClickhouseQueryFastBuilder } from "./queryBuilder.js";apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (3)
315-316: Avoid double unflattening on span event metadata.spanEvent.properties was already unflattened by parseEventsField; stringify directly.
- metadata: JSON.stringify(unflattenAttributes(spanEvent.properties as Attributes)), + metadata: JSON.stringify(spanEvent.properties),
98-101: Comment/code mismatch about load shedding drop policy.Comment says “drop LOG events” but code only drops DEBUG_EVENT. Align the comment to prevent confusion.
- // Only drop LOG events during load shedding + // Only drop debug log events during load shedding return event.kind === "DEBUG_EVENT";
1664-1665: Docstring example implies a timezone shift that doesn’t happen.toISOString() is UTC; replacing “T”/removing “Z” does not add +1h. Fix the example.
- // 2024-11-06T20:37:00.123Z -> 2024-11-06 21:37:00.123 + // 2024-11-06T20:37:00.123Z -> 2024-11-06 20:37:00.123apps/webapp/app/v3/services/tracePubSub.server.ts (1)
43-47: Propagate and log Redis subscriber errors; harden unsubscribe.Surface errors to consumers and avoid silent failures; ensure quit() is awaited safely.
- // Define the message handler. - redis.on("message", (_, message) => { - eventEmitter.emit("message", message); - }); + // Define handlers. + redis.on("message", (_, message) => eventEmitter.emit("message", message)); + redis.on("error", (err) => eventEmitter.emit("error", err)); + redis.on("end", () => eventEmitter.emit("end")); + redis.on("reconnecting", () => eventEmitter.emit("reconnecting")); @@ - await redis.unsubscribe(channel); - redis.quit(); + try { + await redis.unsubscribe(channel); + } finally { + await redis.quit(); + } this._subscriberCount--;Also applies to: 49-53
internal-packages/clickhouse/src/client/client.ts (2)
312-326: Guard fast-path hydration against column/row length mismatches.If the SELECT shape diverges from req.columns, silently misaligned rows will be produced.
- for (const row of rows) { + for (const row of rows) { const rowData = row.json(); + if (!Array.isArray(rowData) || rowData.length !== req.columns.length) { + return [ + new QueryError( + `Column count mismatch: expected ${req.columns.length}, got ${Array.isArray(rowData) ? rowData.length : "non-array"}`, + { query: req.query } + ), + null, + ] as const; + }
232-270: queryFast: add optional per-column mappers for hot paths. AllowqueryBuilderFastto accept an optional ColumnMapper API to enforce per-column type coercions for existing builders (e.g., span_id, parent_span_id in getTraceEvents and getTaskEventDetailedSummary) without impacting performance.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
apps/webapp/app/services/apiRateLimit.server.ts(1 hunks)apps/webapp/app/services/authorizationRateLimitMiddleware.server.ts(3 hunks)apps/webapp/app/services/engineRateLimit.server.ts(1 hunks)apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts(1 hunks)apps/webapp/test/authorizationRateLimitMiddleware.test.ts(1 hunks)internal-packages/clickhouse/schema/008_add_task_events_v1_mvs.sql(1 hunks)packages/cli-v3/src/rules/manifest.ts(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- packages/cli-v3/src/rules/manifest.ts
- internal-packages/clickhouse/schema/008_add_task_events_v1_mvs.sql
🧰 Additional context used
📓 Path-based instructions (8)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
apps/webapp/app/services/apiRateLimit.server.tsapps/webapp/app/services/authorizationRateLimitMiddleware.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/test/authorizationRateLimitMiddleware.test.tsapps/webapp/app/services/engineRateLimit.server.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
apps/webapp/app/services/apiRateLimit.server.tsapps/webapp/app/services/authorizationRateLimitMiddleware.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/test/authorizationRateLimitMiddleware.test.tsapps/webapp/app/services/engineRateLimit.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
When importing from @trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from @trigger.dev/core’s package.json
Files:
apps/webapp/app/services/apiRateLimit.server.tsapps/webapp/app/services/authorizationRateLimitMiddleware.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/test/authorizationRateLimitMiddleware.test.tsapps/webapp/app/services/engineRateLimit.server.ts
{apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Files:
apps/webapp/app/services/apiRateLimit.server.tsapps/webapp/app/services/authorizationRateLimitMiddleware.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/services/engineRateLimit.server.ts
apps/webapp/app/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Modules intended for test consumption under apps/webapp/app/**/*.ts must not read environment variables; accept configuration via options instead
Files:
apps/webapp/app/services/apiRateLimit.server.tsapps/webapp/app/services/authorizationRateLimitMiddleware.server.tsapps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/services/engineRateLimit.server.ts
**/*.test.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
Our tests are all vitest
Files:
apps/webapp/test/authorizationRateLimitMiddleware.test.ts
{apps/webapp/**/__tests__/**/*.{ts,tsx},apps/webapp/**/*.{test,spec}.{ts,tsx}}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
{apps/webapp/**/__tests__/**/*.{ts,tsx},apps/webapp/**/*.{test,spec}.{ts,tsx}}: Do not import app/env.server.ts into tests, either directly or indirectly
Tests should only import classes/functions from files under apps/webapp/app/**/*.ts
Files:
apps/webapp/test/authorizationRateLimitMiddleware.test.ts
**/*.{test,spec}.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{test,spec}.{ts,tsx,js,jsx}: Unit tests must use Vitest
Tests should avoid mocks or stubs and use helpers from @internal/testcontainers when Redis or Postgres are needed
Test files live beside the files under test and should use descriptive describe and it blocks
Files:
apps/webapp/test/authorizationRateLimitMiddleware.test.ts
🧬 Code graph analysis (1)
apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts (3)
apps/webapp/app/env.server.ts (1)
env(1203-1203)internal-packages/clickhouse/src/index.ts (1)
ClickHouse(61-172)apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (1)
ClickhouseEventRepository(77-1657)
🔇 Additional comments (10)
apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.ts (6)
1-9: LGTM!The imports follow coding guidelines (env via env.server, no default exports), and the singleton pattern ensures proper resource management for the ClickHouse repository instance.
11-14: LGTM!Proper validation of the required EVENTS_CLICKHOUSE_URL environment variable with a clear error message.
32-34: LGTM! Past issue resolved.The compression.request configuration now correctly honors the
EVENTS_CLICKHOUSE_COMPRESSION_REQUESTenvironment variable, addressing the previous review feedback.
38-46: LGTM!The repository instantiation correctly passes all configuration parameters from environment variables, following the established pattern.
48-49: LGTM!Clean return of the configured repository instance.
16-17: Verify removal of thesecureURL parameter and document its rationale• Observed
url.searchParams.delete("secure")in both
apps/webapp/app/v3/eventRepository/clickhouseEventRepositoryInstance.server.tsapps/webapp/app/services/clickhouseInstance.server.ts
• No code comments or docs explain whysecureis dropped.
• Confirm that deleting this param won’t disable TLS/SSL (e.g., ensure the URL useshttps://or the client enforces encryption).
• Add a comment or centralize this logic to clarify whysecuremust be removed.apps/webapp/app/services/engineRateLimit.server.ts (1)
21-25: LGTM!The addition of
maxItems: 1000to the limiterCache configuration provides an eviction limit for the in-memory cache, preventing unbounded growth. This change is consistent with similar configurations across other rate limiters in the PR.apps/webapp/app/services/apiRateLimit.server.ts (1)
22-26: LGTM!Consistent with the engine rate limiter, this change adds
maxItems: 1000to cap the limiterCache size and prevent unbounded memory growth.apps/webapp/test/authorizationRateLimitMiddleware.test.ts (1)
368-372: LGTM!The test correctly includes the new
maxItemsfield in the limiterCache configuration, maintaining consistency with the updated schema and production code.apps/webapp/app/services/authorizationRateLimitMiddleware.server.ts (1)
63-67: LGTM!The
maxItemsfield is correctly added to thelimiterCacheconfiguration. The type definition makeslimiterCacheoptional but requiresmaxItemswhen provided, which aligns with the initialization logic that uses a sensible default of 1000.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (3)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (3)
138-140: insertManyImmediate still routes through batching.As noted in previous reviews,
insertManyImmediatecurrently callsthis.insertMany, which enqueues events in the batching scheduler rather than performing a synchronous immediate write and publish. This contradicts the "immediate" contract implied by the method name.
158-160: Attributes guard is already implemented.The previous review flagged potential undefined attributes, but
createEventToTaskEventV1InputAttributes(lines 354-374) already guards against undefined/falsy values at line 355, returning an empty object. This concern has been addressed.
178-181: Redundant BigInt conversion on event.startTime.As noted in previous reviews,
event.startTimeis already abigint(per CreateEventInput type), so wrapping it inBigInt()at line 180 is redundant and could mask type issues. Compare directly:BigInt(r.start_time) > event.startTime.
🧹 Nitpick comments (3)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (3)
88-102: Clarify load shedding configuration.Load shedding is explicitly disabled (
loadSheddingEnabled: falseat line 97), yetloadSheddingThresholdis set to 10000 and anisDroppableEventhandler is configured. This appears inconsistent. Either:
- Remove the load shedding configuration if it's not intended to be used, or
- Set
loadSheddingEnabled: trueif load shedding is desired, or- Add a comment explaining why this is configured but disabled
160-162: Extract hardcoded expiry duration to a constant.The 1-year expiry calculation
new Date(Date.now() + 365 * 24 * 60 * 60 * 1000)is duplicated across multiple methods (at least 5 locations in span event creation). Extract this to a class-level or module-level constant for maintainability and consistency.+const DEFAULT_EVENT_EXPIRY_MS = 365 * 24 * 60 * 60 * 1000; // 1 year + export class ClickhouseEventRepository implements IEventRepository {Then replace occurrences with:
- expires_at: convertDateToClickhouseDateTime( - new Date(Date.now() + 365 * 24 * 60 * 60 * 1000) // 1 year - ), + expires_at: convertDateToClickhouseDateTime( + new Date(Date.now() + DEFAULT_EVENT_EXPIRY_MS) + ),Also applies to: 230-232, 258-260, 290-292, 316-318
616-618: Extract hardcoded 30-day run expiry to a constant.The 30-day expiry calculation
new Date(run.createdAt.getTime() + 30 * 24 * 60 * 60 * 1000)is duplicated across all six run lifecycle methods. Extract this to a class-level or module-level constant for maintainability.+const RUN_EVENT_EXPIRY_DAYS = 30; +const RUN_EVENT_EXPIRY_MS = RUN_EVENT_EXPIRY_DAYS * 24 * 60 * 60 * 1000;Then replace occurrences with:
- const expiresAt = convertDateToClickhouseDateTime( - new Date(run.createdAt.getTime() + 30 * 24 * 60 * 60 * 1000) - ); + const expiresAt = convertDateToClickhouseDateTime( + new Date(run.createdAt.getTime() + RUN_EVENT_EXPIRY_MS) + );Also applies to: 664-666, 704-706, 750-752, 796-798, 840-842
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (5)
**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
**/*.{ts,tsx}: Always prefer using isomorphic code like fetch, ReadableStream, etc. instead of Node.js specific code
For TypeScript, we usually use types over interfaces
Avoid enums
No default exports, use function declarations
Files:
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts
{packages/core,apps/webapp}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.github/copilot-instructions.md)
We use zod a lot in packages/core and in the webapp
Files:
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts
apps/webapp/**/*.{ts,tsx}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
When importing from @trigger.dev/core in the webapp, never import the root package path; always use one of the documented subpath exports from @trigger.dev/core’s package.json
Files:
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts
{apps/webapp/app/**/*.server.{ts,tsx},apps/webapp/app/routes/**/*.ts}
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Access environment variables only via the env export from app/env.server.ts; do not reference process.env directly
Files:
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts
apps/webapp/app/**/*.ts
📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)
Modules intended for test consumption under apps/webapp/app/**/*.ts must not read environment variables; accept configuration via options instead
Files:
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts
🧬 Code graph analysis (1)
apps/webapp/app/v3/eventRepository/clickhouseEventRepository.server.ts (10)
internal-packages/clickhouse/src/index.ts (1)
ClickHouse(61-172)apps/webapp/app/v3/eventRepository/eventRepository.types.ts (11)
IEventRepository(311-417)CreateEventInput(24-58)TraceEventOptions(95-106)EventBuilder(108-114)TraceAttributes(86-91)CompleteableTaskRun(67-80)TraceSummary(272-276)SpanSummary(246-263)SpanDetail(184-223)TraceDetailedSummary(298-301)RunPreparedEvent(180-182)apps/webapp/app/v3/dynamicFlushScheduler.server.ts (1)
DynamicFlushScheduler(21-423)internal-packages/clickhouse/src/taskEvents.ts (8)
TaskEventV1Input(5-22)TaskEventV1Input(24-24)TaskEventSummaryV1Result(40-50)TaskEventSummaryV1Result(52-52)TaskEventDetailsV1Result(111-121)TaskEventDetailsV1Result(123-123)TaskEventDetailedSummaryV1Result(73-84)TaskEventDetailedSummaryV1Result(86-86)apps/webapp/app/v3/services/tracePubSub.server.ts (1)
tracePubSub(62-62)apps/webapp/app/v3/eventRepository/common.server.ts (11)
parseEventsField(88-100)convertDateToNanoseconds(43-45)removePrivateProperties(134-156)extractContextFromCarrier(8-21)getNowInNanoseconds(23-25)calculateDurationFromStart(31-35)generateTraceId(67-69)generateDeterministicSpanId(51-63)generateSpanId(71-73)createExceptionPropertiesFromError(102-131)calculateDurationFromStartJsDate(37-41)packages/core/src/v3/utils/flattenAttributes.ts (3)
unflattenAttributes(241-326)attributes(23-25)value(207-234)packages/core/src/v3/isomorphic/traceContext.ts (1)
serializeTraceparent(27-29)apps/webapp/app/v3/eventRepository/originalRunIdCache.server.ts (1)
originalRunIdCache(62-75)packages/core/src/v3/errors.ts (1)
createJsonErrorObject(222-249)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (23)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (1, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (3, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (4, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (2, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (5, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (6, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (8, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (6, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (2, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (7, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (5, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (8, 8)
- GitHub Check: units / webapp / 🧪 Unit Tests: Webapp (1, 8)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (3, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: units / internal / 🧪 Unit Tests: Internal (4, 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: units / packages / 🧪 Unit Tests: Packages (1, 1)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: typecheck / typecheck
- GitHub Check: Analyze (javascript-typescript)
Issues attributed to commits in this pull requestThis pull request was merged and Sentry observed the following issues:
|
No description provided.