Skip to content

RI-7218 Add telemetry collection for Vector Search page #4811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

valkirilov
Copy link
Member

@valkirilov valkirilov commented Aug 6, 2025

Description

Added telemetry collection events to track users' interaction with the Vector Search page:

  • when users run a query
  • when users copy a query result
  • when users toggle the full-screen mode for a query result
  • when users expand/collapse a query result
  • when users delete a query result
  • when users delete all results
  • when users delete the query in the editor

PS: The diff may look huge, but actually, we have the same changes applied on multiple places + copy/paste of a big component.

How to test it

  1. Go to Vector Search page - http://localhost:8080/{databaseId}/vector-search
  2. Enter a query in the main editor and hit the "Run" button (dispatches SEARCH_COMMAND_SUBMITTED event)
  3. Click on the "Copy" button when you hover over a result of a query (dispatches SEARCH_COMMAND_COPIED event)
  4. Click on the "Run" button on a result of a query, to re-run it (dispatches SEARCH_COMMAND_RUN_AGAIN event)
  5. Click on the "Toggle Full-Screen" button on a result of a query (dispatches SEARCH_RESULTS_IN_FULL_SCREEN event)
  6. Click on the "Expand" button on a result of a query to see the command results (dispatches SEARCH_RESULTS_COLLAPSED event)
  7. Click on the "Collapse" button on a result of a query to hide the command results (dispatches SEARCH_RESULTS_EXPANDED event)
  8. Click on the "Delete" button on a result of a query to remove it (dispatches SEARCH_CLEAR_RESULT_CLICKED event)
  9. Click on the "Clear Results" button to remove all commands (dispatches SEARCH_CLEAR_ALL_RESULTS_CLICKED event)
  10. Click on the "Clear" button to remove the current query from the editor (dispatches SEARCH_CLEAR_EDITOR_CLICKED event)
Screen.Recording.2025-08-07.at.14.03.20.mov

Verify Telemetry events

We have the new events for the Vector Search page collected successfully.

Note: Telemetry is collected only if the user decides to opt-in to that.
You control it from Settings page -> Privacy section -> Usage Data toggle.

image image image image image image image image image

Notes: Since the introduced changes reflect the Workbench telemetry as well, I have checked that we still emit properly the following events:

  • WORKBENCH_RESULTS_COLLAPSED
  • WORKBENCH_RESULTS_EXPANDED
  • WORKBENCH_COMMAND_COPIED
  • WORKBENCH_CLEAR_RESULT_CLICKED

Copy link
Contributor

github-actions bot commented Aug 6, 2025

Code Coverage - Frontend unit tests

St.
Category Percentage Covered / Total
🟢 Statements 81.71% 19589/23973
🟡 Branches 67.06% 8502/12678
🟡 Functions 75.79% 5186/6843
🟢 Lines 82.14% 19172/23342

Test suite run success

4997 tests passing in 659 suites.

Report generated by 🧪jest coverage report action from 264e320

@valkirilov valkirilov force-pushed the fe/feature/RI-7218_vector-search-telemetry-3 branch 2 times, most recently from 1c148ec to 523a81d Compare August 6, 2025 15:27
@valkirilov valkirilov self-assigned this Aug 7, 2025
@valkirilov valkirilov force-pushed the fe/feature/RI-7218_vector-search-telemetry-3 branch from 0b87e6d to f64da8b Compare August 7, 2025 10:59
@valkirilov valkirilov marked this pull request as ready for review August 7, 2025 11:33
@valkirilov valkirilov force-pushed the fe/feature/RI-7218_vector-search-telemetry-2 branch 2 times, most recently from 39baa30 to 4e8a96b Compare August 7, 2025 12:21
@valkirilov valkirilov force-pushed the fe/feature/RI-7218_vector-search-telemetry-3 branch from f64da8b to 4bcf5eb Compare August 7, 2025 12:33
Base automatically changed from fe/feature/RI-7218_vector-search-telemetry-2 to feature/RI-6855/vector-search August 7, 2025 13:08
- refactor the CardHeader component to work with "onQueryCopy" callback
- rework the telemetry collection on "copy command" for the Workbench
- collect telemetry when clicking on the "copy command" button in Vector Search

re #RI-7218
- we can't make the QueryCardHeader component reuable at this point, it need to be reimagined from the ground up, so we decide to copy the current implementation and rework inline what we need to make the telemetry for the vector search

re #RI-7218
@valkirilov valkirilov force-pushed the fe/feature/RI-7218_vector-search-telemetry-3 branch from 4bcf5eb to 910e4a7 Compare August 7, 2025 13:13
Comment on lines 82 to 86
const collectTelemetryQueryReRun = (query: string) => {
sendEventTelemetry({
event: TelemetryEvent.SEARCH_COMMAND_RUN_AGAIN,
eventData: {
databaseId: instanceId,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function already takes a parameter, add another one - instanceId and you can remove it's definition from the component body

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. The telemetry collection is exported as util, outside of the component.

Comment on lines 60 to 61
* We're really sorry to do that at this point, but this family fo components should be refactored to be reusable.
* At this point, it requires a lot of effort to make it reusable, so we decided to copy the component as is, and apply the necessary changes inline.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this adds to the tech debt. What are the changes the original component would require to make it reusable?

Diff checking both of them I currently see a telemetry param change and that's all, so what else is going to be changed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree. It was discussed with @pawelangelow to proceed this way for now. Check the Notes section in the description of the pull request for more information.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, let's copy it for now. We either slow down to do it properly or introduce a tech debt - no "good" scenario :(

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I went for a simplified approach, and I think it worked:

  • removed the copy/paste QueryCardHeader component d6587d2
  • introduced a simple context that for now will keep information where the component is rendered - Workbench page or Vector Search page 49f5bee
  • applied changes to emit telemetry conditionally, based on the state from the context 09b8e86

Comment on lines 63 to 68
const collectTelemetryQueryRun = () => {
sendEventTelemetry({
event: TelemetryEvent.SEARCH_COMMAND_SUBMITTED,
eventData: {
databaseId: instanceId,
commands: [query],
Copy link
Collaborator

@KrumTy KrumTy Aug 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, absolutely not a fan of declaring such functions inside the component, especially since they only require 1-2 params. This adds unnecessary unnecessary complexity - you open the component body and half the code is related to a telemetry callbacks

seeing how much space these telemetry functions take I'd even suggest extracting them in a separate telemetry.ts file or something like that

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, they won't disappear completely as the handlers should live here:

const handleClearResults = () => {
  onAllQueriesDelete()
  sendEventTelemetrySomething()
}
...
and the remaining 3

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both of you are correct. We still have the base handlers as part of the component, because we need them.
But I managed to export the telemetry collection logic into a helper util, outside of the component.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And obviously, all the logic can be external to the component by incorporating a hook :)
But since this is mostly personal preference, I am saying this just for completenes and nothing else :)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initially, I went for a hook, but then @KrumTy shared some concerns on the topic and advised going for a simpler helper function, and keeping it independent of the React lifecycle. I think it works for now.

ViewModeContextProviderProps
> = ({
children,
initialViewMode = ViewMode.Workbench, // Default to Workbench if not provided
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

explicit over default is better in this case imo

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to keep this default value, for backward compatibility and safety.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a new provider, no backwards compatibility is needed as you are defining the interface atm
also, from what I've seen, you are setting it explicitly everywhere (as should be) already

Comment on lines +230 to +239
const telemetryEvent =
viewMode === ViewMode.Workbench
? isOpen
? TelemetryEvent.WORKBENCH_RESULTS_COLLAPSED
: TelemetryEvent.WORKBENCH_RESULTS_EXPANDED
: isOpen
? TelemetryEvent.SEARCH_RESULTS_COLLAPSED
: TelemetryEvent.SEARCH_RESULTS_EXPANDED

sendEvent(telemetryEvent, query)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this starts to look ugly

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we all agree on that. In general, the component should be refactored so it can be reused properly on multiple places without having such tight business logic inside it. Unfortunately, it's not that simple, so that’s why we went for this quick and dirty fix with the conditional telemetry.

Regarding this piece of code, the best thing I can do is to split it over two conditional operators, and it will still be ugly. So after all, I rely on general refactoring to fix this.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So basically, this is a context that shares single value across components, right?
Is there any reason not to just add this to Redux store?

Copy link
Collaborator

@KrumTy KrumTy Aug 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's used in different views now. It would not make sense to dispatch redux actions to update the store as you switch between Search and Workbench tabs

This context solution is the least path of resistance to reusing the QueryCard component. The other being prop drilling, which @valkirilov said was too hard (4-5 levels deep).

The pre-context solution was to copy the whole component and it's test and adjust 3-4 telemetry lines.

Copy link
Collaborator

@KrumTy KrumTy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@valkirilov valkirilov merged commit 5aa7be6 into feature/RI-6855/vector-search Aug 8, 2025
14 checks passed
@valkirilov valkirilov deleted the fe/feature/RI-7218_vector-search-telemetry-3 branch August 8, 2025 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants