Skip to content

Conversation

MitchellShiell
Copy link
Contributor

@MitchellShiell MitchellShiell commented Oct 2, 2025

Remaining Tasks:

  • Update problem solution statement in introduction
  • Add diagram for the life of a sqon filter
  • Add code examples to pair with the life of a sqon section
  • Review

Netlify Page Preview: https://deploy-preview-33--overturedocs.netlify.app/docs/core-software/Arranger/usage/query-processing
Linked to this docs PR: overture-stack/docs#33

Note

Lyric also uses SQON, this should be documented within the Lyric documentation when it is completed

@MitchellShiell
Copy link
Contributor Author

Consider outlining performance/functionality gain versus using simpler and more familiar postgres queries.

Comment on lines 219 to 223
| Operator | Description | Value Type | Example |
| -------- | ------------------------------------------------ | ------------------ | ------- |
| `"in"` | Field value must match one of the listed values | Array | `"value": ["Ontario", "Quebec"]` |
| `">="` | Field value must be greater than or equal to | Scalar (number) | `"value": 1640926800000` |
| `"<="` | Field value must be less than or equal to | Scalar (number) | `"value": 1672549199999` |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good start. Arranger can support more operators than this. We should definitely make a standardized absolute list and ensure compatibility across all services that use SQONs.


```json showLineNumbers
{
"query": "query tableData($sqon: JSON, ...) { file { hits(filters: $sqon, ...) { ... } } }",
Copy link
Member

@justincorrigible justincorrigible Oct 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

multiline query to maintain some readability
also, my suggested comment is optional, meant as an additional breadcrumb for people trying to figure out how things work under the hood

Suggested change
"query": "query tableData($sqon: JSON, ...) { file { hits(filters: $sqon, ...) { ... } } }",
"query": "query tableData($sqon: JSON, ...) {
file { // Arranger's "document type"
hits(filters: $sqon, ...) { ... }
}
}",

Comment on lines +148 to +155
| SQON Component | Elasticsearch Equivalent | Notes |
|----------------|-------------------------|-------|
| `"op": "and"` | `"bool": { "must": [...] }` | AND logic requires all conditions to match |
| `"op": "in"` | `"terms": { "field": [...] }` | IN operation uses Elasticsearch's `terms` query |
| `"fieldName": "data.primary_site"` | `"data.primary_site": [...]` | Field name maps directly to the index mapping |
| `"first": 20` | `"size": 20` | Pagination size parameter |
| `"offset": 0` | `"from": 0` | Pagination offset parameter |

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prettier generally does a good job at inserting spaces in md to make tables readable even without a md render. wonder if the settings in this repo need adjustment to make sure this tool is applied here

| `"offset": 0` | `"from": 0` | Pagination offset parameter |

:::tip
This translation happens automatically ensuring users never interact directly with Elasticsearch.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adjusting this to sound more like it's a convenience feature, not a security concern or a limitation

Suggested change
This translation happens automatically ensuring users never interact directly with Elasticsearch.
This translation happens automatically ensuring users never need to interact directly with Elasticsearch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants