Skip to content
This repository was archived by the owner on Oct 9, 2025. It is now read-only.

Updates for postgREST 11 #417

Merged
merged 8 commits into from
Apr 18, 2023
Merged

Updates for postgREST 11 #417

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
76a9a94
test: postgrest 11 pre-release
steve-chavez Apr 8, 2023
62b49e1
feat: bulk inserts/upserts w/ column defaults
soedirgo Apr 14, 2023
adf4839
test: make snapshots more deterministic
soedirgo Apr 14, 2023
5ad7562
Update docker-compose.yml to postgrest v11
steve-chavez Apr 16, 2023
a9c3629
fix: use `?columns=` on upsert
soedirgo Apr 17, 2023
0557752
test: update snapshots
soedirgo Apr 17, 2023
d4e03a0
feat(filters): likeAllOf, likeAnyOf, ilikeAllOf, ilikeAnyOf
soedirgo Apr 17, 2023
29b3808
Merge branch 'master' into pgrst-11
soedirgo Apr 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions src/PostgrestFilterBuilder.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,32 @@ export default class PostgrestFilterBuilder<
return this
}

likeAllOf<ColumnName extends string & keyof Row>(column: ColumnName, patterns: string[]): this
likeAllOf(column: string, patterns: string[]): this
/**
* Match only rows where `column` matches all of `patterns` case-sensitively.
*
* @param column - The column to filter on
* @param patterns - The patterns to match with
*/
likeAllOf(column: string, patterns: string[]): this {
this.url.searchParams.append(column, `like(all).{${patterns.join(',')}}`)
return this
}

likeAnyOf<ColumnName extends string & keyof Row>(column: ColumnName, patterns: string[]): this
likeAnyOf(column: string, patterns: string[]): this
/**
* Match only rows where `column` matches any of `patterns` case-sensitively.
*
* @param column - The column to filter on
* @param patterns - The patterns to match with
*/
likeAnyOf(column: string, patterns: string[]): this {
this.url.searchParams.append(column, `like(any).{${patterns.join(',')}}`)
return this
}

ilike<ColumnName extends string & keyof Row>(column: ColumnName, pattern: string): this
ilike(column: string, pattern: string): this
/**
Expand All @@ -136,6 +162,32 @@ export default class PostgrestFilterBuilder<
return this
}

ilikeAllOf<ColumnName extends string & keyof Row>(column: ColumnName, patterns: string[]): this
ilikeAllOf(column: string, patterns: string[]): this
/**
* Match only rows where `column` matches all of `patterns` case-insensitively.
*
* @param column - The column to filter on
* @param patterns - The patterns to match with
*/
ilikeAllOf(column: string, patterns: string[]): this {
this.url.searchParams.append(column, `ilike(all).{${patterns.join(',')}}`)
return this
}

ilikeAnyOf<ColumnName extends string & keyof Row>(column: ColumnName, patterns: string[]): this
ilikeAnyOf(column: string, patterns: string[]): this
/**
* Match only rows where `column` matches any of `patterns` case-insensitively.
*
* @param column - The column to filter on
* @param patterns - The patterns to match with
*/
ilikeAnyOf(column: string, patterns: string[]): this {
this.url.searchParams.append(column, `ilike(any).{${patterns.join(',')}}`)
return this
}

is<ColumnName extends string & keyof Row>(
column: ColumnName,
value: Row[ColumnName] & (boolean | null)
Expand Down
49 changes: 36 additions & 13 deletions src/PostgrestQueryBuilder.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,24 +113,31 @@ export default class PostgrestQueryBuilder<
*
* `"estimated"`: Uses exact count for low numbers and planned count for high
* numbers.
*
* @param options.defaultToNull - Make missing fields default to `null`.
* Otherwise, use the default value for the column.
*/
insert<Row extends Relation extends { Insert: unknown } ? Relation['Insert'] : never>(
values: Row | Row[],
{
count,
defaultToNull = true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@steve-chavez @soedirgo
Would this be a breaking change? The behavior prior to this was defaultToNull = false, where the missing fields get their default value specified on the table definition, right?

Copy link

@Vinzent03 Vinzent03 Jul 19, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so.
From the postgREST doc:

Any missing columns in the payload will be inserted as null values. To use the DEFAULT column value instead, use the Prefer: missing=default header.

And the header is only set if defaultToNull is false here. So I think that's correct.

Copy link
Member Author

@steve-chavez steve-chavez Jul 19, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dshukertjr As Vinzent said, there's no breaking change.

defaultToNull was always true before. I would have liked it to make it the default behavior on PostgREST but it would have caused a breaking change (plus right now it also has a bit of perf loss).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see that this is only applicable when you are inserting multiple rows in bulk huh? I was thinking it applies to when you are inserting a single row as well. Maybe we could add a note about that in the comments

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it also applies when inserting a single row. But this only takes effect when specifying &columns=..., both for single & bulk inserts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I think you're right, columns is only set when doing a bulk insert 😬 I'll add a comment about this and make the behavior consistent in v3. Thanks for the catch!

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So there is currently a difference in inserting a single row as an object and a single row in an array with length 1?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wasn't intentional, but yes

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tried that and I think they are equal. I ran the following code in the test db of postgrest-js and other than the actual value of username they are equal:

let res1 = await postgrest.from('users').insert({ username: 'bot1' }).select()
let res2 = await postgrest
  .from('users')
  .insert([{ username: 'bot2' }])
  .select()
console.log(res1)
console.log(res2)

Especially is status = ONLINE, which is the default value of that column. So in both cases the default column is used and not null.

I ran further test understood it now. The missing fields are only mapped to null if a field is missing in one row, but present in another, because then that column is listed in &columns=. To use the default value in that case as well, you have to use the defaultToNull=false flag. For missing fields in a bulk insert with only one row, all fields are listed in &columns= and therefore the missing fields are mapped to the default.
So there is a difference in fields missing in all rows and fields missing in only a proper subset.
I just want to really understand that to properly document and implement it in postgrest-dart.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Vinzent03 I think that's correct - so there isn't a difference between single row insert vs. bulk insert with 1 row after all, sorry for the back and forth.

}: {
count?: 'exact' | 'planned' | 'estimated'
defaultToNull?: boolean
} = {}
): PostgrestFilterBuilder<Schema, Relation['Row'], null> {
const method = 'POST'

const prefersHeaders = []
const body = values
if (this.headers['Prefer']) {
prefersHeaders.push(this.headers['Prefer'])
}
if (count) {
prefersHeaders.push(`count=${count}`)
}
if (this.headers['Prefer']) {
prefersHeaders.unshift(this.headers['Prefer'])
if (!defaultToNull) {
prefersHeaders.push('missing=default')
}
this.headers['Prefer'] = prefersHeaders.join(',')

Expand All @@ -147,7 +154,7 @@ export default class PostgrestQueryBuilder<
url: this.url,
headers: this.headers,
schema: this.schema,
body,
body: values,
fetch: this.fetch,
allowEmpty: false,
} as unknown as PostgrestBuilder<null>)
Expand Down Expand Up @@ -185,39 +192,56 @@ export default class PostgrestQueryBuilder<
*
* `"estimated"`: Uses exact count for low numbers and planned count for high
* numbers.
*
* @param options.defaultToNull - Make missing fields default to `null`.
* Otherwise, use the default value for the column. This only applies when
* inserting new rows, not when merging with existing rows under
* `ignoreDuplicates: false`.
*/
upsert<Row extends Relation extends { Insert: unknown } ? Relation['Insert'] : never>(
values: Row | Row[],
{
onConflict,
ignoreDuplicates = false,
count,
defaultToNull = true,
}: {
onConflict?: string
ignoreDuplicates?: boolean
count?: 'exact' | 'planned' | 'estimated'
defaultToNull?: boolean
} = {}
): PostgrestFilterBuilder<Schema, Relation['Row'], null> {
const method = 'POST'

const prefersHeaders = [`resolution=${ignoreDuplicates ? 'ignore' : 'merge'}-duplicates`]

if (onConflict !== undefined) this.url.searchParams.set('on_conflict', onConflict)
const body = values
if (this.headers['Prefer']) {
prefersHeaders.push(this.headers['Prefer'])
}
if (count) {
prefersHeaders.push(`count=${count}`)
}
if (this.headers['Prefer']) {
prefersHeaders.unshift(this.headers['Prefer'])
if (!defaultToNull) {
prefersHeaders.push('missing=default')
}
this.headers['Prefer'] = prefersHeaders.join(',')

if (Array.isArray(values)) {
const columns = values.reduce((acc, x) => acc.concat(Object.keys(x)), [] as string[])
if (columns.length > 0) {
const uniqueColumns = [...new Set(columns)].map((column) => `"${column}"`)
this.url.searchParams.set('columns', uniqueColumns.join(','))
}
}

return new PostgrestFilterBuilder({
method,
url: this.url,
headers: this.headers,
schema: this.schema,
body,
body: values,
fetch: this.fetch,
allowEmpty: false,
} as unknown as PostgrestBuilder<null>)
Expand Down Expand Up @@ -254,21 +278,20 @@ export default class PostgrestQueryBuilder<
): PostgrestFilterBuilder<Schema, Relation['Row'], null> {
const method = 'PATCH'
const prefersHeaders = []
const body = values
if (this.headers['Prefer']) {
prefersHeaders.push(this.headers['Prefer'])
}
if (count) {
prefersHeaders.push(`count=${count}`)
}
if (this.headers['Prefer']) {
prefersHeaders.unshift(this.headers['Prefer'])
}
this.headers['Prefer'] = prefersHeaders.join(',')

return new PostgrestFilterBuilder({
method,
url: this.url,
headers: this.headers,
schema: this.schema,
body,
body: values,
fetch: this.fetch,
allowEmpty: false,
} as unknown as PostgrestBuilder<null>)
Expand Down
4 changes: 1 addition & 3 deletions src/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,7 @@ export interface PostgrestResponseFailure extends PostgrestResponseBase {
// TODO: in v3:
// - remove PostgrestResponse and PostgrestMaybeSingleResponse
// - rename PostgrestSingleResponse to PostgrestResponse
export type PostgrestSingleResponse<T> =
| PostgrestResponseSuccess<T>
| PostgrestResponseFailure
export type PostgrestSingleResponse<T> = PostgrestResponseSuccess<T> | PostgrestResponseFailure
export type PostgrestMaybeSingleResponse<T> = PostgrestSingleResponse<T | null>
export type PostgrestResponse<T> = PostgrestSingleResponse<T[]>

Expand Down
56 changes: 56 additions & 0 deletions test/basic.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1078,6 +1078,62 @@ describe("insert, update, delete with count: 'exact'", () => {
`)
})

test('bulk insert with column defaults', async () => {
let res = await postgrest
.from('channels')
.insert([{ id: 100 }, { slug: 'test-slug' }], { defaultToNull: false })
.select()
.rollback()
expect(res).toMatchInlineSnapshot(`
Object {
"count": null,
"data": Array [
Object {
"data": null,
"id": 100,
"slug": null,
},
Object {
"data": null,
"id": 4,
"slug": "test-slug",
},
],
"error": null,
"status": 201,
"statusText": "Created",
}
`)
})

test('bulk upsert with column defaults', async () => {
let res = await postgrest
.from('channels')
.upsert([{ id: 1 }, { slug: 'test-slug' }], { defaultToNull: false })
.select()
.rollback()
expect(res).toMatchInlineSnapshot(`
Object {
"count": null,
"data": Array [
Object {
"data": null,
"id": 1,
"slug": null,
},
Object {
"data": null,
"id": 6,
"slug": "test-slug",
},
],
"error": null,
"status": 201,
"statusText": "Created",
}
`)
})

test("update with count: 'exact'", async () => {
let res = await postgrest
.from('messages')
Expand Down
2 changes: 1 addition & 1 deletion test/db/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
version: '3'
services:
rest:
image: postgrest/postgrest:v10.1.2
image: postgrest/postgrest:v11.0.0
ports:
- '3000:3000'
environment:
Expand Down
86 changes: 86 additions & 0 deletions test/filters.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,49 @@ test('like', async () => {
`)
})

test('likeAllOf', async () => {
const res = await postgrest
.from('users')
.select('username')
.likeAllOf('username', ['%supa%', '%bot%'])
expect(res).toMatchInlineSnapshot(`
Object {
"count": null,
"data": Array [
Object {
"username": "supabot",
},
],
"error": null,
"status": 200,
"statusText": "OK",
}
`)
})

test('likeAnyOf', async () => {
const res = await postgrest
.from('users')
.select('username')
.likeAnyOf('username', ['%supa%', '%kiwi%'])
expect(res).toMatchInlineSnapshot(`
Object {
"count": null,
"data": Array [
Object {
"username": "supabot",
},
Object {
"username": "kiwicopple",
},
],
"error": null,
"status": 200,
"statusText": "OK",
}
`)
})

test('ilike', async () => {
const res = await postgrest.from('users').select('username').ilike('username', '%SUPA%')
expect(res).toMatchInlineSnapshot(`
Expand All @@ -199,6 +242,49 @@ test('ilike', async () => {
`)
})

test('ilikeAllOf', async () => {
const res = await postgrest
.from('users')
.select('username')
.ilikeAllOf('username', ['%SUPA%', '%bot%'])
expect(res).toMatchInlineSnapshot(`
Object {
"count": null,
"data": Array [
Object {
"username": "supabot",
},
],
"error": null,
"status": 200,
"statusText": "OK",
}
`)
})

test('ilikeAnyOf', async () => {
const res = await postgrest
.from('users')
.select('username')
.ilikeAnyOf('username', ['%supa%', '%KIWI%'])
expect(res).toMatchInlineSnapshot(`
Object {
"count": null,
"data": Array [
Object {
"username": "supabot",
},
Object {
"username": "kiwicopple",
},
],
"error": null,
"status": 200,
"statusText": "OK",
}
`)
})

test('is', async () => {
const res = await postgrest.from('users').select('data').is('data', null)
expect(res).toMatchInlineSnapshot(`
Expand Down
2 changes: 1 addition & 1 deletion test/index.test-d.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ const postgrest = new PostgrestClient<Database>(REST_URL)
if (error) {
throw new Error(error.message)
}
expectType<{ bar: Json, baz: string }>(data)
expectType<{ bar: Json; baz: string }>(data)
}

// rpc return type
Expand Down
Loading