Skip to content

Conversation

@G8XSU
Copy link
Contributor

@G8XSU G8XSU commented Jul 25, 2023

We add delete functionality as both:

  • Transaction support in PutObject Api, which allows simultaneous delete of keys while writing other keys. This is necessary in order to support CRUD transactions.
  • An idempotent api to allow for applications which can safely merge deletes and don't require transactions with put&delete.

This follows similar pattern followed in popular KVStore Databases, such as aws-dynamodb which allows for both delete api and transaction write api.

For now, ignore the naming for "PutObject" API, will rename it to WriteObject in separate rename-commit.

@G8XSU G8XSU requested a review from jkczyz July 25, 2023 20:15
@G8XSU G8XSU mentioned this pull request Jul 20, 2023
31 tasks
Copy link
Contributor

@jkczyz jkczyz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK.

Comment on lines +90 to +95
batchQueries.addAll(vssPutRecords.stream()
.map(vssRecord -> buildPutObjectQuery(dsl, vssRecord)).toList());
batchQueries.addAll(vssDeleteRecords.stream()
.map(vssRecord -> buildDeleteObjectQuery(dsl, vssRecord)).toList());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does there need to be a check for distinct keys across both lists?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will implement request validation as part of api class and not db-implementation.
Currently we assume all requests are valid and there are couple of pending request validations to be added.
For example, we are also not checking if keys in put_items are unique currently or size or number of keys. (will work on this separately)

@G8XSU
Copy link
Contributor Author

G8XSU commented Aug 2, 2023

Ready for review ! :D

@G8XSU G8XSU requested a review from jkczyz August 2, 2023 00:52
// Multiple items in the `delete_items` field, along with the `transaction_items`, are written in a
// database transaction in an all-or-nothing fashion.
//
// All items within a single `PutObjectRequest` must have distinct keys.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to do so in this PR, but we should be consistent with using ticks for identifiers. Or possibly not use them at all, for that matter. In Rust, they are used to format the markdown. But that's not the case for proto docs, presumably.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Opening another PR to consistently add ticks around all identifiers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Part of #16

KeyValue.newBuilder()
.setKey(GLOBAL_VERSION_KEY)
.setVersion(request.getGlobalVersion())
.setValue(ByteString.EMPTY)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there different behavior if this is omitted?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really,
it was just some inconcstency that i noticed and just to make it more explicit.

@G8XSU G8XSU requested a review from jkczyz August 2, 2023 22:56
@G8XSU G8XSU merged commit 9c94974 into lightningdevkit:main Aug 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants