feat: Introduce snapshot summary properties

[dentiny]

Which issue does this PR close?

• Closes Feature request: support to update snapshot summary #1329

TLDR:

• I want to add customized metadata (with size controlled within 200B) for each snapshot, summary properties is the perfect place for this feature request

What changes are included in this PR?

I add a new table update action for snapshot summary properties.

Are these changes tested?

Yes, unit tests are added.

[jonathanc-n]

Looks good to me, sorry for the misunderstanding. Thanks for this pr @dentiny!

[Xuanwo]

Nice change, thank you!

[Xuanwo]

One thing left here is to format the code.

[dentiny]

Hi @Xuanwo , I think the linting issue has been fixed, also merged with main branch; could you please take another look when you have some time? Thank you!

[dentiny]

@Xuanwo friendly ping :)

[Xuanwo]

Nice change, thank you @dentiny!

[liurenjie1024]

This table update doesn't exist in table updates, see https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml

[liurenjie1024]

Hi, @Xuanwo @dentiny We need to revert this pr, the new added update doesn't exist in openapi spec: https://github.com/apache/iceberg/blob/cab0decbb0e32bf314039e30807eb033c50665d5/open-api/rest-catalog-open-api.yaml#L3058

[dentiny]

Hi, @Xuanwo @dentiny We need to revert this pr, the new added update doesn't exist in openapi spec: https://github.com/apache/iceberg/blob/cab0decbb0e32bf314039e30807eb033c50665d5/open-api/rest-catalog-open-api.yaml#L3058

Hi @liurenjie1024 , there's a SnapshotUpdate action in Java implementation, which supports setting summary property.
https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/SnapshotUpdate.java

[liurenjie1024]

Also I don't think we should allow user to udpate snapshot summaries alone, it's dangerous.

[dentiny]

Also I don't think we should allow user to udpate snapshot summaries alone, it's dangerous.

HI @liurenjie1024, curious if you have other suggestions to update snapshot summary?
My feature request is to write customized metadata for snapshot.

The Java impl I linked above seems to fulfill the requirement, and is exposed as public?
Would like to know your thoughts :)

[liurenjie1024]

Also I don't think we should allow user to udpate snapshot summaries alone, it's dangerous.

HI @liurenjie1024, curious if you have other suggestions to update snapshot summary? My feature request is to write customized metadata for snapshot.

The Java impl I linked above seems to fulfill the requirement, and is exposed as public? Would like to know your thoughts :)

The SnapshotUpdate is a super interface of tx actions that could produce a new snapshot, so in theory you could use any tx action that produces a new snapshot to do that. Currently, you could you FastAppend.

[dentiny]

The SnapshotUpdate is a super interface of tx actions that could produce a new snapshot, so in theory you could use any tx action that produces a new snapshot to do that. Currently, you could you FastAppend.

Yeah I actually checked existing interface before I made this PR, I didn't any interface exposing the capability to set / update snapshot summary:

iceberg-rust/crates/iceberg/src/transaction/append.rs

Lines 40 to 58 in 3ea6f47

	#[allow(clippy::too_many_arguments)]
	pub(crate) fn new(
	tx: Transaction<'a>,
	snapshot_id: i64,
	commit_uuid: Uuid,
	key_metadata: Vec<u8>,
	snapshot_properties: HashMap<String, String>,
	) -> Result<Self> {
	Ok(Self {
	snapshot_produce_action: SnapshotProduceAction::new(
	tx,
	snapshot_id,
	key_metadata,
	commit_uuid,
	snapshot_properties,
	)?,
	check_duplicate: true,
	})
	}

[liurenjie1024]

The SnapshotUpdate is a super interface of tx actions that could produce a new snapshot, so in theory you could use any tx action that produces a new snapshot to do that. Currently, you could you FastAppend.

Yeah I actually checked existing interface before I made this PR, I didn't any interface exposing the capability to set / update snapshot summary:

iceberg-rust/crates/iceberg/src/transaction/append.rs

Lines 40 to 58 in 3ea6f47

	#[allow(clippy::too_many_arguments)]
	pub(crate) fn new(
	tx: Transaction<'a>,
	snapshot_id: i64,
	commit_uuid: Uuid,
	key_metadata: Vec<u8>,
	snapshot_properties: HashMap<String, String>,
	) -> Result<Self> {
	Ok(Self {
	snapshot_produce_action: SnapshotProduceAction::new(
	tx,
	snapshot_id,
	key_metadata,
	commit_uuid,
	snapshot_properties,
	)?,
	check_duplicate: true,
	})
	}

I think you find the correct way to do it in #1391