Skip to content

Conversation

@ebyhr
Copy link
Contributor

@ebyhr ebyhr commented Aug 7, 2024

Description

Enable ErrorProne MissingOverride

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • Test A
  • Test B

Test Configuration:

  • Firmware version:
  • Hardware:
  • Toolchain:
  • SDK:

Checklist:

Please delete options that are not relevant.

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • Any dependent changes have been merged and published in downstream modules
  • If adding new functionality, I have discussed my implementation with the community using the linked GitHub issue
  • I have signed and submitted the ICLA and if needed, the CCLA. See Contributing for details.

@ebyhr ebyhr requested a review from a team as a code owner August 7, 2024 00:49
@ebyhr ebyhr force-pushed the ebi/errorprone-missing-override branch from cb71068 to 2b62bb1 Compare August 7, 2024 01:04
build.gradle Outdated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change needs to go into build-logic/src/main/kotlin/polaris-java.gradle.kts now (right at the top)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your review. Updated polaris-java.gradle.kts instead.

@ebyhr ebyhr force-pushed the ebi/errorprone-missing-override branch from 2b62bb1 to 6bfb85c Compare August 7, 2024 22:58
@ebyhr ebyhr force-pushed the ebi/errorprone-missing-override branch from 6bfb85c to 9e9963a Compare August 7, 2024 23:05
Copy link
Contributor

@flyrain flyrain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@flyrain flyrain merged commit 32eebae into apache:main Aug 8, 2024
@flyrain
Copy link
Contributor

flyrain commented Aug 8, 2024

Thanks @ebyhr for the PR. Thanks @snazy and @findepi for the review.

snazy added a commit to snazy/polaris that referenced this pull request Nov 20, 2025
* fix(deps): update dependency io.projectreactor.netty:reactor-netty-http to v1.2.9 (apache#2326)

* Add getting-started example with external authentication (apache#2244)

* chore(deps): update quay.io/keycloak/keycloak docker tag to v26.3.2 (apache#2331)

* fix(deps): update immutables to v2.11.3 (apache#2333)

* JWTBroker: move error message (apache#2330)

This change moves the `LOGGER.error` call when a token cannot be verified from `verify()` to `generateFromToken()`.

On the token generation path, this should be a no-op; however, on the authentication path, this log message was excessive, especially when using mixed authentication since a failure to decode a token is perfectly normal when the token is from an external IDP.

* Let CI archive html test reports (apache#2327)

when having to debug CI test failures its much more convenient to be
able to download the html report compared to the XML reports (as the
latter requires to you find the right file/failure manually).

* Make S3 `roleARN` optional (apache#2329)

Fixes apache#2325

* Remove spotbugs-annotations (apache#2320)

we dont seem to be running spotbugs/findbugs in our build, so depending
on the annotations is not necessary.

also fix name of common-codec lib.

* Remove redundant locations when constructing access policies (apache#2149)

Iceberg tables can technically store data across any number of paths, but Polaris currently uses 3 different locations for credential vending:
1. The table's base location
2. The table's `write.data.path`, if set
3. The table's `write.metadata.path`, if set

This was intended to capture scenarios where e.g. (2) is not a child path of (1), so that the vended credentials can still be valid for reading the entire table. However, there are systems that seem to always set (2) and (3), such as:

1. `s3:/my-bucket/base/iceberg`
2. `s3:/my-bucket/base/iceberg/data`
3. `s3:/my-bucket/base/iceberg/metadata`

In such cases the extra paths (e.g. extra resources in the AWS Policy) are redundant. In one such case, these redundant paths caused the policy to exceed the maximum allowable 2048 characters.

This PR removes redundant paths -- those that are the child of another path -- from the list of accessible locations tracked for a given table and does some slight refactoring to consolidate the logic for extracting these paths from a TableMetadata.

* Remove CallContext from IcebergPropertiesValidation (apache#2338)

it is sufficient to pass the `RealmConfig`.
same applies to helpers in `PolarisEndpoints`.

* Add entitySubType param to BasePersistence.listEntities (apache#2317)

`BasePersistence.listEntities` has 3 variants:
```
Page<EntityNameLookupRecord> listEntities(..., PageToken);

Page<EntityNameLookupRecord> listEntities(..., Predicate<PolarisBaseEntity>, PageToken)

<T> Page<T> listEntities(..., Predicate<PolarisBaseEntity>, Function<PolarisBaseEntity, T>, PageToken);
```

the 1st method exists to only return the subset of entity properties required to build an `EntityNameLookupRecord`.

the 3rd method supports a predicate and transformer function on the underlying `PolarisBaseEntity`, which means it has to load all entity properties.

the 2nd method is weird as it supports a full `Predicate<PolarisBaseEntity>`, which means it has to load all entity properties under the hood for filtering but then throws most of them away to return a `EntityNameLookupRecord`.
this explains why the implementations of the 2nd method simply forward to the 3rd method usually.
any performance benefits of returning a `EntityNameLookupRecord` are lost.

as it turns out the 2nd method is only used, because methods 1 and 3 dont support passing a `PolarisEntitySubType` parameter to filter down the retrieved data.
Note that the sub type property is available from both the `PolarisBaseEntity` as well as the `EntityNameLookupRecord`.

By adding this parameter, the 2nd method can go away completely.
we can even push down the sub type filtering into the queries of some of our persistence implementations.
other existing implementations are free to decide whether they want to push it down as well or filter on the query results in memory.

note that since we have no `TransactionalPersistence` implementation in the codebase that provides an optimized variant of method 1 we can have a default method in the interface that forwards to method 3.

* Add PyIceberg example (apache#2315)

It is not obvious how to connect PyIceberg to a Polaris catalog.

This PR clears that up by providing an example in the getting-started section of the documentation.

* fix(docs): fix some broken url. (apache#2335)

* fix(docs): fix entity doc API links. (apache#2316)

* fix(deps): update dependency io.netty:netty-codec-http2 to v4.2.4.final (apache#2342)

* NoSQL: Misc ports

* Adopt to the state of apache#2131 (OSS NoSQL PR / idgen)
* Track "base locations" and use an index to detect conflicts (via PolarisMetaStoreManager.hasOverlappingSiblings). Feature must be enabled in the Polaris config. Implementation prepared for intentional overlaps. Backwards compatible, except for checks against already existing tables.
* Cosmetic changes (bunch of)

* Some more adoptions from OSS

... based on a `git diff` against the OSS `persistence-nosql` PR branch.

* Last merged commit 4c23eb7

---------

Co-authored-by: Mend Renovate <[email protected]>
Co-authored-by: Alexandre Dutra <[email protected]>
Co-authored-by: Christopher Lambert <[email protected]>
Co-authored-by: Eric Maynard <[email protected]>
Co-authored-by: Frederic Khayat <[email protected]>
Co-authored-by: Yujiang Zhong <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants