Fix LICENSE and NOTICE in Spark plugin #1834
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jbonofre
requested review from
HonahX,
MonkeyCanCode,
RussellSpitzer,
ajantha-bhat,
collado-mike,
dennishuo,
dimas-b,
ebyhr,
eric-maynard,
flyrain,
jackye1995,
pingtimeout,
singhpk234,
snazy,
takidau and
vvcephei
as code owners
Contributor
|
@jbonofre Thanks a lot for helping on the license. Regarding to the shaded Jackson library, it was shaded again by the polaris plugin because we are not able to reuse the shaded Jackson library due to inconsistency issue. so we introduces the dependency again, and shaded in the packaged jar, and since we also included the whole iceberg spark client, which also shaded the jackson library, so we end up shaded jackson twice. Can we keep both shade for the current jar? |
gh-yzou
approved these changes
Jun 9, 2025
Member
Author
|
@gh-yzou yeah, we can keep the duplicate dependencies (there are several dependencies). Let me do a quick check (listing the duplicate dependencies). |
flyrain
reviewed
Jun 9, 2025
Comment on lines
207
to
208
| * plugins/spark/v3.5/spark/src/main/java/org/apache/polaris/spark/PolarisSparkCatalog.java | ||
| * plugins/spark/v3.5/spark/src/test/java/org/apache/polaris/spark/PolarisInMemoryCatalog.java |
Contributor
There was a problem hiding this comment.
These two lines are also not needed per offline discussion, as they don't copy any code from Iceberg. Can you confirm, @gh-yzou ?
Contributor
There was a problem hiding this comment.
Yes, those two are not copied from iceberg, those are Polaris specific with generic table implementations
flyrain
approved these changes
Jun 9, 2025
github-project-automation
bot
moved this from PRs In Progress
to Ready to merge
in Basic Kanban Board
github-project-automation
bot
moved this from Ready to merge
to Done
in Basic Kanban Board
Contributor
snazy
added a commit
to snazy/polaris
that referenced
this pull request
Jun 13, 2025
* main: Update dependency io.netty:netty-codec-http2 to v4.2.2.Final (apache#1819) * main: Update dependency com.google.cloud:google-cloud-storage-bom to v2.53.0 (apache#1811) * main: Pin actions/setup-java action to c5195ef (apache#1809) * main: Update dependency boto3 to v1.38.31 (apache#1803) * Downgrade open api generator to 7.11 (apache#1823) * main: Update dependency io.smallrye.config:smallrye-config-core to v3.13.1 (apache#1826) * JDBC: Use PreparedStatement (apache#1802) * Remove "unused" files in client/python (apache#1678) * main: Update dependency org.junit:junit-bom to v5.13.1 (apache#1832) * main: Update dependency boto3 to v1.38.32 (apache#1828) * main: Update dependency io.opentelemetry:opentelemetry-bom to v1.51.0 (apache#1829) * Spark: Add license and notice to spark client jar and push jar to maven (apache#1830) * Update versions in LICENSE and NOTICE (apache#1833) * Core: Fix maxCode generation for ConnectionTypeEnum (apache#1827) * main: Update dependency com.github.ben-manes.caffeine:caffeine to v3.2.1 (apache#1836) * main: Update dependency software.amazon.awssdk:bom to v2.31.59 (apache#1835) * Fix the issue where the Polaris Server exposes backend metadata in the error response body when the database is not bootstrapped. (apache#1837) * Fix LICENSE and NOTICE in Spark plugin (apache#1834) * main: Update dependency io.micrometer:micrometer-bom to v1.15.1 (apache#1842) * main: Update dependency boto3 to v1.38.33 (apache#1841) * NoSQL: bump dependencies * Last merged commit 9934fc4 --------- Co-authored-by: Mend Renovate <[email protected]> Co-authored-by: Honah (Jonas) J. <[email protected]> Co-authored-by: Prashant Singh <[email protected]> Co-authored-by: JB Onofré <[email protected]> Co-authored-by: gh-yzou <[email protected]> Co-authored-by: Pooja Nilangekar <[email protected]> Co-authored-by: Bo Wang <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes
LICENSEandNOTICEin the Polaris Spark plugin.@flyrain @gh-yzou I'm observing some "issues" in the Spark plugin. I see some packages are shaded multiple times. For instance Jackson is shaded in Iceberg and we shade again in Polaris Spark plugin.
Imho, the Polaris Spark plugin needs some cleanup in the bundle artifact.