Skip to content

Commit d2054fc

Browse files
panbingkundongjoon-hyun
authored andcommitted
[SPARK-44128][BUILD] Upgrade netty to 4.1.93
### What changes were proposed in this pull request? This pr aims to upgrade netty from 4.1.92 to 4.1.93. ### Why are the changes needed? 1.v4.1.92 VS v4.1.93 netty/netty@netty-4.1.92.Final...netty-4.1.93.Final 2.The new version brings some bug fix, eg: - Reset byte buffer in loop for AbstractDiskHttpData.setContent ([#13320](netty/netty#13320)) - OpenSSL MAX_CERTIFICATE_LIST_BYTES option supported ([#13365](netty/netty#13365)) - Adapt to DirectByteBuffer constructor in Java 21 ([#13366](netty/netty#13366)) - HTTP/2 encoder: allow HEADER_TABLE_SIZE greater than Integer.MAX_VALUE ([#13368](netty/netty#13368)) - Upgrade to latest netty-tcnative to fix memory leak ([#13375](netty/netty#13375)) - H2/H2C server stream channels deactivated while write still in progress ([#13388](netty/netty#13388)) - Channel#bytesBefore(un)writable off by 1 ([#13389](netty/netty#13389)) - HTTP/2 should forward shutdown user events to active streams ([#13394](netty/netty#13394)) - Respect the number of bytes read per datagram when using recvmmsg ([#13399](netty/netty#13399)) 3.The release notes as follows: - https://netty.io/news/2023/05/25/4-1-93-Final.html 4.Why not upgrade to `4-1-94-Final` version? Because the return value of the 'threadCache()' (from `PoolThreadCache` to `PoolArenasCache`) method of the netty Inner class used in the 'arrow memory netty' version '12.0.1' has changed and belongs to break change, let's wait for the upgrade of the 'arrow memory netty' before upgrading to the '4-1-94-Final' version. The reference is as follows: https://github.com/apache/arrow/blob/6af660f48472b8b45a5e01b7136b9b040b185eb1/java/memory/memory-netty/src/main/java/io/netty/buffer/PooledByteBufAllocatorL.java#L164 https://github.com/netty/netty/blob/da1a448d5bc4f36cc1744db93fcaf64e198db2bd/buffer/src/main/java/io/netty/buffer/PooledByteBufAllocator.java#L732-L736 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass GA. Closes #41681 from panbingkun/upgrade_netty. Authored-by: panbingkun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 1fcd537 commit d2054fc

File tree

2 files changed

+23
-19
lines changed

2 files changed

+23
-19
lines changed

dev/deps/spark-deps-hadoop-3-hive-2.3

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -183,24 +183,24 @@ metrics-jmx/4.2.18//metrics-jmx-4.2.18.jar
183183
metrics-json/4.2.18//metrics-json-4.2.18.jar
184184
metrics-jvm/4.2.18//metrics-jvm-4.2.18.jar
185185
minlog/1.3.0//minlog-1.3.0.jar
186-
netty-all/4.1.92.Final//netty-all-4.1.92.Final.jar
187-
netty-buffer/4.1.92.Final//netty-buffer-4.1.92.Final.jar
188-
netty-codec-http/4.1.92.Final//netty-codec-http-4.1.92.Final.jar
189-
netty-codec-http2/4.1.92.Final//netty-codec-http2-4.1.92.Final.jar
190-
netty-codec-socks/4.1.92.Final//netty-codec-socks-4.1.92.Final.jar
191-
netty-codec/4.1.92.Final//netty-codec-4.1.92.Final.jar
192-
netty-common/4.1.92.Final//netty-common-4.1.92.Final.jar
193-
netty-handler-proxy/4.1.92.Final//netty-handler-proxy-4.1.92.Final.jar
194-
netty-handler/4.1.92.Final//netty-handler-4.1.92.Final.jar
195-
netty-resolver/4.1.92.Final//netty-resolver-4.1.92.Final.jar
196-
netty-transport-classes-epoll/4.1.92.Final//netty-transport-classes-epoll-4.1.92.Final.jar
197-
netty-transport-classes-kqueue/4.1.92.Final//netty-transport-classes-kqueue-4.1.92.Final.jar
198-
netty-transport-native-epoll/4.1.92.Final/linux-aarch_64/netty-transport-native-epoll-4.1.92.Final-linux-aarch_64.jar
199-
netty-transport-native-epoll/4.1.92.Final/linux-x86_64/netty-transport-native-epoll-4.1.92.Final-linux-x86_64.jar
200-
netty-transport-native-kqueue/4.1.92.Final/osx-aarch_64/netty-transport-native-kqueue-4.1.92.Final-osx-aarch_64.jar
201-
netty-transport-native-kqueue/4.1.92.Final/osx-x86_64/netty-transport-native-kqueue-4.1.92.Final-osx-x86_64.jar
202-
netty-transport-native-unix-common/4.1.92.Final//netty-transport-native-unix-common-4.1.92.Final.jar
203-
netty-transport/4.1.92.Final//netty-transport-4.1.92.Final.jar
186+
netty-all/4.1.93.Final//netty-all-4.1.93.Final.jar
187+
netty-buffer/4.1.93.Final//netty-buffer-4.1.93.Final.jar
188+
netty-codec-http/4.1.93.Final//netty-codec-http-4.1.93.Final.jar
189+
netty-codec-http2/4.1.93.Final//netty-codec-http2-4.1.93.Final.jar
190+
netty-codec-socks/4.1.93.Final//netty-codec-socks-4.1.93.Final.jar
191+
netty-codec/4.1.93.Final//netty-codec-4.1.93.Final.jar
192+
netty-common/4.1.93.Final//netty-common-4.1.93.Final.jar
193+
netty-handler-proxy/4.1.93.Final//netty-handler-proxy-4.1.93.Final.jar
194+
netty-handler/4.1.93.Final//netty-handler-4.1.93.Final.jar
195+
netty-resolver/4.1.93.Final//netty-resolver-4.1.93.Final.jar
196+
netty-transport-classes-epoll/4.1.93.Final//netty-transport-classes-epoll-4.1.93.Final.jar
197+
netty-transport-classes-kqueue/4.1.93.Final//netty-transport-classes-kqueue-4.1.93.Final.jar
198+
netty-transport-native-epoll/4.1.93.Final/linux-aarch_64/netty-transport-native-epoll-4.1.93.Final-linux-aarch_64.jar
199+
netty-transport-native-epoll/4.1.93.Final/linux-x86_64/netty-transport-native-epoll-4.1.93.Final-linux-x86_64.jar
200+
netty-transport-native-kqueue/4.1.93.Final/osx-aarch_64/netty-transport-native-kqueue-4.1.93.Final-osx-aarch_64.jar
201+
netty-transport-native-kqueue/4.1.93.Final/osx-x86_64/netty-transport-native-kqueue-4.1.93.Final-osx-x86_64.jar
202+
netty-transport-native-unix-common/4.1.93.Final//netty-transport-native-unix-common-4.1.93.Final.jar
203+
netty-transport/4.1.93.Final//netty-transport-4.1.93.Final.jar
204204
objenesis/3.3//objenesis-3.3.jar
205205
okhttp/3.12.12//okhttp-3.12.12.jar
206206
okio/1.15.0//okio-1.15.0.jar

pom.xml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -212,7 +212,11 @@
212212
<commons-cli.version>1.5.0</commons-cli.version>
213213
<bouncycastle.version>1.60</bouncycastle.version>
214214
<tink.version>1.9.0</tink.version>
215-
<netty.version>4.1.92.Final</netty.version>
215+
<!--
216+
Please don't upgrade the version to 4.1.94.Final,
217+
it needs to wait for `arrow-memory-netty` to be greater than 12.0.1.
218+
-->
219+
<netty.version>4.1.93.Final</netty.version>
216220
<!--
217221
If you are changing Arrow version specification, please check
218222
./python/pyspark/sql/pandas/utils.py, and ./python/setup.py too.

0 commit comments

Comments
 (0)