[SPARK-27755][BUILD] Update zstd-jni to 1.4.0-1

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to update zstd-jni library to 1.4.0-1 which improves the level 1 compression speed performance by 6% in most scenarios. The following is the full release note.

• https://github.com/facebook/zstd/releases/tag/v1.4.0

How was this patch tested?

Pass the Jenkins.

[SparkQA]

Test build #105481 has finished for PR 24632 at commit 017841c.

• This patch fails due to an unknown error code, -9.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Retest this please.

[kiszk]

Looks good to update zstd

[Fokko]

Interesting performance improvement. LGTM

[viirya]

Looks good. The license file currently in Spark isn't updated to date with https://github.com/luben/zstd-jni/blob/master/LICENSE in the end date. Shall we update it together?

[SparkQA]

Test build #105483 has finished for PR 24632 at commit 017841c.

• This patch fails SparkR unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[srowen]

The license looks like the same BSD 2-clause? OK by me.

[dongjoon-hyun]

Thank you for review, @kiszk , @Fokko , @viirya , @srowen .
Yep. The license is the same with the existing one in LICENSE-binary.

[SparkQA]

Test build #4781 has finished for PR 24632 at commit 017841c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Thank you, guys. Merged to master!

[melin]

[dongjoon-hyun]

Hi, @melin . Could you write in English? This is mainly for Hadoop 3 profile in Apache Spark. Spark 3.0 will support Hadoop 3.2.0+.