[SPARK-30199][DSTREAM] Recover spark.(ui|blockManager).port from checkpoint

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to recover spark.ui.port and spark.blockManager.port from checkpoint like spark.driver.port.

Why are the changes needed?

When the user configures these values, we can respect them.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the Jenkins with the newly added test cases.

[dongjoon-hyun]

cc @dbtsai

[ScrapCodes]

Hi @dongjoon-hyun, Can you please explain, how it is useful to recover these port numbers from checkpoint. As these can be picked up from configuration, and may not always be desirable to be picked from checkpoint. For instance the ports availability change between the two runs.

[SparkQA]

Test build #115087 has finished for PR 26827 at commit 771ac88.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #115089 has finished for PR 26827 at commit cfa7494.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Thank you for review, @ScrapCodes . This PR aims to keep only user-given port numbers like spark.driver.port. You can see the UT about that.

As these can be picked up from configuration, and may not always be desirable to be picked from checkpoint. For instance the ports availability change between the two runs.

[dongjoon-hyun]

Hi, @jerryshao . Could you review this PR? This is basically the same with your spark.driver.port PR (#5060)

[jerryshao]

The fix LGTM.

[dongjoon-hyun]

Thank you so much, @jerryshao !

[dongjoon-hyun]

Merged to master.