[SPARK-5208][DOC] Add more documentation to Netty-based configs

[sarutak]

#3713 added some documentation about Netty-based configs but I think we need more. I think following configs can be useful for performance tuning.

• spark.shuffle.io.mode
• spark.shuffle.io.backLog
• spark.shuffle.io.receiveBuffer
• spark.shuffle.io.sendBuffer

[SparkQA]

Test build #25437 has finished for PR 4012 at commit a6e2aa7.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[pwendell]

@sarutak when we added the netty shuffle we actually decided not to expose these in order to keep the overall # of configurations manageable. We couldn't think of a user scenario where these would make a large difference (correct me if that is wrong @aarondav).

Did you have a specific use case in mind, or was this mostly for completeness reasons?

[aarondav]

I agree with Patrick -- it's a very case-by-case basis, but we decided not to document these:

• spark.shuffle.io.mode: We could not see a difference between NIO and EPOLL from a performance perspective, and EPOLL is less stable, so we did not want to suggest to use it.
• spark.shuffle.io.backLog: This is more suitable for a serving layer than a shuffle layer, I can't think why someone would want to specify this.
• spark.shuffle.io.receiveBuffer/sendBuffer: These are very low-level tuning parameters that are likely not to achieve a noticeable performance benefit, so documenting them might just lead people to think they're more important than they are.

[pwendell]

Okay @sarutak can you close this issue then? Looks like we intentionally left these out for now

[sarutak]

O.K, I'll close.