[SPARK-7698] Cache and reuse buffers in ExecutorMemoryAllocator when using heap allocation #6227
Conversation
|
Merged build triggered. |
|
Merged build started. |
|
Test build #32963 has started for PR 6227 at commit |
|
For https://gist.github.com/680ee530655941defcb2, this patch gives a roughly 3x speedup. |
|
Test build #32963 has finished for PR 6227 at commit
|
|
Merged build finished. Test PASSed. |
|
Test PASSed. |
|
Leaving the [WIP] tag on for now while we discuss a few different design decisions. |
|
@rxin @zsxwing In the long term, I think that we should consider a more complete design for buffer pooling in our allocator, including thinking through how / whether we want to support pooling for off-heap modes, how we want to match up allocation requests with things in the pages, whether we want to have more manual control over purging pages from the pool, etc. For the immediate 1.4 term, though, I think a super-simple approach like the one in this patch offers a nice improvement. Because the number of different allocation sizes is relatively small (one or two sizes, tops), I think the simple approach is fine for starters. |
There was a problem hiding this comment.
Just a minor question: I think you want to use LinkedList as Stack since you use pop. Right? If so, here you should use push. push calls addFirst, pop calls removeFirst, while add calls addLast.
There was a problem hiding this comment.
Java's linked list is doubly-linked, so I don't think that this makes a perf. difference or anything, which is why I was a little sloppy here.
There was a problem hiding this comment.
I thought there was a special reason to use pop :)
|
On closer inspection, I'm thinking that we should probably prefer WeakReferences to SoftReferences, since it's probably better to allow this memory to be released sooner in response to memory demand rather than having the heap grow in order to try to keep empty pages in the pool. |
|
@JoshRosen why not round up |
|
+1. WeakReferences is better for cache usage. |
|
@zsxwing I think that all of our internal requests are already power-of-2 sized, so I don't think that's a concern yet. Right now, I think we'll only end up allocating pages whose sizes are drawn from a very small set (maybe 4 or fewer standard page sizes, tops). We might consider adding the rounding later, though. |
|
I see. LGTM except the |
JoshRosen
changed the title
|
Just pushed a commit to fix the WeakReference thing :) |
|
Merged build triggered. |
|
Merged build started. |
|
Test build #33114 has started for PR 6227 at commit |
|
LGTM |
|
Test build #33114 has finished for PR 6227 at commit
|
|
Merged build finished. Test PASSed. |
|
Test PASSed. |
|
LGTM |
|
Thanks for the review. I'm going to merge this into master and branch-1.4 (1.4.0). |
…using heap allocation When on-heap memory allocation is used, ExecutorMemoryManager should maintain a cache / pool of buffers for re-use by tasks. This will significantly improve the performance of the new Tungsten's sort-shuffle for jobs with many short-lived tasks by eliminating a major source of GC. This pull request is a minimum-viable-implementation of this idea. In its current form, this patch significantly improves performance on a stress test which launches huge numbers of short-lived shuffle map tasks back-to-back in the same JVM. Author: Josh Rosen <[email protected]> Closes #6227 from JoshRosen/SPARK-7698 and squashes the following commits: fd6cb55 [Josh Rosen] SoftReference -> WeakReference b154e86 [Josh Rosen] WIP sketch of pooling in ExecutorMemoryManager (cherry picked from commit 7956dd7) Signed-off-by: Josh Rosen <[email protected]>
…using heap allocation When on-heap memory allocation is used, ExecutorMemoryManager should maintain a cache / pool of buffers for re-use by tasks. This will significantly improve the performance of the new Tungsten's sort-shuffle for jobs with many short-lived tasks by eliminating a major source of GC. This pull request is a minimum-viable-implementation of this idea. In its current form, this patch significantly improves performance on a stress test which launches huge numbers of short-lived shuffle map tasks back-to-back in the same JVM. Author: Josh Rosen <[email protected]> Closes apache#6227 from JoshRosen/SPARK-7698 and squashes the following commits: fd6cb55 [Josh Rosen] SoftReference -> WeakReference b154e86 [Josh Rosen] WIP sketch of pooling in ExecutorMemoryManager
…using heap allocation When on-heap memory allocation is used, ExecutorMemoryManager should maintain a cache / pool of buffers for re-use by tasks. This will significantly improve the performance of the new Tungsten's sort-shuffle for jobs with many short-lived tasks by eliminating a major source of GC. This pull request is a minimum-viable-implementation of this idea. In its current form, this patch significantly improves performance on a stress test which launches huge numbers of short-lived shuffle map tasks back-to-back in the same JVM. Author: Josh Rosen <[email protected]> Closes apache#6227 from JoshRosen/SPARK-7698 and squashes the following commits: fd6cb55 [Josh Rosen] SoftReference -> WeakReference b154e86 [Josh Rosen] WIP sketch of pooling in ExecutorMemoryManager
…using heap allocation When on-heap memory allocation is used, ExecutorMemoryManager should maintain a cache / pool of buffers for re-use by tasks. This will significantly improve the performance of the new Tungsten's sort-shuffle for jobs with many short-lived tasks by eliminating a major source of GC. This pull request is a minimum-viable-implementation of this idea. In its current form, this patch significantly improves performance on a stress test which launches huge numbers of short-lived shuffle map tasks back-to-back in the same JVM. Author: Josh Rosen <[email protected]> Closes apache#6227 from JoshRosen/SPARK-7698 and squashes the following commits: fd6cb55 [Josh Rosen] SoftReference -> WeakReference b154e86 [Josh Rosen] WIP sketch of pooling in ExecutorMemoryManager
5 participants
When on-heap memory allocation is used, ExecutorMemoryManager should maintain a cache / pool of buffers for re-use by tasks. This will significantly improve the performance of the new Tungsten's sort-shuffle for jobs with many short-lived tasks by eliminating a major source of GC.
This pull request is a minimum-viable-implementation of this idea. In its current form, this patch significantly improves performance on a stress test which launches huge numbers of short-lived shuffle map tasks back-to-back in the same JVM.