-
Notifications
You must be signed in to change notification settings - Fork 28.9k
[SPARK-12760] [DOCS] invalid lambda expression in python example for … #10867
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Does it still execute without error on a cluster? (even if it doesn't actually increment the counter in the way someone might expect.) Certainly if it doesn't compile we need to change this, but want to make sure the result with "global" executes too. |
|
Jenkins test this please |
fc9f16a to
2e1c016
Compare
|
@srowen it compiles for local, let me test that on a cluster I noticed that the next line is actually also invalid python I just updated the PR |
|
@srowen I tested the python code in cluster mode (5 ec2 workers) and this works fine |
|
LGTM then. I can merge this with the other doc updates |
|
Jenkins test this please |
|
Test build #49919 has finished for PR 10867 at commit
|
…local vs cluster srowen thanks for the PR at #10866! sorry it took me a while. This is related to #10866, basically the assignment in the lambda expression in the python example is actually invalid ``` In [1]: data = [1, 2, 3, 4, 5] In [2]: counter = 0 In [3]: rdd = sc.parallelize(data) In [4]: rdd.foreach(lambda x: counter += x) File "<ipython-input-4-fcb86c182bad>", line 1 rdd.foreach(lambda x: counter += x) ^ SyntaxError: invalid syntax ``` Author: Mortada Mehyar <[email protected]> Closes #10867 from mortada/doc_python_fix. (cherry picked from commit 56f57f8) Signed-off-by: Sean Owen <[email protected]>
|
Merged to master/1.6 |
…local vs cluster
@srowen thanks for the PR at #10866! sorry it took me a while.
This is related to #10866, basically the assignment in the lambda expression in the python example is actually invalid