Skip to content

Conversation

@brkyvz
Copy link
Contributor

@brkyvz brkyvz commented Nov 30, 2015

Python tests require access to the KinesisTestUtils file. When this file exists under src/test, python can't access it, since it is not available in the assembly jar.

However, if we move KinesisTestUtils to src/main, we need to add the KinesisProducerLibrary as a dependency. In order to avoid this, I moved KinesisTestUtils to src/main, and extended it with ExtendedKinesisTestUtils which is under src/test that adds support for the KPL.

cc @zsxwing @tdas

@SparkQA
Copy link

SparkQA commented Dec 1, 2015

Test build #46910 has finished for PR 10050 at commit 2e27264.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@zsxwing
Copy link
Member

zsxwing commented Dec 1, 2015

@brkyvz could you enable KinesisStreamTests in this PR? I just disabled it to unblock other PR.

@brkyvz
Copy link
Contributor Author

brkyvz commented Dec 1, 2015

jenkins retest this please

@SparkQA
Copy link

SparkQA commented Dec 1, 2015

Test build #46926 has finished for PR 10050 at commit 42d163c.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@tdas
Copy link
Contributor

tdas commented Dec 2, 2015

jenkins, test this please

@brkyvz
Copy link
Contributor Author

brkyvz commented Dec 2, 2015

jenkins test this please

@SparkQA
Copy link

SparkQA commented Dec 2, 2015

Test build #47091 has finished for PR 10050 at commit 9c208d7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 2, 2015

Test build #47090 has finished for PR 10050 at commit 9c208d7.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets name it KinesisDataGenerator, to avoid confusion with "producer" in KPL.
KinesisProducer --> KinesisDataGenerator
KinesisClientProducer --> SimpleDataGenerator
KinesisProducerLibraryProducer --> KPLDataGenerator

@SparkQA
Copy link

SparkQA commented Dec 3, 2015

Test build #47102 has finished for PR 10050 at commit 0192d2d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 4, 2015

Test build #2169 has finished for PR 10050 at commit 0192d2d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Dec 4, 2015

Test build #2170 has finished for PR 10050 at commit 0192d2d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@tdas
Copy link
Contributor

tdas commented Dec 4, 2015

LGTM. Merging this to master and 1.6. Thanks!

@asfgit asfgit closed this in 302d68d Dec 4, 2015
asfgit pushed a commit that referenced this pull request Dec 4, 2015
Python tests require access to the `KinesisTestUtils` file. When this file exists under src/test, python can't access it, since it is not available in the assembly jar.

However, if we move KinesisTestUtils to src/main, we need to add the KinesisProducerLibrary as a dependency. In order to avoid this, I moved KinesisTestUtils to src/main, and extended it with ExtendedKinesisTestUtils which is under src/test that adds support for the KPL.

cc zsxwing tdas

Author: Burak Yavuz <[email protected]>

Closes #10050 from brkyvz/kinesis-py.
@brkyvz brkyvz deleted the kinesis-py branch February 3, 2019 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants