File tree Expand file tree Collapse file tree 1 file changed +33
-0
lines changed Expand file tree Collapse file tree 1 file changed +33
-0
lines changed Original file line number Diff line number Diff line change 1+ # Notes for Parquet compatibility tests
2+
3+ The following directories and files are used for Parquet compatibility tests:
4+
5+ ```
6+ .
7+ ├── README.md # This file
8+ ├── avro
9+ │ ├── parquet-compat.avdl # Testing Avro IDL
10+ │ └── parquet-compat.avpr # !! NO TOUCH !! Protocol file generated from parquet-compat.avdl
11+ ├── gen-java # !! NO TOUCH !! Generated Java code
12+ ├── scripts
13+ │ └── gen-code.sh # Script used to generate Java code for Thrift and Avro
14+ └── thrift
15+ └── parquet-compat.thrift # Testing Thrift schema
16+ ```
17+
18+ Generated Java code are used in the following test suites:
19+
20+ - ` org.apache.spark.sql.parquet.ParquetAvroCompatibilitySuite `
21+ - ` org.apache.spark.sql.parquet.ParquetThriftCompatibilitySuite `
22+
23+ To avoid code generation during build time, Java code generated from testing Thrift schema and Avro IDL are also checked in.
24+
25+ When updating the testing Thrift schema and Avro IDL, please run ` gen-code.sh ` to update all the generated Java code.
26+
27+ ## Prerequisites
28+
29+ Please ensure ` avro-tools ` and ` thrift ` are installed. You may install these two on Mac OS X via:
30+
31+ ``` bash
32+ $ brew install thrift avro-tools
33+ ```
You can’t perform that action at this time.
0 commit comments