Skip to content

Commit f2208cd

Browse files
committed
Adds README.md for Thrift/Avro code generation
1 parent 1d390aa commit f2208cd

File tree

1 file changed

+33
-0
lines changed

1 file changed

+33
-0
lines changed

sql/core/src/test/README.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Notes for Parquet compatibility tests
2+
3+
The following directories and files are used for Parquet compatibility tests:
4+
5+
```
6+
.
7+
├── README.md # This file
8+
├── avro
9+
│   ├── parquet-compat.avdl # Testing Avro IDL
10+
│   └── parquet-compat.avpr # !! NO TOUCH !! Protocol file generated from parquet-compat.avdl
11+
├── gen-java # !! NO TOUCH !! Generated Java code
12+
├── scripts
13+
│   └── gen-code.sh # Script used to generate Java code for Thrift and Avro
14+
└── thrift
15+
└── parquet-compat.thrift # Testing Thrift schema
16+
```
17+
18+
Generated Java code are used in the following test suites:
19+
20+
- `org.apache.spark.sql.parquet.ParquetAvroCompatibilitySuite`
21+
- `org.apache.spark.sql.parquet.ParquetThriftCompatibilitySuite`
22+
23+
To avoid code generation during build time, Java code generated from testing Thrift schema and Avro IDL are also checked in.
24+
25+
When updating the testing Thrift schema and Avro IDL, please run `gen-code.sh` to update all the generated Java code.
26+
27+
## Prerequisites
28+
29+
Please ensure `avro-tools` and `thrift` are installed. You may install these two on Mac OS X via:
30+
31+
```bash
32+
$ brew install thrift avro-tools
33+
```

0 commit comments

Comments
 (0)