Commit b1493d8
[SPARK-35398][SQL] Simplify the way to get classes from ClassBodyEvaluator in
### What changes were proposed in this pull request?
SPARK-35253 upgraded janino from 3.0.16 to 3.1.4, `ClassBodyEvaluator` provides the `getBytecodes` method to get
the mapping from `ClassFile#getThisClassName` to `ClassFile#toByteArray` directly in this version and we don't need to get this variable by reflection api anymore.
So the main purpose of this pr is simplify the way to get `bytecodes` from `ClassBodyEvaluator` in `CodeGenerator#updateAndGetCompilationStats` method.
### Why are the changes needed?
Code simplification.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass the Jenkins or GitHub Action
- Manual test:
1. Define a code fragment to be tested, for example:
```
val codeBody = s"""
public java.lang.Object generate(Object[] references) {
return new TestMetricCode(references);
}
class TestMetricCode {
public TestMetricCode(Object[] references) {
}
public long sumOfSquares(long left, long right) {
return left * left + right * right;
}
}
"""
```
2. Create a `ClassBodyEvaluator` and `cook` the `codeBody` as above, the process of creating `ClassBodyEvaluator` can extract from `CodeGenerator#doCompile` method.
3. Get `bytecodes` using `ClassBodyEvaluator#getBytecodes` api(after this pr) and reflection api(before this pr) respectively, then assert that they are the same. If the `bytecodes` not changed, we can be sure that metrics state will not change. The test code example as follows:
```
import scala.collection.JavaConverters._
val bytecodesFromApi = evaluator.getBytecodes.asScala
val bytecodesFromReflectionApi = {
val scField = classOf[ClassBodyEvaluator].getDeclaredField("sc")
scField.setAccessible(true)
val compiler = scField.get(evaluator).asInstanceOf[SimpleCompiler]
val loader = compiler.getClassLoader.asInstanceOf[ByteArrayClassLoader]
val classesField = loader.getClass.getDeclaredField("classes")
classesField.setAccessible(true)
classesField.get(loader).asInstanceOf[java.util.Map[String, Array[Byte]]].asScala
}
assert(bytecodesFromApi == bytecodesFromReflectionApi)
```
Closes #32536 from LuciferYang/SPARK-35253-FOLLOWUP.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: Takeshi Yamamuro <[email protected]>CodeGenerator.updateAndGetCompilationStats method1 parent 186477c commit b1493d8
File tree
1 file changed
+2
-12
lines changed- sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen
1 file changed
+2
-12
lines changedLines changed: 2 additions & 12 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
21 | | - | |
22 | 21 | | |
23 | 22 | | |
24 | 23 | | |
| |||
28 | 27 | | |
29 | 28 | | |
30 | 29 | | |
31 | | - | |
32 | | - | |
| 30 | + | |
33 | 31 | | |
34 | 32 | | |
35 | 33 | | |
| |||
1434 | 1432 | | |
1435 | 1433 | | |
1436 | 1434 | | |
1437 | | - | |
1438 | | - | |
1439 | | - | |
1440 | | - | |
1441 | | - | |
1442 | | - | |
1443 | | - | |
1444 | | - | |
1445 | | - | |
| 1435 | + | |
1446 | 1436 | | |
1447 | 1437 | | |
1448 | 1438 | | |
| |||
0 commit comments