Skip to content

Commit 8f6b185

Browse files
itholicMaxGekk
authored andcommitted
[SPARK-40663][SQL] Migrate execution errors onto error classes: _LEGACY_ERROR_TEMP_2251-2275
### What changes were proposed in this pull request? This PR proposes to migrate 25 execution errors onto temporary error classes with the prefix `_LEGACY_ERROR_TEMP_2251` to `_LEGACY_ERROR_TEMP_2275`. The error classes are prefixed with `_LEGACY_ERROR_TEMP_` indicates the dev-facing error messages, and won't be exposed to end users. ### Why are the changes needed? To speed-up the error class migration. The migration on temporary error classes allow us to analyze the errors, so we can detect the most popular error classes. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? ``` $ build/sbt "sql/testOnly org.apache.spark.sql.SQLQueryTestSuite" $ build/sbt "test:testOnly *SQLQuerySuite" $ build/sbt -Phive-thriftserver "hive-thriftserver/testOnly org.apache.spark.sql.hive.thriftserver.ThriftServerQueryTestSuite" ``` Closes apache#38175 from itholic/SPARK-40540-2251-2275. Authored-by: itholic <[email protected]> Signed-off-by: Max Gekk <[email protected]>
1 parent c7fa847 commit 8f6b185

File tree

5 files changed

+264
-81
lines changed

5 files changed

+264
-81
lines changed

core/src/main/resources/error/error-classes.json

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4303,6 +4303,140 @@
43034303
"Not enough memory to build and broadcast the table to all worker nodes. As a workaround, you can either disable broadcast by setting <autoBroadcastjoinThreshold> to -1 or increase the spark driver memory by setting <driverMemory> to a higher value<analyzeTblMsg>"
43044304
]
43054305
},
4306+
"_LEGACY_ERROR_TEMP_2251" : {
4307+
"message" : [
4308+
"<execName> does not support the execute() code path."
4309+
]
4310+
},
4311+
"_LEGACY_ERROR_TEMP_2252" : {
4312+
"message" : [
4313+
"Cannot merge <className> with <otherClass>"
4314+
]
4315+
},
4316+
"_LEGACY_ERROR_TEMP_2253" : {
4317+
"message" : [
4318+
"Data source <sourceName> does not support continuous processing."
4319+
]
4320+
},
4321+
"_LEGACY_ERROR_TEMP_2254" : {
4322+
"message" : [
4323+
"Data read failed"
4324+
]
4325+
},
4326+
"_LEGACY_ERROR_TEMP_2255" : {
4327+
"message" : [
4328+
"Epoch marker generation failed"
4329+
]
4330+
},
4331+
"_LEGACY_ERROR_TEMP_2256" : {
4332+
"message" : [
4333+
"Foreach writer has been aborted due to a task failure"
4334+
]
4335+
},
4336+
"_LEGACY_ERROR_TEMP_2258" : {
4337+
"message" : [
4338+
"Error reading delta file <fileToRead> of <clazz>: key size cannot be <keySize>"
4339+
]
4340+
},
4341+
"_LEGACY_ERROR_TEMP_2259" : {
4342+
"message" : [
4343+
"Error reading snapshot file <fileToRead> of <clazz>: <message>"
4344+
]
4345+
},
4346+
"_LEGACY_ERROR_TEMP_2260" : {
4347+
"message" : [
4348+
"Cannot purge as it might break internal state."
4349+
]
4350+
},
4351+
"_LEGACY_ERROR_TEMP_2261" : {
4352+
"message" : [
4353+
"Clean up source files is not supported when reading from the output directory of FileStreamSink."
4354+
]
4355+
},
4356+
"_LEGACY_ERROR_TEMP_2262" : {
4357+
"message" : [
4358+
"latestOffset(Offset, ReadLimit) should be called instead of this method"
4359+
]
4360+
},
4361+
"_LEGACY_ERROR_TEMP_2263" : {
4362+
"message" : [
4363+
"Error: we detected a possible problem with the location of your checkpoint and you",
4364+
"likely need to move it before restarting this query.",
4365+
"",
4366+
"Earlier version of Spark incorrectly escaped paths when writing out checkpoints for",
4367+
"structured streaming. While this was corrected in Spark 3.0, it appears that your",
4368+
"query was started using an earlier version that incorrectly handled the checkpoint",
4369+
"path.",
4370+
"",
4371+
"Correct Checkpoint Directory: <checkpointPath>",
4372+
"Incorrect Checkpoint Directory: <legacyCheckpointDir>",
4373+
"",
4374+
"Please move the data from the incorrect directory to the correct one, delete the",
4375+
"incorrect directory, and then restart this query. If you believe you are receiving",
4376+
"this message in error, you can disable it with the SQL conf",
4377+
"<StreamingCheckpointEscapedPathCheckEnabled>."
4378+
]
4379+
},
4380+
"_LEGACY_ERROR_TEMP_2264" : {
4381+
"message" : [
4382+
"Subprocess exited with status <exitCode>. Error: <stderrBuffer>"
4383+
]
4384+
},
4385+
"_LEGACY_ERROR_TEMP_2265" : {
4386+
"message" : [
4387+
"<nodeName> without serde does not support <dt> as output data type"
4388+
]
4389+
},
4390+
"_LEGACY_ERROR_TEMP_2266" : {
4391+
"message" : [
4392+
"Invalid `startIndex` provided for generating iterator over the array. Total elements: <numRows>, requested `startIndex`: <startIndex>"
4393+
]
4394+
},
4395+
"_LEGACY_ERROR_TEMP_2267" : {
4396+
"message" : [
4397+
"The backing <className> has been modified since the creation of this Iterator"
4398+
]
4399+
},
4400+
"_LEGACY_ERROR_TEMP_2268" : {
4401+
"message" : [
4402+
"<nodeName> does not implement doExecuteBroadcast"
4403+
]
4404+
},
4405+
"_LEGACY_ERROR_TEMP_2269" : {
4406+
"message" : [
4407+
"<globalTempDB> is a system preserved database, please rename your existing database to resolve the name conflict, or set a different value for <globalTempDatabase>, and launch your Spark application again."
4408+
]
4409+
},
4410+
"_LEGACY_ERROR_TEMP_2270" : {
4411+
"message" : [
4412+
"comment on table is not supported"
4413+
]
4414+
},
4415+
"_LEGACY_ERROR_TEMP_2271" : {
4416+
"message" : [
4417+
"UpdateColumnNullability is not supported"
4418+
]
4419+
},
4420+
"_LEGACY_ERROR_TEMP_2272" : {
4421+
"message" : [
4422+
"Rename column is only supported for MySQL version 8.0 and above."
4423+
]
4424+
},
4425+
"_LEGACY_ERROR_TEMP_2273" : {
4426+
"message" : [
4427+
"<message>"
4428+
]
4429+
},
4430+
"_LEGACY_ERROR_TEMP_2274" : {
4431+
"message" : [
4432+
"Nested field <colName> is not supported."
4433+
]
4434+
},
4435+
"_LEGACY_ERROR_TEMP_2275" : {
4436+
"message" : [
4437+
"Dataset transformations and actions can only be invoked by the driver, not inside of other Dataset transformations; for example, dataset1.map(x => dataset2.values.count() * x) is invalid because the values transformation and count action cannot be performed inside of the dataset1.map transformation. For more information, see SPARK-28702."
4438+
]
4439+
},
43064440
"_LEGACY_ERROR_TEMP_2276" : {
43074441
"message" : [
43084442
"Hive table <tableName> with ANSI intervals is not supported"

0 commit comments

Comments
 (0)