[SPARK-47270][SQL] Dataset.isEmpty projects CommandResults locally

[wForget]

What changes were proposed in this pull request?

Similar to #40779, Dataset.isEmpty should also not trigger job execution on CommandResults.

This PR converts CommandResult to LocalRelation in Dataset.isEmpty method.

Why are the changes needed?

A simple spark.sql("show tables").isEmpty shouldn not require an executor.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Added new UT.

Was this patch authored or co-authored using generative AI tooling?

No

[wForget]

@peter-toth @HyukjinKwon @cloud-fan could you please take a look?

[peter-toth]

I don't fully get this issue. In #40779 the isTemporary column had to be casted to string so a job was triggered. But why does isEmpty trigger a job? Also, do other APIs (like head()) trigger jobs on CommandResults that shouldn't?

[wForget]

I don't fully get this issue. In #40779 the isTemporary column had to be casted to string so a job was triggered. But why does isEmpty trigger a job? Also, do other APIs (like head()) trigger jobs on CommandResults that shouldn't?

isEmpty added an empty project and will also trigger a job.

select().limit(1).queryExecution

[wForget]

Also, do other APIs (like head()) trigger jobs on CommandResults that shouldn't?

The head() method will not trigger a job. Because CollectLimitExec.executeCollect() calls child.executeTake(CommandResultExec.executeTake) which will not trigger a job.

[peter-toth]

Looks good, thanks for clarifying my questions.

[cloud-fan]

thanks, merging to master!