Skip to content

Conversation

@delamarch3
Copy link
Contributor

@delamarch3 delamarch3 commented Jul 27, 2025

Which issue does this PR close?

Rationale for this change

Make temp_directory and max_temp_directory_size confgurable

What changes are included in this PR?

  • Accept temp_directory and max_temp_directory_size as a runtime variable
  • Add a test to check the max directory size is being enforced after setting
  • Update config entries/docs

Are these changes tested?

Yes

Are there any user-facing changes?

New config variables

@github-actions github-actions bot added core Core DataFusion crate execution Related to the execution crate labels Jul 27, 2025
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Jul 27, 2025
@delamarch3 delamarch3 marked this pull request as ready for review July 27, 2025 19:24
Copy link
Member

@xudong963 xudong963 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @delamarch3

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @delamarch3 and @xudong963

@alamb alamb merged commit f2086f3 into apache:main Jul 29, 2025
28 checks passed
Standing-Man pushed a commit to Standing-Man/datafusion that referenced this pull request Aug 4, 2025
…iables (apache#16934)

* add temp_dir runtime config variable

* rename temp_dir to temp_directory

* add max_temp_directory_size config option

* tidy up

* update test for acceptable max temp directory size

* add new runtime config entries to docs

* update test name
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate documentation Improvements or additions to documentation execution Related to the execution crate

Projects

None yet

3 participants