fix intermittent issues with viewer's "Download Data"

[carmichaelong]

After about a month or two, the Download Data button on the viewer's end can initiate a download (initiating the API to gather and zip files), but then the dialogue box never shows a final download button. Redeploying the API fixes the issue temporarily.

We should find and fix the root cause of this issue. One lead was that maybe old .zip files were not being cleared properly and the API worker fills up, but we would need to verify this first.

[carmichaelong]

Not sure if this is the actual underlying issue until some errors crop up and we can check with ECS Exec (to see if files are being left behind), but possible theory:

Sentry shows two recurring errors on this line that recursively deletes a target folder, which may be leaving behind stale files and filling the ECS ephemeral storage that has 20GB of space:

1. FileNotFoundError due to a race condition. It's fixed but would need to bump python version. There might be other ways to mitigate, such as by catching this exact error and ignoring it (unless it's the top-level path passed in).
2. OSError: Directory not empty. They are choosing not to address this issue because of possible workaround, such as using the ignore_errors=True optional argument, using retries, or catching this specific error and handling it.