Skip to content

zig package manager should retry fetching on temporary errors #17472

@mitchellh

Description

@mitchellh

Zig Version

0.12.0-dev.790+ad6f8e3a5

Steps to Reproduce and Observed Behavior

My project (Ghostty) has ~20 dependencies it has to fetch from the internet given an uncached build. These are fetched from multiple sources (github, gitlab, self-hosted, etc.). Each push results in ~7 separate CI runs which run uncached. As a result, each Git push results in ~140 new HTTP fetches for these large files.

Even with this relatively small number, its big enough that I'm regularly seeing temporary errors fail CI. These are the sorts of errors that an immediate retry fixes the issue 90% of the time, and a very short backoff fixes the issue the remaining 10% of the time. So far, none of the failures have been real persisted outages.

Expected Behavior

Zig should retry on fetch errors.

The ones I've seen but this likely isn't exhaustive:

  • error.TemporaryNameServerFailure temporary is even in the name!
  • HTTP 500 errors, namely 502 which is usually a single backend down while others are still up
  • HTTP 400 errors even, I've received a 404 before when the package existed, likely a CDN issue where some edges have it and others don't

My recommendation would be to do a short series of backoffs. Even backing off a few times with a maximum total wait time of 10 seconds would probably fix 90% of my issues, but perhaps this can be configurable because for CI I'd happily configure this to upwards of a few minutes.

The backoff should probably increase, whether it be exponential, fibbonacci, whatever.

As a workaround, I can make my whole CI job retry, but the CI job does a lot of other setup and no other steps in the job are flaky so this would probably result in retrying genuine failures and cost me more money. It'd be much more convenient and faster if Zig retried.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementSolving this issue will likely involve adding new logic or components to the codebase.zig build systemstd.Build, the build runner, `zig build` subcommand, package management

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions