Skip to content

Conversation

grdsdev
Copy link
Contributor

@grdsdev grdsdev commented Oct 4, 2025

Summary

Fixes #1244 by preventing httpx.Client base_url mutation when a custom httpx client is shared across services.

Problem

When passing a custom httpx.Client via SyncClientOptions(httpx_client=...), the same client instance was shared across all Supabase services (PostgREST, Storage, Auth, Functions). Each service mutated the httpx.Client.base_url when initialized, causing subsequent requests to other services to hit the wrong API endpoints.

Symptoms:

  • 404 errors when Storage methods hit /rest/v1/object/list/... instead of /storage/v1/object/list/...
  • Cryptic KeyError: 'error' because the 404 response format differs between PostgREST and Storage APIs
  • Non-deterministic failures depending on which service was accessed most recently

Root Cause

The issue occurred in how services handle injected httpx_client:

# storage3/_sync/client.py
if http_client is not None:
    http_client.base_url = base_url  # Mutates shared client!

Timeline:

  1. Storage service initialized → httpx_client.base_url = ".../storage/v1/" (Works)
  2. PostgREST service accessed → httpx_client.base_url = ".../rest/v1/" Clobbers storage URL
  3. Storage methods called → still points to /rest/v1/ → 404 → KeyError: 'error'

Solution

Create independent copies of the httpx_client for each service using copy.copy(). This ensures each service gets its own instance with a separate base_url that can be mutated independently, while still sharing the underlying connection pool and transport configuration.

Changes

  • Modified _sync/client.py to copy httpx_client for auth, postgrest, storage, and functions services
  • Modified _async/client.py with the same fix for async clients
  • Added test_httpx_client_base_url_isolation for both sync and async clients
  • Tests verify that accessing PostgREST after Storage no longer mutates Storage's base_url

Test Plan

✅ New tests added: test_httpx_client_base_url_isolation (sync and async)
✅ All existing tests pass (34 passed, 12 xfailed, 86 xpassed)
✅ Fix verified to resolve the issue described in #1244

Additional Notes

  • The fix is minimal and uses Python's standard copy.copy() method
  • Preserves the shared connection pool benefits of httpx
  • Prevents base_url mutation conflicts between services
  • Works for both sync and async clients

🤖 Generated with Claude Code

When a custom httpx.Client was passed via ClientOptions, the same
instance was shared across all Supabase services (PostgREST, Storage,
Auth, Functions). Each service mutated the base_url when initialized,
causing subsequent requests to other services to hit the wrong API
endpoints, resulting in 404 errors and KeyError exceptions.

This fix creates independent copies of the httpx_client for each
service using copy.copy(), ensuring each service has its own instance
with a separate base_url that can be mutated independently.

Changes:
- Copy httpx_client for each service initialization (auth, postgrest,
  storage, functions) in both sync and async clients
- Add test_httpx_client_base_url_isolation for both sync and async
  clients to verify services maintain independent base_urls

Fixes #1244

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@grdsdev grdsdev requested review from silentworks and a team October 4, 2025 10:03
@coveralls
Copy link

Pull Request Test Coverage Report for Build 18242876423

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 8 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.03%) to 93.897%

Files with Coverage Reduction New Missed Lines %
src/supabase/_async/client.py 4 97.06%
src/supabase/_sync/client.py 4 97.01%
Totals Coverage Status
Change from base Build 18231163444: 0.03%
Covered Lines: 8754
Relevant Lines: 9323

💛 - Coveralls

@o-santi
Copy link
Contributor

o-santi commented Oct 6, 2025

I'm not convinced that this is the correct approach to fixing it. copy.copy is only a shallow copy, so any inner objects from httpx_client will still be shared. If we wanted to be absolutely sure that these are different objects, we'd need to do a copy.deepcopy, which not only is very slow but may be forgotten in some place and cause the error again.

I think a much better approach is to avoid modifying input objects in storage and postgrest. I will focus on that instead and report back with the results.

Copy link
Contributor

@o-santi o-santi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think copy.copy is enough to fixing. At least, we need copy.deepcopy but even then, I'm not sure this is the correct approach. See my other comment for further info.

@grdsdev
Copy link
Contributor Author

grdsdev commented Oct 6, 2025

@o-santi feel free to ditch this PR in favor of yours.

@silentworks
Copy link
Contributor

My thoughts on this is that we want to share the configuration like headers across requests but we don't want the base_url to be mutated across request (per service library use). For this instance I think it would be best to supply the base_url as part of the actual request only at request time.

I think in almost all of the libraries we have a _request method which then calls the underlying httpx request method with the method, url, json=json properties. I think at this point we could attach the base_url and it would then prevent it's mutation on the httpx_client object itself.

Below is an example of the code inside of storage-py that would get modified.

async def _request(
    self,
    method: RequestMethod,
    url: str,
    json: Optional[dict[Any, Any]] = None,
) -> Response:
    try:
        response = await self._client.request(method, f"{self.base_url}{url}", json=json)
        response.raise_for_status()
    except HTTPStatusError as exc:
        resp = exc.response.json()
        raise StorageApiError(resp["message"], resp["error"], resp["statusCode"])

    return response

What are your thoughts on this approach @o-santi?

@o-santi
Copy link
Contributor

o-santi commented Oct 6, 2025

Yep, I was basically going to do exactly that, minus the manual concatenation of the url. I will open a new PR with these changes shortly.

@grdsdev
Copy link
Contributor Author

grdsdev commented Oct 6, 2025

@o-santi I'm not sure of the Python side, but if possible lets try to not do URL manipulation using raw string, and leverage some URL type/URL library to handle it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Shared httpx.Client causes base_url clobber between PostgREST and Storage - results in 404 and KeyError
5 participants