Use fetch instead of axios

[glynnbird]

Overview

Replaces axios HTTP library with fetch, powered-by undici. The upshot of this is that we reduce the number of dependencies to zero.

Comments and advice welcome.

fetch

Some history: originally Nano was built on top of the request library which was later deprecated. At this point I reworked it to use axios instead. This PR eliminates axios and other axios-related dependencies and instead uses the new kid on the block: the fetch API.

The fetch feature has found widespread adoption in web browsers as a means of handling outbound HTTP requests. It has found its way into Node.js as a global function and is marked as an experimental feature in Node 18/19 and is mainstream in Node 20 and beyond.

Node.js's fetch capability is powered by the undici package which is bundled with Node.js and in turn uses Node's low-level network libraries instead of being based on the higher-level http/https built-in modules. It purports to be significantly faster (according to its own benchmarks) than traffic routed through http/https modules, as is the case with other HTTP libraries like axios & request.

Automated testing

Replacing axios with fetch means also ditching the nock library for mocking HTTP requests and responses, because nock works by intercepting requests originating from the http/https layer, which is bypassed by undici. Fortunately, undici provides its own Mocking tooling.

The outcome

This branch's needs no runtime dependencies. Current dependencies:

  "dependencies": {
    "http-cookie-agent": "^4.0.2",
    "@types/tough-cookie": "^4.0.2",
    "axios": "^1.1.3",
    "qs": "^6.11.0",
    "tough-cookie": "^4.1.2",
    "node-abort-controller": "^3.0.1"
  },
  "devDependencies": {
    "@types/node": "^18.11.9",
    "jest": "^29.2.2",
    "nock": "^13.2.9",
    "standard": "^17.0.0",
    "typescript": "^4.8.4"
  }

Post-PR dependencies:

  "dependencies": {
  },
  "devDependencies": {
    "undici": "^6.2.1",
    "@types/node": "^22.3.0",
    "typescript": "^5.5.4"
  }

Backwards compatibility

None of Nano's API has changed except when a user is supplying non-default connection handling parameters. Gone is requestDefaults which dates back to the "request" days and instead an optional agentOptions can be provided which is documented in the README and in TypeScript.

const agentOptions = {
  bodyTimeout: 30000,
  headersTimeout: 30000,
  keepAliveMaxTimeout: 600000,
  keepAliveTimeout: 30000,
  keepAliveTimeoutThreshold: 1000,
  maxHeaderSize: 16384,
  maxResponseSize: -1,
  pipelining: 6,
  connect: { 
    timeout: 10000
  },
  strictContentLength: true,
  connections: null,
  maxRedirections: 0
}
const undici = require('undici')
const undiciOptions = new undici.Agent(agentOptions)
const nano = Nano({ url: 'http://127.0.0.1:5984', undiciOptions })

Node versioning

As Node 16 is EOL, we can release v11 of Nano and make it the Node 18+ version. The v10 series of Nano would still be supported for the time being for older versions of node.

Testing recommendations

The test suite has been rewritten to use the built-in Node.js test runner (one fewer dependency!) and uses the undici.MockAgent to simulate responses for each of Nano's API calls, just as Nock did previously.

Run with:

npm run test

GitHub issue number

Fixes #307

Related Pull Requests

n/a

Checklist

• Code is written and works correctly;
• Changes are covered by tests;
• Documentation reflects the changes;

[glynnbird]

My simple benchmarking of small HTTP requests measures that Nano "11" would be 0 to 10% faster than Nano 10.1.

Testing method: create simple Node Express app to deliver canned responses to nano requests. Time Nano 10.1 & Nano 11 in doing different nano calls over many cycles and calculate the average response time (ms).

	Nano 10.1	Nano "11"	% speed up
/_all_dbs	0.59	0.5309	10.0
/db/id	0.573	0.543	5.2
/db/_all_docs?limit=100&include_docs=true	0.793	0.779	1.8
/db/_all_docs?limit=10000&include_docs=true	23.425	23.388	0.2

^ my reading of these findings is that the smaller the response payload, the more the efficiency gains of using undici for outbound requests shows up. For larger response bodies, the benefit is swallowed up by the physics of transferring packets over the network.

[glynnbird]

@janl

also: should we drop the callback variant?

It's pretty easy to stop the code handling callbacks, then it's just removing some tests, changing the README and migration guide. It would be a bigger breaking change ofc. what do you think?

[janl]

It would be a bigger breaking change ofc. what do you think?

we don’t have to drop callbacks in this PR, but when we release main with this PR, we should bump the major version number and when we do that, we can also drop the callback api

[glynnbird]

I went ahead and removed callbacks from the code, the Typescript definitions, the tests, the migration guide and the README. The work is in a branch here https://github.com/apache/couchdb-nano/tree/remove-callbacks (branched from the fetch branch).

We can either

1. Release the fetch branch as Nano 11 and then have a further release later on to remove callbacks
2. or release fetch and remove callbacks at the same time in Nano 11

[janl]

I’d say 2 and commit to supporting Nano 11 for another 12 months so folks have time to upgrade?

[glynnbird]

I've merged in the remove-callbacks branch, so this PR now also removes support for callbacks as well as making this library dependency free 💥

[fabiansefranek]

any updates on this?

[digitalextremist]

Seems like an exciting PR @glynnbird! As mentioned in #349 and #307, I am very interested in this one too!

Anything I can do to help move this along into nano? Or are we doing individual cherry-picked forks?

New to nano and CouchDB ... and see it has a very long turn-around time it seems. Makes me wonder if nano is more a reference implementation and all those "actually using it" are using their own fork, and I just did not do that yet?

[digitalextremist]

No conflicts! Nice work @glynnbird!

[rnewson]

Left some comments (about comments...)

+1 to the no-deps movement and the code change is mechanical in many places and you clearly understand the axios->fetch changes well and have described the breaking changes aspect.

This needs squashing before merging into one commit unless there's any meaningful split worth keeping (I don't see one fwiw)

[glynnbird]

Thanks @rnewson !

[digitalextremist]

🎊 🚢 Nice work @glynnbird; and I see that this made it into 11.0.0 an hour ago too!

Have the updated package deployed in a staging environment. I see the breaking changes, with notes on Node versions; am using Bun 1.2.21 and will be monitoring. Thankfully already using await everywhere.

Thank you!