docs(NODE-2329): write upgrade and migration guide

docs/CHANGES_4.0.0.md

@@ -1,39 +1,262 @@
-# Changes in 4.x
+# Changes in 4.x (and how to migrate!)
 
-WIP
+_Hello dear reader, **thank you** for adopting version 4.x of the mongodb driver, from the bottom of our developer hearts we thank you so much for taking the time to upgrade to our latest and greatest offering of a stunning database experience.
+We hope you enjoy your upgrade experience and this guide gives you all the answers you are searching for.
+If anything, and we mean anything, hinders you upgrade experience please let us know via [JIRA](https://jira.mongodb.org/browse/NODE).
+We know breaking changes are hard but they are sometimes for the best.
+Anyway, enjoy the guide, see you at the end!_
 
-## Versioned API
+## Key Changes
 
-Versioned API is a new feature in MongoDB 5.0 that allows user-selectable API versions, subsets of MongoDB server semantics, to be declared on a client. During communication with a server, clients with a declared API version will force the server to behave in a manner compatible with the API version. Declaring an API version on a client can be used to ensure consistent responses from a server, providing long term API stability for an application. The declared API version is applied to all commands run through the client, including those sent through the generic RunCommand helper. Specifying versioned API options in the command document AND declaring an API version on the client is not supported and will lead to undefined behaviour.
+### Typescript
 
-### Declare an API version on a client:
+We've migrated the driver to Typescript! Users can now harness the power of type hinting and intellisense in editors that support it to develop their MongoDB applications. Users don't even have to adopt Typescript to get the benefits. Along with the type hinting there's consistent docs formatting that editors should be able to display while developing.
+Recently we migrated our BSON library to TypeScript as well, this version of the driver pulls in those changes.
+
+#### Community Types users (@types/mongodb)
+
+If you are a user of the community types (@types/mongodb) there will likely be compilation errors while adopting the types from our codebase.
+Unfortunately we could not achieve a one to one match in types due to the details of writing the codebase in Typescript vs definitions for the user layer API along with the breaking changes of this major version. Please let us know if there's anything that is a blocker to upgrading [on JIRA](https://jira.mongodb.org/browse/NODE).
+
+### Node.js Version
+
+We now require node 12.9 or greater for version 4 of the driver.
+If that's outside your support matrix at this time, that's okay!
+Bug fix support for our 3.x branch will not be ending until summer 2022!
+
+### Cursor changes
+
+Affected classes:
+
+- `AbstractCursor`
+- `FindCursor`
+- `AggregationCursor`
+- `ChangeStreamCursor`
+  - This is the underlying cursor for `ChangeStream`
+- `ListCollectionsCursor`
+
+Our Cursor implementation has been updated to clarify what is possible before and after execution of an operation. Take this example:
+
+```javascript
+const cursor = collection.find({ a: 2.3 }).skip(1);
+for await (const doc of cursor) {
+  console.log(doc);
+  fc.limit(1); // bad.
+}
+```
+
+Prior to the this release there was inconsistency surrounding how the cursor would error if a setting like limit was applied after cursor execution had begun. Now, an error along the lines of: `Cursor is already initialized` is thrown.
+
+#### Stream API
+
+The Cursor no longer extends Readable directly, it must be transformed into a stream by calling cursor.stream(), for example:
+
+```javascript
+const cursor = collection.find({});
+const stream = cursor.stream();
+stream.on('data', data => console.log(data));
+stream.on('end', () => client.close());
+```
+
+### MongoClientOptions interface
+
+With type hinting users should find that the options passed to a MongoClient are completely enumerated and easily discoverable.
+In 3.x there were options, like `maxPoolSize`, that were only respected when `useUnifiedTopology=true` was enabled, vs `poolSize` when `useUnifiedTopology=false`.
+We've de-duped these options and put together some hefty validation to help process all options upfront to give early warnings about incompatible settings in order to help your app get up and running correctly quicker!
+
+#### Unified Topology Only
+
+We internally now only manage a Unified Topology when you connect to your MongoDB.
+The [differences are described in detail here](https://mongodb.github.io/node-mongodb-native/3.6/reference/unified-topology/).
+
+Feel free to remove the `useUnifiedTopology` and `useNewUrlParser` options at your leisure, they are no longer used by the driver.
+
+**NOTE:** With the unified topology, in order to connect to replicaSet nodes that have not been initialized you must use the new `directConnection` option.
+
+#### Authentication
+
+Specifying username and password as options is only supported in these two formats:
+
+- `new MongoClient(url, { auth: { username: '', password: '' } })`
+- `new MongoClient('mongodb://username:[email protected]')`
+
+#### Check Server Identity Inconsistency
+
+Specifying `checkServerIdentity === false` (along with enabling tls) is different from leaving it `undefined`.
+The 3.x version intercepted `checkServerIdentity: false` and turned it into a no-op function which is the required way to skip checking the server identity by nodejs.
+Setting this option to `false` is only for testing anyway as it disables essential verification to TLS.
+So it made sense for our library to directly expose the option validation from Node.js.
+If you need to test TLS connections without verifying server identity pass in `{ checkServerIdentity: () => {} }`.
+
+#### Kerberos / GSSAPI
+
+`gssapiServiceName` has been removed.
+Users should use authMechanismProperties.SERVICE_NAME like so:
+
+- In a URI query param: `?authMechanismProperties=SERVICE_NAME:alternateServiceName`
+- Or as an option: `{ authMechanismProperties: { SERVICE_NAME: 'alternateServiceName' } }`
+
+### db.collection no longer accepts a callback
+
+The only option that required the use of the callback was strict mode. Users who wish to ensure operations only execute against existing collections should use `db.listCollections` directly.
+The strict option would return an error if the collection does not exist.
+
+Another way to assert a collection's existence is using this snippet below:
 
 ```javascript
-// Declare API version "1" for the client
-client = new MongoClient(uri, { serverApi: { version: '1' } });
+const collections = (await db.listCollections({}, { nameOnly: true }).toArray()).map(
+  ({ name }) => name
+); // map to get string[]
+if (!collections.includes(myNewCollectionName)) {
+  await db.createCollection(myNewCollectionName);
+}
+```
+
+### BulkWriteError renamed to MongoBulkWriteError
+
+In 3.x we exported both the names above, we now only export `MongoBulkWriteError`.
+Users testing for `BulkWriteError`s should be sure to import the new class name `MongoBulkWriteError`.
+
+### Db no longer emits events
+
+The Db instance is no longer an EventEmitter, all events your application is concerned with can be listened to directly from the `MongoClient` instance.
 
-cursor = client.db('database').collection('coll').find(...);
+### Collection.group() removed
+
+The collection `group()` helper has been deprecated in MongoDB since 3.4 and is now removed from the Driver.
+The same functionality can be achieved using the aggregation pipeline's `$group` operator.
+
+### GridStore removed
+
+The deprecated GirdStore API has been removed from the driver.
+For more information on GridFS [see the mongodb manual](https://docs.mongodb.com/manual/core/gridfs/).
+
+Below are some snippets that represent equivalent operations:
+
+#### Construction
+
+```javascript
+// old way
+const gs = new GridStore(db, filename, mode[, options])
+// new way
+const bucket = new GridFSBucket(client.db('test')[, options])
 ```
 
-### Strict mode
+#### File seeking
 
-Declaring a `strict` API version will cause the MongoDB server to reject all commands that are not part of the declared API version. This includes command options and aggregation pipeline stages. For example, the following `find` call would fail because the `tailable` option is not part of version 1:
+Since GridFSBucket uses the Node.js Stream API you can replicate file seek-ing by using the start and end options creating a download stream from your GridFSBucket
 
 ```javascript
-// Declare API version "1" for the client, with strict on
-client = new MongoClient(uri, { serverApi: { version: '1', strict: true } });
+bucket.openDownloadStreamByName(filename, { start: 23, end: 52 });
+```
 
-// Fails with an error
-cursor = client.db('database').collection('coll').find({ ... }, { tailable: true });
+#### File Upload & File Download
+
+```javascript
+await client.connect();
+const filename = 'test.txt'; // whatever local file name you want
+const db = client.db();
+const bucket = new GridFSBucket(db);
+
+fs.createReadStream(filename)
+  .pipe(bucket.openUploadStream(filename))
+  .on('error', console.error)
+  .on('finish', () => {
+    console.log('done writing to db!');
+
+    bucket
+      .find()
+      .toArray()
+      .then(files => {
+        console.log(files);
+
+        bucket
+          .openDownloadStreamByName(filename)
+          .pipe(fs.createWriteStream('downloaded_' + filename))
+          .on('error', console.error)
+          .on('finish', () => {
+            console.log('done downloading!');
+            client.close();
+          });
+      });
+  });
 ```
 
-### Deprecation Errors
+Notably, **GridFSBucket does not need to be closed like GridStore.**
+
+#### File Deletion
 
-The `deprecationErrors` option can be used to enable command failures when using functionality that is deprecated from version 1. Note that at the time of this writing, no deprecations in version 1 exist.
+Deleting files hasn't changed much:
 
 ```javascript
-// Declare API version "1" for the client, with deprecationErrors on
-client = new MongoClient(uri, { serverApi: { version: '1', deprecationErrors: true } });
+GridStore.unlink(db, name, callback); // Old way
+bucket.delete(file_id); // New way!
+```
+
+#### Finding File Metadata
 
-// Note: since API version "1" is the initial version, there are no deprecated commands to provide as an example yet.
+File metadata that used to be accessible on the GridStore instance can be found by querying the bucket
+
+```typescript
+const fileMetaDataList: GridFSFile[] = bucket.find({}).toArray();
 ```
+
+## Intentional Breaking Changes
+
+- [`NODE-3368`](https://jira.mongodb.org/browse/NODE-3368): make name prop on error classes read-only ([#2879](https://github.com/mongodb/node-mongodb-native/pull/2879))
+- [`NODE-3291`](https://jira.mongodb.org/browse/NODE-3291): standardize error representation in the driver ([#2824](https://github.com/mongodb/node-mongodb-native/pull/2824))
+- [`NODE-3272`](https://jira.mongodb.org/browse/NODE-3272): emit correct event type when SRV Polling ([#2825](https://github.com/mongodb/node-mongodb-native/pull/2825))
+- [`NODE-2752`](https://jira.mongodb.org/browse/NODE-2752): remove strict/callback mode from Db.collection helper ([#2817](https://github.com/mongodb/node-mongodb-native/pull/2817))
+- [`NODE-1812`](https://jira.mongodb.org/browse/NODE-1812): replace returnOriginal with returnDocument option ([#2803](https://github.com/mongodb/node-mongodb-native/pull/2803))
+- [`NODE-3157`](https://jira.mongodb.org/browse/NODE-3157): update find and modify interfaces for 4.0 ([#2799](https://github.com/mongodb/node-mongodb-native/pull/2799))
+- [`NODE-2978`](https://jira.mongodb.org/browse/NODE-2978): remove deprecated bulk ops ([#2794](https://github.com/mongodb/node-mongodb-native/pull/2794))
+- [`NODE-2317`](https://jira.mongodb.org/browse/NODE-2317): remove deprecated items ([#2740](https://github.com/mongodb/node-mongodb-native/pull/2740)) ([listed below](#removed-deprecations))
+- [`NODE-2961`](https://jira.mongodb.org/browse/NODE-2961): clarify empty BulkOperation error message ([#2697](https://github.com/mongodb/node-mongodb-native/pull/2697))
+- [`NODE-1709`](https://jira.mongodb.org/browse/NODE-1709): stop emitting topology events from `Db` ([#2251](https://github.com/mongodb/node-mongodb-native/pull/2251))
+- [`NODE-2704`](https://jira.mongodb.org/browse/NODE-2704): integrate MongoOptions parser into driver ([#2680](https://github.com/mongodb/node-mongodb-native/pull/2680))
+- [`NODE-2757`](https://jira.mongodb.org/browse/NODE-2757): add collation to FindOperators ([#2679](https://github.com/mongodb/node-mongodb-native/pull/2679))
+- [`NODE-1722`](https://jira.mongodb.org/browse/NODE-1722): remove top-level write concern options ([#2642](https://github.com/mongodb/node-mongodb-native/pull/2642))
+- [`NODE-2602`](https://jira.mongodb.org/browse/NODE-2602): createIndexOp returns string, CreateIndexesOp returns array ([#2666](https://github.com/mongodb/node-mongodb-native/pull/2666))
+- [`NODE-2936`](https://jira.mongodb.org/browse/NODE-2936): conform CRUD result types to specification ([#2651](https://github.com/mongodb/node-mongodb-native/pull/2651))
+- [`NODE-1487`](https://jira.mongodb.org/browse/NODE-1487): remove deprecated Collection.group helper ([#2609](https://github.com/mongodb/node-mongodb-native/pull/2609))
+- [`NODE-2590`](https://jira.mongodb.org/browse/NODE-2590): adds async iterator for custom promises ([#2578](https://github.com/mongodb/node-mongodb-native/pull/2578))
+- [`NODE-2458`](https://jira.mongodb.org/browse/NODE-2458): format sort in cursor and in sort builder ([#2573](https://github.com/mongodb/node-mongodb-native/pull/2573))
+- [`NODE-2324`](https://jira.mongodb.org/browse/NODE-2324): remove Cursor#transformStream ([#2574](https://github.com/mongodb/node-mongodb-native/pull/2574))
+- [`NODE-2816`](https://jira.mongodb.org/browse/NODE-2816): remove deprecated find options ([#2571](https://github.com/mongodb/node-mongodb-native/pull/2571))
+- [`NODE-2820`](https://jira.mongodb.org/browse/NODE-2820): pull CursorStream out of Cursor ([#2543](https://github.com/mongodb/node-mongodb-native/pull/2543))
+- [`NODE-2320`](https://jira.mongodb.org/browse/NODE-2320): remove deprecated GridFS API ([#2290](https://github.com/mongodb/node-mongodb-native/pull/2290))
+- [`NODE-2850`](https://jira.mongodb.org/browse/NODE-2850): only store topology on MongoClient ([#2594](https://github.com/mongodb/node-mongodb-native/pull/2594))
+
+## Removed deprecations
+
+- `Collection.prototype.find / findOne options:`
+  - `fields` - use `projection` instead
+- `Collection.prototype.save` - use `insertOne` instead
+- `Collection.prototype.dropAllIndexes`
+- `Collection.prototype.ensureIndex`
+- `Collection.prototype.findAndModify` - use `findOneAndUpdate`/`findOneAndReplace` instead
+- `Collection.prototype.findAndRemove` - use `findOneAndDelete` instead
+- `Collection.prototype.parallelCollectionScan`
+- `MongoError.create`
+- `Topology.destroy`
+- `Cursor.prototype.each` - use `forEach` instead
+- `Db.prototype.eval`
+- `Db.prototype.ensureIndex`
+- `Db.prototype.profilingInfo`
+- `MongoClient.prototype.logout`
+- `MongoClient.prototype.addUser: Creating a user without roles`
+- `MongoClient.prototype.connect`
+- `Remove MongoClient.isConnected` - calling connect is a no-op if already connected
+- `Remove MongoClient.logOut`
+- `require('mongodb').instrument`
+  - Use command monitoring: `client.on('commandStarted', (ev) => {})`
+- Top-Level export no longer a function: `typeof require('mongodb') !== 'function'`
+  - Must construct a MongoClient and call `.connect()` on it.
+- No more `Symbol` export, now `BSONSymbol` which is a deprecated BSON type
+  - Users should use plain strings instead of BSONSymbol
+  - existing bson symbols in your database will be deserialized to a BSONSymbol instance
+- No more `connect` export, use `MongoClient` construction
+
+---
+
+_And that's a wrap, thanks for upgrading! You've been a great audience!_