You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: locale/en/docs/guides/dont-block-the-event-loop.md
+32-13Lines changed: 32 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -217,15 +217,31 @@ And particularly complicated regexps are not supported by node-re2.
217
217
If you're trying to match something "obvious", like a URL or a file path, find an example in a [regexp library](http://www.regexlib.com) or use an npm module, e.g. [ip-regex](https://www.npmjs.com/package/ip-regex).
218
218
219
219
### Blocking the Event Loop: Node core modules
220
-
Node exposes synchronous versions of several expensive APIs, including:
220
+
Several Node core modules have synchronous expensive APIs, including:
221
221
-[Encryption](https://nodejs.org/api/crypto.html)
222
222
-[Compression](https://nodejs.org/api/zlib.html)
223
-
-[File I/O](https://nodejs.org/api/fs.html)
224
-
-[Child process interaction](https://nodejs.org/api/child_process.html)
225
-
226
-
In a server, *you should not use the synchronous APIs from these modules*.
227
-
These APIs are expensive, because they involve significant computation or require I/O.
228
-
If you execute them on the Event Loop, they will take far longer to complete than a typical JavaScript instruction, blocking the Event Loop.
These APIs are expensive, because they involve significant computation (encryption, compression), require I/O (file I/O), or potentially both (child process). These APIs are intended for scripting convenience, but are not intended for use in the server context. If you execute them on the Event Loop, they will take far longer to complete than a typical JavaScript instruction, blocking the Event Loop.
227
+
228
+
In a server, *you should not use the following synchronous APIs from these modules*:
229
+
- Encryption:
230
+
-`crypto.randomBytes` (synchronous version)
231
+
-`crypto.randomFillSync`
232
+
-`crypto.pbkdf2Sync`
233
+
- You should also be careful about providing large input to the encryption and decryption routines.
234
+
- Compression:
235
+
-`zlib.inflateSync`
236
+
-`zlib.deflateSync`
237
+
- File system:
238
+
- Do not use the synchronous file system APIs. For example, if the file you access is in a [distributed file system](https://en.wikipedia.org/wiki/Clustered_file_system#Distributed_file_systems) like [NFS](https://en.wikipedia.org/wiki/Network_File_System), access times can vary widely.
239
+
- Child process:
240
+
-`child_process.spawnSync`
241
+
-`child_process.execSync`
242
+
-`child_process.execFileSync`
243
+
244
+
This list is reasonably complete as of Node v9.
229
245
230
246
### Blocking the Event Loop: JSON DOS
231
247
`JSON.parse` and `JSON.stringify` are other potentially expensive operations.
@@ -261,6 +277,10 @@ took = process.hrtime(n);
261
277
console.log('JSON.parse took '+ took);
262
278
```
263
279
280
+
There are npm modules that offer asynchronous JSON APIs. See for example:
281
+
-[JSONStream](https://www.npmjs.com/package/JSONStream), which has stream APIs.
282
+
-[Big-Friendly JSON](https://github.com/philbooth/bfj), which has stream APIs as well as asynchronous versions of the standard JSON APIs using the partitioning-on-the-Event-Loop paradigm outlined below.
283
+
264
284
### Complex calculations without blocking the Event Loop
265
285
Suppose you want to do complex calculations in JavaScript without blocking the Event Loop.
266
286
You have two options: partitioning or offloading.
@@ -350,7 +370,7 @@ If you rely on only one Worker Pool, e.g. the Node Worker Pool, then the differi
350
370
351
371
For this reason, you might wish to maintain a separate Computation Worker Pool.
352
372
353
-
#### Conclusions
373
+
#### Offloading: conclusions
354
374
For simple tasks, like iterating over the elements of an arbitrarily long array, partitioning might be a good option.
355
375
If your computation is more complex, offloading is a better approach: the communication costs, i.e. the overhead of passing serialized objects between the Event Loop and the Worker Pool, are offset by the benefit of using multiple cores.
356
376
@@ -424,24 +444,23 @@ The downside of this approach is that Workers in all of these Worker Pools will
424
444
Remember that each CPU-bound Task makes progress only while it is scheduled.
425
445
As a result, you should only consider this approach after careful analysis.
426
446
427
-
### Conclusions
447
+
### Worker Pool: conclusions
428
448
Whether you use only the Node Worker Pool or maintain separate Worker Pool(s), you should optimize the Task throughput of your Pool(s).
429
449
430
450
To do this, minimize the variation in Task times by using Task partitioning.
431
451
432
452
## The risks of npm modules
433
-
Node developers benefit tremendously from the [npm ecosystem](https://www.npmjs.com/), with hundreds of thousands of modules offering functionality to accelerate your development process.
453
+
While the Node core modules offer building blocks for a wide variety of applications, sometimes something more is needed. Node developers benefit tremendously from the [npm ecosystem](https://www.npmjs.com/), with hundreds of thousands of modules offering functionality to accelerate your development process.
434
454
435
-
However, the vast majority of these modules are written by third-party developers and undergo no formal verification process.
436
-
A developer using an npm module should be concerned about two things, though the latter is frequently forgotten.
455
+
Remember, however, that the majority of these modules are written by third-party developers and are generally released with only best-effort guarantees. A developer using an npm module should be concerned about two things, though the latter is frequently forgotten.
437
456
1. Does it honor its APIs?
438
457
2. Might its APIs block the Event Loop or a Worker?
439
458
Many modules make no effort to indicate the cost of their APIs, to the detriment of the community.
440
459
441
460
For simple APIs you can estimate the cost of the APIs; the cost of string manipulation isn't hard to fathom.
442
461
But in many cases it's unclear how much an API might cost.
443
462
444
-
*If you are calling an API that might do something expensive, ask the developers to document its cost, or examine the source code yourself (and submit a PR documenting the cost).*
463
+
*If you are calling an API that might do something expensive, double-check the cost. Ask the developers to document it, or examine the source code yourself (and submit a PR documenting the cost).*
445
464
446
465
Remember, even if the API is asynchronous, you don't know how much time it might spend on a Worker or on the Event Loop in each of its partitions.
447
466
For example, suppose in the `asyncAvg` example given above, each call to the helper function summed *half* of the numbers rather than one of them.
0 commit comments