Skip to content

Conversation

lforst
Copy link
Contributor

@lforst lforst commented Aug 9, 2024

Resolves #13057

This PR adds an integration to instrument fs API. We are using OTEL's @opentelemetry/instrumentation-fs instrumentation under the hood.

The integration creates spans with naming patterns of fs.readFile, fs.unlink, and so on.

Users can configure via option whether they want to include path arguments as attributes, and whether to include error messages as an attribute when an fs call fails.

I have found that the integration can slow down applications massively when there are a lot of fs calls, for example when running a dev server - so I put a disclaimer in the JS doc, and we should also put a similar disclaimer in the docs when we write them.

Comment on lines 35 to 44
app.get('/readFile', async (_, res) => {
await new Promise<void>(resolve => {
fs.readFile(path.join(__dirname, 'some-file.txt'), 'utf-8', () => {
resolve();
});
});
await fs.promises.readFile(path.join(__dirname, 'some-file-promises.txt'), 'utf-8');
await util.promisify(fs.readFile)(path.join(__dirname, 'some-file-promisify.txt'), 'utf-8');
res.send('done');
});

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [a file system access](1), but is not rate-limited. This route handler performs [a file system access](2), but is not rate-limited.
Comment on lines +87 to +97
app.get('/mkdtemp', async (_, res) => {
await new Promise<void>(resolve => {
fs.mkdtemp(path.join(os.tmpdir(), 'foo-'), () => {
resolve();
});
});
await fs.promises.mkdtemp(path.join(os.tmpdir(), 'foo-'));
await util.promisify(fs.mkdtemp)(path.join(os.tmpdir(), 'foo-'));

res.send('done');
});

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [a file system access](1), but is not rate-limited. This route handler performs [a file system access](2), but is not rate-limited.
@lforst lforst marked this pull request as ready for review August 9, 2024 09:46
@lforst lforst requested review from AbhiPrasad, mydea and s1gr1d August 9, 2024 09:47
@@ -0,0 +1,11 @@
some-file.txt.copy
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just make this: some-file.txt.* ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and maybe move them out of the root of the package, into e.g. fixtures/ or something like this?

Copy link
Member

@mydea mydea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

Comment on lines +26 to +33
app.get('/readFile-error', async (_, res) => {
try {
await fs.promises.readFile(path.join(__dirname, 'fixtures', 'some-file-that-doesnt-exist.txt'), 'utf-8');
} catch {
// noop
}
res.send('done');
});

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [a file system access](1), but is not rate-limited.
Comment on lines +46 to +65
app.get('/copyFile', async (_, res) => {
await new Promise<void>(resolve => {
fs.copyFile(
path.join(__dirname, 'fixtures', 'some-file.txt'),
path.join(__dirname, 'fixtures', 'some-file.txt.copy'),
() => {
resolve();
},
);
});
await fs.promises.copyFile(
path.join(__dirname, 'fixtures', 'some-file-promises.txt'),
path.join(__dirname, 'fixtures', 'some-file-promises.txt.copy'),
);
await util.promisify(fs.copyFile)(
path.join(__dirname, 'fixtures', 'some-file-promisify.txt'),
path.join(__dirname, 'fixtures', 'some-file-promisify.txt.copy'),
);
res.send('done');
});

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [a file system access](1), but is not rate-limited. This route handler performs [a file system access](2), but is not rate-limited.
Comment on lines +67 to +93
app.get('/link', async (_, res) => {
await new Promise<void>(resolve => {
fs.link(
path.join(__dirname, 'fixtures', 'some-file.txt'),
path.join(__dirname, 'fixtures', 'some-file.txt.link'),
() => {
resolve();
},
);
});
await fs.promises.link(
path.join(__dirname, 'fixtures', 'some-file-promises.txt'),
path.join(__dirname, 'fixtures', 'some-file-promises.txt.link'),
);
await util.promisify(fs.link)(
path.join(__dirname, 'fixtures', 'some-file-promisify.txt'),
path.join(__dirname, 'fixtures', 'some-file-promisify.txt.link'),
);

await Promise.all([
fs.promises.unlink(path.join(__dirname, 'fixtures', 'some-file.txt.link')),
fs.promises.unlink(path.join(__dirname, 'fixtures', 'some-file-promises.txt.link')),
fs.promises.unlink(path.join(__dirname, 'fixtures', 'some-file-promisify.txt.link')),
]);

res.send('done');
});

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [a file system access](1), but is not rate-limited. This route handler performs [a file system access](2), but is not rate-limited. This route handler performs [a file system access](3), but is not rate-limited. This route handler performs [a file system access](4), but is not rate-limited. This route handler performs [a file system access](5), but is not rate-limited.
Comment on lines +107 to +133
app.get('/symlink', async (_, res) => {
await new Promise<void>(resolve => {
fs.symlink(
path.join(__dirname, 'fixtures', 'some-file.txt'),
path.join(__dirname, 'fixtures', 'some-file.txt.symlink'),
() => {
resolve();
},
);
});
await fs.promises.symlink(
path.join(__dirname, 'fixtures', 'some-file-promises.txt'),
path.join(__dirname, 'fixtures', 'some-file-promises.txt.symlink'),
);
await util.promisify(fs.symlink)(
path.join(__dirname, 'fixtures', 'some-file-promisify.txt'),
path.join(__dirname, 'fixtures', 'some-file-promisify.txt.symlink'),
);

await Promise.all([
fs.promises.unlink(path.join(__dirname, 'fixtures', 'some-file.txt.symlink')),
fs.promises.unlink(path.join(__dirname, 'fixtures', 'some-file-promises.txt.symlink')),
fs.promises.unlink(path.join(__dirname, 'fixtures', 'some-file-promisify.txt.symlink')),
]);

res.send('done');
});

Check failure

Code scanning / CodeQL

Missing rate limiting

This route handler performs [a file system access](1), but is not rate-limited. This route handler performs [a file system access](2), but is not rate-limited. This route handler performs [a file system access](3), but is not rate-limited. This route handler performs [a file system access](4), but is not rate-limited. This route handler performs [a file system access](5), but is not rate-limited.
@lforst lforst merged commit 2a1d8ee into develop Aug 9, 2024
@lforst lforst deleted the lforst-fs-instrumentation branch August 9, 2024 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add opt-in instrumentation for fs

2 participants