pkg(profiling-node): port profiling-node repo to monorepo #10151
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
JonasBa
force-pushed
the
jb/pkg/profiling-node
branch
from
10caf39 to
14d19b4
Compare
size-limit report 📦
|
JonasBa
force-pushed
the
jb/pkg/profiling-node
branch
from
8d98bdd to
febd09b
Compare
|
I think I spent like 1 day on the fact that running lerna run build between node-gyp configure node-gyp build somehow breaks gyp... |
|
Seems like we might be gucci on the prebuild, however fetching dep for the repo is so slow on windows it times out. Going to try and bump it |
JonasBa
commented
Jan 19, 2024
JonasBa
commented
Jan 19, 2024
JonasBa
commented
Jan 19, 2024
dev-packages/e2e-tests/test-applications/node-profiling/build.mjs
Outdated
Show resolved
Hide resolved
JonasBa
commented
Jan 19, 2024
anonrig
reviewed
Jan 19, 2024
| "build:transpile": { | ||
| "dependsOn": [ | ||
| "^build:transpile", | ||
| "^build:types" |
There was a problem hiding this comment.
Shouldn't this be included in the nx.json in root?
JonasBa
commented
Jan 19, 2024
AbhiPrasad
reviewed
Jan 22, 2024
AbhiPrasad
approved these changes
Jan 22, 2024
| - name: Extract Profiling Node Prebuilt Binaries | ||
| # @TODO: v4 breaks convenient merging of same name artifacts | ||
| # https://github.com/actions/upload-artifact/issues/478 | ||
| uses: actions/download-artifact@v3 |
There was a problem hiding this comment.
Can we use the workaround in actions/upload-artifact#478 (comment) here?
Lms24
reviewed
Jan 23, 2024
There was a problem hiding this comment.
Generally looks good to me! Still had a couple of questions regarding publishing config.
As well as two requests:
- If not done yet, can you go through
docs/new-sdk-release-checklist.mdto make sure we're not missing something? Some points might not apply, given it's a migration rather than a first time publish but it probably makes sense to check anyway. - Also, I'd recommend pushing a
release/testbranch to the repo and checking the build artifacts to make sure that everything is in the tarball as expected. From what I've seen we should be good but it probably makes sense anyway to double check.
Last question: Once we cut the first release with this package, the version will jump from 1.x to 7.x. Are you fine with that or do we need to stay on 1.x for a while? (maybe that's the reason for the separate version bump files but I'm not sure if craft/our publishing can actually handle this).
Yes, definitely. I did not know it existed, but I'll go through :)
Good idea.
Yes that is ok. I also removed node 14 from the precompile matrix so a new major change will be welcome |
|
@lms @AbhiPrasad to minimize impact on CI times, I made the change to only run the compile bindings job if profiling or nodejs package has changed or if we are on a release branch. |
AbhiPrasad
approved these changes
Jan 24, 2024
There was a problem hiding this comment.
Let's merge!
The amount of beers I owe you continue to climb @JonasBa - I'll be bankrupt at this rate 🤣
Lms24
approved these changes
Jan 24, 2024
|
@lms @AbhiPrasad let me update this branch and merge it first thing start of next week. I will also update final artifact action to v4 |
JonasBa
force-pushed
the
jb/pkg/profiling-node
branch
from
0a9387a to
8fa4055
Compare
JonasBa
force-pushed
the
jb/pkg/profiling-node
branch
from
e10e477 to
0ba4761
Compare
onurtemizkan
pushed a commit
that referenced
this pull request
Feb 4, 2024
Ports profiling-node to monorepo
This should only require CI step changes and no changes to the source
code (except inheriting from base configs in monorepo, but those were
already ported to profiling-node so they shouldn't result in any actual
changes to the codebase).
TODO:
- [x] verify pkg cmds work and we have no lint/ts issues.
- [x] verify tests pass
- [x] verify ci commands work
- [x] prebuild binaries
- [x] port build to rollup
- [x] create e2e verdaccio config
- [x] ensure e2e tests pass and the app can build.
I'm opening this as ready to review. Currently e2e test on profiling
fails as the package is missing. I'm not exactly sure why that is so
hoping I can get a helping hand on that. The condition for a successful
e2e test is to just execute a node script which triggers a profile and
attempt to bundle profiling-node. The bundler test is there to ensure
that we do in fact provide all of the statically required prebuild
binaries and ensure bundlers can resolve them - else we risk breaking
build tooling for folks bundling their code, which can be a common
optimization in serverless environments.
The good:
- Node profiling can resolve some issues around types which kept falling
out fo sync with the SDK
- We can follow the changes in JS SDK and core packages along as well as
their versioning
- The integration between the rest of the packages is tighter and safer,
as it should be impossible for the sentry-javascript packages we rely on
now to break in profiling-node.
The unfortunate:
- CI timings will increase mostly because of the performance of windows
runners. Installing repository dependencies on there takes roughly 10
minutes. I have tried leveraging the cache as much as I could, but since
our dep paths are marked as `~/path/to/cached_dep` and
`${{github.workspace}}/path/to/dep` it means they cannot be cached cross
os as paths differ. I did not change those paths in this PR, but it can
be an optimization as I suspect most of the packages are pure js and
nothing would break.
- When we pack artifacts in yarn build:tarball, we need skip
profiling-node and assemble it separately. This is because we need to
pull in the prebuilt binaries, else we'll only have the packed binary
for the os we are running the action on. This same step needs to be
replicated in e2e tests which prepare the tarball as well.
Couple of things I had to fix:
- Our build tooling was not windows compatible (util polyfill was using
awrong os path separator which wound up creating an incompatible build
output on windows)
- CI is finicky. I learned the hard way that node-gyp configure and
build need to be sequential, else all hell breaks loose and for reasons
which I didn't bother to investigate, python path and tooling is never
correctly resolved (even when specified via gyp arg)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ports profiling-node to monorepo
This should only require CI step changes and no changes to the source code (except inheriting from base configs in monorepo, but those were already ported to profiling-node so they shouldn't result in any actual changes to the codebase).
TODO:
I'm opening this as ready to review. Currently e2e test on profiling fails as the package is missing. I'm not exactly sure why that is so hoping I can get a helping hand on that. The condition for a successful e2e test is to just execute a node script which triggers a profile and attempt to bundle profiling-node. The bundler test is there to ensure that we do in fact provide all of the statically required prebuild binaries and ensure bundlers can resolve them - else we risk breaking build tooling for folks bundling their code, which can be a common optimization in serverless environments.
The good:
The unfortunate:
~/path/to/cached_depand${{github.workspace}}/path/to/depit means they cannot be cached cross os as paths differ. I did not change those paths in this PR, but it can be an optimization as I suspect most of the packages are pure js and nothing would break.Couple of things I had to fix: