Skip to content

Conversation

@behnazh-w
Copy link
Member

@behnazh-w behnazh-w commented Mar 20, 2024

This PR extends and makes changes to the static analysis of CI configurations in Macaron with the high-level goal of finding build and deploy commands more accurately. To achieve that, some of the abstractions had to be replaced to allow writing customized analyses, such as detecting build language setup in a GitHub Actions workflow or detecting reachable secrets. The list of extensions and changes are as follows:

  • Replaces BashCommands which consisted of build tool commands collected after analyzing GitHub Actions and passed to checks with BuildToolCommand. With this PR BuildToolCommand are created by each build tool instead, and can be detected by calling a build tool API as part of the check. This delegation of analysis to each build tool allows to customize them per build tool. Moreover, now each build tool has direct access to the parsed AST of GitHub Actions and bash scripts, which opens opportunities to add new analyses more easily.
  • The build related checks are refactored and simplified to use BuildToolCommand.
  • The callgraph analysis, which needs to be implemented for each CI service, is extended with new node types for GitHub Actions. The callgraph plays the role of Intermediate Representation and is available to all checks.
  • The mcn_build_script_1 check does not depend on any checks and always runs by default based on a customer request.
  • The mcn_build_as_code_1 check now reports deploy commands with confidence scores. The confidence scores are computed based on additional facts collected from GitHub Actions, such as the CI event, reachable secrets, and name of the workflows.
  • New analysis is added to resolve the value of expression variables, which is used for other analysis, such as reachable secrets and build language detection.
  • New abstractions are added to model third-party GitHub Actions. This feature is used to collect data about build Language setup.
  • Fixes this parsing issue: Error when parsing the bash script in reactor/reactor-core's GitHub Action workflow #107
  • Changes the check result log level to debug mode.

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Mar 20, 2024
@behnazh-w behnazh-w marked this pull request as ready for review March 20, 2024 01:22
@behnazh-w behnazh-w requested a review from tromai as a code owner March 20, 2024 01:22
@behnazh-w behnazh-w force-pushed the improve-build-checks branch from 596c233 to ed96dae Compare March 20, 2024 05:12
behnazh-w added a commit that referenced this pull request Mar 22, 2024
This PR temporarily excludes mcn_provenance_available_1 check for micronaut-core integration test because provenances have failed to publish due to an issue in [email protected].

It also excludes mcn_infer_artifact_pipeline_1, which is due to a non-deterministic behavior in deploy command detection, which will be fixed in PR #673.

Signed-off-by: behnazh-w <[email protected]>
Copy link
Contributor

@tromai tromai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I have put down all of my questions for now. Thanks for the changes!

@behnazh-w behnazh-w force-pushed the improve-build-checks branch from ed96dae to 071e99a Compare March 26, 2024 22:29
@behnazh-w behnazh-w self-assigned this Mar 26, 2024
@behnazh-w
Copy link
Member Author

behnazh-w commented Mar 26, 2024

This commit 071e99a adds node_id as an optional attribute of all callgraph nodes, makes the build tool detection deterministic by always sorting it based on the object string representation, and improves the store_inferred_provenance function.

@tromai
Copy link
Contributor

tromai commented Mar 27, 2024

This commit adds node_id as an optional attribute of all callgraph nodes, makes the build tool detection deterministic by always sorting it based on the object string representation, and improves the store_inferred_provenance function.

@behnazh-w I think the commit is not available anymore (I got 404 when I clicked on the link).

@behnazh-w
Copy link
Member Author

This commit adds node_id as an optional attribute of all callgraph nodes, makes the build tool detection deterministic by always sorting it based on the object string representation, and improves the store_inferred_provenance function.

@behnazh-w I think the commit is not available anymore (I got 404 when I clicked on the link).

Please check again.

@tromai
Copy link
Contributor

tromai commented Mar 27, 2024

I can access it again. Thanks @behnazh-w

@behnazh-w behnazh-w force-pushed the improve-build-checks branch from cbe9501 to b4506de Compare March 27, 2024 14:30
@behnazh-w behnazh-w force-pushed the improve-build-checks branch from 6a74072 to a882a62 Compare March 29, 2024 00:43
Copy link
Contributor

@tromai tromai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the changes.

@behnazh-w behnazh-w merged commit a3842b4 into staging Apr 2, 2024
@behnazh-w behnazh-w deleted the improve-build-checks branch May 15, 2024 00:53
art1f1c3R pushed a commit that referenced this pull request Nov 29, 2024
This PR temporarily excludes mcn_provenance_available_1 check for micronaut-core integration test because provenances have failed to publish due to an issue in [email protected].

It also excludes mcn_infer_artifact_pipeline_1, which is due to a non-deterministic behavior in deploy command detection, which will be fixed in PR #673.

Signed-off-by: behnazh-w <[email protected]>
art1f1c3R pushed a commit that referenced this pull request Nov 29, 2024
… commands (#673)

This PR extends and makes changes to the static analysis of CI configurations in Macaron with the high-level goal of finding build and deploy commands more accurately. To achieve that, some of the abstractions had to be replaced to allow writing customized analyses, such as detecting build language setup in a GitHub Actions workflow or detecting reachable secrets. Here are the summary of changes:

* BashCommands which consisted of build tool commands collected after analyzing GitHub Actions and passed to checks is replaced with BuildToolCommand.
* The build related checks are refactored and simplified to use BuildToolCommand.
* The callgraph analysis, which needs to be implemented for each CI service, is extended with new node types for GitHub Actions. The callgraph plays the role of Intermediate Representation and is available to all checks.
* The mcn_build_script_1 check does not depend on any checks and always runs by default based on a customer request.
* The mcn_build_as_code_1 check now reports deploy commands with confidence scores.
* New analysis is added to resolve the value of expression variables, which is used for other analysis, such as reachable secrets and build language detection.
* New abstractions are added to model third-party GitHub Actions. This feature is used to collect data about build Language setup

Signed-off-by: behnazh-w <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants