chore: update sphinx documentation about the feature

Signed-off-by: Trong Nhan Mai <[email protected]>

Author: Trong Nhan Mai

docs/source/_static/examples/oracle-quickstart/oci-micronaut/policies/oci-micronaut-purl.dl

@@ -0,0 +1,51 @@
+/* Copyright (c) 2023 - 2023, Oracle and/or its affiliates. All rights reserved. */
+/* Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/. */
+
+
+#include "prelude.dl"
+
+/**
+ *  This is an example policy for OCI Micronaut project using PURL as the identifier.
+ *  See:
+ *      https://github.com/oracle-quickstart/oci-micronaut
+ *      https://github.com/package-url/purl-spec
+ */
+
+Policy("oci_micronaut_dependencies", parent, "") :-
+    check_passed(parent, "mcn_build_service_1"),
+    !violating_dependencies(parent, "mcn_build_service_1"), // There should not be any violating dependencies.
+    verify_provenance(dependency, "micronaut-projects/micronaut-core").
+
+// Projects that violate an expected check result.
+.decl violating_dependencies(parent: number, property: symbol)
+violating_dependencies(parent, property) :-
+    is_check(property),
+    transitive_dependency(parent, dependency), // note that since macaron by default does not traverse
+                                               // to transitive, dependencies, in most cases this is
+                                               // identical to `dependency(parent, repo)`.
+    !check_passed(dependency, property),
+    !exception_dependencies(dependency).
+
+// Exceptions for violating dependencies.
+.decl exception_dependencies(dependency: number)
+exception_dependencies(dependency) :-
+    is_repo(dependency, "github.com/mapstruct/mapstruct", _).
+
+exception_dependencies(dependency) :-
+    is_repo(dependency, "github.com/mysql/mysql-connector-j", _).
+
+exception_dependencies(dependency) :-
+    is_repo(dependency, "github.com/aws/aws-msk-iam-auth", _).
+
+exception_dependencies(dependency) :-
+    is_repo(dependency, "github.com/h2database/h2database", _).
+
+// Projects that we expect to generate a provenance.
+.decl verify_provenance(repo_num: number, repo_name: symbol)
+verify_provenance(repo_num, repo_name) :-
+    is_repo(repo_num, repo_name, _),
+    check_passed(repo_num, "mcn_provenance_level_three_1"),
+    check_passed(repo_num, "mcn_provenance_expectation_1").
+
+// Apply the policy.
+apply_policy_to("oci_micronaut_dependencies", component_id) :- is_component(component_id, "<target_software_component_purl>").

docs/source/_static/examples/oracle-quickstart/oci-micronaut/policies/oci-micronaut.dl renamed to docs/source/_static/examples/oracle-quickstart/oci-micronaut/policies/oci-micronaut-repo.dl

@@ -35,6 +35,7 @@
     "sphinx.ext.autosectionlabel",
     "sphinx_autodoc_typehints",
     "numpydoc",
+    "sphinx_tabs.tabs",
 ]
 autosectionlabel_prefix_document = True
 autosectionlabel_maxdepth = 2

docs/source/conf.py

@@ -20,7 +20,7 @@ Usage
 .. code-block:: shell
 
     usage: ./run_macaron.sh analyze
-        [-h] [-sbom SBOM_PATH] [-rp REPO_PATH] [-b BRANCH]
+        [-h] [-sbom SBOM_PATH] [-purl PURL] [-rp REPO_PATH] [-b BRANCH]
         [-d DIGEST] [-pe PROVENANCE_EXPECTATION] [-c CONFIG_PATH]
         [--skip-deps] [-g TEMPLATE_PATH]
 
@@ -36,11 +36,14 @@ Options
 
     The path to the SBOM of the analysis target.
 
+.. option:: -purl PACKAGE_URL, --package-url PACKAGE_URL
+
+    The PURL string as the unique identifier of the analysis target.
+
 .. option:: -rp REPO_PATH, --repo-path REPO_PATH
 
     The path to the repository, can be local or remote
 
-
 .. option:: -b BRANCH, --branch BRANCH
 
     The branch of the repository that we want to checkout. If not set, Macaron will use the default branch

docs/source/pages/cli_usage/action_analyze.rst

@@ -41,17 +41,35 @@ The report files of Macaron (from using the :ref:`analyze action <analyze-action
 Unique result path
 ''''''''''''''''''
 
-For each target repository, Macaron creates a directory under ``reports`` to store the report files. This directory
-path is formed from the git host name (e.g ``github.com``), the owner and the name of that
-repository. The final path is created using the following template:
+For each target software component, Macaron creates a directory under ``reports`` to store the report files. This directory
+path is formed from the PURL string of that component. The final path is created using the following template:
 
 .. code-block::
 
-    <path_to_output>/reports/<git_service_name>/<owner>/<repo_name>
+    <path_to_output>/reports/<purl_type>/<purl_namespace>/<purl_name>
 
-.. note:: The git host name has all occurrence of ``.`` in the URL replaced by ``_``.
+For more information on the three fields ``type``, ``namespace`` and ``name`` of a PURL string, please see
+`PURL Specification <https://github.com/package-url/purl-spec/blob/master/PURL-SPECIFICATION.rst>`_.
 
-For example, the reports for `<https://github.com/micronaut-projects/micronaut-core>`_ repository will be stored under
+Typically, when a repository path is provided as the main software component of the :ref:`analyze action <analyze-action-cli>`,
+a PURL is generated from the repository path, which is then later used in generating the unique report path.
+
+For example, when run this command:
+
+.. code-block::
+
+  ./run_macaron.sh analyze -rp https://github.com/micronaut-projects/micronaut-core
+
+The report files will be stored into:
+
+.. code-block::
+
+  <path_to_ouput>/reports/github_com/micronaut-projects/micronaut-core
+
+.. note:: In the unique path, only ASCII letters, digits and ``-`` are allowed. Prohibited characters are changed into
+  ``_``. No changes to the letter case are made.
+
+For example, the reports for `<https://github.com/micronaut-projects/micronaut-core>`_ will be stored under
 ``<path_to_output>/reports/github_com/micronaut-projects/micronaut-core``.
 
 ''''''''''''

docs/source/pages/output_files.rst

@@ -90,6 +90,50 @@ To analyze a repository on a self-hosted GitLab instance, you need to do the fol
 
 - Obtain a GitLab access token having at least the ``read_repository`` permission and store it into the ``MCN_SELF_HOSTED_GITLAB_TOKEN`` environment variable. For more detailed instructions, see `GitLab documentation <https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#create-a-personal-access-token>`_.
 
+''''''''''''''''''''''''''''''''''''''''''''''''''''
+Providing a PURL string instead of a repository path
+''''''''''''''''''''''''''''''''''''''''''''''''''''
+
+Instead of providing the repository path for the analysis software component, a `PURL <https://github.com/package-url/purl-spec/blob/master/PURL-SPECIFICATION.rst>`_. string
+for the target git repository could be used.
+
+To begin with, we follow the same steps for the configuration if needed (especially for self-hosted GitLab instances). The PURL string for a git repository could be created using the following format:
+
+.. code-block::
+
+  pkg:<git_service_domain>/<organization>/<name>
+
+The list bellow shows examples for the corresponding PURL string for different git repositories:
+
+.. list-table:: Example of PURL strings for git repositories.
+   :widths: 50 50
+   :header-rows: 1
+
+   * - Repository path
+     - PURL string
+   * - ``https://github.com/micronaut-projects/micronaut-core``
+     - Both ``pkg:github/micronaut-projects/micronaut-core`` and ``pkg:github.com/micronaut-projects/micronaut-core`` are applicable as ``github`` is a pre-defined type as mentioned `here <https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst>`_.
+   * - ``https://bitbucket.org/snakeyaml/snakeyaml``
+     - Both ``pkg:github/micronaut-projects/micronaut-core`` and ``pkg:github.com/micronaut-projects/micronaut-core`` are applicable as ``bitbucket`` is a pre-defined type as mentioned `here <https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst>`_.
+   * - ``https://internal.gitlab.com/foo/bar``
+     - ``pkg:internal.gitlab.com/foo/bar``
+   * - ``https://gitlab.com/gitlab-org/gitlab``
+     - ``pkg:gitlab.com/gitlab-org/gitlab``
+
+After the PURL string has been obtained, the analysis could be run with:
+
+.. code-block:: shell
+
+  ./run_macaron.sh analyze -purl <purl_string>
+
+You can also provide the PURL string together with the repository path. In this case, the PURL string will be used as the unique identifier to the analysis target:
+
+.. code-block:: shell
+
+  ./run_macaron.sh analyze -purl <purl_string> -rp <repo_path> -b <branch> -d <digest>
+
+.. note:: When provide the PURL and the repository path, both branch name and commit digest must be provided as well.
+
 -------------------------------------------------
 Verifying provenance expectations in CUE language
 -------------------------------------------------
@@ -218,8 +262,13 @@ Running the policy engine
 Macaron's policy engine accepts policies specified in `Datalog <https://en.wikipedia.org/wiki/Datalog>`_. An example policy
 can verify if a project and all its dependencies pass certain checks. We use `Soufflé <https://souffle-lang.github.io/index.html>`_
 as the Datalog engine in Macaron. Once you run the checks on a target project as described :ref:`here <analyze-action>`,
-the check results will be stored in ``macaron.db`` in the output directory. We can pass the check results to the policy
-engine and provide a Datalog policy file to be enforced by the policy engine.
+the check results will be stored in ``macaron.db`` in the output directory. Because the check results of different software component can be stored into the database at ``macaron.db``, we must specify the target software component in the Datalog policy file to be enforced by the policy engine. There are two ways that we could
+specify the target software component in the Datalog policy file:
+
+#. Using the complete name of the target component (e.g. ``github.com/oracle-quickstart/oci-micronaut``)
+#. Using the PURL string of the target component (e.g. ``pkg:github.com/oracle-quickstart/oci-micronaut@<commit_sha>``).
+
+However, the two methods would mostly share the same policy definition in the Datalog files.
 
 We use `Micronaut MuShop <https://github.com/oracle-quickstart/oci-micronaut>`_ project as a case study to show how to run the policy engine.
 Micronaut MuShop is a cloud-native microservices example for Oracle Cloud Infrastructure. When we run Macaron on the Micronaut MuShop GitHub
@@ -233,13 +282,35 @@ Now we can run the policy engine over these results and enforce a policy:
 
 .. code-block:: shell
 
-  ./run_macaron.sh verify-policy -o outputs -d outputs/macaron.db --file oci-micronaut.dl
+  ./run_macaron.sh verify-policy -o outputs -d outputs/macaron.db --file <policy_file>
+
+In this example, the Datalog policy files for both ways (as mentioned previously) are provided in `oci-micronaut-repo.dl <../_static/examples/oracle-quickstart/oci-micronaut/policies/oci-micronaut-repo.dl>`__ and `oci-micronaut-purl.dl <../_static/examples/oracle-quickstart/oci-micronaut/policies/oci-micronaut-purl.dl>`__.
+
+The differences between the two policy files can be observed below:
+
+.. tabs::
+
+  .. code-tab:: prolog Using repository complete name
+
+    apply_policy_to("oci_micronaut_dependencies", repo_id) :- is_repo(repo_id, "github.com/oracle-quickstart/oci-micronaut", _).
+
+  .. code-tab:: prolog Using PURL string
+
+    apply_policy_to("oci_micronaut_dependencies", component_id) :- is_component(component_id, "<target_software_component_purl>").
+
+The PURL string for the target software component could be obtained when the :ref:`analyze command <analyze-action>` finishes. For example:
+
+.. code:: shell
+
+  > ./run_macaron.sh analyze -rp https://github.com/oracle-quickstart/oci-micronaut
+  > ...
+  > 2023-08-15 14:36:56,672 [INFO] PURL of the main target software component is 'pkg:github.com/oracle-quickstart/oci-micronaut@3ebe0c9520a25feeae983eac6eb956de7da29ead'.
+  > 2023-08-15 14:36:56,672 [INFO] Analysis Completed!
 
-In this example, the Datalog policy file is provided in `oci-micronaut.dl <../_static/examples/oracle-quickstart/oci-micronaut/policies/oci-micronaut.dl>`__.
 This example policy can verify if the Micronaut MuShop project and all its dependencies pass the ``build_service`` check
 and the Micronaut provenance documents meets the expectation provided as a `CUE file <../_static/examples/micronaut-projects/micronaut-core/policies/micronaut-core.cue>`__.
 
-Thanks to Datalog’s expressive language model, it’s easy to add exception rules if certain dependencies do not meet a
+Thanks to Datalog's expressive language model, it's easy to add exception rules if certain dependencies do not meet a
 requirement. For example, `the Mysql Connector/J <https://slsa.dev/spec/v0.1/requirements#build-service>`_ dependency in
 the Micronaut MuShop project does not pass the ``build_service`` check, but can be manually investigated and exempted if trusted. Overall, policies expressed in Datalog can be
 enforced by Macaron as part of your CI/CD pipeline to detect regressions or unexpected behavior.

docs/source/pages/using.rst

@@ -75,6 +75,7 @@ docs = [
     "sphinx-autodoc-typehints >=1.19.4,<2.0.0",
     "sphinx-rtd-theme >=1.0.0,<2.0.0",
     "numpydoc >=1.5.0,<2.0.0",
+    "sphinx_tabs >=3.4.1,<4.0.0",
 ]
 hooks = [
     "pre-commit >=3.0.0,<3.4.0",