docs: add a tutorial for malware detection check

docs/source/index.rst

@@ -55,9 +55,6 @@ the requirements that are currently supported by Macaron.
    * - SLSA level
      - SLSA spec v0.1
      - Concrete check
-   * - 0+
-     - **Provenance verified** - Provenance is available and verified.
-     - See :doc:`SLSA Build Levels </pages/checks/slsa_builds>`
    * - 1
      - **Scripted build** - All build steps were fully defined in a “build script”.
      - Identify and validate build script(s).
@@ -70,6 +67,9 @@ the requirements that are currently supported by Macaron.
    * - 2
      - **Build service** - All build steps are run using some build service (e.g. GitHub Actions)
      - Identify and validate the CI service(s) used for the build process.
+   * - 2+
+     - **Provenance verified** - Provenance is available and verified.
+     - See :doc:`SLSA Build Levels </pages/checks/slsa_builds>`
    * - 3
      - **Trusted builders** - Guarantees the identification of the top-level build configuration used to initiate the build. The build is verified to be hermetic, isolated, parameterless, and executed in an ephemeral environment.
      - Identify and validate that the builder used in the CI pipeline is a trusted one.
@@ -92,6 +92,19 @@ the requirements that are currently supported by Macaron.
      - **Provenance derived commit** - Check if the analysis target's commit matches the commit in the provenance.
      - If there is no commit, this check will fail.
 
+****************************************************************************************
+Macaron checks that report integrity issues but do not map to SLSA requirements directly
+****************************************************************************************
+
+.. list-table::
+   :widths: 20 40
+   :header-rows: 1
+
+   * - Check name
+     - Description
+   * - Detect malicious metadata
+     - This check analyzes the metadata of a package and reports malicious behavior. This check currently supports PyPI packages.
+
 ----------------------
 How does Macaron work?
 ----------------------

docs/source/pages/supported_technologies/index.rst

@@ -79,6 +79,9 @@ Package Registries
    * - `npm Registry <https://registry.npmjs.org>`_
      - Projects built with npm or Yarn and published on the npm registry.
      - :doc:`page </pages/supported_technologies/npm_registry>`
+   * - `Python Package Index (PyPI) <https://pypi.org/>`_
+     - Projects built with Pip or Poetry and published on the PyPI registry.
+     - :doc:`page </pages/supported_technologies/pypi_registry>`
 
 -----------
 Provenances
@@ -115,3 +118,4 @@ See also
    witness
    maven_central
    npm_registry
+   pypi_registry

docs/source/pages/supported_technologies/pypi_registry.rst

@@ -0,0 +1,6 @@
+.. Copyright (c) 2024 - 2024, Oracle and/or its affiliates. All rights reserved.
+.. Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.
+
+===========================
+Python Package Index (PyPI)
+===========================

docs/source/pages/tutorials/detect_malicious_package.rst

@@ -0,0 +1,243 @@
+.. Copyright (c) 2024 - 2024, Oracle and/or its affiliates. All rights reserved.
+.. Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.
+
+.. _detect-malicious-package:
+
+----------------------------
+Detecting malicious packages
+----------------------------
+
+In this tutorial we show how to use Macaron to find malicious packages. Imagine you’ve discovered a Python package you want to add as a dependency to your project, but you’re unsure whether you can trust its maintainers. In this case, you can run Macaron to see if it can detect any malicious behavior. Note that Macaron is an analysis tool and can either miss a malicious behavior or report a false positive.
+
+.. list-table::
+   :widths: 25
+   :header-rows: 1
+
+   * - Supported packages
+   * - Python packages (PyPI)
+
+.. contents:: :local:
+
+
+**********
+Background
+**********
+
+Detecting malicious behavior in open-source software has been a focus for the `Open Source Security Foundation <https://github.com/ossf>`_ (OpenSSF) community in recent years. One significant initiative is :term:`SLSA`, which offers practical recommendations to enhance the integrity of software packages and infrastructure. Macaron is designed to detect poorly maintained or malicious packages by implementing checks inspired by the SLSA specification. However, some forms of attacks currently fall outside the scope of SLSA version 1—notably, SLSA doesn't address the issue of malicious maintainers. Our primary goal is to make it more difficult for malicious actors to compromise critical supply chains and infrastructure. To achieve this, we're developing new methods to detect when maintainers of open source projects are untrustworthy and deliberately spreading malware.
+
+******************************
+Installation and Prerequisites
+******************************
+
+Skip this section if you already know how to install Macaron.
+
+.. toggle::
+
+    Please follow the instructions :ref:`here <installation-guide>`. In summary, you need:
+
+        * Docker
+        * the ``run_macaron.sh``  script to run the Macaron image.
+
+    .. note:: At the moment, Docker alternatives (e.g. podman) are not supported.
+
+
+    You also need to provide Macaron with a GitHub token through the ``GITHUB_TOKEN``  environment variable.
+
+    To obtain a GitHub Token:
+
+    * Go to ``GitHub settings`` → ``Developer Settings`` (at the bottom of the left side pane) → ``Personal Access Tokens`` → ``Fine-grained personal access tokens`` → ``Generate new token``. Give your token a name and an expiry period.
+    * Under ``"Repository access"``, choosing ``"Public Repositories (read-only)"`` should be good enough in most cases.
+
+    Now you should be good to run Macaron. For more details, see the documentation :ref:`here <prepare-github-token>`.
+
+***********
+Run Macaron
+***********
+
+In this tutorial, we run Macaron on the ``django`` Python package as an example with and without its dependencies to check malicious behavior and apply a policy to fail if the corresponding check fails.
+
+
+'''''''''''''''''''''''''''''''''''''
+Analyzing django without dependencies
+'''''''''''''''''''''''''''''''''''''
+
+First, we need to run the ``analyze`` command of Macaron to run a number of :ref:`checks <checks>` on the ``django`` package. In this tutorial, we are interested in the results of the ``mcn_detect_malicious_metadata_1`` check. Check :ref:`this tutorial <include_exclude_checks>` if you would like to exclude other checks.
+
+.. code-block:: shell
+
+  ./run_macaron.sh analyze -purl pkg:pypi/[email protected] --skip-deps
+
+.. note:: By default, Macaron clones the repositories and creates output files under the ``output`` directory. To understand the structure of this directory please see :ref:`Output Files Guide <output_files_guide>`.
+
+.. code-block:: shell
+
+  open output/reports/pypi/django/django.html
+
+.. _fig_django-malware-check:
+
+.. figure:: ../../_static/images/tutorial_django_5.0.6_detect_malicious_metadata_check.png
+   :alt: Check ``mcn_detect_malicious_metadata_1`` result for ``[email protected]``
+   :align: center
+
+The image above shows the result of the ``mcn_detect_malicious_metadata_1`` check for ``[email protected]``. The check has passed, which means this package is not malicious. If a package is malicious, this check fails. If the ecosystem is not supported, the check returns ``UNKNOWN``. You can also see the result of individual heuristics applied in this check under the ``Justification`` column.
+
+Now we can write a policy to ensure that all versions of ``django`` pass the ``mcn_detect_malicious_metadata_1`` check. The policy will be enforced against the output of the ``analyze`` command that is cached in the local database at ``output/macaron.db``.
+
+.. code-block:: shell
+
+  ./run_macaron.sh verify-policy --database output/macaron.db --file policy.dl
+
+Where the policy looks like below:
+
+.. code-block:: prolog
+
+  #include "prelude.dl"
+
+  Policy("check-django", component_id, "Check django artifacts.") :-
+      check_passed(component_id, "mcn_detect_malicious_metadata_1").
+
+
+  apply_policy_to("check-django", component_id) :-
+      is_component(component_id, purl),
+      match("pkg:pypi/django@.*", purl).
+
+The ``match`` constraint in this policy allows us to apply the policy on all versions of ``django``. The result of this command should show that the policy succeeds with a zero exit code (if a policy fails to pass, Macaron returns a none-zero error code):
+
+.. code-block:: javascript
+
+    passed_policies
+        ['check-django']
+    component_satisfies_policy
+        ['1', 'pkg:pypi/[email protected]', 'check-django']
+    failed_policies
+    component_violates_policy
+
+Note that the ``match`` constraint applies a regex pattern and can be expanded to ensure the ``mcn_detect_malicious_metadata_1`` check passes on all Python packages analyzed so far by Macaron:
+
+.. code-block:: prolog
+
+  apply_policy_to("check-django", component_id) :-
+      is_component(component_id, purl),
+      match("pkg:pypi.*", purl).
+
++++++++++++++++++++++++++++++++++++++++
+Verification Summary Attestation report
++++++++++++++++++++++++++++++++++++++++
+
+Additionally, Macaron generates a Verification Summary Attestation (:term:`VSA`) report that contains the policy, and information about the analyzed artifact. See :ref:`this page <vsa>` for more details. For instance, the VSA report for the ``check-django`` policy shown above can be viewed by running this command:
+
+.. toggle::
+
+  .. code-block:: shell
+
+    cat output/vsa.intoto.jsonl | jq -r '.payload' | base64 -d | jq
+
+  .. code-block:: json
+
+    {
+      "_type": "https://in-toto.io/Statement/v1",
+      "subject": [
+        {
+          "uri": "pkg:pypi/[email protected]"
+        }
+      ],
+      "predicateType": "https://slsa.dev/verification_summary/v1",
+      "predicate": {
+        "verifier": {
+          "id": "https://github.com/oracle/macaron",
+          "version": {
+            "macaron": "0.11.0"
+          }
+        },
+        "timeVerified": "2024-08-09T02:28:41.968492+00:00",
+        "resourceUri": "pkg:pypi/[email protected]",
+        "policy": {
+          "content": "  #include \"prelude.dl\"\n\n  Policy(\"check-django\", component_id, \"Check django artifacts.\") :-\n check_passed(component_id, \"mcn_detect_malicious_metadata_1\").\n\n\n  apply_policy_to(\"check-django\", component_id) :-\n is_component(component_id, purl),\n match(\"pkg:pypi/django@.*\", purl)."
+        },
+        "verificationResult": "PASSED",
+        "verifiedLevels": []
+      }
+    }
+
+.. _django_with_deps:
+
+''''''''''''''''''''''''''''''''''
+Analyzing django with dependencies
+''''''''''''''''''''''''''''''''''
+
+Macaron supports analyzing a package's dependencies and performs the same set of checks on them as it does on the main target package. To analyze the dependencies of ``[email protected]`` Python package, you can either :ref:`generate an SBOM <python-sbom>` yourself or :ref:`point Macaron to a virtual environment <python-venv-deps>` where ``django`` is installed.
+
+
+Let's assume ``/tmp/.django_venv`` is the virtual environment where ``[email protected]`` is installed. Run Macaron as follows to analyze ``django`` and its dependencies.
+
+.. code-block:: shell
+
+  ./run_macaron.sh analyze -purl pkg:pypi/[email protected] --python-venv "/tmp/.django_venv"
+
+
+By default Macaron only checks the direct dependencies. To turn on recursive dependency analysis, add the following to the ``configurations.ini`` file:
+
+.. code-block:: ini
+
+  [dependency.resolver]
+  recursive = True
+
+And pass that to the ``analyze`` command:
+
+.. code-block:: shell
+
+  ./run_macaron.sh --defaults-path configurations.ini analyze -purl pkg:pypi/[email protected] --python-venv "/tmp/.django_venv"
+
+To learn more about changing configurations see :ref:`here <change-config>`.
+
+Now we can enforce the policy below to ensure that the ``mcn_detect_malicious_metadata_1`` check always passes on ``django`` and its dependencies, indicating that none of the dependencies have malicious behavior.
+
+.. code-block:: prolog
+
+  #include "prelude.dl"
+
+  Policy("check-dependencies", component_id, "Check the dependencies of django.") :-
+    transitive_dependency(component_id, dependency),
+    check_passed(component_id, "mcn_detect_malicious_metadata_1"),
+    check_passed(dependency, "mcn_detect_malicious_metadata_1").
+
+  apply_policy_to("check-dependencies", component_id) :-
+      is_component(component_id, purl),
+      match("pkg:pypi/django@.*", purl).
+
+As you can see below, the policy passes because Macaron doesn't detect malicious behavior for ``django`` or any of its transitive dependencies.
+
+.. code-block:: javascript
+
+  passed_policies
+      ['check-dependencies']
+  component_satisfies_policy
+      ['1', 'pkg:pypi/[email protected]', 'check-dependencies']
+  failed_policies
+  component_violates_policy
+
+''''''''''''''''''''''''''''''''''''''''
+Require a confidence level in the policy
+''''''''''''''''''''''''''''''''''''''''
+
+Macaron also provides a confidence score for each check result, represented as a value ranging from ``0`` to ``1`` (inclusive). You can incorporate this score into your policy to ensure checks meet a required level of confidence. Currently, Macaron :class:`has these confidence levels <macaron.slsa_analyzer.checks.check_result.Confidence>`. For instance, you might adjust the :ref:`check-dependencies policy shown earlier <django_with_deps>` to require that the ``mcn_detect_malicious_metadata_1`` check passes with a high confidence, i.e., ``1``:
+
+.. code-block:: prolog
+
+  #include "prelude.dl"
+
+  Policy("check-dependencies", component_id, "Check the dependencies of django with high confidence.") :-
+      transitive_dependency(component_id, dependency),
+      check_passed_with_confidence(component_id, "mcn_detect_malicious_metadata_1", confidence),
+      check_passed_with_confidence(dependency, "mcn_detect_malicious_metadata_1", confidence),
+      confidence = 1.
+
+  apply_policy_to("check-dependencies", component_id) :-
+      is_component(component_id, purl),
+      match("pkg:pypi/django@.*", purl).
+
+***********
+Future Work
+***********
+
+We are actively working on the malware detection analysis check in Macaron — to improve precision, support more ecosystems, and in particular, perform more advanced source code analysis. Stay tuned and feel free to contribute to improve this check.

docs/source/pages/tutorials/exclude_include_checks.rst

@@ -1,6 +1,8 @@
 .. Copyright (c) 2024 - 2024, Oracle and/or its affiliates. All rights reserved.
 .. Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/.
 
+.. _include_exclude_checks:
+
 =====================================
 Exclude and include checks in Macaron
 =====================================

docs/source/pages/tutorials/index.rst

@@ -17,8 +17,9 @@ For the full list of supported technologies, such as CI services, registries, an
 .. toctree::
    :maxdepth: 1
 
-   detect_malicious_java_dep
    commit_finder
-   exclude_include_checks
-   generate_verification_summary_attestation
+   detect_malicious_package
    npm_provenance
+   detect_malicious_java_dep
+   generate_verification_summary_attestation
+   exclude_include_checks