oracle / macaron

Macaron is an extensible supply-chain security analysis framework from Oracle Labs that supports a wide range of build systems and CI/CD services. It can be used to prevent supply chain attacks or check conformance to frameworks, such as SLSA.

Macaron

Macaron is a supply chain security analysis tool from Oracle Labs, which focuses on the build integrity of an artifact and the artifact dependencies. It is based on the Supply chain Levels for Software Artifacts (SLSA) specification, which aims at preventing some of the software supply chain attacks as the systems get more complex, especially with respect to the use of open-source third-party code in applications. Attacks include stealing credentials, injecting malicious code etc., and it is critical to have security assurance on the third-party code to guarantee that the integrity of the code has not been compromised.

Macaron uses SLSA requirements specifications v0.1 to define concrete rules for protecting software integrity that can be checked for compliance requirements automatically. Macaron provides a customizable checker platform that makes it easy to define checks that depend on each other. This is particularly useful for implementing checks for SLSA levels. In addition, Macaron also checks a user-specified policy for a software component to detect unexpected behavior in the build process. We currently support the following build tools:

• Maven and Gradle Java build systems
• Pip or Poetry package managers for Python
• npm and Yarn for JavaScript
• Go
• Docker

For the full list of supported technologies, such as CI services, registries, and provenance types see this page. Macaron is a work-in-progress project. We plan to support more build systems and technologies in the future.

Table of Contents

• Getting started
• Contributing
• Defining new checks
• Publications
• Security
• License

Getting started

• To learn how to download and run Macaron, see our documentation here.
• Check out our tutorials to see how Macaron can detect software supply chain issues.
• You can also watch this demo to learn more about Macaron.

Contributing

This project welcomes contributions from the community. Before submitting a pull request, please review our contribution guide.

Defining new checks

After cloning a repository, Macaron parses the CI configuration files and bash scripts that are triggered by the CI, creates call graphs and other intermediate representations as abstractions. Using such abstractions, Macaron implements concrete checks to gather facts and metadata based on a security specification.

To learn how to define your own checks, see the steps in the checks documentation.

Publications

• Behnaz Hassanshahi, Trong Nhan Mai, Alistair Michael, Benjamin Selwyn-Smith, Sophie Bates, and Padmanabhan Krishnan: Macaron: A Logic-based Framework for Software Supply Chain Security Assurance. SCORED 2023. Best paper award 🏆

  @inproceedings{10.1145/3605770.3625213,
    author = {Hassanshahi, Behnaz and Mai, Trong Nhan and Michael, Alistair and Selwyn-Smith, Benjamin and Bates, Sophie and Krishnan, Padmanabhan},
    title = {Macaron: A Logic-Based Framework for Software Supply Chain Security Assurance},
    year = {2023},
    isbn = {9798400702631},
    publisher = {Association for Computing Machinery},
    url = {https://doi.org/10.1145/3605770.3625213},
    doi = {10.1145/3605770.3625213},
    booktitle = {Proceedings of the 2023 Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses},
    pages = {29–37},
    series = {SCORED'23}
  }

Generating SLSA provenances for Macaron itself

We have integrated SLSA provenance generation for our Docker image and release artifacts. However, due to a strict policy regarding the use of third-party GitHub Actions, we cannot generate the provenances in this repository yet until this issue is resolved.

Security

Please consult the security guide for our responsible security vulnerability disclosure process.

License

Copyright (c) 2022, 2024 Oracle and/or its affiliates. Macaron is licensed under the Universal Permissive License (UPL), Version 1.0.

macaron's Issues

Fix code scanning alert - Clear-text logging of sensitive information

Tracking issue for:

• https://github.com/oracle-samples/macaron/security/code-scanning/1

Prepare the GitHub Actions release workflows for the release

• Set the event type to push
• Refactor the workflows and clean up

Use the slsa-verifier tool as a Go dependencies instead of building the binary separately

Description/Requirements
At the moment, Macaron is using slsa-verifier to verify the content of SLSA provenances. We uses it directly out of the box, therefore, it is built from source to generate a binary which will then be invoked by Macaron. For example, this is the step where we prepare the slsa-verifier binary Makefile.

This works fine, however, we are doing an extra step to generate the binary from source (as compare to using Go to build the whole Go module automatically.) and the version of slsa-verifier is hard-coded not in the conventional go.mod. This would make it very difficult to maintain and update automatically by Dependabot.

Solution
We could address the above issue by writing a simple wrapper using slsa-verifier as a third-party dependency Go library.
In terms of licensing, we won't have any issue as slsa-verifier has been approved prior to open-sourcing.

CycloneDX Gradle plugin has non-detereminitic behavior

The CycloneDX Gradle plugin produces different BOM files in different runs, which results in failing Gradle dependency analysis integration tests.

Run CUE policies per dependency

Our current CUE policy integration does not apply policies per analysis target, which gets problematic while analyzing dependencies.

Distinguish reports by commit SHA in addition to remote URL

Currently a report is output for each repository URL analyzed, even if it is analyzed twice on different commits.

Since repositories are to be identified with both a remote url, branch and commit SHA: #33, this should be reflected in the reports.

https://github.com/oracle-samples/macaron/blob/main/src/macaron/slsa_analyzer/analyzer.py#L351

SQLite Database Errors happen during the integration tests, but does not affect the final results.

DESCRIPTION
In the integration tests, at this test case - https://github.com/oracle-samples/macaron/blob/38ddeb8c1061a9a98f36b89efcc3de51f4ab5ffa/scripts/dev_scripts/integration_tests.sh#L218

There is an SQLite Database error - https://github.com/oracle-samples/macaron/actions/runs/4474590474/jobs/7863208199#step:8:7114. It seems that Macaron was attempting to insert a dependency relationship that already exists in the database. This error does not affect the final outcome of the test.

Error: -21 01:50:33,316 [ERROR] Database error (sqlite3.IntegrityError) UNIQUE constraint failed: _dependency.dependent_repository, _dependency.dependency_repository 

[SQL: INSERT INTO _dependency (dependent_repository, dependency_repository) VALUES (?, ?)] 

[parameters: (50, 51)] 


(Background on this error at: https://sqlalche.me/e/20/gkpj)

Treat the main target as empty if it's not provided together with the SBOM.

Description
We allow the user to run the analysis for the dependencies listed in the SBOM file (provided via -sbom) but the main target software component doesn't have a repository available.
For example:

macaron analyze -sbom <path_to_sbom> -purl <no_repo_component_purl> 

Load checks from a configurable location

Right now we load the checks from a specific location. It would be good to allow users to pass the location from where additional checks can be loaded.

• Keep the current location as default.
• Allow additional locations to load checks.
• Improve the check loading process to make sure the checks are valid, but also define tables necessary for the ORM mapping.

Enable support for SBOM as an input for the Docker image

Description
This issue is for adding the support for SBOM as an input to the Docker image. This task was missed during the development of #105

Error when resolving a CycloneDx SBOM component missing both version number and external references.

PROBLEM
At this line, we are reading the value of key version in item which is a DependencyInfo (a TypedDict). However, if in the CycloneDx SBOM, a component does not have any defined version value , an AttributeError could be raised at this line.
For example, this component would be enough to cause the issue.

{
    "bomFormat": "CycloneDX",
    "specVersion": "1.4",
    "serialNumber": "...",
    "version": 1,
    "metadata": {
        "timestamp": "...",
        "component": {
            ...
    },
    "components": [
        ...
        {
            "group": "far",
            "name": "bar",
            "type": "foo",
            "bom-ref": "boo"
        },
        ...
    ],
    ...
}

Sphinx ^= 6.1.0 couldn't build our documentation

Description
In the latest version of Sphinx (from 6.1.0 to 6.1.3.), there is a breaking change that prevent the Macaron's documentation from being built successfully.
When building the documentation with make clean docs, there following error messages is seen (some part of the message is removed for visibility):

Running Sphinx v6.1.0
making output directory... done
[autosummary] generating autosummary for: "***"
building [mo]: targets for 0 po files that are out of date
writing output... 
building [html]: targets for 18 source files that are out of date
updating environment: [new config] 18 added, 0 changed, 0 removed
ERROR: [numpydoc] While processing docstring for 'macaron.code_analyzer.call_graph.Node'                                                                                                            

Extension error (numpydoc.numpydoc):
Handler <function mangle_docstrings at 0x7fbe18f79d00> for event 'autodoc-process-docstring' threw an exception (exception: Expected a class or None, but got ~Node)
make[1]: *** [Makefile:23: html] Error 2
make[1]: Leaving directory '***'
make: *** [Makefile:230: docs/_build/html/index.html] Error 2

The message indicates that the error happened for the global variable here.

Solution

• We could revert the version of Sphinx back to 6.0.0 so that the error won't happen.

Check missing license header

If a file has the correct copyright header but misses a license header, the copyright-checker.sh script, which is used as a pre-commit hook, currently doesn't check and try to add the license, e.g.,

#!/usr/bin/env bash

# Copyright (c) 2022 - 2023, Oracle and/or its affiliates. All rights reserved.

# ...

passes the pre-commit hook test.

The report files are empty when we couldn't resolve the full name for a repo

Description
When there is an error while resolving the the full_name for a repository (here , the empty full_name is stored in the AnalyzeContext instance in that repository and the analysis continue normally. This is not expected as full_name is used in multiple places during the analysis and storing the results.

Replicate
This issue is discovered when we were trying to analyze a git repository that is cloned from a git bundle. The remote url set that by repository is a local path to the original git bundle. Therefore, Macaron could not resolve the full_name of the repository from analyzing the remote URL.
Here is a sample instructions to replicate it (given that we already have a file.bundle generate from a sample repo):

git clone name.bundle # The repo will be extract to ./name
macaron -t $GITHUB_TOKEN -o output -lr $(pwd) analyze -rp name

Blocking issues:
#138

An Exception happens in a check will make the Runner not available for the rest of the analysis

Description
This issue is discovered together with issue #131 . When an exception happens, we could not properly re use the Runner which run that check. This will make the rest of the analysis failed with the following error message:

2023-03-29 12:04:07,600 [CRITICAL] Could not find any available runners. Stop the analysis...

Reproduce
We can run the analysis against a sample config where the repository mentioned in #131 (https://github.com/java-json-tools/json-patch) is one of the dependencies. For example:

target:
  id: apache/maven
  path: https://github.com/apache/maven

dependencies:
- id: exception-happen-repo
  path: https://github.com/java-json-tools/json-patch
- id: The analysis will failed to run against this dependency because the Runner is not available.
  path: https://github.com/ben-manes/caffeine.git

Refactor policy engine front end

The CLI front end for the policy engine needs to improve and be easier to use.

Support for SLSA v1.0

This issue tracks support for SLSA v1.0.

Allow disabling a check

• We need to allow a list of checks to be disabled from the CLI and handle it properly in the report.
• Add an option to disable all the default checks.

Add Python as a supported build tool

Initially, this will involve determining which scripts/services identify that a repo is a Python project, and adding these checks in a similar way to how they were done for Gradle and Maven in build_tool/.

Using undefined local variable in build_as_code_check

Description
At this line https://github.com/oracle-samples/macaron/blob/dbf572463ab56368fa763e0291ca01d58cf5201f/src/macaron/slsa_analyzer/checks/build_as_code_check.py#L199, we are using the variable deploy_cmd. However, this variable is not defined for this logic branch. This will cause an Exception.

Reproduce
The issue can be reproduced by running against this repository: https://github.com/java-json-tools/json-patch

macaron -t $GITHUB_TOKEN -o output analyze -rp https://github.com/java-json-tools/json-patch --skip-deps

Generate a Verification Summary Attestation (VSA)

Following the spec and schema described by the SLSA framework we need to generate a compatible Verification Summary Attestation (VSA) and also ingest an existing VSA for an artifact.

Refactor tests and remove unittest.TestCase subclasses

Due to limited support of unittest.TestCase subclasses in pytest, we should refactor the tests and remove the custom MacaronTestCase class to be able to use all of the features in pytest.

Deploy Macaron as a Docker image

Description/Requirements

We need to setup the Docker image for Macaron. The Docker image will be use the latest Oracle linux 9 slim - container-registry.oracle.com/os/oraclelinux:9-slim. For more information, please see h.

The required run time components in the deployed Docker are:

• Needed to be installed from source (because oraclelinux:9-slim only has up to Python3.9):
  • Python 3.11.1
• Run time libraries - Can be installed from the yum repos of oracle linux 9.
  • sqlite-devel-3.34.1-5.el9
  • git-2.31.1-2.el9.2
  • java-11-openjdk-devel-1:11.0.17.0.8-2.0.1.el9_0
• The Go packages will be compiled and copied to the Docker image during build time. Therefore, we don't need to install Go in the final Docker image.
• The maven wrapper files in resources/ will be downloaded and verified using GitHub Actions before copied to the Docker image.

Other requirements

• The output files generated by the Docker image must not have root owner as we want to make them available for the user.
• Make sure that it fits to be used in the release pipeline for Macaron (#1).
• The final Docker image must be signed (probably using Sigstore) and published to GH Container registry
  
• Use Docker multi-stage build to minimize the final Docker image size.

Accept an SBOM as the input for Macaron

Description
Up until now, Macaron only accepts the repo URL of the main analysis target (e.g. --rp https://github.com/micronaut-projects/micronaut-core) as the entrypoint. This behavior, however, comes as a limitation when we want to support running the analysis against a generated SBOM of the analysis target.

Add support for detecting python deployments that use Github Actions

When adding support for python build tools as part of the build_as_code check in #67, it was found that many projects use Github Actions to publish packages. For example, Flask and urllib3 both use: pypa/gh-action-pypi-publish.

We need to add support for this Action to detect deployment.

Add THIRD_PARTY_LICENSES.txt file to the repository

Do not update a start year in the copyright header

In the copyright-checker.sh script when a start date exists in the copyright header, we should not try to update it. This is particularly problematic when a file is moved or renamed.

Merge generated BOMs for mono repositories with multiple projects

For mono repositories that contain multiple projects, we can use the CycloneDX CLI tool to merge individual BOMs into a single BOM to be processed by Macaron.

Collapse long justifications in html report

#46 Shows all verified artifacts in the report justification, for micronaut projects this ends up more than a single page long so should be hidden and expandable by default.

Add shellcheck as a Git pre-commit hook

One option is to use shellcheck-py.

Improve integration tests

The integration tests need to be improved to log debug information more effectively. Also, some of the integration tests that take a long time, e.g., micronaut-core are analyzed with various configurations. We need to pick a different target repository that can be analyzed more quickly with similar configurations.

Remove the login steps in _build_docker.yaml

We could remove this step because the macaron-base will be public after the next release and therefore doesn't need logging in to ghcr.io to access it.

Set Python version for pyupgrade to py3.11

Looks like I forgot to change the Python version for pyupgrade pre-commit hook to --py311-plus. Make sure to run make check to allow pyupgrade rewrite all necessary files.

Error when parsing the bash script in reactor/reactor-core's GitHub Action workflow

DESCRIPTION
In one of the integration tests run (not for main and staging) - https://github.com/oracle-samples/macaron/actions/runs/4443293977/jobs/7800423867?pr=67#step:8:3644.
There is an error where our bash parser couldn't parse the bash script here - https://github.com/reactor/reactor-core/blob/9ca0484e13e8f2369fa42b595da5e6d21c5d865b/.github/workflows/publish.yml#LL131C7-L135C26.
This is the error message of Macaron for this repository:

2023-03-17 01:59:10,787 [INFO] =====================================
2023-03-17 01:59:10,788 [INFO] Analyzing io.projectreactor:reactor-core
2023-03-17 01:59:10,788 [INFO] Repo path: https://github.com/reactor/reactor-core
2023-03-17 01:59:10,788 [INFO] =====================================
2023-03-17 01:59:10,788 [INFO] Preparing the repository for the analysis (path=https://github.com/reactor/reactor-core, branch=, digest=)
2023-03-17 01:59:10,788 [INFO] The path to repo https://github.com/reactor/reactor-core is a remote path.
2023-03-17 01:59:11,171 [INFO] Cloning the repo https://github.com/reactor/reactor-core to /home/runner/work/macaron/macaron/output/git_repos/github_com/reactor/reactor-core
2023-03-17 01:59:15,431 [INFO] Stashing any uncommitted changes.
2023-03-17 01:59:15,534 [INFO] Forcefully reset the repository.
2023-03-17 01:59:15,730 [INFO] Find branch main in the remote's refs of the local repository. The branch is checked out directly.
2023-03-17 01:59:15,831 [INFO] Successfully checked out branch main.
2023-03-17 01:59:15,834 [INFO] Pulling the latest changes of branch main fast-forward only.
2023-03-17 01:59:16,139 [INFO] Successfully checked out commit 9ca0484e13e8f2369fa42b595da5e6d21c5d865b.
2023-03-17 01:59:16,143 [INFO] The full name of this repository is reactor/reactor-core
2023-03-17 01:59:16,146 [INFO] Running the analysis on branch main, commit_sha 9ca0484e13e8f2369fa42b595da5e6d21c5d865b, commit_date: 2023-03-14T16:15:35+02:00
2023-03-17 01:59:16,153 [INFO] Detect git service github for reactor/reactor-core.
2023-03-17 01:59:16,153 [INFO] Checking if the repo reactor/reactor-core uses build tool gradle
2023-03-17 01:59:16,158 [INFO] The repo uses gradle build tool.
2023-03-17 01:59:16,159 [INFO] The repo uses github_actions CI service.
Error: -17 01:59:16,176 [ERROR] Error while parsing bash script: Command '['/home/runner/work/macaron/macaron/src/macaron/bin/bashparser', '-input', 'git config --local user.name \'reactorbot\'\ngit config --local user.email \'[email protected]\'\ngit tag -m "Release milestone ${{ needs.prepare.outputs.fullVersion }}" v${{ needs.prepare.outputs.fullVersion }} ${{ github.sha }}\ngit push --tags\n']' returned non-zero exit status 2.
Error: -17 01:59:16,178 [ERROR] Error while parsing bash script: Command '['/home/runner/work/macaron/macaron/src/macaron/bin/bashparser', '-input', 'git config --local user.name \'reactorbot\'\ngit config --local user.email \'[email protected]\'\ngit tag -m "Release version ${{ needs.prepare.outputs.fullVersion }}" v${{ needs.prepare.outputs.fullVersion }} ${{ github.sha }}\ngit push --tags\n']' returned non-zero exit status 2.
2023-03-17 01:59:16,180 [CRITICAL] Could not find any available runners. Stop the analysis...

This repository is analyzed as the dependency of `https://github.com/micronaut-projects/micronaut-core, branch=3.8.x, digest=68f9bb0a78fa930865d37fca39252b9ec66e4a43

REPRODUCE
The bug could be reproduced by running Macaron with this command: macaron -t $GITHUB_TOKEN -o output analyze -rp https://github.com/reactor/reactor-core -b main -d 9ca0484e13e8f2369fa42b595da5e6d21c5d865b --skip-deps

Generating the documentation using make docs has a warning

When running make docs from the root path of the repository, this is the warning we receive:

WARNING: Failed guarded type import with ImportError("cannot import name '_RegistryType' from 'sqlalchemy.orm._typing' (.../.venv/lib/python3.11/site-packages/sqlalchemy/orm/_typing.py)")

This likely happens because of this plugin (the code to issue this warning is located here). However, more investigation is needed.

Accept provenance as input

Currently Macaron requires the URL to the target remote repo or path to the local repo as input and tries to discover a provenance automatically. As new feature, we could add a new input argument to be the provenance itself in case it's already known and discovered.

Enable Go analysis by CodeQL and Dependabot

We already analyze Python in codeql-analysis.yaml and dependabot-automerge.yaml. We need to support Go as well.

Ensure atomicity in SQLAlchemy transactions

We need to rethink the transactions in SQLAlchemy and make sure sessions are managed per analysis.

Update THIRD_PARTY_LICENSES.txt for SQLAlchemy, CUE and Souffle

This ticket documents the task to update the THIRD_PARTY_LICENSES.txt file with the new third party dependencies for the open source v0.1.0 release.

Resolve the main target repository through the SBOM itself

Description
Scenario:

• The main target software component (via -rp or --purl) is available but its repository is not available
• An SBOM is available (via -sbom)

Macaron should try to extract the information for the main software component from the SBOM itself.

Blocking issues
#157

Macaron pull the latest when analyzing a local cloned repository

Description
When analyzing a locally cloned repository, by default, if the user doesn't provide the branch name or the commit digest, Macaron will try to analyze the latest commit of the default branch. However, to analyze the latest commit, Macaron performs a git pull -f --ff-only which will fetch from the remote repository. This is not as expected as the reason we have the local repo feature is to analyze repositories that Macaron couldn't access (behind SSO, internal repos, etc.).
Note that we should only not fetch from remote for the main target but not for the dependencies.

Add support for the container provenance generator

Add support for the official slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml container provenance generator.

Update the Documentation on using `GITHUB_TOKEN` to store the Github PAT

Description
We are setting GITHUB_TOKEN as the environmental variable in our Github Action runner, which is responsible for running the tests, packaging the Python dist, etc. :

https://github.com/oracle-samples/macaron/blob/babd2e784fcea81ed6b383ad3ad992f9d6ab8cc7/.github/workflows/_build.yaml#L83

However, this is not consistent with the instructions given in README.md.

Solution
Update the instructions in README.md to be consistent.

Map a commit sha to an artifact and provenance

Macaron can analyze a specific <branch, commit sha> of a repo. However, the provenance analyzer looks for the provenance in the latest release. We need to search the releases to match the provided commit sha if possible and analyze the corresponding artifact and provenance. We can add an optional artifact version input to assist the search.

If we cannot find such mapping, we can fall back to analyze the latest release and analyze the repo with its corresponding <branch, commit sha>.

Check supported events for GitHub Actions build type

Based on the specification for GitHub Actions workflow build type, the build service check for GitHub Actions needs to check whether any of the following event types are used:

• create: Creation of a git tag or branch.
• deployment: Creation of a deployment.
• release: Creation or update of a GitHub release.
• push: Creation or update of a git tag or branch.
• workflow_dispatch: Manual trigger of a workflow.

The Dependency Resolver fails when parsing a CycloneDx component missing both version and externalReferences

Description
The error happens when we are trying to parse the dependencies from a CycloneDx SBOM. Given the following component in the components section of a CycloneDx SBOM:

{
    "group": "boo",
    "name": "foo",
    "type": "library",
    "bom-ref": "bah"
},

The above component does not have version and externalReferences.
Because it does not have version, this block will produce a DependencyInfo (a typed dictionary) instance with the following value:

{
    "version": None,
    "group": "boo",
    "name": "foo",
    "url": "",
    "note": "",
    "available": "AVAILABLE",
}

When we reach this block, item.get("version", "") will return a None value, but not an empty string. This will cause the expression item.get("version", "").lower() to have the error AttributeError: 'NoneType' object has no attribute 'lower'.

Replicate
Run Macaron with the following sample SBOM:

{
  "bomFormat" : "CycloneDX",
  "specVersion" : "1.4",
  "version" : 1,
  "metadata" : {
    "timestamp" : "2023-01-27T11:23:44Z",
    "tools" : [
      {
        "vendor" : "CycloneDX",
        "name" : "cyclonedx-gradle-plugin",
        "version" : "1.7.3"
      }
    ],
    "component" : {
      "group" : "com.amazonaws",
      "name" : "aws-lambda-java-events",
      "version" : "3.11.0",
      "purl" : "pkg:maven/com.amazonaws/[email protected]?type=jar",
      "modified" : false,
      "externalReferences" : [
        {
          "type" : "distribution",
          "url" : "https://oss.sonatype.org/service/local/staging/deploy/maven2/"
        },
        {
          "type" : "vcs",
          "url" : "https://github.com/aws/aws-lambda-java-libs.git"
        }
      ],
      "type" : "library",
      "bom-ref" : "pkg:maven/com.amazonaws/[email protected]?type=jar"
    }
  },
  "components" : [
    {
      "publisher" : "Joda.org",
      "group" : "joda-time",
      "name" : "joda-time",
      "description" : "Date and time library to replace JDK date handling",
      "purl" : "pkg:maven/joda-time/[email protected]?type=jar",
      "modified" : false,
      "type" : "library",
      "bom-ref" : "pkg:maven/joda-time/[email protected]?type=jar"
    }
  ],
  "dependencies" : [
    {
      "ref" : "pkg:maven/com.amazonaws/[email protected]?type=jar",
      "dependsOn" : [
        "pkg:maven/joda-time/[email protected]?type=jar"
      ]
    }
  ]
}

Support running the analysis with the SBOM without having to provide the repo path

Description
This ticket is created to track the enhancement for running the analysis given an SBOM.
At the moment, Macaron needs to have the git URL of the main repo or the config file as an entry point to the analysis. In some cases, we don't have the git URL to begin with or we cannot access the main target repository because it's behind authentication wall, etc.
In those cases, it's very important that we:

• Could extract the necessary information about the main target from the given SBOM without having to provide the git URL beforehand.
• If the main target cannot be accessed we reflect it in the reports and keep on analyzing the dependencies.

Blocking

• #95
• #157
• #162

Issue with the e2e configuration integration tests

The e2e configuration here provides a tag here to checkout.
If Macaron is run once with this config, and in the next run set the branch and commit to empty, it is expected to successfully checkout the latest commit in the default branch, but can't and the analysis fails.
This is the log for macaron -o output -t $GITHUB_TOKEN analyze -rp https://github.com/qos-ch/slf4j

2023-02-08 10:20:13,128 [INFO] Setting the output directory to output
2023-02-08 10:20:13,128 [INFO] The log file of Macaron will be stored in debug.log
2023-02-08 10:20:13,156 [INFO] =====================================
2023-02-08 10:20:13,156 [INFO] Analyzing https://github.com/qos-ch/slf4j
2023-02-08 10:20:13,156 [INFO] Repo path: https://github.com/qos-ch/slf4j
2023-02-08 10:20:13,156 [INFO] =====================================
2023-02-08 10:20:13,156 [INFO] Preparing the repository for the analysis (path=https://github.com/qos-ch/slf4j, branch=, digest=)
2023-02-08 10:20:13,156 [INFO] The path to repo https://github.com/qos-ch/slf4j is a remote path.
2023-02-08 10:20:14,830 [INFO] The clone dir output/git_repos/github_com/qos-ch/slf4j is not empty. No cloning is proceeded.
2023-02-08 10:20:14,849 [INFO] HEAD is a detached symbolic reference, setting main branch to empty string
2023-02-08 10:20:14,860 [INFO] Stashing any uncommitted changes.
2023-02-08 10:20:15,010 [INFO] Forcefully reset the repository.
2023-02-08 10:20:15,025 [INFO] Find branch master in the remote's refs of the local repository. The branch is checked out directly.
2023-02-08 10:20:15,104 [INFO] Successfully checked out branch master.
2023-02-08 10:20:15,107 [INFO] Pulling the latest changes of branch master fast-forward only.
2023-02-08 10:20:17,836 [ERROR] Cmd('git') failed due to: exit code(1)
  cmdline: git pull -f --ff-only
  stderr: 'fatal: couldn't find remote ref refs/heads/v_1.7.36'
2023-02-08 10:20:17,837 [ERROR] Cannot pull the latest changes for branch master.
2023-02-08 10:20:17,837 [ERROR] Cannot checkout the specific branch or commit of the target repo.
2023-02-08 10:20:17,837 [ERROR] Cannot prepare the repository for analysis.
2023-02-08 10:20:17,837 [INFO] Analysis has failed.

Allow querying results for dependencies

We need to improve querying results from the DB, JSON, and HTML reports for a dependency. For example if I want to know which dependencies have had provenances but failed l3 check, I have to check each dependency manually. The question is if we should improve the HTML report, or directly query the DB? Or would it make sense to add a SQL query CLI with some predefined queries?

Add support for repos containing submodules

We need to make a decision on how to approach analyzing repos that contain submodules.

This issue was identified after running Macaron on OpenStack.