in-toto / in-toto-java

A Java implementation of in-toto runlib

in-toto

in-toto provides a framework to protect the integrity of the software supply chain. It does so by verifying that each task in the chain is carried out as planned, by authorized personnel only, and that the product is not tampered with in transit.

in-toto requires a project owner to create a layout. A layout lists the sequence of steps of the software supply chain, and the functionaries authorized to perform these steps. When a functionary performs a step in-toto gathers information about the used command and the related files and stores it in a link metadata file. As a consequence link files provide the required evidence to establish a continuous chain that can be validated against the steps defined in the layout.

The layout, signed by the project owners, together with the links, signed by the designated functionaries, are released as part of the final product, and can be validated manually or via automated tooling in, e.g. a package manager.

Getting Started

Installation

in-toto is available on PyPI and can be installed via pip. See in-toto.readthedocs.io to learn about system dependencies and installation alternatives and recommendations.

pip install in-toto

Create layout, run supply chain steps and verify final product

Layout

The in-toto software supply chain layout consists of the following parts:

• expiration date
• readme (an optional description of the supply chain)
• functionary keys (public keys, used to verify link metadata signatures)
• signatures (one or more layout signatures created with the project owner key(s))
• software supply chain steps correspond to steps carried out by a functionary as part of the software supply chain. The steps defined in the layout list the functionaries who are authorized to carry out the step (by key id). Steps require a unique name to associate them (upon verification) with link metadata that is created when a functionary carries out the step using the in-toto tools. Additionally, steps must have material and product rules which define the files a step is supposed to operate on. Material and product rules are described in the section below.
• inspections define commands to be run during the verification process and can also list material and product rules.

Take a look at the demo layout creation example for further information on how to create an in-toto layout.

Artifact Rules

A software supply chain usually operates on a set of files, such as source code, executables, packages, or the like. in-toto calls these files artifacts. A material is an artifact that will be used when a step or inspection is carried out. Likewise, a product is an artifact that results from carrying out a step.

The in-toto layout provides a simple rule language to authorize or enforce the artifacts of a step and to chain them together. This adds the following guarantees for any given step or inspection:

• Only artifacts authorized by the project owner are created, modified or deleted,
• each defined creation, modification or deletion is enforced, and also
• restricted to the scope of its definition, which chains subsequent steps and inspections together.

Note that it is up to you to properly secure your supply chain, by authorizing, enforcing and chaining materials and products using any and usually multiple of the following rules:

• CREATE <pattern>
• DELETE <pattern>
• MODIFY <pattern>
• ALLOW <pattern>
• DISALLOW <pattern>
• REQUIRE <file>
• MATCH <pattern> [IN <source-path-prefix>] WITH (MATERIALS|PRODUCTS) [IN <destination-path-prefix>] FROM <step>

Rule arguments specified as <pattern> allow for Unix shell-style wildcards as implemented by Python's fnmatch.

in-toto's Artifact Rules, by default, allow artifacts to exist if they are not explicitly disallowed. As such, a DISALLOW * invocation is recommended as the final rule for most step definitions. To learn more about the different rule types, their guarantees and how they are applied, take a look at the Artifact Rules section of the in-toto specification.

Carrying out software supply chain steps

in-toto-run

in-toto-run is used to execute a step in the software supply chain. This can be anything relevant to the project such as tagging a release with git, running a test, or building a binary. The relevant step name and command are passed as arguments, along with materials, which are files required for that step's command to execute, and products which are files expected as a result of the execution of that command. These, and other relevant details pertaining to the step are stored in a link file, which is signed using the functionary's key.

If materials are not passed to the command, the link file generated just doesn't record them. Similarly, if the execution of a command via in-toto-run doesn't result in any products, they're not recorded in the link file. Any files that are modified or used in any way during the execution of the command are not recorded in the link file unless explicitly passed as artifacts. Conversely, any materials or products passed to the command are recorded in the link file even if they're not part of the execution of the command.

See this simple usage example from the demo application for more details. For a detailed list of all the command line arguments, run in-toto-run --help or look at the online documentation.

in-toto-record

in-toto-record works similar to in-toto-run but can be used for multi-part software supply chain steps, i.e. steps that are not carried out by a single command. Use in-toto-record start ... to create a preliminary link file that only records the materials, then run the commands of that step or edit files manually and finally use in-toto-record stop ... to record the products and generate the actual link metadata file. For a detailed list of all command line arguments and their usage, run in-toto-record start --help or in-toto-record stop --help, or look at the online documentation.

Release final product

In order to verify the final product with in-toto, the verifier must have access to the layout, the *.link files, and the project owner's public key(s).

Verification

Use in-toto-verify on the final product to verify that

• the layout was signed with the project owner's private key(s),
• has not expired,
• each step was performed and signed by the authorized functionary,
• the functionaries used the commands, they were supposed to use,
• materials and products of each step were in place as defined by the rules, and
• run the defined inspections

For a detailed list of all command line arguments and their usage, run in-toto-verify --help or look at the online documentation.

Signatures

in-toto-sign is a metadata signature helper tool to add, replace, and verify signatures within in-toto Link or Layout metadata, with options to:

• replace (default) or add signature(s), with layout metadata able to be signed by multiple keys at once while link metadata can only be signed by one key at a time
• write signed metadata to a specified path (if no output path is specified, layout metadata is written to the path of the input file while link metadata is written to <name>.<keyid prefix>.link)
• verify signatures

This tool serves well to re-sign test and demo data. For example, it can be used if metadata formats or signing routines change.

For a detailed list of all command line arguments and their usage, run in-toto-sign --help or look at the online documentation.

in-toto demo

You can try in-toto by running the demo application. The demo basically outlines three users viz., Alice (project owner), Bob (functionary) and Carl (functionary) and how in-toto helps to specify a project layout and verify that the layout has been followed in a correct manner.

Specification

You can read more about how in-toto works by taking a look at the specification.

Security Issues and Bugs

See SECURITY.md.

Governance and Contributing

For information about in-toto's governance and contributing guidelines, see GOVERNANCE.md and CONTRIBUTING.md.

Acknowledgments

This project is managed by Prof. Santiago Torres-Arias at Purdue University. It is worked on by many folks in academia and industry, including members of the Secure Systems Lab at NYU and the NJIT Cybersecurity Research Center.

This research was supported by the Defense Advanced Research Projects Agency (DARPA), the Air Force Research Laboratory (AFRL), and the US National Science Foundation (NSF). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of DARPA, AFRL, and NSF. The United States Government is authorized to reproduce and distribute reprints notwithstanding any copyright notice herein.

in-toto-java's Issues

use non-regex string replace in JSONEncoder for speed-up

replaceAll below compiles and uses a regex on each call. This could be sped up by pre-compiling the regex, or, even better, using a non-regex string replace function if available.

in-toto-java/src/main/java/io/github/intoto/legacy/lib/JSONEncoder.java

Lines 30 to 33 in 3ed0888

	static String canonicalizeString(String src) {
	String pattern = "([\\\\\"])";
	return String.format("\"%s\"", src.replaceAll(pattern, "\\\\$1"));
	}

This issue was initially raised by @jku in secure-systems-lab/securesystemslib#372 and fixed by @PradyumnaKrishna in secure-systems-lab/securesystemslib#410.

A similar issues exists in secure-systems-lab/go-securesystemslib#24

Update the SLSA Provenance feilds with v0.2 updates

The current in-toto-java supports v0.1 of SLSA Provenance. There are not very major changes from v0.1 to v0.2. This issue tracks the progress of the same.

Broken link (404 Page not found) in Readme.md

https://github.com/in-toto/in-toto-java/blob/master/README.md

Models in the 0.2 directory should use the proper spec?

@adityasaky Hey I just noticed that the files in 0.2 directory reference to the 0.1 spec in comments and most importantly in the pedicateType. Should these refer to the 0.2 version?

gpg-plugin runs in CI system

As it happened when re-tagging v0.2 it appears that the gpg-plugin, reuqired to push into maven central. We need tto update our test script to ignore thee gpg plugin

Non-printable canonicalization differs from reference implementation

Signatures of metadata that contain non-printable unicode literals created in in-toto-java cannot be verified using the reference implementation.

Background (json):
The JSON libraries used here in java (google/gson) and in the reference implementation (Python json) decode non-printable unicode literals as ascii byte strings and encode them in reverse fashion. This behavior is the same in both implementations.

# Python json example (same in java)
>>> import json
>>> json.loads('"\\u001b"')
'\x1b'
>>> json.dumps('\x1b')
'"\\u001b"'

Issue (different canonicalization):
The issue is that for canonicalization (in order to create/verify signatures) in-toto-java mostly uses the standard json encoder. The reference implementation, on the other hand, uses custom canonicalization, which does not turn the non-printable back into a unicode literal but instead keeps the ascii byte string.

Possible solutions:
(a) Fully port the reference implementation's canonicalization routine to java (like in-toto-golang does)
(b) Use a canonicalization specification with broader support in all implementations (see secure-systems-lab/securesystemslib#159)

Support in-toto final product verification

Currently in-toto-java may be used by functionaries to generate and sign in-toto link metadata, as evidence for steps in the software supply chain.

This is a feature request to support full final product verification as described in section "5.2 Verifying the final product" of the in-toto specification.

See verifylib.in_toto_verify in the reference implementation and verifylib.InTotoVerify in the golang implementation for two fully compliant examples.

Use a single JSON library

At the moment both Jackson and GSON are configured as dependencies. It would be better if there was only one dependency for processing JSON. Hopeful Jackson will be the chosen one.

in-toto-java/pom.xml

Lines 57 to 74 in 73afcb4

	<!-- Jackson Dependencies -->
	<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
	<artifactId>jackson-databind</artifactId>
	<version>2.14.1</version>
	</dependency>
	<dependency>
	<groupId>com.fasterxml.jackson.datatype</groupId>
	<artifactId>jackson-datatype-jsr310</artifactId>
	<version>2.14.1</version>
	</dependency>
	<!-- gson: object to json converter -->
	<dependency>
	<groupId>com.google.code.gson</groupId>
	<artifactId>gson</artifactId>
	<version>2.10.1</version>
	<scope>compile</scope>
	</dependency>

A small typo in Readme.md

The line be "The keys in the project where generated with:" should be "The keys in the project were generated with:"

Proposal: Updated implementation for the In-toto spec

cc @SantiagoTorres @TomHennen

Hello! :)

As described in this request we want to contribute a new version of the Java client. We would like to implement something like the following:

The above diagram contains the models used by the Java library to represent the various entities required from the different specs (in-toto, SLSA, DSSE) to generate build provenance.

A few notes on the proposed classes:

• The Recipe entity has additional properties: arguments and environment that are classified as generic objects in the spec. For this reason it would be expected that builders using this library would extend from the default Recipe and create their own custom class.
• The entities are contained within packages with naming that matches the origin of the spec, i.e. Provenance from SLSA, Envelope from DSEE, etc. It would be expected that the implementation for the different specs resides in the corresponding GitHub project.

Proposed changes for: io.github.intoto

Structural changes

• Create a new branch to contain the new version for the proposal
• Add a legacy package that contains the existing implementation as is.
• Clean up and update the pom.xml

API changes

• Remove signing aspect in the library
• Create helper classes to convert to JSON

General changes

• Replace gson with Jackson and annotations
• Only use lowercase letter with no underscores in package names
• Do not have multiple public nested classes

Currently the library has a POJO (the Link) that not only holds information about the entity it represents but also is able to transform it self to JSON and sign it self.

A proposed change would be for serialization to JSON to happen in a helper class using Jackson. Something like:

Statement myStatement = new Statement()
    .setSubject(…)
    .setPedicateType(…)
    .setPredicate(…);
String jsonString = IntotoHelper.toJson(myStatement);

For this to happen, the new POJOs will have Jackson annotations where required. The IntotoHelper class would instantiate Jackson and configure it so that it outputs the correct JSON format.
Currently POJOs that want to be signable extend from Metablock. Metablock exposes the following method:

sign(Key privateKey)

The proposed change is to have an external class (probably an implementation of an interface in the DSSE repo) perform the signing. Allowing users to either provide a key to sign or sign the payloads using their own mechanism (i.e Cloud KMS).

Proposed changes in: io.github.dsse

Create a new java library that contains Envelope, Signature and helper classes to implement signing. This library should have the implementations to sign with a given Key (similar to the current implementation in the Java lib) but also allow the capability for users to define their own Signers that would allow the dsse library to correctly create the Envelopes.

Proposed changes in: io.github.slsa

Create a new Java library that contains implementation of the Provenance spec. This would contain the models to represent a Provenance as well as helper classes to serialize this into JSON.

Questions

• How and where to run CI/CD tests? Github actions? Cloud Build?
• Should we target JDK 11 or JDK 17?
• Please let me know if something does not make sense, if I made a mistake or would like to see a different implementation.

Update Maven setup

Found the following banner while working on #100

[INFO] --- surefire:2.22.2:test (default-test) @ in-toto ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------

 +-------------------------------------------------------------------------------+
 | WARNING:                                                                      |
 | The junit-platform-surefire-provider has been deprecated and is scheduled to  |
 | be removed in JUnit Platform 1.4. Please use the built-in support in Maven    |
 | Surefire >= 2.22.0 instead.                                                   |
 | » https://junit.org/junit5/docs/current/user-guide/#running-tests-build-maven |
 +-------------------------------------------------------------------------------+

Caused by

in-toto-java/pom.xml

Lines 130 to 145 in c7f5b1a

	<plugin>
	<artifactId>maven-surefire-plugin</artifactId>
	<version>2.22.2</version>
	<dependencies>
	<dependency>
	<groupId>org.junit.platform</groupId>
	<artifactId>junit-platform-surefire-provider</artifactId>
	<version>1.3.2</version>
	</dependency>
	<dependency>
	<groupId>org.junit.jupiter</groupId>
	<artifactId>junit-jupiter-engine</artifactId>
	<version>5.9.2</version>
	</dependency>
	</dependencies>
	</plugin>

Access Denied error when running "mvn test"

I'll get follwoing error when runnig "mvn test" in travis-ci, which says access was denied:

[ERROR] Plugin org.apache.maven.plugins:maven-resources-plugin:2.6 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-resources-plugin:jar:2.6: Could not transfer artifact org.apache.maven.plugins:maven-resources-plugin:pom:2.6 from/to central (https://repo.maven.apache.org/maven2): Access denied to: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom , ReasonPhrase:Forbidden. -> [Help 1]

I temporarily use "sudo: true" in .travis.yml configuration file at 72b39c1 as a workaround. But this bug needs to be fixed in future.

in-toto-java/.travis.yml

Line 2 in 72b39c1

sudo: true