see screenshot below

@jossef @cx-monicac the idea here is to support this output to make it easier to ingest by security tools, one of those is CxOne

I tested with the parameter set and it appears that 2ms is still scanning all confluence spaces (see screenshot below)

arguments

• --stdout-format - output format of the stdout (default yaml)
  • yaml - as of today
  • sarif - as of #65
• --report-path - an array of file paths to output the report. The file's extension sets the format. Supported extensions:
  • .sarif - as of #65
  • .yaml - as of what we print today to stdout

Update supporting services, how to use, how to contribute.

optional, when specified, instead of cli args, values of args should be loaded from a config file
I give you the freedom to decide the file format (json, yaml, properties, ...)
and its structure

Description:
The Checkmarx 2MS tool is a powerful secret leakage detection tool that helps developers identify sensitive data and other secrets that may have been unintentionally leaked within their code repositories. To integrate this tool effectively into the development workflow, we need to add support for GitHub Actions to Checkmarx 2MS.

Technical Details:
To add GitHub Actions support to Checkmarx 2MS, we will create a custom action that can be used within GitHub workflows. This action will leverage the Checkmarx 2MS tool to scan a specified code repository for potential secret leakage issues and provide detailed results to the user. The action should be configurable, allowing users to specify the repository to scan, the API key to use for authentication, and any other relevant options.

Once the custom action is created, we can add it to the GitHub Marketplace, making it easily accessible for users. Additionally, we will provide documentation on how to integrate this action into existing workflows and best practices for using the Checkmarx 2MS tool for secret leakage detection within the GitHub ecosystem.

when provided, the value is expected to be a regular expression.
if provided and valid input, this regex should be converted to a rule and included in the rule set.

usage

./2ms --regex "((25[0-5]|(2[0-4]|1\d|[1-9]|)\d)\.?\b){4}"

After #34, we want to CD to DockerHub: https://hub.docker.com/r/checkmarx/2ms.

The release shouldn't be for each commit, but we should build a process of manually (by label or by workflow_dispatch) releasing, to include more content in each release, and to avoid many small releases.

repository.go is quite straight-forward implementation.

suggesting to use Gitleaks engine to scan historical commits if they contain secrets

With the demand for GPT related keys we should add support to detect those as well.

This means developing the detection and contributing it to gitleaks, so they'll add it to their big list of key already detected. See https://github.com/gitleaks/gitleaks/tree/master/cmd/generate/config/rules

Please talk to me before submitting the PR to them.

Reference (in Hebrew) about GPT stolen keys at https://www.geektime.co.il/gpt-4-pirates-are-rampant/

CC @Alexaro1cx

2ms currently only has flags for the scan. We want to create commands to support different operations.
Reference to the concept can be seen at https://kubernetes.io/docs/reference/kubectl/ and https://docs.kics.io/latest/commands/

list - list plugins / engines / rules
scan - (default) scan for secrets
help - show cli help (same as -h today)

global flags
-v version
-h help

scan options
--log-level (same as today)
--all-plugins (default)
--plugin-include string, comma separated (add to plugins list, starts from empty)
--plugin-exclude string, comma separated (remove from all plugin list, starts from full)
--all-rules (default)
--rule-include string, comma separated
--rule-exclude string, comma separated
--config (file path)

list sub commands
plugins - list plugins
rules - list rules
config - list configs detected (env variables / files)

Notice each plugins brings its own flags to the scan command
e.g. confluence has
--confluence-url (mandatory)
--confluence-username (optional, for private instances)
--confluence-token(optional, for private instances)
--confluence-spaces (comma separated list, or all if not present)

if time allows:
scan sub commands
(plugin name) - scan with just one plugin (e.g. 2ms scan confluence)
confluence

Description:
The goal of this issue is to containerize the 2MS tool for easy deployment and management in any infrastructure. This will involve generating a Dockerfile for the project and adding it to Docker Hub.

Technical Details:
We will be following the best practices recommended by KICS to ensure that the containerized 2MS tool is secure and efficient.

2ms should support scanning of bitbucket (and similarly structured git based) repositories

Description

When using 2ms filesystem --path somedir/ on a non empty dir, sometimes there's a crash (possible data-race).

Steps to reproduce

1. Create a directory mkdir testdir
2. Create a file in the dir echo "someinfo" > testdir/testfile.txt
3. Run 2ms filesystem --path testdir

Expected Behavior

Expecting the command to succeed and report no secrets.

Current Behavior

Sometimes ( around 1 / 3 times ) the program panics with this error stack:

panic: sync: WaitGroup is reused before previous Wait has returned

goroutine 1 [running]:
sync.(*WaitGroup).Wait(0x0?)
	/usr/local/go/src/sync/waitgroup.go:118 +0x7f
github.com/checkmarx/2ms/cmd.postRun(0xc00012a600?, {0xa79f14?, 0x2?, 0x2?})
	/home/<redacted>/Downloads/2ms/cmd/main.go:171 +0x3a
github.com/spf13/cobra.(*Command).execute(0xc00012a600, {0xc0001242c0, 0x2, 0x2})
	/home/<redacted>/go/pkg/mod/github.com/spf13/[email protected]/command.go:936 +0xa2b
github.com/spf13/cobra.(*Command).ExecuteC(0xef3ce0)
	/home/<redacted>/go/pkg/mod/github.com/spf13/[email protected]/command.go:1044 +0x3bd
github.com/spf13/cobra.(*Command).Execute(...)
	/home/<redacted>/go/pkg/mod/github.com/spf13/[email protected]/command.go:968
github.com/checkmarx/2ms/cmd.Execute()
	/home/<redacted>/Downloads/2ms/cmd/main.go:107 +0x5e5
main.main()
	/home/<redacted>/Downloads/2ms/main.go:22 +0x1fa
exit status 2

I removed my information from the stack trace and replaced with

The idea is a directory in the project, combined with .go files, and each is like a "plugin"
for ease of maintenance and future contribution

Couln't understand why but without the sleep we can't get all elements.

Maybe it would make sense to spend some time investigating if it's realated to issues on goroutines management or it's related to some kind of limitations.

Originally posted by @joaopedrocsilva in #52 (comment)

use gitleaks with common config
suggest args to expose for user

similar to what you can do in .gitignore but as optional args

add a number argument that is going to limit the historical scans when using the git plugin

Today we run the full list of rules as defined at secrets/secrets.go#L111

Add the following sub-commands to the list command to all the user know which rules are available:
rules - list rules

Add the following flags for the scan command to allow the user to control which rules are being used:

--all-rules (default)
--rule-include string, comma separated (adds to list, starts from empty)
--rule-exclude string, comma separated (remove from list, starts from full list)

• history revisions
• multiple sub-pages

Only if appear on the same file - when results corresponding to the same page they should appear once with all the versions where the secret is present.

In #81, I added a validation pipeline to make sure the PR title is following the semantic-release conventions and will bump the SemVer.

But semantic-release is using a small subset of Angular Conventional Commit, and I need to research it to find how to control both commitlint and semantic-release in a simple way.

semantic-release/semantic-release#2820

Running 2ms without parameters gives this result:

$ ./2ms
Summary:
- Total items scanned: 0
- Total items with secrets: 0
Detailed Report:

Should the confluence switches be mandatory if the confluence plugin is enabled/running?
Or at least give a warning 2ms is missing some info for a confluence scan.

to receive and read cli arguments

Implement the Paligo scanner from inside Checkmarx and run it in parallel with the current implementation in Checkmarx to compare the results and replace it.

We need to create a config file for golangci-lint and add these linters:

• goimports

Refactor the return type of getItems func. Replace the entire json structure for the body.value field (the valid one).

Get contents for older versions, also consider to add an option to scan it or no

Steps to reproduce

running the command: ./2ms repository --path "<file_to_scan>"
where <file_to_scan> is this file.

Expected result

2ms finds a hardcoded secret at line 8

Actual result

See here what the condition should be to identify if the PR is from a fork or not.

With your PR, update the pull_request_template.md and remove the comment about adding the label manually.

I've tried

--confluence https://checkmarx.atlassian.net/wiki/

the actual value passed: https//checkmarx.atlassian.net/wiki/

params - discord token?
TBD

add options to filter out specific secrets results by their id (in case known issues or false positives)

Provide an example call with flags set when printing help -h / --help

./2ms --confluence [https://<MyConfluence>.atlassian.net/wiki](https://%3Cmyconfluence%3E.atlassian.net/wiki) --confluence-spaces <MySpace> --confluence-username <MyUsername> --confluence-token <MyToken>

this should be generic as plugins will be added