oxsecurity / megalinter Goto Github PK

Is your feature request related to a problem? Please describe.

I ran mega-linter-runner on a local repository, which uses Terraform. Terraform clones dependencies into a .terraform/ directory (just like node_modules/ for npm). Our .gitignore ignores this with **/.terraform/, so I expected mega-linter-runner will understand that I don't want to lint this directory. However, I am getting results for various files within, such as

In /tmp/lint/deploy/terraform/dev/.terraform/modules/deployment.eks/examples/launch_templates/pre_userdata.sh line 1:

Describe the solution you'd like

Read exclude patterns from .gitignore and append them to all the other user-defined excludes.

Describe alternatives you've considered

N/A

Additional context

N/A

Describe the bug
Running the the python 'flavor' image errors out instead of displaying the final SUMMARY table.
I was able to reproduce the error messages / symptom when running from both 'local' mega-linter-runner and also in my GitLab CI pipeline.
I was also able to reproduce this error with the 'documentation' flavor. The error does not show when running the full mega-linter image.

To Reproduce

1. $> npx mega-linter-runner -p ~/repo/data/ci --flavor python
2. 3. See error

Expected behavior
mega-linter runs and display 'SUMMARY' table without errors

Error Output
[Text Reporter] Generated TEXT report: /builds/data/ci/report/linters_logs/SUCCESS-YAML_V8R.log
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/megalinter/MegaLinter.py", line 27, in run_linters
linter.run()
File "/megalinter/Linter.py", line 419, in run
return_code, stdout = self.process_linter(file)
File "/megalinter/Linter.py", line 528, in process_linter
return_code, return_output = self.execute_lint_command(command)
File "/megalinter/Linter.py", line 564, in execute_lint_command
process = subprocess.run(
File "/usr/local/lib/python3.8/subprocess.py", line 489, in run
with Popen(*popenargs, **kwargs) as process:
File "/usr/local/lib/python3.8/subprocess.py", line 854, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/usr/local/lib/python3.8/subprocess.py", line 1702, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'shfmt'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/megalinter/run.py", line 15, in
linter.run()
File "/megalinter/MegaLinter.py", line 125, in run
self.process_linters_parallel(active_linters, linters_do_fixes)
File "/megalinter/MegaLinter.py", line 208, in process_linters_parallel
updated_linters = pool_result.get()
File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 771, in get
raise self._value
FileNotFoundError: [Errno 2] No such file or directory: 'shfmt'

Additional context
macOS 11.1
zsh in iTerm2

https://pkgs.alpinelinux.org/package/edge/community/x86_64/cppcheck

http://cppcheck.sourceforge.net/

1. Maybe you would like to experiment with a command line setting --cache-from for docker build.
   https://docs.semaphoreci.com/ci-cd-environment/docker-layer-caching/#about-cache-from

It allows CI to reuse image layers from previous builds even if local build agent image cache does not have them. I'm not sure. Maybe this is not a real issue due to good docker cache in GitHub actions. Or because all layers are updated and regenerated too frequently.

If we reuse layers more - it will take less time for end users to download them. And it will take less disk space.

Please note I did not use this command line option before. I'm just going to try it in CI on my current project.

2. I would also recommend to reorder blocks inside of Dockerfile so that dependencies which are changed more frequently and moved later.

Example

$ git diff --check
Dockerfile:46: leftover conflict marker

Describe the bug
Setting the config value for the *_DISABLE_ERRORS_IF_LESS_THAN throws an error:

ValueError: invalid literal for int() with base 10: 'YAML_V8R_DISABLE_ERRORS_IF_LESS_THAN'

To Reproduce
Steps to reproduce the behavior:

1. Create a YAML file with a single schema error in it
2. Set YAML_V8R_DISABLE_ERRORS_IF_LESS_THAN: 1 in .mega-linter.yml
3. Run mega-linter-runner

Expected behavior
Single error is ignored

Actual Behavior
Error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/megalinter/run.py", line 9, in <module>
    linter = megalinter.Megalinter({"cli": True})
  File "/megalinter/MegaLinter.py", line 94, in __init__
    self.load_linters()
  File "/megalinter/MegaLinter.py", line 336, in load_linters
    all_linters = linter_factory.list_all_linters(linter_init_params)
  File "/megalinter/linter_factory.py", line 30, in list_all_linters
    descriptor_linters = build_descriptor_linters(
  File "/megalinter/linter_factory.py", line 92, in build_descriptor_linters
    linter_instance = linter_class(linter_init_params, instance_attributes)
  File "/megalinter/Linter.py", line 213, in __init__
    self.load_config_vars()
  File "/megalinter/Linter.py", line 387, in load_config_vars
    self.disable_errors_if_less_than = int(
ValueError: invalid literal for int() with base 10: 'YAML_V8R_DISABLE_ERRORS_IF_LESS_THAN'

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
I tried this with other linters and ran into the same issue, so looks like a common problem because I believe this is a common property

sfdx-scanner uses PMD which relies on JAVA, so java must be included in salesforce descriptor

Use regex to get the number of errors in logs, when the linter is called in "project" mode

The current one is so ugly (built win paint.net in 5mn :D )

Only constraint: have the llama included 🍡

Describe the bug
The current v4 image is displaying python errors instead of outputting the expected table of results

To Reproduce

1. Setup mega-linter runner locally via these instructions
2. run npx mega-linter-runner

• for what it may be worth, I'm getting the same error from the 'latest' / insiders image, and the few flavors I've tried

3. See error output

Expected behavior
mega-linter completes successfully, including showing the results table, similar to the example shown at https://nvuillam.github.io/mega-linter/mega-linter-runner/

Error Output

[Text Reporter] Generated TEXT report: /tmp/lint/report/linters_logs/SUCCESS-JSON_V8R.log
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/megalinter/MegaLinter.py", line 27, in run_linters
    linter.run()
  File "/megalinter/Linter.py", line 451, in run
    reporter.produce_report()
  File "/megalinter/reporters/TextReporter.py", line 76, in produce_report
    text_report_lines += self.master.complete_text_reporter_report(self)
  File "/megalinter/linters/CSpellLinter.py", line 64, in complete_text_reporter_report
    data = json.load(json_file)
  File "/usr/local/lib/python3.8/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/local/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 33 column 5 (char 634)
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/megalinter/run.py", line 15, in <module>
    linter.run()
  File "/megalinter/MegaLinter.py", line 125, in run
    self.process_linters_parallel(active_linters, linters_do_fixes)
  File "/megalinter/MegaLinter.py", line 208, in process_linters_parallel
    updated_linters = pool_result.get()
  File "/usr/local/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
json.decoder.JSONDecodeError: Expecting value: line 33 column 5 (char 634)

That way, contributors will know that they don't have to modify both CHANGELOG files, for example

• update how-to-contribute instructions

Ups, wrong one :|

Is your feature request related to a problem? Please describe.
I have to disable the Vue.js plugin for eslint to be able to use mega-linter.

Describe the solution you'd like
It would be great if the plugin were supported.

Describe alternatives you've considered
A separate action to lint my .vue files.

Additional context
There is of course a chance this is user error, however this page suggests it isn't supported yet: https://github.com/nvuillam/mega-linter/blob/master/docs/descriptors/javascript_eslint.md#installation-on-mega-linter-docker-image

Describe the bug
mega-linter-runner claims a ruby file that goes against my configuration is OK. When I push up the bad change the Github Action seems to catch the issue with no problems.

To Reproduce
Steps to reproduce the behavior:

1. Create a file called test.rb containing a single line: p 'hello world!'
2. Run mega-linter-runner or in my case I usually do mega-linter-runner -e 'SHOW_ELAPSED_TIME=true' -e 'LINTER_RULES_PATH=https://raw.githubusercontent.com/joe-sharp/linter-configs/main' (I tried both)
3. Optionally also verify .ruby-lint.yml by running rubocop -c .ruby-lint.yml test.rb
4. mega-linter-runner passes the file, rubocop or Mega Linter in a Github Action will fail on the file for:

test.rb:1:1: C: Style/FrozenStringLiteralComment: Missing frozen string literal comment.

Expected behavior
Locally the same rubocop rules should apply to mega-linter-runner as they do the Github Action (assuming the same config is provided.)

Screenshots

### Processing [RUBY] files
- Using [rubocop v0.82.0] https://rubocop.org/
- Mega-Linter key: [RUBY_RUBOCOP]
- Rules config: [https://raw.githubusercontent.com/joe-sharp/linter-configs/main/.ruby-lint.yml]
[rubocop] test.rb - SUCCESS - 0 error(s)
✅ Linted [RUBY] files with [rubocop] successfully - (1.59s)
[Text Reporter] Generated TEXT report: /tmp/lint/report/linters_logs/SUCCESS-RUBY_RUBOCOP.log

and with just a local config:

### Processing [RUBY] files
- Using [rubocop v0.82.0] https://rubocop.org/
- Mega-Linter key: [RUBY_RUBOCOP]
- Rules config: [/.ruby-lint.yml]
[rubocop] test.rb - SUCCESS - 0 error(s)
✅ Linted [RUBY] files with [rubocop] successfully - (1.73s)
[Text Reporter] Generated TEXT report: /tmp/lint/report/linters_logs/SUCCESS-RUBY_RUBOCOP.log

Additional context
Here is the draft PR I created to ensure my rules worked at all:
https://github.com/joe-sharp/linter-configs/pull/11/checks?check_run_id=1767368673

### Processing [RUBY] files
- Using [rubocop v0.82.0] https://rubocop.org/
- Mega-Linter key: [RUBY_RUBOCOP]
- Rules config: [/github/workspace/.ruby-lint.yml]
[rubocop] test.rb - ERROR - 1 error(s)
--Error detail:
Inspecting 1 file
C

Offenses:

test.rb:1:1: C: Style/FrozenStringLiteralComment: Missing frozen string literal comment.
p 'hello world!'
^

1 file inspected, 1 offense detected

❌ Linted [RUBY] files with [rubocop]: Found 1 error(s) - (1.41s)

Maybe it is a good idea to rename default dir for reports to from report to .report? So excluding it by default at any directory level will less likely affect any useful directory in linted project?

I can easily imaging to have folder report with some report generator code inside.

First, tried to say hi via the blue bird but the "This account doesn't exist". It's linked in the footer of your doc home page:

• https://nvuillam.github.io/mega-linter/ footer links to
• https://twitter.com/nvuillam
  which gives the missing account error.

Expectation: Either a valid twitter account should be linked or the link should be removed. Is there a valid twitter account?

What I was trying to say was: Super happy to find this project as I've run up against the limitations of the bash version when adapting it for bitbucket, and to a lesser degree codebuild. Additionally in my port of the super-linter I parse the linter annotations and make line-by-line annotations against the bitbucekt PR api (for example) as well as adding xunit output.

Also thrilled to see flavors and am curious about docker-in-docker as a solution for certain environments. I've still not scanned the docs but I'm hoping the flavors still detect all formats and warn or fail if the flavor is incomplete and/or a separate lightweight image can do that.

Great stuff, hoping to throw up a PR soon (but I must admit this is a bit shaving the yak for me).

https://pypi.org/project/restructuredtext-lint/

Add hyperlink to new issue with pre-input text containing required list of linters

Count copy-pastes correctly with jscpd

ex: Found 239 clones

Following the mega-linter contribution guidelines on a Ubuntu 18.04 system I ran into some errors which I'd like to address:

A) Install dev-requirements with --user option by default:

The sequence to install the pre-requisites:

python3 -m pip install -U pip
python3 -m pip install -r requirements.dev.txt

could be improved with the --user option:

python3 -m pip install --user -U pip # update pip
python3 -m pip install --user -r requirements.dev.txt # install development requirements

B) Align python3 usage in build.sh with installing the pre-requisites

The build.sh script is using python not python3 to run the build script on line 6 leading to a syntax error:

$ bash build.sh
bash build.sh
  File "./.automation/build.py", line 47
    f"{REPO_HOME}/megalinter/descriptors/schemas/megalinter-descriptor.jsonschema.json"
                                                                                      ^
SyntaxError: invalid syntax

This could be improved by testing if python3 command is available and use it when. Otherwise keep the existing behavior and use the python command:

if type python3 >/dev/null 2>/dev/null; then
  python3 ./.automation/build.py
else
  python ./.automation/build.py
fi

C) Force property of logging in build script throw ValueError Exception

$ bash build.sh
bash build.sh
Traceback (most recent call last):
  File "./.automation/build.py", line 1782, in <module>
    handlers=[logging.StreamHandler(sys.stdout)],
  File "/usr/lib/python3.6/logging/__init__.py", line 1829, in basicConfig
    raise ValueError('Unrecognised argument(s): %s' % keys)
ValueError: Unrecognised argument(s): force

This could be improved by catching the ValueError and repeat without force (I could not figure out the required version nor which module that actually is):

build.py 1783 ff:

    try:
        logging.basicConfig(
            force=True,
            level=logging.INFO,
            format="%(asctime)s [%(levelname)s] %(message)s",
            handlers=[logging.StreamHandler(sys.stdout)],
        )
    except ValueError:
        logging.basicConfig(
            level=logging.INFO,
            format="%(asctime)s [%(levelname)s] %(message)s",
            handlers=[logging.StreamHandler(sys.stdout)],
        )

D) Build script generates white-space errors

Many text files in the repository are missing a newline at the end of file. Some of them are generated by build script.

This could be improved by adding the missing newlines and fixing build script to add them as well (which I could try for the case of adding one linter, other scenarios might reveal more which is to be seen).

E) YAML was out of date

My systems YAML extension in Python was not compatible with the build script leading to errors:

$ bash build.sh
Traceback (most recent call last):
  File "./.automation/build.py", line 1786, in <module>
    collect_linter_previews()
  File "./.automation/build.py", line 1610, in collect_linter_previews
    linters = megalinter.linter_factory.list_all_linters()
  File "/home/mot/PhpstormProjects/mega-linter/megalinter/linter_factory.py", line 31, in list_all_linters
    descriptor_file, linters_init_params
  File "/home/mot/PhpstormProjects/mega-linter/megalinter/linter_factory.py", line 61, in build_descriptor_linters
    language_descriptor = yaml.load(f, Loader=yaml.FullLoader)
AttributeError: module 'yaml' has no attribute 'FullLoader'

I could fix this issue by upgrading it (from 3.12 to latest):

$ python3 -m pip install --user -U PyYAML
Requirement already satisfied: PyYAML in /usr/lib/python3/dist-packages (3.12)
Collecting PyYAML
  Downloading PyYAML-5.4.1-cp36-cp36m-manylinux1_x86_64.whl (640 kB)
     |████████████████████████████████| 640 kB 7.0 MB/s
Installing collected packages: PyYAML
Successfully installed PyYAML-5.4.1

An improvement could be to document this for the requirements or if the minimum version (which I don't know what it is) of PyYAML for FullLoader is known, to add it to the dev-requirements.txt file so that doing the pre-requisites already covers this.

Implement a mechanism allowing Mega-Linter to run with descriptor yaml files defined outside of Mega-Linter

Can be used to add additional linters to existing flavors:

• install additional linters at runtime
• add plugin yaml descriptor dynamically when calling Mega-Linter

Could be very useful for https://github.com/cookiejar/cookietemple linter integration :)

example config in .mega-linter.yml:

PLUGINS
  - https://raw.githubusercontent.com/cookiejar/megalinter-plugin-cookiejar/master/cookietemplate.mega-linter-descriptor.yml

Today: forced to 0.5.0 because test cases are failing with 0.5.1

Describe the bug
This is a somewhat two layer issue for the local version: mega-linter-runner. In my repository Mega-Linter reads linter config files from .github/linters, however when running mega-linter-runner locally the path is not used so I get a different set of rules. I think a parameter similar to the -p flag would probably be best and possibly required here, but I also tried manually passing the environment variable LINTER_RULES_PATH without success.

To Reproduce
Steps to reproduce the behavior:

1. Clone https://github.com/joe-sharp/treehouse-vue.git NOTE: make sure to recurse submodules or it will not have the config files: git clone https://github.com/joe-sharp/treehouse-vue.git --recurse-submodules
2. Optionally fork it and test Mega-Linter GitHub Action works
3. Run mega-linter-runner -e 'SHOW_ELAPSED_TIME=true' -e 'DEFAULT_BRANCH=main'
4. Optionally add -e 'LINTER_RULES_PATH=.github/linters'

Expected behavior
I expect .github/linters to be mapped to Docker by default, and additionally it would be nice to have an option of overriding the directory to say project root. (What happens if I passed a subdirectory in the project with the -p flag without this feature?)

Additional context
Since you recently recommended I move my config to project root I think I should add context as to why I haven't. I have set up a repository: https://github.com/joe-sharp/linter-configs that I will incorporate as a submodule in other projects, allowing me to maintain consistent rules across all my projects. The repo you clone in the steps to reproduce is the first example of that. Also, if you don't mind waiting a few days tops, I plan on attempting to fix this and would be delighted to open a PR assuming I am successful, but I thought I would document the issue first. 😄

https://github.com/remarkjs/remark-lint

Do not allow "quick build" and do not upload from forked repositores, as they require docker credentials to manage cache builds

https://github.com/nvuillam/mega-linter/pull/292/checks?check_run_id=1758825615

Related to #292

Problem

I have the following problem. My project contains a number of Docker files with names like Dockerfile-*.
I want to lint them. But Dockerfile liters do not find them.

Proposal

1. Add possibility to customize file_names for particular linter and/or globally

2. Optional: Add possibility to customize file_extensions for particular linter and/or globally

3. Add support for regular expressions of simple file masks inside of file_names.

4. Optional: Compile regular expressions used many times: filter_regex_include, filter_regex_exclude, file_names
   https://docs.python.org/3/library/re.html#re.compile

5. Optional: Join multiple regular expressions to match faster (when we need to match an array of regular expressions to an array of files):
   https://stackoverflow.com/a/3040797/739731

6. (Not related): Some documentation is missing for config options. For example REPORT_OUTPUT_FOLDER, GITHUB_WORKSPACE and others

What do you think about these proposals?
What is a good idea to implement and what is not?

P.S. I'm ready to implement these features. I just want to know they will not break something I do not know. Or maybe you have better ideas how to improve the code or solve my problem.

Mega-Linter would run faster on my project if I had a flavor containing the following list of linters:

PYTHON_PYLINT, PYTHON_BLACK, PYTHON_FLAKE8, PYTHON_ISORT, BASH_EXEC, BASH_SHELLCHECK, BASH_SHFMT, COPYPASTE_JSCPD, CSS_STYLELINT, DOCKERFILE_DOCKERFILELINT, DOCKERFILE_HADOLINT, GIT_GIT_DIFF, JAVASCRIPT_ES, JAVASCRIPT_STANDARD, JSON_JSONLINT, JSON_ESLINT_PLUGIN_JSONC, JSON_V8R, MARKDOWN_MARKDOWNLINT, MARKDOWN_MARKDOWN_LINK_CHECK, MARKDOWN_MARKDOWN_TABLE_FORMATTER, OPENAPI_SPECTRAL, SPELL_MISSPELL, SPELL_CSPELL, XML_XMLLINT, YAML_YAMLLINT, YAML_V8R

Would it be possible to create one ? Thanks ☺️

When updating Mega-Linter image, check its consistency and security using dockle

https://github.com/goodwithtech/dockle#ignore-the-specified-checkpoints

Describe the bug

.mega-linter.yml is not validated because of missing jsonschema properties.
If found that UPDATED_SOURCES_* are missing but maybe some others should be missing too.

❌ [ERROR] .mega-linter.yml
    Validating .mega-linter.yml against schema from https://raw.githubusercontent.com/nvuillam/mega-linter/master/megalinter/descriptors/schemas/megalinter-configuration.jsonschema.json ...
    
    Errors:
    [
      {
        keyword: 'additionalProperties',
        dataPath: '',
        schemaPath: '#/additionalProperties',
        params: { additionalProperty: 'UPDATED_SOURCES_REPORTER' },
        message: 'should NOT have additional properties'
      }
    ]
    
    ❌ .mega-linter.yml is invalid

To Reproduce

1. Go to your project working directory
2. Declare UPDATED_SOURCES_REPORTER: false in .mega-linter.yml
3. Run Mega-Linter (with the YAML V8R)
4. See error in V8R report

Expected behavior
UPDATED_SOURCES_REPORTER should be accepted as described in the documentation

Additional context

Run in a local docker container using:

docker run  -v $(pwd):/tmp/lint nvuillam/mega-linter:v4

Workaround

Excludes .megalinter.yml from V8R Linter using YAML_V8R_FILTER_REGEX_EXCLUDE: '.mega-linter.yml'

Hey, so I've been poking around.

In the short term I'm hacking some things in in a very ugly way. I'll admit that my experience is patchy/incomplete with docker and coding in python and I'm working mostly on my laptop rather than using github automations, for example. Oh and patchy with github actions too.

One of the things I noticed testing with bitbucket is the image build time is long because of the setup.py refetching everything when I need to change a single python file.

Does adding a python lib layer make sense as a PR?

I quickly hacked something like:

##################
# Get base image #
##################
FROM python:3.8-alpine

ARG GITHUB_VERSION=4.20.0
LABEL GITHUB_VERSION=v$GITHUB_VERSION

# APK Packages used by mega-linter pylib
# this must be a subset of main libraries 
# this selection is a little shotgun, it appears to work though
# what happens if they don't match?
RUN apk add --update --no-cache \
    coreutils \
    curl \
    file \
    gcc \
    gnupg \
    icu-libs \
    jq \
    krb5-libs \
    libcurl libintl libssl1.1 libstdc++ \
    libffi-dev \
    linux-headers \
    make \
    musl-dev \
    py3-setuptools \
    readline-dev

################################
# Installs python dependencies #
################################
COPY megalinter /megalinter
RUN python /megalinter/setup.py install --user

###########################
# Get the build arguments #
###########################
ARG BUILD_DATE
ARG BUILD_REVISION
ARG BUILD_VERSION

#################################################
# Set ENV values used for debugging the version #
#################################################
ENV BUILD_DATE=$BUILD_DATE
ENV BUILD_REVISION=$BUILD_REVISION
ENV BUILD_VERSION=$BUILD_VERSION


#########################################
# Label the instance and set maintainer #
#########################################
LABEL com.github.actions.name="Mega-Linter Python Requirements" \
      com.github.actions.description="Python libs for the ultimate linters aggregator to make sure your projects are clean" \
      maintainer="Nicolas Vuillamy <[email protected]>" \
      org.opencontainers.image.created=$BUILD_DATE \
      org.opencontainers.image.revision=$BUILD_REVISION \
      org.opencontainers.image.version=$BUILD_VERSION \
      org.opencontainers.image.authors="Nicolas Vuillamy <[email protected]>" \
      org.opencontainers.image.url="https://nvuillam.github.io/mega-linter" \
      org.opencontainers.image.source="https://github.com/nvuillam/mega-linter" \
      org.opencontainers.image.documentation="https://nvuillam.github.io/mega-linter" \
      org.opencontainers.image.vendor="Nicolas Vuillamy" \
      org.opencontainers.image.description="Lint your code base with GitHub Actions - Python libs"

which is pushed as jlongman/mega-linter-pylib:v$GITHUB_VERSION (well jlongman/mega-linter-pylib:test for now).

then into the main Dockerfile and the flavor I was targeting as a smaller test flavor:

# FROM jlongman/mega-linter-pylib:v$GITHUB_VERSION as mega-linter-pylib
FROM jlongman/mega-linter-pylib:test as mega-linter-pylib
# etc
COPY --from=mega-linter-pylib /root/.local /root/.local
COPY megalinter /megalinter

Note, reviewing this I can't find where I took GITHUB_VERSION from. The intention is to strongly link the pylib image with any of the users images contexts.

I might look at pushing the hacky version for review, in case it helps comprehension. Looking at it the way the autogenerated Dockerfiles are done is not included in what I've done, but I'm not sure if this is worth the effort without an intention to merge.

As I said, it allowed me to iterate faster, in this particular case I was trying to set up the git proxy so that bitbucket's git would automatically authenticate with the pipeline's ownership context, as well as solve a misunderstanding of how GitPython was working locally.

Snakemake maintainers fixed the bug -> snakemake/snakemake#712 (comment)

TODO: remove hardcode of version number in snakemake descriptor