jenkinsci / plugin-installation-manager-tool Goto Github PK

View Code? Open in Web Editor NEW

371.0 19.0 193.0 13.53 MB

Plugin Manager CLI tool for Jenkins

License: MIT License

Java 99.68% Perl 0.20% Dockerfile 0.12%

cli plugin-manager jenkins hacktoberfest

plugin-installation-manager-tool's People

Contributors

Stargazers

Watchers

Forkers

batmat stopalopa kwhetstone pierrebtz timja th3w4y martinda leemeador dsonsem2020 cah-prakash-kalaiarasu01 midalias huaqiangli hint66 sladyn98 echoofinsanity rhenwood3995 ricksbrown syedibrahim01 stefanbirkner sumith08 rudolfwalter oganga yingpingep russh01 eshack94 guptaricha98 babusekaran hixio-mh halkeye novarstar42 sanjaygeeky rhysm oleg-nenashev vsilverman viraj-levelops dinwath nilmapeiris zhaoying818 iberz hp77-creator skundrik pellegrinipierre aajimal garethjevans yabberyabber kwin mbarbero kyounger fr-allen group-ppp linuxsuren fmaree ktanable1 venkikanki pwillis-els vishalbhosale5000 shbalicloudbees v1v kalmalyzer eugenelesnov persianyagami90xs gggk hectoralicea flavorzyb juanmacoo gcuisinier shreyarao0303 uday-create jehan8 basil markewaite yjuns nicolasiltis cronik impact-maker snehilms nhojpatrick ldr426 bpd1069 justcrossheaven urosorolicki 00mjk daves2ng navi019595 pratster95 csseow quinndiggity shilihaha akachiw anushkashreyam tsoliangwu0130 xwzl woma702 jamesrobson-secondmind umar9052 thechandangiri gilbertnuga tas-sos codingtim instabase

plugin-installation-manager-tool's Issues

Fails to run when plugins directory is a docker mounted volume

Because in docker when a directory is marked as a volume, it gets mounted into the container, which means it can't actually be deleted.

$ rm -rf /usr/share/jenkins/ref/plugins
rm: cannot remove '/usr/share/jenkins/ref/plugins': Device or resource busy

$ java -jar /usr/lib/jenkins-plugin-manager.jar -w /usr/share/jenkins/jenkins.war --verbose -f /var/jenkins_home/plugins.txt
File containing list of plugins to be downloaded: /var/jenkins_home/plugins.txt
Reading in plugins from /var/jenkins_home/plugins.txt

No directory to download plugins entered. Will use default of /usr/share/jenkins/ref/plugins
Using update center https://updates.jenkins.io/update-center.json from JENKINS_UC environment variable
Using experimental update center https://updates.jenkins.io/experimental/update-center.json from JENKINS_UC_EXPERIMENTAL environment variable
Using incrementals mirror https://repo.jenkins-ci.org/incrementals from JENKINS_INCREMENTALS_REPO_MIRROR environment variable
No CLI option or environment variable set for plugin info, using default of https://updates.jenkins.io/plugin-versions.json
Will use war file: /usr/share/jenkins/jenkins.war
java.io.UncheckedIOException: Unable to delete: /usr/share/jenkins/ref/plugins
        at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:129)
        at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:113)
        at io.jenkins.tools.pluginmanager.cli.Main.main(Main.java:63)
Caused by: java.io.IOException: Unable to delete directory /usr/share/jenkins/ref/plugins.
        at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1212)
        at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:127)
        ... 2 more
Unable to delete: /usr/share/jenkins/ref/plugins

I think the fix is to delete the contents of the directory, and not the directory itself but having trouble figuring out a clean solution for that. or maybe a flag to make it not delete the directory which would help #108

-d flag removes contents of already existing directory, though does not say in README

Using the -d or --plugin-download-directory will remove the directory/contents specified with the flag, even if there are contents there. This is not reflected in the README:

--plugin-download-directory or -d: (optional) Path to the directory in which to install plugins, which can also be set via
 the PLUGIN_DIR environment variable. Directory will be created if it does not exist. If no directory is entered, directory
 will default to C:\ProgramData\Jenkins\Reference\Plugins if detected OS is Windows, or /usr/share/jenkins/ref/plugins 
otherwise.

I don't believe this should be the default behavior. There should be at least a warning letting the user know that it whatever is there will be removed or that location already exists. I just found this out the hard way as I pointed -d to my current plugins folder which caused the removal of all of my plugins, and now Jenkins wants to uninstall them on next reboot since they're missing.

Please feel free to correct me if I have something wrong.

Add support for Jenkins BOM as a source of plugin versions when using YAML or TXT plugin lists

Jenkins Plugin Bill of Materials provides a list of plugins which are cross-verified to some extent: https://github.com/jenkinsci/bom . When using pom.xml as a plugin list input, it is possible to use BOM via Maven features. But it is not possible for a YAML configuration. It would be awesome to support it so that users can delegate version management of some components they bundle

Example of a cocnfig file:

versionSources:
  - bom:
      baseline: 2.277
      version: 27
plugins:
  - artifactId: git
  - artifactId: job-import-plugin
    source:
      version: 2.1
  - artifactId: docker
  - artifactId: cloudbees-bitbucket-branch-source
  - artifactId: script-security
  - artifactId: workflow-step-api

Bug (or "undocumented feature?") in dependency resolution using `--latest true`

I'm not sure if this is expected behavior, but it certainly was unexpected to me, and seems like it might be a bug. Slightly complicated to explain, but I'll try...

Expected behavior:

User A has a list of pinned plugins to install. They have the entire set of plugins (+ dependencies) that are needed. They pass the list of plugins to the install tool.
The tool goes through each item and collects its dependencies.
Seeing as the entire list of dependencies has been provided, and all of them are pinned, the tool should only install what has been provided and nothing more, and return successfully.

But this is what is happening:

User A has a list of pinned plugins to install. They have the entire set of plugins (+ dependencies) that are needed. They pass the list of plugins to the install tool.
The tool goes through each item and collects its dependencies. It then retrieves the latest version of each dependency (due to --latest default of true), regardless of what has been pinned.
The tool fails with an error that it can't resolve the newer dependency it wanted to collect, because that dependency was already provided and pinned to an earlier version. The tool ignores the fact that the pinned version actually satisfies the dependency.

The "bug" here is that the list User A is providing is complete and accurate. All plugins + dependencies have been provided and pinned to their minimum required versions. But the tool is erroring out, seemingly due to ignoring the pinned items in favor of the latest dependencies it wants to install.

Maybe I missed something and something else is going on? But I've combed through all the MANIFEST.MF files of all the plugins and even the plugin-versions.json file, and all the dependencies shown below seem to be correct. It seems to be just --latest true that is failing.

The plugins below has been crafted to provide the minimum versions of all plugins + dependencies in this chain.

Sample failure with --latest true:

jenkins-plugin-cli --no-download --verbose --latest true --plugins active-directory:2.20 apache-httpcomponents-client-4-api:4.5.10-1.0 aws-credentials:1.23 aws-java-sdk:1.11.341 aws-parameter-store:1.2.2 configuration-as-code:1.27 configuration-as-code-secret-ssm:1.0.1 credentials:2.3.5 credentials-binding:1.23 display-url-api:2.3.1 git:4.2.1 git-client:3.0.0 github:1.29.2 github-api:1.90 jackson2-api:2.7.3 job-dsl:1.72 jsch:0.1.55.1 ldap:2.2 mailer:1.32.1 matrix-auth:2.6.2 plain-credentials:1.6 role-strategy:3.1 scm-api:2.6.3 script-security:1.75 ssh-credentials:1.17.3 structs:1.20 timestamper:1.11.2 token-macro:2.8 workflow-api:2.39 workflow-scm-step:2.9 workflow-step-api:2.22
No .txt or .yaml file containing list of plugins to be downloaded entered.
No directory to download plugins entered. Will use default of /usr/share/jenkins/ref/plugins
Using update center https://updates.jenkins.io/update-center.json from JENKINS_UC environment variable
Using experimental update center https://updates.jenkins.io/experimental/update-center.json from JENKINS_UC_EXPERIMENTAL environment variable
Using incrementals mirror https://repo.jenkins-ci.org/incrementals from JENKINS_INCREMENTALS_REPO_MIRROR environment variable
No CLI option or environment variable set for plugin info, using default of https://updates.jenkins.io/plugin-versions.json
No war entered. Will use default of /usr/share/jenkins/jenkins.war
Jenkins version: 2.272

Retrieving update center information
Created cache at: /var/jenkins_home/.cache/jenkins-plugin-management-cli
Cache miss for: update-center-2.272
Cache miss for: experimental-update-center-2.272
Cache miss for: plugin-versions
Couldn't find checksum for active-directory at version: 2.20
Setting checksum for: active-directory to CGamy8Tm1yzx0TkE6/Sj+6UwPWqodup52RAGxP5MWjU=

active-directory depends on: 
mailer 1.32.1
Skipping dependency mailer:1.32.1 and its sub-dependencies, because there is a higher version defined on the top level - mailer:1.32.1
Couldn't find checksum for apache-httpcomponents-client-4-api at version: 4.5.10-1.0
Setting checksum for: apache-httpcomponents-client-4-api to kTaiJaAAbVqBISY2AWNVjkhhl6DbXKVW6sEDKxTYdNs=

apache-httpcomponents-client-4-api has no dependencies
Couldn't find checksum for aws-credentials at version: 1.23
Setting checksum for: aws-credentials to IP5LXBJzruZrKK21BMzQ+7C0TiQLhL3HTvejh0nucm4=

aws-credentials depends on: 
credentials 2.3.14
aws-java-sdk 1.11.854
credentials-binding 1.24
io.jenkins.tools.pluginmanager.impl.PluginDependencyStrategyException: Plugin aws-credentials:1.23 depends on credentials:2.3.14, but there is an older version defined on the top level - credentials:2.3.5
	at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveRecursiveDependencies(PluginManager.java:883)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.findPluginsAndDependencies(PluginManager.java:493)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:157)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:117)
	at io.jenkins.tools.pluginmanager.cli.Main.main(Main.java:76)
Plugin aws-credentials:1.23 depends on credentials:2.3.14, but there is an older version defined on the top level - credentials:2.3.5

Sample success with --latest false:

jenkins-plugin-cli --no-download --verbose --latest false --plugins active-directory:2.20 apache-httpcomponents-client-4-api:4.5.10-1.0 aws-credentials:1.23 aws-java-sdk:1.11.341 aws-parameter-store:1.2.2 configuration-as-code:1.27 configuration-as-code-secret-ssm:1.0.1 credentials:2.3.5 credentials-binding:1.23 display-url-api:2.3.1 git:4.2.1 git-client:3.0.0 github:1.29.2 github-api:1.90 jackson2-api:2.7.3 job-dsl:1.72 jsch:0.1.55.1 ldap:2.2 mailer:1.32.1 matrix-auth:2.6.2 plain-credentials:1.6 role-strategy:3.1 scm-api:2.6.3 script-security:1.75 ssh-credentials:1.17.3 structs:1.20 timestamper:1.11.2 token-macro:2.8 workflow-api:2.39 workflow-scm-step:2.9 workflow-step-api:2.22
No .txt or .yaml file containing list of plugins to be downloaded entered.
No directory to download plugins entered. Will use default of /usr/share/jenkins/ref/plugins
Using update center https://updates.jenkins.io/update-center.json from JENKINS_UC environment variable
Using experimental update center https://updates.jenkins.io/experimental/update-center.json from JENKINS_UC_EXPERIMENTAL environment variable
Using incrementals mirror https://repo.jenkins-ci.org/incrementals from JENKINS_INCREMENTALS_REPO_MIRROR environment variable
No CLI option or environment variable set for plugin info, using default of https://updates.jenkins.io/plugin-versions.json
No war entered. Will use default of /usr/share/jenkins/jenkins.war
Jenkins version: 2.272

Retrieving update center information
Created cache at: /var/jenkins_home/.cache/jenkins-plugin-management-cli
Cache miss for: update-center-2.272
Cache miss for: experimental-update-center-2.272
Cache miss for: plugin-versions
Couldn't find checksum for active-directory at version: 2.20
Setting checksum for: active-directory to CGamy8Tm1yzx0TkE6/Sj+6UwPWqodup52RAGxP5MWjU=

active-directory depends on: 
mailer 1.5
Skipping dependency mailer:1.5 and its sub-dependencies, because there is a higher version defined on the top level - mailer:1.32.1
Couldn't find checksum for apache-httpcomponents-client-4-api at version: 4.5.10-1.0
Setting checksum for: apache-httpcomponents-client-4-api to kTaiJaAAbVqBISY2AWNVjkhhl6DbXKVW6sEDKxTYdNs=

apache-httpcomponents-client-4-api has no dependencies
Couldn't find checksum for aws-credentials at version: 1.23
Setting checksum for: aws-credentials to IP5LXBJzruZrKK21BMzQ+7C0TiQLhL3HTvejh0nucm4=

aws-credentials depends on: 
credentials 2.1.16
aws-java-sdk 1.10.16
credentials-binding 1.7
Skipping dependency credentials:2.1.16 and its sub-dependencies, because there is a higher version defined on the top level - credentials:2.3.5
Skipping dependency aws-java-sdk:1.10.16 and its sub-dependencies, because there is a higher version defined on the top level - aws-java-sdk:1.11.341
Skipping dependency credentials-binding:1.7 and its sub-dependencies, because there is a higher version defined on the top level - credentials-binding:1.23
Couldn't find checksum for aws-java-sdk at version: 1.11.341
Setting checksum for: aws-java-sdk to 6+fMp3VdwCDROO9Tenv3th47utQffw9r5ABJF8B2kjg=

aws-java-sdk depends on: 
apache-httpcomponents-client-4-api 4.5.3-2.0
jackson2-api 2.7.0
Skipping dependency apache-httpcomponents-client-4-api:4.5.3-2.0 and its sub-dependencies, because there is a higher version defined on the top level - apache-httpcomponents-client-4-api:4.5.10-1.0
Skipping dependency jackson2-api:2.7.0 and its sub-dependencies, because there is a higher version defined on the top level - jackson2-api:2.7.3
Setting checksum for: aws-parameter-store to bUDzx5vvdyQ4QYkW3C2UtO46UVR5v0yCWWUNrWFh71w=
Setting checksum for: aws-parameter-store to bUDzx5vvdyQ4QYkW3C2UtO46UVR5v0yCWWUNrWFh71w=

aws-parameter-store depends on: 
aws-credentials 1.23
aws-java-sdk 1.11.264
Skipping dependency aws-credentials:1.23 and its sub-dependencies, because there is a higher version defined on the top level - aws-credentials:1.23
Skipping dependency aws-java-sdk:1.11.264 and its sub-dependencies, because there is a higher version defined on the top level - aws-java-sdk:1.11.341
Setting checksum for: aws-parameter-store to bUDzx5vvdyQ4QYkW3C2UtO46UVR5v0yCWWUNrWFh71w=
Couldn't find checksum for configuration-as-code at version: 1.27
Setting checksum for: configuration-as-code to 3i717v6SHSrnZTGJ7RbUsLa+X8VMegbOCdNg4v0HpCo=

configuration-as-code has no dependencies
Setting checksum for: configuration-as-code-secret-ssm to uxb74m+8kzBnotLH0YjXi4uv8sBeR713izCG+ZKGGbA=
Setting checksum for: configuration-as-code-secret-ssm to uxb74m+8kzBnotLH0YjXi4uv8sBeR713izCG+ZKGGbA=

configuration-as-code-secret-ssm depends on: 
configuration-as-code 1.2
aws-java-sdk 1.11.341
Skipping dependency configuration-as-code:1.2 and its sub-dependencies, because there is a higher version defined on the top level - configuration-as-code:1.27
Skipping dependency aws-java-sdk:1.11.341 and its sub-dependencies, because there is a higher version defined on the top level - aws-java-sdk:1.11.341
Setting checksum for: configuration-as-code-secret-ssm to uxb74m+8kzBnotLH0YjXi4uv8sBeR713izCG+ZKGGbA=
Couldn't find checksum for credentials at version: 2.3.5
Setting checksum for: credentials to tgVI8cfMc/mk8TzxP7yzUeKUXhudHmow5qOym+CvO5Q=

credentials depends on: 
structs 1.20
Skipping dependency structs:1.20 and its sub-dependencies, because there is a higher version defined on the top level - structs:1.20
Couldn't find checksum for credentials-binding at version: 1.23
Setting checksum for: credentials-binding to 0tsPEoS2UEI3UUApcvSLUqkiisETySJxCNGeY9bXejo=

credentials-binding depends on: 
workflow-step-api 2.22
credentials 2.3.5
plain-credentials 1.6
ssh-credentials 1.17.3
structs 1.20
Skipping dependency workflow-step-api:2.22 and its sub-dependencies, because there is a higher version defined on the top level - workflow-step-api:2.22
Skipping dependency credentials:2.3.5 and its sub-dependencies, because there is a higher version defined on the top level - credentials:2.3.5
Skipping dependency plain-credentials:1.6 and its sub-dependencies, because there is a higher version defined on the top level - plain-credentials:1.6
Skipping dependency ssh-credentials:1.17.3 and its sub-dependencies, because there is a higher version defined on the top level - ssh-credentials:1.17.3
Skipping dependency structs:1.20 and its sub-dependencies, because there is a higher version defined on the top level - structs:1.20
Couldn't find checksum for display-url-api at version: 2.3.1
Setting checksum for: display-url-api to FIjOqVuLDR7dgw56pJRYvubMsIGRcZlq/ykbE+m5mjw=

display-url-api has no dependencies
Couldn't find checksum for git at version: 4.2.1
Setting checksum for: git to Xc9x7ta81coOYHFXmic4C3szqD33QMG9VkmS25bXklA=

git depends on: 
workflow-scm-step 2.9
workflow-step-api 2.20
credentials 2.3.0
git-client 3.0.0
mailer 1.23
scm-api 2.6.3
script-security 1.66
ssh-credentials 1.17.3
structs 1.20
Skipping dependency workflow-scm-step:2.9 and its sub-dependencies, because there is a higher version defined on the top level - workflow-scm-step:2.9
Skipping dependency workflow-step-api:2.20 and its sub-dependencies, because there is a higher version defined on the top level - workflow-step-api:2.22
Skipping dependency credentials:2.3.0 and its sub-dependencies, because there is a higher version defined on the top level - credentials:2.3.5
Skipping dependency git-client:3.0.0 and its sub-dependencies, because there is a higher version defined on the top level - git-client:3.0.0
Skipping dependency mailer:1.23 and its sub-dependencies, because there is a higher version defined on the top level - mailer:1.32.1
Skipping dependency scm-api:2.6.3 and its sub-dependencies, because there is a higher version defined on the top level - scm-api:2.6.3
Skipping dependency script-security:1.66 and its sub-dependencies, because there is a higher version defined on the top level - script-security:1.75
Skipping dependency ssh-credentials:1.17.3 and its sub-dependencies, because there is a higher version defined on the top level - ssh-credentials:1.17.3
Skipping dependency structs:1.20 and its sub-dependencies, because there is a higher version defined on the top level - structs:1.20
Couldn't find checksum for git-client at version: 3.0.0
Setting checksum for: git-client to DKFkrtQQCZDhroGQH2Ld7hBc9+vP7AOpcVErPjZgRsA=

git-client depends on: 
apache-httpcomponents-client-4-api 4.5.10-1.0
credentials 2.3.0
jsch 0.1.55.1
ssh-credentials 1.17.2
structs 1.20
Skipping dependency apache-httpcomponents-client-4-api:4.5.10-1.0 and its sub-dependencies, because there is a higher version defined on the top level - apache-httpcomponents-client-4-api:4.5.10-1.0
Skipping dependency credentials:2.3.0 and its sub-dependencies, because there is a higher version defined on the top level - credentials:2.3.5
Skipping dependency jsch:0.1.55.1 and its sub-dependencies, because there is a higher version defined on the top level - jsch:0.1.55.1
Skipping dependency ssh-credentials:1.17.2 and its sub-dependencies, because there is a higher version defined on the top level - ssh-credentials:1.17.3
Skipping dependency structs:1.20 and its sub-dependencies, because there is a higher version defined on the top level - structs:1.20
Couldn't find checksum for github at version: 1.29.2
Setting checksum for: github to P0s/MQxAXzTNZyxZ2NxuRTx4gTgPy0yywZlKbg5z4SE=

github depends on: 
credentials 2.1.8
display-url-api 2.0
git 3.4.0
github-api 1.90
plain-credentials 1.1
scm-api 2.2.0
structs 1.10
token-macro 1.11
Skipping dependency credentials:2.1.8 and its sub-dependencies, because there is a higher version defined on the top level - credentials:2.3.5
Skipping dependency display-url-api:2.0 and its sub-dependencies, because there is a higher version defined on the top level - display-url-api:2.3.1
Skipping dependency git:3.4.0 and its sub-dependencies, because there is a higher version defined on the top level - git:4.2.1
Skipping dependency github-api:1.90 and its sub-dependencies, because there is a higher version defined on the top level - github-api:1.90
Skipping dependency plain-credentials:1.1 and its sub-dependencies, because there is a higher version defined on the top level - plain-credentials:1.6
Skipping dependency scm-api:2.2.0 and its sub-dependencies, because there is a higher version defined on the top level - scm-api:2.6.3
Skipping dependency structs:1.10 and its sub-dependencies, because there is a higher version defined on the top level - structs:1.20
Skipping dependency token-macro:1.11 and its sub-dependencies, because there is a higher version defined on the top level - token-macro:2.8
Couldn't find checksum for github-api at version: 1.90
Setting checksum for: github-api to jsEMmLTMED2Cbi4qEBtL9Es7kbOVwz3nRIADQxunNs4=

github-api depends on: 
jackson2-api 2.7.3
Skipping dependency jackson2-api:2.7.3 and its sub-dependencies, because there is a higher version defined on the top level - jackson2-api:2.7.3
Couldn't find checksum for jackson2-api at version: 2.7.3
Setting checksum for: jackson2-api to bfobo9syrkdjCyOTmcP8jZEC/wtSOlwz3TK8iNmHSDc=

jackson2-api has no dependencies
Couldn't find checksum for job-dsl at version: 1.72
Setting checksum for: job-dsl to UaAZG3Ovv1xl+AZAILiRdkSmiZ+qKW6gfi3xjP9niTQ=

job-dsl depends on: 
structs 1.13
script-security 1.54
Skipping dependency structs:1.13 and its sub-dependencies, because there is a higher version defined on the top level - structs:1.20
Skipping dependency script-security:1.54 and its sub-dependencies, because there is a higher version defined on the top level - script-security:1.75
Couldn't find checksum for jsch at version: 0.1.55.1
Setting checksum for: jsch to k0FRYA3UKwOY6IH7unBlbDj9am8SqHTSP8jcNeNxxNQ=

jsch depends on: 
ssh-credentials 1.14
Skipping dependency ssh-credentials:1.14 and its sub-dependencies, because there is a higher version defined on the top level - ssh-credentials:1.17.3
Setting checksum for: ldap to waxLrSaPc/LvbBgjbgg11WeD2jMXJ3Rg/TGBKxXECvs=
Setting checksum for: ldap to waxLrSaPc/LvbBgjbgg11WeD2jMXJ3Rg/TGBKxXECvs=

ldap depends on: 
mailer 1.32.1
Skipping dependency mailer:1.32.1 and its sub-dependencies, because there is a higher version defined on the top level - mailer:1.32.1
Setting checksum for: ldap to waxLrSaPc/LvbBgjbgg11WeD2jMXJ3Rg/TGBKxXECvs=
Setting checksum for: mailer to CwReqH78KJ/MrVWsUeTJA5RdloO3G0hmfhlEDkKi02g=
Setting checksum for: mailer to CwReqH78KJ/MrVWsUeTJA5RdloO3G0hmfhlEDkKi02g=

mailer depends on: 
display-url-api 2.3.1
Skipping dependency display-url-api:2.3.1 and its sub-dependencies, because there is a higher version defined on the top level - display-url-api:2.3.1
Setting checksum for: mailer to CwReqH78KJ/MrVWsUeTJA5RdloO3G0hmfhlEDkKi02g=
Couldn't find checksum for matrix-auth at version: 2.6.2
Setting checksum for: matrix-auth to ZnV+9pzC4a45xwovy9WZluON6phRi0o5Xf5CZH1S3ls=

matrix-auth has no dependencies
Couldn't find checksum for plain-credentials at version: 1.6
Setting checksum for: plain-credentials to JrwSClZ0r6VZJSfjPF/LCC9BTV9MO13fsvRertn4MHM=

plain-credentials depends on: 
credentials 2.1.16
Skipping dependency credentials:2.1.16 and its sub-dependencies, because there is a higher version defined on the top level - credentials:2.3.5
Setting checksum for: role-strategy to xf/+FgDzUkb3pdruf51USfQUI1dA7pM+x7iRImujFzQ=
Setting checksum for: role-strategy to xf/+FgDzUkb3pdruf51USfQUI1dA7pM+x7iRImujFzQ=

role-strategy depends on: 
matrix-auth 2.2
Skipping dependency matrix-auth:2.2 and its sub-dependencies, because there is a higher version defined on the top level - matrix-auth:2.6.2
Setting checksum for: role-strategy to xf/+FgDzUkb3pdruf51USfQUI1dA7pM+x7iRImujFzQ=
Couldn't find checksum for scm-api at version: 2.6.3
Setting checksum for: scm-api to gyYtQGhirVX/kNNgIKg/iOvXHmb0nHXJpRQPQxdquik=

scm-api depends on: 
structs 1.9
Skipping dependency structs:1.9 and its sub-dependencies, because there is a higher version defined on the top level - structs:1.20
Setting checksum for: script-security to 46ntxSuoQmaVu6ShnDGuQcOk6WwgpH/PX+gOd4UZxLg=
Setting checksum for: script-security to 46ntxSuoQmaVu6ShnDGuQcOk6WwgpH/PX+gOd4UZxLg=

script-security has no dependencies
Setting checksum for: script-security to 46ntxSuoQmaVu6ShnDGuQcOk6WwgpH/PX+gOd4UZxLg=
Couldn't find checksum for ssh-credentials at version: 1.17.3
Setting checksum for: ssh-credentials to Y/5ezKJXbwSYzSg5I+0gxBYHMKLOjjUGZuapxpkjfKs=

ssh-credentials depends on: 
credentials 2.2.0
Skipping dependency credentials:2.2.0 and its sub-dependencies, because there is a higher version defined on the top level - credentials:2.3.5
Setting checksum for: structs to fnhhNWo3qmpydGLXrqcW3ZMHBxJS9zScJybWSnc/6zo=
Setting checksum for: structs to fnhhNWo3qmpydGLXrqcW3ZMHBxJS9zScJybWSnc/6zo=

structs has no dependencies
Setting checksum for: structs to fnhhNWo3qmpydGLXrqcW3ZMHBxJS9zScJybWSnc/6zo=
Couldn't find checksum for timestamper at version: 1.11.2
Setting checksum for: timestamper to Kp52J/R0R6hbWIVsnuBV4m4XhTONp5XN+SQHGYQr8xg=

timestamper depends on: 
workflow-api 2.39
workflow-step-api 2.19
Skipping dependency workflow-api:2.39 and its sub-dependencies, because there is a higher version defined on the top level - workflow-api:2.39
Skipping dependency workflow-step-api:2.19 and its sub-dependencies, because there is a higher version defined on the top level - workflow-step-api:2.22
Couldn't find checksum for token-macro at version: 2.8
Setting checksum for: token-macro to WwaKWNi7yRp0z/0TfqoZYC19jVzpAYzACtZ+M7fDKDw=

token-macro depends on: 
workflow-step-api 2.14
structs 1.14
Skipping dependency workflow-step-api:2.14 and its sub-dependencies, because there is a higher version defined on the top level - workflow-step-api:2.22
Skipping dependency structs:1.14 and its sub-dependencies, because there is a higher version defined on the top level - structs:1.20
Couldn't find checksum for workflow-api at version: 2.39
Setting checksum for: workflow-api to 7r0mqAtz8/K0G4++p4ncAVTDX9qzP9SGBRE6KYZ5wZI=

workflow-api depends on: 
workflow-step-api 2.16
scm-api 2.2.6
structs 1.17
Skipping dependency workflow-step-api:2.16 and its sub-dependencies, because there is a higher version defined on the top level - workflow-step-api:2.22
Skipping dependency scm-api:2.2.6 and its sub-dependencies, because there is a higher version defined on the top level - scm-api:2.6.3
Skipping dependency structs:1.17 and its sub-dependencies, because there is a higher version defined on the top level - structs:1.20
Couldn't find checksum for workflow-scm-step at version: 2.9
Setting checksum for: workflow-scm-step to Y7EF7XduJp9pFQUiGNXNcx2mDKviKRu/0HGOpOOEuYE=

workflow-scm-step depends on: 
workflow-step-api 2.9
Skipping dependency workflow-step-api:2.9 and its sub-dependencies, because there is a higher version defined on the top level - workflow-step-api:2.22
Couldn't find checksum for workflow-step-api at version: 2.22
Setting checksum for: workflow-step-api to 65xkoZQdOvMgRRJn4kj+Og/50jboDD9Xbx06Zm1QKjM=

workflow-step-api depends on: 
structs 1.20
Skipping dependency structs:1.20 and its sub-dependencies, because there is a higher version defined on the top level - structs:1.20
Done

Security warnings should be shown by default

plugin-installation-manager-tool/plugin-management-cli/src/main/java/io/jenkins/tools/pluginmanager/cli/CliOptions.java

Lines 77 to 80 in 79b38b4

    
           @Option(name = "--view-security-warnings", 
        
                   usage = "Show if any security warnings exist for the requested plugins", 
        
                   handler = BooleanOptionHandler.class) 
        
           private boolean showWarnings;

There seems to be no real reason for this to be disabled by default.

--view-security-warnings should be made a no-op for compatibility, and a new option --hide-security-warnings (or similar) should be added to explicitly opt out of showing them.

Add an option to generate a new updates file in `available-updates`

Currently the tool generates the following output:

Available updates:
blueocean-config (1.14.0) has an available update: 1.23.2
jdk-tool (1.2) has an available update: 1.4
github-branch-source (2.3.2) has an available update: 2.7.1
blueocean-pipeline-api-impl (1.14.0) has an available update: 1.23.2
workflow-multibranch (2.20) has an available update: 2.21
docker-workflow (1.14) has an available update: 1.23
maven-plugin (2.17) has an available update: 3.6
pipeline-stage-tags-metadata (1.3.2) has an available update: 1.6.0
git-client (2.7.3) has an available update: 3.2.1
scm-api (2.2.8) has an available update: 2.6.3
email-ext (2.65) has an available update: 2.69
github (1.26.0) has an available update: 1.30.0

It would be great if the tool had support for generating a plugins.txt or YAML with the resulting list of plugins after the update. It would save a lot of time on managing dependencies when using the tool.

Ability to update plugins file to latest version

It would be great to have the ability to update the plugins.yaml file to the latest version of all plugins.

The way that I currently do that is within an Ansible playbook that does the following:

Make a new copy of plugins.yaml on the Jenkins master
Remove all version: lines from that file
Run java -jar jenkins-plugin-manager-{{ jenkins_pimt_version }}.jar -w {{ jenkins_war }} -f /tmp/plugins.yaml -d {{ jenkins_home }}/plugins --available-updates --latest
Run java -jar {{ jenkins_pimt_location }}/jenkins-plugin-manager-{{ jenkins_pimt_version }}.jar -w {{ jenkins_war }} -f /tmp/plugins.yaml -d {{ jenkins_home }}/plugins --no-download --list > /tmp/plugin-versions.txt
Copy /tmp/plugin-versions.txt to the control machine
Run a Python script which processes the output of /tmp/plugin-versions.txt and generates plugins.yaml as output

This is quite a bit of work, and it's also error prone. It also means that our plugins.yaml has to track everything, making it act like a lockfile. This also means that removing plugins (which does happen occasionally) is cumbersome, since that plugin's dependencies are still tracked in plugins.yaml and need to be removed manually.

It would be nice if plugin-installation-manager-tool had an option to generate an updated version of plugins.yaml based on the current version and whatever plugin updates were available. Truthfully, our team doesn't much care about plugin dependency versions or transitive dependency versions -- we only care about the versions of the plugins we directly want to install. But that means when we run plugin-installation-manager-tool, I would like two things to happen:

The plugins installed from plugins.yaml are always installed at the exact version specified
Everything else (all dependencies, transitive dependencies, etc) are updated if necessary

This issue was created as a result of the discussion in #85.

Some features slightly fragile

Given the following plugins.txt file:

active-directory
aws-parameter-store
configuration-as-code
configuration-as-code-secret-ssm
git
github
job-dsl
ldap
role-strategy
timestamper

The following command returns an error:

docker run \
    --rm -it \
    -v `pwd`/plugins.txt:/app/plugins.txt \
    jenkins/jenkins \
        jenkins-plugin-cli \
            --plugin-file /app/plugins.txt \
            --view-all-security-warnings \
            --no-download
java.util.ArrayList cannot be cast to java.lang.Comparable

This seems to be a bug, can anyone confirm?

If the plugins.txt contains git:4.1.9, a different command throws an error when it cannot resolve a version dependency:

docker run \
    --rm -it \
    -v `pwd`/plugins.txt:/app/plugins.txt \
    jenkins/jenkins \
        jenkins-plugin-cli \
            --plugin-file /app/plugins.txt \
            --view-security-warnings \
            --no-download \
            --skip-failed-plugins
Downloaded file is not a valid ZIP
Unable to resolve dependencies for git
Plugin github:1.32.0 depends on git:4.5.1, but there is an older version defined on the top level - git:4.1.9

For this second error, it would be useful if there were a "--force" or "--ignore-errors" option. The "--skip-failed-plugins" option doesn't seem to change the result.

Use latest compatible version of plugins when generating YAML file

Given that the tool requires a Jenkins JAR, it would be nice if it only considered the latest compatible version of plugins when generating the plugins.yaml file. I routinely run into this issue, for example most recently after having updated Jenkins to 2.263.2, and then noticing that the tool had upgraded branch-api to an incompatible version.

We update plugins by running this command:

java -jar <path to tool JAR file> \
          --available-updates \
          --output YAML \
          --plugin-file /tmp/plugins.yaml \
          --war <path to Jenkins WAR file> \
        > /tmp/updated-plugins.yaml

...and then we take the resulting plugins.yaml file, put it in source control, etc. When deploying Jenkins, we then run the following command:

    java -jar <path to tool JAR file> \
      --plugin-download-directory /opt/jenkins/home/plugins \
      --plugin-file <path to plugins.yaml> \
      --war <path to Jenkins WAR file> \
      --latest false \
      --verbose

We only learn about the incompatibility when running the second deployment step, where we get an error message like branch-api (2.6.3) requires a greater version of Jenkins (2.264) than 2.263.2 in /opt/jenkins/lib/jenkins-2.263.2.war. In short, it would be really nice if the tool would generate a YAML file with the latest compatible version. I'm not sure if this is easy/possible, but given that the WAR file is being passed to the tool I would hope that it is.

Basic usage throws org.json.JSONException

Probably something I'm overlooking but running the most basic example fails for me:

jenkins@5e3eb1c93a51:~$ java -jar jenkins-plugin-manager-1.1.2.jar --war /usr/share/jenkins/jenkins.war --plugin-file /var/jenkins_home/plugins.txt --verbose
File containing list of plugins to be downloaded: /var/jenkins_home/plugins.txt
Reading in plugins from /var/jenkins_home/plugins.txt

No directory to download plugins entered. Will use default of /usr/share/jenkins/ref/plugins
Using update center https://updates.jenkins.io from JENKINS_UC environment variable
Using experimental update center https://updates.jenkins.io/experimental from JENKINS_UC_EXPERIMENTAL environment variable
Using incrementals mirror https://repo.jenkins-ci.org/incrementals from JENKINS_INCREMENTALS_REPO_MIRROR environment variable
No CLI option or environment variable set for plugin info, using default of https://updates.jenkins.io/current/plugin-versions.json
Will use war file: /usr/share/jenkins/jenkins.war
Jenkins version: 2.235.1
No version specific update center for Jenkins version 2.235.1

Retrieving update center information
org.json.JSONException: A JSONObject text must begin with '{' at 1 [character 2 line 1]
	at org.json.JSONTokener.syntaxError(JSONTokener.java:505)
	at org.json.JSONObject.<init>(JSONObject.java:215)
	at org.json.JSONObject.<init>(JSONObject.java:399)
	at io.jenkins.tools.pluginmanager.impl.CacheManager.retrieveFromCache(CacheManager.java:73)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.getJson(PluginManager.java:561)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.getUCJson(PluginManager.java:591)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:141)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:112)
	at io.jenkins.tools.pluginmanager.cli.Main.main(Main.java:37)
A JSONObject text must begin with '{' at 1 [character 2 line 1]

I'm running this command from inside a docker container using the jenkins/jenkins:2.235.1 image.

Plugin file:

blueocean:1.23.2
configuration-as-code:1.41
gitlab-plugin:1.5.13
ssh-credentials:1.18.1
job-dsl:1.77

What I am doing wrong here?

Build update center tree structure?

Below I describe a use case for plugin-installation-manager-tool that is not currently catered for. I was going to just do it but it would be a non-trivial amount of work so I thought I'd float it first. In the meantime we are going to have to continue using custom scripts to augment plugin-installation-manager-tool.

There are actually two different consumers for the downloaded plugins:

Jenkins (obviously)
(On-prem) Jenkins Update Center (UC)

Jenkins requires the plugins to be downloaded into a flat directory structure, that's what plugin-installation-manager-tool does. Job done.

Since our Jenkins is not allowed to connect to the cloud we must also take the plugins and upload them to an on-premises server, it's not a true Update Center, but we upload them in the same format:

host.example.com/updates.jenkins.io/download/plugins/{plugin-name}/{plugin-version}/{plugin-name}.hpi

We have a script that downloads them into the correct directory structure, which is basically a very selective mirror of the Jenkins Update Center, just the plugins we need.

I was thinking plugin-installation-manager-tool could potentially do this too, either with a different mode OR it might make sense to build this tree structure in the cache directory. That way plugins would only ever be downloaded once, multiple runs of the script would potentially be much faster, only downloading plugins they had not seen before.

running behind proxy

What I need to define in order to run behind proxy?

I tried to define the env http/s_proxy but without success

Plugins cannot be downloaded when some sites are blocked but not others

Hello:

Sometime around August 19th, our jenkins builds using version 2.0.0 stopped working. Upon further investigation, it appears that jenkins.io may have changed their behavior:

$ curl -I https://updates.jenkins.io/download/plugins/authentication-tokens/1.4/authentication-tokens.hpi
HTTP/1.1 302 Found
Date: Fri, 28 Aug 2020 20:30:53 GMT
Server: Apache/2.4.29 (Ubuntu)
Location: https://get.jenkins.io/plugins/authentication-tokens/1.4/authentication-tokens.hpi
Content-Type: text/html; charset=iso-8859-1

$ curl -I https://get.jenkins.io/plugins/authentication-tokens/1.4/authentication-tokens.hpi
HTTP/1.1 302 Found
Server: nginx/1.19.1
Date: Fri, 28 Aug 2020 20:31:03 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Cache-Control: private, no-cache
Link: <https://ftp.yz.yamagata-u.ac.jp/pub/misc/jenkins/plugins/authentication-tokens/1.4/authentication-tokens.hpi>; rel=duplicate; pri=1; geo=jp
Link: <https://ftp-nyc.osuosl.org/pub/jenkins/plugins/authentication-tokens/1.4/authentication-tokens.hpi>; rel=duplicate; pri=2; geo=us
Link: <https://mirror.xmission.com/jenkins/plugins/authentication-tokens/1.4/authentication-tokens.hpi>; rel=duplicate; pri=3; geo=us
Link: <https://mirror.serverion.com/jenkins/plugins/authentication-tokens/1.4/authentication-tokens.hpi>; rel=duplicate; pri=4; geo=de
Location: https://ftp-chi.osuosl.org/pub/jenkins/plugins/authentication-tokens/1.4/authentication-tokens.hpi
Strict-Transport-Security: max-age=15724800; includeSubDomains

If any site in Link or Location is not reachable, the plugin manager might not work with the following message:

Unable to resolve plugin URL https://updates.jenkins.io/download/plugins/authentication-tokens/1.4/authentication-tokens.hpi, or download plugin authentication-tokens to file

Could the tool be updated such that it only returns this message if all of the links are unreachable? This is especially a problem for those behind proxies that allow only certain domains to be accessed.

Thanks!

2.1.3 does not build on OpenJDK16

I'm sorry to report that the latest release (2.1.3) does not build with docker OpenJDK16 (openjdk:16-jdk-alpine3.12), attaching output from mvn package:

[ERROR] Failures: 
[ERROR]   CacheManagerTest.cacheManagerWithNonReadableJsonFileForKey:159 
Expecting:
 <"">
to start with:
 <"Cache ignored file the-cache-key.json because it cannot be read.
java.nio.file.AccessDeniedException:">

[ERROR]   CacheManagerTest.cacheReturnsNullWhenCachedFileCannotBeRead:148 
Expecting:
 <{}>
to be equal to:
 <null>
but was not.
[INFO] 
[ERROR] Tests run: 121, Failures: 2, Errors: 0, Skipped: 1

Error installing latest versions of pipeline-model-definition and workflow-multibranch at the same time

When trying to install these two plugins via the jenkins-plugin-manager:

pipeline-model-definition:latest
workflow-multibranch:latest

the following error is thrown:

Plugin pipeline-model-definition:1.7.1 depends on workflow-multibranch:2.22, 
but there is an older version defined on the top level - workflow-multibranch:latest

Error "Downloaded file is not a valid ZIP" is not clear

If a plugin cannot be found a more useful error message should be given than "Downloaded file is not a valid ZIP". For example
some plugins have inconsistent names and may or may not end with -plugin. The install-plugins.sh used to try both. I was getting this error message but it was not clear and I had to dig into why the execution was failing.

Example

java -jar jenkins-plugin-manager-2.5.0.jar --jenkins-version 2.235.5 --plugins shelve-project:3.0
War not found, installing all plugins: C:\ProgramData\Jenkins\jenkins.war
Downloaded file is not a valid ZIP
Unable to resolve dependencies for shelve-project
Downloaded file is not a valid ZIP
Unable to download shelve-project

Correcting the issue by fixing the plugin identifier

java -jar jenkins-plugin-manager-2.5.0.jar --jenkins-version 2.235.5 --plugins shelve-project-plugin:3.0
War not found, installing all plugins: C:\ProgramData\Jenkins\jenkins.war
Done

error: unexpected EOF

Hi,

sometimes I got below error.

<snip>
Checksum valid for: jira
jira downloaded successfully
Checksum valid for: p4
p4 downloaded successfully
Checksum valid for: ace-editor
ace-editor downloaded successfully
error: unexpected EOF

I think plugin download finished but what is error: unexpected EOF
I am using --verbose option but I can't find any clue.

Tool has no support for proxy servers

Currently the installation tool has no support for proxy servers. This would make a massive difference for adoption in larger corporate offices.

Posting snippets of the log first part repeats for each section


credentials depends on: 
structs 1.7

structs has no dependencies

blueocean depends on: 
blueocean-bitbucket-pipeline 1.21.0
blueocean-commons 1.21.0
blueocean-config 1.21.0
blueocean-core-js 1.21.0
blueocean-dashboard 1.21.0
blueocean-events 1.21.0
blueocean-git-pipeline 1.21.0
blueocean-github-pipeline 1.21.0
blueocean-i18n 1.21.0
blueocean-jira 1.21.0
blueocean-jwt 1.21.0
blueocean-personalization 1.21.0
blueocean-pipeline-api-impl 1.21.0
blueocean-pipeline-editor 1.21.0
blueocean-rest-impl 1.21.0
blueocean-rest 1.21.0
blueocean-web 1.21.0
jenkins-design-language 1.21.0
blueocean-autofavorite 1.2.3
blueocean-display-url 2.2.0
pipeline-build-step 2.7
pipeline-milestone-step 1.3.1

End Error

Resolving dependencies of scm-api by downloading plugin to temp file /tmp/scm-api5982136854335144858.jpi and parsing MANIFEST.MF
Unable to resolve plugin URL https://updates.jenkins.io/download/plugins/scm-api/2.0.1-beta-1/scm-api.hpi, or download plugin scm-api to file
First download attempt of scm-api unsuccessful, reattempting
Unable to resolve plugin URL https://updates.jenkins.io/download/plugins/scm-api-plugin/2.0.1-beta-1/scm-api-plugin.hpi, or download plugin scm-api-plugin to file
io.jenkins.tools.pluginmanager.impl.DownloadPluginException: Unable to resolve dependencies for scm-api-plugin
	at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveDependenciesFromManifest(PluginManager.java:572)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveDirectDependencies(PluginManager.java:692)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveRecursiveDependencies(PluginManager.java:716)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.findPluginsAndDependencies(PluginManager.java:436)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:118)
	at io.jenkins.tools.pluginmanager.cli.Main.main(Main.java:37)
Unable to resolve dependencies for scm-api-plugin
ERROR: Service 'jenkins' failed to build: The command '/bin/sh -c java -Dhttp.proxyHost=xx.xx.xx.xx -Dhttp.proxyPort=8080 -Dhttps.proxyHost=xx.xx.xx.xx -Dhttps.proxyPort=8080 -jar jenkins-plugin-manager-1.0.1.jar -f /usr/share/jenkins/ref/jenkins-plugins.txt --verbose' returned a non-zero code: 1

Optional dependencies aren't handled correctly

I just installed some plugins with this tool (without using --latest) and got this error:

Jenkins Git plugin (3.9.3)
Update required: Jenkins Parameterized Trigger plugin (2.21) to be updated to 2.33 or higher
Jenkins promoted builds plugin (3.5)
Update required: Config File Provider Plugin (2.16.4) to be updated to 2.18 or higher

These dependencies are optional.
Looks like Jenkins Parameterized Trigger plugin (2.21) and Config File Provider Plugin (2.16.4) were required by some other plugins.
And Jenkins expects optional dependency not to be installed at all or to have specified version.

Jenkins version 2.222.3.

PluginDependencyStrategyException - Considers 'latest' as an older version

Steps to recreate issue:

Create a plugins.txt file with the below 2 lines:

artifactory:3.7.3
gradle:latest

Run jenkins-plugin-cli. Verbose output below:

jenkins-plugin-cli -l --verbose --no-download -f plugins_test.txt
File containing list of plugins to be downloaded: plugins_test.txt
Reading in plugins from plugins_test.txt

No directory to download plugins entered. Will use default of /usr/share/jenkins/ref/plugins
Using update center https://updates.jenkins.io/update-center.json from JENKINS_UC environment variable
Using experimental update center https://updates.jenkins.io/experimental/update-center.json from JENKINS_UC_EXPERIMENTAL environment variable
Using incrementals mirror https://repo.jenkins-ci.org/incrementals from JENKINS_INCREMENTALS_REPO_MIRROR environment variable
No CLI option or environment variable set for plugin info, using default of https://updates.jenkins.io/plugin-versions.json
No war entered. Will use default of /usr/share/jenkins/jenkins.war
Jenkins version: 2.249.1

Retrieving update center information
Returning cached value for: update-center-2.249.1
Returning cached value for: experimental-update-center-2.249.1
Returning cached value for: plugin-versions

artifactory depends on:
maven-plugin 3.7
workflow-cps 2.83
workflow-job 2.40
workflow-step-api 2.22
ant 1.11
apache-httpcomponents-client-4-api 4.5.10-2.0
config-file-provider 3.7.0
credentials 2.3.13
gradle 1.36
ivy 2.1
junit 1.37
mailer 1.32.1
matrix-project 1.17
run-condition 1.3
ssh-credentials 1.18.1
token-macro 2.12
io.jenkins.tools.pluginmanager.impl.PluginDependencyStrategyException: Plugin artifactory:3.7.3 depends on gradle:1.36, but there is an older version defined on the top level - gradle:latest
        at io.jenkins.tools.pluginmanager.impl.PluginManager.resolveRecursiveDependencies(PluginManager.java:843)
        at io.jenkins.tools.pluginmanager.impl.PluginManager.findPluginsAndDependencies(PluginManager.java:484)
        at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:147)
        at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:112)
        at io.jenkins.tools.pluginmanager.cli.Main.main(Main.java:63)
Plugin artifactory:3.7.3 depends on gradle:1.36, but there is an older version defined on the top level - gradle:latest

The code incorrectly considers latest as an older version to 1.36. However, flipping the dependencies in the plugins text file works correctly.

gradle:latest
artifactory:3.7.3

Note that this is a simplistic file and we have a huge number of plugins that we want to automatically install in our jenkins instances and would like to list our plugins in alphabetical order for better readability and management.

"Unable to create plugin directory" message does not contain a path

When there is no download directory specified, the operation fails without providing a path. It would be great to have one in the log

onenashev@LAPTOP-25K65LOS:/c/Users/Oleg/Documents/jenkins/demo/demo-jenkins-config-as-code$ java -jar jenkins-plugin-manager-1.0.2.jar --list -f plugins.txt --no-download
File containing list of plugins to be downloaded: plugins.txt
Reading in plugins from plugins.txt

No directory to download plugins entered. Will use default of /usr/share/jenkins/ref/plugins
No CLI option or environment variable set for update center, using default of https://updates.jenkins.io/update-center.actual.json
No CLI option or environment variable set for experimental update center, using default of https://updates.jenkins.io/experimental/update-center.actual.json
No CLI option or environment variable set for incrementals mirror, using default of https://repo.jenkins-ci.org/incrementals
No war entered. Will use default of /usr/share/jenkins/jenkins.war
Unable to create plugin directory

Allow downloading plugin via FTP

It is not possible to download .hpi file if it is on FTP server.

In plugins.txt:
...
my_plugin::ftp://example.com/jenkins_plugins/my_pluing.hpi
...

Error: Not able to resolve URL for plugin

Version 2.0.1 incorrectly resolves required versions in dependencies

Looks like there is a regression after switching to "download latest by default" in 2.0.0

2020-09-17 11:49:21
Step 4/4 : RUN java -jar /app/bin/jenkins-plugin-manager.jar   --war /app/jenkins/jenkins.war   --plugins docker-workflow:1.22 workflow-durable-task-step:2.35
2020-09-17 11:49:21

2020-09-17 11:49:21
 ---> Running in 90ef99b3c292
2020-09-17 11:49:22
Unable to open /app/jenkins/jenkins.war
2020-09-17 11:49:22
Unable to get version from war file
2020-09-17 11:49:26
War not found, installing all plugins: /app/jenkins/jenkins.war
2020-09-17 11:49:26
Plugin docker-workflow:1.22 depends on workflow-durable-task-step:2.36, but there is an older version defined on the top level - workflow-durable-task-step:2.35

At the same time, even the most recent release of Docker Pipeline plugin requires 2.35:

Subsequent installation of plugins

Hi,

i tried to install a single plugin on an existing jenkins server. Here is an example:

java -jar ${REF}/jenkins-plugin-manager.jar --war ${JENKINS_WAR} --latest false --plugin-download-directory ${JENKINS_HOME}/plugins --verbose --plugins startup-trigger-plugin:latest

${JENKINS_HOME}/plugins is already there with several plugins. When i run this command, the existing plugins will all being deleted and just the given (in this case startup-trigger-plugin) will be installed. Is it possible to install just the single plugin in addition to the existing (without bash'ing around with backing up the folder)?

I use plugin manager 2.3.0.

Here is the overall output of this experiment:

jenkins@544bbbf6fc9d:~/plugins$ ls
ace-editor                              configuration-as-code      emailext-template         junit                        pipeline-input-step               script-security          workflow-basic-steps
ace-editor.jpi                          configuration-as-code.jpi  emailext-template.jpi     junit.jpi                    pipeline-input-step.jpi           script-security.jpi      workflow-basic-steps.jpi
antisamy-markup-formatter               credentials                external-monitor-job      ldap                         pipeline-milestone-step           snakeyaml-api            workflow-cps
antisamy-markup-formatter.jpi           credentials-binding        external-monitor-job.jpi  ldap.jpi                     pipeline-milestone-step.jpi       snakeyaml-api.jpi        workflow-cps-global-lib
apache-httpcomponents-client-4-api      credentials-binding.jpi    git-client                lockable-resources           pipeline-model-api                ssh-agent                workflow-cps-global-lib.jpi
apache-httpcomponents-client-4-api.jpi  credentials.jpi            git-client.jpi            lockable-resources.jpi       pipeline-model-api.jpi            ssh-agent.jpi            workflow-cps.jpi
authentication-tokens                   display-url-api            git-server                mailer                       pipeline-model-definition         ssh-credentials          workflow-durable-task-step
authentication-tokens.jpi               display-url-api.jpi        git-server.jpi            mailer.jpi                   pipeline-model-definition.jpi     ssh-credentials.jpi      workflow-durable-task-step.jpi
bouncycastle-api                        docker-build-step          handlebars                matrix-auth                  pipeline-model-extensions         ssh-slaves               workflow-job
bouncycastle-api.jpi                    docker-build-step.jpi      handlebars.jpi            matrix-auth.jpi              pipeline-model-extensions.jpi     ssh-slaves.jpi           workflow-job.jpi
branch-api                              docker-commons             jackson2-api              matrix-project               pipeline-rest-api                 structs                  workflow-multibranch
branch-api.jpi                          docker-commons.jpi         jackson2-api.jpi          matrix-project.jpi           pipeline-rest-api.jpi             structs.jpi              workflow-multibranch.jpi
build-timeout                           docker-java-api            jaxb                      modernstatus                 pipeline-stage-step               token-macro              workflow-scm-step
build-timeout.jpi                       docker-java-api.jpi        jaxb.jpi                  modernstatus.jpi             pipeline-stage-step.jpi           token-macro.jpi          workflow-scm-step.jpi
build-timestamp                         docker-plugin              jdk-tool                  momentjs                     pipeline-stage-tags-metadata      trilead-api              workflow-step-api
build-timestamp.jpi                     docker-plugin.jpi          jdk-tool.jpi              momentjs.jpi                 pipeline-stage-tags-metadata.jpi  trilead-api.jpi          workflow-step-api.jpi
cloudbees-folder                        docker-workflow            job-dsl                   pam-auth                     pipeline-stage-view               windows-slaves           workflow-support
cloudbees-folder.jpi                    docker-workflow.jpi        job-dsl.jpi               pam-auth.jpi                 pipeline-stage-view.jpi           windows-slaves.jpi       workflow-support.jpi
command-launcher                        durable-task               jquery-detached           pipeline-build-step          plain-credentials                 workflow-aggregator
command-launcher.jpi                    durable-task.jpi           jquery-detached.jpi       pipeline-build-step.jpi      plain-credentials.jpi             workflow-aggregator.jpi
config-file-provider                    email-ext                  jsch                      pipeline-graph-analysis      scm-api                           workflow-api
config-file-provider.jpi                email-ext.jpi              jsch.jpi                  pipeline-graph-analysis.jpi  scm-api.jpi                       workflow-api.jpi
jenkins@544bbbf6fc9d:~/plugins$ java -jar ${REF}/jenkins-plugin-manager.jar --war ${JENKINS_WAR} --latest false --plugin-download-directory ${JENKINS_HOME}/plugins --verbose --plugins startup-trigger-plugin:latest
Picked up JAVA_TOOL_OPTIONS: -Djavax.net.ssl.trustStore=/etc/ssl/certs.jks
No .txt or .yaml file containing list of plugins to be downloaded entered.
Plugin download location: /data/jenkins/plugins
No CLI option or environment variable set for update center, using default of https://updates.jenkins.io/update-center.json
No CLI option or environment variable set for experimental update center, using default of https://updates.jenkins.io/experimental/update-center.json
No CLI option or environment variable set for incrementals mirror, using default of https://repo.jenkins-ci.org/incrementals
No CLI option or environment variable set for plugin info, using default of https://updates.jenkins.io/plugin-versions.json
Will use war file: /usr/share/jenkins/jenkins.war
Jenkins version: 2.265

Retrieving update center information
Returning cached value for: update-center-2.265
Returning cached value for: experimental-update-center-2.265
Returning cached value for: plugin-versions
Couldn't find checksum for startup-trigger-plugin at version: latest

startup-trigger-plugin has no dependencies
Setting checksum for: startup-trigger-plugin to bnC0ncuO7CsH3u2ta/yQ9/BX1Efx8vjyxbFBtjhZWNw=
Will use url: https://updates.jenkins.io/download/plugins/startup-trigger-plugin/2.9.3/startup-trigger-plugin.hpi to download startup-trigger-plugin plugin

Downloading plugin startup-trigger-plugin from url: https://updates.jenkins.io/download/plugins/startup-trigger-plugin/2.9.3/startup-trigger-plugin.hpi
Checksum valid for: startup-trigger-plugin
startup-trigger-plugin downloaded successfully
Done
jenkins@544bbbf6fc9d:~/plugins$ ls
startup-trigger-plugin.jpi
jenkins@544bbbf6fc9d:~/plugins$

Thanks in advance

Regards Dave

Support to use the latest compatible version for a given plugin and jenkins core

Not sure if this is a valid requirement but I'd say #65 is a good way to prevent any issues when upgrading a plugin that requires a bump for the jenkins core.

I'd consider another use case when using latest should install the latest supported version for the given core. I don't know if that's something that was discarded in the past.

For instance:

Let's say, given the 2.252 core version and workflow-basic-steps as of today:

the latest version for workflow-basic-steps is 2.23 but it requires 2.263

$ curl -s https://updates.jenkins.io/current/update-center.actual.json | jq '.plugins["workflow-basic-steps"].version'
"2.23"
$ curl -s https://updates.jenkins.io/current/update-center.actual.json | jq '.plugins["workflow-basic-steps"].requiredCore'
"2.263"

Therefore when doing jenkins-plugin-cli --plugins workflow-basic-steps there is an error

io.jenkins.tools.pluginmanager.impl.VersionCompatibilityException: 
workflow-basic-steps (2.23) requires a greater version of Jenkins (2.263) than 2.252 in /usr/share/jenkins/jenkins.war
	at io.jenkins.tools.pluginmanager.impl.PluginManager.checkVersionCompatibility(PluginManager.java:430)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:153)
	at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:112)
	at io.jenkins.tools.pluginmanager.cli.Main.main(Main.java:63)

workflow-basic-steps (2.23) requires a greater version of Jenkins (2.263) than 2.252 in /usr/share/jenkins/jenkins.war

As far as I see the file plugin-versions.json contains all the versions and required core version too:

$ curl -s https://updates.jenkins.io/current/plugin-versions.json | jq -r '.plugins["workflow-basic-steps"]["2.23"].requiredCore'   
2.263
$ curl -s https://updates.jenkins.io/current/plugin-versions.json | jq -r '.plugins["workflow-basic-steps"]["2.22"].requiredCore' 
2.249.1

What I'd expect is given workflow-basic-steps:latest and jenkins-core 2.252 the upgrade should be done to 2.22 which it's the latest supported version for that particular core.

Is there any specific flag that can help me to accomplish that particular requirement? Or at leats, to avoid the upgrade of workflow-basic-steps:latest and move forward to the next plugins?

I've been using some combinations of flags but I cannot find if there is one for such.

Thanks

Unable to download plugin

Hello,

I'm running a dockerized Jenkins using the official image and installing my plugins this way:

jenkins-plugin-cli -f /usr/share/jenkins/ref/plugins.txt

It's been more than 12 hours than my builds (running on GitHub Actions) systematically fail to download all the plugins mentioned in the plugins.txt file. The error is always:

Unable to download [PLUGIN_NAME]

It is always a different plugin.

I tried with HTTPS and HTTP, with updates.jenkins.io and with a direct mirror address, it never worked.

Is there any way I can circomvent this issue?

Add support for generating a new plugin file based on current Jenkins folder

Ideally it should support the following outputs:

console (human readable)
txt
yaml

The --list flag along with the --output flags can probably be used for this

Also relevant #226 and #173

Available updates updates past max supported version

workflow-durable-task-step:2.35

with the current LTS 2.235.5 gets upgraded to 2.36 which requires 2.248

Add reference to the -help command when running without options

The tool fails when running without options. It would be better to print a message about the -help command in such case.

onenashev@LAPTOP-25K65LOS:/c/Users/Oleg/Documents/jenkins/demo/demo-jenkins-config-as-code$ java -jar jenkins-plugin-manager-1.0.2.jar
No .txt or .yaml file containing list of plugins to be downloaded entered.
No directory to download plugins entered. Will use default of /usr/share/jenkins/ref/plugins
No CLI option or environment variable set for update center, using default of https://updates.jenkins.io/update-center.actual.json
No CLI option or environment variable set for experimental update center, using default of https://updates.jenkins.io/experimental/update-center.actual.json
No CLI option or environment variable set for incrementals mirror, using default of https://repo.jenkins-ci.org/incrementals
No war entered. Will use default of /usr/share/jenkins/jenkins.war
Unable to create plugin directory

Internal hosted UC not supported

We are hosting update center in Nexus which has a self-signed certificate so when I run this command it fails with the following error:

io.jenkins.tools.pluginmanager.impl.UpdateCenterInfoRetrievalException: Error getting update center json at io.jenkins.tools.pluginmanager.impl.PluginManager.getJson(PluginManager.java:578) at io.jenkins.tools.pluginmanager.impl.PluginManager.getUCJson(PluginManager.java:597) at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:146) at io.jenkins.tools.pluginmanager.impl.PluginManager.start(PluginManager.java:113) at io.jenkins.tools.pluginmanager.cli.Main.main(Main.java:63) Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316) at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639) at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223) at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037) at sun.security.ssl.Handshaker.process_record(Handshaker.java:965) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379) at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498) at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:268) at java.net.URL.openStream(URL.java:1068) at org.apache.commons.io.IOUtils.toString(IOUtils.java:2795) at io.jenkins.tools.pluginmanager.impl.PluginManager.getJson(PluginManager.java:572) ... 4 more Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:450) at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:317) at sun.security.validator.Validator.validate(Validator.java:262) at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:330) at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237) at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621) ... 19 more Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:445) ... 25 more Error getting update center json

I tried adding our ca certificate to /usr/local/share/certificates and then running update-ca-certificates but it still doesn't work.
Perhaps we need a flag to ignore self-signed certificates eg when using the script I could pass CURL_OPTS -k

Running "jenkins-plugin-manager.jar --list" deletes all installed plugins

Running /usr/lib/jenkins-plugin-manager.jar --list deletes all installed plugins.

Example:

jenkins@676e1f797b5e:/$ ls -l $REF/plugins
total 2484
-rw-r--r-- 1 jenkins jenkins 2262190 Aug 24 19:06 configuration-as-code.jpi
-rw-r--r-- 1 jenkins jenkins  277413 Aug 24 19:06 snakeyaml-api.jpi

jenkins@676e1f797b5e:/$ java -jar /usr/lib/jenkins-plugin-manager.jar --list

Installed plugins:

Bundled plugins:

Set of all requested plugins:

Set of all requested plugins that will be downloaded:

Set of all existing plugins and plugins that will be downloaded:
Done

jenkins@676e1f797b5e:/$ ls -l $REF/plugins
total 0

jenkins@676e1f797b5e:/$ java -jar /usr/lib/jenkins-plugin-manager.jar --version
2.0.0

Jenkins plugin manager rejects plugins.txt.override file

We switched from install-plugins.sh to jenkins-plugin-manager in our Dockerfile and noticed that it doesn't support specifying the plugins.txt.override file as a parameter.

$ java -jar /usr/lib/jenkins-plugin-manager.jar -f /usr/share/jenkins/ref/plugins.txt.override
Unknown file type, file must have .yaml/.yml or .txt extension

We worked around the issue like this for the time being:

$ java -jar /usr/lib/jenkins-plugin-manager.jar -p $(< /usr/share/jenkins/ref/plugins.txt.override)

I think it would still make sense to allow .override files in this case...

Move Plugin Installation Manager issues from Jenkins Jira to GitHub Issues

When the project started, it was using Jenkins Jira instead of GitHub Issues. Many issues are still open here: https://issues.jenkins-ci.org/issues/?jql=project%20%3D%20JENKINS%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20component%20%3D%20plugin-installation-manager-tool%20 . It would be great to scrub the issues and to move relevant ones to GitHub Issues

Automate attaching jar to github releases during release

plugin-installation-manager add '-plugin' suffix to my plugins files

Hi,
I am trying to install plugins from a plugins txt file and it adds '-plugin' suffix for every plugin and as a result it does not appear in the jenkins update center.
e.g:

Downloading plugin conditional-buildstep-plugin from url: https://updates.jenkins.io/download/plugins/conditional-buildstep-plugin/1.3.6/conditional-buildstep-plugin.hpi

while in plugins.txt file:
conditional-buildstep:1.3.6

Command:
java -jar plugin-management-cli/target/jenkins-plugin-manager-1.0.3-SNAPSHOT.jar --war ../jenkins.war --plugin-file ../plugins.txt

--latest ignores whether the plugin is actually available or not

If I want to resolve the latest dependencies for my Jenkins plugins I often run into a scenario where a plugin is in the process of being bumped to a newer version (by the maintainer), and this is reflected on the jenkins update center as a new directory, however the actual jpi hasn't been uploaded yet so it fails to download. This is troublesome as if I test that the plugins are available in a test environment before rolling to production, there could be a plugin version bump in progress that causes a failure in the time between testing in the test environment and moving it to production.

Perhaps a --latest-available flag that checks if there is a jpi available and if not rolls to the previous release of a plugin would avoid this? To play devils advocate, I can't see why someone would want this behaviour using --latest in the first place.

Add support for Basic HTTP authentication

We have developed some in-house Jenkins plugins to suit some specific use cases in our company, which aren't really general purpose enough to merit the update center (i.e we rely on some local implementation details to speed development and avoid generalising the problem too much).

As such the plugins are hosted on our Artifactory service which requires authorisation to access.

As the install-plugins.sh script used curl under the hood, we were able to auth via a .netrc file. Unfortunately we aren't able to migrate to this tool as I don't think it supports any auth mechanism.

Updates check fails due to the missing uithemes plugin

UI Themes plugin is hidden in the update center, and not mentioned in the dependency tree, but the https://github.com/jenkins-infra/update-center2/blob/434567c532f60c87d6d056e04b3884daf42deb0e/resources/artifact-ignores.properties#L139

Steps to reproduce

Check out https://github.com/oleg-nenashev/demo-jenkins-config-as-code
Run make show-updates. It will invoke java -jar tmp/jenkins-plugin-manager.jar --list -f plugins.txt --no-download -d tmp/plugins -w tmp/jenkins.war --available-updates

Diagnostics info

Available updates:
github-branch-source (2.7.1) has an available update: 2.8.0
snakeyaml-api (1.26.1) has an available update: 1.26.4
pipeline-stage-tags-metadata (1.6.0) has an available update: 1.7.0
manage-permission (1.0) has an available update: 1.0.1
pipeline-model-definition (1.6.0) has an available update: 1.7.0
data-tables-api (1.10.20-14) has an available update: 1.10.21-1
github-api (1.111) has an available update: 1.112.0
echarts-api (4.6.0-8) has an available update: 4.7.0-4
Unable to find plugin uithemes in update center https://updates.jenkins.io/update-center.actual.json

Plugins.txt

ace-editor:1.1
authorize-project:1.3.0
blueocean:1.23.2
cloud-stats:0.25
cloudbees-folder:6.12
cobertura:1.16
code-coverage-api:1.1.6
configuration-as-code:1.41
docker-commons:1.16
docker-workflow:1.23
email-ext:2.69
extended-read-permission:3.2
favorite:2.3.2
filesystem_scm:2.1
git-client:3.2.1
git:4.2.2
github-api:1.111
github-branch-source:2.7.1
github:1.30.0
jackson2-api:2.11.0
jacoco:3.0.4
javadoc:1.5
jdk-tool:1.4
job-restrictions:0.8
jsch:0.1.55.2
junit:1.27
locale:1.4
lockable-resources:2.8
mailer:1.32
manage-permission:1.0
matrix-auth:2.6.1
maven-plugin:3.6
momentjs:1.1.1
monitoring:1.83.0
ownership:0.12.1
parallel-test-executor:1.12
parameterized-trigger:2.36
pipeline-build-step:2.12
pipeline-model-definition:1.6.0
pipeline-stage-tags-metadata:1.6.0
pipeline-stage-view:2.13
role-strategy:2.16
run-condition:1.3
scm-api:2.6.3
security-inspector:0.5
ssh-credentials:1.18.1
timestamper:1.11.3
token-macro:2.12
warnings-ng:8.1.0
workflow-basic-steps:2.20
workflow-cps:2.80
workflow-durable-task-step:2.35
workflow-job:2.39
workflow-multibranch:2.21
workflow-scm-step:2.11
yet-another-docker-plugin:0.2.0

Plugin Installation Manager should respect versions required on the top-level

Currently the tool may override a plugin version explicitly required by a user. It happens if there is a transient dependency which requires higher version. IMO the tool should fail the operation by default in such case. We could offer alternate strategies for more intelligent behavior.

Follow-up to https://github.com/jenkinsci/plugin-installation-manager-tool/pull/102/files#r440811107

plugin.txt format differs

install-plugins.sh takes the custom url as 4th parameter

https://github.com/jenkinsci/docker/blob/37a170fbb09fb50a63612551ca1d16f7752013bb/install-plugins.sh#L41-L44

But

plugin-installation-manager-tool/plugin-management-library/src/main/java/io/jenkins/tools/pluginmanager/util/PluginListParser.java

Lines 149 to 155 in d6be728

    
           private Plugin parsePluginLine(String pluginLine) { 
        
               String[] pluginInfo = pluginLine.split(":", 3); 
        
               String pluginName = pluginInfo[0]; 
        
               String pluginVersion = "latest"; 
        
               String pluginUrl = null; 
        
               String groupId = null;

ignores the lockfile parameter

so my existing plugins.txt with
html5-notifier:custom::https://files.halkeye.net/html5-notifier-plugin.hpi does not function

Is this intentional (should I update my plugins.txt) or accidental (bug and needs a PR)

[Request] Pin/lock versions of specified plugins and all sub-dependencies

I would like the Plugin Installation Manager Tool to be able to pin/lock the versions of all the plugins given on the command-line or in a file as well as their sub-dependencies. This behavior would imitate the locking functionality of tools like Bundler, Gradle, and Yarn.

One major benefit of locking all plugin versions is that the Jenkins configuration becomes 100% reproducible. Another benefit is that security scanning of all plugins and sub-dependencies becomes possible. Here is an example of what this pinning/locking might look like in a CI pipeline:

# Input file.
cat <<"EOF" >plugins.txt
blueocean:1.21.0
junit:latest
EOF
# Lock all plugins and sub-dependencies from specs in plugins.txt and output to plugins.yml.
java -jar plugin-management-cli.jar --lock-file plugins.yml --plugin-file plugins.txt
# View security warnings for all locked plugins and sub-dependencies.
java -jar plugin-management-cli.jar --no-download --plugin-file plugins.yml --view-security-warnings

The output YAML file would contain all the plugins from the input file and their sub-dependencies locked to specific versions:

plugins:
  - artifactId: blueocean
    source:
      version: 1.21.0
# All of blueocean:1.21.0's sub-dependencies would be locked here.
  - artifactId: junit
    source:
      version: 1.28 # This is the latest version of the junit plugin at time of writing.
# All of junit:1.28's sub-dependencies would be locked here.

Support Maven pom with dependencies as input to the tool

We would like to propose to support Maven pom file as an input to the plugin manager tool.
E.g.

<dependency>
    <groupId>org.jenkins-ci.plugins</groupId>
    <artifactId>git</artifactId>
    <version>4.4.1</version>
</dependency>

This way we can use dependabot in our repository to check for updates and create PRs, if an update is available. At the moment keeping an eye on the updates requires too much of our time, the same can be said regarding the equivalent implementation of the dependabot using the plugin manager tool.

We love reusing/contributing to existing tools.

What do you think about it? Have you tried anything similar already? Have we missed an important detail, that can block the implementation of this feature?

Plugin validation is incorrect

I get an error message when running the plugin-installation-manager:

Plugin gradle:1.35 depends on workflow-basic-steps:2.23, but there is an older version defined on the top level - workflow-basic-steps:2.22

The problem is that gradle:1.35 dependency is declared as: Pipeline: Basic Steps ≥ 2.16.1 (see site: https://plugins.jenkins.io/gradle/#dependencies)

I need to use Pipeline: Basic Steps 2.22 because it is compatible with the version of Jenkins I'm using. Going to 2.23 requires my Jenkins to be more advanced than the LTS release. That also means I can't use the set up with the jenkinsci/jenkinsfile-runner docker component.

Just because Pipeline: Basic Steps 2.23 was released 2 weeks ago and now requires Jenkins version 2.263 doesn't mean I want to go to that version. By policy and self defense our team needs to stay on the LTS release path. The latest LTS version of Jenkins is 2.249.3 at the time this bug is introduced.

Make --latest the default

--latest only affects dependency resolution

It's not great to request a plugin and to see it install 5 other dependencies and see that they now need updating, which leads to either having to specify every plugin, or learning about the --latest flag.

Proposal:

Changing the flag to take an argument:

--latest=true
--latest=false

And have it default to true

Not able to download plugins through private repo

Trying to use the tool behind company firewall by using Artifactory as a proxy.
Have been able to download update json files by using JENKINS_PLUGIN_INFO, JENKINS_INCREMENTALS_REPO_MIRROR, JENKINS_UC_EXPERIMENTAL, JENKINS_UC environment variables.

But for the actual download of the plugins it is always trying wrong url. Either trying the direkte url to the Jenkins update center or wrong path pointing at company Artifactory.

If it is a bug or missing feature, not sure.

Maybe use something like JENKINS_UC_DOWNLOAD environment variable or add a new command line flag to enable download plugins from company artifact proxy.

Split CLI to subcommands (and migrate to picocli)

Plugin Installation Manager Tool implements multiple commands at the moment, and options are a bit confusing. The tool would benefit from sub-commands (install-plugins, list-updates) that could simplify the interface for users and allow to easily extend it going forward.

Such a migration would also allow using Plugin Installation Manager as a subcommand on its own, e.g. inside Jenkinsfile Runner CLI which needs plugin management.

FTR https://groups.google.com/g/jenkinsci-dev/c/jTkOpdZ7z1s for the discussion. Reference migration: jenkinsci/jenkinsfile-runner#432

Update tests to use the SystemOutRule rule

Currently several tests redirect the STDOUT to verify the output. In such case the tool's output does not get displayed in the execution log. Instead of that, it would be better to use a special JUnit Rule which can capture the output.

See the example in #102

    @Rule
    public final SystemOutRule systemOutRule = new SystemOutRule().enableLog();

    assertThat(systemOutRule.getLog(), not(containsString("uithemes")));

jenkins-update-center parameter does not take complete URL

Trying to use this tool in stead of the old shell script, however I find it strange that it does not take the complete customized URLs.

I'm executing this in my Dockerfile:

JENKINS_UC=$ARTIFACTORY_URL/jenkins-update-remote
...
jenkins-plugin-cli --plugin-file /usr/share/jenkins/ref/plugins.txt --jenkins-update-center https://artifactory.domain.local/artifactory/jenkins-update-remote/update-center.json --verbose

Using JENKINS_UC works in the old shell script, however it looks like it is completely ignored, and thus the need for a "manullay" inserted URL for --jenkins-update-center.

As for experimental and incremental, I have no use for them. All my plugins in plugins.txt are versioned, i.e.

# Active Directory Plugin
active-directory:2.20

# Amazon Elastic Container Service Plugin
amazon-ecs:1.37

# Anchore Container Image Scanner Plugin
anchore-container-scanner:1.0.22

# Amazon Web Services SDK Plugin
aws-java-sdk:1.11.854

...

And the verbose output starts like this, which look fine to me:

File containing list of plugins to be downloaded: /usr/share/jenkins/ref/plugins.txt
Reading in plugins from /usr/share/jenkins/ref/plugins.txt

No directory to download plugins entered. Will use default of /usr/share/jenkins/ref/plugins
Using update center https://artifactory.domain.local/artifactory/jenkins-update-remote/update-center.json specified with CLI option
Using experimental update center https://updates.jenkins.io/experimental/update-center.json from JENKINS_UC_EXPERIMENTAL environment variable
Using incrementals mirror https://repo.jenkins-ci.org/incrementals from JENKINS_INCREMENTALS_REPO_MIRROR environment variable
No CLI option or environment variable set for plugin info, using default of https://updates.jenkins.io/plugin-versions.json
No war entered. Will use default of /usr/share/jenkins/jenkins.war
Jenkins version: 2.268

Retrieving update center information
Created cache at: /root/.cache/jenkins-plugin-management-cli
Cache miss for: update-center-2.268
Cache miss for: experimental-update-center-2.268
Cache miss for: plugin-versions

Then the actual output is (partial output only):

Downloading plugin pipeline-milestone-step from url: https://artifactory.domain.local/download/plugins/pipeline-milestone-step/1.3.1/pipeline-milestone-step.hpi
Unable to resolve plugin URL https://artifactory.domain.local/download/plugins/resource-disposer/0.14/resource-disposer.hpi, or download plugin resource-disposer to file
Will use url: https://artifactory.domain.local/download/plugins/job-dsl/1.77/job-dsl.hpi to download job-dsl plugin

Downloading plugin job-dsl from url: https://artifactory.domain.local/download/plugins/job-dsl/1.77/job-dsl.hpi
Unable to resolve plugin URL https://artifactory.domain.local/download/plugins/jacoco/3.1.0/jacoco.hpi, or download plugin jacoco to file
Will use url: https://artifactory.domain.local/download/plugins/authorize-project/1.3.0/authorize-project.hpi to download authorize-project plugin

Downloading plugin authorize-project from url: https://artifactory.domain.local/download/plugins/authorize-project/1.3.0/authorize-project.hpi
io.jenkins.tools.pluginmanager.impl.DownloadPluginException: Unable to download workflow-scm-step
        at io.jenkins.tools.pluginmanager.impl.PluginManager.lambda$null$5(PluginManager.java:457)
        at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
        at java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
        at java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:731)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Unable to download workflow-scm-step
Unable to resolve plugin URL https://artifactory.domain.local/download/plugins/workflow-multibranch/2.22/workflow-multibranch.hpi, or download plugin workflow-multibranch to file
Will use url: https://artifactory.domain.local/download/plugins/active-directory/2.20/active-directory.hpi to download active-directory plugin

Downloading plugin active-directory from url: https://artifactory.domain.local/download/plugins/active-directory/2.20/active-directory.hpi
Unable to resolve plugin URL https://artifactory.domain.local/download/plugins/blueocean-rest-impl/1.24.3/blueocean-rest-impl.hpi, or download plugin blueocean-rest-impl to file
Will use url: https://artifactory.domain.local/download/plugins/run-condition/1.3/run-condition.hpi to download run-condition plugin

"Available updates" shows updates if 'latest' is used

jenkins-plugin-cli --available-updates -f plugins.txt
... shows IMHO all plugins, including those that are definitely already on latest version (i.e. up-to-date):

Available updates:
ace-editor (latest) has an available update: 1.1
analysis-model-api (latest) has an available update: 9.1.0
antisamy-markup-formatter (latest) has an available update: 2.1
blueocean-autofavorite (latest) has an available update: 1.2.4
blueocean-display-url (latest) has an available update: 2.4.0
blueocean (latest) has an available update: 1.24.1
bouncycastle-api (latest) has an available update: 2.18
branch-api (latest) has an available update: 2.6.0
...

E.g. "Branch API v2.6.0" is (already) installed (in this version and is the latest available version) => Should not be shown for option "--available-updates" (I had hoped/expected)!?

Not all plugins are updated

I would like to use the plugin-installation-manager-tool to update plugins on a Jenkins instance, which seems to kind of work, but not completely. I am running the tool from Ansible like so (forgive the Jinja2 templating):

java -jar /opt/jenkins-pimt/jenkins-plugin-manager-1.0.2.jar -w {{ jenkins_war }} -f {{ jenkins_home }}/plugins.yaml -d {{ jenkins_home }}/plugins --available-updates --latest

After running the above command, Jenkins is restarted (just to be safe!), and then the tool is run again with the --no-download --list arguments. The output gets parsed by a separate Python script which regenerates plugins.yaml. In my local git diff, I can see that a number of plugins were successfully updated, such as credentials from 2.3.1 to 2.3.3. However, when I go to Jenkins' plugin management page, I see a number of other plugins still want to be updated:

If I re-run my Ansible playbook, these plugins don't get updated. I can't seem to figure out why some plugins can be updated but others cannot. All of the above plugins are listed in the plugins.yaml file. Why don't they get updated?

	@Option(name = "--view-security-warnings",
	usage = "Show if any security warnings exist for the requested plugins",
	handler = BooleanOptionHandler.class)
	private boolean showWarnings;

	private Plugin parsePluginLine(String pluginLine) {
	String[] pluginInfo = pluginLine.split(":", 3);
	String pluginName = pluginInfo[0];
	String pluginVersion = "latest";
	String pluginUrl = null;
	String groupId = null;