mlabouardy / butler

Export/Import Jenkins jobs & plugins 📤

CLI to import/export Jenkins jobs & plugins.

Usage

Download

Below are the available downloads for the latest version of Butler (1.0.0). Please download the proper package for your operating system and architecture.

Linux:

wget https://s3.us-east-1.amazonaws.com/butlercli/1.0.0/linux/butler

Windows:

wget https://s3.us-east-1.amazonaws.com/butlercli/1.0.0/windows/butler

Mac OS X:

wget https://s3.us-east-1.amazonaws.com/butlercli/1.0.0/osx/butler

OpenBSD:

wget https://s3.us-east-1.amazonaws.com/butlercli/1.0.0/openbsd/butler

FreeBSD:

wget https://s3.us-east-1.amazonaws.com/butlercli/1.0.0/freebsd/butler

Installation

To install the library and command line program, use the following:

go get -u github.com/mlabouardy/butler

Available Commands

Generic environment variables

Username flag may also be provided via environment variable JENKINS_USER and the password via JENKINS_PASSWORD. In order to always skip folders, you may set the environment variable JENKINS_SKIP_FOLDER.

Jobs Management

$ butler jobs export --server localhost:8080 --skip-folder

$ butler jobs import --server localhost:8080

Plugins Management

$ butler plugins export --server localhost:8080

$ butler plugins import --server localhost:8080

Credentials Management

$ butler credentials decrypt --server localhost:8080 --folder foo/bar > decryptedCredentials.json

$ cat decryptedCredentials.json | butler credentials apply --server localhost:8080 --folder bar/foo

Tutorials

• Butler CLI: Import/Export Jenkins Plugins & Jobs

butler's Issues

x509: certificate signed by unknown authority

Hi there,

Description

I just noticed that butler fails to import jobs, when Jenkins runs behind a self signed TLS certificate:

macbookpro:jenkins phil$ butler jobs import --server jenkins.example.com --username admin --password <my-secret> 
Import job: Git_Sync
Get https://jenkins.example.com:443/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb): x509: certificate signed by unknown authority

Related Issue

None

Changes proposed

Would it be possible to add an argument --skip-tls-check or something similar do disable the check if the TLS certificate is officially signed?

Screenshots

None

Thanks

Regards,
Philip

GetCrumb

Description

Not to much of an issue. But I had a rare case on a Jenkins installation where I had disabled CSRF & tried running butler jobs import & received a 404 response.

Maybe instead of saying Not found 404 - indicate that the program was unable to retrieve a crumb ( because some people disable it ).

Related Issue

Changes proposed

Screenshots

X 509 certificate issue

Description

is there any way to pass nocertificatecheck parameter ,since our Jenkins uses self signed when i run butler command it says "Get https://xxxxxx8443/api/xml: x509: certificate signed by unknown authority"

Related Issue

Changes proposed

-noCertificateCheck as parameter to butler

Screenshots

Utility forces http (no SSL support?)

Description

Jenkins can and should be run over SSL/https. This utility does not allow you to specify https. Does it handle the protocol?

./butler jobs export --server https://myhost:port--username mtdeguzis --password $(read -s password && echo $password)

Get http://https//myhost:port/view/all/api/xml: dial tcp: lookup https on *************:53: no such host

Related Issue

Changes proposed

Screenshots

Butler is using system Jenkins URL internally and fails if it does not match the server URL argument

Description

Our project Jenkins is set up for external access over an SSH tunnel only. Jenkins URL (Manage System > Jenkins Location) = http://localhost:58080/: the tunneled URL.
Locally on the same server, Jenkins is available over http://localhost:8080. If I try to run jobs export on the same server over http://localhost:8080, Butler fails.

Workaround: configure Jenkins temporarily to use the same URL.

Related Issue

N/A

Changes proposed

Fix this

Screenshots

Terminal log:

$ butler jobs export --server localhost:8080 --username {user} --password {password}
Exporting job: CaseStudy-BLAST-generate-dataset
Get http://localhost:58080/job/CaseStudy-BLAST-generate-dataset//config.xml: dial tcp [::1]:58080: getsockopt: connection refused

Migrating a set of jobs or just one job?

Does this have the ability to target one job or a set of jobs? A --job or --jobs (with nargs * for a list) would be really nice. I want to have a Dev to Prod workflow, but not import/export all jobs.

Not recursive

Description

When using Folders plugin and Multibranch plugin the structure become a tree like:
jobs:

• foldernameA:
  • multibranch-project1:
    • branchA
    • branchB
  • multibranch-project2:
    • branchX
    • branchY
• foldernameB
  ...

Butler is exporting only the foldernameA and foldernameB xml that does not contain the rest of the tree

Changes proposed

add --recursive with short form -r and export the whole tree when present

Or do it recursive by default and add --no-recursive as option

404 while downloading the utility from S3 bucket

Description

The utility is no longer available on the S3 bucket URL mentioned in the README

Screenshots

Cant export from Jenkins ver. 1.643 and no output at all

Description

I am trying to export the jobs from a Jenkins ver. 1.643 installation, but when i run the command i get absolutely not output at all. Using the same command against a 2.121 instance works. Does this mean that butler only works for jenkins >=2?

Export jobs are not running now for jobs

Earlier, I used to run the export tasks using butler.Now, recently when I check today, the export tasks are not running for jobs.But working for plugins.

And the plugins exported, I tried to import but getting import errors like for email-ext.

Also, one more issue, how to manage export/import for jenkins user with singlesignon?

Getting Error message "plugins cannot be installed" while importing plugins on jenkins

@mlabouardy @icereed ### Description
While importing plugins, I am getting error message "plugins cannot be installed" on Jenkins version above 2.190 onwards.
Up to which version of Jenkins provides support ?

Related Issue

Changes proposed

If it doesn't provide the support then please provide support for higher version of Jenkins.

Screenshots

Enhancement Request: User / password secure handling

Description

Ideally there should be a prompt or method to read a file for user/password handling. Having this in my bash history isn't ideal.

Crude hackish workaround

./butler jobs export --server https://udajenkins:8444 --username mtdeguzis --password $(read -s password && echo $password)

Enhance diagnostics on job import failure

Description

I'm getting Job couldn't not be imported: check if it is already existing and verify all plugins used in this job are installed on the target jenkins instance.

There needs to be a way to get more details, otherwise this is very hard to troubleshoot. All the plugins used in the job seem to be in place. I went through the exercise of tweaking all the plugin versions in the imported config.xml to match the target server to no avail.

Related Issue

None

Changes proposed

Print additional (diagnostic) info on all failures.

Screenshots

Not needed

Unauthorized 401

Description

I keep getting Unauthorized 401 when I use the butler to import to my Jenkins .

my jenkins is

Related Issue

Changes proposed

Screenshots

Death project? any active fork?

Description

No commits since Dec. 2018

If there is any active fork please add the link in this discussion