circleci-public / circleci-config-generator

Script to generate a basic configuration file for CircleCI 2.0

CircleCI Config Generator

The repo contains a shell script to generate a basic configuration file for CircleCI 2.0

Reason to use this

If a project (git repo) is connected to CircleCI and has no circle.yml (for CircleCI 1.0) or .circleci/config.yml (for CircleCI 2.0) file, CircleCI will try to build on CircleCI 1.0 using 'inference' (we inspect the files and 'infer' what tests to run).

CircleCI 1.0 will be EOL soon (https://circleci.com/sunset1-0/). So all projects will need a .circleci/config.yml file to build on 2.0.

This script is helpful if you're building on 1.0 without a config file (you do not have a circle.yml file at the root of your project). It will help you get started on CircleCI 2.0.

Running this script locally from the root of your project will:

1. Create a new Git branch called circleci-20-test and check out that branch
2. Generate a .circleci/config.yml file in the new branch
3. Optionally push a new commit on that branch and build it on CircleCI 2.0

Requirements

You will need:

1. Git and Curl installed on your local computer
2. Project checked out on your local computer
3. Push access to your GitHub or Bitbucket repo
4. A CircleCI API token (see below)

This script makes use of the CircleCI 'config translation' tool which is documented here: https://circleci.com/docs/2.0/config-translation/

How to use

1. Download this script https://raw.githubusercontent.com/CircleCI-Public/circleci-config-generator/master/cci-config-generator.sh
2. Make sure the script is in the root of your project
3. Make the script executable: chmod +x cci-config-generator.sh
4. Create an API token in the CircleCI UI: https://circleci.com/account/api
5. Copy the API token to your clipboard
6. Run the script ./cci-config-generator.sh
7. Paste in your API token when prompted
8. The script will generate the config, and optionally create a branch and kick off a build

Did it work? Next steps

Visit your project page on CircleCI https://circleci.com/gh/ORG/REPO to see the new build that ran on CircleCI 2.0

If your build is green - congratulations you're now using CircleCI 2.0. Read the docs to take advantage of more CircleCI 2.0 features: https://circleci.com/docs/2.0/

It the build is red, you can troubleshoot any error messages using our docs: https://circleci.com/docs/2.0/

If you need more help, please open a support ticket here: https://support.circleci.com/hc/en-us/requests/new

circleci-config-generator's Issues

Does Not Reliably Detect Origin / Remotes

The current script does not properly handle repositories that have:

• Multiple remotes
• A remote that does not end its URL in .git.

For example. here's some output from git remote -v that is not parsed properly:

matt    [email protected]:matt/some-project (fetch)
matt    [email protected]:matt/some-project (push)
origin  [email protected]:GuyPaddock/some-project.git (fetch)
origin  [email protected]:GuyPaddock/some-project.git (push)
production      [email protected]:org/some-project.git (fetch)
production      [email protected]:org/some-project.git (push)

This leads to the following error:

fatal: remote error:
   is not a valid repository name
  Email [email protected] for help

It looks like the following line:

project="$(git remote -v | awk -F'[@:.]' 'NR==1{ print $4 }')"

Should be changed to:

project="$(git remote -v | grep origin | sed 's/\t/ /' | cut -d' ' -f2 | sort | uniq | awk -F'[@:.]' 'NR==1{ print $4 }')"

This ensures:

• The origin is used instead of the first repo that appears.
• The delimiters between fields is always a space (instead of tabs between most fields and a space between the remote URL and the (fetch) or (push) indicator.

php configuration translation problem

Hello,
Thank you for this nice tool :)

I used the config-translator to convert a PHP project from circleci 1.0 to 2.0. But I have the following error :

#!/bin/bash --login
ln -fs << $HOME/.phpenv/versions/7.0.24/libexec/apache2/libphp5.so/usr/lib/apache2/modules/libphp5.so;
phpenv global 7.0.24 2>/dev/null

/bin/bash: line 1: warning: here-document at line 0 delimited by end-of-file (wanted `$HOME/.phpenv/versions/7.0.24/libexec/apache2/libphp5.so/usr/lib/apache2/modules/libphp5.so')
ln: missing file operand
Try 'ln --help' for more information.
Exited with code 1

I think the translator did not translate

  php:
    version: 7.0.24

correctly. Would you please have an idea how to fix this ?

This error has already been posted many times but I couldn't find an answer :
https://discuss.circleci.com/t/config-translation-generated-broken-config/21094
https://discuss.circleci.com/t/upgrading-to-2-0-config-translation-error-deprecated-base-image/23866

Thank you very much ! :)

Mathieu.

--
Here is my old circle.yml :

machine:
  php:
    version: 7.0.24
  hosts:
    localhost: 127.0.0.1
  services:
    - mysql
dependencies:
  post:
    - wget https://phar.phpunit.de/phpunit-5.6.2.phar
    - chmod +x phpunit-5.6.2.phar
    - mv phpunit-5.6.2.phar phpunit
database:
  override:
    - mysql -u ubuntu -e "CREATE DATABASE db_test;SET @@global.sql_mode='';"
checkout:
  post:
    - cd library && git clone https://github.com/runkit7/runkit7.git && cd runkit7 && phpize && ./configure && make && make install && (echo "extension=runkit.so" && echo "runkit.internal_override=1") > /opt/circleci/php/7.0.24/etc/conf.d/runkit.ini
    - cd library && tar -zxf install.tar.gz
    - pear install channel://pear.php.net/XML_Serializer-0.21.0
test:
  override:
    - mkdir -p $CIRCLE_TEST_REPORTS/phpunit
    - ./phpunit --log-junit $CIRCLE_TEST_REPORTS/phpunit/junit.xml
- ./phpunit --coverage-html $CIRCLE_ARTIFACTS

And the translated file .circleci/config.yml :

# This configuration was automatically generated from a CircleCI 1.0 config.
# It should include any build commands you had along with commands that CircleCI
# inferred from your project structure. We strongly recommend you read all the
# comments in this file to understand the structure of CircleCI 2.0, as the idiom
# for configuration has changed substantially in 2.0 to allow arbitrary jobs rather
# than the prescribed lifecycle of 1.0. In general, we recommend using this generated
# configuration as a reference rather than using it in production, though in most
# cases it should duplicate the execution of your original 1.0 config.
version: 2
jobs:
  build:
    working_directory: ~/ircf/myproject
    parallelism: 1
    shell: /bin/bash --login
    # CircleCI 2.0 does not support environment variables that refer to each other the same way as 1.0 did.
    # If any of these refer to each other, rewrite them so that they don't or see https://circleci.com/docs/2.0/env-vars/#interpolating-environment-variables-to-set-other-environment-variables .
    environment:
      CIRCLE_ARTIFACTS: /tmp/circleci-artifacts
      CIRCLE_TEST_REPORTS: /tmp/circleci-test-results
    # In CircleCI 1.0 we used a pre-configured image with a large number of languages and other packages.
    # In CircleCI 2.0 you can now specify your own image, or use one of our pre-configured images.
    # The following configuration line tells CircleCI to use the specified docker image as the runtime environment for you job.
    # We have selected a pre-built image that mirrors the build environment we use on
    # the 1.0 platform, but we recommend you choose an image more tailored to the needs
    # of each job. For more information on choosing an image (or alternatively using a
    # VM instead of a container) see https://circleci.com/docs/2.0/executor-types/
    # To see the list of pre-built images that CircleCI provides for most common languages see
    # https://circleci.com/docs/2.0/circleci-images/
    docker:
    - image: circleci/build-image:ubuntu-14.04-XXL-upstart-1189-5614f37
      command: /sbin/init
    steps:
    # Machine Setup
    #   If you break your build into multiple jobs with workflows, you will probably want to do the parts of this that are relevant in each
    # The following `checkout` command checks out your code to your working directory. In 1.0 we did this implicitly. In 2.0 you can choose where in the course of a job your code should be checked out.
    - checkout
    # Prepare for artifact and test results  collection equivalent to how it was done on 1.0.
    # In many cases you can simplify this from what is generated here.
    # 'See docs on artifact collection here https://circleci.com/docs/2.0/artifacts/'
    - run: mkdir -p $CIRCLE_ARTIFACTS $CIRCLE_TEST_REPORTS
    # This is based on your 1.0 configuration file or project settings
    - run:
        working_directory: ~/ircf/myproject
        command: |-
          ln -fs << $HOME/.phpenv/versions/7.0.24/libexec/apache2/libphp5.so/usr/lib/apache2/modules/libphp5.so;
          phpenv global 7.0.24 2>/dev/null
    - run:
        working_directory: ~/ircf/myproject
        command: |-
          printf '127.0.0.1       localhost
          ' | sudo tee -a /etc/hosts
    - run:
        working_directory: ~/ircf/myproject
        command: 'sudo service mysql status || sudo service mysql start; '
    # Checkout
    #   This would typically go in either a build or a build-and-test job when using workflows
    # This is based on your 1.0 configuration file or project settings
    - run: cd library && git clone https://github.com/runkit7/runkit7.git && cd runkit7 && phpize && ./configure && make && make install && (echo "extension=runkit.so" && echo "runkit.internal_override=1") > /opt/circleci/php/7.0.24/etc/conf.d/runkit.ini
    - run: cd library && tar -zxf install.tar.gz
    - run: pear install channel://pear.php.net/XML_Serializer-0.21.0
    # Dependencies
    #   This would typically go in either a build or a build-and-test job when using workflows
    # Restore the dependency cache
    - restore_cache:
        keys:
        # This branch if available
        - v1-dep-{{ .Branch }}-
        # Default branch if not
        - v1-dep-master-
        # Any branch if there are none on the default branch - this should be unnecessary if you have your default branch configured correctly
        - v1-dep-
    # This is based on your 1.0 configuration file or project settings
    - run: wget https://phar.phpunit.de/phpunit-5.6.2.phar
    - run: chmod +x phpunit-5.6.2.phar
    - run: mv phpunit-5.6.2.phar phpunit
    # Save dependency cache
    - save_cache:
        key: v1-dep-{{ .Branch }}-{{ epoch }}
        paths:
        # This is a broad list of cache paths to include many possible development environments
        # You can probably delete some of these entries
        - vendor/bundle
        - ~/virtualenvs
        - ~/.m2
        - ~/.ivy2
        - ~/.bundle
        - ~/.go_workspace
        - ~/.gradle
        - ~/.cache/bower
    # This is based on your 1.0 configuration file or project settings
    - run: mysql -u ubuntu -e "CREATE DATABASE db_test;SET @@global.sql_mode='';"
    # Test
    #   This would typically be a build job when using workflows, possibly combined with build
    # This is based on your 1.0 configuration file or project settings
    - run: mkdir -p $CIRCLE_TEST_REPORTS/phpunit
    - run: ./phpunit --log-junit $CIRCLE_TEST_REPORTS/phpunit/junit.xml
    - run: ./phpunit --coverage-html $CIRCLE_ARTIFACTS
    # Teardown
    #   If you break your build into multiple jobs with workflows, you will probably want to do the parts of this that are relevant in each
    # Save test results
    - store_test_results:
        path: /tmp/circleci-test-results
    # Save artifacts
    - store_artifacts:
        path: /tmp/circleci-artifacts
    - store_artifacts:
        path: /tmp/circleci-test-results

Doesn't allow for projects with "." in their names.

Your awk expression doesn't allow for "." to be in the project name.

Does not support https:// repos

$ git remote -v

origin    https://github.com/me/repo (fetch)
origin    https://github.com/me/repo (push)
upstream  https://github.com/them/repo (fetch)
upstream  https://github.com/them/repo (push)

Fails (on Mac OSX) with...

$ ./cci-config-generator.sh
Gathering info please wait...
fatal: 'git@com/me/repo (fetch).com:.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Paste your CircleCI API token here:

README.md points to https://circleci.com/account/api that returns HTTP 404

Steps to reproduce:

1. Open https://github.com/CircleCI-Public/circleci-config-generator/blob/master/README.md
2. How-to use, step 4 has https://circleci.com/account/api

Actual result:

$ curl -I https://circleci.com/account/api
HTTP/1.1 404 Not Found

Expected result:
A working link

(I don't know valid expected link for instruction. I found only parametrized one for my account: https://circleci.com/gh///edit#api - and not yet sure if it's that one mentioned in instruction)