mserranom / bbrun Goto Github PK

Hi, I tried to use Private images hosted by AWS ECR (EC2 Container Registry) as you can see in documentation, and I receive an error.
The context is, I have a custom pipelines image, hosted in private AWS repository, and I want to use it, to build code with controled environment.

executing step in "[object Object]" docker: invalid reference format. See 'docker run --help'.
If I use a public image, everything works ok.
Thanks

Currently deployment to npm.com is performed after a master push in Travis.

After this change:

• Deployment should be performed after a tag is created (via github release)
• Package version is taken from tag name

If I run it interactively, it works.

My script called deploy.sh
bbrun $1 $2 $3 --pipeline branches:$BRANCH --env "AWS_SECRET_ACCESS_KEY=xxx, AWS_ACCESS_KEY_ID=yyy"

I made a script to basically copy everything to a /tmp area, and only them run bb in it. Else, it ends changing my project files, since I do npm install and I kind of change a few file contents depending on the deployment environment (dev / prod strings, stuff like that). But when running on Bitbucket, everything happens in a docker and gets disposed.

Anyway, the problem is that it fails directly. I also commented out my aws commands on yml file and it keeps saying "my aws command" is faulty. Interactively, it works like a charm.

At the moment I am at a feature branch, say "feature-1".
But I want to test the scripts for a different branch, say "dev".

I tried
bbrun --pipline branches:dev
and
bbrun --pipline dev

but only the default scripts will be executed.

The bitbucket-pipelines.yml looks like:

image: node:6.10.3
clone: 
  depth: 5
pipelines: 
     
  branches:
    dev:
     -step:
        scripts:
          - echo dev
    master:
     -step:
        scripts:
          - echo master
    default:
     -step:
        scripts:
          - echo default

Can you explain how i can tell bbrun to excute the dev scripts ?
Thank you for your help !

Many pipelines need a custom auth.json that will be passed as env to be secure. Ie:

        name: Initial Setup
        caches:
          - composer
        script:
          - echo "$AUTH" > auth.json
          - php -d memory_limit=-1 $(which composer) install --prefer-dist
        artifacts:
          - vendor/**

The parseVars function is very simple and need to be more complex for this scenarios.

using node:12-alpine as my base image
after running bbrun --pipeline branches:development I get the following error

  throw err;
  ^

Error: Cannot find module '/ws/bash'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:982:15)
    at Function.Module._load (internal/modules/cjs/loader.js:864:27)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)
    at internal/main/run_main_module.js:18:47 {
  code: 'MODULE_NOT_FOUND',
  requireStack: []
}```

I think Bitbucket use the environment Variable "CI" to indicate that the script is running on a pipeline.
Should bbrun automatically set CI to "true" ?

returning following error steps.forEach is not a function when running the following

command:

bbrun --template test-pipelines.yml

test-pipelines.yml

image: node:10.14.1

definitions:
  caches:
    node: node_modules
  steps:
    - step: &Test
        name: Run Tests
        caches:
          - node
        script:
          - npm run test:pipeline-ob
          - npm run test:pipeline-webauth
pipelines:
  default:
    "**":
      - step: &Test

Often a Bitbucket pipeline has some sort of deployment step that'll execute some SSH commands on a remote server, rsync some files etc.

At the moment debugging this scenario is quite difficult if you have a local Docker container acting as the "remote server", but the containers running each step don't have access to this "remote server" because they run on different Docker networks.

One way of tackling this is by giving the ability to specify the Docker network to run the containers in, an example would be:

bbrun --network example-network --env "SSH_HOST=example, SSH_USER=bitbucket

Notice the --network example-network, this would allow each step to have access to your container running outside of bbrun using the container name of the "remote server" (assuming that your container is "example" and connected to the "example-network").

It would be beneficial to add a flag for ssh keys, because a pipeline could be installing a private package.

BBrun should support persisting files between steps. As it stands, each call to src/docker.js#run runs a separate container, erasing created files.

Please document which node version needed to use bbrun
I found out, that node 12.0.0 is needed?

• Create standalone binary
• Create brew formula
• Upload in every deployment

As in:

options:
  size: 2x  # all steps in this repo get 8GB memory
 
pipelines:
  default:
    - step:
        size: 2x  # or configure at step level
        script:
          - ...

More informatio in Pipelines stocking stuffers: test reporting, Docker run and large builds

Currently v0.8.0 is the version of bbrun listed on npm and yarn. Ideally, this could be solved alongside #14.

I managed to create a null step in an otherwise valid bitbucket-pipelines.yml, and got the uninformative error above. I'm proposing (1) guarding each of the config[attr] || inScopeAttr in bbrun.js and (2) providing a traceback to the null/undefined part of a pipeline.

Problem: The engines dependency is too strict in package.json, requiring node 8.5.0 exactly.

Recommended Solution: Be more forgiving. "node": ">= 8.5.0"

Current Behaviour:
When the --interactive flag is passed, the container is run with /bin/bash. However in the course of a normal pipeline run, sh is used though the .bbrun.sh file has it's first line pointing to bash via #!/usr/bin/env bash.
This causes bash specific features like the source command that's used to run a script that doesn't have executable permission to fail (like source venv/bin/activate that's common in python projects)

Expected Behaviour:
Since we already use /bin/bash in the interactive shell, we could use /bin/bash for non-interactive mode script execution as well.

I started a bbrun without parameters and the fist default step was successful.
But then also another default step wich has deployment: staging & trigger: manual set.
This should be prevented!

attached you will find a test bitbucket-pipline.yml:

pipelines:
  default:
    - step:
        name: Build & Test
        script:
          - /bin/echo "Build & Test"
    - step:
        name: Deploy to STAGING
        deployment: staging
        trigger: manual
        script:
          - /bin/echo "Deploy to STAGING - this should not be run!"
    - step:
        name: Deploy to LIVE
        deployment: production
        trigger: manual
        script:
          - /bin/echo "Deploy to LIVE - this should not be run!"

For expediency, bitbucket pipelines support caching directories between steps. bbrun should be able to emulate this.

Would be helpful to have examples available for documentation

How can I pass --net=host to "docker run"?
Additionally, I think that if you send this all the time, there would be no harm at all even if you have a direct internet connection.

After #19 was fixed, this no longer breaks the builds, but builds are executed sequentially. They should be executed in parallel.

As described in Use services and databases in Bitbucket Pipelines

Both in README.md, and when running bbrun --help the --pipelines argument is not correct.

First off, thank you for your work creating this tools, it's made debugging BB pipelines locally so much easier.

I noticed that the working directory always seems to be /ws even if a different one is specified in the Dockerfile WORKDIR. Is there a command line flag or other option that can be setting to specific this?

I appreciate any help you can give me.

Hello, this is not related to your add-on more a docker thing. but maybe you know the answer... is it possible to debug a pipeline script like in vagrant? In vagrant i can enter "vagrant ssh" and can go into the machine. Currently it sucks very when i wait 20 minutes for execution and then everything breaks up because of a error in the build script. I mean thanks to your module its much better then real testing but a command for going into the machine would be awesome.

Bitbucket can run steps in parallel (docs). Currently, bbrun expects each item in a pipeline to contain a script member. This causes parallel: [...steps] to fail.

Currently docker commands are run with --rm option, that destroys the container after execution.

A --keep-container will not destroy the containers, which is useful for debugging purposes.

Hi @mserranom . I think I got all access to everything.
On NPM the organization, for me, is empty.

There is the release.sh script, which I would assume to run locally to get something released to GH.
You also have Travis config file.

Can I think if GitHub actions, from Master, calling all steps AND releasing the package to NPMJS?
One thing I ended using in a few Node projects was to set the package.json version ONLY when releasing (like it was 1.0.0 and at release time become 1.0.20200120) and never push this back to the repo. Instead, just tag the repo with v1.0.20200120. Everything gets automatic then, and releases can be fully automatic from GH itself. Travis would not be needed anymore (or use Travis and forget GH).

What do you think? I would like to have latest master released to NPMJS.

Best
Marco

Current Behavior:
Currently each time a pipeline is run, one docker container per step is created. The steps may install packages that utilize disk space. At the end of pipeline completion the containers aren't removed.

Expected Behavior:
After completion of pipeline run, the containers should be removed from docker, cleaning up any space utilized.

Used the hello-world bitbucket-pipelines.ymland changed the docker image to docker:stable
See more here

So the file ends-up like this:

pipelines:
  default:
    - step:
          image: docker:stable
          script:
            - echo "hello world!"

After running bbrun I get the following error:

executing step in "docker:stable"
/usr/local/bin/docker-entrypoint.sh: exec: line 35: bash: not found

If you check the line 35 of docker-entrypoint.sh file in the docker image repo you see it's just an "exec" command.

Checking on docker.js file, I see it uses bash as shell and Alpine Linux doesn't uses bash it uses sh instead.

I've tested changing the bash references with sh on docker.js and it works both for Ubuntu and Alpine images. Ideally we should point the shell to be used dynamically. Maybe using $SHELL or $0 enviroment variables. I've tried quickly but couldn't make it work. So I'll submit a Pull Request with what I have shortly.

Currently the execution environment doesn't support running docker commands against a docker daemon.

Potentially we can use a local Docker daemon and add support as described in Run Docker commands in Bitbucket Pipelines

That particular container, which I use on bitbucket, works perfectly nice there.
But when running locally, I got an error.

Why? Because there is a mandatory entrypoint which runs and fails, perhaps because it is not inside Pipelines.

Solution: remove entrypoint on bbrun ... I am not sure if this has side effects, but by doing that, my problem was gone.

docker.js : 36
:run --rm -P --entrypoint="" -v ${pwd()}:${workDir} -w ${workDir} ${image} bash ${BUILD_SCRIPT};