Coder Social home page Coder Social logo

gliderlabs / herokuish Goto Github PK

View Code? Open in Web Editor NEW
1.4K 26.0 151.0 2.71 MB

Utility for emulating Heroku build and runtime tasks in containers

License: MIT License

Shell 39.63% Go 2.55% Clojure 1.79% Java 4.62% Ruby 0.94% Python 5.42% JavaScript 0.65% PHP 0.68% HTML 14.98% Scala 8.66% Makefile 10.58% Dockerfile 2.30% CSS 1.50% Twig 4.88% Procfile 0.81%

herokuish's Introduction

herokuish

Build Status IRC Channel Docker Hub

A command line tool for emulating Heroku build and runtime tasks in containers.

Herokuish is made for platform authors. The project consolidates and decouples Heroku compatibility logic (running buildpacks, parsing Procfile) and supporting workflow (importing/exporting slugs) from specific platform images like those in Dokku/Buildstep, Deis, Flynn, etc.

The goal is to be the definitive, well maintained and heavily tested Heroku emulation utility shared by all. It is based on the Heroku:20, and Heroku:22 system images. Together they form a toolkit for achieving Heroku compatibility.

Herokuish is a community project and is in no way affiliated with Heroku.

Getting herokuish

Download and uncompress the latest binary tarball from releases.

For example, you can do this directly in your Dockerfiles installing into /bin as one step:

RUN curl --location --silent https://github.com/gliderlabs/herokuish/releases/download/v0.9.2/herokuish_0.9.2_linux_x86_64.tgz \
    | tar -xzC /bin

Herokuish depends on Bash (4.0 or newer) and a handful of standard GNU utilties you probably have. It likely won't work on Busybox, though neither will any Heroku buildpacks.

Using herokuish

Herokuish is meant to work behind the scenes inside a container. It tries not to force decisions about how you construct and operate containers. In fact, there's nothing that even ties it specifically to Docker. It focuses on neatly emulating Heroku, letting you design and orchestrate containers around it.

$ herokuish

Available commands:
  buildpack                Use and install buildpacks
    build                    Build an application using installed buildpacks
    install                  Install buildpack from Git URL and optional committish
    list                     List installed buildpacks
    test                     Build and run tests for an application using installed buildpacks
  help                     Shows help information for a command
  paths                    Shows path settings
  procfile                 Use Procfiles and run app commands
    exec                     Run as unprivileged user with Heroku-like env
    parse                    Get command string for a process type from Procfile
    start                    Run process type command from Procfile through exec
  slug                     Manage application slugs
    export                   Export generated slug tarball to URL (PUT) or STDOUT
    generate                 Generate a gzipped slug tarball from the current app
    import                   Import a gzipped slug tarball from URL or STDIN
  test                     Test running an app through Herokuish
  version                  Show version and supported version info

Main functionality revolves around buildpack commands, procfile/exec commands, and slug commands. They are made to work together, but can be used independently or not at all.

For example, build processes that produce Docker images without producing intermediary slugs can ignore slug commands. Similarly, non-buildpack runtime images such as google/python-runtime might find procfile commands useful just to support Procfiles.

herokuish exec will by default drop root privileges through use of setuidgid, but if already running as a non-root user setuidgid will fail, you can opt-out from this by setting the env-var HEROKUISH_SETUIDGUID=false.

Buildpacks

Herokuish does not come with any buildpacks, but it is tested against recent versions of Heroku supported buildpacks. You can see this information with herokuish version. Example output:

$ herokuish version
herokuish: 0.3.0
buildpacks:
  heroku-buildpack-multi     cddec34
  heroku-buildpack-nodejs    v60
  heroku-buildpack-php       v43
  heroku-buildpack-python    v52
  heroku-buildpack-ruby      v127
  ...

You can install all supported buildpacks with herokuish buildpack install, or you can manually install buildpacks individually with herokuish buildpack install <url> [committish]. You can also mount a directory containing your platform's supported buildpacks (see Paths, next section), or you could bake your supported buildpacks into an image. These are the types of decisions that are up to you.

Paths

Use herokuish paths to see relevant system paths it uses. You can use these to import or mount data for use inside a container. They can also be overridden by setting the appropriate environment variable.

$ herokuish paths
APP_PATH=/app                    # Application path during runtime
ENV_PATH=/tmp/env                # Path to files for defining base environment
BUILD_PATH=/tmp/build            # Working directory during builds
CACHE_PATH=/tmp/cache            # Buildpack cache location
IMPORT_PATH=/tmp/app             # Mounted path to copy to app path
BUILDPACK_PATH=/tmp/buildpacks   # Path to installed buildpacks

Entrypoints

Some subcommands are made to be used as default commands or entrypoint commands for containers. Specifically, herokuish detects if it was called as /start, /exec, or /build which will shortcut it to running those subcommands directly. This means you can either install the binary in those locations or create symlinks from those locations, allowing you to use them as your container entrypoint.

Help

Don't be afraid of the help command. It actually tells you exactly what a command does:

$ herokuish help slug export
slug-export <url>
  Export generated slug tarball to URL (PUT) or STDOUT

slug-export ()
{
    declare desc="Export generated slug tarball to URL (PUT) or STDOUT";
    declare url="$1";
    if [[ ! -f "$slug_path" ]]; then
        return 1;
    fi;
    if [[ -n "$url" ]]; then
        curl -0 -s -o /dev/null --retry 2 -X PUT -T "$slug_path" "$url";
    else
        cat "$slug_path";
    fi
}

Using Herokuish to test Heroku/Dokku apps

Having trouble pushing an app to Dokku or Heroku? Use Herokuish with a local Docker instance to debug. This is especially helpful with Dokku to help determine if it's a buildpack issue or an issue with Dokku. Buildpack issues should be filed against Herokuish.

Running an app against Herokuish

docker run --rm -v /abs/app/path:/tmp/app gliderlabs/herokuish /bin/herokuish test

Mounting your local app source directory to /tmp/app and running /bin/herokuish test will run your app through the buildpack compile process. Then it starts your web process and attempts to connect to the web root path. If it runs into a problem, it should exit non-zero.

::: BUILDING APP :::
-----> Ruby app detected
-----> Compiling Ruby/Rack
-----> Using Ruby version: ruby-1.9.3
  ...

You can use this output when you submit issues.

Running an app tests using Heroku buildpacks

docker run --rm -v /abs/app/path:/tmp/app gliderlabs/herokuish /bin/herokuish buildpack test

Mounting your local app source directory to /tmp/app and running /bin/herokuish buildpack test will run your app through the buildpack test-compile process. Then it will run test command to execute application tests.

-----> Ruby app detected
-----> Setting up Test for Ruby/Rack
-----> Using Ruby version: ruby-2.3.3
  ...
-----> Detecting rake tasks
-----> Running test: bundle exec rspec
       .
       Finished in 0.00239 seconds (files took 0.07525 seconds to load)
       1 example, 0 failures

If you are on macOS, you'll want to explicitly set the platform:

docker run --platform linux/amd64 --rm -v /abs/app/path:/tmp/app gliderlabs/herokuish /bin/herokuish buildpack test

However, there is a risk of compatibility issues when running on a different platform than the one you are developing on. If you are getting strange compilation or segfaults, try running the build process on an x86 platform.

Troubleshooting

If you run into an issue and looking for more insight into what herokuish is doing, you can set the $TRACE environment variable.

$ docker run --rm -e TRACE=true -v /abs/app/path:/tmp/app gliderlabs/herokuish /bin/herokuish test
+ [[ -d /tmp/app ]]
+ rm -rf /app
+ cp -r /tmp/app /app
+ cmd-export paths
+ declare 'desc=Exports a function as a command'
+ declare fn=paths as=paths
+ local ns=
++ cmd-list-ns
++ sort
++ grep -v :
++ for k in '"${!CMDS[@]}"'
++ echo :help
...
++ unprivileged /tmp/buildpacks/custom/bin/detect /tmp/build
++ setuidgid u33467 /tmp/buildpacks/custom/bin/detect /tmp/build
++ true
+ selected_name=
+ [[ -n /tmp/buildpacks/custom ]]
+ [[ -n '' ]]
+ title 'Unable to select a buildpack'
----->' Unable to select a buildpack
+ exit 1

You can also set a custom buildpack:

docker run -e BUILDPACK_URL="https://github.com/custom/buildpack.git#with-a-branch" -e STACK=heroku-20 -e TRACE=true --rm -v ./:/tmp/app -it gliderlabs/herokuish /bin/herokuish test

Note that the underlying buildpacks will not trace their commands with TRACE=true is enabled. They need to independently set set -x in order to trace execution.

Contributing

Pull requests are welcome! Herokuish is written in Bash and Go. Please conform to the Bash styleguide used for this project when writing Bash.

Developers should have Go installed with cross-compile support for Darwin and Linux. Tests will require Docker to be available. If you have OS X, we recommend boot2docker.

For help and discussion beyond Github Issues, join us on Freenode in #gliderlabs.

Releases

Anybody can propose a release. First bump the version in Makefile and Dockerfile, make sure CHANGELOG.md is up to date, and make sure tests are passing. Then open a Pull Request from master into the release branch. Once a maintainer approves and merges, Github Actions will build a release and upload it to Github.

Acknowledgements

This project was sponsored and made possible by the Deis Project.

That said, herokuish was designed based on the experience developing and re-developing Heroku compatibility in Dokku, Deis, and Flynn. Herokuish is based on code from all three projects, as such, thank you to all the contributors of those projects.

In fact, since I hope this is the final implementation of Heroku emulation I'm involved with, I'd like to finally thank Matt Freeman (@nonuby). I've been more or less copy-and-pasting code he originally wrote for the now defunct OpenRuko since 2012.

Lastly, thank you Heroku for pioneering such a great platform and inspiring all of us to try and take it further.

License

BSD

herokuish's People

Contributors

adzap avatar cellane avatar coffee2codenl avatar dependabot[bot] avatar dzaporozhets avatar friism avatar gjreasoner avatar iloveitaly avatar jesseshieh avatar johanneswuerbach avatar josegonzalez avatar joshmanders avatar karanthukral avatar mgood avatar michaelshobbs avatar miyucy avatar mlandauer avatar natanrolnik avatar nbyl avatar nolith avatar nschlemm avatar paperback avatar patrickjahns avatar peterhellberg avatar progrium avatar rvalyi avatar shelnutt2 avatar thenathanblack avatar tnir avatar webknjaz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

herokuish's Issues

Buildpack name and order

I'm... not wild about heroku-buildpack-multi being arbitrarily special-cased in the buildpack loop (https://github.com/gliderlabs/herokuish/blob/master/include/buildpack.bash#L76). I could well have a meta-buildpack similar in nature that I'd want to run before other buildpacks, or even before buildpack-multi.

I'd rather the build command take an optional list of buildpacks to execute (defaulting to using ls $BUILDPACKS_DIR, optionally with the special case for hoisting heroku-buildpack-multi). This is how I'm planning to handle buildpack execution order in Plushu: right now, each buildpack is installed with a numeric prefix on its directory name (which is hacky, fragile and hard to work with, which is why I'm planning to switch to the separate-list approach). Anyway, with that naming scheme, this heroku-buildpack-multi special case wouldn't even trigger.

This also lets platforms smoothly disable/switch out buildpacks for different build scenarios (maybe one user is on the "velvet deluxe Groovy build system" plan while one is on the "barebones cot-on-the-floor cheapskate Maven runner" plan).

Slowness on App start

I am using herokuish in dokku and there is a long delay before app starts. I have narrowed it down to this line:

https://github.com/gliderlabs/herokuish/blob/master/include/procfile.bash#L91

Since the container image is AUFS, the chown is causing every single file in /app to be modified, which means copying every file. Chown may be the most efficient under normal circumstances, but when it is causing the file to be copied due to a copy-on-write filesystem, it often takes 5-10 minutes. This even though there are only a few files that aren't already owned by the unprivileged user. Maybe overlay-fs would not have this issue - I don't have it set up anywhere to test.

I am testing a patch to do

find /app \! -user $unprivileged_user | xargs -0 -r chown $unprivileged_user:$unprivileged_group

Which at least prevents writing every file and in my case runs in a few seconds compared to 3-10 min.

Is this affecting others, or is there something I can do differently to avoid this?

latest nodejs buildpack fails with permission denied

Using this sample repo fork (https://github.com/michaelshobbs/node-js-sample) I get the following permission denied error when deploying with dokku.

-----> Checking startup method
       Found Procfile

-----> Finalizing build
       Creating runtime environment
remote: /tmp/buildpacks/custom/lib/build.sh: line 212: /tmp/buildpacks/custom/export: Permission denied
       Exporting binary paths

-----> Build failed

       We're sorry this build is failing! If you can't find the issue in application code,
       please submit a ticket so we can help: https://help.heroku.com/
       You can also try reverting to our legacy Node.js buildpack:
       heroku config:set BUILDPACK_URL=https://github.com/heroku/heroku-buildpack-nodejs#v63

       Love,
       Heroku
root@c14760c734ec:/# ls -la /tmp/buildpacks/
total 60
drwxr-xr-x 27 u29276 u29276 4096 Mar 30 23:18 .
drwxrwxrwt 13 root   root   4096 Mar 30 23:18 ..
drwxr-xr-x  6 root   root   4096 Mar 30 23:18 custom
drwxr-xr-x 10 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-clojure
drwxr-xr-x 12 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-go
drwxr-xr-x 12 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-gradle
drwxr-xr-x  6 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-grails
drwxr-xr-x 12 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-java
drwxr-xr-x  4 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-multi
drwxr-xr-x  8 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-nodejs
drwxr-xr-x  8 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-php
drwxr-xr-x 10 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-play
drwxr-xr-x  8 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-python
drwxr-xr-x 14 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-ruby
drwxr-xr-x 14 u29276 u29276 4096 Mar 30 23:18 heroku-buildpack-scala
root@c14760c734ec:/# ls -la /tmp/buildpacks/custom/
total 44
drwxr-xr-x  6 root   root   4096 Mar 30 23:18 .
drwxr-xr-x 27 u29276 u29276 4096 Mar 30 23:18 ..
-rw-r--r--  1 root   root     17 Mar 30 23:18 .gitignore
-rw-r--r--  1 root   root   1075 Mar 30 23:18 LICENSE
-rw-r--r--  1 root   root   7008 Mar 30 23:18 README.md
drwxr-xr-x  2 root   root   4096 Mar 30 23:18 bin
-rw-r--r--  1 root   root   1391 Mar 30 23:18 concurrency
drwxr-xr-x  2 root   root   4096 Mar 30 23:18 lib
drwxr-xr-x  3 root   root   4096 Mar 30 23:18 test
drwxr-xr-x  2 root   root   4096 Mar 30 23:18 vendor

EDIT: I believe this to be the offending code. However, this is fairly irrelevant since /tmp/buildpacks/custom should probably be owned by the anonymous user?

https://github.com/heroku/heroku-buildpack-nodejs/blob/40214da800dadf60b9a746514ea55140f5863380/lib/build.sh#L210-L214

ref: progrium/buildstep#145

how to start the app

after i build the app with docker,and i commit it to a image,but how to start the app? use /start ?
in addition,after i build the app,how can i export the /app,when i use "herokuish slug generate",but only include the src app?

Incorrect parsing of Procfile commands

Procfile command parsing is using eval (https://github.com/gliderlabs/herokuish/blame/41aeaed24f778bf9582f9112e2cf79418f7fc8cc/include/procfile.bash#L21):

unprivileged /bin/bash -c "$(eval echo $@)"

Doing $(eval <blah>) doesn't serve a purpose, it just breaks commands that include quotes.

my-quotes.sh:

#!/bin/sh
echo "echo \$@ = $(echo $@)"
echo "eval echo \$@ = $(eval echo $@)"

Execution example:

./my-quotes.sh sh -c \"echo \$PORT\"
echo $@ = sh -c "echo $PORT"
eval echo $@ = sh -c echo 10

Note that the quotes have been dropped. Also, variable expansion has taken place, which we do want in Herokuish. However, the variable expansion will be performed by /bin/bash -c on the same line in procfile.bash.

question: (basic) best practice for using herokuish to create a runnable image

hi,

apologies in advance for beginner question (feel free to redirect me to more appropriate forum).

i have been using tutum/buildstep to build a runnable image like so as part of a home-grown build pipeline using jenkins, git and a docker runtime platform.

i recently wanted to use an iojs runtime in a node app, and it turns out that requires a more recent node buildpack than the one baked into the version of tutum/buildstep on dockerhub.

trying to gain some background context, i did some research and came across this thread about preferring herokuish over buildstep.

along those lines, i would like to use herokuish directly without tutum or progrium buildsteps, but i'm not clear on exactly how to accomplish that.

for example would i use:

  1. some form of docker run gliderlabs/herokuish at the command line, or
  2. FROM gliderlabs/herokuish with some other commands in a Dockerfile

any guidance or examples would be appreciated

best,
tony.

run herokuish as unprivileged user

Currently, herokuish is required to be called as a privileged user and then drop the privilege to an unprivileged user. Would you be interested in a patch allowing herokuish to be called by an unprivileged user.

For example, in the unprivileged function, if the user is not privileged, herokuish can just invoke the command, instead of call setuidgid.

Some test like if [[ "id -un" != "$unprivileged_user"]] ... should do the trick, I think.

Obviously, I may be using it wrong, so please point me to the right direction if this is the case.

Procfile parse not working after slug import

Hello there,

I generated a slug using herokuish. I now try to run a process from it but fail to parse the Procfile. Here's what I do :

FROM gliderlabs/herokuish

COPY bar.tgz /bar.tgz

RUN cat /bar.tgz | /bin/herokuish slug import

CMD /bin/herokuish procfile parse

The command exists with a 0 status code and nothing is displayed. The Procfile exists in /app. What could possibly go wrong ?

custom the url for download

how can i set the url for download?
ex: i want to deploy a node app,when i use the cmd:
docker run --rm -e TRACE=true -v /root/nodejs-flynn-example:/tmp/app gliderlabs/herokuish /bin/herokuish test
it alwalys fail to download the node-v0.10.42-linux-x64.tar.gz from the url :http://s3pository.heroku.com/node/v0.10.42/node-v0.10.42-linux-x64.tar.gz

i want to set the url ,but how?

the logs:

::: BUILDING APP :::
-----> Node.js app detected

-----> Creating runtime environment

   NPM_CONFIG_LOGLEVEL=error
   NPM_CONFIG_PRODUCTION=true
   NODE_ENV=production
   NODE_MODULES_CACHE=true

-----> Installing binaries
engines.node (package.json): 0.10.x
engines.npm (package.json): 2.x

   Resolving node version 0.10.x via semver.io...
   Downloading and installing node 0.10.42...
   Unable to download node 0.10.42; does it exist?

-----> Build failed

   We're sorry this build is failing! You can troubleshoot common issues here:
   https://devcenter.heroku.com/articles/troubleshooting-node-deploys

   If you're stuck, please submit a ticket so we can help:
   https://help.heroku.com/

   Love,
   Heroku

Logs are not visible, if Procfile has a shell script #2667

Hey,

If my Procfile is like:

web: sh sh/warmup.sh & java -jar server/webapp-runner.jar --port $PORT target/*.war

I don't see application logs, but they start working if I run it like this

web: java -jar server/webapp-runner.jar --port $PORT target/*.war

The Idea is to run a warm up script(wampup.sh) that hits the web process before it starts receiving requests.

More on that here: https://devcenter.heroku.com/articles/warming-up-a-java-process

I first thought that this is something related to dokku but then I tried a lot of scenarios and it seems that its related to herokuish.

More details here: dokku/dokku#2667

So I have tried running sample java and node js apps in:

  1. Dokku
  2. herokuish - (FROM gliderlabs/herokuish:latest)
  3. and just docker

But the problem comes up only in 1 and 2.

Any suggestions on fixing this?

Support for Heroku-16 Docker image

Any plans for supporting the Ubuntu 16.04 based heroku-16 Docker image in the near future?
It will be a great addition, considering that the resulting Docker images will be a third of the size...

[label:enhancement]

Silent failure on systems with Debian bash completion

Background: I'm currently working to improve our developer experience by rolling herokuish into a Vagrant image, discovered this issue in the final steps.

Symptoms: When running herokuish procfile start web on a system with Debian's bash completion installed, herokuish exits silently with exit code 1

Trace: trace.txt

Diagnosis:
The problem occurs at procfile-load-profile, during the loading of bash completion. The problem occurs because bash completion and set -e are incompatible, as documented in this debian bug.
Ordinarily the bash completion doesn't run for interactive shells, but it detects interactivity by checking PS1 (i.e. if [ -n "$BASH_VERSION" -a -n "$PS1" -a -z "$BASH_COMPLETION_COMPAT_DIR" ]; then, which is expected behaviour as per bash's documentation), while herokuish sets it's own PS1 unconditionally.

Unfortunately I don't know what the best way to solve this is, otherwise I would provide a PR - does herokuish actually need to set that PS1? Should it only be set immediately prior to procfile-exec?

In the meantime I've modified the interactivity check in my VM, but that's not great long term.

remote: /tmp/bashenv.252471714: line 388: export: `=': not a valid identifier

I tried to use dokku master and get some error on herokuish.

vagrant@dokku:~/node-js-sample$ git push dokku master
Counting objects: 406, done.
Compressing objects: 100% (315/315), done.
Writing objects: 100% (406/406), 214.48 KiB | 0 bytes/s, done.
Total 406 (delta 61), reused 406 (delta 61)
-----> Cleaning up...
-----> Building node-js-sample from herokuish...
-----> Adding BUILD_ENV to build environment...
remote: /tmp/bashenv.252471714: line 388: export: `=': not a valid identifier
remote: /tmp/bashenv.252471714: line 388: export: `=': not a valid identifier
To [email protected]:node-js-sample
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to '[email protected]:node-js-sample'

I think 465bbe1 does not treat empty lines wrongly.

Problem with starting build

So, I have problem with using Herokuish. I run /build, commit image and, then, when I try to run /start, I get:

cat: /app/Procfile: No such file or directory

Below is how it happened...

Also, as you can see below, I checked and /app/Procfile is there...

root@SDE-1:~/php-getting-started# pwd
/root/php-getting-started
root@SDE-1:~/php-getting-started# docker run -ti -v /root/php-getting-started:/tmp/app gliderlabs/herokuish /build
-----> PHP app detected
-----> Bootstrapping...
-----> Installing system packages...
       NOTICE: No runtime required in composer.json; requirements
       from dependencies in composer.lock will be used for selection
       - php (7.0.2)
       - Apache (2.4.16)
       - Nginx (1.8.0)
-----> Enabling PHP extensions...
       - ext-zend-opcache (automatic)
-----> Installing dependencies...
       Composer version 1.0.0-alpha11 2015-11-14 16:21:07
       Loading composer repositories with package information
       Installing dependencies from lock file
       - Installing psr/log (1.0.0)
       Downloading: 100%         

       - Installing monolog/monolog (1.16.0)
       Downloading: 100%         

       - Installing symfony/routing (v2.7.3)
       Downloading: 100%         

       - Installing symfony/http-foundation (v2.7.3)
       Downloading: 100%         

       - Installing symfony/event-dispatcher (v2.7.3)
       Downloading: 100%         

       - Installing symfony/debug (v2.7.3)
       Downloading: 100%         

       - Installing symfony/http-kernel (v2.7.3)
       Downloading: 100%         

       - Installing pimple/pimple (v1.1.1)
       Downloading: 100%         

       - Installing silex/silex (v1.3.1)
       Downloading: 100%         

       - Installing twig/twig (v1.19.0)
       Downloading: 100%         

       - Installing symfony/twig-bridge (v2.7.3)
       Downloading: 100%         

       Generating optimized autoload files
-----> Preparing runtime environment...
-----> Checking for additional extensions to install...
-----> Discovering process types
       Procfile declares types -> web
root@SDE-1:~/php-getting-started# docker ps -a
CONTAINER ID        IMAGE                           COMMAND                  CREATED             STATUS                          PORTS                                              NAMES
aef017370e74        gliderlabs/herokuish            "/build"                 2 minutes ago       Exited (0) About a minute ago                                                      jolly_franklin
6ea4ecf478f1        rancher/agent-instance:v0.6.0   "/etc/init.d/agent-in"   16 hours ago        Up 16 hours                     0.0.0.0:500->500/udp, 0.0.0.0:4500->4500/udp       68c23dba-1aa0-4ae8-bdc6-c342125ba540
9eec76436db4        registry:2                      "/bin/registry /etc/d"   17 hours ago        Up 16 hours                     0.0.0.0:5000->5000/tcp                             registry
aa424cafcedd        nemanjan00/jenkins              "/entrypoint.sh"         17 hours ago        Up 16 hours                     0.0.0.0:50000->50000/tcp, 0.0.0.0:8081->8080/tcp   sleepy_swartz
1bf82997a41f        rancher/agent:v0.8.2            "/run.sh run"            18 hours ago        Up 16 hours                                                                        rancher-agent
95151b0b13ad        rancher/server                  "/usr/bin/s6-svscan /"   18 hours ago        Up 16 hours                     3306/tcp, 0.0.0.0:8080->8080/tcp                   ecstatic_snyder
root@SDE-1:~/php-getting-started# docker commit aef017370e74 php-example
sha256:d244c6464fec96876caae4ffe62205d70d70b3c4010b4c8c2d91c77b2b2cd865
root@SDE-1:~/php-getting-started# docker run -ti php-example /start web
cat: /app/Procfile: No such file or directory
root@SDE-1:~/php-getting-started# docker run -ti php-example /bin/ls /app
Procfile   app.json   composer.lock  vendor
README.md  composer.json  php-min    web
root@SDE-1:~/php-getting-started# docker run -ti php-example /bin/cat /app/Procfile
web: vendor/bin/heroku-php-apache2 web/

start exited with no output

Im having a hard time figuring out why when I run herokuish procfile start web it does not start the application and just exits.

# ls -a /app/
.  ..  .heroku  .profile.d  .project  .release  Dockerfile  
Procfile  a  node_modules  package.json  server.j


# ls -a /tmp/app
.  ..  .heroku  .profile.d  .project  .release  Dockerfile  
Procfile  a  node_modules  package.json  server.js

# cat /app/Procfile 
web: node server.js

# ls /app/.profile.d/
nodejs.sh


# cat /app/.profile.d/nodejs.sh 
calculate_concurrency() {
  MEMORY_AVAILABLE=${MEMORY_AVAILABLE-$(detect_memory 512)}
  WEB_MEMORY=${WEB_MEMORY-512}
  WEB_CONCURRENCY=${WEB_CONCURRENCY-$((MEMORY_AVAILABLE/WEB_MEMORY))}
  if (( WEB_CONCURRENCY < 1 )); then
    WEB_CONCURRENCY=1
  fi
  WEB_CONCURRENCY=$WEB_CONCURRENCY
}

log_concurrency() {
  echo "Detected $MEMORY_AVAILABLE MB available memory, $WEB_MEMORY MB limit per process (WEB_MEMORY)"
  echo "Recommending WEB_CONCURRENCY=$WEB_CONCURRENCY"
}

detect_memory() {
  local default=$1
  local limit=$(ulimit -u)

  case $limit in
    256) echo "512";;      # Standard-1X
    512) echo "1024";;     # Standard-2X
    16384) echo "2560";;   # Performance-M
    32768) echo "14336";;  # Performance-L
    *) echo "$default";;
  esac
}

export PATH="$HOME/.heroku/node/bin:$HOME/.heroku/yarn/bin:$PATH:$HOME/bin:$HOME/node_modules/.bin"
export NODE_HOME="$HOME/.heroku/node"
export NODE_ENV=${NODE_ENV:-production}

calculate_concurrency

export MEMORY_AVAILABLE=$MEMORY_AVAILABLE
export WEB_MEMORY=$WEB_MEMORY
export WEB_CONCURRENCY=$WEB_CONCURRENCY

if [ "$LOG_CONCURRENCY" = "true" ]; then
  log_concurrency
fi

This is the trace output.ls /ap
https://gist.github.com/FLYBYME/a6e9cf6d5efaacc1342e0c212b27d828

Flaky buildpack detection

When I run docker run --rm -v /tmp/app:/tmp/app gliderlabs/herokuish /bin/herokuish buildpack build, the buildpack detected can be flaky. Any help is appreciated!

My app has a .buildpack file which means the 00_buildpack-multi should be detected.
My app also has a package.json file, which means 02_buildpack-nodejs should also be detected, but because the buildpacks are ordered, I expect 00_buildpack-multi to be selected.

When I run the build, I get 02_buildpack-nodejs roughly 95% of the time and 00_buildpack-multi 5% of the time. 00_buildpack-multi is not even detected.

I did some testing and found that whatever buildpack is listed first gets randomly skipped and I'm not sure why. I manually deleted all the buildpacks, and installed 3 identical buildpacks. Sometimes herokuish reports 3 buildpacks detected, sometimes 2 buildpacks detected and it's always the first one that is skipped. I wasn't able to modify herokuish, but it almost seems as if buildpack-list() is not getting the first buildpack sometimes.

More details:

I'm trying to build an elixir app. I use 00_buildpack-multi so that I can use https://github.com/HashNuke/heroku-buildpack-elixir and https://github.com/gjaldon/heroku-buildpack-phoenix-static, but phoenix creates a package.json file to compile static assets.

What I'm seeing in my shell

~/tmp/dummy-app$ ls -a
.  ..  .buildpacks  package.json
~/tmp/dummy-app$ sudo docker run --rm -v $PWD:/tmp/app gliderlabs/herokuish /bin/herokuish buildpack build
-----> Node.js app detected
~/tmp/dummy-app$ sudo docker run --rm -v $PWD:/tmp/app gliderlabs/herokuish /bin/herokuish buildpack build
-----> Node.js app detected
~/tmp/dummy-app$ sudo docker run --rm -v $PWD:/tmp/app gliderlabs/herokuish /bin/herokuish buildpack build
-----> Node.js app detected
~/tmp/dummy-app$ sudo docker run --rm -v $PWD:/tmp/app gliderlabs/herokuish /bin/herokuish buildpack build              
-----> Warning: Multiple default buildpacks reported the ability to handle this app. The first buildpack in the list below will be used.
       Detected buildpacks: multi nodejs
-----> Multipack app detected
=====> Downloading Buildpack: https://github.com/HashNuke/heroku-buildpack-elixir

Versions

~/tmp/dummy-app$ sudo docker run --rm -v $PWD:/tmp/app gliderlabs/herokuish /bin/herokuish version
herokuish: 0.3.22
buildpacks:
  heroku-buildpack-multi     v1.0.0
  heroku-buildpack-ruby      v148
  heroku-buildpack-nodejs    v91
  heroku-buildpack-clojure   v75
  heroku-buildpack-python    v85
  heroku-buildpack-java      v48
  heroku-buildpack-gradle    v18
  heroku-buildpack-grails    v21
  heroku-buildpack-scala     v72
  heroku-buildpack-play      v26
  heroku-buildpack-php       v114
  heroku-buildpack-go        v52
  heroku-buildpack-erlang    fa17af9
  buildpack-nginx            v6
~/tmp/dummy-app$ uname -a
Linux foo 4.4.0-62-generic #83-Ubuntu SMP Wed Jan 18 14:10:15 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
~/tmp/dummy-app$ sudo docker version
Client:
 Version:      1.13.0
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:58:26 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.0
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:58:26 2017
 OS/Arch:      linux/amd64
 Experimental: false
~/tmp/dummy-app$ sudo docker images
REPOSITORY             TAG                 IMAGE ID            CREATED             SIZE
gliderlabs/herokuish   latest              14a0c05d8c2d        29 hours ago        1.4 GB

can i build herokuish on other os

i think the size of image gliderlabs/herokuish is a bit big,it have 1.307 GB .so i want to build the herokuish with other os from the src code? can i ? if yes ,and how?

where are the executable now?

When running a compiled node image, it works if I run /start web, but doesn't work if I just call npm or node.

Higher-level, it doesn't seem possible to execute files with these binaries in Dokku plugins, so for example in a node script:

#!/usr/bin/env node

console.log('hi');

Will not be able to find the node executable, if I just try to run them on the container via:

COMMAND=$(cat <<EOF
cd "/app" &&
make build
EOF
)

id=$(docker run -d $IMAGE /bin/bash -c "$COMMAND")
docker attach $id
test $(docker wait $id) -eq 0
docker commit $id $IMAGE > /dev/null

Is there something I need to update in the plugins for herokuish?

Confused about exec vs. start & creating a runnable image

I was following the example given in #62 to create a runnable image. When I used their sample dockerfile to do so, I made one modification to the code to have it call npm start rather than the specific file stuff, since I always run my node apps that way. The result was as follows:

`FROM gliderlabs/herokuish

RUN mkdir -p /app
COPY . /app
RUN /build

ENTRYPOINT ["/exec", "npm", "start]`

This builds fine, but running it results in the error: setuidgid: usage: setuidgid account child

Conversely, if I change it to rely on a Procfile and simply change the last line to

ENTRYPOINT ["/start", "web"]

It works as expected, and I can navigate to my app through the exposed port.

Given that the Procfile defines web: npm start, I'm confused as to why I get the error on exec but not on start. Any suggestions as to why this might be the case? The docs say

start Run process type command from Procfile through exec

Which seems to indicate that exec is still being run under the hood.

how to deploy app

can i use herokuish to deploy my app? i can use
docker run --rm -v /abs/app/path:/tmp/app gliderlabs/herokuish /bin/herokuish test
to test my app,but how to deploy it?
anyone can help me?

heroku-buildpack-elixir does not see MIX_ENV

The README said to file buildpack issues against herokuish, but this is when using https://github.com/HashNuke/heroku-buildpack-elixir with Catalyze, which has said they use herokuish, and not Dokku or Heroku itself.

I had the MIX_ENV=qa set in the catalyze vars, which is how Catalyze has you set environment variables:

$ catalyze vars list 
BUILDPACK_URL=https://github.com/HashNuke/heroku-buildpack-elixir.git
MIX_ENV=qa
...

It wasn't seeing the MIX_ENV setting in the catalyze vars:

-----> elixir app detected
-----> Checking Erlang and Elixir versions
Will use the following versions:
* Stack cedar-14
* Erlang 18.2.1
* Elixir 1.2.3
Will export the following config vars:
* Config vars MIX_ENV DATABASE_URL PORT HOST_NAME LOG_LEVEL HOST_NAME INTERPRETER_APP_UUID RABBITMQ_URL REDIS_URL
* MIX_ENV=prod

Catalyze support then advised me to use a pre-build hook to set /tmp/env/MIX_ENV myself. So I added this script at .catalyze/pre-build:

#!/bin/sh -e

mkdir -p /tmp/env

if [ -n "$MIX_ENV" ]; then
  echo $MIX_ENV > /tmp/env/MIX_ENV
else
  echo "MIX_ENV not set"
  exit 1
fi

The script worked:

remote: [buildstep] Running pre-build hook
remote: [buildstep] Building application
-----> Fetching custom buildpack
-----> elixir app detected
-----> Checking Erlang and Elixir versions
       Will use the following versions:
       * Stack cedar-14
       * Erlang 18.2.1
       * Elixir 1.2.3
       Will export the following config vars:
       * Config vars MIX_ENV DATABASE_URL PORT HOST_NAME LOG_LEVEL HOST_NAME INTERPRETER_APP_UUID RABBITMQ_URL REDIS_URL
       * MIX_ENV=qa

But, is the MIX_ENV not being set in the first place a bug in herokuish or its interaction with the buildpack?

Silent failure if daemontools isn't installed

I ran herokuish against my django app, using the default python:3.5 docker image. When running 'herokuish test', it complained that it was 'Unable to select a buildpack'. The buildpack detect function worked fine when run as root, and it was only after some digging that I realised my docker image didn't have the 'setuidgid' function, and such was failing silently. Installing daemontools in the container fixed the problem.

It would be good if there was a warning / check for this without failing silently.

Error: EACCES, mkdir '/app/.npm' when using node buildpack

I just tried building my app using heroku-buildpack-nodejs and got this:

----> Building image codetunes
docker run --name build-1438609566-4081 -it -v /Users/teamon/myapp/.build-1438609566-4081:/app gliderlabs/herokuish /build
-----> Node.js app detected

-----> Creating runtime environment

       NPM_CONFIG_LOGLEVEL=error
       NPM_CONFIG_PRODUCTION=true
       NODE_MODULES_CACHE=true

-----> Installing binaries
       engines.node (package.json):  unspecified
       engines.npm (package.json):   unspecified (use default)

       Resolving node version (latest stable) via semver.io...
       Downloading and installing node 0.12.7...
       Using default npm version: 2.11.3

-----> Restoring cache
       Loading 1 from cacheDirectories (default):
       - node_modules (not cached - skipping)

-----> Building dependencies
       Pruning any extraneous modules
       Installing node modules (package.json)
       npm ERR! Linux 3.18.11-tinycore64
       npm ERR! argv "/tmp/build/.heroku/node/bin/node" "/tmp/build/.heroku/node/bin/npm" "install" "--unsafe-perm" "--userconfig" "/tmp/build/.npmrc"
       npm ERR! node v0.12.7
       npm ERR! npm  v2.11.3
       npm ERR! path /app/.npm
       npm ERR! code EACCES
       npm ERR! errno -13

       npm ERR! Error: EACCES, mkdir '/app/.npm'
       npm ERR!     at Error (native)
       npm ERR!  { [Error: EACCES, mkdir '/app/.npm']
       npm ERR!   errno: -13,
       npm ERR!   code: 'EACCES',
       npm ERR!   path: '/app/.npm',
       npm ERR!   parent: 'codetunes2' }
       npm ERR!
       npm ERR! Please try running this command again as root/Administrator.

       npm ERR! Please include the following file with any support request:
       npm ERR!     /tmp/build/npm-debug.log

-----> Build failed

       We're sorry this build is failing! You can troubleshoot common issues here:
       https://devcenter.heroku.com/articles/troubleshooting-node-deploys

       Some possible problems:

       - Node version not specified in package.json
       https://devcenter.heroku.com/articles/nodejs-support#specifying-a-node-js-version

       Love,
       Heroku

rake aborted!

Any ideas what could be the cause?

Problem parsing .env file ?

Clone the java-getting-started app:

$ git clone https://github.com/heroku/java-getting-started.git
Cloning into 'java-getting-started'...
remote: Counting objects: 184, done.
remote: Total 184 (delta 0), reused 0 (delta 0), pack-reused 184
Receiving objects: 100% (184/184), 81.08 KiB | 0 bytes/s, done.
Resolving deltas: 100% (53/53), done.
Checking connectivity... done.

test it:

$ docker run --rm -v `pwd`/java-getting-started:/tmp/app gliderlabs/herokuish /bin/herokuish test
::: BUILDING APP :::
/app/.env: line 1: GeV: command not found

I guess, this is due to the fact, that the .env file contains a space in its value:

ENERGY=20 GeV

Indeed, when I quote the value, I can go further into the build:

ENERGY="20 GeV"

child process does not receive process signals

I have (as I assume many do) the need to catch SIGTERM upon container shutdown to handle some cleanup. Currently, it seems, herokuish does not pass this through. See below:

example app code (repo):

var express = require('express');
var app = express();

app.set('port', (process.env.PORT || 5000));
app.use(express.static(__dirname + '/public'));

app.get('/', function(request, response) {
  response.send('Hello World!');
});

process.on( "SIGINT", function() {
  console.log('CLOSING [SIGINT]');
  process.exit();
} );

process.on( "SIGTERM", function() {
  console.log('CLOSING [SIGTERM]');
  process.exit();
} );

app.listen(app.get('port'), function() {
  console.log("Node app is running at localhost:" + app.get('port'));
});

Dockerfile.herokuish:

FROM gliderlabs/herokuish:latest
EXPOSE 5000

ADD . /app
WORKDIR /app

RUN herokuish buildpack build
CMD herokuish procfile start web

Procfile

web: node index.js

docker

$ docker run -d node-js-app setuidgid u17287 /bin/bash -c "PATH=/app/.heroku/node/bin:$PATH node index.js"
4690edb831d53225e1886ce11764176edfd6788a7c98c3a768707fc5a2f28d82
$ docker ps
CONTAINER ID        IMAGE               COMMAND                CREATED              STATUS              PORTS               NAMES
4690edb831d5        node-js-app         "setuidgid u17287 /b   5 seconds ago        Up 4 seconds        5000/tcp            fervent_lovelace
f477a105c114        node-js-app         "/bin/sh -c 'herokui   About a minute ago   Up About a minute   5000/tcp            serene_goldstine
$ docker stop fervent_lovelace
fervent_lovelace
$ docker logs fervent_lovelace
Node app is running at localhost:5000
CLOSING [SIGTERM]

herokuish

$ docker run -d node-js-app
2b671f27cd1f9568310d2440ce1b334bf5eb6c1e17031cc0ff182d3f49444d40

$ docker ps
CONTAINER ID        IMAGE               COMMAND                CREATED             STATUS              PORTS               NAMES
2b671f27cd1f        node-js-app         "/bin/sh -c 'herokui   5 seconds ago       Up 4 seconds        5000/tcp            admiring_heisenberg

$ docker logs admiring_heisenberg
Node app is running at localhost:5000

Multiline execs

This is probably a bash issue, but I could use some guidance and some other people could use this as a reference.

I'm really struggling to execute a multi-line command within a container:

makehook() {
  IMAGE=$1
  HOOK=$2

  COMMAND=$(cat <<EOF
[[ -f "Makefile" ]]
&& grep -q "^$HOOK:" Makefile
&& echo "------> Triggering: make $HOOK"
&& make $HOOK
&& echo "------> Triggered: make $HOOK"
EOF
)

  id=$(docker run -d $IMAGE /exec "$COMMAND")
  docker attach $id
  test "$(docker wait $id)" -eq 0
  docker commit $id $IMAGE > /dev/null
}

Yields the following:

remote: /bin/bash: line 1: ------: command not found
remote: /bin/bash: line 2: dokku.post-build-buildstep: command not found
remote: /bin/bash: line 3: ------: command not found

Basically, I think it's the arrows. It'd be nice to have them outside the container, but I only want output if the file exists.

/cc @michaelshobbs @josegonzalez (if you can take a look too, I've spent hours on this issue :-X)

herokuish procfile start deletes my local project

Hello,
I could be doing something incorrectly, but I am trying to launch herokuish image as part of a cluster using docker-compose.

my docker-compose.yml has the following:

web:
  build: .
  env_file: .env
  command: herokuish procfile start
  ports:
   - "5000:5000"
  volumes:
   - .:/app
  links:
   - mongo
mongo:
  image: stepsaway/mongo:2.6.9
  ports:
   - "27017:27017"

My docker files look like

FROM gliderlabs/herokuish:latest

WORKDIR /tmp/app
ADD . /tmp/app

RUN herokuish buildpack build

WORKDIR /app
ADD . /app

I run docker-compose build and then docker-compose up. It is my understanding that during the up command the Procfile should be read and ran which should bring up my webapp. However, what I am seeing is the following:

Recreating sapi_mongo_1...
Recreating sapi_web_1...
Attaching to sapi_web_1
web_1 | rm: cannot remove '/app': Device or resource busy
web_1 | cat: /app/Procfile: No such file or directory
sapi_web_1 exited with code 6
Gracefully stopping... (press Ctrl+C again to force)

Followed by my entire project being deleted on my laptop. Is this a bug or am I not understanding how I should be using herokuish correctly.

Thanks for any help and thanks for creating this image.

Push tests out to buildpacks repos

Since Herokuish is based on the Heroku official image and is packaged up nicely to be run in Docker, we should try and get buildpack repo maintainers to let us submit a PR that uses Herokuish to test their buildpack the way we test them here. Some of them already use TravisCI, but that lets us submit a CircleCI manifest to just test against Herokuish. The argument being it's pretty damn close to testing the buildpack in action in production.

This would simplify and alleviate a lot of complexity in Herokuish when it comes to automatically testing new buildpacks ... they'll have already been tested.

Unable to use custom buildpack

Im not sure if im doing something wrong or its a legit issue. Tried testing the application (both on OSX and Linux) with a custom static buildpack. Build command always exited with non-zero status.
I dont thinks is an issue with that particular buildpack, using any third-party one resulted in the same problem.

In the test application i created file .buildpacks with the following content:

https://github.com/abhishekmunie/heroku-buildpack-static.git

Then ran test command:

 docker run --rm -e TRACE=1 -v $(pwd):/tmp/app gliderlabs/herokuish /bin/herokuish test

Here's the tail of the log:

+ selected_path=/tmp/buildpacks/00_buildpack-multi
++ unprivileged /tmp/buildpacks/00_buildpack-multi/bin/detect /tmp/build
++ setuidgid u2143 /tmp/buildpacks/00_buildpack-multi/bin/detect /tmp/build
+ selected_name=Multipack
+ [[ -n /tmp/buildpacks/00_buildpack-multi ]]
+ [[ -n Multipack ]]
+ title 'Multipack app detected'
----->' Multipack app detected
-----> Multipack app detected == --* ]]
-----> Multipack app detected == ==* ]]
-----> Multipack app detected'
-----> Multipack app detected
+ read line
+ cd /tmp/build
+ unprivileged /tmp/buildpacks/00_buildpack-multi/bin/compile /tmp/build /tmp/cache /tmp/env
+ setuidgid u2143 /tmp/buildpacks/00_buildpack-multi/bin/compile /tmp/build /tmp/cache /tmp/env
+ [[ =====> Downloading Buildpack: https://github.com/abhishekmunie/heroku-buildpack-static.git == --* ]]
+ [[ =====> Downloading Buildpack: https://github.com/abhishekmunie/heroku-buildpack-static.git == ==* ]]
=====>' Downloading Buildpack: https://github.com/abhishekmunie/heroku-buildpack-static.git
=====> Downloading Buildpack: https://github.com/abhishekmunie/heroku-buildpack-static.git
+ read line

Test command just exists and there's no error or anything useful in the log.
Using latest gliderlabs/herokuish docker image. Versions:

herokuish: 0.3.19
buildpacks:
  heroku-buildpack-multi     v1.0.0
  heroku-buildpack-ruby      v146
  heroku-buildpack-nodejs    v91
  heroku-buildpack-clojure   v75
  heroku-buildpack-python    v82
  heroku-buildpack-java      v46
  heroku-buildpack-gradle    v18
  heroku-buildpack-grails    v21
  heroku-buildpack-scala     v72
  heroku-buildpack-play      v26
  heroku-buildpack-php       v110
  heroku-buildpack-go        v44
  heroku-buildpack-erlang    fa17af9
  buildpack-nginx            v6
  emberjs.tgz                latest

Add support for .tgz buildpacks

It'd be nice if it were possible to install buildpacks from a tgz file. For example the Ember JS buildpack requires you to install from their prebuilt tgz instead of their git repository โ€“

This buildpack has a binary component, so it needs to be compiled beforehand. It's easiest just to use the buildpack with the prebuilt binary.

https://codon-buildpacks.s3.amazonaws.com/buildpacks/heroku/emberjs.tgz

buildpack-install assumes any buildpack URL is a git repository though, so trying to use their URL fails. I'm not sure if tgz buildpacks are common, but if Heroku is planning on making more like the EmberJS buildpack, it'd be nice to support them.

/tmp is way too small

Depending on your setup /tmp may be mounted on a really small tmpfs (I think this depends on the operating system, but on Fedora 22 this is 64MB), which if not enough for herokuish to deploy real applications.

The problem is that herokuish is using /tmp/build as a temporary "build area" and when deploying application composed of multiple git repositories, this will quickly fill up the small RAM disk mounted /tmp.

For now I'm using a workaround - by designating /tmp as a volume, it gets allocated a docker FS layer of itself, which allows /tmp to be the size of available space on the drive where docker stores its file system layers:

docker run --rm -v /abs/app/path:/tmp/app -v /tmp gliderlabs/herokuish /bin/herokuish test

Fails silently when custom buildpack fails to download from Git

The logic in buildpack.bash attempts to git ls-remote $url, and falls back to trying to pipe from curl into tar, with arguments based on file extension.

In my case, I was using a buildpack with a GitHub, but ls-remote failed due to a connectivity issue. The following case statement failed to match any extensions, leaving tar_args empty. Which caused tar to fail as tar without one of -Acdrtux is invalid.

Update node.js buildpack for io.js support

Hopefully this is the correct repo to file this against.

Using dokku 0.3.16, it seems the detection of engines.iojs in package.json isn't working and it falls back to node.js:

-----> Node.js app detected

       PRO TIP: Specify a node version in package.json
       See https://devcenter.heroku.com/articles/nodejs-support

-----> Defaulting to latest stable node: 0.12.2

This feature has been working on official Heroku since around February, maybe the buildpack used here or by dokku is outdated?

failing installation behind proxy: wrong docker argument ?

Description of problem:

I try to install dokku on Ubuntu behind a corporate proxy following the official documentation. The installation fails during the installation of herokuish with following output:

Adding proxy settings to docker build: --build-args http_proxy=<PROXY_ADDRESS> --build-args https_proxy=<PROXY_ADDRESS>
unknown flag: --build-args
See 'docker build --help'.
dpkg: error processing package herokuish (--configure):
 subprocess installed post-installation script returned error exit status 125

I think this is a very simple issue, because the official docker docs says the flag is called --build-arg instead of --build-args! see : https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg

Could you check this?

Procfile parsing seems very fragile

Earlier today, while trying to deploy an app to Dokku I had a simple Procfile which looked like this

web:python server.py

Can you spot the error? Yeah, I couldn't either. As it turns out, herokuish expects a space after the colon and threw this nasty stack trace in my face. Neither foreman nor "real" Heroku seems to care about the presence of the space, or lack thereof. The error message is also really cryptic and nondescript, and it was just really a random guess of mine to try it with a space. A better error message, or more lenient parsing of the Procfile would be really nice.

Permission denied on /tmp directory

Since I updated the herokuish image, I cannot compile nginx as a build step due to:

...
mkdir: cannot create directory '/tmp/cg2779-8969': Permission denied
mkdir: cannot create directory '/tmp/cg-2779': Permission denied
...

I've mounted a container to check it and these are the /tmp permissions:

# stat -c '%a %n' /tmp
1755 /tmp

Changing permissions to standard

# chmod 1777 /tmp

fixes the compilation issue, as expected.

Shouldn't it be world-writable as in any standard *nix setup?

buildback not included in slug, wrong path ?

Hello,

I was playing with herokuish and stumbled upon this :

When mounting a volume with my app in /tmp/app and doing this in the container :

/bin/herokuish buildpack build
/bin/herokuish slug generate

Then the slug I get in /tmp/slug.tgz is the source as imported in /tmp/app and not the app I just built in /app.

cannot install herokuish with docker-ce

I installed docker-ce with viewing https://docs.docker.com/engine/installation/linux/ubuntu/ .
And install dokku with viewing http://dokku.viewdocs.io/dokku/#install-apt .
But I cannot install herokuish with dependency error.

vagrant@vagrant:~$ sudo apt install herokuish
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 herokuish : PreDepends: docker-engine-cs (>= 1.9.1) but it is not installable or
                         docker-engine (>= 1.9.1) but it is not installable
E: Unable to correct problems, you have held broken packages.

Related: dokku/dokku@1e387b3

Changes not persisted between buildpacks?

I must be misunderstanding how herokuish works; hopefully someone can explain what's going on here.

In my .buildpacks, I specify 2 buildpacks -- the official Heroku python buildpack and a custom one that I am developing for the tox testing tool.

When the python buildpack executes, I can see it install python and pip in /app/.heroku/python/bin and then it uses pip to install a bunch of packages. Great.

Now the tox buildpack executes and it tries to do /app/.heroku/python/bin/pip install tox and it fails, because the whole /app/.heroku directory from the previous step is not there.

I added some debugging:

After the end of python buildpack execution:

$ hostname
8ab2d0316829

$ ls -al /app
...
       drwxr-xr-x  2 u17823 u17823  4096 Aug 21 13:52 etc
       -rw-r--r--  1 u17823 u17823   278 Aug 21 13:52 .gitignore
       drwxr-xr-x  4 u17823 u17823  4096 Aug 21 13:52 .heroku
       -rw-r--r--  1 u17823 u17823 43093 Aug 21 13:52 junit.xml
       -rw-r--r--  1 u17823 u17823   105 Aug 21 13:52 MANIFEST.in
...

At the beginning of tox buildpack execution:

$ hostname
8ab2d0316829

$ ls -al /app
...
       drwxr-xr-x  2 u17823 u17823  4096 Aug 21 13:52 etc
       -rw-r--r--  1 u17823 u17823   278 Aug 19 17:49 .gitignore
       -rw-r--r--  1 u17823 u17823 43093 Aug 19 18:02 junit.xml
       -rw-r--r--  1 u17823 u17823   105 Aug 19 17:49 MANIFEST.in
...

Note that the .heroku directory is missing. I also just noticed that some of the timestamps are different, which is peculiar.

Here's my Dockerfile:

FROM gliderlabs/herokuish

RUN mkdir -p /app
ADD . /app
RUN /build

CMD ["/exec", "pserve", "/app/etc/app.mt3.ini"]

Cc: @progrium, @sudarkoff

persistent user AND userid

When using herokuish (+dokku) and docker volumes, the randomized user id is less than ideal. Thoughts on allowing a $USERID variable to be passed in?

I'm willing to do the work but wanted feedback on the implementation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.