jwilder / docker-squash Goto Github PK

View Code? Open in Web Editor NEW

2.0K 2.0K 158.0 61 KB

Squash docker images to make them smaller

License: MIT License

Makefile 3.04% Go 96.26% Dockerfile 0.70%

docker-squash's People

Contributors

Stargazers

Watchers

Forkers

spuder snewhouse oligot pinglamb wheelcomplex elyase aiden0z rafecolton kgrvamsi shingonoide ddollar cetex atlashealth sheikhsameer1609 dweinstein zhitaoli jwhitcraft ph-one devopsbrett myyk arcolombo dilgerma adfernandes amesianx serkanh mba811 th3architect iconnect jduhamel coryb mihirv0ra sentinelleader mbankole snakeliwei makicha sunnycmf soualid cloudxtreme colstrom wrouesnel derdmitry excavador chriskite jkwroi24io9342fsdjkladf apiaryio nealhardesty oliverzeyen hayd wingerd jwgoh yantis mhaberler kissingwolf novboy dockernuts ieee0824 binarytemple-external arvinhub scriptarchitect docker-buea valerioz aeppert ramfree17 vsajip sudhish74 jeremypogue vintage-chou learn-docker harendranathvegi9 perillamint aland-zhang mohitsharma44 moran0013 pombredanne fuhbar richvred shangwen pjferrell sonnt85 cbepx uliul-devops vermuz caoguanghui mrgoogol balancetan iskibinskahfa meua jpalmergithub rgrsoucy l3x danihe amoschov m4t7e0 therevoman awhawks jiangrensheng jeremylin06 onophris stormning toddtaoli

docker-squash's Issues

Panic instead of error on incorrect input

I had some difficulty resolving below error. It turned out that some use of sudo was actually messing up the pipes and thus giving docker-squash incorrect input. But that is not really apparent from a panic...

Can be reproduced by doing echo "foo" | docker-squash ....

panic: runtime error: index out of range

goroutine 1 [running]:
runtime.panic(0x546cc0, 0x6f14d7)
/usr/lib/go/src/pkg/runtime/panic.c:266 +0xb6
main.main()
/tmp/go/docker-squash/main.go:66 +0x2228

goroutine 3 [runnable]:
os/signal.loop()
/usr/lib/go/src/pkg/os/signal/signal_unix.go:19
created by os/signal.init·1
/usr/lib/go/src/pkg/os/signal/signal_unix.go:27 +0x31
2014/09/14 15:10:04 write /dev/stdout: broken pipe

ctrl-c not working

if docker-squash is run without any input, it hangs. ctrl-c doesn't kill it.

Preserve whiteouts - deleted files reappear in squashed image.

docker-squash doesn't seem to preserve whiteouts. I am using docker 1.11 which doesn't seem to be compatible with the manifest.json so I built docker-squash using this procedure:

mkdir /tmp/docker-squash-build && cd /tmp/docker-squash-build
export GOPATH=`pwd`
git clone https://github.com/jwilder/docker-squash.git
cd docker-squash
git remote add navad https://github.com/navad/docker-squash.git
git fetch navad
git merge --no-ff navad/support-docker-manifest
# Need to use tar instead of gtar:
# Revert https://github.com/jwilder/docker-squash/pull/55/commits/9cc83428a812557bc862e1cf454e703016fc0689
git revert 9cc83428a812557bc862e1cf454e703016fc0689
glock sync ../docker-squash/
GOBIN=`pwd` make
sudo install docker-squash /usr/local/bin

Then once I had docker-squash built:

cat <<EOF | docker build -t docker-squash-test:latest -
FROM busybox:latest
RUN touch /blah
RUN rm /bin/telnetd
RUN touch /blah
EOF
docker save docker-squash-test:latest | sudo docker-squash -t docker-squash-test:squashed  | docker load

Then, I tried to stat /bin/telnetd in both the latest and squashed tag. I would think it should be deleted in both tags, but that only seems to be the case for the latest tag:

$ docker run --rm -it docker-squash-test:latest stat /bin/telnetd
stat: can't stat '/bin/telnetd': No such file or directory
$ docker run --rm -it docker-squash-test:squashed stat /bin/telnetd
  File: /bin/telnetd
  Size: 1031256     Blocks: 2016       IO Block: 4096   regular file
Device: 56h/86d Inode: 23          Links: 365
Access: (0755/-rwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2016-06-15 20:01:47.000000000
Modify: 2016-03-18 16:38:21.000000000
Change: 2016-06-15 20:01:35.000000000

Edit: Use busybox:latest to reproduce instead of ubuntu:14.04

go get not running the package gotten

followed the instructions on the official golang page and installed go on my Ubuntu 16.04LTS laptop. anytime i want to work i export my GOPATH e.g export GOPATH=/home/x/go/src/github.com/x/GoWebDev/GoLearning/. I did go get and it downloads the package into ~/go/github.com but when I try to run the program it outputs an error cannot find package "github.com/nu7hatch/gouuid" in any of: /usr/local/go/src/github.com/nu7hatch/gouuid (from $GOROOT) /home/x/go/src/github.com/x/GoWebDev/GoLearning/src/github.com/nu7hatch/gouuid (from $GOPATH) what is wrong?

tar: This does not look like a tar archive

I had an issue beforehand (#65) so I figured I'd try it out on a different Docker version.

On Debian Jessie with Docker version 1.12.1

When trying to squash an image, the following output comes up:

$ sudo docker-squash -i image.tar -o squashed.tar -t squashed -verbose
Loading export from image.tar using /tmp/docker-squash649489719 for tempdir
Loaded image w/ 13 layers
Extracting layers...
  -  /tmp/docker-squash649489719/15ca8d025fb12dd0390b50364bf71b5573ba07a4d92d266cf23b7c3c29a87e4b/layer.tar
  -  /tmp/docker-squash649489719/457485a3a84745e213bbb7fc065eacc2826c1d0dacc8743dcb47383b9140b248/layer.tar
tar: This does not look like a tar archive
tar: Exiting with failure status due to previous errors

ERROR: exit status 2
Removing tempdir /tmp/docker-squash649489719

Same output when trying it through STDIN
Any ideas?

throws null pointer exception

First: Listing docker images.

root@agallego:/tmp/alex# docker images
REPOSITORY                    TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
concord/client_devbox         latest              8d2b61676ad1        23 minutes ago      2.401 GB
concord/light_client_devbox   latest              7bbcba9980f2        25 minutes ago      2.401 GB
<none>                        <none>              702ee68f9a1f        40 minutes ago      2.476 GB

When trying to squash, with either `-from` , no -from argument or a hash id it breaks like this.


root@agallego:/tmp/alex# docker save concord/client_devbox | docker-squash -from 8d2b61676ad1 -t concord/client_devbox_3 | docker load 
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x50 pc=0x404c22]

goroutine 1 [running]:
main.(*Export).InsertLayer(0xc20801e340, 0xc2080429c0, 0x40, 0xc208038240, 0x0, 0x0)
        /Users/jason/go/src/github.com/jwilder/docker-squash/export.go:395 +0xa72
main.main()
        /Users/jason/go/src/github.com/jwilder/docker-squash/main.go:130 +0xee7

goroutine 5 [syscall]:
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
        /usr/local/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 6 [chan receive]:
main.shutdown(0xc20803c540, 0x25)
        /Users/jason/go/src/github.com/jwilder/docker-squash/main.go:22 +0x67
created by main.main
        /Users/jason/go/src/github.com/jwilder/docker-squash/main.go:73 +0x7d8
Error response from daemon: EOF

refactor to be library?

Hey,

I like the docker-squash project and was hoping to incorporate it into one of my projects, but I cant really do that with all the code in the "main" package. I was wondering if you be willing to take patches to transform it into a library + main. I am roughly thinking:

move main.go into ./main/main.go
change all the other *.go to be in "dockersquash" package
tweak the build so it can still produce a binary

also, was wondering how attached you were to the log.go code, vs something like http://github.com/op/go-logging which might be easier to integrate for library usage.

If you are amenable then I will work on a pull request.

There will still be sub-optimal issues (like requiring sudo and using external tar) but I think those can be fixed similar to what was done in #15 just in smaller isolated pull requests.

-Cory

Docker-squash doesn't work

Hello I have docker version: Docker version 17.03.1-ce, build c6d412e
And when I run;
docker save whoami:latest | sudo docker-squash -t whoami:squashed2 | docker load

It return me: Loaded image: whoami:latest
but whoami:squashed2 is not there
and image whoami:latest is not squashed..

Error on Mac Elcapitan.

I tried to use this on Mac Elcapitan but I did run into the following error:

tar: Option --xattrs is not supported
Usage:
List: tar -tf
Extract: tar -xf
Create: tar -cf [filenames...]
Help: tar --help

ERROR: exit status 1
invalid argument

I am not sure what could be done to get over the issue. Thanks for your help.

Squashed image is different from original

I am new to docker so I may be doing something wrong here, but as far as I can tell, something is being lost in the squashing process.

I am squashing an image based in python:2.7 and after the squash python is broken.

The difference between the two images seems to be that the squashed image is missing the python2.7 dir in /usr/local/lib/

Steps to reproduce:

# setup
docker pull python:2.7

# squash
docker save `docker inspect --format '{{ .Id }}' python:2.7` | sudo docker-squash -verbose -t python:squash | docker load

# make sure it works pre-squash
docker run -it python:2.7 python -c 'print "it works"'

# show it fails post-squash
docker run -it python:squash python -c 'print "it works"'

# good one has /usr/local/lib/python2.7 
docker run -it python:2.7 ls -alh /usr/local/lib/

# squashed one doesn't
docker run -it python:squash ls -alh /usr/local/lib/

Docker-squash not properly working

With the command docker save d5b63701456a | sudo /usr/local/bin/docker-squash -verbose -t test:squashed | docker load I don't get any new images, though there's no error.

Output:

Loading export from STDIN using /tmp/docker-squash299353940 for tempdir
Loaded image w/ 13 layers
Extracting layers...
  -  /tmp/docker-squash299353940/f3376dbeb3b0e4f36439bca922ec3cd370ef05c5243a37f038170716d556aeb3/layer.tar
  -  /tmp/docker-squash299353940/09967547b5df888f5e066a52d72526b1c68f6837e216799c91c26499dd42c6a7/layer.tar
  -  /tmp/docker-squash299353940/457485a3a84745e213bbb7fc065eacc2826c1d0dacc8743dcb47383b9140b248/layer.tar
  -  /tmp/docker-squash299353940/a238fce9a846a5209b8c1d0bc4c620b4cac48fb512c5b391f4bc655a80b3aa79/layer.tar
  -  /tmp/docker-squash299353940/a4d961cdf6bb153d27f8f58f8e2eaec75241ac60e51dd39aecbc34f2164db202/layer.tar
  -  /tmp/docker-squash299353940/ac03b9ff3e20837ea0e0095e093ee76c8bcffc18fb29ce5d70cc317c67129cef/layer.tar
  -  /tmp/docker-squash299353940/b13b4991fd92f5c3f893ec70993dca32bdc80785030e538b575fa68612a9c4e9/layer.tar
  -  /tmp/docker-squash299353940/bf6a503f7a3e7527729622ed72da1999d3d8b5a674ba3e18cd4ef1e01d1b5c97/layer.tar
  -  /tmp/docker-squash299353940/f30d7a68d1426e11ddea1ea508144e78cd1581f2aa710a28bc1157da72d659bb/layer.tar
  -  /tmp/docker-squash299353940/15ca8d025fb12dd0390b50364bf71b5573ba07a4d92d266cf23b7c3c29a87e4b/layer.tar
  -  /tmp/docker-squash299353940/17215e1d258f4f6306eec1764f142fc4fd3cf85d050a88b1580e16ae98626cb6/layer.tar
  -  /tmp/docker-squash299353940/3b5667d6dbc444eb8baaac05ca46bdc18f6d872d0c8d899b9064749a1cbe80f0/layer.tar
  -  /tmp/docker-squash299353940/626f3bd3ed630a7876e2f1f002b7ea8a4020cacf3e0319498831ee2c604c6ee7/layer.tar
Inserted new layer 4455bd2b5c88 after 15ca8d025fb1
  -  15ca8d025fb1
  -> 4455bd2b5c88 /bin/sh -c #(squash) from 15ca8d025fb1
  -  3b5667d6dbc4
  -  626f3bd3ed63
  -  17215e1d258f
  -  ac03b9ff3e20
  -  bf6a503f7a3e
  -  457485a3a847
  -  f3376dbeb3b0
  -  a238fce9a846
  -  b13b4991fd92
  -  a4d961cdf6bb
  -  09967547b5df
  -  f30d7a68d142 /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "node /srv/node/Centra
Squashing from 4455bd2b5c88 into 4455bd2b5c88
  -  Deleting whiteouts for layer 3b5667d6dbc4
  -  Deleting whiteouts for layer 626f3bd3ed63
  -  Deleting whiteouts for layer 17215e1d258f
  -  Deleting whiteouts for layer ac03b9ff3e20
  -  Deleting whiteouts for layer bf6a503f7a3e
  -  Deleting whiteouts for layer 457485a3a847
  -  Deleting whiteouts for layer f3376dbeb3b0
  -  Deleting whiteouts for layer a238fce9a846
  -  Deleting whiteouts for layer b13b4991fd92
  -  Deleting whiteouts for layer a4d961cdf6bb
  -  Deleting whiteouts for layer 09967547b5df
  -  Deleting whiteouts for layer f30d7a68d142
  -  Rewriting child history
  -  Removing 3b5667d6dbc4. Squashed. ()
  -  Removing 626f3bd3ed63. Squashed. ()
  -  Removing 17215e1d258f. Squashed. ()
  -  Removing ac03b9ff3e20. Squashed. ()
  -  Removing bf6a503f7a3e. Squashed. ()
  -  Removing 457485a3a847. Squashed. ()
  -  Removing f3376dbeb3b0. Squashed. ()
  -  Removing a238fce9a846. Squashed. ()
  -  Removing b13b4991fd92. Squashed. ()
  -  Removing a4d961cdf6bb. Squashed. ()
  -  Removing 09967547b5df. Squashed. ()
  -  Replacing f30d7a68d142 w/ new layer 9d2df5ab1cca (/bin/sh -c #(nop) CMD ["/bin/sh" "-c" "node /sr...)
Tarring up squashed layer 4455bd2b5c88
Removing extracted layers
Tagging 9d2df5ab1cca as test:squashed
Tarring new image to STDOUT
Done. New image created.
  -  15ca8d025fb1 292.471209 years  130.9 MB
  -  4455bd2b5c88 26 seconds /bin/sh -c #(squash) from 15ca8d025fb1 640.8 MB
  -  9d2df5ab1cca 16 seconds /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "node /srv/node/Cen... 70.64 MB
Removing tempdir /tmp/docker-squash299353940

When docker-squashing into an output file and trying to load it from that, I get the following error:
$ docker load -i squashed.tar Error response from daemon: open /var/lib/docker/tmp/docker-import-102261036/7dc29a8f47df2223a335cd818b446df8cdde86863a46c13a18211916eb5a759c/layer.tar: no such file or directory

I'm using SLES 12 SP 1 Docker version 1.10.3

Any ideas?

osx_install.md could be updated

It could mention that there is a binary release at https://github.com/jwilder/docker-squash/releases

OSX tar: Option --xattrs is not supported

It seems it's not playing so nice with tar on OSX:

$ docker save 091eebea9efd | sudo docker-squash -t squash -verbose | docker load
Loading export from STDIN using /tmp/docker-squash978633966 for tempdir
Loaded image w/ 17 layers
Extracting layers...
  -  /tmp/docker-squash978633966/e566a8a9474f62235548281ed5e276a559dca94a9ebd8f1acb5e8cd287bc7b08/layer.tar
tar: Option --xattrs is not supported
Usage:
  List:    tar -tf <archive-filename>
  Extract: tar -xf <archive-filename>
  Create:  tar -cf <archive-filename> [filenames...]
  Help:    tar --help

ERROR: exit status 1
Removing tempdir /tmp/docker-squash978633966
open /var/lib/docker/tmp/docker-import-172480019/repositories: no such file or directory

Any suggestions would be greatly recevied.

Thanks

Nathan

Docker image fails to build

package github.com/docker/docker/pkg/units: cannot find package "github.com/docker/docker/pkg/units" in any of:
	/usr/local/go/src/github.com/docker/docker/pkg/units (from $GOROOT)
	/go/src/github.com/docker/docker/pkg/units (from $GOPATH)

This package is referenced in export.go.

It appears that the package was replaced in docker by go-units.

See this PR which mentioned this: #61

Squashed image not rerunning the image history

Extracting layers...

/tmp/docker-squash184390964/53b8e727e33ca37e5a9938107bd6b030c5c18804a1cdf145ebb5b37efa57c15a/layer.tar
/tmp/docker-squash184390964/57fdabbcbbd733d21a5f4f764102ad9fdc3523289cf8faf7e3bb55e180e11c80/layer.tar
/tmp/docker-squash184390964/5d5dc6a600125e54bfb94f3ea1e5ed401b7cd84263ba29d05092a9d75a0bb07d/layer.tar
/tmp/docker-squash184390964/7397b069ec3e2327b89cd45c6172dbdf543876947678b1fc48cbc9829ca426ee/layer.tar
/tmp/docker-squash184390964/b5ae67ec3dcb6288dd181642ec57f00acb51422638b239119c405283b1fd2e16/layer.tar
/tmp/docker-squash184390964/faa7ad651836c42be41ef3ebb36358e116cef1d996251431e1b0fc64dc6021cc/layer.tar
/tmp/docker-squash184390964/2587e40a84bfd5c037e735d2913444a010d33a41931245b6a3c7b83718c66783/layer.tar
/tmp/docker-squash184390964/27a7e9df4b42ff368c7541849df3479f88d4c1c91d617c3cd8a30e0705c77ca3/layer.tar
/tmp/docker-squash184390964/f61d7ac5814b84215efff3569417ca8ee50d7688b82802355a9187d570f78a76/layer.tar
/tmp/docker-squash184390964/48863186f5e0baef52a091fdf45962a95a2e2c99582cda7640ac93fe6441a275/layer.tar
/tmp/docker-squash184390964/b0a16415dc07708bf06923be992a374c91fc067c46a2e3dfba3b26cb7cbec2f8/layer.tar
Inserted new layer fcb5419f383b after 5d5dc6a60012
5d5dc6a60012
-> fcb5419f383b /bin/sh -c #(squash) from 5d5dc6a60012
48863186f5e0
57fdabbcbbd7
53b8e727e33c
b0a16415dc07
f61d7ac5814b
7397b069ec3e
2587e40a84bf
27a7e9df4b42
faa7ad651836
b5ae67ec3dcb /bin/sh -c apk del gcc g++ linux-headers paxctl gnupg && rm
Squashing from fcb5419f383b into fcb5419f383b
Deleting whiteouts for layer 48863186f5e0
Deleting whiteouts for layer 57fdabbcbbd7
Deleting whiteouts for layer 53b8e727e33c
Deleting whiteouts for layer b0a16415dc07
Deleting whiteouts for layer f61d7ac5814b
Deleting whiteouts for layer 7397b069ec3e
Deleting whiteouts for layer 2587e40a84bf
Deleting whiteouts for layer 27a7e9df4b42
Deleting whiteouts for layer faa7ad651836
Deleting whiteouts for layer b5ae67ec3dcb
Rewriting child history
Removing 48863186f5e0. Squashed. ()
Removing 57fdabbcbbd7. Squashed. ()
Removing 53b8e727e33c. Squashed. ()
Removing b0a16415dc07. Squashed. ()
Removing f61d7ac5814b. Squashed. ()
Removing 7397b069ec3e. Squashed. ()
Removing 2587e40a84bf. Squashed. ()
Removing 27a7e9df4b42. Squashed. ()
Removing faa7ad651836. Squashed. ()
Removing b5ae67ec3dcb. Squashed. (/bin/sh -c apk del gcc g++ linux-headers paxctl...)
Tarring up squashed layer fcb5419f383b

Cannot mkdir: No space left on device

When I use the docker squash to reduce my docker image size with the command "docker save | sudo docker-squash -t newtag | docker load", I met the problem such like
tar: home/work_87586/dbzip_shiphome: Cannot mkdir: No space left on device

And my image size is 26.45G.
Here is the details info of my container(which run from the image which I want to reduce size)
[root@f7ece21 /]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/loop0 100G 29G 71G 29% /
tmpfs 7.3G 0 7.3G 0% /dev
shm 64M 0 64M 0% /dev/shm
tmpfs 7.3G 0 7.3G 0% /sys/fs/cgroup
/dev/loop0 100G 29G 71G 29% /etc/hosts

How can i fix this kind of problem?

cannot find package "github.com/docker/docker/pkg/units"

Could you please add some instructions to the README.md for people who don't know golang?

I tried to Google this and found these instructions for installing Go packages: http://stackoverflow.com/questions/10772799/how-to-install-golang-3rd-party-projects-from-download-sources

That seems strange and overly complicated but if it's right I'll give it a whirl.

It seems like "units" is the Go package I need to build this, but Googling "golang package units", I can't find anything plausible in the first two pages.

Building docker-squash
export.go:19:2: cannot find package "github.com/docker/docker/pkg/units" in any of:
        /usr/lib/go/src/pkg/github.com/docker/docker/pkg/units (from $GOROOT)
        /root/t/docker-squash/src/github.com/docker/docker/p

tl;dr would be nice if what I needed to know about Go to build this was included in the README.md since the solution doesn't seem to be immediately searchable =)

Thanks

OSX Build failing

Followed all the instructions on osx_install.md.

[ankush@goten:~/git/go/src/github.com/jwilder/docker-squash on master]
$ GOBIN=$(pwd) make
Building docker-squash
# _/Users/ankushagarwal/git/go/src/github.com/jwilder/docker-squash
./export.go:355: cannot use size (type int64) as type float64 in argument to units.HumanSize
make: *** [docker-squash] Error 2

$ go version                                                                                                                                                                    2 ↵
go version go1.4 darwin/amd64

OSX Version
10.10.1 (14B25)

-from should support "last squash" or "last from" so it's easier to build multiple squashed layers on top of each other.

Hi!

I'm building an environment where I have a minimal base-image, i then add layers on top of it to keep the required image size small so we can keep the filesystem usage on the servers (which is in ram) as low as possible.

When i squash the build of base-image/biglayer1/biglayer2/biglayer3 where i only want to squash biglayer3 docker-squash will squash it down to a new layer directly on top of biglayer1, so now i have biglayer2, and biglayer2&3 in two images, not biglayer2 and the addon biglayer3 where needed.

This is what it looks like when only first and second layer (java, java/mesos, java/zookeeper) has been squashed. every layer depends on the previous one.

docker images -a --tree
Warning: '--tree' is deprecated, it will be removed soon. See usage.
└─16a45869d60a Virtual Size: 95.27 MB Tags: trusty/minbase:latest
  └─0131ed4dbe05 Virtual Size: 414.3 MB Tags: trusty/java:latest
    ├─4251ca7211b9 Virtual Size: 418.9 MB
    │ └─e7f4bca2c41a Virtual Size: 418.9 MB Tags: trusty/zookeeper:latest
    └─840bad29bff6 Virtual Size: 577.8 MB
      └─4435bbaee79f Virtual Size: 577.8 MB Tags: trusty/mesos:latest
        └─82c13b47631d Virtual Size: 723.7 MB
          └─33621f55ac5b Virtual Size: 751 MB
            └─1c9e071a8c56 Virtual Size: 1.145 GB
              └─0a8936865afc Virtual Size: 1.176 GB
                └─30d31d294d37 Virtual Size: 1.176 GB
                  └─518bca946236 Virtual Size: 1.177 GB
                    └─76430b0182b9 Virtual Size: 1.178 GB
                      └─d4667f91fa20 Virtual Size: 1.178 GB
                        └─49497d6ebcd4 Virtual Size: 1.178 GB
                          └─ed12d73f94c3 Virtual Size: 1.178 GB
                            └─f16a5b30bcaa Virtual Size: 1.178 GB
                              └─e67fd997dbe8 Virtual Size: 1.178 GB Tags: trusty/aurora_tmp:latest

If i squash aurora it will end up directly under the java-layer, not under java/mesos/aurora where it belongs.

docker save trusty/aurora_tmp | sudo ./docker-squash -t trusty/aurora | docker load
docker images -a --tree
Warning: '--tree' is deprecated, it will be removed soon. See usage.
└─16a45869d60a Virtual Size: 95.27 MB Tags: trusty/minbase:latest, 127.0.0.1:5000/trusty/minbase:latest
  └─0131ed4dbe05 Virtual Size: 414.3 MB Tags: trusty/java:latest
    ├─037c9eaf296d Virtual Size: 709.4 MB
    │ └─b995defbf385 Virtual Size: 709.4 MB
    │   └─b009ced4b759 Virtual Size: 709.4 MB Tags: trusty/aurora:latest <- this should be beneath mesos-layer.
    ├─4251ca7211b9 Virtual Size: 418.9 MB
    │ └─e7f4bca2c41a Virtual Size: 418.9 MB Tags: trusty/zookeeper:latest
    └─840bad29bff6 Virtual Size: 577.8 MB
      └─4435bbaee79f Virtual Size: 577.8 MB Tags: trusty/mesos:latest

If the "-from" flag supported something like "last-defined from" or "first layer after last squash" it would be layering aurora on top of mesos properly instead of creating a new aurora&mesos layer.

Tagging with private repository names fails with error

When tagging output image with localhost:5000/imgname:tag the process ends up with fatal error:

FATA[1068] Error: Illegal tag name (5000/imgname): only [A-Za-z0-9_.-] are allowed, minimum 2, maximum 30 in length.

It is possibly a bug with regular expression query.

Is docker-squash abandoned?

Dear @jwilder and/or related, are we still going to move this forward?

I tried to fix the build issue with package [go-]units, in the hope of making it into Homebrew, but found out several people having been dealing with it according to a number of latest pull requests (#55, #48) . But none of them is merged or actively followed. The same applies to issues, so does the commit history, which was last updated half a year ago.

Let's try to be positive, maybe someone capable can lend a hand?

Silently fails with Docker 1.10RC2

I tried with Docker 1.10RC2 and there's no error messages but in the end I don't seem to get a new image created. I'm using this form:

docker save img | docker-squash -t img-squashed | docker load

Any ideas to make it work? We regularly use docker-squash to deploy apps at Cosmology@Home so it'd be great to keep it working.

Bug: panic: runtime error: invalid memory address or nil pointer dereference

reproduce:

docker save -o compressed.tar init:compressed
sudo docker-squash -i ~/compressed.tar -o squashed.tar

images history:

arthurwangs-MacBook-Pro:docker-squash arthur$ docker history init:compressed
IMAGE               CREATED             CREATED BY          SIZE                COMMENT
1d345e8af171        2 days ago                              953.6 MB            Imported from -

docker-squash must the newest version since I just download it at 3 days ago.

docker version:

arthurwangs-MacBook-Pro:docker-squash arthur$ docker version
Client:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:49:29 UTC 2015
 OS/Arch:      darwin/amd64

Server:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:49:29 UTC 2015
 OS/Arch:      linux/amd64

Error 1473, pushing a loaded docker image from a squashed tar file.

Hi. First this software is great. I was able to squash a very large image down to 10GB, and run the new docker image.

Squash version

sudo docker-squash -v
v0.1.0

the issue I am having is that after squashing the docker image, I can not pushed the new squashed image onto the docker hub.

For my example , I squashed my 40GB image into a file called squashed.tar. then loaded the squashed.tar into a new image called arcolombo/artemisunlayer:v1. I can create containers successfully and everything looks great.
docker save > aretemis.v156.squashed.tar
sudo docker-squash -i artemis.v156.squashed.tar -o squashed.tar -t arcolombo/artemisunlayer:v1
cat squashed.tar | docker load

anthonycolombo@quigonjinn:/Documents$ ls
Artemis artemis.v156.squashed.tar squashed.tar
anthonycolombo@quigonjinn:/Documents$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
arcolombo/artemisunlayer v1 3b15a479aa85 31 minutes ago 10.21 GB
arcolombo/artemis v156 de9bc789af1d 8 days ago 39.91 GB

when I run sudo docker push arcolombo/artemisunlayer:v1

the error is

The push refers to a repository [arcolombo/artemisunlayer](len: 1)
3b15a479aa85: Image already exists
f43a789562e2: Image already exists
20895e1cb336: Image already exists
8bf26c72ef69: Image already exists
90e1f131e119: Image push failed
FATA[1416] Error pushing to registry: Put https://registry-1.docker.io/v2/arcolombo/artemisunlayer/blobs/uploads/4accb54c-a964-4153-ac24-7a8610662ec0?_state=gx-cY2jt1fzbwLyOcnLYZKxt_e4-1cCJ8bK8WUvQwiV7Ik5hbWUiOiJhcmNvbG9tYm8vYXJ0ZW1pc3VubGF5ZXIiLCJVVUlEIjoiNGFjY2I1NGMtYTk2NC00MTUzLWFjMjQtN2E4NjEwNjYyZWMwIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDE1LTEwLTEyVDE4OjQ1OjQxLjAyMDAwNDg5N1oifQ%3D%3D&digest=sha256%3A1da5e7d6dc4f04d8038b88fe7fd4e726d71e24fc98f4095c711d63b67d760979: use of closed network connection

I am using a wired network for ethernet ? This sounds like a docker bug? but just looking for a confirm?

thank you again for your great software

Anthony C.

anthonycolombo@quigonjinn:/Documents$ sudo docker info
[sudo] password for anthonycolombo:
Containers: 1
Images: 129
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 131
Dirperm1 Supported: true
Execution Driver: native-0.2
Kernel Version: 3.16.0-50-generic
Operating System: Ubuntu 14.04.3 LTS
CPUs: 4
Total Memory: 7.712 GiB
Name: quigonjinn
ID: U5LV:PN5L:LQZY:B36M:IJF7:LYRS:LEQB:2NMY:RNVY:IYYV:GWRI:YRBO
Username: arcolombo
Registry: [https://index.docker.io/v1/]
WARNING: No swap limit support
anthonycolombo@quigonjinn:/Documents$ sudo docker version
Client version: 1.6.2
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): 7c8fca2
OS/Arch (client): linux/amd64
Server version: 1.6.2
Server API version: 1.18
Go version (server): go1.4.2
Git commit (server): 7c8fca2
OS/Arch (server): linux/amd64

So I've tested the docker-squash on a very large image (40GB) and realized that perhaps maybe this error of push a docker squashed image was specific to my image.

I tested docker-squash on a test image that is very small , 4GB, and squashed it, and tried to then push the docker squashed loaded image, and generated the error.

anthonycolombo@quigonjinn:~/Documents/trnaDB$ sudo docker push arcolombo/newtag
The push refers to a repository [docker.io/arcolombo/newtag](len: 1)
378d25546ce5: Image successfully pushed
dbddfc1d9325: Image successfully pushed
7325be38363d: Pushing 2.997 GB/3.025 GB
EOF

Here are the commands use to generate this error

sudo docker save 40f425f4e713 > image.tar
sudo docker-squash -i image.tar -o mut.squashed.tar -t arcolombo/newtag
cat mut.squashed.tar | sudo docker load
sudo docker push arcolombo/newtag

anthonycolombo@quigonjinn:~/Documents/trnaDB$ sudo docker push arcolombo/newtag
The push refers to a repository [docker.io/arcolombo/newtag](len: 1)
378d25546ce5: Image successfully pushed
dbddfc1d9325: Image successfully pushed
7325be38363d: Pushing 2.997 GB/3.025 GB
EOF

It appears that there is an unexpected EOF in the middle of these squashed layers.

docker-squash without sudo

Is it possible to use docker-squash without sudo, i'm trying to write a script to build our web app, and trying to add docker-squash to the bash script. But its failing without sudo. All the examples in the project README.md are also with sudo. Is there any workaround to use docker-squash without sudo ?

Untar fork/exec /usr/bin/docker: argument list too long

Hi,

Thanks for doing this project1 I am generating an intermediate image due to huge build who's artifacts I remove but leaves my image bloated.

I'm using docker 1.3.2. and a pull of your repo from today. I was excited to see that docker-squash has shrunken my image from 9.9G to 2.6 but docker is gacking on loading it back in. I can save/load other images with docker but I think something must happening with the generated json or something.

Can you point me at a way to troubleshoot this? Here is my pipeline:

sudo docker save 3e8458581665 | sudo docker-squash -verbose -t hhvm_builder_squashed | sudo docker load
                  -snip-
  -  Replacing 3e8458581665 w/ new layer 5eb58410aab4 (/bin/sh -c #(nop) CMD [/bin/bash])
Tarring up squashed layer 327e680e08d5
Removing extracted layers
Tagging 5eb58410aab4 as hhvm_builder_squashed:latest
Tarring new image to STDOUT
Done. New image created.
                  -snip-
  -  c8ce5a628d02 9 seconds /bin/sh -c #(nop) ENV LD_RUN_PATH=/opt/hhvm-dev/rtf/lib:/... 1.024 kB
  -  5eb58410aab4 8 seconds /bin/sh -c #(nop) CMD [/bin/bash] 1.024 kB
Removing tempdir /tmp/docker-squash547210872
2014/12/15 15:44:35 Error: Untar fork/exec /usr/bin/docker: argument list too long

FWIW, there is a seemingly unrelated docker issue -- though it is does involve a forthcoming docker squash command -- that matches the error I'm getting here. Have you run into this before?

Thanks
Chris

invalid memory address or nil pointer dereference

Got the following when trying to squash library/centos:7. This is using Docker 1.5 on ubuntu and 0.0.11.

docker save library/centos:7 | sudo ./dockerbin/docker-squash -t squashed -verbose | docker load
Loading export from STDIN using /tmp/docker-squash897833692 for tempdir
Loaded image w/ 3 layers
  -  library/centos (1 tags)
Extracting layers...
  -  /tmp/docker-squash897833692/511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158/layer.tar
  -  /tmp/docker-squash897833692/5b12ef8fd57065237a6833039acc0e7f68e363c15d8abb5cacce7143a1f7de8a/layer.tar
  -  /tmp/docker-squash897833692/dade6cb4530a852b83bc34f6f3904a10879632947c01f1ae0447ff9dbb37e12b/layer.tar
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x50 pc=0x4042a2]

goroutine 16 [running]:
runtime.panic(0x5842e0, 0x693cf3)
    /usr/local/go/src/pkg/runtime/panic.c:279 +0xf5
main.(*Export).InsertLayer(0xc2080423c0, 0xc20800e400, 0x40, 0xc208095d98, 0x0, 0x0)
    /home/jwilder/go/src/github.com/jwilder/docker-squash/export.go:393 +0x7d2
main.main()
    /home/jwilder/go/src/github.com/jwilder/docker-squash/main.go:130 +0xd37

goroutine 19 [finalizer wait]:
runtime.park(0x41ea00, 0x697d08, 0x696829)
    /usr/local/go/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x697d08, 0x696829)
    /usr/local/go/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
    /usr/local/go/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1445

goroutine 20 [syscall]:
os/signal.loop()
    /usr/local/go/src/pkg/os/signal/signal_unix.go:21 +0x1e
created by os/signal.init·1
    /usr/local/go/src/pkg/os/signal/signal_unix.go:27 +0x32

goroutine 21 [chan receive]:
main.shutdown(0xc208042380, 0x1b)
    /home/jwilder/go/src/github.com/jwilder/docker-squash/main.go:22 +0x60
created by main.main
    /home/jwilder/go/src/github.com/jwilder/docker-squash/main.go:73 +0x6f2
FATA[0006] Error: EOF

It seems not working

I have a 900+M big image:

arthurwangs-MacBook-Pro:docker-tar arthur$ docker history 552bb0624af2
IMAGE               CREATED             CREATED BY          SIZE                COMMENT
552bb0624af2        About an hour ago   /bin/bash           110.1 MB            
1d345e8af171        3 days ago                              953.6 MB            Imported from -

I use squash to compress it , but it still remain in 1GB+ size:

arthurwangs-MacBook-Pro:docker-tar arthur$ docker save 552bb0624af2 | sudo docker-squash -verbose -t root:1 | docker load
Password:
Loading export from STDIN using /tmp/docker-squash046536096 for tempdir
Loaded image w/ 2 layers
Extracting layers...
  -  /tmp/docker-squash046536096/1d345e8af17112fa644f04ea6d93d10241b344e412f968bab81e337c9dd1bdae/layer.tar
  -  /tmp/docker-squash046536096/552bb0624af20d79ed1fe374eda40b4459e933aca812bc85d4784aeb44913320/layer.tar
Inserted new layer e56cfb97521e after 1d345e8af171
  -  1d345e8af171 
  -> e56cfb97521e /bin/sh -c #(squash) from 1d345e8af171
  -  552bb0624af2 /bin/bash
Squashing from e56cfb97521e into e56cfb97521e
  -  Deleting whiteouts for layer 552bb0624af2
  -  Rewriting child history
  -  Removing 552bb0624af2. Squashed. (/bin/bash)
Tarring up squashed layer e56cfb97521e
Removing extracted layers
Tagging e56cfb97521e as root:1
Tarring new image to STDOUT
Done. New image created.
  -  1d345e8af171 3 days  977.4 MB
  -  e56cfb97521e 19 seconds /bin/sh -c #(squash) from 1d345e8af171 110.3 MB
Removing tempdir /tmp/docker-squash046536096
The image root:1 already exists, renaming the old one with ID c47f98670d7d to empty string
arthurwangs-MacBook-Pro:docker-tar arthur$ docker image
docker: 'image' is not a docker command.
See 'docker --help'.
arthurwangs-MacBook-Pro:docker-tar arthur$ docker images
REPOSITORY              TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
root                    1                   e56cfb97521e        2 minutes ago       1.064 GB
<none>                  <none>              c47f98670d7d        About an hour ago   1.064 GB
newtag                  latest              e25fb3d04c4f        About an hour ago   1.064 GB
rcontainer              iptableconfig       552bb0624af2        About an hour ago   1.064 GB
<none>                  <none>              8102a6a36300        4 hours ago         990.1 MB
init                    compressed          1d345e8af171        3 days ago          953.6 MB

Is there any steps Im missing ?

COPY behaves differently than ADD while squashing

The COPY command behaves similarly to ADD in a Dockerfile, yet when squashed, the COPY commands are replaced after the squashed image, while the ADD commands are removed.

Chances are, including a check for COPY on line 591 of export.go will fix it. For now, I've replaced COPY with ADDs in my Dockerfile.

-t reports "bad tag format" for any value

Including -t squash, as given in the README.

 docker save $IMAGE | sudo docker-squash -t squash -verbose | docker load
ERROR: bad tag format: squash

Add cryptographic checksums for releases

Currently, I can't find MD5s or SHAs for the binaries in the releases section. Would it be possible to have them added so that I can be sure that the binary is in fact what I expect it to be?

docker/docker/pkg/units not found

Trying to install on Mac El Cap
units does not appear to be in the docker/docker/pkg repo

TheBeast:nodecontainerbuddybase donvawter$ go get github.com/docker/docker/pkg/units
package github.com/docker/docker/pkg/units: cannot find package "github.com/docker/docker/pkg/units" in any of:
/usr/local/Cellar/go/1.5.3/libexec/src/github.com/docker/docker/pkg/units (from $GOROOT)
/Users/donvawter/.go/src/github.com/docker/docker/pkg/units (from $GOPATH)

Error in setting up env to compile on Mac.

I followed the instruction to compile the code on Mac but I am still having the issue of not being able to get the units. I saw a post mentioning synching using glock. So I installed glock and ran the command:

open ~/.go/src/github.com/jwilder/docker-squash/GLOCKFILE: no such file or directory

It still gave me an error. Do I need to clone the project into .go folder? I am just not sure if that is supposed to be the case. Thanks for your help.

Broken link

The readme instructs the user to download from this location

https://github.com/jwilder/docker-squash/releases/download/v0.0.8/docker-sqush-linux-amd64-v0.0.8.tar.gz

however that link doesn't exist

OSX: tar: Option -O is not permitted in mode -c

Using master branch with 3dc9b8d

Tarring up squashed layer 2f0b6a270d34
Removing extracted layers
Tagging a99f495e54a5 as squash:latest
Tarring new image to STDOUT
tar: Option -O is not permitted in mode -c
ERROR: exit status 1
Removing tempdir /tmp/docker-squash388277481
invalid argument

gtar --version
tar (GNU tar) 1.29

Configuration commands can be combined

Some of the configuration commands can be combined into single commands (or removed). I'm not sure if this is something we'd want to do for all of them, but it can certainly remove some layers. For example, the following two docker files are equivalent:

7 Layers:

FROM ubuntu:14.04.2

CMD command_one

ENV env_one=1
ENV env_two=2
ENV env_three=3
ENV env_four=4

CMD command_two
CMD command_three

Two Layers:

FROM ubuntu:14.04.2

ENV env_one=1 \
    env_two=2 \ 
    env_three=3 \
    env_four=4

CMD command_three

Does not seem to work with Docker. 1.10

I simply cannot see my new image after the job's done.

Has anyone tried docker-squash with the latest Docker version?

Unable to squash

Hello,
I am running this command on Mac OS Sierra:
sudo docker-squash -i igorganapolsky.tar -o igorganapolsky-squashed.tar

However, I get this error:

tar: Option --xattrs is not supported
Usage:
List: tar -tf
Extract: tar -xf
Create: tar -cf [filenames...]
Help: tar --help
ERROR: exit status 1

What is causing this?

Thanks,
igor

Does a docker-squash remove the intermediate layers from the build cache?

From moby/moby#13490

Specifically:

Mark .. Squash / Flatten layers. (#332, #12198, #4232, #9591). Squashing layers will remove the intermediate layers from the final image, however, secrets used in those intermediate layers will still end up in the build cache.

The issues are talking about PRs containing squash commands that are probably implemented differently but the concept is the same I believe. I'm wondering if docker-squash addressed this concern.

Fails on squashed.tar loading

$ docker save > image.tar
$ sudo docker-squash -i image.tar -o squashed.tar -t newtag
$ cat squashed.tar | docker load
$ docker images

Fails silently after doing cat squashed.tar | docker load. Image therefore is not in the list of images.

Docker version 1.10.0, build 590d5108

Pre-squashed image id still being search on docker load

Docker Load Error:

FATA[0000] Error response from daemon: could not find image: no such id: 0925b626df7d82abca7a9d3073b1549cfba51c9eab1b55161f1b17d5fc2c1c36

Image id from pre-squash is causing error on docker load. Theoretically, the image should be combined with the newly created squashed image, so it shouldn't exist anymore? It even says "removing 0925..." but for some reason docker load is requiring the image.

Loading export from ubuntu.tar using /tmp/docker-squash776676198 for tempdir
Loaded image w/ 6 layers
  -  127.0.0.1:5000/ubuntu (1 tags)
Extracting layers...
  -  /tmp/docker-squash776676198/0925b626df7d82abca7a9d3073b1549cfba51c9eab1b55161f1b17d5fc2c1c36/layer.tar
  -  /tmp/docker-squash776676198/29460ac934423a55802fcad24856827050697b4a9f33550bd93c82762fb6db8f/layer.tar
  -  /tmp/docker-squash776676198/83e4dde6b9cfddf46b75a07ec8d65ad87a748b98cf27de7d5b3298c1f3455ae4/layer.tar
  -  /tmp/docker-squash776676198/b670fb0c7ecd3d2c401fbfd1fa4d7a872fbada0a4b8c2516d0be18911c6b25d6/layer.tar
  -  /tmp/docker-squash776676198/d2a0ecffe6fa4ef3de9646a75cc629bbd9da7eead7f767cb810f9808d6b3ecb6/layer.tar
  -  /tmp/docker-squash776676198/dbdf96cbef6e46a54db522f963378758144163bbdc9ab99d6913703ea5557a5d/layer.tar
Inserted new layer bcc84cd3273a after 83e4dde6b9cf
  -  83e4dde6b9cf /bin/sh -c #(nop) ADD file:c8f078961a543cdefaeb73d06e55440d0
  -> bcc84cd3273a /bin/sh -c #(squash) from 83e4dde6b9cf
  -  b670fb0c7ecd /bin/sh -c echo '#!/bin/sh' > /usr/sbin/policy-rc.d   && echo
  -  29460ac93442 /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$/\1/g' /etc/apt/
  -  d2a0ecffe6fa /bin/sh -c #(nop) CMD ["/bin/bash"]
  -  dbdf96cbef6e bash
  -  dbdf96cbef6e bash
  -  0925b626df7d bash
Squashing from bcc84cd3273a into bcc84cd3273a
  -  Deleting whiteouts
  -  Rewriting child history
  -  Removing b670fb0c7ecd. Squashed. (/bin/sh -c echo '#!/bin/sh' > /usr/sbin/policy-...)
  -  Removing 29460ac93442. Squashed. (/bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$/\1...)
  -  Replacing d2a0ecffe6fa w/ new layer 7c0f35e741b7 (/bin/sh -c #(nop) CMD ["/bin/bash"])
  -  Removing dbdf96cbef6e. Squashed. (bash)
  -  Removing 0925b626df7d. Squashed. (bash)
Tarring up squashed layer bcc84cd3273a
Removing extracted layers
Tagging 7c0f35e741b7 as squashed/ubuntu:latest
Tarring new image to squashed.tar
Done. New image created.
  -  83e4dde6b9cf 6 days /bin/sh -c #(nop) ADD file:c8f078961a543cdefaeb73d06e5544... 197.2 MB
  -  bcc84cd3273a 7 seconds /bin/sh -c #(squash) from 83e4dde6b9cf 329.6 MB
  -  7c0f35e741b7 5 seconds /bin/sh -c #(nop) CMD ["/bin/bash"] 1.024 kB
Removing tempdir /tmp/docker-squash776676198

root@docker-fast:/home# cat squashed.tar | docker load
FATA[0002] Error response from daemon: could not find image: no such id: 0925b626df7d82abca7a9d3073b1549cfba51c9eab1b55161f1b17d5fc2c1c36

Idea: Remove brew from dependency requirements

Perhaps instead of requiring that someone use brew to install things you can have another docker which has necessary pre-requisites to compile to go source. Ideally, the only pre-requisites would be go and docker.

Not Squashing....

Trying to squashed a Debian Jessie image I removed several item with using:

apt-get update && \
    apt-get upgrade -y && \
    apt-get clean -y && \
    apt-get autoclean -y && \
    apt-get autoremove -y && \
    rm -rf /usr/share/locale/* && \
    rm -rf /var/cache/debconf/*-old && \
    rm -rf /var/lib/apt/lists/* && \
    rm -rf /usr/share/doc/* && \
    exit

However, the image after running docker squash is still the same size:

rancher@rancheros:~$ docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
raido/min-jessie      latest             9b58925bbb33        12 seconds ago      125.1 MB
debian                   jessie              d1f66aef36c9        10 days ago         125.1 MB


rancher@rancheros:~$ docker save 9b58925bbb33 | sudo ./docker-squash -from root -t raido/min-jessie -verbose | docker load
Loading export from STDIN using /tmp/docker-squash029083388 for tempdir
Loaded image w/ 3 layers
Extracting layers...
  -  /tmp/docker-squash029083388/d0ca40da9e35b5da0b9f2859242ded0dd868d5244b73f4e72226f72012358e96/layer.tar
  -  /tmp/docker-squash029083388/d1f66aef36c95d9c80da2c35a51c432c7cb12f7033377b7c22263870cf1f2295/layer.tar
  -  /tmp/docker-squash029083388/9b58925bbb33968daf4a10efb4ed4ed0b0c036b033d4c2d666d986a096a3bdce/layer.tar
Inserted new layer d8dd2e1f8a00 after d0ca40da9e35
  -  d0ca40da9e35 /bin/sh -c #(nop) ADD file:5f0fca122586fe4ea79c5f22acef96c38
  -> d8dd2e1f8a00 /bin/sh -c #(squash) from d0ca40da9e35
  -  d1f66aef36c9 /bin/sh -c #(nop) CMD ["/bin/bash"]
  -  9b58925bbb33 /bin/bash
Squashing from d8dd2e1f8a00 into d8dd2e1f8a00
  -  Deleting whiteouts for layer d1f66aef36c9
  -  Deleting whiteouts for layer 9b58925bbb33
  -  Rewriting child history
  -  Replacing d1f66aef36c9 w/ new layer fbb3358df2b7 (/bin/sh -c #(nop) CMD ["/bin/bash"])
  -  Removing 9b58925bbb33. Squashed. (/bin/bash)
Tarring up squashed layer d8dd2e1f8a00
Removing extracted layers
Tagging fbb3358df2b7 as raido/min-jessie:latest
Tarring new image to STDOUT
Done. New image created.
  -  d0ca40da9e35 10 days /bin/sh -c #(nop) ADD file:5f0fca122586fe4ea79c5f22acef96... 130.9 MB
  -  d8dd2e1f8a00 1 seconds /bin/sh -c #(squash) from d0ca40da9e35 10.24 kB
  -  fbb3358df2b7 1 seconds /bin/sh -c #(nop) CMD ["/bin/bash"] 1.024 kB
Removing tempdir /tmp/docker-squash029083388
rancher@rancheros:~$ docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             VIRTUAL SIZE
raido/min-jessie      latest              fbb3358df2b7        12 seconds ago      125.1 MB
debian                   jessie              d1f66aef36c9        10 days ago         125.1 MB

Remove xattrs for Ubuntu precise

It looks like in Ubuntu 12.04, there is no tar --xattrs option. Is it truly needed here and can it be removed?

Crashes

After installing docker-squash, it crashes while trying to compress an image
Running 0.8.0 on cent 6.5 with docker 1.2

[foo@bar ~]$ sudo docker save 12341f681234 | sudo /usr/local/bin/docker-squash -t foo/bar:squash -verbose -from root | sudo docker load
Loading export from STDIN using /tmp/docker-squash792282776 for tempdir
Loaded image w/ 28 layers
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x402b9e]

goroutine 16 [running]:
runtime.panic(0x584120, 0x693cf3)
    /usr/local/go/src/pkg/runtime/panic.c:279 +0xf5
main.(*Export).firstLayer(0xc20803e040, 0x5aacb0, 0x9, 0x0)
    /home/jwilder/go/src/github.com/jwilder/docker-squash/export.go:247 +0xfe
main.(*Export).FirstSquash(0xc20803e040, 0xc208022540)
    /home/jwilder/go/src/github.com/jwilder/docker-squash/export.go:261 +0x3b
main.main()
    /home/jwilder/go/src/github.com/jwilder/docker-squash/main.go:98 +0xb65

goroutine 19 [finalizer wait, 2 minutes]:
runtime.park(0x41e910, 0x697d08, 0x696829)
    /usr/local/go/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x697d08, 0x696829)
    /usr/local/go/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
    /usr/local/go/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
    /usr/local/go/src/pkg/runtime/proc.c:1445

goroutine 20 [syscall, 2 minutes]:
os/signal.loop()
    /usr/local/go/src/pkg/os/signal/signal_unix.go:21 +0x1e
created by os/signal.init·1
    /usr/local/go/src/pkg/os/signal/signal_unix.go:27 +0x32

goroutine 21 [chan receive]:
main.shutdown(0xc20803e380, 0x1b)
    /home/jwilder/go/src/github.com/jwilder/docker-squash/main.go:22 +0x60
created by main.main
    /home/jwilder/go/src/github.com/jwilder/docker-squash/main.go:76 +0x7bc
2014/08/22 17:46:11 Error: EOF

I've tried reinstalling docker-squash, running with sudo, running as root, it always crashes with this error.

Support label feature

Hi,

It seems like the current docker-squash does not support Labels feature from docker after 1.6 (https://docs.docker.com/userguide/labels-custom-metadata/). It seems like a pretty decent feature for managing a lot of containers.

The fix should be a simple 1-2 line change by defining "Label map[string]string" in ContainerConfig/Config. I can send the PR if you'd like.

Thanks!

tar --xattrs only works on centos

I'm trying to use docker-squash within google/golang images. but got this error message.

root@d9598f24a46c:/gopath# docker save ce4190a0e33a | bin_debian7/docker-squash -t squash | docker load
tar: unrecognized option '--xattrs'
Try `tar --help' or `tar --usage' for more information.
ERROR: exit status 64
2014/11/11 07:59:16 Error: EOF

tar --xattrs seems only works on CentOS. both debian7 and max 10.9 don't support this option.

utils.go

17  cmd := exec.Command("tar", "--same-owner", "--xattrs", "--overwrite",
18          "--preserve-permissions", "-xf", src, "-C", dest)

centos6

$ cat /etc/centos-release 
CentOS release 6.6 (Final)
root ~ [8:44:57]
$ tar --help | grep xattrs
      --no-xattrs            Don't extract the user/root xattrs from the
      --xattrs               Save the user/root xattrs to the archive

google/golang

root@d9598f24a46c:/gopath# cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 7 (wheezy)"
NAME="Debian GNU/Linux"
VERSION_ID="7"
VERSION="7 (wheezy)"
ID=debian
ANSI_COLOR="1;31"
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support/"
BUG_REPORT_URL="http://bugs.debian.org/"
root@d9598f24a46c:/gopath# tar --help | grep xattrs
root@d9598f24a46c:/gopath#

OSX

oliveagle ~ [16:10:32]
$ tar --help | grep xattrs
oliveagle ~ [16:47:14]
$ uname
Darwin