petejkim / goop Goto Github PK

goop is so GREAT and I love it! Used it alot since I found it, thanks a lot for the tool.

But I have a minor problem recurring in my development process, in the process I often need to add new dependencies to the project, but I have problems installing ONLY the new dependencies without touching others.
The first thing I tried was adding an entry in the Goopfile then goop install, but goop install just ignore the changed Goopfile and picked-up the content of Goopfile.lock.
Then I tried goop update, it do install the new dependencies, but as the help text indicates it also update all dependencies to latest version, I'd rather not to do it this way as it may compromise the stability of my project.
Then I tried adding entries directly in Goopfile.lock then goop install, but this does not add entries of 'dependencies of dependencies' to the Goopfile.lock, thus leaving unlocked versions flying around.

So what should I do in this situation? I think my use case is a very general one, so maybe there is already a way to do it and I missed it.

Any differentiators worth considering?
https://github.com/mattn/gom
thanks!

if I add one dep in Goopfile, such as

github.com/gohttp/app #5fe28eda25acc96b48d8f2ddb651ea9104cc3e7c

I want to do goop install --auto-parse

and the deps of github.com/gohttp/app will be parsed, and saved to Goopfile

so, the result will be:

github.com/gohttp/app #5fe28eda25acc96b48d8f2ddb651ea9104cc3e7c
github.com/justinas/alice #e82c7d9ec9ebfd83e90d4fd39ce934f5fc0b0b5f
github.com/bmizerany/pat #b8a35001b773c267eb260a691f4e5499a3531600

In 1.5 the go command will read the environment variable GO15VENDOREXPERIMENT. If enabled, it will use the local "/vendor/" folder for dependencies.

Will you consider supporting this approach?

goop install does not build binaries for some packages. Specifically, Ginkgo.

Since goop does add .vendor/bin to the path, it seems that the expectation is that packages with binaries will build them and stick them in there.

I note that Ginkgo has its main package in a subpackage of the repo (under ginkgo/). Possibly this is an issue that needs to be solved in that package instead?

Imagine I depend on package X. Package X is a subfolder of some larger repository Y. I put package X into my goopfile and specify the revision of Y that I want. goop install will bring in package X, but it also drags in the dependencies of all the stuff in Y.

Real world use case, consider this Goopfile:

github.com/influxdb/influxdb/client #632dba81dcb9769cbc7005a83d52f28fc5c6c7cc

InfluxDB is a time series database written in Go... in other words, it is a big project. The official Go client for influx happens to live in their repo as well. The Go client has zero dependencies (stdlib only) but Influx's own dependency tree is considerably larger. To prove that the official client has no dependencies:

$ go get -u -v github.com/influxdb/influxdb/client
github.com/influxdb/influxdb (download)
github.com/influxdb/influxdb/client
$ 

Now try goop install on that Goopfile instead. Goop clones influx (as expected), checks it out to the right revision (as expected), but then starts computing the dependency tree for influx itself, and starts cloning all the dependency repositories as well. That ends up failing because influx has a bzr dependency, but the point is that none of those dependencies need to be cloned at all, because my dependency is on the client and not on the parent package.

I was going to ask on the mailing list for influx why the client was moved back into the main repo, but ultimately, this is not their issue. Proper dependency resolution for subpackages is a dependency manager (ie goop) problem.

Go version: 1.3
OS: OS X

Goopfile

github.com/zenazn/goji

Error message:

=> Installing github.com/zenazn/goji...
WORK=/var/folders/cg/gtgtb84d4cx135ml8wtpc88r0000gn/T/go-build596648851
mkdir -p $WORK/github.com/zenazn/goji/bind/_obj/
mkdir -p $WORK/github.com/zenazn/goji/
cd /private/tmp/test/.vendor/src/github.com/zenazn/goji/bind
/usr/local/Cellar/go/1.3/libexec/pkg/tool/darwin_amd64/6g -o $WORK/github.com/zenazn/goji/bind.a -trimpath $WORK -p github.com/zenazn/goji/bind -complete -D _/private/tmp/test/.vendor/src/github.com/zenazn/goji/bind -I $WORK -pack ./bind.go ./einhorn.go ./systemd.go
mkdir -p $WORK/github.com/zenazn/goji/graceful/_obj/
cd /private/tmp/test/.vendor/src/github.com/zenazn/goji/graceful
/usr/local/Cellar/go/1.3/libexec/pkg/tool/darwin_amd64/6g -o $WORK/github.com/zenazn/goji/graceful.a -trimpath $WORK -p github.com/zenazn/goji/graceful -complete -D _/private/tmp/test/.vendor/src/github.com/zenazn/goji/graceful -I $WORK -pack ./einhorn.go ./graceful.go ./middleware.go ./net.go ./signal.go
mkdir -p $WORK/github.com/zenazn/goji/web/_obj/
cd /private/tmp/test/.vendor/src/github.com/zenazn/goji/web
/usr/local/Cellar/go/1.3/libexec/pkg/tool/darwin_amd64/6g -o $WORK/github.com/zenazn/goji/web.a -trimpath $WORK -p github.com/zenazn/goji/web -complete -D _/private/tmp/test/.vendor/src/github.com/zenazn/goji/web -I $WORK -pack ./atomic.go ./fast_router.go ./func_equal.go ./middleware.go ./mux.go ./pattern.go ./router.go ./web.go
mkdir -p /private/tmp/test/.vendor/pkg/darwin_amd64/github.com/zenazn/goji/
mv $WORK/github.com/zenazn/goji/bind.a /private/tmp/test/.vendor/pkg/darwin_amd64/github.com/zenazn/goji/bind.a
mv $WORK/github.com/zenazn/goji/graceful.a /private/tmp/test/.vendor/pkg/darwin_amd64/github.com/zenazn/goji/graceful.a
mv $WORK/github.com/zenazn/goji/web.a /private/tmp/test/.vendor/pkg/darwin_amd64/github.com/zenazn/goji/web.a
mkdir -p $WORK/github.com/zenazn/goji/web/middleware/_obj/
mkdir -p $WORK/github.com/zenazn/goji/web/
cd /private/tmp/test/.vendor/src/github.com/zenazn/goji/web/middleware
/usr/local/Cellar/go/1.3/libexec/pkg/tool/darwin_amd64/6g -o $WORK/github.com/zenazn/goji/web/middleware.a -trimpath $WORK -p github.com/zenazn/goji/web/middleware -complete -D _/private/tmp/test/.vendor/src/github.com/zenazn/goji/web/middleware -I $WORK -I /private/tmp/test/.vendor/pkg/darwin_amd64 -pack ./envinit.go ./logger.go ./middleware.go ./nocache.go ./options.go ./realip.go ./recoverer.go ./request_id.go ./terminal.go ./writer_proxy.go
mkdir -p /private/tmp/test/.vendor/pkg/darwin_amd64/github.com/zenazn/goji/web/
mv $WORK/github.com/zenazn/goji/web/middleware.a /private/tmp/test/.vendor/pkg/darwin_amd64/github.com/zenazn/goji/web/middleware.a
mkdir -p $WORK/github.com/zenazn/goji/_obj/
mkdir -p $WORK/github.com/zenazn/
cd /private/tmp/test/.vendor/src/github.com/zenazn/goji
/usr/local/Cellar/go/1.3/libexec/pkg/tool/darwin_amd64/6g -o $WORK/github.com/zenazn/goji.a -trimpath $WORK -p github.com/zenazn/goji -complete -D _/private/tmp/test/.vendor/src/github.com/zenazn/goji -I $WORK -I /private/tmp/test/.vendor/pkg/darwin_amd64 -pack ./default.go ./goji.go
mkdir -p /private/tmp/test/.vendor/pkg/darwin_amd64/github.com/zenazn/
mv $WORK/github.com/zenazn/goji.a /private/tmp/test/.vendor/pkg/darwin_amd64/github.com/zenazn/goji.a
goop: readdirent: invalid argument

What's the rationale for prepending to GOPATH rather than replacing it entirely? Replacing it would result in (more) hermetic builds.

warning: code.google.com is shutting down; import path code.google.com/p/go.tools/go/vcs will stop working

receiving this error during goop install. is this library available through any other sources? thanks!

Python developer here. You wouldn't believe how useful it is to allow comments on dependencies declarations, we do that all the time in the python world (in our requirements.txt files, used to install dependencies with the pip python tool).
Things like "don't update to newer versions, this is the last compatible one", or "we are using this for feature/module x", etc.

seems the easiest / best way to use goop afaik, would be nice if it was mentioned in the readme

It would be nice if I could specify the directory where goop installs requirements. For instance, I'd want to use a .goenv directory instead - I use .virtualenv for python, and .rbenv for ruby.

I'm not sure what this would look like - it looks like you might parse the Goopfile after you create the NewGoop instance, so adding it to the Goopfile is out. Perhaps an environment variable? GOOP_VENDOR_DIR?

I've been able to use goop with a trivial main.go example, but I've been unable to figure out how to use goop with a folder structure containing a package main and one or more other packages.

My folder structure looks like below:

Goopfile
src/myproj
   - main.go
   - client/client.go

I'm using goop go build src/myproj/main.go to build package main, but for the life of me I can't figure out how to build/import the client package. I would really appreciate an example that works.

Support exporting env to fish shell. See rbenv/rbenv@be5e1a4

Using ... at the end of a go get, will fetch all of the packages under that path. But in some projects this will import unnecessary packages.. for example like, https://github.com/garyburd/go-oauth. Adding that to your Goopfile will try to fetch the "examples" package and add the appengine as a dependency, which clear is not a good thing :|

go get github.com/nitrous-io/goop
package code.google.com/p/go.tools/go/vcs: Get https://code.google.com/p/go/source/checkout?repo=tools: x509: failed to load system roots and no roots provided

Where did I go wrong?

The project go-restful imports broken packages in it's examples/ directory, for example:

https://github.com/emicklei/go-restful/blob/master/examples/google_app_engine/restful-appstats-integration.go

This makes using go-restful impossible with goop.

This common inclusion of an examples/ directory should probably be counted as a place not to look for imports, if you have that within your power.

Error log from a Goopfile containing nothing but github.com/emicklei/go-restful:

$ GOPATH=$(pwd) goop install
=> Fetching github.com/emicklei/go-restful...
Cloning into '/api/.vendor/tmp/src/github.com/emicklei/go-restful'...
remote: Reusing existing pack: 1862, done.
remote: Counting objects: 5, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 1867 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (1867/1867), 4.80 MiB | 932 KiB/s, done.
Resolving deltas: 100% (959/959), done.
Checking out "c12ca7685cf7b6e123f90cd30556d1065e37a5e6"
=> Fetching dependencies for github.com/emicklei/go-restful...
github.com/gorilla/schema (download)
import "appengine": import path doesn't contain a slash
package github.com/gorilla/schema
    imports appengine: unrecognized import path "appengine"
import "appengine/memcache": import path doesn't contain a hostname
package github.com/gorilla/schema
    imports appengine/memcache: unrecognized import path "appengine/memcache"
github.com/mjibson/appstats (download)
import "appengine/user": import path doesn't contain a hostname
package github.com/gorilla/schema
    imports github.com/mjibson/appstats
    imports appengine/user: unrecognized import path "appengine/user"
import "appengine_internal": import path doesn't contain a slash
package github.com/gorilla/schema
    imports github.com/mjibson/appstats
    imports appengine_internal: unrecognized import path "appengine_internal"
code.google.com/p/goprotobuf (download)
import "appengine/datastore": import path doesn't contain a hostname
package github.com/gorilla/schema
    imports github.com/mjibson/appstats
    imports code.google.com/p/goprotobuf/proto
    imports appengine/datastore: unrecognized import path "appengine/datastore"
goop: Command failed with exit status 1

I will work around this by forging the import path, and working from a clone of the repository in my own name with the examples directory removed.

cannot find package "code.google.com/p/go.tools/go/vcs" in any of...

The typical use of goop is to have a Goopfile (and .lock) at the root of a project. Running goop install creates a .vendor in the project directory wherever the Goopfiles are defined.

A pattern that I like, started by Ben Johnson, is having a cmd directory that is actually the main package of the project to create the cli binary. This is a good pattern because it encourages writing your project as a library that can be imported by others. As a simple example see: https://github.com/boltdb/boltd

Now when the current working directory is $GOPATH/src/github.com/boltdb/boltd/cmd/boltd .. and then running goop go build it would be nice for goop to go up the chain until it finds the first Goopfile/.lock/.vendor and use that namespace.

Hope that makes sense. Otherwise this means you have to run goop in the root of the project everytime, which is fine but this isn't clear. The downside of searching the children is when do you stop, in case someone forgets to have a Goopfile somewhere ... but, maybe that isn't so bad.

go get github.com/nitrous-io/goop
imports code.google.com/p/go.tools/go/vcs: exec: "hg": executable file not found in $PATH

this seems strange ...

I might be mistaken, but it seems that goop creates a lock file for the top level dependencies only. For example, if I list module 'A' as a dependency and it has module 'B' as a dependency, my lock file will contain a revision hash for module A but nothing about module B.

Is that correct? If so, how can we make sure that changes to module B would not affect our locked versions in future?

Hi there,

I think I may have found a bug with how Goopfile works. It seems that if you have multiple dependencies located in the same GitHub repo, the individual dependencies must be listed in the order oldest to newest or else they do not get updated properly.

For example, for my project called MumbleDJ, I need to fetch five dependencies: gumble, gumble_ffmpeg, gumbleutil, gcfg, and jsonq. If I put the dependencies in my Goopfile in this order, it does not completely update all of the dependencies (the commit specified for gumble is newer than the commit for gumble_ffmpeg, and gumble_ffmpeg is newer than gumbleutil):

github.com/layeh/gumble/gumble #80248757d681a7563d7257ff645ea27250f9d42d
github.com/layeh/gumble/gumble_ffmpeg #7ca5605a1e735c95442683ef47be117b0b9cb042
github.com/layeh/gumble/gumbleutil #29411f8e62add7e8f0856b35dc925cf879be3aab
code.google.com/p/gcfg #c2d3050044d0
github.com/jmoiron/jsonq #7c27c8eb9f6831555a4209f6a7d579159e766a3c

However, if I put them in this order, the updates are fetched correctly:

github.com/layeh/gumble/gumbleutil #29411f8e62add7e8f0856b35dc925cf879be3aab
github.com/layeh/gumble/gumble_ffmpeg #7ca5605a1e735c95442683ef47be117b0b9cb042
github.com/layeh/gumble/gumble #80248757d681a7563d7257ff645ea27250f9d42d
code.google.com/p/gcfg #c2d3050044d0
github.com/jmoiron/jsonq #7c27c8eb9f6831555a4209f6a7d579159e766a3c

The same dependencies and SHA hashes produce different results when put in different orders. Maybe I've missed something about how goop works, but it doesn't seem to make sense to me that the order of the dependencies in Goopfile should matter.

Thanks!

It seems like having support for privately-hosted git and hg servers is not a bad idea. Many organizations rely on having git hosted internally, for any number of reasons, and yet still have same problems that Goop could potentially solve for them.

I run into this with my privately-run Git instance:

=> Fetching [email protected]/sz/sandboxy.git...
goop: unrecognized import path "[email protected]/sz/sandboxy.git"

I use /.vendor/bin, someone might wish something else.

BTW This is the same "way" npm&&nodejs behaves

By default, Goop use go get -u [REPO], however go get -u cant fetch private dependencies.

The workaround right now is to use .netrc file to specify credentials for the remote repository (i.e. github), but it would be nice to be able to specify a git src and use it.

This would be nice to build an initial Goopfile that would collect all the deps and ready to be edited by hand

Like the virtualenv in Python, we can add the new CMD goop active and goop deactive to setup the environment.

And we can update the PS1 to indicate the current environment usage.