loov / goda Goto Github PK

None of the examples in the README seems to do anything interesting.
An example:

$ goda graph "github.com/loov/goda/..."
digraph G {
    node [penwidth=2 fontsize=10 shape=rectangle target="_graphviz"];
    edge [tailport=e penwidth=2];
    compound=true;
    rankdir=LR;
    newrank=true;
    ranksep="1.5";
    quantum="0.5";
}

Currently the code loads all the std library packages, which takes a bunch of time. It's only used to remove std library packages from the output.

https://github.com/loov/goda/blob/v0.5.3/internal/pkgset/std.go#L13

Instead of loading the std packages separately, write a func that detects based on packages.Package whether it's a std library package or not.

I have very often wanted to run a command to see why a dependency is required. This could be similar to go mod why, although that command is lacking a bit of functionality.

Essentially, given an import tree of

.
 ├  a
   ├ b
   ├ c
 ├ d
   ├  c 
 ├  c 

I would like a command like: goda why . c I would expect:

$ goda why . c
. -> a -> c
. -> d -> c
. -> c

From this, I can see all of the places I need to clean up usages of c if I want to drop the dependency.

I have done this exact thing before using a bunch of ad-hoc goda commands, resulting in out binary sizes being cut in half, so I think it would be pretty useful. It feels like all of the information is there, but not presentable in this way. Its also possible I just haven't found the right expressions to represent this.

Alternatives:

• Modify graph/tree in some way to allow filtering out packages not in the path of c. For example, the above tree would be filtered to:

.
 ├  a
   ├ c
 ├ d
   ├  c 
 ├  c 

Here the tree is pretty much the same, but in our real repo the tree is 12k lines so its not usable for this use case. The same with the graph - its too big to read without filtering

Finalize expression graph syntax such that is nice to use.

Expressions must be type-able without quotes on Windows and Unix shells; scripts.

These for example aren't allowed:

# ^ has special meaning in Windows
goda list github.com/loov/goda ^ github.com/loov/...
# & has special meaning in bash
goda list github.com/loov/goda & github.com/loov/...
# * has special meaning in bash
goda list github.com/loov/goda * github.com/loov/...

Using invalid paths probably makes the most sense since they cannot have special meaning in shells.

• https://docstore.mik.ua/orelly/unix/upt/ch08_19.htm, https://www.tldp.org/LDP/abs/html/special-chars.html
• https://ss64.com/nt/syntax-esc.html, https://ss64.com/nt/syntax-redirection.html

Currently all the tools work on packages by default. But it would be good to have a mode for printing only module information rather than the full package information.

First, I would like to say, that this is a very useful project/tool, and I have been exactly for 👍

goda graph github.com/loov/goda/...:root | dot -Tsvg -o graph.svg

Regarding the issue. I'm visualizing internal package dependencies of my wip/hobby project, and there are many levels. In combination with "longer" package names, the graph is quite very wide, aspect ratio of ~ 7:1.

Would it be possible, to add flag --direction vertical to set graph's direction to be vertical? Or, maybe --direction top-bottom or --direction to-bottom indicating also, that it goes from top to bottom, not just any "vertical".

It would be useful to see which packages import "C".

goda graph "reach(., C:root)"

Transitive edges sometimes aren't useful.

This is more of a question that an issue. Not sure why I am seeing circles in the some packages.

Not able to upload an image here but here is some ascii art with an emoticon.

cmd ---> ⭕️ --> router

I'm not sure what other details I can provide that would be helpful since the repo is private but happy to provide whatever additional details I can.

$ goda graph -cluster -short "github.com/me/app:all" | dot -Tsvg -o graph.svg

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x1408ce1]

goroutine 1 [running]:
github.com/loov/goda/internal/pkgtree.(*Tree).NodeRepo(...)
	/Users/me/go/pkg/mod/github.com/loov/[email protected]/internal/pkgtree/tree.go:75
github.com/loov/goda/internal/pkgtree.From(0xc000076000, 0xb061, 0x5c0, 0xee)
	/Users/me/go/pkg/mod/github.com/loov/[email protected]/internal/pkgtree/tree.go:26 +0x121
github.com/loov/goda/internal/graph.(*Dot).WriteClusters(0xc000154980, 0xc000076000, 0x0, 0x0)
	/Users/me/go/pkg/mod/github.com/loov/[email protected]/internal/graph/dot.go:131 +0x76
github.com/loov/goda/internal/graph.(*Dot).Write(0xc000154980, 0xc000076000, 0xc0008d2270, 0x1)
	/Users/me/go/pkg/mod/github.com/loov/[email protected]/internal/graph/dot.go:106 +0x3b
github.com/loov/goda/internal/graph.(*Command).Execute(0xc000154640, 0x157f370, 0xc00011a000, 0xc00012e660, 0x0, 0x0, 0x0, 0x0)
	/Users/me/go/pkg/mod/github.com/loov/[email protected]/internal/graph/cmd.go:136 +0x339
github.com/google/subcommands.(*Commander).Execute(0xc000130200, 0x157f370, 0xc00011a000, 0x0, 0x0, 0x0, 0xffffffff)
	/Users/me/go/pkg/mod/github.com/google/[email protected]/subcommands.go:209 +0x347
main.main()
	/Users/me/go/pkg/mod/github.com/loov/[email protected]/main.go:38 +0x39d

Ensure we don't blind people with the output.

It would be useful to see which packages import "unsafe".

goda graph "reach(., unsafe:root)"

Currently you can use "-std" however that will display too much data.

Add format that can be used with https://github.com/golang/tools/blob/master/cmd/digraph/digraph.go.

goda is a great package -- thanks!
Is it possible to shift to html output (like pprof -http=:8080 profile.out)?

Flags for goda and go list differ for no good reason, try to improve it.

Currently the syntax for Package and Modules differ.

Currently to include test packages, you can use test=1(something) but that's awkward to type, not that easy to remember and requires shell quotes.

I find myself needing this qualifier a lot, because test code is a large part of any repository and a big part of refactoring calculations.

How about using a :test qualifier instead?

So, for example, ./...:test would include all the test code in ./....

./...:test:all would include all dependencies of all packages and tests below ..

With the advent of Go modules one can no longer ignore the side-effects of including files targeting a GOOS or a GOOS/GOARCH pair one does not need: their deps will end up in go module deps, and go will pull all the other modules those deps require, recursively.

So when you integrate software for Linux, for example, you now have to deal with a lot of windows or freebsd or whatever- support modules you have no actual need of, just because they are declared in module deps.

Therefore, we'd like to trim Go modules to only the GOOS (or alternatively GOOS+GOARCHES couples) we integrate for, to avoid this problem. Unfortunately go list and friends are especially useless, because they only know to list

• "all source files" or
• "all source files for this GOOS and Go build tags" + "other source files"

Which is not the correct splitting point for us. Go build tags can include other things that Go arches, and what we really need is "all source files for this GOOS, including all arches and all build tags" or "all source files for this GOOS and GOARCH, including all the combinations of build tags which are not GOARCH bits"

So if goda could provide this kind of listing, that would be useful to us.

When I run goda cut, I see a lot of test package results like this:

github.com/influxdata/idpe/client/v2 [github.com/influxdata/idpe/client/v2.test]                                                                            in:0     pkgs:1   size:45.6KB    loc:1499
github.com/influxdata/idpe/client/v2.test                                                                                                                   in:0     pkgs:1   size:2.9KB     loc:58
github.com/influxdata/idpe/client/v2_test [github.com/influxdata/idpe/client/v2.test]                                                                       in:0     pkgs:1   size:6.0KB     loc:234
github.com/influxdata/idpe/client/v2                                                                                                                        in:24    pkgs:1   size:21.3KB    loc:706

There's actually only one package there, but it's appearing multiple times in the results (because of the external test shenanigans).

It's probably not helpful to include these entries in the cut results because it's unlikely we're going to consider removing tests.

In fact, these results are kinda confusing all round unless you're familiar with the way that test packages are dealt with. I know these package need to be a part of the package sets internally, but I wonder if there might be a way to hide them from user view, at least most of the time?

Add syntax for being able to draw the graph based on direct dependencies.

So given github.com/loov/goda -> golang.org/x/tools/... it would show all packages in github.com/loov/goda that directly or indirectly depend on golang.org/x/tools.

Figure out whether there's a better symbol than ->.

Hi!
How to exclude several repos from graph drawing?
Syntax like - github.com... - google.golang.org... - gopkg.in... doesn't seem to work.
Thanks!

Add FolderSize func that shows the directory size rather than the amount of code used for a particular package.

It would be nice to specify things like:

graph list github.com/loov/goda -> ( golang.org/x/tools/... $ )

Ensure we can also print information about test packages.

Also ensure we have the capability to mix these two. For example:

go list TEST=1 ./... - TEST=0 ./...

Would be nice capability to find out what packages tests depend on.

Hi,

I don't know if it's a RFE or if the functionality already exists in goda, without good documentation

Dependency cycles are the curse of software integration. When A depends on B and B depends directly or indirectly on A, it's no longer possible to define a clean modular integration plan where each component is tested after the ones it depends on, you have to import the whole cycle in bulk, which makes in impossible to split tasks between different teams, and scale your organization.

Therefore, as an integrator, I would be interested in a command, that tells me if it is possible to split the code tree of A, in

• something that depends on B, and
• something that depends neither on B nor on the first part.

With A being a tree of Go packages (typically, nowadays, the packages included in a Go module).

If it possible to do this with Goda? I already have the code to walk a source code directory and identify all the module trees within (I can share it if you want, it's GPLv3, but relicensing it is no problem).

I really enjoy goda! It's been very helpful on debugging. Now I want to use it as part of our CI to build and test the application when the relevant packages have changed.

Our CI is air gapped and all the packages vendored, but when I call goda list, I see it's trying to fetch the packages from the internet.

Is there a way to enable vendored package in goda?

Feature Request
We run our own internal GoDoc. Being able to point at that instead of pkg.go.dev would make this package usable in our GoDoc.

What's currently happening
Clicking a node yields documentation for pkg.go.dev which only serves public packages

What you want to happen
Be able to override the pkg.go.dev URL.

GraphML can be used to import the data into yEd and similar tools, which allow better editing.

• https://github.com/loov/layout/tree/master/format/graphml
• https://github.com/storj/archview/blob/master/graph/graphml.go

Currently it seems build tags are not respected. I would like files like // +build !bloat to be ignored, since they will not end up in my final binaries anyways. Specifically with graph, but likely others apply as well

During output it would be nice to collapse some particular nodes into a single node.

For example, these could be collapsed:

1. modules
2. */internal/*
3. golang.org/x/tools/*
4. everything except the project module (or multiple modules)

This sounds like pkggraph needs collapsing and uncollapsing capabilities so that the code can be written as:

goda graph -m .
goda graph -collapse "*/internal/*" .
goda graph -collapse "golang.org/x/tools/*" .

# combining all together
goda graph -m -uncollapse github.com/loov/goda -collapse "*/internal/*" .

From the list of subcommands, nm is the one which could have a more intuitive name. Depending on future developments I would recommend binary or memory as a replacement.

There is a function sat8.
The good part is that the comment does explain the content of what it does. The improvable part is its naming. I propose it should either be renamed into something like scaleTo256 or scaleTo255, or the documentation should expand with the explanation of what does the sat8 stand for.

https://github.com/loov/goda/blob/master/graph/color.go#L54

Currently, when "exporting" a graph to SVG, the links in the nodes point to godoc.org. This should be changed to point to pkg.go.dev.

It would be nice to see differences between used packages based on platform. For example:

go list GOOS=window ./... - GOOS=linux ./...

Could be used to see which packages Windows build uses, but Linux build doesn't.

From:

1. https://www.reddit.com/r/golang/comments/as76tb/go_mod_download_cache_is_huge/
2. https://github.com/disintegratedcircuit/gomodtest

go.mod:

module gomodtest

require (
	github.com/opentracing/opentracing-go v1.0.2 // indirect
	github.com/pkg/errors v0.8.1 // indirect
	github.com/prometheus/client_golang v0.9.2 // indirect
	github.com/prometheus/common v0.2.0 // indirect
	github.com/prometheus/prometheus v2.7.0+incompatible
	gopkg.in/yaml.v2 v2.2.2 // indirect
)

I am attempting to use goda to produce the same form of output as go mod graph, for use in an application that parses the latter. goda graph -type edges comes close, save for the lack of version suffixes. There does not appear to be a format string argument that will provide this.

It would be helpful for version information to at least be available (as a label-formatting directive) when generating dependency output.

It would be nice to determine what causes significant size differences in binaries.

What?

goda help message says: "Use "goda flags" for a list of top-level flags", but typing that does not show any flag info.

Replicate

go build
goda flags

Expected

Some flag info

Got

Same help as when just typing goda

Main packages are by definition roots, but we usually don't want to remove them;
currently there's no way to make an expression that finds all sources that are not
main packages.

How about a :main qualifier that selects only main packages. Then I can do (for example):

goda list ./...:source - ./...:main

to find all non-main source packages.

From:

1. https://www.reddit.com/r/golang/comments/as76tb/go_mod_download_cache_is_huge/
2. https://github.com/disintegratedcircuit/gomodtest

go.mod:

module gomodtest

require (
	github.com/opentracing/opentracing-go v1.0.2 // indirect
	github.com/pkg/errors v0.8.1 // indirect
	github.com/prometheus/client_golang v0.9.2 // indirect
	github.com/prometheus/common v0.2.0 // indirect
	github.com/prometheus/prometheus v2.7.0+incompatible
	gopkg.in/yaml.v2 v2.2.2 // indirect
)

goda list "shared(./...:root, ./pkg/encryption + ./pkg/identity + ./pkg/macaroon + ./pkg/paths + ./pkg/pb (without private endpoints) + ./pkg/peertls + ./pkg/peertls/extensions + ./pkg/peertls/tlsopts + ./pkg/pkcrypto + ./pkg/ranger + ./pkg/rpc + ./pkg/rpc/rpcpeer + ./pkg/rpc/rpcpool + ./pkg/rpc/rpcstatus + ./pkg/signing + ./pkg/storj + ./private/bloomfilter + ./private/errs2 + ./private/fpath + ./private/groupcancel + ./private/memory + ./private/readcloser + ./private/sync2 + ./private/testcontext + ./private/testidentity + ./private/testpeertls + ./private/testrand + ./private/useragent)"

Gives:

panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x67994f]

goroutine 1 [running]:
github.com/loov/goda/pkgset.Calc.func2(0xc00010e280, 0x78e180, 0xc0000c14d0, 0xc000537560, 0x0, 0x0)
        f:/Code/github.com/loov/goda/pkgset/calc.go:103 +0xc4f
github.com/loov/goda/pkgset.Calc.func1(0xc00010e280, 0xc00009a860, 0x2, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0)
        f:/Code/github.com/loov/goda/pkgset/calc.go:42 +0xf3
github.com/loov/goda/pkgset.Calc.func2(0xc00010e280, 0x78e180, 0xc0000c1500, 0xc00007fbc0, 0x0, 0x0)
        f:/Code/github.com/loov/goda/pkgset/calc.go:84 +0xb53
github.com/loov/goda/pkgset.Calc.func1(0xc00010e280, 0xc00009a880, 0x2, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0)
        f:/Code/github.com/loov/goda/pkgset/calc.go:42 +0xf3
github.com/loov/goda/pkgset.Calc.func2(0xc00010e280, 0x78e180, 0xc0000c1530, 0xc00007fbb0, 0x0, 0x0)
...