anticycle / anticycle

Track down import cycles in your GO project.

Anticycle

Anticycle is a tool for static code analysis which search for dependency cycles. It scans recursively all source files and parses theirs dependencies. Anticycle does not compile the code, so it is ideal for searching for complex, difficult to debug cycles.

Anticycle is published under GNU GENERAL PUBLIC LICENSE Version 3, from 29 June 2007

Installation

You may need sudo privileges to unpack into system specific directories.

1. Download binary from latest release.
2. Unpack binary to your PATH for example: tar xvf anticycle.linux-amd64.tar.gz -C /usr/local/bin
3. Verify installation: anticycle -version

Usage

anticycle [options] [directory]

Options

-all                 Output all packages, with and without cycles.

-format="text"       Output format. Available: text, json.

-exclude=""          A space-separated list of directories that should 
                     not be scanned. The list will be added to the 
                     default list of directories.
-excludeDefault=""   A space-separated list of directories that should 
                     not be scanned. The list will override the default.
-showExcluded        Shows list of excluded directories.

-help                Shows this help text.
-version             Shows version and build hash.

Directory

An optional path to the analyzed project. If the directory is not defined, the current working directory will be used.

Example

Analyze recursively from current working directory but skip internal/ anywhere in dir tree.

$ anticycle -exclude="internal"

Analyze recursively given directory with JSON output format

$ anticycle -all -format=json $GOPATH/src/github.com/anticycle/anticycle

Example output

Real case scenario:

Command without flags and directories.

$ cd $GOPATH/src/github.com/Juniper/contrail
$ anticycle
Found 4 cycles

db -> models -> db
ipam -> models -> db -> models
models -> db -> models
services -> models -> db -> models

Details

[db -> models] "github.com/Juniper/contrail/pkg/models"
   pkg/db/address_manager.go
   pkg/db/address_manager_test.go
   pkg/db/db.go
   pkg/db/db_test.go
   pkg/db/useragent_kv.go

[models -> db] "github.com/Juniper/contrail/pkg/db"
   pkg/models/validation.go

[services -> models] "github.com/Juniper/contrail/pkg/models"
   pkg/services/contrail_service_test.go
   pkg/services/event_test.go
   pkg/services/list_response_test.go

[ipam -> models] "github.com/Juniper/contrail/pkg/models"
   pkg/types/ipam/address_manager.go

How to read:

[package -> wants] "fully/qualified/import/name"
   path/to/affected/file.go
   path/to/another/file.go

This gives us a hint that few files in db package wants to import models, but validation.go file in models want to import db package.

The cycle looks like: db -> models -> db.

Development

Require GO v1.11.x

Make sure you have GO in version 1.11. If not, follow official instructions.

Makefile

Use Make to run tests, benchmarks, build process and more.

sudo apt install make
$ make help

Download project

$ go get github.com/anticycle/anticycle
$ make devdeps install test-all

After each change use make install to update dev binary. Then run sanity tests. Sanity tests are called on built binary, not the source code.

Run tests

Make sure you had installed anticycle in your OS before run sanity or acceptance tests.

$ make test-unit test-sanity test-acceptance

or just

$ make test

Build artifacts

Artifacts will be moved to /bin directory after building.

$ make build tarball

Contribution

Your help is very desirable. We are open to all kinds of contribution:

• Reporting found bugs
• Requesting new features
• Typos or bug fixing
• Filling gaps in documentation
• Creating new features
• Building tools for other contributors

If you don't know how to GO, you can report a bug or request a new feature by creating new issue at https://github.com/anticycle/anticycle/issues

If you want to be involved in development:

1. Fork this repository.
2. Create new issue with description of what you are plan to do.
3. Create new feature or bug branch in your forked repository.
4. Make changes, and create pull request to our master branch.
5. Attach pull request to issue and wait for community response.

Remember to

• always follow conventions found in this repository
• be kind to other contributors and never attack them for any reason
• if you are reviewing pull requests, focus on source code, never on developer who wrote it
• always write a unittests
• if it is required create new sanity or end-to-end test

anticycle's Issues

anticycle process got killed

Describe the bug
hardware & software: macbook pro, macos mojave-10.14.5
process always got killed even though I just run "anticycle -version"

To Reproduce
Steps to reproduce the behavior:

1. download anticycle-v1.0.0.darwin-amd64.tar.gz
2. run "tar xvf anticycle-v1.0.0.darwin-amd64.tar.gz -C /usr/local/bin"
3. run "anticycle -version"

process got killed

Expected behavior
should show version infomation

Environment (please complete the following information):
GOARCH="amd64"
GOBIN="/Users/yuanxie/work/go_proj/bin"
GOCACHE="/Users/yuanxie/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/yuanxie/work/go_proj"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/yuanxie/work/forder-server/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/kp/ymr5srdj4_s44jdjbmqqvxhh0000gn/T/go-build066343451=/tmp/go-build -gno-record-gcc-switches -fno-common"

Version v1.0.0, binary file

CI release job failed

Describe the bug
https://circleci.com/gh/anticycle/anticycle/204

#!/bin/bash -eo pipefail
./build/tarball.sh /tmp/artifacts
/bin/bash: ./build/tarball.sh: No such file or directory
Exited with code 127

Remove dep gopkg files

Since Anticycle started to use go.mod it is not necessary to keep Gopkg.lock and Gopkg.toml files. Support for dep should be dropped.

Add 'go vet' to CI workflow

https://golang.org/cmd/vet/

Sanity tests for CLI interface

We need few sanity tests with most basic scenarios:

• Analyze single file with no imports
• Analyze single file with multiple imports
• Analyze broken file

False positive match when external package has the same name as internal one

There can be false positive match if external library have the same package name as project package.
For example:

Internal package types and external package types

pkg/types
github.com/gogo/protobuf/types

Files:

pkg/common/common.go imports protobuf/types
pkg/types/types.go imports pkg/common

This will generate false positive because package short name overlaps with external library.

Add imports to public package in doc.go

Godoc didn't fetch all docs. We need to add imports to doc.go so parser could fetch rest of the files.
In doc.go file few lines are rendered as source code. Need fix.

Use gotestsum in CircleCI

CircleCI supports insight statistics from tests.
Instructions https://circleci.com/blog/level-up-go-test-with-gotestsum

Find cycles in package imports

Update docs

README.md is out of sync with sourcecode. Need update.

Map with all possible cycles

Is your feature request related to a problem? Please describe.
Proposal to add cycles map for every possible imports chain which leads to cycle.
In text format it could be printed at the beginning of the report, and in JSON format as separate metadata key.

Describe the solution you'd like
For JSON output:

{
  "metadata": {
    "cycles": [
      ["foo", "bar", "baz", "bar"],
      ["bar", "baz", "bar"],
      ["baz", "bar", "baz"]
    ]
  },
  "cycles": ["current report output here"]
}

In this structure, metadata will hold all extra information, and cycles will be output which is the same as current output.

For text output:

Found 3 cycles:
foo -> bar -> baz -> bar
bar -> baz -> bar
baz -> bar -> baz

[current report output here]

Add 'go clean' to uninstall target in Makefile

Go clean removes object files and cached files.

Make doc.go files

Like in https://golang.org/src/encoding/gob/doc.go

Line number of affected import

Is your feature request related to a problem? Please describe.
List of imports could be very long in some cases. Output should suggest line number of affected import.

Describe the solution you'd like
In text format:

[db -> models] "affected/import/package"
   /path/to/file.go:12

In JSON format (only cycle fragmet):

{
  "cycles": [
    {
      "affectedImport": {
        "name": "testdata/notAffectedFiles/baz",
        "nameShort": "baz",
        "line": 12
      },
      "affectedFile": "testdata/notAffectedFiles/bar/bar.go"
    }
  ]
}

Compress binaries in CI workflow and upload to github release

Is your feature request related to a problem? Please describe.
CI should be responsible for preparing compressed binaries and upload them to github release.

Run all tests with -race flag

Is your feature request related to a problem? Please describe.
All tests in CI and run locally should be call with -race flag even if there is no concurrency right now.
It may change in future and usage of gorutines may cause undetected bugs.

Describe the solution you'd like
Add -race flag to all tests calls in makefile

Additional context
It is easy to forget about things you should remember of.

Simple CI for unit tests

We need simple pipeline which runs unit tests only: Travis / CircleCI.
Investigation needed which open source tool suites our needs the most.

Add 'golint' to CI workflow

https://github.com/golang/lint

Create .deb package in CI

Is your feature request related to a problem? Please describe.
Create .deb package as part of CI release pipeline.

Standardize CLI output

Right now CLI always returns data but the goal of the tool is to return info about cycles, not all dependency graph. Output should be readable in form of a text report. New flag to specify different output format should be added like -format=json.

Changes:

• By default return only cycles or nothing if no cycles have been found
• Add flag to allow return all dependency graph
• Add flag to output data in JSON format
• Add -version flag
• Flags: -exclude and -excludeOnly should be change to take space-separated directories

Text representation of cycles

Is your feature request related to a problem? Please describe.
With big cycles and multiple files current text representation of cycles is not readable.

Describe the solution you'd like
General Map of package cyckle like foo -> bar -> baz -> foo
And then table with specific files, and imports which participate in cycle.

Additional context
Attached file.
cyckle-report.txt

Better documented Makefile

Makefile is used only for development purposes. Make a better Makefile.

Inspiration: https://suva.sh/posts/well-documented-makefiles/

Make artifacts without 3rd party tools

Right now we are using gox to create binaries for different platforms.
We can do it without any 3rd-party tools. Cross compilation is built in Golang.

Relative paths comparison error

Describe the bug
In test we are using absolute paths everywhere but in real world anticycle will scan relative paths as well.
Right now path mismatch results as empty cycles list for packages witch in fact have cycles.

Add support for go modules

Is your feature request related to a problem? Please describe.
Anticycle can be used as 3rd-party dependency. We should add mod.go feature.

Remove duplicates in map cycles

Is your feature request related to a problem? Please describe.
Remove duplicates from cycles map.
For example:

db -> models -> db is the same as models -> db -> models

Describe the solution you'd like

1. Add first cycle to map
2. If next cycle has at least 2 packages the same as cycles in map skip
3. Repeat step 2 for rest of the cycles

Additional context

Upgrade to GO v.1.11

There are changes between v1.10 and v.1.11 which reflects in golden files.
All goldens and Circle CI container must be updated .

Run tests with coverage report

Is your feature request related to a problem? Please describe.
Right now test coverage is used separately and not in CI. It would be nice to have coverage print in test results like:
ok fmt 0.056s coverage: 91.7% of statements

Describe the solution you'd like

test: ## run tests
	go test -race -covermode=count -coverprofile=none ./pkg/...
	go test -race -covermode=count -coverprofile=none ./internal/...

Anticycle prints files which don't participate in cycle

Describe the bug
Anticycle prints all files in package even if the don't participate in cycle. This creates clutter and is hard to read.

To Reproduce
Steps to reproduce the behavior:

1. New test case is required

Expected behavior
Anticycle should output only files affected with cycle not all files which belongs to affected package.
Letter "C" in front of import name is not readable enough with big output.

Resolve linter errors

Add 'gofmt' to CI workflow and check differences

https://golang.org/cmd/gofmt/

Makefile:8: recipe for target 'install' failed

Bug in Make install recipe:

go install cmd/anticycle
can't load package: package cmd/anticycle: cannot find package "." in:
	/usr/lib/go-1.10/src/cmd/anticycle

Create .rpm package in CI

Is your feature request related to a problem? Please describe.
Create .rpm package as part of CI release pipeline.

Shrink size of binaries

Remove debugging features from release binaries and compress with UPX

Create issue templates

make deps is confusing

Is your feature request related to a problem? Please describe.
make deps indicate that it is all about production dependencies, but in fact it only installs development dependencies. It may be confused.

Describe the solution you'd like
Change make deps to make devdeps

Anticycle show packages which lead to but are not involved in cycle

Describe the bug
When some package imports affected package automatically is mark as affected.
This is 100% correct but not helping at all.

Expected behavior
I'd like to see only these packages which are strictly involved in cycle.
All other packages which are incidentally affected should be removed from report.

Additional context
This log clearly show that cycle exists between db -> models -> db packages.

But from ipam you can get to models from which you can get to db and here cycle begins.

Packages: ipam and services should be removed from report because they create unnecessary information noise.

[db -> models] "github.com/Juniper/contrail/pkg/models"
   pkg/db/address_manager.go
   pkg/db/address_manager_test.go
   pkg/db/db.go
   pkg/db/db_test.go
   pkg/db/useragent_kv.go

[models -> db] "github.com/Juniper/contrail/pkg/db"
   pkg/models/validation.go

[services -> models] "github.com/Juniper/contrail/pkg/models"
   pkg/services/contrail_service_test.go
   pkg/services/event_test.go
   pkg/services/list_response_test.go

[ipam -> models] "github.com/Juniper/contrail/pkg/models"
   pkg/types/ipam/address_manager.go

Acceptance tests which fully cover CLI

Is your feature request related to a problem? Please describe.
We need to write acceptance tests which cover all features in CLI with all possible combinations.

Describe the solution you'd like
It may be done in a way the sanity-tests are created.

Additional context
Unit tests don't cover using built and compressed binary file.