kowainik / smuggler

🚣 Smuggle all imports

smuggler

“So many people consider their work a daily punishment. Whereas I love my work as a translator. Translation is a journey over a sea from one shore to the other. Sometimes I think of myself as a smuggler: I cross the frontier of language with my booty of words, ideas, images, and metaphors.”

― Amara Lakhous, Clash of Civilizations Over an Elevator in Piazza Vittorio

Haskell Source Plugin which removes unused imports automatically.

How to use

Add smuggler to the dependencies of your project. Then add the following compiler options:

-fplugin=Smuggler.Plugin

Lovely addition to this package is that it automatically supports on-the-fly feature if you use it with ghcid. Smuggler doesn't perform file changes when there are no unused imports. So you can just run ghcid as usual:

ghcid --command='cabal repl'

For contributors

Requirements:

• ghc-8.6.5
• cabal >= 3.0 or stack >= 2.0

Cabal: How to build?

cabal update
cabal build

Stack: How to build?

stack build

Run tests

cabal test --enable-tests

smuggler's Issues

Pass path to file as CLI argument

This is convenient for testing. In case we want to test on different files without recompilation.

Add tests for implicit imports

I would like to have two tests:

1. When the implicit import is used and not deleted by smuggler
2. When the implicit import is not used and got deleted by smuggler

Write usage example and some documentaiton

• Improve README.md
• Add .cabal description

Upgrade to 8.6.5

Some infelicities

This is just the plugin I was looking for, and I have been giving it a whirl.

I used ghc 8.8.3 (by adjusting the cabal bounds).

-  build-depends:       base ^>= 4.12
+  build-depends:       base ^>= 4.13
   ghc-options:         -Wall
                        -Wincomplete-uni-patterns
                        -Wincomplete-record-updates
@@ -62,8 +62,8 @@ library
   build-depends:       bytestring ^>= 0.10
                      , containers >= 0.5 && < 0.7
                      , filepath ^>= 1.4
-                     , ghc ^>= 8.6.0
-                     , ghc-exactprint ^>= 0.6
+                     , ghc ^>= 8.8.3
+                     , ghc-exactprint ^>= 0.6.2
                      , unordered-containers ^>= 0.2.7

This doesn't compile, so I also have to

diff --git a/src/Smuggler/Loc.hs b/src/Smuggler/Loc.hs
index 6580578..b2ae1f2 100644
--- a/src/Smuggler/Loc.hs
+++ b/src/Smuggler/Loc.hs
@@ -4,8 +4,7 @@ module Smuggler.Loc
        , unL
        ) where
 
-import SrcLoc (GenLocated (L), Located, SrcSpan (RealSrcSpan), srcSpanStartCol, srcSpanStartLine,
-               unLoc)
+import SrcLoc (GenLocated (L), Located, SrcSpan (RealSrcSpan), srcSpanStartCol, srcSpanStartLine)
 
 -- | Returns location in the way of @line:col@.
 showLoc :: SrcSpan -> String
@@ -18,4 +17,4 @@ showL name showA (L l nm) = "(" ++ showLoc l ++ name ++ showA nm ++ ")"
 
 -- | Shorter synonim for 'unLoc'.
 unL :: GenLocated l e -> e
-unL = unLoc
+unL (L _ e) = e

Perhaps the answer is to use ghc-lib.

The tests now don't pass

Linking /Users/xxx/Documents/Haskell/smuggler/dist-newstyle/build/x86_64-osx/ghc-8.8.3/smuggler-0.2.0.0/t/smuggler-test/build/smuggler-test/smuggler-test ...
Running 1 test suites...
Test suite smuggler-test: RUNNING...

Test test/Test/BoolPartlyUsedLast failed:
Expected:
- import Data.Bool ( not)
But got:


Test test/Test/TypeConstructorUsed failed:
Expected:
- import Data.Bool (Bool (True))
But got:


Test test/Test/TypeUsed failed:
Expected:
- import Data.Bool (Bool)
But got:


Test test/Test/WildcardUsed failed:
Expected:
- import Data.Bool (Bool (..))
But got:


Test test/Test/TypeConstructorUnused failed:
Expected:
- import Data.Bool (Bool ( True))
But got:


Test test/Test/TypeConstructorUnusedComma failed:
Expected:
- import Data.Bool (Bool (False))
But got:


Test test/Test/WildcardUnused failed:
Expected:
- import Data.Bool ( not)
But got:


Test test/Test/ConstructorsUnused failed:
Expected:
- import Data.Bool ( not)
But got:


Test test/Test/TypeUnused failed:
Expected:
- import Data.Bool ( not)
But got:

Cleaning tests
Some tests failed

Test suite smuggler-test: FAIL

I tried running the plugin on a large project with a lot of redundant imports, https://github.com/agda/agda. There were several issues:

• because smuggler reads files using ByteString it mangles Unicode characters in the code. Eg:

--- | Get the next version of the concrete name. For instance, @nextName "x" = "x₁"@.
+-- | Get the next version of the concrete name. For instance, @nextName "x" = "xâ<U+0082><U+0081>"@.

• a good number of redundant imports get zapped, but there is a fair bit of mangling (where two adjacent imports get joined into a single line, with the second import keyword missing). I am not sure how close the result is to the minimal imports that GHC sees with -ddump-minimal-imports
• some imports are redundant, but needed for backward compatibility (eg, around Monad.Fail and Data.Monoid and Data.Semigroup.

To make this app even more useful, it would be helpful to have it generate a minimal exports list (based on what is used elsewhere in the project)!

Thanks for all the hard work that has clearly gone into all this.

Remove multi-line imports

If whole multi-line import is unused, we delete only first line. We should do this deletion for every line:

smuggler/src/Smuggler/Plugin.hs

Line 85 in 4fbaafe

| null used = map (srcSpanStartLine loc,) [1..100]

Collect mapping from import list entries to their location

It's needed to use remoceAnnAtLoc function. hsModImports from this type should be used:

• https://hackage.haskell.org/package/ghc-8.4.1/docs/HsSyn.html#t:HsModule

Think on how to make testing automatic

Currently I just test on Main.hs file and I don't like it. But plugin in running during compilation. So we need some way to test it automatically...

Fix stack.yaml

Provided stack.yaml yields:

λ ~/code/smuggler/ master STACK_YAML=stack-mac-8.6.1.yaml stack build
Getting project config file from STACK_YAML environment
Error parsing targets: The project contains no local packages (packages not marked with 'extra-dep')

Compress imports

Sometimes we have imports like these:

import Foo (Foo)
import Foo.Bar (Bar)

And often they can be compressed to something like this:

import Foo (Bar, Foo)

I think it would be good to implement this feature.

Fix warnings in app/Main

There are some warnings about unused imports in app/Main that need fixing.

Learn how to build 'TcGblEnv'

This environment:

• https://hackage.haskell.org/package/ghc-8.4.1/docs/TcRnTypes.html#t:TcGblEnv

Probably with the help of this function:

• https://hackage.haskell.org/package/ghc-8.4.1/docs/TcRnMonad.html#v:initTc

Reuse more logic from GHC regarding detecting of unused imports

See here:

• #67 (comment)

Improve tests output messages

Currently, we have

smuggler/test/Test.hs

Lines 24 to 26 in 7746b69

	if result
	then putStrLn "Success"
	else putStrLn "Fail" >> exitFailure

which is not informative at all. We need somehow show which tests are failing, what was the expected and real result of the smuggler work.

cabal.project should not hard code location of ghc

For example, on NixOS, this is not where GHC is located and the line has to be removed before the project is built.

Get 'HscEnv' somehow for 'Env'

This is Env:

• https://hackage.haskell.org/package/ghc-8.2.2/docs/TcRnTypes.html#t:Env

Learn how to run Source Plugins

We should have some simple dummy example of running Source Plugins over files inside our project.

Here is the example:

• https://github.com/ghc/ghc/blob/master/docs/users_guide/extending_ghc.rst#source-plugins

This could be useful to actually building with newer GHC since source plugins are avalaible only in GHC-8.6:

• http://mpickering.github.io/posts/2018-06-24-haskell-nix-plugins.html

Include dependencies in the hash

Let's say I have three modules: A, B, and C:

module A (X) where
data X

module B (Y) where
data Y

module C where
import A
import B
data Z = MkZ X Y

Then I change B:

module B (X, Y) where
import A (X)
data Y

Now import A in C is redundant, because B reexports X. However, C itself hasn't changed.

This means that using hash-store to decide whether recompilation is needed is not sufficient, because redundancy of imports is not a property of a particular module, but a property of this module and all its dependencies.

Check whether it works with `ghci` and `ghcid`

I wonder how it works with ghci and even ghcid.

Check if cache directory exists

Tests sometimes fail locally with

smuggler-test: .smuggler: removeDirectoryRecursive:getSymbolicLinkStatus: does not exist (No such file or directory)

We should run removeDirectoryRecursive only if .smuggler exists?

Launch 'rnImports'

Support GHC-8.8.1

Print minimal imports

Basically conversion from implicit imports to explicit imports.

Requires contribution to GHC:

• https://ghc.haskell.org/trac/ghc/ticket/15439

Investigate how to remove symbol from 'Anns'

Use debugAST function to see how it works.

Check GHC for existing solutions

Maybe we can reuse some existing algos.

Check whether it's possible to delete data using source plugins

Don't do anything if file fails to compile

This requires some code refactoring but it's a good addition.

Add Travis CI build

Optimize whole import deletion

Currently we just assume and length of line is 100 and we try to delete entry on every position. But this can be optimized to delete only from starting positions of entries:

smuggler/src/Smuggler/Plugin.hs

Line 85 in 4fbaafe

| null used = map (srcSpanStartLine loc,) [1..100]

Get all 'Name's inside module

Just print them to console for now. We need pairs of symbol name and module where it came from.

Learn to remove commas in data type list

Use ByteString instead of FilePath in Parser

This is related to the to-do found here https://github.com/kowainik/smuggler/blob/master/src/Smuggler/Plugin.hs#L62

Stack extra-deps

I am not sure if this should be merged in or provided by each user individually but I had to add some extra-deps to my stack.yaml in order to build:

allow-newer: true
extra-deps:
 - ghc-exactprint-0.5.8.1@sha256:f76eed0976b854ce03928796e9cff97769e304618ca99bc0f6cdccab31e539d0
 - hash-store-0.1.0@sha256:aa7a3e9c1e4d6eb426a0682b8d2164c3d954b23d34e9ca33c9f04fbc9e5198a3
 - unordered-containers-0.2.9.0@sha256:124a5241860711a0e5c10e94d8aec8a127f0094df7a537cc1786e198d7571dfe
 - base16-bytestring-0.1.1.6@sha256:ead535a30f93189068b3457496e30e3cb8a55d48b2d46f39325233d0d9987d2c
 - blake2-0.2.0@sha256:2175e66e5f4a72b47407cb79b252b0adc5d82f34a958418b75e1d7871187b507
 - free-5.1@sha256:71aa48c5a5dde71ed2a7f3d1d23c6d3a66580859444f55c8b38dd1bb205aa298
 - ghc-paths-0.1.0.9@sha256:ed18df7b53699446dabd66f6e019e5b156820f0301d8dec0d6385f13b797e0ba
 - hashable-1.2.7.0@sha256:03b6836ca9cd3ad0e5a2f3cce989b001dd0e05f306a873db3196037adb30e0a4
 - syb-0.7@sha256:9c559edeef6b9eb1362d7e16fbb15e786be871050cf085a46fcead282bb86e89
 - comonad-5.0.4@sha256:522ca45fedc212d86b3f628275974378e821e7f3caf8055a64189ae4f8c59dbf
 - distributive-0.6@sha256:26507cf231eb10db1eb54bc358477418ce87d5077ff76c66743925fb49494b54
 - exceptions-0.10.0@sha256:ef6066c13de2dfd191dc810f9dfa9a474318abc9bb458413ffbdd2ece482312a
 - profunctors-5.3@sha256:ce641b72fd014b1f7b42a438f8c0360cb93fd277c7b843b7ef94a7c083c31ded
 - semigroupoids-5.3.1@sha256:ed631b231bf49ee972482aa98b3ba1487041328f7168c8773e473b2fa4ebe4a4
 - transformers-base-0.4.5.2@sha256:e4d8155470905ba2942033a1537fc4cf91927d1c9b34693fd57ddf3bc02334af
 - base-orphans-0.8@sha256:4885caa82ab94fa5cb841d6d06a43a97cdc7e835f346be4f11c1cc5707b886db
 - bifunctors-5.5.3@sha256:8f6e90c3ae7a04dba36669e4fad5b9fbc973dbfc6978b4c7d410f9875e0e9d00
 - cabal-doctest-1.0.6@sha256:c0b4a5b1ff38d2867e7003b4be59f3bd7e8e204ab8c988d96d3a77472ae671cd
 - contravariant-1.5@sha256:83f4d20073414561f2352aeb8f48207def31cc933eb058c141968fbbefaeb0d5
 - semigroups-0.18.5@sha256:aaf0d3d5050aa120b2eb98682bc7f8aeed947c1eb51a80375f32775af3d2ff68
 - tagged-0.8.6@sha256:b5e1de3b14ff198b548928f686e5e8783301dc231413f751c23f4ee10cd47a7c
 - transformers-compat-0.6.2@sha256:3569e3a38065c66c99eca44ce67fba2747628d1b87d8c1b6bba2e910d79f0df1
 - StateVar-1.1.1.1@sha256:cd3b516a49faf0627ea31885e012611e63600824976dcb276bcb2e92cdb0790f
 - th-abstraction-0.2.8.0@sha256:db5a2ee91cce9afbe7e251c925c2d2b78725b515f6655eb5079aaeeb935c247b

Stack build osx for 8.6+ failing

I tried to follow the instructions to get things built:
STACK_YAML=stack-mac-8.6.1.yaml stack build

but I get this error:

Getting project config file from STACK_YAML environment
Error parsing targets: The project contains no local packages (packages not marked with 'extra-dep')

Any idea what I might be doing wrong, I've deleted my .stack-work/ but still seems that I'm running on ghc 8.4.3 🤔

Get 'IORef UniqSupply' for 'Env'

Again, this is Env:

• https://hackage.haskell.org/package/ghc-8.2.2/docs/TcRnTypes.html#t:Env

Try to rewrite file source code using `ghc-exactprint`

Search for imports and remove them

Can be done after ghc release with findImportUsage function exported.

Remove duplicate imports

Simpler version of #41 issue. Consider the following situation:

import App.Db (WithDb, query)
import App.Db (WithDb, execute)

It would be really good if smuggler can group modules with same names with join their import lists, like in the following snippet:

import App.Db (WithDb, execute, query)

Such situation often happen after resolving conflicts when rebasing over some branch. If such feature is implemented, then people will be able to just remove >>>>> git markers and smuggler will compress imports for them.

Learn how to build 'TcLclEnv'

This data type:

• https://hackage.haskell.org/package/ghc-8.4.1/docs/TcRnTypes.html#t:TcLclEnv

Probably will use this function:

• https://hackage.haskell.org/package/ghc-8.4.1/docs/TcRnMonad.html#v:initTcWithGbl

Write juicy documentation

• Add logo
• Add slogan
• Describe how to use
• Record demo video