Coder Social home page Coder Social logo

orijtech / structslop Goto Github PK

View Code? Open in Web Editor NEW
790.0 9.0 30.0 47 KB

structslop is a static analyzer for Go that recommends struct field rearrangements to provide for maximum space/allocation efficiency.

License: Apache License 2.0

Go 100.00%
golang

structslop's Introduction

structslop

Build status

Package structslop defines an Analyzer that checks if struct fields can be re-arranged to optimize size.

Installation

With Go modules:

go get github.com/orijtech/structslop/cmd/structslop

Without Go modules:

$ cd $GOPATH/src/github.com/orijtech/structslop
$ git checkout v0.0.6
$ go get
$ install ./cmd/structslop

Usage

You can run structslop either on a Go package or Go files, the same way as other Go tools work.

Example:

$ structslop github.com/orijtech/structslop/testdata/src/struct

or:

$ structslop ./testdata/src/struct/p.go

Sample output:

/go/src/github.com/orijtech/structslop/testdata/struct/p.go:30:9: struct has size 24 (size class 32), could be 16 (size class 16), you'll save 50.00% if you rearrange it to:
struct {
	y uint64
	x uint32
	z uint32
}

/go/src/github.com/orijtech/structslop/testdata/struct/p.go:36:9: struct has size 40 (size class 48), could be 24 (size class 32), you'll save 33.33% if you rearrange it to:
struct {
	_  [0]func()
	i1 int
	i2 int
	a3 [3]bool
	b  bool
}

/go/src/github.com/orijtech/structslop/testdata/struct/p.go:59:9: struct has size 40 (size class 48), could be 32 (size class 32), you'll save 33.33% if you rearrange it to:
struct {
	y uint64
	t *httptest.Server
	w uint64
	x uint32
	z uint32
}

/go/src/github.com/orijtech/structslop/testdata/struct/p.go:67:9: struct has size 40 (size class 48), could be 32 (size class 32), you'll save 33.33% if you rearrange it to:
struct {
	y uint64
	t *s
	w uint64
	x uint32
	z uint32
}

Example, for the first report above, the output meaning:

  • The current struct size is 24, the size that the Go runtime will allocate for that struct is 32.
  • The optimal struct size is 16, the size that the Go runtime will allocate for that struct is 16.
  • The layout of optimal struct.
  • The percentage savings with new struct layout.

That said, some structs may have a smaller size, but for efficiency, the Go runtime will allocate them in the same size class, then those structs are not considered sloppy:

type s1 struct {
	x uint32
	y uint64
	z *s
	t uint32
}

and:

type s2 struct {
	y uint64
	z *s
	x uint32
	t uint32
}

have the same size class 32, though s2 layout is only 24 byte in size.

However, you can still get this information when you want, using -verbose flag:

$ structslop -verbose ./testdata/src/verbose/p.go
/go/src/github.com/orijtech/structslop/testdata/src/verbose/p.go:17:8: struct has size 0 (size class 0)
/go/src/github.com/orijtech/structslop/testdata/src/verbose/p.go:19:9: struct has size 1 (size class 8)
/go/src/github.com/orijtech/structslop/testdata/src/verbose/p.go:23:9: struct has size 32 (size class 32), could be 24 (size class 32), optimal fields order:
struct {
	y uint64
	z *s
	x uint32
	t uint32
}

Note

For applying suggested fix, use -apply flag, instead of -fix.

Development

Go 1.20+

Running test

Add test case to testdata/src/struct directory, then run:

go test

Contributing

TODO

structslop's People

Contributors

atc0005 avatar cuonglm avatar elias-orijtech avatar rootulp avatar savetherbtz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

structslop's Issues

The results for fieldalignment and structslop are not the same. Who is better?

package main

type Car struct {
	flag        bool
	age         int32
	F1          int8
	age2        int32
	age3        int16
	F2          int64
	F3          *int32
	InnerStruct struct {
		InnerByte byte
		//InnerStr string
	}
	F4   []byte
	Name string
	F5   error
}

func main() {

}

fieldalignment

fieldalignment -fix main.go  
/Users/TiM/Downloads/GolandProjects/helloworld/main.go:3:10: struct of size 104 could be 88

fieldalignment result

type Car struct {
	F5          error
	F3          *int32
	Name        string
	F4          []byte
	F2          int64
	age         int32
	age2        int32
	age3        int16
	flag        bool
	InnerStruct struct{ InnerByte byte }
	F1          int8
}

structslop

structslop -apply main.go     
/Users/TiM/Downloads/GolandProjects/helloworld/main.go:3:10: struct has size 104 (size class 112), could be 88 (size class 96), you'll save 14.29% if you rearrange it to:
struct {
        F4          []byte
        Name        string
        F5          error
        F3          *int32
        F2          int64
        age         int32
        age2        int32
        age3        int16
        flag        bool
        InnerStruct struct{ InnerByte byte }
        F1          int8
}

structslop result

type Car struct {
        F4          []byte
        Name        string
        F5          error
        F3          *int32
        F2          int64
        age         int32
        age2        int32
        age3        int16
        flag        bool
        InnerStruct struct{ InnerByte byte }
        F1          int8
}

The results for fieldalignment and structslop are not the same

test the under Car with fieldalignment

type Car struct {
        F4          []byte
        Name        string
        F5          error
        F3          *int32
        F2          int64
        age         int32
        age2        int32
        age3        int16
        flag        bool
        InnerStruct struct{ InnerByte byte }
        F1          int8
}
fieldalignment -fix main.go
/Users/TiM/Downloads/GolandProjects/helloworld/main.go:3:10: struct with 64 pointer bytes could be 48

index out of range crash

When running structslop against pkg/sql of CockroachDB I got the following:

panic: runtime error: index out of range [11] with length 10

goroutine 10501 [running]:
github.com/orijtech/structslop.run.func1(0x138cca0, 0xc006541960)
	/Users/yuzefovich/go/pkg/mod/github.com/orijtech/[email protected]/structslop.go:98 +0xbf2
golang.org/x/tools/go/ast/inspector.(*Inspector).Preorder(0xc02cc427a0, 0xc004e81b20, 0x1, 0x1, 0xc004f6fb30)
	/Users/yuzefovich/go/pkg/mod/golang.org/x/[email protected]/go/ast/inspector/inspector.go:77 +0xa2
github.com/orijtech/structslop.run(0xc02cc400c0, 0xc02cc400c0, 0x0, 0x0, 0x0)
	/Users/yuzefovich/go/pkg/mod/github.com/orijtech/[email protected]/structslop.go:68 +0x1fa
golang.org/x/tools/go/analysis/internal/checker.(*action).execOnce(0xc02ec706e0)
	/Users/yuzefovich/go/pkg/mod/golang.org/x/[email protected]/go/analysis/internal/checker/checker.go:690 +0x87e
sync.(*Once).doSlow(0xc02ec706e0, 0xc00004f790)
	/usr/local/opt/go/libexec/src/sync/once.go:66 +0xec
sync.(*Once).Do(...)
	/usr/local/opt/go/libexec/src/sync/once.go:57
golang.org/x/tools/go/analysis/internal/checker.(*action).exec(0xc02ec706e0)
	/Users/yuzefovich/go/pkg/mod/golang.org/x/[email protected]/go/analysis/internal/checker/checker.go:579 +0x65
golang.org/x/tools/go/analysis/internal/checker.execAll.func1(0xc02ec706e0)
	/Users/yuzefovich/go/pkg/mod/golang.org/x/[email protected]/go/analysis/internal/checker/checker.go:567 +0x34
created by golang.org/x/tools/go/analysis/internal/checker.execAll
	/Users/yuzefovich/go/pkg/mod/golang.org/x/[email protected]/go/analysis/internal/checker/checker.go:573 +0x125

Unable to use in windows environment

When using the proposed installation steps (both cases) the executable produced under the darwin_amd64 folder is (understandably) not valid for windows.

When I tried building the project myself for windows (GOOS=windows;GOARCH=amd64) and running it I get this error .
"The specified executable is not a valid application for this OS platform"

Error when checking a package or (.go) file

Using Go 1.18 and structslop gives me error like the following: package "database/sql/driver" without types was imported from "command-line-arguments" or similar errors (just different package) depending on .go file imported packages.

Print field's type as legal Go syntax.

Currently, for struct like:

type s struct{
    Jump obj.As
    Index int
}

The warning message printed as:

struct{Jump cmd/internal/obj.As; Index int}

It should be:

struct{Jump obj.As; Index int}

instead.

Add more test cases

When #5 were merged, we should add more test cases.

What can be edge case for this?

why this re-arrange algorithm works?

structslop/structslop.go

Lines 246 to 267 in 13637e2

sort.Slice(fields, func(i, j int) bool {
ti, tj := fields[i].Type(), fields[j].Type()
si, sj := sizes.Sizeof(ti), sizes.Sizeof(tj)
if si == 0 && sj != 0 {
return true
}
if sj == 0 && si != 0 {
return false
}
ai, aj := sizes.Alignof(ti), sizes.Alignof(tj)
if ai != aj {
return ai > aj
}
if si != sj {
return si > sj
}
return false
})

why sort by <align of field, size of field> can minimize the sizeof struct?
is there some formal prove?

Output is hard to read

I ran structslop -fix ./... and got the following output. It does not look too bad when copy-pasted here, but it is impossible to read in the terminal.

/home/jacob/Git/fyne/internal/driver/glfw/canvas.go:22:15: struct has size 256 (size class 256), could be 240 (size class 240), rearrange to struct{shortcut fyne.ShortcutHandler; size fyne.Size; content fyne.CanvasObject; menu fyne.CanvasObject; context driver.WithContext; painter gl.Painter; onTypedRune func(rune); menuTree *renderCacheTree; overlays *overlayStack; onTypedKey func(*fyne.KeyEvent); onKeyDown func(*fyne.KeyEvent); menuFocusMgr *app.FocusManager; contentFocusMgr *app.FocusManager; contentTree *renderCacheTree; refreshQueue chan fyne.CanvasObject; dirtyMutex *sync.Mutex; onKeyUp func(*fyne.KeyEvent); sync.RWMutex; texScale float32; detectedScale float32; scale float32; dirty bool; padded bool} for optimal size (6.25% savings)
/home/jacob/Git/fyne/internal/driver/glfw/window.go:47:13: struct has size 400 (size class 416), could be 376 (size class 384), rearrange to struct{pending []func(); clipboard fyne.Clipboard; mousePressed fyne.CanvasObject; mouseLastClick fyne.CanvasObject; mouseOver desktop.Hoverable; mouseDraggedOffset fyne.Position; mouseDragged fyne.Draggable; title string; icon fyne.Resource; mousePos fyne.Position; mouseDragPos fyne.Position; mouseClickCount int; eventQueue chan func(); height int; width int; mainmenu *fyne.MainMenu; canvas *glCanvas; cursor *glfw.Cursor; ypos int; xpos int; mouseButton desktop.MouseButton; onCloseIntercepted func(); onClosed func(); mouseCancelFunc context.CancelFunc; viewport *glfw.Window; viewLock sync.RWMutex; eventLock sync.RWMutex; createLock sync.Once; eventWait sync.WaitGroup; mouseDragStarted bool; visible bool; centered bool; shouldExpand bool; decorate bool; fullScreen bool; fixedSize bool; master bool} for optimal size (7.69% savings)
/home/jacob/Git/fyne/dialog/fileitem.go:19:21: struct has size 136 (size class 144), could be 128 (size class 128), rearrange to struct{widget.BaseWidget; name string; location fyne.URI; picker *fileDialog; isCurrent bool; dir bool} for optimal size (11.11% savings)
/home/jacob/Git/fyne/test/testcanvas.go:30:17: struct has size 168 (size class 176), could be 160 (size class 160), rearrange to struct{fyne.ShortcutHandler; size fyne.Size; content fyne.CanvasObject; painter SoftwarePainter; hovered desktop.Hoverable; onTypedRune func(rune); onTypedKey func(*fyne.KeyEvent); overlays *internal.OverlayStack; focusMgr *app.FocusManager; propertyLock sync.RWMutex; scale float32; padded bool} for optimal size (9.09% savings)
/home/jacob/Git/fyne/internal/driver/gomobile/canvas.go:20:19: struct has size 264 (size class 288), could be 248 (size class 256), rearrange to struct{shortcut fyne.ShortcutHandler; content fyne.CanvasObject; windowHead fyne.CanvasObject; menu fyne.CanvasObject; dragging fyne.Draggable; painter gl.Painter; size fyne.Size; focused fyne.Focusable; touchLastTapped fyne.CanvasObject; onTypedKey func(event *fyne.KeyEvent); onTypedRune func(rune); touched map[int]mobile.Touchable; touchCancelFunc context.CancelFunc; touchTapCount int; lastTapDown map[int]time.Time; lastTapDownPos map[int]fyne.Position; overlays *internal.OverlayStack; refreshQueue chan fyne.CanvasObject; minSizeCache map[fyne.CanvasObject]fyne.Size; scale float32; inited bool; padded bool} for optimal size (11.11% savings)

Impossible to read in terminal:
image

Perhaps a few lines in between would make it more readable? Maybe even better formatting of it?

README: Install path is deprecated with latest Go

I am getting this (in my home directory, with go version go1.17.2 darwin/amd64):

% go get github.com/orijtech/structslop/cmd/structslop
go: downloading github.com/orijtech/structslop v0.0.6
go: downloading golang.org/x/tools v0.0.0-20200917221617-d56e4e40bc9d
go: downloading github.com/dave/dst v0.26.2
go: downloading golang.org/x/mod v0.3.0
go get: installing executables with 'go get' in module mode is deprecated.
	Use 'go install pkg@version' instead.
	For more information, see https://golang.org/doc/go-get-install-deprecation
	or run 'go help get' or 'go help install'.

With Go 1.17 the new way is:

% go install github.com/orijtech/structslop/cmd/structslop@latest

is this linter compatible with Go 1.20+ ?

@odeke-em

I enabled this linter in a Go 1.19 Docker image (golang:1.19.7) without issue, but for a Go 1.20 image (golang:1.20.2) I get this error:

structslop: internal error: package "errors" without types was imported from "github.com/..."

To reproduce:

  1. docker image pull golang:1.20.2
  2. docker container run -it --rm golang:1.20.2
  3. git clone https://github.com/atc0005/check-whois
  4. cd check-whois
  5. go install github.com/orijtech/structslop/cmd/[email protected]
    • same result as go install github.com/orijtech/structslop/cmd/structslop@latest , but noting specific version for clarity
  6. structslop ./...

Output:

root@6c4091e45d1e:/go/check-whois# structslop ./...
structslop: internal error: package "errors" without types was imported from "github.com/atc0005/check-whois/internal/domain"

This produces no errors and no output:

  1. docker image pull golang:1.19.7
  2. docker container run -it --rm golang:1.19.7
  3. git clone https://github.com/atc0005/check-whois
  4. cd check-whois
  5. go install github.com/orijtech/structslop/cmd/[email protected]
  6. structslop ./...

Do not print old struct in report message

Currently, the report message includes both original struct and optimal struct. This can lead to ugly message, something like:

struct{x uint32; y uint64; z uint32} has size 24, could be 16, rearrange to struct{y uint64; x uint32; z uint32} for optimal size

It can be better if we can include struct name in report, something like:

struct X has size 24, could be 16, rearrange to struct{y uint64; x uint32; z uint32} for optimal size

structslop: internal error: package "context" without types was imported

OS: Windows 10 64bit
Go version: 1.18.4
structslop v0.0.6
Installed using go get i.e modules
My target project mod is using Go version 1.18

Usage: within target project
structslop ./

Expected outcome:
Suggestions or nothing if no suggestions.

Actual outcome:
structslop: internal error: package "context" without types was imported from "docgenapi [docgenapi.test]"

Maybe i'm reading this wrong perhaps? The message doesn't give any useful information about how to correct my code. Not sure what i'm meant to do from this.

Any advice appreciated.

distribution: prepare to host and distribute structslop for bootstrapping

We need to investigate how to distribute structslop for use in test binaries given that including it in the Go standard library’s cmd/vet makes its outputs mandatory actions, yet the amount of code out there organized in certain ways is immense. We should see how other passes like staticcheck are distributed. With a surge in popularity/usage, we can examine usage patterns, demand and that will justify if we should donate it to the Go project.

Making this into a linter.

I'm sorry if something like this exists, but could we maybe turn this into a lint/format rule?

I'd love to work on this with anyone if they're interested! : )

align values on pointer-size values to avoid false positives that ignore padding

The current code just performs a packing based off reducing the memories, without accounting for the fact that all struct fields have to be aligned, for example I just added structslop to the Go standard library in a private fork but it is reporting unrealistic sizes that are also odd; there is no way we'll have odd sizes, yet padding exists.

For an exhibit here is an error message

./mbitmap.go:132:15: struct{bytep *uint8; mask uint8; index uintptr}
has size 24, could be 17, rearrange to struct{bytep *uint8; index uintptr; 
mask uint8} for optimal size

the error says the value is 24, but could be 17; that's impossible; the size will be 24 on both 32-bit and 64-bit machines given that we pad.

I believe the remedy for this could be rounding up to the next multiple of the pointer size

show percentage savings after showing optimal size change

Right now we just show numeric values in improvements, but it might be useful and create easier buy-in if we can show how much has been saved, especially after adding support for size classes as per #22 (comment) so

<LINENO_POS>: struct has size 112 (size class <CLASS_SIZE>), rearrange to
<REARRANGEMENT> for optimal size (<PERCENT_DECREASE>% savings)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.