koykov / inspector

Inspection framework for arbitrary Go structs.

Inspector

Inspector is a code-generation framework of special wrappers of arbitrary types and data structures that allows to read/write/iterate fields without using reflection and produce minimum or zero allocations.

Each type-wrapper (next "inspector" or "type-inspector") has name of original type with suffix Inspector and implements Inspector interface.

Intro

The main idea: you may read/write/iterate (inspect) structure of arbitrary type in common way. The historical reason of development that framework was dyntpl package and many others. Usually this problem solves using reflection, but that way is extremely slow in general and produces huge amount of allocs on every use of reflect.Value type. There is good library reflect2 that solves 'reflect.Value' problem, but it is also slow. The perfect way if to use type assertion together with hardcoded combinations of all possible paths to type fields. Unfortunately this way isn't a pure "dynamic solution". Let's consider that problem using example:

Let we have type T:

type T struct {
    L1 *L1
}

type L1 struct {
    L2 *L2
}

type L2 struct {
    L3 *L3
}

type L3 struct {
    S string
    I int64
    F float64
}

with many nested subtypes. And we need to read data of fields for arbitrary path, eg:

• obj.L1.L2.L3.S
• obj.L1.L2.L3.F
• ...

The hard requirement is a "dynamic" paths to fields - at any time may require to read value of any field.

Let's check how that problem may be solved ob showcase:

reflect

Solution https://github.com/koykov/versus/blob/master/inspector2/reflect.go#L8

It's a pure dynamic solution, it solves the problem, but benchmark looks bad:

BenchmarkReflect/obj.L1.L2.L3.S-8         	 3119121	       375.6 ns/op	      64 B/op	       5 allocs/op

Speed is not acceptable and too many allocations - they will trigger problems with GC.

reflect2

Solution https://github.com/koykov/versus/blob/master/inspector2/reflect2.go#L12

It's also pure dynamic solution, and it should solve problem with allocations (cost of use reflect.Value). It's true, but benchmarks shows problems:

BenchmarkReflect2/obj.L1.L2.L3.S-8         	 2973918	       391.3 ns/op	       0 B/op	       0 allocs/op

Allocations problem is solved, but speed is much worse than native reflect due to internal design of reflect2.frozenConfig (it uses sync.Map inside and speed reduces due to sync operations).

inspector

Solution https://github.com/koykov/versus/blob/master/inspector2/inspector_test.go#L13 Benchmark https://github.com/koykov/versus/blob/master/inspector2/inspector_test.go#L18

BenchmarkInspector/obj.L1.L2.L3.S-8         	159301698	         7.596 ns/op	       0 B/op	       0 allocs/op

And that speed is acceptable. See explanation is the next chapter.

How it works

As mentioned, the perfect way is using type assertion together with hardcoded combinations of all possible paths to type fields, i.e. zero reflection. Type T allows the following paths combinations:

• obj.L1.L2.L3.S
• obj.L1.L2.L3.I
• obj.L1.L2.L3.F
• obj.L1.L2.L3
• obj.L1.L2
• obj.L1

Type is simple, thus has only 6 combinations. The code considers these combinations looks the following https://github.com/koykov/versus/blob/master/inspector2/inspector2_ins/t_ins.go#L31. So primitive and thus so fast.

But what about big types? They provide hundreds/thousands combinations and write code manually is so big and boring work. The further support is a problem as well. Therefore, this work was automatized writing special code-generation tool.

Additional features

Fields values reading is so important feature, but inspectors also provide features:

• write values to fields
• compare fields values
• iterate iterable fields по заданному пути
• deep comparison of a whole types по заданному пути
• deep copy of types
• reset types

Basic types support

static

For support of basic types (int, uint, float64, ...) was developed special static inspector. It uses in dyntpl and decoder packages for primitive types.

strings

Special for types string and [][]byte was developed inspector strings. See test/bench.

map[string]any

Popular type uses together with encoding/json supported by inspector map[string]any. See test/bench https://github.com/koykov/inspector/blob/master/test/stranymap_test.go

inspector's Issues

Map inspector

Hello, hope this issue finds you well.

I would like to propose an inspector of sort to traverse JSON/Msgpack maps, i.e. map[string]any.

It should work like a recursive StaticInspector:

switch x := v.(type) {
case int:
    // ...
case int8:
    // ...

case map[string]any:
    MapInspector.Traverse(x)
// ...
}

Could really use that for nested map traversal

[inspc] File/path targets

Hello,

I'm using your Go project and it's been working great so far, thank you for all your hard work! I wanted to suggest a couple of improvements that could potentially enhance its usability and flexibility.

Ability to compile specific files or paths without GOPATH binding:
Currently, it seems the project relies heavily on the GOPATH for identifying and compiling Go source files. However, in certain situations, it could be beneficial to compile Go source files or directories that are outside the GOPATH or do not adhere to its conventional structure. By supporting this, users will have the flexibility to work with Go code in different ways and it could possibly improve the workflow in non-traditional or complex project structures.

Excluding specific Go structs from compilation using go:generate:
The second suggestion is to provide a way to exclude certain Go structs from the compilation process, preferably using go:generate. This could help in situations where certain structs are meant for auxiliary or testing purposes and not meant to be included in the final binary. It will provide an effective way to selectively include/exclude parts of the code during compilation.

These enhancements, I believe, would provide more flexibility and control to the users of your project. I would be glad to provide more information or use-case scenarios if it helps. Please let me know what you think about these proposed features.

Thanks once again for your work on this project!

Best,
Boris

[naming] Inspector* within inspector package

Hello,

I would suggest you to rid of Inspector suffixes in inspector package, with accordance to Google Go Style Guide.
e.g. inspector.BaseInspector => inspector.Base, inspector.GetInspector => inspector.Get and so on.