fatih / structs

Utilities for Go structs

Archived project. No maintenance.

This project is not maintained anymore and is archived. Feel free to fork and make your own changes if needed. For more detail read my blog post: Taking an indefinite sabbatical from my projects

Thanks to everyone for their valuable feedback and contributions.

Structs

Structs contains various utilities to work with Go (Golang) structs. It was initially used by me to convert a struct into a map[string]interface{}. With time I've added other utilities for structs. It's basically a high level package based on primitives from the reflect package. Feel free to add new functions or improve the existing code.

Install

go get github.com/fatih/structs

Usage and Examples

Just like the standard lib strings, bytes and co packages, structs has many global functions to manipulate or organize your struct data. Lets define and declare a struct:

type Server struct {
	Name        string `json:"name,omitempty"`
	ID          int
	Enabled     bool
	users       []string // not exported
	http.Server          // embedded
}

server := &Server{
	Name:    "gopher",
	ID:      123456,
	Enabled: true,
}

// Convert a struct to a map[string]interface{}
// => {"Name":"gopher", "ID":123456, "Enabled":true}
m := structs.Map(server)

// Convert the values of a struct to a []interface{}
// => ["gopher", 123456, true]
v := structs.Values(server)

// Convert the names of a struct to a []string
// (see "Names methods" for more info about fields)
n := structs.Names(server)

// Convert the values of a struct to a []*Field
// (see "Field methods" for more info about fields)
f := structs.Fields(server)

// Return the struct name => "Server"
n := structs.Name(server)

// Check if any field of a struct is initialized or not.
h := structs.HasZero(server)

// Check if all fields of a struct is initialized or not.
z := structs.IsZero(server)

// Check if server is a struct or a pointer to struct
i := structs.IsStruct(server)

Struct methods

The structs functions can be also used as independent methods by creating a new *structs.Struct. This is handy if you want to have more control over the structs (such as retrieving a single Field).

// Create a new struct type:
s := structs.New(server)

m := s.Map()              // Get a map[string]interface{}
v := s.Values()           // Get a []interface{}
f := s.Fields()           // Get a []*Field
n := s.Names()            // Get a []string
f := s.Field(name)        // Get a *Field based on the given field name
f, ok := s.FieldOk(name)  // Get a *Field based on the given field name
n := s.Name()             // Get the struct name
h := s.HasZero()          // Check if any field is uninitialized
z := s.IsZero()           // Check if all fields are uninitialized

Field methods

We can easily examine a single Field for more detail. Below you can see how we get and interact with various field methods:

s := structs.New(server)

// Get the Field struct for the "Name" field
name := s.Field("Name")

// Get the underlying value,  value => "gopher"
value := name.Value().(string)

// Set the field's value
name.Set("another gopher")

// Get the field's kind, kind =>  "string"
name.Kind()

// Check if the field is exported or not
if name.IsExported() {
	fmt.Println("Name field is exported")
}

// Check if the value is a zero value, such as "" for string, 0 for int
if !name.IsZero() {
	fmt.Println("Name is initialized")
}

// Check if the field is an anonymous (embedded) field
if !name.IsEmbedded() {
	fmt.Println("Name is not an embedded field")
}

// Get the Field's tag value for tag name "json", tag value => "name,omitempty"
tagValue := name.Tag("json")

Nested structs are supported too:

addrField := s.Field("Server").Field("Addr")

// Get the value for addr
a := addrField.Value().(string)

// Or get all fields
httpServer := s.Field("Server").Fields()

We can also get a slice of Fields from the Struct type to iterate over all fields. This is handy if you wish to examine all fields:

s := structs.New(server)

for _, f := range s.Fields() {
	fmt.Printf("field name: %+v\n", f.Name())

	if f.IsExported() {
		fmt.Printf("value   : %+v\n", f.Value())
		fmt.Printf("is zero : %+v\n", f.IsZero())
	}
}

Credits

• Fatih Arslan
• Cihangir Savas

License

The MIT License (MIT) - see LICENSE.md for more details

structs's Issues

panic when using both flatten and omitempty

Hello!

When flattening a struct, if all of the fields in the struct have the omitempty option at least one of them must be set or the library will panic. This means I can't safely combine "flatten" and "omitempty". It would be nice if the two could be used together safely.

I believe this happens because of this check: https://github.com/fatih/structs/blob/master/structs.go#L527

The embedded struct is converted to a map. If the map is empty the function returns a the struct value instead of a map which results in a panic in the parent function since it expects a map[string]interface{} (https://github.com/fatih/structs/blob/master/structs.go#L142).

For example:

package main

import (
	"fmt"

	"github.com/fatih/structs"
)

type Parent struct {
       Embedded `structs:",flatten"`
}

type Embedded struct {
        A string `structs:",omitempty"`
        B string `structs:",omitempty"`
}

func main() {
        p := &Parent{}
        fmt.Println(structs.Map(p))
}

Results in this error:

panic: interface conversion: interface {} is main.Embedded, not map[string]interface {}

goroutine 1 [running]:
github.com/fatih/structs.(*Struct).FillMap(0xc420014240, 0xc420012240)
	/home/serenst/go/src/github.com/fatih/structs/structs.go:144 +0x569
github.com/fatih/structs.(*Struct).Map(0xc420014240, 0xc42000a320)
	/home/serenst/go/src/github.com/fatih/structs/structs.go:83 +0x65
github.com/fatih/structs.Map(0x493640, 0xc42000a320, 0x0)
	/home/serenst/go/src/github.com/fatih/structs/structs.go:449 +0x43
main.main()
	/home/serenst/bugreport.go:20 +0x6a
exit status 2

creating an exported field

Is it possible to create a field if it is unsettable?

Range over nested slice of structs (using fields)

Hi,

I would like some help on how to iterate of the slice of structs like below. Im actually creating a recursive function that essentially walks the try down to primitives and their values. Ive covered all relect types except for handling the slice and how to use / convert a field (e.g E below), that is of stype slice (or array), and get access to the slices indexes instances and their X.Y and X.Z values?

A {
   B {
      C string
      D bool
   }
   E []*X
}

X {
   Y string
   Z int
}

I start by using

structs.New()

on an instance of the root struct, and then range over the fields as below.

for _, f := range struct.Fields() {

Its when one of the fields (kind) is a slice what do I do now to get / range over the slice indexes and their fields? I get a panic add this as the input to structs.New() again as its NOT a struct.

Im not sure how best to proceed handling the slice / array? Ive been looking at the tests but maybe missing something obvious?

Thoughts / helps much welcome

Kurt

data type limitation

Is there a limitation that this package only work on int value. I tried with int64/uint/uint64 and somehow it does not seems to change values

Ref:
package main

import (
"fmt"

"github.com/fatih/structs"

)

type Data struct {
IntVal uint64
StringVal string
NewStruct NewStruct
}

type NewStruct struct {
InVal int
}

func main() {
var vals Data
vals.IntVal = 1
vals.NewStruct.InVal = 1
vals.StringVal = "Hello"
fmt.Println("Values are", vals)
s := structs.New(&vals)
//t := s.Field("IntVal")
//fmt.Printf("Type of %s is %s\n", "IntVal", t.Kind())
// s.Field("StringVal").Set("Nopes")
s.Field("IntVal").Set(2)
s.Field("NewStruct").Field("InVal").Set(2)
fmt.Println("New Values are", vals)
//doIt(&vals)
//fmt.Println("New Values after doIt", vals)
return
}

Output:
Values are {1 Hello {1}}
New Values are {1 Hello {2}}

Nested array structs dont pick up custom tags

Consider following example where struct person consists of array of addresses:

package main

import (
    "fmt"

    "github.com/fatih/structs"
)

func main() {
    type address struct {
        Country string `structs:"country"`
    }

    type person struct {
        Name      string    `structs:"name"`
        Addresses []address `structs:"addresses"`
    }

    p := person{Name: "test", Addresses: []address{address{Country: "England"}}}
    fmt.Printf("%+v", structs.Map(p))
}

The output is (custom tag for country is not picked):
map[name:test addresses:[{Country:England}]]

while the expected output is:
map[name:test addresses:[{country:England}]]

Versions and tags

Hi there,

I'm busy packaging structs for Debian as a dependency for minio. For packaging to work correctly, we need to be able to scan for updated versions, and for that we need releases or tags. Please read https://wiki.debian.org/debian/watch and specifically https://wiki.debian.org/debian/watch#GitHub for more information on tags/release information in github that is needed to make packaging easier.

Can you please tag or release your software accordingly.

Regards
Henti

Field.Set("field", value) methods

Add IsEmpty() function

Checks whether the fields are initialized or not.

Add structs.Names() method

Just to return the names of each struct field. It'll be handy to use in a for range:

type Server struct {
    Name     string
    Number   int
    Disabled bool
}

s := new(Server)
for _, name := range structs.Names(s) {
    // prints: "Name", "Number" and "Disabled"
    fmt.Println(name)
}

Custom Map function

Similar to the json.Marshal() it would be really great to have the ability to add my own Map() function to a struct in order to overwrite the default mapping.

Example use case:

type foo struct {
	data bar
}

type bar struct {
	Name  string
	Value string
}

func main() {
	f := &foo{
		data: bar{
			Name:  "foo",
			Value: "bar",
		},
	}
	fmt.Println(structs.Map(f))
}

In this example the actual data is hidden away and only accessible from the foo struct itself (at least outside the package). So I would want to build a function foo.Map() that can extract the data. In a nested case it is necessary that the map package calls the custom Map function if it exists.

add options to use custom struct tag name instead of `structs:"..."`

I would like to use json:"..." instead of structs:"..", for now I am forced to use both when a struct is used by encoding/json and fatih/structs with converting structs to map[string]interface{}.

type ErrorResponse struct {
    Error string `json:"error" structs:"error"`
}

Field.Set: use of reflect.Indirect

Hello,

Thanks for this useful package, but I have a question: why did you use reflect.Indirect when setting a field instead of using the value of the field itself? When setting pointers, it seems that it just does not work when using the value that the field points to.

can map function parse struct's tag

Global omitempty?

Thanks for creating this package, it's super simple and useful.

Do you know if it's possible to request an omitempty policy at a global / struct level without having to specify it for every field?

Thanks!

New() with options

Hi,

I would like to change the tag structs to column in structures tag. How can I set the new tag ? I'd love to see something like this:

structs.WithTag("column").New(something).Map()

or

structs.NewWithOptions(structs.Options{
    Tag: "column",
     Object: something,
}).Map()

What do you think about it ?

Custom tag name doesn't propagate to inner structs

Add Fields() or SortedFields() function

Can it work when fields start with an lowercase

I tried using lowercase fields with

type photoUploadForm struct {
userid string form:"userid" valid:"Required"
caption string form:"caption" valid:"Required"
}

m := structs.Map(ph)

m is always empty.

Add Addr method for Field

In order to allow for modification of nested structures, we must have a way to retrieve a pointer for an individual field inside the first struct, so we may call structs.New on it later and enable modification of individual fields.

I'm asuming we need to call reflect.Addr on the unexported reflect.Value.

https://golang.org/pkg/reflect/#Value.Addr

Report fields of anonymous fields

From the documentation of the encoding/json package:

Anonymous struct fields are usually marshaled as if their inner exported fields were fields in the outer struct, subject to the usual Go visibility rules amended as described in the next paragraph...

For my use case it would be really nice if structs.Map worked the same way including all the inner fields of an anonymous struct field instead of the anonymous struct field itself.

I need to slightly tweak the JSON produced by marshaling a certain struct with an anonymous field. My plan is to use structs.Map() to turn my struct into a map[string]interface{} which would let me easily tweak what I need to before marshaling. Trouble is that structs.Map doesn't seem to treat anonymous fields the same way that the json marshaler treats them.

Map() generates inconsistent type

When using Map() method, field with value of struct generates map. However field with pointer of struct doesn't generate map, but original struct instead:

package main

import (
    "fmt"
    "github.com/fatih/structs"
)

type single struct{ String string }

type arr struct {
    ArrStruct []single
    ArrPtr    []*single
}

func main() {
    str := single{"hi"}
    ptr := &single{"hi"}
    a := arr{ArrStruct: []single{str}, ArrPtr: []*single{ptr}}
    m := structs.Map(a)
    fmt.Printf("a %#v\nm %#v\n", a, m)
}

Generates:

a main.arr{ArrStruct:[]main.single{main.single{String:"hi"}}, ArrPtr:[]*main.single{(*main.single)(0xc82000a300)}}
m map[string]interface {}{"ArrStruct":[]interface {}{map[string]interface {}{"String":"hi"}}, "ArrPtr":[]*main.single{(*main.single)(0xc82000a300)}}

where ArrStruct converts to []interface{}{map[string]interface{}} but ArrPtr converts to []*main.single

It would be nice to be in sync, i.e. everything converts map[string]interface{}

stack overflow on structs.Map()

			type Parent struct {
				Child   *Parent
				LogData string
			}

			myVar := Parent{}
			myVar.LogData = "Hello"
			myVar.Child = &myVar

If myVar is passed to structs.Map(myVar) will cause a stack overflow. I know you can specify not to recurse on the struct field definition, but I am trying to guard against the case that I don't know what the struct will look like nor am I defining the struct itself.

Could a default level be specified, or some other guard to prevent from panic.

Map() panics with nil pointer

Steps to reproduce:

type A struct {
    Unused *B
}

type B struct {
}

func main() {
    a := A{} 
    s := structs.New(a).Map()
    fmt.Println(s)
}

http://play.golang.org/p/YAh_AJSEQi

Output:

panic: not struct

Checking for a nil pointer on IsStruct() and returning false solved it for me, but I'm not sure this is the right solution. If my approach is fine, I'll be happy to submit a PR.

I get empty return for this

Can anyone help me to see what's wrong with the following line please?

https://github.com/suntong/lang/blob/820cf694de6b099256f429b9fff91265461eef25/lang/Go/src/ds/structs_example.go#L131-L147

it prints:

map[string]interface {}{}

while the previous test prints the following, which is what I want:

map[string]interface {}{"Servers":[]main.Server{main.Server{Name:"Arslan", ID:123456, Enabled:true}, main.Server{Name:"Arslan", ID:123456, Enabled:true}}}

How to fix it to make it working? Thx!

Why structs.Map() convert anonymous field in sub map?

type A{
  Name string
}

type B{
 A
 Surname string
}
var t B;
structs.Map(t)

generate a map like
map[A:map[Name:] Surname:]
but A is an anonymous field I'm expecting a flat map like
map[Name:Surname:]

Omitempty option for creating maps

It would be lovely to have an ",omitempty" option to struct members, so they can be omitted if empty similar to the json encoder.

Add tests for anonymous fields

Nested structs

How to set values in nested structs?

I have something like this:

struct1 {
   struct2 {
      field1 string
      field2 bool
   }
   struct3 {
      field3 string
      field4 bool
   }
}

I got all fields but I can't set it...
I made a little recursion about this

func some(c interface{}){
  for _, f := range structs.Fields(c) {
   if f.Kind() == reflect.Struct {
      some(f.Value(), dev)
      //how to get a adressed value?
      continue
   } else {
      f.Set(v) //here an Error: field is not settable
   }
  }
}

some(struct1)

thank you!

Add ToSlice() function

Creates a []interface{} of the field values.

Values that are structs lose their tags

I'm using this library to get the keys for a struct, and then marshalling to json for values that are objects. It appears though that when the value is mapped, it loses its tags:

type ApiArg struct {
	Title         string       `json:"title"`
        RightsHolder RightsHolder `json:"rights_holder"`
}

type RightsHolder struct {
	FirstName string `json:"first_name"`
	LastName  string `json:"last_name"`
	Email      string `json:"email"`
}

apiArg := ApiArg{Title: "blah", RightsHolder: RightsHolder{FirstName: "John", LastName: "Doe", Email: "[email protected]"}}
apiArgMap := structs.Map(apiArg)
for key, val := range apiArgMap {
  valBytes, _ := json.Marshal(val)
  valString := string(valBytes)
}

FirstName, LastName, Email marshal as FirstName, etc, instead of first_name, last_name, email.

Changing the struct field names to lowercase makes it so they don't end up showing up in the map though.

How to set a ptr field to nil

I'm trying to unset a pointer field (ie, set it to nil), like so:

            if err := f.Set(nil); err != nil {
                fmt.Println(err.Error())
            }

However, I get this error:

wrong kind. got: invalid want: ptr

I believe this is because reflect.Value.Kind() will return Invalid for 'zero' values.

Is there a way around this?

Thanks!

Is there a way to do a non recursive nested struct map conversion?

I have some fields within my stuct that use null.String types which are structs themselves. However, these are wrappers for scalar types so I don't want to create nested maps for these. Is there an option to turn off recursive map conversion?

panic on nested map

I got a panic error if my struct has an nested map. Not sure but, i think it starts appear after go compiler updated. Now it 1.6.

reflect: Elem of invalid type
/usr/lib/go/src/runtime/panic.go:426 (0x42d659)
        gopanic: reflectcall(nil, unsafe.Pointer(d.fn), deferArgs(d), uint32(d.siz), uint32(d.siz))
/usr/lib/go/src/reflect/type.go:599 (0x5b3283)
        (*rtype).Elem: panic("reflect: Elem of invalid type")
/opt/project/src/github.com/fatih/structs/structs.go:535 (0x6cccd5)
        (*Struct).nested: v := val.Type().Elem()
/opt/project/src/github.com/fatih/structs/structs.go:120 (0x6c9c2e)
        (*Struct).FillMap: finalVal = s.nested(val)
/opt/project/src/github.com/fatih/structs/structs.go:84 (0x6c97db)
        (*Struct).Map: s.FillMap(out)
...

golang 1.7 issue

Here for tracking. Something in koding/multiconfig is panic-ing.

via koding/multiconfig#50

"""
It's issue with http://github.com/fatih/structs, the check for IsExported should be adapted as described in golang/go#12367
"""

which says

Code that assumes

f.PkgPath != nil 
means a field is unexported and must be ignored must now be revised to check for

f.PkgPath != nil && !f.Anonymous
for it to walk into the embedded structs to look for exported fields contained within.

Any way to override field names when converting a struct to a map?

JSON has the following:

type ErrorResponse struct {
	ErrorCode int    `json:"error_code"`
	ErrorStr  string `json:"error_str"`
}

which is a bit friendlier (BUT not Go style)...

Add IsStruct() function

Returns true if the given type is struct or a pointer to struct.

Cannot flatten the nested struct.

When I use flatten,
type Message struct { Model structs:",flatten" Name structs:"name" }

  `type Model struct {
CreatedAt rdb.Timestamp ` structs:"created_at"`
UpdatedAt rdb.Timestamp `structs:"updated_at"`
  }`

message is the instance of Message
when I use structs.Map(message)

It always return map[string]interface{} contains map[string]interface{} of Model. Seems Model fields cannot be flatten to same level of Name. Something wrong with my code?

Can't set value on embedded struct fields

structs.Field.Set fails when trying to set a value on an embedded struct field because CanSet returns false.

It should not fail since the documentation of CanSet says:

A Value can be changed only if it is addressable and was not obtained by the use of unexported struct fields

Here is an example that you can copy-paste:

package my_test

import (
    "reflect"
    "testing"

    "github.com/fatih/structs"
)

type Base struct {
    ID int
}

type User struct {
    Base
    Name string
}

// This test fails
func TestEmbeddedWithStructs(t *testing.T) {
    u := User{}
    s := structs.New(&u)
    f := s.Field("Base").Field("ID")
    err := f.Set(10)
    if err != nil {
        t.Errorf("Error %v", err)
    }
    if f.Value().(int) != 10 {
        t.Errorf("Value should be equal to 10, got %v", f.Value())
    }
}

// This test works fine
func TestEmbeddedWithReflect(t *testing.T) {
    u := User{}
    s := reflect.ValueOf(&u).Elem()
    f := s.FieldByName("Base").FieldByName("ID")
    if !f.CanSet() {
        t.Error("CanSet should be true")
    }
    f.Set(reflect.ValueOf(10))
    if f.Interface().(int) != 10 {
        t.Errorf("Value should be equal to 10, got %v", f.Interface())
    }
}

Latest version panics on interface{}

The following program will panic, because interface type in struct.
Older Versions supported it.

package main

import (
    "fmt"

    "github.com/fatih/structs"
)

type MyStruct struct {
    PublicKey interface{}
}

func main() {

    mystruct := MyStruct{PublicKey: []byte("test")}
    s := structs.Map(mystruct)
    fmt.Printf("s %+v\n", s)
}

Tried to structs.Map an x509 certificate from crypto/x509 and it panics.
Older Version, commit e5ca5fe, does not have this problem.
Panic trace shows structs.go:119, where the last commit was around.

what's the reasoning behind omitempty working differently from encoding/json?

Hi, love this library, thank you so much.

Question I was recently caught out by the way omit empty works in structs tags. For encoding/json if you pass a pointer it omits the field, but if you pass structs a nil it sets the value to null. It took me a bit to understand that structs was detecting uninitialized fields to decide whether to omit empty. Is there a rational behind this different from the encoding/json? Thanks

structs.Map return empty map

test := &commonlyUsedRedisTable{protocol: "test", status: "test"}
m := structs.Map(test)
fmt.Println(m)

   print: map[]

Is there any option to create structs from map?

What I need currently is to be able to convert map into a struct. Is it possible currently?

Proposal: Create a map and omit nil values

Hey,

would you accept a pull request for a method that creates a map while omitting keys with nil values?

If yes, how would you like the method to be called?

Cheers

structs.Values() does not parse time.Time field

The title is self explanatory. Here is also an example:

type MyTime struct {
Timestamp time.Time json:"timestamp"
}

structs.Names(obj) returns correctly the Timestamp field.
structs.Values(obj) does not return the Timestamp value.

Add tests for embedded structs

Get tag-value without omitempty

Excellent package! One question though: I sometimes need to get the tag() from a struct value, but then without the "omitempty" part. Does it make sense to create a dedicated method for that? So instead of func (f *Field) Tag(key string) string to have something like func (f *Field) TagWithoutOmitEmpty(key string) string?

Invalid Type

Hi @fatih

I have a struct like this:

type RStruct struct {
    Items []struct {
        Percentage  int         `json:"percentage"`
        Host             string      `json:"host"`
        Indices          interface{} `json:"indices"`
        IP               []string    `json:"ip"`
    } `json:"items"`
}

However, when I try to convert the returned the return object, I get an error:

Any thoughts?

Add ability to retrieve tag names from struct

I would like to build a SQL statement without using * using an empty struct. It would be nice if I could retrieve all the "tag names", similar to how Names() works. For example:

type Person struct {
  FirstName string `structs:"first_name"`
  LastName string `structs:"last_name"`
}

p := new(Person)

names := structs.Names(p)
// names = []string{"FirstName", "LastName"}

// not currently possible
tagNames := structs.TagNames(p)
// tagNames = []string{"first_name", "last_name"}

I would argue that this is acceptable in this library since it deals with a tag that this package manages. A temporary workaround is to export the struct to a map and extract the keys from that. (but it would be great if this library could remove that boilerplate)

Map() time.Time

time.Time is converted to an empty map.

"asis" field tag?

I have another issue. I have a struct with time.Time fields. This whole structure, once converted to a map, will get converted to JSON. However, if I use structure.Map, it'll also process thise time.Time fields. Is there any way we can instruct structure to keep those fields` values as is?