Coder Social home page Coder Social logo

mergo's Introduction

Mergo

GitHub release GoCard Test status OpenSSF Scorecard OpenSSF Best Practices Coverage status Sourcegraph FOSSA status

GoDoc Become my sponsor Tidelift

A helper to merge structs and maps in Golang. Useful for configuration default values, avoiding messy if-statements.

Mergo merges same-type structs and maps by setting default values in zero-value fields. Mergo won't merge unexported (private) fields. It will do recursively any exported one. It also won't merge structs inside maps (because they are not addressable using Go reflection).

Also a lovely comune (municipality) in the Province of Ancona in the Italian region of Marche.

Status

Mergo is stable and frozen, ready for production. Check a short list of the projects using at large scale it here.

No new features are accepted. They will be considered for a future v2 that improves the implementation and fixes bugs for corner cases.

Important notes

1.0.0

In 1.0.0 Mergo moves to a vanity URL dario.cat/mergo. No more v1 versions will be released.

If the vanity URL is causing issues in your project due to a dependency pulling Mergo - it isn't a direct dependency in your project - it is recommended to use replace to pin the version to the last one with the old import URL:

replace github.com/imdario/mergo => github.com/imdario/mergo v0.3.16

0.3.9

Please keep in mind that a problematic PR broke 0.3.9. I reverted it in 0.3.10, and I consider it stable but not bug-free. Also, this version adds support for go modules.

Keep in mind that in 0.3.2, Mergo changed Merge()and Map() signatures to support transformers. I added an optional/variadic argument so that it won't break the existing code.

If you were using Mergo before April 6th, 2015, please check your project works as intended after updating your local copy with go get -u dario.cat/mergo. I apologize for any issue caused by its previous behavior and any future bug that Mergo could cause in existing projects after the change (release 0.2.0).

Donations

If Mergo is useful to you, consider buying me a coffee, a beer, or making a monthly donation to allow me to keep building great free software. 😍

Donate using Liberapay Become my sponsor

Mergo in the wild

Mergo is used by thousands of projects, including:

Install

go get dario.cat/mergo

// use in your .go code
import (
    "dario.cat/mergo"
)

Usage

You can only merge same-type structs with exported fields initialized as zero value of their type and same-types maps. Mergo won't merge unexported (private) fields but will do recursively any exported one. It won't merge empty structs value as they are zero values too. Also, maps will be merged recursively except for structs inside maps (because they are not addressable using Go reflection).

if err := mergo.Merge(&dst, src); err != nil {
    // ...
}

Also, you can merge overwriting values using the transformer WithOverride.

if err := mergo.Merge(&dst, src, mergo.WithOverride); err != nil {
    // ...
}

If you need to override pointers, so the source pointer's value is assigned to the destination's pointer, you must use WithoutDereference:

package main

import (
	"fmt"

	"dario.cat/mergo"
)

type Foo struct {
	A *string
	B int64
}

func main() {
	first := "first"
	second := "second"
	src := Foo{
		A: &first,
		B: 2,
	}

	dest := Foo{
		A: &second,
		B: 1,
	}

	mergo.Merge(&dest, src, mergo.WithOverride, mergo.WithoutDereference)
}

Additionally, you can map a map[string]interface{} to a struct (and otherwise, from struct to map), following the same restrictions as in Merge(). Keys are capitalized to find each corresponding exported field.

if err := mergo.Map(&dst, srcMap); err != nil {
    // ...
}

Warning: if you map a struct to map, it won't do it recursively. Don't expect Mergo to map struct members of your struct as map[string]interface{}. They will be just assigned as values.

Here is a nice example:

package main

import (
	"fmt"
	"dario.cat/mergo"
)

type Foo struct {
	A string
	B int64
}

func main() {
	src := Foo{
		A: "one",
		B: 2,
	}
	dest := Foo{
		A: "two",
	}
	mergo.Merge(&dest, src)
	fmt.Println(dest)
	// Will print
	// {two 2}
}

Note: if test are failing due missing package, please execute:

go get gopkg.in/yaml.v3

Transformers

Transformers allow to merge specific types differently than in the default behavior. In other words, now you can customize how some types are merged. For example, time.Time is a struct; it doesn't have zero value but IsZero can return true because it has fields with zero value. How can we merge a non-zero time.Time?

package main

import (
	"fmt"
	"dario.cat/mergo"
    "reflect"
    "time"
)

type timeTransformer struct {
}

func (t timeTransformer) Transformer(typ reflect.Type) func(dst, src reflect.Value) error {
	if typ == reflect.TypeOf(time.Time{}) {
		return func(dst, src reflect.Value) error {
			if dst.CanSet() {
				isZero := dst.MethodByName("IsZero")
				result := isZero.Call([]reflect.Value{})
				if result[0].Bool() {
					dst.Set(src)
				}
			}
			return nil
		}
	}
	return nil
}

type Snapshot struct {
	Time time.Time
	// ...
}

func main() {
	src := Snapshot{time.Now()}
	dest := Snapshot{}
	mergo.Merge(&dest, src, mergo.WithTransformers(timeTransformer{}))
	fmt.Println(dest)
	// Will print
	// { 2018-01-12 01:15:00 +0000 UTC m=+0.000000001 }
}

Contact me

If I can help you, you have an idea or you are using Mergo in your projects, don't hesitate to drop me a line (or a pull request): @im_dario

About

Written by Dario Castañé.

License

BSD 3-Clause license, as Go language.

FOSSA Status

mergo's People

Contributors

abicky avatar andy-miracl avatar arnaudbriche avatar bionoren avatar darccio avatar dependabot[bot] avatar edofic avatar fossabot avatar ghouscht avatar github-actions[bot] avatar gwenaelp avatar hosszukalman avatar masahide avatar nbrownus avatar nl5887 avatar novas0x2a avatar peerxu avatar requilence avatar robstrong avatar silvin-lubecki avatar svyotov avatar tariq1890 avatar theobrigitte avatar timoreimann avatar umairidris avatar untoreh avatar vdemeester avatar vincentcr avatar yannh avatar zhiyu0729 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mergo's Issues

Invalid merge of two map[string]interface{}

Merge of

func main() {
	map1 := map[string]interface{}{
		"first": 1,
	}
	map2 := map[string]interface{}{
		"first": map[string]interface{}{
			"second" : 2,
		},
	}

	mergo.Merge(&map1, map2)

	fmt.Println(map1)
}

will returns

map[first:1]

Simulating patch-like functionality

Given the yamls like below:

default.yaml:

services:
  A:
    host: "google.com"
    path: "search"
    scheme: "https"
  B:
    host: "bing.com"
    path: "find"
logging:
  level: WARN

staging.yaml:

services:
  A:
    path: "search2"

I want this merged yaml:

services:
  A:
    host: "google.com"
    path: "search2"
    scheme: "https"
  B:
    host: "bing.com"
    path: "find"
logging:
  level: WARN

Is there a way to achieve this kind of behavior thru mergo? Currently, it converts services["A"].host and services["A"].scheme to nil in the merged object representation.

Map with time.Time

Hi,

I can't manage to copy time variables from a map to a struct :

type Bla struct {
    Name  string
    Birth time.Time
}
data := map[string]interface{}{
        "Name":  "Bla",
        "Birth": time.Now(),
}

b := Bla{}

err := mergo.Map(&b, data)
fmt.Println(err)
// nil

fmt.Println(b.Birth.IsZero())
// true

Am i missing something?

panic: 'reflect: call of reflect.Value.NumField...' at anonymous field

Panic with the following test code. 😇

//panic: reflect: call of reflect.Value.NumField on int64 Value

type testStruct struct {
    time.Duration
}

func TestMergePanicCase(t *testing.T) {
    to := testStruct{}
    from := testStruct{}
    if err := Merge(&to, from); err != nil {
        t.Fail()
    }
}
--- FAIL: TestMergePanicCase (0.00s)
panic: reflect: call of reflect.Value.NumField on int64 Value [recovered]
        panic: reflect: call of reflect.Value.NumField on int64 Value

goroutine 38 [running]:
testing.tRunner.func1(0xc4200de000)
        /usr/local/go1.9.2/src/testing/testing.go:711 +0x2d2
panic(0x55f500, 0xc42000b1c0)
        /usr/local/go1.9.2/src/runtime/panic.go:491 +0x283
reflect.flag.mustBe(0x186, 0x19)
        /usr/local/go1.9.2/src/reflect/value.go:201 +0xc8
reflect.Value.NumField(0x5733c0, 0xc420014f18, 0x186, 0x5785a0)
        /usr/local/go1.9.2/src/reflect/value.go:1176 +0x34
github.com/imdario/mergo.hasExportedField(0x5733c0, 0xc420014f18, 0x186, 0x0)
        /home/yamasaki_masahide/go/src/github.com/imdario/mergo/merge.go:16 +0x5a
github.com/imdario/mergo.hasExportedField(0x5785a0, 0xc420014f18, 0x199, 0xc42004deb8)
        /home/yamasaki_masahide/go/src/github.com/imdario/mergo/merge.go:19 +0x18a
github.com/imdario/mergo.deepMerge(0x5785a0, 0xc420014f18, 0x199, 0x5785a0, 0x6773e0, 0x99, 0xc42008de40, 0x0, 0x677300, 0x99, ...)
        /home/yamasaki_masahide/go/src/github.com/imdario/mergo/merge.go:49 +0xf68
github.com/imdario/mergo.merge(0x573540, 0xc420014f18, 0x5785a0, 0x6773e0, 0x578500, 0x15fe8d01, 0xc420014f18)
        /home/yamasaki_masahide/go/src/github.com/imdario/mergo/merge.go:166 +0x250
github.com/imdario/mergo.Merge(0x573540, 0xc420014f18, 0x5785a0, 0x6773e0, 0x46e176, 0x5a223bab)
        /home/yamasaki_masahide/go/src/github.com/imdario/mergo/merge.go:146 +0x4e
github.com/imdario/mergo.TestMergePanicCase(0xc4200de000)
        /home/yamasaki_masahide/go/src/github.com/imdario/mergo/mergo_test.go:674 +0x78
testing.tRunner(0xc4200de000, 0x596f80)
        /usr/local/go1.9.2/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
        /usr/local/go1.9.2/src/testing/testing.go:789 +0x2de
FAIL    github.com/imdario/mergo        0.007s

shell returned 1

Times are being overwritten even when src is empty

I'm using mergo with structs that can have empty times. I expected it not to overwrite times in destination when the value in source is empty (time.IsEmpty() returns true).

Sample failing test:

type structWithTime struct {
	Birth time.Time
}

func TestEmptyTime(t *testing.T) {
	now := time.Now()
	dst := structWithTime{now}
	src := structWithTime{}
	if err := MergeWithOverwrite(&dst, src); err != nil {
		t.FailNow()
	}
	if dst.Birth.IsZero() {
		t.Fatalf("time.Time should not have been overwritten: dst.Birth(%v) != now(%v)", dst.Birth, now)
	}
}

Output:

--- FAIL: TestEmptyTime (0.00s)
	mergo_test.go:630: time.Time should not have been overwritten: dst.Birth(0001-01-01 00:00:00 +0000 UTC) != now(2018-01-11 14:00:18.433651 -0200 -02 m=+0.002753105)

Is this expected behavior or can I go ahead fork & change it?

Merge slice with override no effect

package test5

import (
	"testing"

	"github.com/imdario/mergo"
)

type Strudent struct {
	Name  string
	Books []string
}

func TestSliceNotOverride(t *testing.T) {
	var s1 = Strudent{
		Name:  "Jack",
		Books: []string{"a", "B"},
	}

	var s2 = Strudent{
		Name:  "Tom",
		Books: []string{"1"},
	}

	err := mergo.Merge(&s2, s1, mergo.WithOverride)
	if err != nil {
		t.Log("merge error")
		t.Fail()
	}
	if len(s2.Books) != 2 {
		t.Log("slice not ovrride")
		t.Fail()
	}
}

I expect s2's Books are overrided by s1's Books, however the test failed.

Semantic versioning?

Hello,

I see you doing versioning, and that's awesome! But it's not semantic. Please consider adding v as a prefix.

For tools (like dep or vgo) to automatically pick-up release tags those need to be prefixed with v, like your latest release 0.3.2, should be v0.3.2. dep and vgo will pick those up automatically in that case.

Thanks!

struct with unexported fields

I came across an issue where struct with unexported fields are not merged.

A good example is type time.Time

package main

import (
	"fmt"
	"github.com/imdario/mergo"
	"time"
)

type Foo struct {
	A string
	B int64
	C time.Time
}

func main() {
	src := Foo{
		A: "one",
		C: time.Date(2009, time.November, 10, 23, 0, 0, 0, time.UTC),
	}

	dest := Foo{
		B: 2,
		C: time.Date(2010, time.December, 11, 23, 0, 0, 0, time.UTC),
	}

	mergo.MergeWithOverwrite(&dest, src)

	fmt.Println(dest)
	// Expected output
	// {one 2 2009-11-10 23:00:00 +0000 UTC}
	// Actual output
	// {one 2 2010-12-11 23:00:00 +0000 UTC}
}

MergeWithOverwrite nil pointer source overwrites non-nil destination

Thanks for creating and maintaining this package, it is a great help.

I found a bug with structs and nil pointers.
There was a check when the destination was nil but not vice versa.
So in an example where the destination has a pointer that represents a value, and the source is nil. The source should not overwrite the destination... but it currently does.

Here is a little command to reproduce the issue.

I will send a PR with the fix.

package main

import (
	"fmt"
	"github.com/imdario/mergo"
)

type simpleTest struct {
	Value int
}

type pointerTest struct {
	C *simpleTest
}

func main() {

	a := pointerTest{&simpleTest{19}}
	b := pointerTest{nil}

	fmt.Printf("a = %+v\n", a)
	fmt.Printf("b = %+v\n", b)

	if err := mergo.MergeWithOverwrite(&a, b); err != nil {
		fmt.Printf("error = %s", err)
	}
	fmt.Printf("a post merge = %+v\n", a)

	if a.C == nil {
		fmt.Printf("Expected a.C.Value = 19 but received a.C = nil")
	} else {
		fmt.Printf("Success! a.C.Value = %v", a.C.Value)
	}

}

Running ...

~/G/w/s/g/i/cmd ❯❯❯ go run main.go
a = {C:0x42119e1f0}
b = {C:<nil>}
a post merge = {C:<nil>}
Expected a.C.Value = 19 but received a.C = nil%

Merge With Override And AppendSlice fails for map

Hi @imdario

We merged two maps with Override and also AppendSlice , which is taking Override as high precedence than AppendSlice for slices too.

Expected behaviour: We want Override for non-slice and AppendSlice for slice. The following Test should PASS, but it's failing currently

func TestMergeWithOverrideAndAppendSlice(t *testing.T) {
	aMap := map[string]interface{}{
		"key1": []int{3},
		"key2": "valueA2",
	}

	bMap := map[string]interface{}{
		"key1": []int{1, 2},
		"key2": "valueB2",
		"key3": "valueB3",
	}

	expectedMap := map[string]interface{}{
		"key1": []int{1, 2, 3},
		"key2": "valueB2",
		"key3": "valueB3",
	}
	err := Merge(&aMap, bMap, WithAppendSlice, WithOverride)

	if err != nil {
		t.Errorf("Test failed with error: %v", err)
	}

	if !reflect.DeepEqual(aMap, expectedMap) {
		t.Errorf("Actual: %#v did not match \nExpected: %#v", aMap, expectedMap)
	}
}

We went through the code and found it's because of this part of code

https://github.com/imdario/mergo/blob/master/merge.go#L139

Could you explain the meaning for this code?

How to reset field to zero value?

Seems there is no way to reset a field to zero value, e.g. reset true with false, reset 1 with 0. Probably the lib shouldn't follow a pointer if it points to primitive types.

type ValueResetExample struct {
	A *bool
	B bool
	C *int
	D int
}

func valueReset() {
	bt := true
	bf := false
	i1 := 1
	i0 := 0
	dst := ValueResetExample{
		A: &bt,
		B: bt,
		C: &i1,
		D: i1,
	}
	src := ValueResetExample{
		A: &bf,
		B: bf,
		C: &i0,
		D: i0,
	}

	err := mergo.MergeWithOverwrite(&dst, src)
	if err != nil {
		panic(err)
	}
	// expected result: A: false, B: false, C: 0, D: 0
	fmt.Printf("A: %v, B: %v, C: %v, D: %v\n", *dst.A, dst.B, *dst.C, dst.D)

	err = mergo.MergeWithOverwrite(&src, dst)
	if err != nil {
		panic(err)
	}
	fmt.Printf("A: %v, B: %v, C: %v, D: %v\n", *src.A, src.B, *src.C, src.D)
}

Output

A: true, B: true, C: 1, D: 1
A: true, B: true, C: 1, D: 1

Errors reported have non-idiomatic naming

The global error vars, NilArgumentsErr, DifferentArgumentsTypesErr and NotSupportedErr should probably have names like ErrNilArguments, etc. This style of error naming is more consistent with other golang code.

Why is "You can only merge same-type structs"?

In our microservices we have a strict separation of transport and business logic. That means we implement an endpoint and a service. Both very often have very common request/response structures. So instead of writing all the struct mapping down and having very verbose code I would like to merge these structs dynamically. In the readme it says this.

You can only merge same-type structs ...

I am wondering for what reason this is. Based on my quick testing removing these lines works as expected with the following example.

package main

import (
	"fmt"

	"github.com/imdario/mergo"
)

type Thir struct {
	Val string
}

type Sec struct {
	F string
	X Thir
}

type Foo struct {
	A string
	B int64
	C Sec
	D string
	F string
}

type Bar struct {
	A string
	B int64
	C Sec
	E int
	F int
}

func main() {
	src := Foo{
		A: "one",
		C: Sec{
			F: "C.F",
			X: Thir{
				Val: "Foo.C.X.Val",
			},
		},
		D: "I am D",
		F: "I am F",
	}

	dest := Bar{
		A: "two",
		C: Sec{
			F: "Bar.C.F",
		},
		B: 2,
		E: 2,
		F: 2,
	}

	err := mergo.Merge(&dest, src)
	if err != nil {
		panic(err)
	}

	fmt.Println(dest)
	// Will print
	// {two 2 {Bar.C.F {Foo.C.X.Val}} 2 2}
}

Emptying dst slice

is this behavior correct?
I was expecting the dst.Data to be empty after the merge since I'm using the Overwrite option.

package main

import (
	"fmt"

	"github.com/imdario/mergo"
)

type My struct {
	Data []int
}

func main() {
	dst := My{Data: []int{1, 2, 3}}
	new := My{}

	mergo.MergeWithOverwrite(&dst, new)
	fmt.Println(dst)
}

Map results in surprising key case

Map() results in a map with keys that are surprising. For example A struct with member CIDR, its resulting key in the map is cDIR. A member with Name results in a key with name, less surprising but still odd. It's like the Map() call is taking the inverse of the first character always. Is this by design? I think this behavior is in conflict with the documentation: "Keys are capitalized to find each corresponding exported field". The godoc is closer to the behavior.

import (
  "fmt"
  "testing"
)

type Record struct {
  Data    map[string]interface{}
  Mapping map[string]string
}

func StructToRecord(in interface{}) *Record {
  rec := Record{}
  rec.Data = make(map[string]interface{})
  rec.Mapping = make(map[string]string)
  typ := reflect.TypeOf(in)
  for i := 0; i < typ.NumField(); i++ {
    field := typ.Field(i)
    dbFieldName := field.Tag.Get("db")

    fmt.Printf("%d %v, tags: %v\n", i, field.Name, dbFieldName)
    if dbFieldName != "" {
      rec.Mapping[field.Name] = dbFieldName
    }
  }

  mergo.Map(&rec.Data, in)
  return &rec
}

func TestStructToRecord(t *testing.T) {

  type A struct {
    Name string `json:"name" db:"name"`
    CIDR string `json:"cidr" db:"cidr"`
  }

  type Record struct {
    Data    map[string]interface{}
    Mapping map[string]string
  }

  a := A{Name: "David", CIDR: "10.0.0.0/8"}
  rec := StructToRecord(a)
  fmt.Printf("rec: %+v\n", rec)
  if len(rec.Mapping) < 2 {
    t.Fatalf("struct to record failed, no mapping, struct missing tags?, rec: %+v, a: %+v ", rec, a)
  }
}

Output:

go test -v
=== RUN   TestStructToRecord
0 Name, tags: name
1 CIDR, tags: cidr
rec: &{Data:map[name:David cIDR:10.0.0.0/8] Mapping:map[Name:name CIDR:cidr]}
--- PASS: TestStructToRecord (0.00s)
PASS
ok             0.003s

Merging maps within structs

Considering this example

type Test struct {
      A map[string]string
      B string
      C string

}

func test() {
      foo := new(Test)
      foo.B = "one"

      bar := new(Test)
      bar.A = map[string]string{"biz": "baz"}
      bar.B = "two"
      bar.C = "three"

      mergo.Merge(foo, bar)
}

What am I doing wrong to be getting this error?

Adding This line solves the problem and merges happen as expected. But its not ideal.

Foo.A = map[string]string{}

panic: assignment to entry in nil map

goroutine 1 [running]:
reflect.Value.SetMapIndex(0x8123380, 0x184382a8, 0xd5, 0x8122d80, 0x1840a178, 0x58, 0x8122d80, 0x1840a180, 0x58)
        /usr/local/go/src/reflect/value.go:1440 +0x21a
github.com/imdario/mergo.deepMerge(0x8123380, 0x184382a8, 0xd5, 0x8123380, 0x184382b8, 0xd5, 0x18436480, 0x1, 0x0, 0x0)
        /home/vagrant/go/src/github.com/imdario/mergo/merge.go:56 +0x7b3
github.com/imdario/mergo.deepMerge(0x813e540, 0x184382a0, 0xd9, 0x813e540, 0x184382b0, 0xd9, 0x18436480, 0x0, 0x0, 0x0)
        /home/vagrant/go/src/github.com/imdario/mergo/merge.go:36 +0x9a5
github.com/imdario/mergo.Merge(0x811b080, 0x184382a0, 0x811b080, 0x184382b0, 0x0, 0x0)
        /home/vagrant/go/src/github.com/imdario/mergo/merge.go:96 +0x1bb
main.test()
        /home/vagrant/go/src/github.com/Jmeyering/zealot/main.go:38 +0x12e
main.main()
        /home/vagrant/go/src/github.com/Jmeyering/zealot/main.go:41 +0x28

goroutine 2 [runnable]:
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:90
runtime.goexit()
        /usr/local/go/src/runtime/asm_386.s:2287 +0x1

goroutine 3 [runnable]:
runtime.bgsweep()
        /usr/local/go/src/runtime/mgc0.go:82
runtime.goexit()
        /usr/local/go/src/runtime/asm_386.s:2287 +0x1

goroutine 4 [runnable]:
runtime.runfinq()
        /usr/local/go/src/runtime/malloc.go:712
runtime.goexit()
        /usr/local/go/src/runtime/asm_386.s:2287 +0x1
exit status 2

Issue with type based on time.Time

I have a Date type:

type Date time.Time

The type is used for date-only JSON representation. I AM using a pointer to Date:

type Document struct {
   Created    *Date
}

and MergeWithOverwrite does not work unfortunately.

Could you help with this?

Inconsistent merge behavior

The first test, when merging maps with succeed, while the second, where the same maps are being merged as fields to structs it will fail.
I am not sure if this is intended behavior or not, IMO this is a bug.

package mergo

import (
	"testing"

	"github.com/magiconair/properties/assert"
)

func TestMergoSimpleMap(t *testing.T) {
	dst := map[string]string{"key1": "loosethis", "key2": "keepthis"}
	src := map[string]string{"key1": "key10"}
	exp := map[string]string{"key1": "key10", "key2": "keepthis"}
	Merge(&dst, src, WithAppendSlice, WithOverride)
	assert.Equal(t, dst, exp)
}

type CustomStruct struct {
	SomeMap map[string]string
}

func TestMergoComplexStructMap(t *testing.T) {
	dst := map[string]CustomStruct{
		"Normal": CustomStruct{SomeMap: map[string]string{"key1": "loosethis", "key2": "keepthis"}},
	}
	src := map[string]CustomStruct{
		"Normal": CustomStruct{SomeMap: map[string]string{"key1": "key10"}},
	}
	exp := map[string]CustomStruct{
		"Normal": CustomStruct{SomeMap: map[string]string{"key1": "key10", "key2": "keepthis"}},
	}
	Merge(&dst, src, WithAppendSlice, WithOverride)
	assert.Equal(t, dst, exp)
}

Fails to merge when using a pointer to a struct

I don't seem to find out why the tests pass in stuff like complexTest but fails to merge correctly stuff like the following:

package main

import (
    "fmt"
    "github.com/imdario/mergo"
)

type Person struct {
    ID      int
    Name    string
    Address string
}

var defaultPerson = Person{
    ID:      1,
    Name:    "Homer",
    Address: "742 Evergreen Terrace",
}

func main() {
    a := Person{Name: `John`}
    if err := mergo.Merge(&a, defaultPerson); err != nil {
        fmt.Printf("ERR: %+v", err)
    }

    fmt.Printf("A is: %+v\n", a)
    // Prints A is: {ID:1 Name:Homer Address:742 Evergreen Terrace}
    // should print A is: {ID:1 Name:John Address:742 Evergreen Terrace}
}

Merge is broken for structs with slices

At some point after 6633656 (the version currently vendored by kubernetes, which works fine), merging broke for structs containing byteslices.

Here's a simplificed repro

package main

import (
        "fmt"
        "github.com/imdario/mergo"
)

type Foo struct {
        Str    string
        Bslice []byte
}

func main() {
        dest := Foo{Str: "a"}

        toMerge := Foo{
                Str:    "b",
                Bslice: []byte{1, 2},
        }

        mergo.Merge(&dest, toMerge)

        fmt.Println(dest)
        // Should print
        // {"b" [1,2]}
        // However, it prints
        // {"a" [1,2]}
}

Add benchmarks

to make we keep improving performance… or at least we don't degrade it 👼

panic: runtime error: invalid memory address or nil pointer dereference [recovered]

I'm getting an error

panic: runtime error: invalid memory address or nil pointer dereference [recovered]
        panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0xa0 pc=0x535086]

goroutine 24 [running]:
testing.tRunner.func1(0xc82012e510)
        /usr/local/go/src/testing/testing.go:450 +0x171
github.com/imdario/mergo.deepMerge(0x8057e0, 0xc82014f5c0, 0x15, 0x8057e0, 0xc82014f050, 0x15, 0xc820051b28, 0x2, 0x1, 0x0, ...)
        github.com/imdario/mergo/merge.go:49 +0x836
github.com/imdario/mergo.deepMerge(0x802ea0, 0xc820168e10, 0x54, 0x802ea0, 0xc820168e00, 0x54, 0xc8200b3b18, 0x1, 0x1, 0x0, ...)
        github.com/imdario/mergo/merge.go:73 +0x556
github.com/imdario/mergo.deepMerge(0x8057e0, 0xc8200e93f0, 0xd5, 0x8057e0, 0xc82014f170, 0x15, 0xc8200b3b18, 0x0, 0xc82014f101, 0x0, ...)
        github.com/imdario/mergo/merge.go:53 +0x8c9
github.com/imdario/mergo.merge(0x7e6040, 0xc8200e93f0, 0x8057e0, 0xc82014f170, 0xc820134d01, 0x0, 0x0)
        github.com/imdario/mergo/merge.go:109 +0x2cd
github.com/imdario/mergo.MergeWithOverwrite(0x7e6040, 0xc8200e93f0, 0x8057e0, 0xc82014f170, 0x0, 0x0)
        github.com/imdario/mergo/merge.go:95 +0x50

Map() map[string]interface{} -> struct, overwrite not working for zero values

package main

import (
	"fmt"
	"github.com/imdario/mergo"
)

type Foo struct {
	A int
	B int
	C int
}

func main() {
	dst := Foo{A: 0, B: 1, C: 2}

	src := map[string]interface{}{
		"A": 3, "B": 4, "C": 0,
	}

	mergo.Map(&dst, src, mergo.WithOverride)
	fmt.Printf("%+v\n", dst)
}

Expected result:
{A:3 B:4 C:0}

Actual result:
{A:3 B:4 C:2}

Looks like Overwrite ignores Value if it is zero.
But it must overwrite struct field in case map have the corresponding key set. Regardless of it's value

merging structs with opiton "WithOverride" not consistent

Hi,
In my opinion merging nested structs isn't really consistent when using the option "WithOverride". If you nest a struct "by value" or "by pointer" the behaviour of merge with override is different. (Possibly a duplicate of #14)
Example Code:

package main
import (
        "fmt"
        "log"

        "github.com/imdario/mergo"
)

type PtrTest struct {
        Str    string
        Number int
        Nested *Nested
}

type Test struct {
        Str    string
        Number int
        Nested Nested
}

type Nested struct {
        NestedStr    string
        NestedNumber int
}

func main() {
        dstPtr := &PtrTest{
                Str: "This is a non empty test string",
                Nested: &Nested{
                        NestedStr:    "This is a non empty string in a nested struct",
                        NestedNumber: 42,
                },
        }

        srcPtr := &PtrTest{
                Str:    "This is also a string",
                Number: 4,
                Nested: &Nested{},
        }

        if err := mergo.Merge(dstPtr, srcPtr, mergo.WithOverride); err != nil {
                log.Fatal(err)
        }
        fmt.Printf("Result with 'Nested' as pointer: %v\n", dstPtr.Nested)

        dst := &Test{
                Str: "This is a non empty test string",
                Nested: Nested{
                        NestedStr:    "This is a non empty string in a nested struct",
                        NestedNumber: 42,
                },
        }

        src := &Test{
                Str:    "This is also a string",
                Number: 4,
                Nested: Nested{},
        }

        if err := mergo.Merge(dst, src, mergo.WithOverride); err != nil {
                log.Fatal(err)
        }
        fmt.Printf("Result with 'Nested' as value: %v\n", dst.Nested)
}

This code prints the following two lines:

Result with 'Nested' as pointer: &{ 0}
Result with 'Nested' as value: {This is a non empty string in a nested struct 42}

Quote from the documentation: WithOverride will make merge override non-empty dst attributes with non-empty src attributes values.

I expected that behaviour also from nested structs (by pointer). Maybe this is just a misunderstanding of mine, because in my example the pointer to the nested struct isn't empty, but the element to which the pointer points is.
Actually if you remove the pointer to the empty element the example works like I'd expected it. However I got a use case which this solution isn't really applicable 😃.

Without looking at your code I think it should be possible to make a consistent use of "WithOverride" possible (also with my example). However I think it won't be very easy and maybe adds a bit of complexity.
If there is interest I can take a look at it and try to implement it. If not I think it should at least be documented for other people.

Please let me know your opinion about that.
Thanks for your work and regards,
Thomas

Inconsistent behaviour when merging maps

I'm pasting a test case that shows inconsistent behaviour when merging maps. I hope it's clear enough. This problem appeared with commit 65847ea that fixed issue #23.

type typeA struct {
    A *typeB
}

type typeB struct {
    B map[string]typeC
}

type typeC struct{ C string }

func TestInconsistentMergeBehaviourOnMaps(t *testing.T) {
    dst := typeA{
        A: &typeB{
            B: map[string]typeC{"C": typeC{"value"}},
        },
    }
    src := typeA{
        A: &typeB{
            B: map[string]typeC{},
        },
    }
    MergeWithOverwrite(&dst.A.B, src.A.B)
    if dst.A.B["C"].C != "value" {
        t.Error("Bad merge (1)") // No problem here.
    }
    MergeWithOverwrite(&dst, src)
    if dst.A.B["C"].C != "value" {
        t.Error("Bad merge (2)") // Problem here.
    }
}

bool custom Transformer for non zero-value false

hi @iamdario,

I recently introduced mergo on one of our projects.

I came up with an issue very similar to #24 :

import (
  "testing"
  "github.com/imdario/mergo"
  "github.com/stretchr/testify/assert"
)
func TestBoolean(t *testing.T) {
  type Foo struct {
    Bar bool `json:"bar"`
  }

  f1 := Foo{Bar: true}
  f2 := Foo{Bar: false}

  mergo.Merge(&f2, f1)
  assert.False(t, f2.Bar)
}

Do you think it could be handled with some custom transformers?

Merge() doesn't work with two pointer values

I was looking at this package and noticed that it will fail if both parameters to Merge() are pointers to structs. One could ofc dereference src, but should it be necessary?

To illustrate:

a := &simpleTest{}
b := &simpleTest{42}
if err := Merge(a, b); err != nil {
    // boom
}

Bug: Map fails when struct has an interface member

mergo panics when the following code is run:

type S struct {
	Member interface{}
}

func main(){
	m := make(map[string]interface{})
	m["Member"] = "anything"

	st := &S{}
	mergo.Map(st, m)

	fmt.Printf("%+v", st)
}

producing panic: reflect: call of reflect.Value.IsNil on string Value. This panic occurs if any interface type at all is specified for Member

Issue with boolean type (bool)

If struct has a boolean fields, merge result is prefer true value, but this is not correct.
Example:

package main

import (
    "fmt"
    "github.com/imdario/mergo"
)

type Foo struct {
    A string
    B int64
    C bool
}

func main() {
    src := Foo{
        A: "one",
        C: true,
    }

    dest := Foo{
        A: "two",
        B: 2,
        C: false,
    }

    mergo.Merge(&dest, src)

    fmt.Println(dest)
    // Will print
    // {two 2 true}
    // but expected result is {two 2 false}
}

panic: reflect: reflect.Value.Set using unaddressable value merging nested structures

Hi,

I want to do a deep merge where non-empty dst attributes are overwritten with non-empty src attributes values but slices are appended, is this possible?

Can someone help me understand why this does't work?

package main

import (
        "fmt"

        "github.com/imdario/mergo"
)

type My struct {
        Name  string
        Data  []int
        Other map[string]Other
}

type Other struct {
        Foo string
        Bar int
        Baz []string
        Faz map[string]string
}

func main() {
        dst := My{
                Data: []int{1, 2, 3},
                Name: "Dest",
                Other: map[string]Other{
                        "stuff": Other{
                                Foo: "bar",
                                Bar: 0,
                        },
                },
        }

        new := My{
                Data: []int{4, 5},
                Name: "Source",
                Other: map[string]Other{
                        "stuff": Other{
                                Foo: "bar",
                                Bar: 1,
                        },
                        "Morestuff": Other{
                                Foo: "foo",
                                Bar: 2,
                        },
                },
        }

        new2 := My{
                Name: "Source2",
                Other: map[string]Other{
                        "stuff": Other{
                                Foo: "bar",
                                Bar: 1,
                        },
                        "Morestuff": Other{
                                Foo: "foo",
                                Bar: 2,
                                Baz: []string{"something", "wrong"},
                                Faz: map[string]string{
                                        "some": "some2",
                                },
                        },
                },
        }

        fmt.Println(dst)
        fmt.Println(new)
        fmt.Println(new2)
        mergo.Merge(&dst, &new, mergo.WithAppendSlice, mergo.WithOverride)
        fmt.Println(dst)
        mergo.Merge(&dst, &new2, mergo.WithAppendSlice, mergo.WithOverride)
        fmt.Println(dst)
}

I get:

$ go run test.go
{Dest [1 2 3] map[stuff:{bar 0 [] map[]}]}
{Source [4 5] map[Morestuff:{foo 2 [] map[]} stuff:{bar 1 [] map[]}]}
{Source2 [] map[stuff:{bar 1 [] map[]} Morestuff:{foo 2 [something wrong] map[some:some2]}]}
{Source [1 2 3 4 5] map[Morestuff:{foo 2 [] map[]} stuff:{bar 1 [] map[]}]}
panic: reflect: reflect.Value.Set using unaddressable value

goroutine 1 [running]:
reflect.flag.mustBeAssignable(0x95)
        /usr/local/Cellar/go/1.11.1/libexec/src/reflect/value.go:234 +0x157
reflect.Value.Set(0x10b0760, 0xc000022470, 0x95, 0x10b0760, 0xc000074480, 0x15)
        /usr/local/Cellar/go/1.11.1/libexec/src/reflect/value.go:1397 +0x2f
github.com/imdario/mergo.deepMerge(0x10b0760, 0xc000022470, 0x95, 0x10b0760, 0xc000022430, 0x95, 0xc000095b08, 0x3, 0xc00000a1c0, 0x0, ...)
        /Users/ldavim/go/src/github.com/imdario/mergo/merge.go:83 +0x1388
github.com/imdario/mergo.deepMerge(0x10bcde0, 0xc000022440, 0x99, 0x10bcde0, 0xc000022400, 0x99, 0xc000095b08, 0x2, 0xc00000a1c0, 0x0, ...)
        /Users/ldavim/go/src/github.com/imdario/mergo/merge.go:72 +0x23a7
github.com/imdario/mergo.deepMerge(0x10b0700, 0xc0000741a8, 0x195, 0x10b0700, 0xc000074268, 0x195, 0xc000095b08, 0x1, 0xc00000a1c0, 0x0, ...)
        /Users/ldavim/go/src/github.com/imdario/mergo/merge.go:115 +0x986
github.com/imdario/mergo.deepMerge(0x10b9ba0, 0xc000074180, 0x199, 0x10b9ba0, 0xc000074240, 0x199, 0xc000095b08, 0x0, 0xc00000a1c0, 0x199, ...)
        /Users/ldavim/go/src/github.com/imdario/mergo/merge.go:72 +0x23a7
github.com/imdario/mergo.merge(0x10a4780, 0xc000074180, 0x10a4780, 0xc000074240, 0xc000095ce8, 0x2, 0x2, 0x0, 0x0)
        /Users/ldavim/go/src/github.com/imdario/mergo/merge.go:251 +0x29f
github.com/imdario/mergo.Merge(0x10a4780, 0xc000074180, 0x10a4780, 0xc000074240, 0xc000095ce8, 0x2, 0x2, 0x0, 0x0)
        /Users/ldavim/go/src/github.com/imdario/mergo/merge.go:206 +0x71
main.main()
        /Users/ldavim/Desktop/test.go:72 +0xa28
exit status 2

So, the first merge works but the second one fails.

Thanks

Fail on conflict

I can see that mergo merges when the destination is non-zero and that there is also a forced overwrite option, but I have a use case where I want to report an error (and abort the merge), if an attempt is made to merge a field that already is non-zero in the destination object. I'd like to submit a PR to support this behaviour. Do you agree that this is useful?

Slice and Maps are handled differently

Say you have something similar to the example you provided in the README , but instead deal with maps and slices, giving you something like this:

package main

import (
	"fmt"
	"github.com/imdario/mergo"
)

type Foo struct {
	A map[string]string
	B []string
}

func main() {
	src := Foo{
		A: make(map[string]string),
		B: []string{"something"},
	}
	src.A["first"] = "1"

	dest := Foo{
		A: make(map[string]string),
		B: []string{"else"},
	}
	dest.A["second"] = "2"
	fmt.Println(src)
	// Will print
	// {map[first:1] [something]}
	fmt.Println(dest)
	// Will print
	// {map[second:2] [else]}

	mergo.Merge(&dest, src)

	fmt.Println(dest)
	// Will print
	// {map[second:2 first:1] [else]}
	// Expected print
	// {map[second:2 first:1] [something else]}
}

Why is it that the behavior of Merge is to append to maps but completely over-write the slice?

MergeWithOverwrite behavior on struct field inside a map

Hi,

I would expect that fields from a struct on a map would only be overwritten by MergeWithOverwrite if the src struct field was not empty valued, but according to the following test, it seems to be overwriting anyway on map to struct and map to struct pointer.

package mergo_test

import "testing"
import "github.com/imdario/mergo"

type Struct struct {
    Value string
}
type StructOfStruct struct {
    Value Struct
}

type MapToStructPtr map[string]*Struct
type MapToStruct map[string]Struct

func TestMapToStructPtr(t *testing.T) {
    var dst = make(MapToStructPtr)
    dst["a"] = &Struct{Value: "1"}
    var src = make(MapToStructPtr)
    src["a"] = &Struct{}
    var expect = make(MapToStructPtr)
    expect["a"] = &Struct{Value: "1"}

    if err := mergo.MergeWithOverwrite(&dst, src); err != nil {
        t.FailNow()
    }

    for k, _ := range expect {
        if ((expect[k] == nil) != (dst[k] == nil)) || (expect[k] != nil && dst[k].Value != expect[k].Value) {
            t.Errorf("Test failed:\ngot  :\n dst[\"%s\"] = %#v\n\nwant :\n dst[\"%s\"] = %#v\n\n", k, dst[k], k, expect[k])
        }
    }
}

func TestMapToStruct(t *testing.T) {
    var dst = make(MapToStruct)
    dst["a"] = Struct{Value: "1"}
    var src = make(MapToStruct)
    src["a"] = Struct{}
    var expect = make(MapToStruct)
    expect["a"] = Struct{Value: "1"}

    if err := mergo.MergeWithOverwrite(&dst, src); err != nil {
        t.FailNow()
    }

    for k, _ := range expect {
        if dst[k].Value != expect[k].Value {
            t.Errorf("Test failed:\ngot  :\n dst[\"%s\"] = %#v\n\nwant :\n dst[\"%s\"] = %#v\n\n", k, dst[k], k, expect[k])
        }
    }
}

func TestStruct(t *testing.T) {
    var dst = Struct{Value: "1"}
    var src = Struct{}
    var expect = Struct{Value: "1"}

    if err := mergo.MergeWithOverwrite(&dst, src); err != nil {
        t.FailNow()
    }

    if dst.Value != expect.Value {
        t.Errorf("Test failed:\ngot  :\n dst = %#v\n\nwant :\n dst = %#v\n\n", dst, expect)
    }
}

func TestStructOfStruct(t *testing.T) {
    var dst = StructOfStruct{Value: Struct{"1"}}
    var src = StructOfStruct{}
    var expect = StructOfStruct{Value: Struct{"1"}}

    if err := mergo.MergeWithOverwrite(&dst, src); err != nil {
        t.FailNow()
    }

    if dst.Value != expect.Value {
        t.Errorf("Test failed:\ngot  :\n dst = %#v\n\nwant :\n dst = %#v\n\n", dst, expect)
    }
}

go test -v mergo_test.go gives me the following output

=== RUN TestMapToStructPtr
--- FAIL: TestMapToStructPtr (0.00s)
        mergo_test.go:31: Test failed:
                got  :
                 dst["a"] = &mergo_test.Struct{Value:""}

                want :
                 dst["a"] = &mergo_test.Struct{Value:"1"}

=== RUN TestMapToStruct
--- FAIL: TestMapToStruct (0.00s)
        mergo_test.go:50: Test failed:
                got  :
                 dst["a"] = mergo_test.Struct{Value:""}

                want :
                 dst["a"] = mergo_test.Struct{Value:"1"}

=== RUN TestStruct
--- PASS: TestStruct (0.00s)
=== RUN TestStructOfStruct
--- PASS: TestStructOfStruct (0.00s)
FAIL
exit status 1
FAIL    command-line-arguments  0.118s

It seems to work the way I expected for struct and struct of struct though.

Unclear documentation

I find the description of the Merge method quite unclear: after reading, I thought that zero-value fields of the destination were overridden by the matching field of the source. I had to dig in the source code and the tests to understand it worked the other way around.

It may have to do with me being tired, but I think that providing a simple and visual example that can be copy-pasted for demonstration would be a nice thing to have in the readme. For example:

package main

import (
    "fmt"
    "github.com/imdario/mergo"
)

type Foo struct {
    A string
    B int64
}

func main() {
    src := Foo{
        A: "one",
    }

    dest := Foo{
        A: "two",
        B: 2,
    }

    mergo.Merge(&dest, src)

    fmt.Println(dest)
    // Will print
    // {one 2}
}

Mapping into struct with pointer values

Hi,
it would be nice if Map/MapWithOverwrite could handle nested pointer values, just like Merge does. For example:

package main

import (
    "fmt"

    "github.com/imdario/mergo"
)

type OuterStruct struct {
    Inner *InnerStruct
}

type InnerStruct struct {
    Field string
}

func main() {
    s := OuterStruct{&InnerStruct{"old"}}

    m := map[string]interface{}{
        "inner": map[string]interface{}{
            "field": "new",
        },
    }

    fmt.Println("before", s.Inner)

    err := mergo.MapWithOverwrite(&s, m)
    if err != nil {
        fmt.Errorf(err.Error())
    }

    fmt.Println("after", s.Inner)
}

results:

before &{old}
after &{old}

instead of:

before &{old}
after &{new}

Edit: Added desired results example

Make a release/tag

Hi there,

It's been a while since the latest release (Feb 2016) and updates have been done since.
To make dependency management easier for applications it would be great if mergo can make a new tag/release for changes.

What do you think?

Merging slices WithAppendSlice produces different merge order than expected

For input (the example from merge_appendslice_test.go)

S1: = Student{"Tom",[]string{"a", "B"}}
S2:= Student{"Tom", []string{"1"}},
err := Merge(&data.S2, data.S1, WithOverride, WithAppendSlice)

I get different output that what I would expect

Exp: []string{"a", "B", "1"}
Act: []string{ "1", "a", "B"}

Since we are merging S2 into S2, the order of the elements I would expect would be fist the elements of the slice of S1 and then the elements of the slice of S2.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.