mitchellh / protoc-gen-go-json

Protobuf compiler plugin to generate Go JSON Marshal/Unmarshal implementations for messages using jsonpb.

protoc-gen-go-json

This is a plugin for the Google Protocol Buffers compiler protoc that generates code to implement json.Marshaler and json.Unmarshaler using protojson.

This enables Go-generated protobuf messages to be embedded directly within other structs and encoded with the standard JSON library, since the standard encoding/json library can't encode certain protobuf messages such as those that contain oneof fields.

Install

go get github.com/mitchellh/protoc-gen-go-json

Also required:

• protoc
• protoc-gen-go

Usage

Define your messages like normal:

syntax = "proto3";

message Request {
  oneof kind {
    string name = 1;
    int32  code = 2;
  }
}

The example message purposely uses a oneof since this won't work by default with encoding/json. Next, generate the code:

protoc --go_out=. --go-json_out=. request.proto

Your output should contain a file request.pb.json.go which contains the implementation of json.Marshal/Unmarshal for all your message types. You can then encode your messages using standard encoding/json:

import "encoding/json"

// Marshal
bs, err := json.Marshal(&Request{
  Kind: &Kind_Name{
    Name: "alice",
  },
}

// Unmarshal
var result Request
json.Unmarshal(bs, &result)

Options

The generator supports options you can specify via the command-line:

• enums_as_ints={bool} - Render enums as integers instead of strings.
• emit_defaults={bool} - Render fields with zero values.
• orig_name={bool} - Use original (.proto file) name for fields.
• allow_unknown={bool} - Allow messages to contain unknown fields when unmarshaling

It also includes the "standard" options available to all protogen-based plugins:

• import_path={path} - Override the import path
• paths=source_relative - Derive the output path from the input path
• etc.

These can be set as part of the --go-json_out value:

protoc --go-json_opt=emit_defaults=true:.

You can specify multiple using a ,:

protoc --go-json_out=enums_as_ints=true,emit_defaults=true:.

Alternatively, you may also specify options using the --go-json_opt value:

protoc --go-json_out:. --go-json_opt=emit_defaults=true,enums_as_ints=true

protoc-gen-go-json's Issues

Maybe we should also support: `generator protoc-gen-go-json hasn't been updated to support optional fields in proto3`

Maybe we should also support: generator protoc-gen-go-json hasn't been updated to support optional fields in proto3

Originally posted by @elvizlai in #9 (comment)

Some unknown struct methods are generated for map<string, HeaderValue>

Hi, thanks for the generator!

message Request {
  string body     = 1;
  map<string, HeaderValue> header = 2;
  string rawQuery = 3;
  string method   = 4;
}


message HeaderValue {
  repeated string values = 1;
}

For some reason generates additional unknown struct marshalers:

// MarshalJSON implements json.Marshaler
func (msg *Request_HeaderEntry) MarshalJSON() ([]byte, error) {
	var buf bytes.Buffer
	err := (&jsonpb.Marshaler{
		EnumsAsInts:  false,
		EmitDefaults: false,
		OrigName:     false,
	}).Marshal(&buf, msg)
	return buf.Bytes(), err
}

// UnmarshalJSON implements json.Unmarshaler
func (msg *Request_HeaderEntry) UnmarshalJSON(b []byte) error {
	return jsonpb.Unmarshal(bytes.NewReader(b), msg)
}

One workaround is probably just to delete them manually.. :D Any ideas why is it doing this?

Maintain comments for licenses and build tags

On generated .pb.go files, comments at the beginning of the .proto file are preserved. For example, assume a .proto file that started with:

/*
Copyright © 2020 Alessandro Segala (@ItalyPaleAle)

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, version 3 of the License.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.
*/

// +build linux

syntax = "proto3";

package foo;

// < rest of file >

When compiling it and using the json plugin, I get two files.

The .pb.go file preserves the comments, so it starts with:

//
//Copyright © 2020 Alessandro Segala (@ItalyPaleAle)
//
//This program is free software: you can redistribute it and/or modify
//it under the terms of the GNU Affero General Public License as published
//by the Free Software Foundation, version 3 of the License.
//
//This program is distributed in the hope that it will be useful,
//but WITHOUT ANY WARRANTY; without even the implied warranty of
//MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
//GNU Affero General Public License for more details.
//
//You should have received a copy of the GNU Affero General Public License
//along with this program.  If not, see <https://www.gnu.org/licenses/>.

// +build linux

// Code generated by protoc-gen-go. DO NOT EDIT.
// versions:
// 	protoc-gen-go v1.25.0-devel
// 	protoc        v3.13.0
// source: foo.proto

package foo

// < rest of file >

However the .pb.json.go file starts with this only:

// Code generated by protoc-gen-go-json. DO NOT EDIT.
// source: foo.proto

package foo

// < rest of file >

As you can see, there are two important things missing:

1. The license
2. Optional build tags such as // +build linux

Please publish a release

Could you please publish a release, so this can be used with gobinaries.com ? Makes using this in a CI server much faster.

Thanks!

MarshalJSON not called in some cases

The generated MarshalJSON() implementation currently has a pointer receiver:

func (msg *Foo) MarshalJSON() ([]byte, error) {
   //...
}

Generally this is fine, as protobufs aren't value-copyable and are almost always used in a pointer context, but if it's not a value receiver, there are subtle places where the json.Marshaler function won't be called properly.

For example:

type EmbedFoo struct {
   Foo
   TimeToLive int `json:"ttl"`
}

func main() {
   foo := Foo{}
   _, _ = json.Marshal(foo)     // FAIL, Does not call MarshalJSON
   _, _ = json.Marshal(&foo)    // OK

   foo2 := EmbedFoo{}
   _, _ = json.Marshal(foo2)    // FAIL, Does not call MarshalJSON
   _, _ = json.Marshal(&foo2)   // FAIL, Does not call MarshalJSON
}

Changing the generated MarshalJSON to have value receiver fixes it (due to the pointer-following logic that json.Marshal() performs):

func (msg Foo) MarshalJSON() ([]byte, error) {
   // ...
}

This would also mirror its common uses within Go itself (including the example in the docs):

func (a *Animal) UnmarshalJSON(b []byte) error {
   // ...
}

func (a Animal) MarshalJSON() ([]byte, error) {
   // ...
}

Note: This only applies to MarshalJSON. UnmarshalJSON should continue to have the pointer receiver.

Added AllowUnknownFields to jsonpb.Unmarshaler

Support EmitDefaultValues

Currently the option emit_defaults maps to EmitUnpopulated. But there is a new option EmitDefaultValues, see protojson#MarshalOptions

Any chance to add this? I would suggest option emit_defaults_without_null or similar

[Feature Request] Generated files should respect go_package option

The default behaviour of the Go protoc plugin is to generate the code in a folder equal to the one declared in the go_package option. For example, I would expect the following proto to be generated in ./github.com/mitchellh/protoc-gen-go-json/e2e/*:

syntax = "proto3";
package e2e;
option go_package = "github.com/mitchellh/protoc-gen-go-json/e2e;e2e";
...

We are generating dozens of files and we heavily rely on this functionality to automatically split the generated Go files into packages. It would be helpful to have that option in this generator as well. Are there any plans to support it?

Move to google.golang.org/protobuf/encoding/protojson

With github.com/golang/protobuf/jsonpb being deprecated this repository should probably instead (or optionally) target google.golang.org/protobuf/encoding/protojson.

Publish new tag

Any chance we could get a new tag, maybe v1.2.0?

There's been a few notable changes, 2263edb...4990573

ERROR IN V1.1.0

when i update v1.1.0

i got this

protoc-gen-go-json: parse error --go-json_out: protoc-gen-go-json: Plugin failed with status code 1. make: *** [gen-grpc] Error 1