Coder Social home page Coder Social logo

protopatch's Introduction

protopatch

go.dev reference build status

Patch protoc plugin output with Go-specific features. The protoc-gen-go-patch command wraps calls to Go code generators like protoc-gen-go or protoc-gen-go-grpc and patches the Go syntax before being written to disk.

Install

go install github.com/alta/protopatch/cmd/protoc-gen-go-patch

Usage

After installing protoc-gen-go-patch, use it by specifying it with a --go-patch_out=... argument to protoc:

protoc \
	-I . \
	-I `go list -m -f {{.Dir}} github.com/alta/protopatch` \
	-I `go list -m -f {{.Dir}} google.golang.org/protobuf` \
	--go-patch_out=plugin=go,paths=source_relative:. \
	--go-patch_out=plugin=go-grpc,paths=source_relative:. \
	*.proto

Features

Patches are defined via an Options extension on messages, fields, oneof fields, enums, and enum values.

  • go.message — message options, which modify the generated Go struct for a message.
  • go.field — message field options, which modify Go struct fields and getter methods.
  • go.oneof — oneof field options, which modify struct fields, interface types, and wrapper types.
  • go.enum — enum options, which modify Go enum types and values.
  • go.value — enum value options, which modify Go const values.

Custom Names

import "patch/go.proto";

message OldName {
	option (go.message).name = 'NewName';
	int id = 1 [(go.field).name = 'ID'];
}

enum Error {
	option (go.enum).name = 'ProtocolError';
	INVALID = 1 [(go.value).name = 'ErrInvalid'];
	NOT_FOUND = 2 [(go.value).name = 'ErrNotFound'];
	TOO_FUN = 3 [(go.value).name = 'ErrTooFun'];
}

Alternate Syntax

Multiple options can be grouped together with a message bounded by {}:

import "patch/go.proto";

message OldName {
	option (go.message) = {name: 'NewName'};
	int32 id = 1 [(go.field) = {name: 'ID'}];
}

enum Error {
	option (go.enum) = {name: 'ProtocolError'};
	INVALID = 1 [(go.value) = {name: 'ErrInvalid'}];
	NOT_FOUND = 2 [(go.value) = {name: 'ErrNotFound'}];
	TOO_FUN = 3 [(go.value) = {name: 'ErrTooFun'}];
}

Struct Tags

message ToDo {
	int32 id = 1 [(go.field).name: 'ID', (go.field).tags = 'xml:"id,attr"'];
	string description = 2 [(go.field).tags: 'xml:"desc"'];
}

Alternate Syntax

Multiple options can be grouped together with a message bounded by {}:

message ToDo {
	int32 id = 1 [(go.field) = {name: 'ID', tags: 'xml:"id,attr"'}];
	string description = 2 [(go.field) = {tags: 'xml:"desc"'}];
}

Embedded Fields

A message field can be embedded in the generated Go struct with the (go.field).embed option. This only works for message fields, and will not work for oneof fields or basic types.

import "patch/go.proto";

message A {
	B b = 1 [(go.field).embed = true];
}

message B {
	string value = 1;
}

The resulting Go struct will partially have the form:

type A struct {
	*B
}

type B struct {
	Value string
}

var a A
a.Value = "value" // This works because B is embedded in A

Alternate Syntax

Multiple options can be grouped together with a message bounded by {}:

import "patch/go.proto";

message A {
	B b = 1 [(go.field) = {embed: true}];
}

message B {
	string value = 1;
}

Linting

Protopatch can automatically “lint” generated names into something resembling idiomatic Go style. This feature should be considered unstable, and the names it generates are subject to change as this feature evolves.

  • Initialisms: names with ID or URL or other well-known initialisms will have their case preserved. For example Id would lint to ID, and ApiBaseUrl would lint to APIBaseURL.
  • Stuttering: it will attempt to remove repeated prefixed names from enum values. An enum value of type Foo named Foo_FOO_BAR would lint to FooBar.

To lint all generated Go names, add option (go.lint).all = true to your proto file. To lint only enum values, add option (go.lint).values = true. To specify one or more custom initialisms, specify an initialism with option (go.lint).initialisms = 'HSV' for the HSV initialism. All names with HSV will preserve its case.

option (go.lint).all = true;
option (go.lint).initialisms = 'RGB';
option (go.lint).initialisms = 'RGBA';
option (go.lint).initialisms = 'HSV';

enum Protocol {
	// PROTOCOL_INVALID value should lint to ProtocolInvalid.
	PROTOCOL_INVALID = 0;
	// PROTOCOL_IP value should lint to ProtocolIP.
	PROTOCOL_IP = 1;
	// PROTOCOL_UDP value should lint to ProtocolUDP.
	PROTOCOL_UDP = 2;
	// PROTOCOL_TCP value should lint to ProtocolTCP.
	PROTOCOL_TCP = 3;
}

message Color {
	oneof value {
		// rgb should lint to RGB.
		string rgb = 1;
		// rgba should lint to RGBA.
		string rgba = 2;
		// hsv should lint to HSV.
		string hsv = 3;
	}
}

protopatch's People

Contributors

adphi avatar altaci avatar connyay avatar dependabot[bot] avatar hanjm avatar houz42 avatar moul avatar ydnar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

protopatch's Issues

Annotations in YML files

Is it possible to write protopatch annotations in a separate YML file, instead of in the Proto file? That way I can avoid importing the protopatch Proto file, which is great if I want to distribute the Proto files to others so they can compile them for their platform/language.

Proto3: add optional support

Proto3 support for optional has been added, see https://github.com/protocolbuffers/protobuf/blob/master/docs/implementing_proto3_presence.md

protopatch does not have support for it:

tests/plugin/validate.proto: is a proto3 file that contains optional fields, but code generator protoc-gen-go-patch hasn't been updated to support optional fields in proto3. Please ask the owner of this code generator to support proto3 optional.--go-patch_out: 
make: *** [tests/plugin/validate.proto] Error 1

Support can be declared using this snippet in main.go:

supportedFeatures := uint64(pluginpb.CodeGeneratorResponse_FEATURE_PROTO3_OPTIONAL)
res.SupportedFeatures = &supportedFeatures

// Write the patched CodeGeneratorResponse to stdout.
return patch.WriteResponse(os.Stdout, res)

But introducing optional keyword in the test proto files breaks code generation.

Missing descriptor.proto in google/protobbuf

I am trying to use this package by importing using go mod. I see the it requires following from go.mod

google.golang.org/protobuf v1.31.0

But, I checked from the source that descriptor.proto is missing in that

hence I see compile errors when I do protoc

google/protobuf/descriptor.proto: File not found.
patch/go.proto: Import "google/protobuf/descriptor.proto" was not found or had errors.
patch/go.proto:46:8: "google.protobuf.MessageOptions" is not defined.
patch/go.proto:50:8: "google.protobuf.FieldOptions" is not defined.
patch/go.proto:54:8: "google.protobuf.OneofOptions" is not defined.
patch/go.proto:58:8: "google.protobuf.EnumOptions" is not defined.
patch/go.proto:62:8: "google.protobuf.EnumValueOptions" is not defined.
patch/go.proto:95:8: "google.protobuf.FileOptions" is not defined.
greet.proto: Import "patch/go.proto" was not found or had errors.

How to fix this?

Support public imports

A proto file imported from another proto file using import public ... will have its exported types aliased in the generated Go code, and vars copied. Currently the Go code generator doesn’t check to see if the imported symbols clobber existing symbols in the importing package.

  • Ensure symbols imported using import public are renamed when aliased or copied into the destination package.
  • Consider an option to rename the symbol in an importing package.

Question: why so many proto imports?

This looks great @ydnar ... thanks for doing this.

I’m wondering why use several different options for this; can’t this be done with a single option go.options for all the things, and figure out at “patch” time which type of patch it should be?

Alternatively, if this isn’t possible or desirable, I think it would be nicer to have a single import for all options.

use of external plugins breaks generated code

We often use other plugins like protoc-gen-validate or protoc-gen-grpc-gateway, but tools renaming fields usually breaks the generated code. The chosen approach by protopatch may solve this issue but does not work right now.
I did implement a naive cache that store protoc-gen-go output and reload it before patching generated code by other plugins, allowing the ast tree to find the type's declarations. I can create a pull request if you wish.
Another approach would have been to re-run protoc-gen-go plugin before each (other) plugin call to retrieve the types declaration.
I don't know which would be best.
Any thought ?

`option (go.lint).all = true;` does not work for enum values imported from other package

Firstly, thanks a lot for this project, it's really great.

Provided awesome/v1/entity.proto:

package awesome.v1;

import "patch/go.proto";

option (go.lint).all = true;

enum MyEnum {
   MY_ENUM_UNSPECIFIED = 0;
}

It generates idiomatic Go identifiers 👍 on entity.pb.go:

const (
      MyEnumUnspecified MyEnum = 0
)

And then awesome/api/v1/api.proto:

package awesome.api.v1;

import "patch/go.proto";
import "awesome/v1/entity.proto"

option (go.lint).all = true;

message MessageWithMyEnum {
    awesome.v1.MyEnum my_enum = 0;
}

But at generate this message, it fails to use the patched identifier on api.pb.go:

func (x *MessageWithMyEnum) GetMyEnum() v1.MyEnum {
    if x != nil {
          return x.MyEnum
    }
    return v1.MY_ENUM_UNSPECIFIED
}

This is worked around with this code:

enum MyEnum {
   MY_ENUM_UNSPECIFIED = 0 [(go.value) = {name:'MyEnumUnspecified'}];
}

Thanks!

lint initialisms missing "IDs"

For example...

  • expected:example_ids -> ExampleIDs
  • actual: example_ids -> ExampleIds

Doesn't look like you can fix this with the lint's initialisms setting, either.

Multiple packages with the same "leaf" package name can cause go.lint options to be ignored.

If multiple packages have the same "leaf" package name (package1/messages/, package2/messages/), then the linting will sometimes be skipped for one of the packages.

It's easiest to describe with a small example:

v1/a/test.proto:

syntax = "proto3";
package test.v1.a;
option go_package = "foo/bar/v1/a";

v2/a/anothertest.proto:

syntax = "proto3";
package test.v1.a;
option go_package = "foo/bar/v2/a";

import "patch/go.proto";
option (go.lint).all = true;

enum SomeEnum {
  SOME_ENUM_UNDEFINED = 0;
  SOME_ENUM_FOO = 1;
}

Then, depending on the order the proto files are supplied to protoc, the generated enum might not be linted.

this failed to lint the enum:
protoc -I . --go-patch_out=plugin=go,paths=source_relative:.make v2\a\anothertest.proto v1\a\test.proto
this works:
protoc -I . --go-patch_out=plugin=go,paths=source_relative:.make v2\a\test.proto v1\a\anothertest.proto

I think this is because Patcher.packagesByName is only keyed by the "leaf" package name, and thus gets overwritten.

Suggestion: Add 'prefix' to enum options

Errors in Go are usually prefixed with Err, and adding a custom name for each enum value is cumbersome and error prone

It seems something like "prefix" feature would make life a bit easier

Compare

enum Errors {
	option (go.enum.options) = {name: 'ProtocolErrors'};
	INVALID = 1 [(go.value.options) = {name: 'ErrInvalid'}];
	NOT_FOUND = 2 [(go.value.options) = {name: 'ErrNotFound'}];
	TOO_FUN = 3 [(go.value.options) = {name: 'ErrTooFun'}];
}

and

enum Errors {
	option (go.enum.options) = {name: 'ProtocolErrors', prefix: 'Err'};
	INVALID = 1;
	NOT_FOUND = 2;
	TOO_FUN = 3;
}

should produce same results.

(I think (might be mistaken though) that contraction rules will replace NOT_FOUND and TOO_FUN with NotFound and TooFun correspondingly.)

Thank you very much!

(Found through gogo/protobuf#691 (comment))

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.