yoheimuta / go-protoparser Goto Github PK
View Code? Open in Web Editor NEWYet another Go package which parses a Protocol Buffer file (proto2+proto3)
License: MIT License
Yet another Go package which parses a Protocol Buffer file (proto2+proto3)
License: MIT License
Hi!
Awesome application, thanks for building this. I hit a minor issue today:
We have enums defined which have custom options pointing to another message for documentation purposes:
e.g. something like this
REQUEST = 1 [(event_additional_data) = {field_number: 101}];
This fails during parsing with:
failed to parse, err found "\"{\"(Token=14, Pos=some.proto:39:54)" but expected [enumValueOption] at /home/hfjn/go/pkg/mod/github.com/yoheimuta/go-protoparser/[email protected]/parser/enum.go:280:found "{" but expected [;]
After checking the code I assume the enum type currently only supports String types?
Thanks!
Here is a simple reprod:
syntax = "proto3";
import "google/protobuf/descriptor.proto";
extend google.protobuf.ServiceOptions {
string service_description = 51000;
}
service MyService{
option(service_description) = "description";;
}
note that there is a mistake in the file, with a double semicolon after description.
when running protolint on this file, we got a crash:
protolint lint test.proto
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x18 pc=0x527104]
goroutine 1 [running]:
github.com/yoheimuta/go-protoparser/v4/parser.(*Service).Accept(0xc0001b45a0, {0xb52a20, 0xc000008bd0})
/home/runner/go/pkg/mod/github.com/yoheimuta/go-protoparser/[email protected]/parser/service.go:90 +0x64
github.com/yoheimuta/go-protoparser/v4/parser.(*Proto).Accept(0xc000222750?, {0xb52a20, 0xc000008bd0})
/home/runner/go/pkg/mod/github.com/yoheimuta/go-protoparser/[email protected]/parser/proto.go:26 +0x69
github.com/yoheimuta/protolint/linter/visitor.RunVisitorAutoDisable({0xb53b88?, 0xc000224a80?}, 0xc000222750, {0xa7185b, 0x10}, 0x5?)
/home/runner/work/protolint/protolint/linter/visitor/hasExtendedVisitor.go:55 +0x119
github.com/yoheimuta/protolint/linter/visitor.RunVisitor(...)
/home/runner/work/protolint/protolint/linter/visitor/hasExtendedVisitor.go:28
github.com/yoheimuta/protolint/internal/addon/rules.QuoteConsistentRule.Apply({{{0xa68c01?, 0x15?}}, 0xc00001e82d?, 0xa?}, 0xc00019e600?)
/home/runner/work/protolint/protolint/internal/addon/rules/quoteConsistentRule.go:60 +0xa7
github.com/yoheimuta/protolint/internal/linter.(*Linter).Run(0xc00000b6e0?, 0xc000177758, {0xc000228180?, 0x7, 0x0?})
/home/runner/work/protolint/protolint/internal/linter/linter.go:33 +0x10e
github.com/yoheimuta/protolint/internal/cmd/subcmds/lint.(*CmdLint).runOneFile(0xc0001c4380, {{0xc00001e810, 0x27}, {0xc00001e82d, 0xa}})
/home/runner/work/protolint/protolint/internal/cmd/subcmds/lint/cmdLint.go:131 +0x151
github.com/yoheimuta/protolint/internal/cmd/subcmds/lint.(*CmdLint).run(0xc0001c4380)
/home/runner/work/protolint/protolint/internal/cmd/subcmds/lint/cmdLint.go:100 +0xd6
github.com/yoheimuta/protolint/internal/cmd/subcmds/lint.(*CmdLint).Run(0xc0001c4380)
/home/runner/work/protolint/protolint/internal/cmd/subcmds/lint/cmdLint.go:77 +0x53
github.com/yoheimuta/protolint/internal/cmd.doLint({0xc0000360e0?, 0x100000000?, 0x0?}, {0xb46980, 0xc00005e028}, {0xb46980, 0xc00005e030})
/home/runner/work/protolint/protolint/internal/cmd/cmd.go:110 +0x2d1
github.com/yoheimuta/protolint/internal/cmd.doSub({0xc0000360d0?, 0x0?, 0x1d8e0249df8?}, {0xb46980?, 0xc00005e028?}, {0xb46980?, 0xc00005e030?})
/home/runner/work/protolint/protolint/internal/cmd/cmd.go:64 +0xac
github.com/yoheimuta/protolint/internal/cmd.Do({0xc0000360d0?, 0x60?, 0x0?}, {0xb46980?, 0xc00005e028?}, {0xb46980?, 0xc00005e030?})
/home/runner/work/protolint/protolint/internal/cmd/cmd.go:49 +0x78
main.main()
/home/runner/work/protolint/protolint/cmd/protolint/main.go:11 +0x5e
protoc correctly handles this by reporting a syntax error.
protolint should not panic, and rather output a syntax error like protoc
Thanks for producing protolint!
Please could the aggregate syntax be supported for (custom) options?
message FooOptions {
optional int32 opt1 = 1;
optional string opt2 = 2;
}
extend google.protobuf.FieldOptions {
optional FooOptions foo_options = 1234;
}
// usage:
message Bar {
optional int32 a = 1 [(foo_options).opt1 = 123, (foo_options).opt2 = "baz"];
// alternative aggregate syntax (uses TextFormat):
optional int32 b = 2 [(foo_options) = { opt1: 123 opt2: "baz" }];
// for completeness, split with (optional) commas:
optional int32 c = 3 [(foo_options) = {
opt1: 123,
opt2: "baz",
}];
}
Unfortunately this is not part of the language spec but is implemented in protoc...
Readme says to do:
go get github.com/yoheimuta/go-protoparser/v4
But this returns an error:
package github.com/yoheimuta/go-protoparser/v4: cannot find package "github.com/yoheimuta/go-protoparser/v4" in any of...
It seems to be invalid according to https://developers.google.com/protocol-buffers/docs/reference/proto3-spec#option, but this is a common style. So I will support this format when the permissive option is true.
Hi, thanks for providing an awesome linter and library.
I'm introducing the linter to my project. And it raises an exception that found "." but expected [TIDENT]. Use -v for more details
.
It fails at parsing like the following statement.
option (.foo.bar) = {…}
It seems invalid, according to https://developers.google.com/protocol-buffers/docs/reference/proto3-spec#option, but protobuf allows this syntax to change the name resolution algorithm.
cf. https://github.com/protocolbuffers/protobuf/blob/b696347f80d324c624e65a0b7e3ea5dac7ec5a41/src/google/protobuf/compiler/parser_unittest.cc#L2126-L2141
Could you support this format?
I'm trying to parse comments from messages formatted according to the google protobuf style guide.
In the following example:
message SearchRequest {
string query = 1;
int32 page_number = 2; // Which page number do we want?
int32 result_per_page = 3; // Number of results to return per page.
}
I would expect the field page_number
to contain the comment "// Which page number do we want?", but instead I'm getting an empty string.
In the same way, I'd expect result_per_page
to contain "// Number of results to return per page", but instead I get the comment of the field above it.
I had a quick look at your unit tests (ie: parser/message_test.go
), and it seems you're following a different style.
message outer {
// option
option (my_option).a = true;
// message
message inner { // Level 2
int64 ival = 1;
}
// field
repeated inner inner_message = 2;
// enum
enum EnumAllowingAlias {
option allow_alias = true;
}
EnumAllowingAlias enum_field =3;
// map
map<int32, string> my_map = 4;
// oneof
oneof foo {
string name = 5;
SubMessage sub_message = 6;
}
// reserved
reserved "bar";
}
I understand people have different styles, but if you claim it "Conforms to the exactly official spec" I would expect comments to be formatted as the official spec too.
Can we file this as a bug?
I'd be glad to help.
Hello!
I use Bazel to build a bunch of my personal projects, and at work as well. I like it a lot: https://bazel.build
In particular, Bazel support for golang is pretty good. There's a neat tool called gazelle
that auto-manages things for you: https://github.com/bazelbuild/bazel-gazelle
I'd like to use your parser in projects that use Bazel for builds, and will probably end up bazel-ifying it in a fork. Mostly this involves adding BUILD.bazel
files, and a WORKSPACE
file. Would you be open to accepting this upstream? I'm happy to do the work of converting your builds, so that your CI builds get the benefits of Bazel caching.
WIP
How I can get information - import is used or not?
Hi,
I was going through the readme, and found that the godoc links don't work, thought of letting you know
It seems to be invalid according to https://developers.google.com/protocol-buffers/docs/reference/proto3-spec, but this is a common style. So I will support this definition by default.
ref.
type MessageBody struct {
Fields []*parser.Field
Enums []*Enum
Messages []*Message
Options []*parser.Option
Oneofs []*parser.Oneof
Maps []*parser.MapField
Groups []*parser.GroupField
Reserves []*parser.Reserved
Extends []*parser.Extend
EmptyStatements []*parser.EmptyStatement
}
could you please add a new field Extensions []*parser.Extension
to the MessageBody struct and the switch statement? Thanks.
String literals that are split between multiple lines in proto annotations cause an error in go-protoparser. Running the parser on _testdata/simple.proto
with the following modification
enum EnumAllowingAlias {
option allow_alias = true;
UNKNOWN = 0;
STARTED = 1;
RUNNING = 2 [(custom_option) = "this is a "
"string on two lines"
];
}
results in this error
failed to parse, err found "\""(Token=11, Pos=simple.proto:12:36) but expected []] at //go/src/github.com/yoheimuta/go-protoparser/parser/enum.go:293:found "\"" but expected [;]
Despite this syntax being accepted by the compiler. It's not clear from the language spec what the proper syntax for multiline literals is, however.
2019/12/11 13:34:59 Lexer encountered the error "found "\x00" but expected [
] at /go/pkg/mod/github.com/yoheimuta/go-protoparser/[email protected]/internal/lexer/scanner/comment.go:12"
Looks like https://github.com/yoheimuta/go-protoparser/blob/master/internal/lexer/scanner/comment.go#L9-L17 is a bit too strict
Why FieldNumber has string type? Can I expect strconv.AtoI always return nil error for FieldNumber?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.