coinbase / protoc-gen-rbi Goto Github PK

If we have an example file:

syntax = "proto3";

package example;

message Request {
  // name for some message
  string name = 1;
}

message Response {
  // greeting response
  string greeting = 1;
}

service Greeter {
  // some description for the hello method
  rpc Hello (Request) returns (Response);
}

but the generated .rbi file doesn't have the comments, so hovering over fields in VSCode doesn't show the comments

# Code generated by protoc-gen-rbi. DO NOT EDIT.
# source: example.proto
# typed: strict

class Example::Request
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  sig { params(str: String).returns(Example::Request) }
  def self.decode(str)
  end

  sig { params(msg: Example::Request).returns(String) }
  def self.encode(msg)
  end

  sig { params(str: String, kw: T.untyped).returns(Example::Request) }
  def self.decode_json(str, **kw)
  end

  sig { params(msg: Example::Request, kw: T.untyped).returns(String) }
  def self.encode_json(msg, **kw)
  end

  sig { returns(::Google::Protobuf::Descriptor) }
  def self.descriptor
  end

  sig do
    params(
      name: T.nilable(String)
    ).void
  end
  def initialize(
    name: ""
  )
  end

  sig { returns(String) }
  def name
  end

  sig { params(value: String).void }
  def name=(value)
  end

  sig { void }
  def clear_name
  end

  sig { params(field: String).returns(T.untyped) }
  def [](field)
  end

  sig { params(field: String, value: T.untyped).void }
  def []=(field, value)
  end

  sig { returns(T::Hash[Symbol, T.untyped]) }
  def to_h
  end
end

class Example::Response
  include ::Google::Protobuf::MessageExts
  extend ::Google::Protobuf::MessageExts::ClassMethods

  sig { params(str: String).returns(Example::Response) }
  def self.decode(str)
  end

  sig { params(msg: Example::Response).returns(String) }
  def self.encode(msg)
  end

  sig { params(str: String, kw: T.untyped).returns(Example::Response) }
  def self.decode_json(str, **kw)
  end

  sig { params(msg: Example::Response, kw: T.untyped).returns(String) }
  def self.encode_json(msg, **kw)
  end

  sig { returns(::Google::Protobuf::Descriptor) }
  def self.descriptor
  end

  sig do
    params(
      greeting: T.nilable(String)
    ).void
  end
  def initialize(
    greeting: ""
  )
  end

  sig { returns(String) }
  def greeting
  end

  sig { params(value: String).void }
  def greeting=(value)
  end

  sig { void }
  def clear_greeting
  end

  sig { params(field: String).returns(T.untyped) }
  def [](field)
  end

  sig { params(field: String, value: T.untyped).void }
  def []=(field, value)
  end

  sig { returns(T::Hash[Symbol, T.untyped]) }
  def to_h
  end
end

It seems like a protobuf type like

message Foo {
  bar string = 1
}

generates a sorbet bound constructor like

  sig do
    params(
      bar: String
    ).void
  end
  def initialize(
    bar: ""
  )
  end

In many cases (for us at least), the caller might be passing a bar value which is of type T.nilable(String). As a result, callers have to explicitly invoke things like Foo.new(bar: somethingNilable || ''), which isn't a huge problem but does become verbose in larger instances with many such fields.

It seems like the constructor above would work well and retain backwards-compatible semantics if it typed bar as T.nilable(String) and coerced nil to "" (for String; the conversion for other primitives would be similar). Presumably the actual matching Ruby code would need to do the actual x.nil? : '' : x conversion.

We have a build system that builds protoc-gen-rbi as part of a larger repository build by effectively downloading protoc-gen-rbi and then running go build. This has the effect of downloading all dependencies at build time, which can be slow or flaky given the number of builds we run and Github's potential for rate limiting.

Would you consider taking a PR that checks in the vendor directory with go mod vendor? This would enable us to run reproducible builds without depending on the entire world.

This does have the downside that we'd pin to specific repository versions of dependencies, but I think from a dependency standpoint, this is nicer than depending on the builder to fetch the versions at compile time.

If this sounds reasonable, I'm happy to submit a PR to this effect!

cc @praboud-stripe

Let me start this off with, I have no exp in Go at all :P
I installed Go v1.16.8 bc I was getting errors following the install instructions using go get ... with v17.
I then was able to install the package, but when trying to run the Usage example, I get Command not found: protoc

I restarted my terminal after installing go and again after installing the package but still receive the error. Any advice?

Problem

With the generated types for enums it's tricky to figure out what the possible variants are without spelunking through .proto files or other usages in the codebase.

For example, with the following enum:

enum Mode {
  INVALID_MODE = 0;
  LIVEMODE = 1;
  TESTMODE = 2;
}

the generated .rbi doesn't help when figuring out the possible variants:

class Common::Feature < ::Google::Protobuf::AbstractMessage
  sig do
    params(
      mode: T.nilable(T.any(Symbol, String, Integer)),
    ).void
  end
  def initialize(
    mode: :INVALID_MODE,
  )
  end

  sig { returns(Symbol) }
  def mode
  end

  sig { params(value: T.any(Symbol, String, Integer)).void }
  def mode=(value)
  end
end

relevant generator code:

Ideas

Add a comment with the Symbol variants:

  # variants: :INVALID_MODE, :LIVEMODE, :TESTMODE
  sig { params(value: T.any(Symbol, String, Integer)).void }
  def mode=(value)
  end

somehow express the union in the type itself

but also keep the more general Symbol type since other variants could be passed in

not sure about this one

Thanks for building and open-sourcing this library! We're currently trying to adopt both sorbet and gRPC, so this looks like a potentially very useful part of our toolchain.

One thing we've noticed in trying to adopt this library is that all primitive (non-message) protobuf types are mapped to a non-nilable sorbet types:

With proto3 protobufs, all fields are considered optional, so this prevents us from creating potentially valid protobufs. Was this an intentional decision when building this library? Would you be open to a contribution to make these types nilable?

protoc v3.12.0 added support for the optional annotation to proto3 (see RC: https://github.com/protocolbuffers/protobuf/releases/tag/v3.12.0-rc1)

Trying optional fields generation now errors out with:
proto3 file that contains optional fields, but code generator protoc-gen-rbi hasn't been updated to support optional fields in proto3. Please ask the owner of this code generator to support proto3 optional.--rbi_out

Hey, I'm running into an issue upgrading to golang/protobuf v1.5.4 that appears to be stemming from this code generator.

The error result is is a proto3 file that contains optional fields, but code generator protoc-gen-rbi hasn't been updated to support optional fields in proto3. Please ask the owner of this code generator to support proto3 optional. However, I know that optional support in this code generator has been around for a few years, and these proto files compile just fine on version 1.5.2 of the protobuf module.

It appears there is some incompatibility here, but I don't know enough Go to determine what exactly is the issue. Any chance we can get some support on moving this module to depend on the recent security-patched protobuf release?

Unreviewed commit(s) COMPARE was merged to master by mdehoog.

Comment with appropriate number of 👍 to approve.

We have the proto file:

com/example/v1alpha/test.proto:

package com.example.v1alpha

message Test { ... }

generated ruby file:

module Com
  module Example
    module V1alpha
      Test = ....

generated rbi file:

class Com::Example::V1Alpha::Test

note the wrong capitalization of A in V1Alpha. It does not match the ruby files.