bufbuild / protovalidate

Protocol Buffer Validation - Go, Java, Python, and C++ Beta Releases!

protovalidate

protovalidate is a series of libraries designed to validate Protobuf messages at runtime based on user-defined validation rules. Powered by Google's Common Expression Language (CEL), it provides a flexible and efficient foundation for defining and evaluating custom validation rules. The primary goal of protovalidate is to help developers ensure data consistency and integrity across the network without requiring generated code.

Note

protovalidate is the spiritual successor to protoc-gen-validate.

We recommend that new and existing projects transition to using protovalidate instead of protoc-gen-validate.

Read our blog post if you want to learn more about the limitations of protoc-gen-validate and how we have designed protovalidate to be better.

What is this repository?

This repository is the core of the protovalidate project. It contains:

• The API definition: used to describe validation constraints
• Documentation: how to apply protovalidate effectively
• Migration tooling: incrementally migrate from protoc-gen-validate
• Examples: example .proto files using protovalidate
• Conformance testing utilities: for acceptance testing of protovalidate implementations

Implementations

Runtime implementations of protovalidate can be found in their own repositories:

• Go: protovalidate-go (beta release)
• C++: protovalidate-cc (beta release)
• Java: protovalidate-java (beta release)
• Python: protovalidate-python (beta release)
• TypeScript: protovalidate-ts (coming soon)

Interested in adding support for another language? Check out our Contributing Guidelines.

Usage

Import protovalidate

To define constraints within your Protobuf messages, import buf/validate/validate.proto into your .proto files:

syntax = "proto3";

package my.package;

import "buf/validate/validate.proto";

Build with buf

Add a dependency on buf.build/bufbuild/protovalidate to your module's buf.yaml:

version: v1
# <snip>
deps:
  - buf.build/bufbuild/protovalidate
# <snip>

After modifying your buf.yaml, don't forget to run buf mod update to ensure your dependencies are up-to-date.

Build with protoc

Add an import path (-I flag) pointing to the contents of the proto/protovalidate directory to your invocation of protoc:

protoc \
  -I ./vendor/protovalidate/proto/protovalidate \
  # <snip>

Implementing validation constraints

Validation constraints can be enforced using the buf.validate Protobuf package. The rules are specified directly in the .proto files.

Let's consider a few examples:

1. Scalar field validation: For a basic User message, we can enforce constraints such as a minimum length for the user's name.

   syntax = "proto3";
   
   import "buf/validate/validate.proto";
   
   message User {
     // User's name, must be at least 1 character long.
     string name = 1 [(buf.validate.field).string.min_len = 1];
   }

2. Map field validation: For a Product message with a map of item quantities, we can ensure that all quantities are positive.

   syntax = "proto3";
   
   import "buf/validate/validate.proto";
   
   message Product {
     // Map of item quantities, all quantities must be positive.
     map<string, int32> item_quantities = 1 [(buf.validate.field).map.values.int32.gt = 0];
   }

3. Well-known type (WKT) validation: For the User message, we can add a constraint to ensure the created_at timestamp is in the past.

   syntax = "proto3";
   
   import "google/protobuf/timestamp.proto";
   import "buf/validate/validate.proto";
   
   message User {
     // User's creation date must be in the past.
     google.protobuf.Timestamp created_at = 1 [(buf.validate.field).timestamp.lt_now = true];
   }

For more advanced or custom constraints, protovalidate allows for CEL expressions that can incorporate information across fields.

1. Field-level expressions: We can enforce that a products' price, sent as a string, includes a currency symbol like "$" or "£". We want to ensure that the price is positive and the currency symbol is valid.

   syntax = "proto3";
   
   import "buf/validate/validate.proto";
   
   message Product {
     string price = 1 [(buf.validate.field).cel = {
       id: "product.price",
       message: "Price must be positive and include a valid currency symbol ($ or £)",
       expression: "(this.startsWith('$') || this.startsWith('£')) && double(this.substring(1)) > 0"
     }];
   }

2. Message-level expressions: For a Transaction message, we can use a message-level CEL expression to ensure that the delivery_date is always after the purchase_date.

   syntax = "proto3";
   
   import "google/protobuf/timestamp.proto";
   import "buf/validate/validate.proto";
   
   message Transaction {
     google.protobuf.Timestamp purchase_date = 1;
     google.protobuf.Timestamp delivery_date = 2;
   
     option (buf.validate.message).cel = {
       id: "transaction.delivery_date",
       message: "Delivery date must be after purchase date",
       expression: "this.delivery_date > this.purchase_date"
     };
   }

3. Producing an error message in the expression: We can produce custom error messages directly in the CEL expressions. In this example, if the age is less than 18, the CEL expression will evaluate to the error message string.

   syntax = "proto3";
   
   import "buf/validate/validate.proto";
   
   message User {
     int32 age = 1 [(buf.validate.field).cel = {
       id: "user.age",
       expression: "this < 18 ? 'User must be at least 18 years old': ''"
     }];
   }

Check out examples for examples on both standard constraints and custom CEL constraints.

Validate Messages

Once the messages are annotated with constraints, use one of the supported language libraries to validate; no additional code generation necessary.

Documentation

protovalidate provides a robust framework for validating Protobuf messages by enforcing standard and custom constraints on various data types, and offering detailed error information when validation violations occur. For a detailed overview of all its components, the supported constraints, and how to use them effectively, please refer to our comprehensive documentation. The key components include:

• Standard Constraints: protovalidate supports a wide range of standard constraints for all field types as well as special functionality for the Protobuf Well-Known-Types. You can apply these constraints to your Protobuf messages to ensure they meet certain common conditions.

• Custom Constraints: With Google's Common Expression Language (CEL), protovalidate allows you to create complex, custom constraints to handle unique validation scenarios that aren't covered by the standard constraints at both the field and message level.

• Error Handling: When a violation occurs, protovalidateprovides detailed error information to help you quickly identify the source and fix for an issue.

protoc-gen-validate

protovalidate is the spiritual successor to protoc-gen-validate, offering all of the same functionality present in the original plugin, without the need for custom code generation, and the new ability to describe complex constraints in CEL.

protovalidate's constraints very closely emulate those in protoc-gen-validate to ensure an easy transition for developers. To migrate from protoc-gen-validate to protovalidate, use the provided migration tool to incrementally upgrade your .proto files.

Ecosystem

• Buf
• CEL Specification

Legal

Offered under the Apache 2 license.

protovalidate's Issues

[Feature Request] Support validation of extension fields

Currently, all four implementations (Go, Java, Python, and C++) ignore extension fields.

Ideally, when building the evaluator for a message, if a message is extendable (its descriptor defines at least one extension range), an additional message-level evaluator would be instantiated whose implementation loops through all present extensions and recursively validates them. Not only should this apply field constraints to the extension fields, but it should also recursively validate nested messages, for extensions whose type is a message (or a list of messages or a map with message values).

Currently, evaluators are created and cached on a per-message basis. Computing an evaluator for a message constructs sub-evaluators for all fields and oneofs. We will need to either augment this to support to creating/caching evaluators keyed by an extension FieldDescriptor or the message evaluator needs to be mutable (at least when "lazy-loading" of evaluators is enabled), so that add'l extension field evaluators can be added as new extensions are encountered when validating message instances.

In order to properly support non-lazy validators, where all constraints must be defined up front, the APIs should be updated to accept extension FieldDescriptors up-front, in addition to accepting the set of known message types up-front. That way, rules can be built ahead-of-time for these extension fields and applied to the relevant messages.

[Feature Request] string.well_known.re2

Feature description:
Validator that checks if a string conforms to valid regular expression (RE2) syntax.

Problem it solves or use case:
Eliminates custom validation logic in RPCs.

Proposed implementation or solution:
Many langs have RE2 compilers.

[Feature Request] CIDR Validation

Feature description:
Add field validation for CIDR ranges, ie. support string cidr = 1 [(buf.validate.field).string.cidr = true];

I found a similar request from the protoc-gen-validate repo that had been closed bufbuild/protoc-gen-validate#461

Problem it solves or use case:
Currently to get validation behavior I am using this pattern

message CidrRange {
  string address_prefix = 1 [(buf.validate.field).string.ipv4 = true];
  google.protobuf.UInt32Value prefix_len = 2 [(buf.validate.field).uint32 = {lte: 32}];
}

however, this isn't particularly ergonomic when used with google.api.http routes since you end up having to do:

?cidr.address_prefix=1.2.3.4&cidr.prefix_len=28

vs

?cidr=1.2.3.4/28

Proposed implementation or solution:
I think doing a similar implementation as IP validators would make sense

protovalidate/proto/protovalidate/buf/validate/validate.proto

Lines 2683 to 2730 in 7e81586

	//`ip` specifies that the field value must be a valid IP
	//(v4 or v6) address, without surrounding square brackets for IPv6 addresses.
	//If the field value isn't a valid IP address, an error message will be
	//generated.
	//
	//```proto
	//message MyString {
	// // value must be a valid IP address
	// string value = 1 [(buf.validate.field).string.ip = true];
	//}
	//```
	bool ip = 14 [(priv.field).cel = {
	id: "string.ip",
	message: "value must be a valid IP address",
	expression: "this.isIp()",
	}];
	//`ipv4` specifies that the field value must be a valid IPv4
	//address. If the field value isn't a valid IPv4 address, an error message
	//will be generated.
	//
	//```proto
	//message MyString {
	// // value must be a valid IPv4 address
	// string value = 1 [(buf.validate.field).string.ipv4 = true];
	//}
	//```
	bool ipv4 = 15 [(priv.field).cel = {
	id: "string.ipv4",
	message: "value must be a valid IPv4 address",
	expression: "this.isIp(4)"
	}];
	//`ipv6` specifies that the field value must be a valid
	//IPv6 address, without surrounding square brackets. If the field value is
	//not a valid IPv6 address, an error message will be generated.
	//
	//```proto
	//message MyString {
	// // value must be a valid IPv6 address
	// string value = 1 [(buf.validate.field).string.ipv6 = true];
	//}
	//```
	bool ipv6 = 16 [(priv.field).cel = {
	id: "string.ipv6",
	message: "value must be a valid IPv6 address",
	expression: "this.isIp(6)",
	}];

1. (buf.validate.field).string.cidr
2. (buf.validate.field).string.cidrv4
3. (buf.validate.field).string.cidrv6

Contribution:
I would be happy to try and implement this myself. I'm not sure how implementations in the language specific repos is done, but I could also help with the Go and Python implementations.

Examples or references:
https://github.com/go-playground/validator/blob/5bf55dc757cad229e8297d42640ec036e2360df7/baked_in.go#L375-L393

[Feature Request] Is it possible to add validated input parameters and rule values in the error Violation?

Feature description:

Example Proto

syntax = "proto3";

package project.api.v1;

import "buf/validate/validate.proto";

option go_package = "project/api/v1;v1";

// 分页参数
message PageParams {
  // 当前页, 从1开始(默认: 1, 最大: 1000)
  uint32 page = 1 [
    (buf.validate.field).uint32 = {gt: 0, lte: 1000}
  ];

  // 每页数量(默认: 20, 最大: 500)
  uint32 pageSize = 2 [
    (buf.validate.field).uint32 = {gt: 0, lte: 500}
  ];
}

go validate

func main() {
	msg := &pb.PageParams{
		Page:     0,
		PageSize: 10,
	}

	v, err := protovalidate.New()
	if err != nil {
		fmt.Println("failed to initialize validator:", err)
	}

	if err = v.Validate(msg); err != nil {
		var valErr *protovalidate.ValidationError
		if ok := errors.As(err, &valErr); ok {
			errPb := valErr.ToProto()
			fmt.Printf("\n%+v\n\n", errPb.GetViolations())
		}
	}
}

// ouput
//
// [field_path:"page"  constraint_id:"uint32.gt_lte"  message:"value must be greater than 0 and less than or equal to 1000"]
//

When I validate, the error returned by page is *validate.Violation, but I can’t get the request value 0 and the rule values: 0 and 1000, can the package add new fields (value and attributes ?? ) return?

Problem it solves or use case:

Users can get these values for more custom operations

I need to perform i18n now, because the semantics of each language are different when translating, and I need to customize the translation, but I can’t get these values, which makes the work very difficult

Proposed implementation or solution:

Contribution:

Examples or references:

Additional context:

Go. no validations during generation but not build/dep errors

Description

In managed mode and noticed no validates are occuring but no errors during generation

Steps to Reproduce

message HyperParameters {
  int32 input_count = 1 [
    (buf.validate.field).int32.gt = 0,
    (buf.validate.field).int32.lte = 4096
  ];

  int32 output_count = 2 [
    (buf.validate.field).int32.gt = 0,
    (buf.validate.field).int32.lte = 4096
  ];

  Activation activation = 3 [ (buf.validate.field).enum.defined_only = true ];
}

Expected Behavior

generates neat.pb.validate.go with no errors however the results are missing but no errors. The behavior is the same for all levels of nesting in all messages.

// ValidateAll checks the field values on HyperParameters with the rules
// defined in the proto definition for this message. If any rules are
// violated, the result is a list of violation errors wrapped in
// HyperParametersMultiError, or nil if none found.
func (m *HyperParameters) ValidateAll() error {
	return m.validate(true)
}

func (m *HyperParameters) validate(all bool) error {
	if m == nil {
		return nil
	}

	var errors []error

	// no validation rules for InputCount

	// no validation rules for OutputCount

	// no validation rules for Activation

....

Environment

• Operating System: Linux
• Version: Ubuntu 22.04
• Protobuf Compiler & Version: buf v1.25.1
• Protovalidate Version: v1.31.0

[Question] Setting not_in or in rules literal returning a syntax error

Description

when using buf generate or buf push, enum rule not_in and in report error: syntax error: unexpected int literal

example proto:

syntax = "proto3";

package greet.v1;

import "buf/validate/validate.proto";

enum MyEnum {
  MY_ENUM_UNSPECIFIED = 0;
  MY_ENUM_VALUE1 = 1;
  MY_ENUM_VALUE2 = 2;
}

message MyMessage {
  // The field `value` must not be equal to any of the specified values.
  MyEnum value = 1 [(buf.validate.field).enum.not_in = { 1, 2 }];
}

buf.yaml:

version: v1
deps:
  - buf.build/bufbuild/protovalidate
breaking:
  use:
    - FILE
lint:
  use:
    - DEFAULT

buf.gen.yaml

version: v1
managed:
  enabled: true
  go_package_prefix:
    default: xxxx
    except:
      - buf.build/bufbuild/protovalidate

plugins:
  - plugin: go
    out: gen
    opt: paths=source_relative

Environment

• Operating System: macOS
• Version: 14.0 (23A344)
• Protobuf Compiler & Version: libprotoc 25.0
• buf version: 1.28.0
• protovalidate version: v0.5.3

[BUG] tools/protovalidate-migrate command error

Description

1. When instantiating Migrator, no values were assigned to Migrator.print and Migrator.write, only cfg was assigned to Migrator

protovalidate/tools/protovalidate-migrate/internal/migrator/migrator.go

Line 42 in 77c3495

func NewMigrator(cfg Config) *Migrator {

2. When executing Migrator.migrate(), use Migrator.print or Migrator.write, but they are both false, making the tool unusable

protovalidate/tools/protovalidate-migrate/internal/migrator/migrator.go

Line 83 in 77c3495

func (m *Migrator) migrate(rootPath, srcPath string, stat fs.DirEntry) error {

Steps to Reproduce

Expected Behavior

Actual Behavior

Screenshots/Logs

Environment

• Operating System:
• Version:
• Compiler/Toolchain:
• Protobuf Compiler & Version:
• Protoc-gen-validate Version:
• Protovalidate Version:

Possible Solution

Additional Context

Integrate with Google ServiceWeaver

@elliotmjackson Please try writing a demo with protovalidate to integrate github.com/ServiceWeaver/weaver together!

[Feature Request] Reusing Custom Validation Rules

Feature description:
I don't know it is possible. But reusing validation rules would be nice.

Problem it solves or use case:
A validation rule may be suitable for multiple fields.
Currently, same custom cel expression duplicated to all fields. This makes harder to maintain custom validation rules.

Proposed implementation or solution:
I don't know if there is a way to extend the rules. But if we could extend the rules, we could apply the rule to the field as normal standard constraint.

For Example;

string xxx = 1 [(buf.validate.field) = {string.<custom_cel_id>: true}];

Contribution:

Examples or references:

Additional context:

[Feature Request] Documentation on version releases and production readyness

Feature description:
Documentation regarding how close is protovalidate to being non-beta. And whether it's production ready.

Problem it solves or use case:
Allows users to decide whether to use as-is in production systems and whether to expect a non-beta release soon.
Protovalidate seems like the best solution to Proto model validation and it would be great to know if it's ready.

Proposed implementation or solution:
A documentation in the repo as an Markdown file. Readme can link to it.

Contribution:
As this would depend on the project maintainers and their plan, I don't see how I could help with the writing of the doc.

Examples or references:
N/A

Additional context:
N/A

Thanks in advance!

[BUG] Inconsistent behavior for required and ignore_empty

This issue represents a few different defects:

1. The current documentation does not fully specify required and ignore_empty field constraints. It is unclear how message fields are handled (is an empty message one that is unset/null or one that is a default/empty message instance?) or how this interacts with field presence and explicitly configured default values (which may be non-zero).
2. The current implementations actually disagree on the meaning of "empty" and how that is applied to the required and ignore_empty field constraints. So once a clear specification is provided for these constraints, some implementations will likely need to be updated to comply with the revised spec.
3. The current conformance tests do not have any cases which enforce how these constraints are applied, which is how we find ourselves with inconsistent implementations.

In addition to resulting in inconsistency, some of the implementations are also written in a way that will likely not work correctly with Protobuf Editions. In particular, some implementations do not fully lean on the reflection support of the relevant Protobuf runtime, which will correctly consider "features" in source files that use Protobuf Editions. This means that the new representation for things like implicit vs. explicit field presence is likely to confuse the implementation logic.

Below is a summary of how the various runtimes actually implement these constraints:

• Go: The Go implementation fully leverages Protobuf reflection to interpret the data and apply these constraints. This is close to an ideal implementation; the only caveat is that some methods in the runtime's reflection API are not well-specified under all conditions, and the implementation is using these methods even for these unclear conditions. This is likely remedied with a few unit tests to make sure the result is expected, even under these conditions.

  To be specific, the implementation leans on FieldDescriptor.HasPresence(), FieldDescriptor.Default(), and Message.Has(FieldDescriptor) to decide if a value is present/non-empty and enforce the required and ignore_empty constraints. The downside is that FieldDescriptor.Default() is being used unconditionally, but the docs for that method do not specify its behavior for non-scalar fields (e.g. lists, maps, and messages).

  Relevant code snippets:

  • https://github.com/bufbuild/protovalidate-go/blob/main/internal/evaluator/field.go#L52
  • https://github.com/bufbuild/protovalidate-go/blob/main/internal/evaluator/builder.go#L255

• Java: The Java implementation is very similar to the Go implementation in structure and also leverages Protobuf reflection. However, it uses some conditions to clarify the case of empty/default message values. (It considers an explicitly present message value to be "empty" as long as it equals the empty/default message instance.)

  Strangely, it does not use the field's default value if it's type is an unsigned integer: it explicitly considers only zero to be empty. This is an awkward inconsistency with the Go implementation and weird "hybrid" between the Go and Python/C++ approaches. (I suspect this may have roots in the fact that the JVM does not actually have unsigned integer types.)

  Relevant code snippets:

  • https://github.com/bufbuild/protovalidate-java/blob/main/src/main/java/build/buf/protovalidate/internal/evaluator/EvaluatorBuilder.java#L175
  • https://github.com/bufbuild/protovalidate-java/blob/main/src/main/java/build/buf/protovalidate/internal/evaluator/FieldEvaluator.java#L66
  • https://github.com/bufbuild/protovalidate-java/blob/main/src/main/java/build/buf/protovalidate/internal/evaluator/ValueEvaluator.java#L54-L59

• C++: The C++ implementation has some quirks. For one, instead of using FieldDescriptor.has_presence(), it only considers fields with message types and fields in oneofs to be "explicit presence". This is a proto3-specific interpretation and must change to properly support proto2 and editions.

  Furthermore, this does not use FieldDescriptor's accessors for the field's default value. Instead, it is hard-coded to only consider zero values (0 numeric values, empty string/bytes, and false) to be "empty", with regards to the ignore_empty constraint. This is inconsistent with the Go and Java implementations and also leads to the potential paradox that a non-zero default value for a field could be considered an invalid value if explicitly set.

  Relevant code snippets:

  • https://github.com/bufbuild/protovalidate-cc/blob/main/buf/validate/internal/constraints.cc#L106-L109
  • https://github.com/bufbuild/protovalidate-cc/blob/main/buf/validate/internal/constraints.cc#L115-L175

• Python: The Python implementation looks almost exactly like the C++ implementation. It does not use FieldDescriptor.has_presence or FieldDescriptor.default_value properties. Instead it has logic similar to the C++ implementation for deciding if a field has explicit presence. Though its implementation is more specific and only considers non-repeated message fields to have explicit presence (this is more correct than the C++ implementation, but FieldDescriptor.has_presence would still be a marked improvement).

  Also like C++, it only considers zero values to be empty and does not consider a field's potential non-zero default value.

  Different from C++: it will compare the value to the zero/empty value to decide if a required field is present. This seems wrong since it implies that an explicitly set field may still be considered absent. It only uses message.HasField(name) when the field is not in a oneof (which seems like it's overly-specific to the awkward implementation detail used in proto3 optional fields).

  Relevant code snippets:

  • https://github.com/bufbuild/protovalidate-python/blob/main/protovalidate/internal/constraints.py#L115
  • https://github.com/bufbuild/protovalidate-python/blob/main/protovalidate/internal/constraints.py#L335

In general, the implementations should really look much more like one another. Even if we add conformance tests to better detect inconsistencies like above, it would instill greater confidence if the implementation code also looked more uniform, using the same conditions and reflection APIs across all implementations, where possible. (FWIW, the Protobuf runtimes do provide reasonably consistent APIs for reflection, across languages.)

In my opinion, the implementation in the Go runtime is closest to what I would intuitively expect -- that a default value, even if set explicitly, is not checked when ignore_empty is true. However, I would argue that the name of this constraint is very confusing since "empty" doesn't intuitively apply to scalar fields at all. I think ignore_default would be more clear, especially since it ignores other constraints not only when the field is absent but also when explicitly set to the default value. The only down-sides to the Go implementation is that its unclear how its usage of FieldDescriptor.Default() interacts with non-scalar fields. Is an explicitly-set empty message considered "empty"? And, either way, should it? My opinion is that an explicitly set field is not empty, only an absent one. But I can see this interpretation being debatable (even if the constraint were named ignore_default).

Some of the inconsistencies are a bit subtle, such that devising a conformance test to distinguish one's behavior from another might be tricky. But we should at least have conformance tests that cover the following scenarios, which should have teased out issues in the above implementations:

• A proto2 optional field is "present" and thus valid per the required constraint as long as it is explicitly set, even if set to its zero or default value. (This would catch potential issues in the Python implementation.)
• We should have test cases that use messages that define default values for all scalar field types, and then verify that they are correctly considered empty. This can be done with a validation that would otherwise fail with the field's default or zero value, which should be skipped. Before such cases can be formulated, we need to first agree on and fully specify the meaning of ignore_empty. (This would catch the inconsistencies between using a field's configured default vs. hard-coding to zero.)
• We should have cases that verify (both positive and negative outcomes) whether absent and present-but-empty message values are considered empty.

Consider using oneof in the schema to prevent conflicting rules

Feature description:
Integrate support for schema-level constraint enforcement within protovalidate.

Problem it solves or use case:
During an evaluation of the protovalidate lint rules, a specific behavior was identified that drew attention. There's a valid concern about potential conflicts that could arise when certain rules are combined, such as the simultaneous application of greater-than and greater-than-equal rules to a single field. The existing lint-level validation might not be capable of detecting these conflicts early in the schema design phase. To alleviate this issue, a proposal is made to introduce the capability of defining oneof constraints directly within the schema structure. This approach would offer more instinctive and coherent constraint application.

Proposed implementation or solution:
The envisaged solution entails empowering developers to define oneof constraints directly within the schema using the protovalidate tool. This mechanism would effectively prevent the coexistence of conflicting rules within the same field. For instance, the current lint rules, such as FIELD_VALIDATE_NOW_WITHIN_MIX, FIELD_VALIDATE_EQUAL_GT_LTE_CONFLICT, etc., could be reimagined as schema-level oneof constraints. This transition necessitates adapting the interpretation of lint rules and their translation into schema-level constraints during the validation process.

Additional context:
Incorporating constraint evaluations directly within the schema framework can furnish developers with immediate insights during the schema design phase. This enhancement serves to curtail the chances of encountering ambiguities and conflicts. By delivering clearer and more direct guidance throughout schema design, the proposed feature has the potential to elevate the usability and dependability of protovalidate.

List of rules to be transformed into schema-level oneof constraints:

• FIELD_VALIDATE_NOW_WITHIN_MIX: Ensures now rules aren't mixed with absolute lt/gt rules.
• FIELD_VALIDATE_NOW_ABS_LT_GT_MIX: Ensures now rules aren't mixed with absolute lt/gt rules.
• FIELD_VALIDATE_EQUAL_GT_LTE_CONFLICT: Ensures no equal values for both gt and lte rules on the same field.
• FIELD_VALIDATE_EQUAL_GT_LT_CONFLICT: Ensures no equal values for both gt and lt rules on the same field.
• FIELD_VALIDATE_LT_LTE_CONFLICT: Ensures no simultaneous use of lt and lte rules on the same field.
• FIELD_VALIDATE_EQUAL_GTE_LT_CONFLICT: Ensures no equal values for both gte and lt rules on the same field.
• FIELD_VALIDATE_GT_GTE_CONFLICT: Ensures no simultaneous use of gt and gte rules on the same field.

Investigate the following

• FIELD_VALIDATE_NOW_BOTH_RULES: Prevents the use of both now rules together.

Not possible

• FIELD_VALIDATE_IN_NOT_IN_CONFLICT: Prevents simultaneous use of in and not_in rules on the same field.

[Question] How is this different from bufbuild/protoc-gen-validate ?

How is this different from bufbuild/protoc-gen-validate ???

Roadmap for TypeScript support

Hello protovalidate team 👋. You mentioned that TypeScript support is underway in the README. I haven't been able to find branches or bufbuild repos relating to TypeScript + protovalidate. Do you have a sense of where this fits on your roadmap? Thanks in advance!

Import error (buf/validate/validate.proto: does not exist)

Description

The final step in the example in the README fails due to an import error.

Steps to Reproduce

1. Set up folder structure

.
|
|____ gen/
|    |____ ... # will be filled
|
|____ examples/
|    |____ transaction.proto # contains example
|
|____ buf.yaml
|
|____ buf.gen.yaml
|
|____ buf.work.yaml

2. Set the file contents

# buf.yaml
version: v1
deps:
  - buf.build/bufbuild/protovalidate

# buf.work.yaml
version: v1
directories:
  - examples

# buf.gen.yaml
version: v1
managed:
  enabled: true
plugins:
  - plugin: buf.build/protocolbuffers/python:v23.4
    out: gen

// examples/transaction.proto example from README.md
syntax = "proto3";

package my.package;

import "google/protobuf/timestamp.proto";
import "buf/validate/validate.proto";

message Transaction {
  uint64 id = 1 [(buf.validate.field).uint64.gt = 999];
  google.protobuf.Timestamp purchase_date = 2;
  google.protobuf.Timestamp delivery_date = 3;

  string price = 4 [(buf.validate.field).cel = {
    id: "transaction.price",
    message: "price must be positive and include a valid currency symbol ($ or £)",
    expression: "(this.startswith('$') or this.startswith('£')) and float(this[1:]) > 0"
  }];

  option (buf.validate.message).cel = {
    id: "transaction.delivery_date",
    message: "delivery date must be after purchase date",
    expression: "this.delivery_date > this.purchase_date"
  };
}

3. Run from root folder

buf mod update

4. Run from root folder

buf generate buf.build/bufbuild/protovalidate

5. Run from root folder

buf generate --include-imports

Expected Behaviour

Should generate the python models.

Actual Behaviour

Everything up to step 4 works. In step 5 I get the error

examples/transaction.proto:6:8:buf/validate/validate.proto: does not exist

Screenshots/Logs

Environment

• Operating System: OSX
• Version: macOS 13.5.2
• Compiler/Toolchain: python 3.11, Buf cli v1.27.0
• Protobuf Compiler & Version: Buf cli v1.27.0, protoc v24.3
• Protoc-gen-validate Version: 1.0.2
• Protovalidate Version: 0.2.1

Also have python packages

• protobuf==4.24.4
• grpcio-tools==1.59.0

Possible Solution

I think I followed the instructions as closely as possible. If not, please let me know.

Either way, it would be nice if this repository had a simple example (a folder containing one single *.proto file + the buf*.yaml files) that worked "out of the box" (assuming Buf cli is installed).

[Feature Request] Support specification validation for google.protobuf.Timestamp

Feature description:
The well-known google.protobuf.Timestamp specification imposes some constraints on the fields (see source):

message Timestamp {
  // Represents seconds of UTC time since Unix epoch
  // 1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
  // 9999-12-31T23:59:59Z inclusive.
  int64 seconds = 1;

  // Non-negative fractions of a second at nanosecond resolution. Negative
  // second values with fractions must still have non-negative nanos values
  // that count forward in time. Must be from 0 to 999,999,999
  // inclusive.
  int32 nanos = 2;
}

This feature request suggests to enhance protovalidate to support this validation out-of-the-box.

Problem it solves or use case:
With the current implementation, developers would have to program this validation outside of protovalidate (or maybe use some complicated CEL expression). In Go, the specification validation is typically done with CheckValid. However, this has some issues:

• Overhead for the developer.
• Inconsistent validation end user experience.

Proposed implementation or solution:
I propose to add an extra validation rule for timestamp. For the implementation I am not sure since I am not experienced with CEL. From what I can see, the general idea is either:

• Have an implementation in CEL using existing expression logic. I tried implementing the constraints as a CEL expression. The validation on seconds seems easy enough, but the nanoseconds seem unavailable in a CEL expression (There might still be a way.)
• Adding a new CEL function and implementing it for each language (like isNaN).

Contribution:
I would be willing to implement this feature; as long as there is consensus on how it should be implemented.

Examples or references:

• protoc-gen-validate-go did seem to produce code which checked the specification validity using CheckValid. See here.

Additional context:

• While this proposal proposes an explicit validation rule, this might actually be something which should be done implicitly. This because the validation 1) is on the specification of the message and 2) is not context-aware.

[Feature Request] ignore `compilation error: no evaluator available for` option

Feature description:
ignore compilation error: no evaluator available for option. I don't want to run fully in Lazy mode, but as we're migrating to protovalidate, not every message will be registered.

As currently written with no codegen, it's hard to write a grpc interceptor to only run validation on messages that have protovalidate options defined. Our interceptor looks like this:

conf.interceptors = append(conf.interceptors, func(c context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
	msg, ok := req.(proto.Message)
	if ok {
		if err := conf.validator.Validate(msg); err != nil {
			return nil, err
		}
	}

	return handler(c, req)
})

If there was codegen, then I could do some type assertion here to check for the Validator interface and skip the call to protovalidate, but that's not available. Adding our own codegen that interprets protovalidate options feels fragile. Maybe we could do some protoreflect magic to check, but that also feels fragile, when the check is already happening inside this library.

Proposed implementation or solution:
A new ValidatorOption IgnoreCompilationErrors or whatever that returns nil instead of an error if a non-registered message is passed through. If that is acceptable, I'd be willing to contribute.

[QUESTION] How to use \ (backslash) in CEL regex with protovalidate?

I tried to write validation for the phone field (E.164 standard)

string phone = 6 [(buf.validate.field).cel = {
    id: "sign_up.phone",
    message: "Phone must be in E.164 format (e.g. [phone])",
    expression: "this.matches('^\\+[1-9]\\d{1,14}$')"
  }];

But I got an error:

{"level":"error","error":"compilation error: failed to compile expression sign_up.phone: ERROR: <input>:1:14: Syntax error: token recognition error at: ''^\\+'\n | this.matches('^\\
+[1-9]\\d{1,14}$')\n | .............^\nERROR: <input>:1:23: Syntax error: token recognition error at: '\\'\n | this.matches('^\\+[1-9]\\d{1,14}$')\n | ......................^\nERROR
: <input>:1:24: Syntax error: mismatched input 'd' expecting ')'\n | this.matches('^\\+[1-9]\\d{1,14}$')\n | .......................^\nERROR: <input>:1:31: Syntax error: token recog
nition error at: '$'\n | this.matches('^\\+[1-9]\\d{1,14}$')\n | ..............................^\nERROR: <input>:1:32: Syntax error: token recognition error at: '')'\n | this.matche
s('^\\+[1-9]\\d{1,14}$')\n | ...............................^","time":"2023-10-31T21:57:51+02:00","message":"failed to validate"}

So, my problem:
I use \ (backslash) in regex in CEL and it should be escaped because I will have no chance to use buf generate. But if I escape it, I have an error. What should I do? Thanks in advance!

[Feature Request] Document `required` on `repeated` fields to validate that a `repeated` field is non-empty.

I'd propose we change the behavior of required on repeated fields to mean non-empty. Justification being principle of least surprise - this is what I'd expect. I haven't thought through the downsides, but I figured worth a quick discussion.

How to use protoc?

I don't use buf to build gen, only protoc.
What can I do?

[BUG] FieldMask example is not working

Description

I tried to validate FieldMask paths as shown in

protovalidate/examples/cel_field_mask.proto

Line 30 in 8151372

google.protobuf.FieldMask field_mask = 2 [(buf.validate.field).cel = {

but it fails at runtime with a compilation error:

compilation error: failed to compile expression valid_field_mask: ERROR: <input>:1:5: unexpected failed resolution of 'google.protobuf.FieldMask'
 | this.paths.all(path, path in ['foo', 'bar'])
 | ....^

Environment

• Operating System: Linux
• Compiler/Toolchain: go1.21.1
• Protobuf Compiler & Version: buf v1.26.1, libprotoc 24.2, protoc-gen-go v1.31.0
• Protovalidate Version: v0.3.2

Additional Context

• protovalidate is used via buf.yaml deps: buf.build/bufbuild/protovalidate
• buf.gen.yaml is not in managed mode

Remove buf.validate.Violations Protobuf message

We don't have a super-concrete use case for having a buf.validate.Violations message, as opposed to using lists of buf.validate.Violation messages. There's a thought that we may want a repeated message wrapper for RPC error details, but we haven't discovered this just yet, and we want to minimize API surface area as much as possible. We'll look to batch this with other breaking changes pre-v1.0.

[Feature Request] Add skipped_fields field constraint for finer-grained skipped behavior

Feature description:

Today, we support the skipped standard constraint on message fields to skip any validation constraints applied to the underlying type. This is currently an all or nothing scenario, when perhaps it's desired to opt-out of one or a few of the message's fields.

Potentially, we can support this via a skipped_fields constraint which takes a FieldMask and can specify which not to evaluate.

Problem it solves or use case:

Example use-case where a message describing a resource elided a constraint for a field and had the RPC request object use a CEL expression to validate that nested field.

Proposed implementation or solution:

# validate.proto

# <snip>

message FieldConstraints {
  # <snip>
  oneof skip {
    bool skipped = 24;
    google.protobuf.FieldMask skipped_fields = XX;
  }
  # <snip>
}

# <snip>

# example.proto

message Foo {
  string id   = 1 [(buf.validate.field).string.uuid = true];
  bytes  data = 2 [(buf.validate.field).bytes.min_len = 42];
}

message Bar {
  Foo foo = 1 [(buf.validate.field).skipped_fields.paths = "id" ];
}

In the example above, Foo by itself will verify the value of the id field to be a UUID and the data field to be of a minimum length. However, if validate is called on a Bar containing a Foo, only the data field will be evaluated as id has been skipped.

[Feature Request] i18n support for error message

Feature description:

It would be good the error message has the i18n feature or can integrate with some i18n libs, then the error message can come out with different language support.

Problem it solves or use case:

Proposed implementation or solution:

This seems not an important feature or even shouldn't be done in this repo, still would be good to have some extra custom fields leave in violations that we can use it as a i18n key or something.

Contribution:

Can help for golang implementation if have a clear goal.

Examples or references:

Additional context:

[Feature Request] Validation rules on methods

Feature description:
Extend the functionality of the protovalidate library to include support for defining and enforcing validation rules on methods within Protocol Buffers service definitions.

Problem it solves or use case:
Currently, the protovalidate library focuses on validating individual message fields, but lacks the ability to validate the parameters and return values of methods defined within Protocol Buffers services. By introducing validation rules on methods, developers can ensure that the inputs and outputs of service methods adhere to specified constraints, enhancing data integrity and API consistency.

Proposed implementation or solution:
The proposed solution involves enhancing the protovalidate library to support validation rules specific to service methods. This includes introducing annotations or configuration options that allow developers to specify validation rules for method parameters and return values within Protocol Buffers service definitions. The library would then enforce these rules during runtime method calls.

Benefits of Method-Level Validation:
Method-level validation provides several benefits, including enhanced data validation, improved API documentation by making validation expectations explicit, and reduced likelihood of malformed data causing errors downstream. This feature also aligns with best practices for ensuring data integrity and robustness in service-oriented architectures.

Examples or references:
Other API frameworks, like gRPC, often include features for specifying validation rules on method parameters and responses. These can serve as references for how such functionality can be integrated into the protovalidate library.

Additional context:
The inclusion of validation rules on methods within the protovalidate library will significantly enhance the data validation capabilities of Protocol Buffers services. Developers will be able to define and enforce constraints on the inputs and outputs of methods, leading to more reliable and robust service interactions. This feature aligns with the evolving demands of API design and data integrity.

[Feature Request] Ruby Support

Any plan to support Ruby or do you accept ruby support PR?

Thanks for the awesome project!

I'm super excited to use this when and where I can. We're gradually moving towards a microservice architecture where all the APIs are defined in protobuf (which is amazing) but ofc each service has to do message validation on it's own :/

[Feature Request] Typescript Support

Feature description:
Introduce TypeScript support to the protovalidate library for performing runtime validation of Protocol Buffers messages in TypeScript projects. This was a feature request in protovalidates predecessor bufbuild/protoc-gen-validate#71

Problem it solves or use case:
The absence of TypeScript support in the protovalidate library hinders developers who work with Protocol Buffers and TypeScript. By adding TypeScript support, developers would be able to validate Protocol Buffers messages at runtime within their TypeScript applications, enhancing data integrity and reducing potential bugs related to data validation.

Proposed implementation or solution:
The proposed solution involves extending the existing protovalidate library to include TypeScript bindings and support. This includes defining TypeScript type definitions that correspond to Protocol Buffers message structures, and implementing runtime validation functions that work seamlessly with TypeScript types.

Contribution:
I am willing to contribute to this feature request by actively participating in discussions, assisting in defining TypeScript type definitions, and collaborating on the implementation of runtime validation functions for TypeScript. If necessary, I can also assist in writing tests and documentation to ensure the feature is thoroughly covered and well-documented.

Examples or references:
While protovalidate doesn't currently have TypeScript support, similar libraries like protobufjs and protobuf-ts provide inspiration for how TypeScript bindings and runtime validation can be integrated for Protocol Buffers messages in a type-safe manner.

Additional context:
Enabling TypeScript support in the protovalidate library would bridge the gap for developers who use Protocol Buffers in TypeScript projects, allowing them to perform runtime validation without relying on separate code generation processes. This aligns with the trend of utilizing TypeScript's strong typing and safety features in modern software development.

[BUG] unable to use protovalidate-java project with the generated protos

Description

I am trying to replace our custom validation implementation with https://github.com/bufbuild/protovalidate.
so we added the dependency in buf.yaml as in https://github.com/plantoncloud/planton-cloud-apis/blob/feat/PC-1147/proto-validate/buf.yaml#L10 and updated the protos and added a custom-constraint using cel expression on a field as in https://github.com/plantoncloud/planton-cloud-apis/blob/feat/PC-1147/proto-validat[…]lanton/apis/v1/commons/testing/resource/field/input/model.proto

Steps to Reproduce

1. clone https://github.com/swarupdonepudi/protovalidate-test ( I created this simple gradle project which contains just one case to setup reproducibility for this issue and nothing else)
2. run ./gradlew build

Expected Behavior

Successful Build with a running test case

Actual Behavior

Failed test case.

> Task :test FAILED

RequestInputFieldsValidatorTest > testRegexFieldsValidation() FAILED
    java.lang.IllegalArgumentException at RequestInputFieldsValidatorTest.java:20

1 test completed, 1 failed

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/swarup/scm/github.com/swarupdonepudi/protovalidate-test/build/reports/tests/test/index.html

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 913ms
3 actionable tasks: 2 executed, 1 up-to-date

Screenshots/Logs

• Contents of test file are at https://github.com/swarupdonepudi/protovalidate-test/blob/main/src/test/java/cloud/planton/commons/lib/protovalidate/RequestInputFieldsValidatorTest.java

• Test case is failing with following error

mergeFrom(Message) can only merge messages of the same type.
java.lang.IllegalArgumentException: mergeFrom(Message) can only merge messages of the same type.
	at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:375)
	at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:370)
	at build.buf.validate.FieldConstraints$Builder.mergeFrom(FieldConstraints.java:1829)
	at build.buf.validate.FieldConstraints$Builder.mergeFrom(FieldConstraints.java:1539)
	at com.google.protobuf.GeneratedMessage$GeneratedExtension.singularFromReflectionType(GeneratedMessage.java:1801)
	at com.google.protobuf.GeneratedMessage$GeneratedExtension.fromReflectionType(GeneratedMessage.java:1785)
	at com.google.protobuf.GeneratedMessageV3$ExtendableMessage.getExtension(GeneratedMessageV3.java:1148)
	at com.google.protobuf.GeneratedMessageV3$ExtendableMessage.getExtension(GeneratedMessageV3.java:1215)
	at build.buf.protovalidate.internal.evaluator.ConstraintResolver.resolveFieldConstraints(ConstraintResolver.java:74)
	at build.buf.protovalidate.internal.evaluator.EvaluatorBuilder.processFields(EvaluatorBuilder.java:156)
	at build.buf.protovalidate.internal.evaluator.EvaluatorBuilder.buildMessage(EvaluatorBuilder.java:113)
	at build.buf.protovalidate.internal.evaluator.EvaluatorBuilder.build(EvaluatorBuilder.java:98)
	at build.buf.protovalidate.internal.evaluator.EvaluatorBuilder.loadOrBuildDescriptor(EvaluatorBuilder.java:384)
	at build.buf.protovalidate.internal.evaluator.EvaluatorBuilder.load(EvaluatorBuilder.java:83)
	at build.buf.protovalidate.Validator.validate(Validator.java:76)
	at cloud.planton.commons.lib.protovalidate.RequestInputFieldsValidatorTest.testRegexFieldsValidation(RequestInputFieldsValidatorTest.java:20)

Environment

• Operating System: MacOS
• Version: MacOS Sonoma
• Compiler/Toolchain: Buf BSR
• Protobuf Compiler & Version: Buf BSR
• Protoc-gen-validate Version: n/a
• Protovalidate Version: 0.1.4

Possible Solution

None We can think of.

Additional Context

• Upon some googling we landed on the following issues reported on the project which seemed very related to the issue that we are running into. But unable to really understand how to fix it.
  • bufbuild/protovalidate-java#12
  • #62

During debugging, we noticed that the descriptor comparison is failing primarily because of different packages for the same class.
In one descriptor proto buf/validate/validate.proto the field options are

options {
  java_package: "build.buf.gen.buf.validate"
  java_outer_classname: "ValidateProto"
  java_multiple_files: true
  go_package: "buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go/buf/validate"
}

In another descriptor proto the filed options are

options {
  java_package: "build.buf.validate"
  java_outer_classname: "ValidateProto"
  java_multiple_files: true
  go_package: "buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go/buf/validate"
}

The main difference there is java_package: "build.buf.validate" while in another it is java_package: "build.buf.gen.buf.validate"

[Feature Request] unique repeated messages by field

Feature description:
There's already a built in rule for scalar uniqueness in repeated fields. Most repeated messages have an id field that defines uniqueness, that could be used.
https://github.com/bufbuild/protovalidate/blob/main/proto/protovalidate/buf/validate/validate.proto#L3199-L3213

Proposed implementation or solution:

This isn't valid CEL, just a rough description of what I'm expecting

  optional bool unique = 3 [(priv.field).cel = {
    id: "repeated.unique_by_field"
    message: "repeated value must contain unique items, identified by an id field"
    expression: "this.unique(this.unique_id)"
  }];

Contribution:
Minimal time to contribute currently

Rework buf.validate.Violation.field_path field

This field effectively has its own custom encoding to represent a field path. Instead, we should attempt to encode what we need to using Protobuf types.

[BUG] Migration tool panics on repeated validation rule

Description

Running protovalidate-migrate -w ./ against the example protobuf raises a panic.

Steps to Reproduce

1. Install the latest migration tool go install github.com/bufbuild/protovalidate/tools/protovalidate-migrate@latest
2. Migate protovalidate-migrate -w ./

syntax = "proto3";

package foo;

import "validate/validate.proto";

message SomeRequest {
  message SomeEntry {
    repeated string actions = 1 [(validate.rules).repeated = {
      unique: true,
      min_items: 1,
      max_items: 10,
      items {
        string {min_len: 1}
      }
    }];
  }
}

Actual Behavior

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x102da6300]

goroutine 19 [running]:
github.com/bufbuild/protocompile/ast.(*RuneNode).Start(0x1?)
	<autogenerated>:1
github.com/bufbuild/protocompile/ast.(*FileInfo).NodeInfo(0x140000c60e0, {0x102f5da28, 0x0})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/file_info.go:152 +0x30
github.com/bufbuild/protocompile/ast.(*FileNode).NodeInfo(...)
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/file.go:97
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.PrinterVisitor.PrintNodes({{0x0, 0x1, 0x0, {0x0, 0x0}, {0x102e94f9f, 0x17}, {0x102e960a6, 0x1b}, 0x0, ...}, ...}, ...)
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/printer.go:38 +0x9c
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.(*MessageLiteralVisitor).VisitMessageFieldNode(0x140000bed10, 0x140000ce580)
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/field.go:256 +0x568
github.com/bufbuild/protocompile/ast.Visit({0x102f5daa0?, 0x140000ce580?}, {0x102f647c8?, 0x140000bed10?})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/walk.go:173 +0x5c0
github.com/bufbuild/protocompile/ast.VisitChildren({0x102f5e5c8?, 0x140000ac6c0?}, {0x102f647c8, 0x140000bed10})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/walk.go:231 +0x7c
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.HandleMessageLiteral({{0x0, 0x1, 0x0, {0x0, 0x0}, {0x102e94f9f, 0x17}, {0x102e960a6, 0x1b}, 0x0, ...}, ...}, ...)
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/field.go:190 +0xd8
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.(*FieldVisitor).HandleOption(0x1400012e200, 0x140000d03c0, 0x0, {0x102f5da28, 0x140000a6560})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/field.go:116 +0x500
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.(*FieldVisitor).VisitCompactOptionsNode(0x1400012e200, 0x140000ac720)
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/field.go:45 +0x150
github.com/bufbuild/protocompile/ast.Visit({0x102f5df28?, 0x140000ac720?}, {0x102f643d8?, 0x1400012e200?})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/walk.go:117 +0x160
github.com/bufbuild/protocompile/ast.VisitChildren({0x102f5e508?, 0x140000c61c0?}, {0x102f643d8, 0x1400012e200})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/walk.go:231 +0x7c
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.(*MessageVisitor).VisitFieldNode(0x1400012e180, 0x80?)
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/message.go:62 +0xa8
github.com/bufbuild/protocompile/ast.Visit({0x102f5de38?, 0x140000c61c0?}, {0x102f64678?, 0x1400012e180?})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/walk.go:129 +0x250
github.com/bufbuild/protocompile/ast.VisitChildren({0x102f5e598?, 0x140000d0460?}, {0x102f64678, 0x1400012e180})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/walk.go:231 +0x7c
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.(*MessageVisitor).VisitMessageNode(0x1400012e100, 0x80?)
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/message.go:58 +0xb0
github.com/bufbuild/protocompile/ast.Visit({0x102f5df00?, 0x140000d0460?}, {0x102f64678?, 0x1400012e100?})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/walk.go:119 +0x188
github.com/bufbuild/protocompile/ast.VisitChildren({0x102f5e598?, 0x140000d0500?}, {0x102f64678, 0x1400012e100})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/walk.go:231 +0x7c
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.(*FileVisitor).VisitMessageNode(0x1400012e080, 0x140000d00a0?)
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/file.go:50 +0xa8
github.com/bufbuild/protocompile/ast.Visit({0x102f5df00?, 0x140000d0500?}, {0x102f64138?, 0x1400012e080?})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/walk.go:119 +0x188
github.com/bufbuild/protocompile/ast.VisitChildren({0x102f5e4d8?, 0x140000d05a0?}, {0x102f64138, 0x1400012e080})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/walk.go:231 +0x7c
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.(*RootVisitor).VisitFileNode(0x1400012e000, 0x140000d05a0)
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/visitor.go:57 +0x278
github.com/bufbuild/protocompile/ast.Visit({0x102f5d960?, 0x140000d05a0?}, {0x102f64288?, 0x1400012e000?})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/[email protected]/ast/walk.go:103 +0x48
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.(*Migrator).MigrateFile(0x140000ac2a0, {0x140000a8080, 0x31}, {0x102f5cf38?, 0x1400009a088?}, {0x102f5cf18?, 0x1400009a090})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/migrator.go:171 +0x1bc
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.(*Migrator).InPlaceMigrate(0x0?, {0x102f5ff78, 0x140000a0f70}, {0x140000a8080, 0x31})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/migrator.go:130 +0x2c8
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.(*Migrator).migrate(0x140000ac2a0, {0x16d15b621?, 0x0?}, {0x140000a8080, 0x31}, {0x102f5f728?, 0x140000ce2c0?})
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/migrator.go:89 +0x284
github.com/bufbuild/protovalidate/tools/protovalidate-migrate/internal/migrator.(*Migrator).Migrate.func1.1()
	/Users/robcrowe/go/pkg/mod/github.com/bufbuild/protovalidate/[email protected]/protovalidate-migrate/internal/migrator/migrator.go:69 +0x38
golang.org/x/sync/errgroup.(*Group).Go.func1()
	/Users/robcrowe/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x58
created by golang.org/x/sync/errgroup.(*Group).Go in goroutine 1
	/Users/robcrowe/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:72 +0x98

[Feature Request] Lint cel expression

Feature description:

I would be happy if cel expression validation is supported.

Problem it solves or use case:

When I buf lint the following proto, the second mistake is caught by buf lint proto/xxx/greeting/v1/greeting.proto:35:79:field xxx.greeting.v1.SayHelloResponse.request_time: option (buf.validate.field).timestamp.lt_nowwww: field lt_nowwww of buf.validate.TimestampRules does not exist, but the first one (cel expression) is not. I'd like to know if the expression is valid or not.

message SayHelloResponse {
  string message_id = 1;

  string message = 2 [(buf.validate.field).cel = {
    id: "sayHelloResponse.message",
    message: "message must start with 'Hello, ' and shorter than 1024",
    expression: "(this.startsWith('Hello, ') && this.string.max_lennnn "
  }];

  google.protobuf.Timestamp request_time = 3  [(buf.validate.field).timestamp.lt_nowwwwwww = true];
}

Proposed implementation or solution:

Is it possible to validate expression like other tools like linting IAM condition that checks cel validity?
https://cloud.google.com/iam/docs/linting-policies

Contribution:

Examples or references:

Additional context:

[Question] buf/validate/validate.proto: does not exist

I have been trying to migrate a project to protovalidate. During the process, I encountered the following error;

<proto file>:buf/validate/validate.proto: does not exist

I have created a dummy repository that closely resembles the project I am trying to migrate;
https://github.com/oguzhand95/buftest

The file structure for the dummy project;

go.mod
Makefile
buf.work.yaml
buf.gen.yaml
api/
    public/
        buf.yaml
        buf.lock
        buftest/
            auth/
                v1/
                    auth.proto

Probably I am doing some silly thing to cause this error, but I'd really appreciate it if anyone could point me in the right direction.

[BUG] Groups (proto2-only feature) are not consistently handled

Groups are not handled uniformly in the various implementations of protovalidate.

In particular, it looks like the Go and Java implementations do not correctly support groups and will skip validating the data inside a group field. However, Python and C++ implementations do not appear to have this defect.

Test cases should likely be added to the conformance tests to verify they work correctly across all implementations.

In particular, both Go and Java implementations only do "embedded message" validation when the field's type is "message". However, group fields will have a type of "group". The Go code uses an thin abstraction over google.protobuf.protobuf.FieldDescriptorProto.Type named protoreflect.Kind. Similarly, the Java code uses a thin abstraction in the Java enum Descriptor.FieldDescriptor.Type. But they both map to the type enum in descriptor.proto, where message and group each have a distinct value (necessary because though they are structurally the same, representing nested messages, they use different wire encoding formats).

• Relevant Go code: https://github.com/bufbuild/protovalidate-go/blob/main/internal/evaluator/builder.go#L303
• Relevant Java code: https://github.com/bufbuild/protovalidate-java/blob/main/src/main/java/build/buf/protovalidate/internal/evaluator/EvaluatorBuilder.java#L242

The Python code, however, simply asks if the field descriptor has an associated message type, which it will for both groups and messages. And the C++ code instead looks at the field descriptor's cpp_type(), which will be CPPTYPE_MESSAGE for both messages and groups. So these two implementations appear to correctly support groups. (A new conformance test would be the best way to verify and assert this to be true.)

• Relevant Python code: https://github.com/bufbuild/protovalidate-python/blob/main/protovalidate/internal/constraints.py#L713
• Relevant C++ code: https://github.com/bufbuild/protovalidate-cc/blob/main/buf/validate/validator.cc#L44

Return Value in Error Message when Validation Fails

Feature description:
The proto validation tooling is fantastic! It's super helpful and easy to understand what the core issue is. Something that would make it even better would be to add the actual value to the message that failed validation.

Problem it solves or use case:
While it's currently very helpful to be able to see exactly which field failed validation, it's VERY difficult to figure out what the original value was. We've had to go to some extreme lengths to print out the entire proto object client-side when a field validation error occurs. Instead, it would be SO much easier if the failure message included the original value that failed the validation. It may make sense as well to make the option configurable in case there are concerns with PII being in the failure message.

Proposed implementation or solution:
Here's a simple example of an orderId not passing the regex validation that would be very helpful to add the failing orderId to the message.

Original Message

"invalid ClickstreamEvent.OrderEvent: embedded message failed validation | caused by: invalid OrderEvent.OrderId: embedded message failed validation | caused by: invalid OrderId.Value: value does not match regex pattern \"^[0-9a-fA-F]{2,64}$\""

Modified Message

"invalid ClickstreamEvent.OrderEvent: embedded message failed validation | caused by: invalid OrderEvent.OrderId: embedded message failed validation | caused by: invalid OrderId.Value: \"INVALID_ORDER_ID\" does not match regex pattern \"^[0-9a-fA-F]{2,64}$\""

In the event that the orderId was empty, it would be helpful to explicitly state that as well.

[BUG] go install issue

Sorry, I'm really new to golang & protobuf in general, I'm trying to install this module using 'go install'. Firstly, is that best approach? If so, this is what i'm seeing:

Description

I'm trying to install with 'go install' but it's failing.

Steps to Reproduce

❯ go install github.com/bufbuild/protovalidate@latest
go: github.com/bufbuild/protovalidate@latest: github.com/bufbuild/[email protected]: parsing go.mod:
module declares its path as: github.com/envoyproxy/protoc-gen-validate
but was required as: github.com/bufbuild/protovalidate

Expected Behavior

I expected the module to be installed

Actual Behavior

It wasn't installed

Screenshots/Logs

n/a

Environment

• Operating System: macOS
• Version: Ventura 13.6
• Compiler/Toolchain: golang
• Protobuf Compiler & Version: libprotoc 3.21.12
• Protovalidate Version: latest

[BUG] No-op validation functions being generated

Description

No validations checks appear in the generated in code. Instead I get "// no validation rules for X" comments.

Steps to Reproduce

use this proto definition

syntax = "proto3";
import "buf/validate/validate.proto";
message HomePageRequest {
  uint64 id = 1 [(buf.validate.field).uint64.gt = 999, (buf.validate.field).required = true];
}

Expected Behavior

The validate function should have a check for greater than 999 and a require check.

Actual Behavior

This is the behavior I get in the generated code

func (m *HomePageRequest) validate(all bool) error {
	if m == nil {
		return nil
	}

	var errors []error

	// no validation rules for Id

	if len(errors) > 0 {
		return HomePageRequestMultiError(errors)
	}

	return nil
}

Screenshots/Logs

Environment

• Operating System: macOS M2
• Version: MacOS 13.2.1 (22D68)
• Compiler/Toolchain:
• Protobuf Compiler & Version:libprotoc 23.3
• Protoc-gen-validate Version:
• Protovalidate Version:v1.0.2

Please let me know if I'm am missing something obvious.

[BUG] Cannot install package

Description

go: github.com/bufbuild/[email protected]: parsing go.mod:
module declares its path as: github.com/envoyproxy/protoc-gen-validate
but was required as: github.com/bufbuild/protovalidate

Steps to Reproduce

Expected Behavior

I can install protovalidate

Actual Behavior

go: github.com/bufbuild/[email protected]: parsing go.mod:
module declares its path as: github.com/envoyproxy/protoc-gen-validate
but was required as: github.com/bufbuild/protovalidate

Environment

• Operating System: macOS
• Version:
• Compiler/Toolchain:
• Protobuf Compiler & Version:
• Protoc-gen-validate Version:
• Protovalidate Version:

Possible Solution

Additional Context

[Feature Request] C# Support

Original Post in PGV: bufbuild/protoc-gen-validate#480

Feature description:
Extend the capabilities of the protovalidate library to include support for performing runtime validation of Protocol Buffers messages in C# projects. Additionally, integrate a CEL (Common Expression Language) interpreter, currently under development at https://github.com/rofrankel/cel-csharp, to facilitate advanced validation expressions.

Problem it solves or use case:
The absence of C# support in the protovalidate library limits developers working with Protocol Buffers and C#. By adding C# support, developers can perform runtime validation of Protocol Buffers messages directly within their C# applications. Furthermore, integrating the CEL interpreter enables complex validation expressions, enhancing data validation capabilities.

Proposed implementation or solution:
The proposed solution involves extending the protovalidate library to include C# bindings and compatibility, which will involve defining C# classes or structures that correspond to Protocol Buffers message structures. Additionally, integrating the CEL interpreter from https://github.com/rofrankel/cel-csharp will facilitate advanced validation expressions within the protovalidate library.

Contribution:
I'm excited about contributing to this feature by participating in discussions around C# bindings, assisting in the integration of the CEL interpreter, and collaborating on the implementation of runtime validation functions within the protovalidate library. If necessary, I can also assist in writing tests and documentation to ensure comprehensive coverage and guidance.

Examples or references:
While the protovalidate library currently does not support C#, reference points such as protobuf-net can be valuable when considering how Protocol Buffers and C# can be integrated. The ongoing development of the CEL interpreter for C# (https://github.com/rofrankel/cel-csharp) can serve as a foundation for enabling complex validation expressions.

Additional context:
Enabling C# support in the protovalidate library aligns with the needs of developers who work with C# and Protocol Buffers. The integration of the CEL interpreter further enhances the library's capabilities, allowing developers to define sophisticated validation expressions. This feature contributes to more robust and accurate data validation, aligning with modern software development practices in the C# ecosystem.

Rename buf.validate.Violation.for_key field

We'd like to see if we can come up with something more self-describing.

[Feature Request] Support for validation of NaN

Feature description:
Enhance the functionality of the protovalidate library by introducing support for validation of NaN (Not-a-Number) values in Protocol Buffers messages across all supported programming languages. This issue was originally raised in bufbuild/protoc-gen-validate#85

Problem it solves or use case:
Currently, the protovalidate library lacks built-in support for validating NaN values, which can lead to potential issues when dealing with floating-point numbers in Protocol Buffers messages. By incorporating NaN validation, developers can ensure the accuracy of floating-point data, prevent unexpected behavior, and maintain data integrity.

Proposed implementation or solution:
The proposed solution involves extending the protovalidate library to include specialized NaN validation logic for floating-point fields in Protocol Buffers messages. Additionally, to facilitate consistent validation across different languages, an isNaN(x) function needs to be implemented as a built-in part of the protovalidate library for every supported programming language.

Contribution:
I'm enthusiastic about contributing to this feature by participating in discussions regarding the best approach to implement NaN validation for floating-point fields. I can also assist in designing the isNaN(x) function and collaborating on its integration into the protovalidate library across various languages.

Examples or references:
Languages like JavaScript, Python, and C++ already have built-in isNaN(x) functions that could serve as references for implementing this feature. Additionally, the IEEE 754 standard for floating-point arithmetic provides guidance on handling special values like NaN.

Additional context:
Adding support for NaN validation and the corresponding isNaN(x) function would significantly enhance the data validation capabilities of the protovalidate library, particularly in scenarios involving floating-point data. This feature aligns well with the industry's emphasis on precision and correctness in handling numerical data across programming languages.

Support fieldmask / Better handling of gRPC Update request messages

It would be great if protovalidate supports fieldmask. I.e. if a call to the validator can include a fieldmask, and only the fields covered by the fieldmask are validated.

This would solve the problem of gRPC Update calls, where the Update contains only a subset of the fields, so a full validation will fail.

There's a few workarounds, sometimes using ignore_empty helps, sometimes it doesn't.
Also adding CEL expressions to the UpdateXYRequest works, but it's not a replacement for field-level rules imho.

[Feature] ip_prefix library implementations

Tracking library implementations of ip_prefix and friends added in #99 (released v0.5.0 and v0.5.1 for the conformance suite)

• bufbuild/protovalidate-go#53 (released v0.4.0)
• bufbuild/protovalidate-python#78 (released v0.3.0)
• bufbuild/protovalidate-java#39 (released v0.1.7)
• bufbuild/protovalidate-cc#45 (in progress)

cc @higebu @paina

[BUG] Proto sources' lack of `options` interferes with protoc codegen for Java under Bazel.

Description

I tried to integrate protovalidate-java into my existing monorepo, which uses Bazel for builds. The generated Java code for my protos annotated with proto field/message options from this repo are incompatible with the JARs published through Maven. I believe this is because the proto sources in this repository lack the options that would cause protoc's Java plugin to generate/expect classes with the correct fully-qualified names.

Steps to Reproduce

1. Import protovalidate protos to Bazel WORKSPACE.

http_archive(
    name = "protovalidate",
    sha256 = "727601c86e8f7b0ea1a00952d431450948933db44f26cf4fab046492f1d1b029",
    strip_prefix = "protovalidate-0.2.7",
    urls = ["https://github.com/bufbuild/protovalidate/archive/refs/tags/v0.2.7.tar.gz"],
)

2. Add validation options to the project's protos and take a dep from the corresponding Bazel proto_librarys.

 syntax = "proto2";
 
 package my.pkg;
 
+import "buf/validate/validate.proto";
+
 option java_multiple_files = true;
 option java_package = "com.myorg.proto";
 
 message Foo {
-  optional fixed64 some_field = 1;
+  optional fixed64 some_field = 1 [(buf.validate.field).cel = {
+    id: "some_field.is_positive",
+    message: "Try to stay positive!",
+    expression: "this > 0",
+  }];
 }

 proto_library(
     name = "my_proto",
     srcs = [
       "my_protos.proto",
     ],
+    deps = [
+	"@protovalidate//proto/protovalidate/buf/validate:validate_proto",
+    ],
 )

3. Set up rules_jvm_external for protovalidate dependency.

maven_install(
  name = "maven",
  artifacts = [
    "build.buf:protovalidate:0.0.1",
  ],
  repositories = [
    "https://repo1.maven.org/maven2",
    "https://buf.build/gen/maven",
  ],
)

4. Call into build.buf.protovalidate.Validator from application.
   Also update corresponding java_library deps

 java_library(
   name = "my_app",
   srcs = ["MyApp.java"],
   deps = [
     "//protos:my_java_proto",
+    "@maven//:build_buf_protovalidate",
     ],
 )

5. Rebuild/run.

Expected Behavior

Validation logic runs. Accepts message or throws an exception from build.buf.protovalidate.exceptions (not 100% sure if I've specified the option correctly, but that's not important yet).

Actual Behavior

Runtime error when decoding performing validation:

java.lang.IllegalArgumentException: mergeFrom(Message) can only merge messages of the same type.
	at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:385)
	at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:370)
	at build.buf.gen.buf.validate.MessageConstraints$Builder.mergeFrom(MessageConstraints.java:541)
	at build.buf.gen.buf.validate.MessageConstraints$Builder.mergeFrom(MessageConstraints.java:411)
	at com.google.protobuf.GeneratedMessage$GeneratedExtension.singularFromReflectionType(GeneratedMessage.java:1801)
	at com.google.protobuf.GeneratedMessage$GeneratedExtension.fromReflectionType(GeneratedMessage.java:1785)
	at com.google.protobuf.GeneratedMessageV3$ExtendableMessage.getExtension(GeneratedMessageV3.java:1132)
	at com.google.protobuf.GeneratedMessageV3$ExtendableMessage.getExtension(GeneratedMessageV3.java:1199)
	at build.buf.protovalidate.internal.evaluator.ConstraintResolver.resolveMessageConstraints(ConstraintResolver.java:40)
	at build.buf.protovalidate.internal.evaluator.EvaluatorBuilder.buildMessage(EvaluatorBuilder.java:107)
	at build.buf.protovalidate.internal.evaluator.EvaluatorBuilder.build(EvaluatorBuilder.java:98)
	at build.buf.protovalidate.internal.evaluator.EvaluatorBuilder.loadOrBuildDescriptor(EvaluatorBuilder.java:384)
	at build.buf.protovalidate.internal.evaluator.EvaluatorBuilder.load(EvaluatorBuilder.java:83)
	at build.buf.protovalidate.Validator.validate(Validator.java:76)

Environment

• Operating System: macOS
• Version: macOS 13.4.1
• Compiler/Toolchain: openjdk 11.0.19 2023-04-18 LTS
• Protobuf Compiler & Version: protoc 3.20.0
• Protoc-gen-validate Version: N/A
• Protovalidate Version: v0.2.7

Possible Solution

Either:

1. Preferred: Add protooptions for Java codegen. e.g.

+option java_multiple_files = true;
+option java_package = "build.buf.gen.buf.validate";
+option java_outer_classname = "ValidateProto";

2. Alternate: Published JARs on Maven should use protoc-compatible Java codegen.

Additional Context

The Java proto runtime (com.google.protobuf:protobuf-java) uses descriptor reference identity to perform type comparisons between message objects. When we have multiple classes for the same message type (because they were generated with different Java codegen settings), we can get type-mismatch errors when decoding messages even when compatible types are used.

In the case described above, the IllegalArgumentException was triggered when merging from an instance of class buf.validate.Validate$MessageConstraints to an instance of class build.buf.gen.buf.validate.MessageConstraints$Builder; both of these types purport to be for message buf.validate.MessageConstraints, but they come from different JARs and their descriptors reference-compare as inequal (even if the contents are probably identical).

This occurs because Bazel's java_proto_library rule generated its own copies of the buf.validate.* message classes, under the packages and names it expected. This happens even without there being java_proto_library rules instantiated for "@protovalidate//proto/protovalidate/buf/validate:validate_proto" and friends; code generation occurs for the entire proto dependency tree using Bazel's aspects mechanism.

Non-solution: excluding one set of codegened proto classes

I tried excluding either side of the message class conflict; neither approach works.

Thebuild.buf.gen.buf.validate.* classes (from Maven) can be excluded using the exclude_artifacts option on maven_install. This leads to a ClassNotFoundException when using build.buf.protovalidate.Validator:

java.lang.NoClassDefFoundError: build/buf/gen/buf/validate/ValidateProto
	at build.buf.protovalidate.internal.evaluator.EvaluatorBuilder.<clinit>(EvaluatorBuilder.java:51)
	at build.buf.protovalidate.Validator.<init>(Validator.java:54)
	... 25 trimmed
Caused by: java.lang.ClassNotFoundException: build.buf.gen.buf.validate.ValidateProto
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 29 more

Thebuf.validate.* classes (from local protoc) can be excluded using a proto_lang_toolchain with a custom blacklisted_protos to suppress codegen on buf.validate proto libs. This leads to compilation errors running javac for dependent protos (proto sources that have used buf.validate options):

error: package buf.validate does not exist
          buf.validate.Validate.getDescriptor(),

I think this gets to the heart off the matter: protoc Java codegen produces references into buf.validate when we use this package's messages as options on our own protos. If the corresponding Java classes for those aren't where protoc expected them to be, we'll get errors. Conversely, the build.buf:protovalidate on Maven has different expectations for the packages and names for these classes and it won't work if its expectations are violated. We need some way to reconcile these expectations.

Workaround: patching

A hacky way to reconcile the expecations locally is to patch one of the two inputs to the build: either the proto sources (through the patches option on http_archive) or the JARs (via shading). I found the former to be easier. I'm currently having some success using this patch:

diff -ru protovalidate-0.2.7/proto/protovalidate/buf/validate/expression.proto protovalidate-0.2.7.patched/proto/protovalidate/buf/validate/expression.proto
--- proto/protovalidate/buf/validate/expression.proto	2023-07-20 18:25:37
+++ proto/protovalidate/buf/validate/expression.proto	2023-07-26 01:24:16
@@ -17,6 +17,9 @@
 package buf.validate;
 
 option go_package = "buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go/buf/validate";
+option java_multiple_files = true;
+option java_package = "build.buf.gen.buf.validate";
+option java_outer_classname = "ExpressionProto";
 
 // `Constraint` represents a validation rule written in the Common Expression
 // Language (CEL) syntax. Each Constraint includes a unique identifier, an
diff -ru protovalidate-0.2.7/proto/protovalidate/buf/validate/priv/private.proto protovalidate-0.2.7.patched/proto/protovalidate/buf/validate/priv/private.proto
--- proto/protovalidate/buf/validate/priv/private.proto	2023-07-20 18:25:37
+++ proto/protovalidate/buf/validate/priv/private.proto	2023-07-26 01:24:25
@@ -19,6 +19,9 @@
 import "google/protobuf/descriptor.proto";
 
 option go_package = "buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go/buf/validate/priv";
+option java_multiple_files = true;
+option java_package = "build.buf.gen.buf.validate.priv";
+option java_outer_classname = "PrivateProto";
 
 extend google.protobuf.FieldOptions {
   // Do not use. Internal to protovalidate library
diff -ru protovalidate-0.2.7/proto/protovalidate/buf/validate/validate.proto protovalidate-0.2.7.patched/proto/protovalidate/buf/validate/validate.proto
--- proto/protovalidate/buf/validate/validate.proto	2023-07-20 18:25:37
+++ proto/protovalidate/buf/validate/validate.proto	2023-07-26 01:24:33
@@ -23,6 +23,9 @@
 import "google/protobuf/timestamp.proto";
 
 option go_package = "buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go/buf/validate";
+option java_multiple_files = true;
+option java_package = "build.buf.gen.buf.validate";
+option java_outer_classname = "ValidateProto";
 
 // MessageOptions is an extension to google.protobuf.MessageOptions. It allows
 // the addition of validation rules at the message level. These rules can be

If I don't do anything else, I still have duplicate JARs on my classpath. However, now they're identical, and because the names match we'll only wind up loading one set at runtime. This duplication could be eliminated by excluding either of the sets of classes, as discussed above, now with no error.

proto/buf/validate/validate.proto: does not exist

Description

I am somehow making it so that I can't use the new validate beta

Steps to Reproduce

1. Create auth.proto file in proto/auth/v1/auth.proto

syntax = "proto3";

package auth.v1;

import "buf/validate/validate.proto";

2. My buf.gen

version: v1
managed:
  enabled: true
  go_package_prefix:
    default: github.com/SuppaDuppa/leprechaun/api/gen
plugins:
  - plugin: buf.build/protocolbuffers/go
    out: gen
    opt: paths=source_relative
  - plugin: buf.build/bufbuild/connect-go
    out: gen
    opt: paths=source_relative

buf.lock

# Generated by buf. DO NOT EDIT.
version: v1
deps:
  - remote: buf.build
    owner: bufbuild
    repository: protovalidate
    commit: ca37dc8895db4729ac94f62f00fbd994
    digest: shake256:42c51d9091a313379dfdb0482cd86d2de412eb856721ac637bc8ff986694fd27924214440681f678c4c9dd2c24af0b4f59caf53e60d6fd1caa48c2d66d6c0425

buf.work

version: v1
directories:
  - proto

buf.yaml

version: v1
deps:
  - buf.build/bufbuild/protovalidate
breaking:
  use:
    - FILE
lint:
  use:
    - DEFAULT
  ignore:
    - buf.build/bufbuild/protovalidate

Expected Behavior

Being able to run buf generate

Actual Behavior

proto/auth/v1/auth.proto:5:8:proto/buf/validate/validate.proto: does not exist

Environment

• Operating System: Linux WSL
• Version: Ubuntu 22.04.2 LTS
• Compiler/Toolchain: gcc --version gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
• Protobuf Compiler & Version: buf 1.20
• Protovalidate Version: 0.1.5 (i just re-updated)

[Feature Request] Add support for ULID as a well_known validation rule

Feature description:

Support for ULID as a well_known validation rule.

Problem it solves or use case:

ULID aims to be a unique identifier that can be generated on independent machines, much like UUID. However, the first component is generated from the timestamp (and the second half is meant to be monotonic for a given timestamp on a given generator, although this is not often followed in practice). This has the effect that it is better suited to use cases where ordering helps with performance, particularly database indices.

https://github.com/ulid/spec

It is possible to use the existing regex validation facility, but doing this separately to ever ULID field is quite redundant and noisy. Additionally, marking a field as ULID carries some semantic information (e.g. for generated documentation). Basically: the standard arguments for adding something as a well-known type.

Proposed implementation or solution:

Add ULID to the well_known oneof, and its implementation could be the following regex or equivalent (this is Python syntax):

[0-7][0-9A-HJKMNP-TV-Z]{25}

Alternatively, for a case insensitive implementation (this is what the ULID spec recommends but is probably less useful in a data interchange format, especially to optimise use as a database index):

[0-7][0-9A-HJKMNP-TV-Za-hjkmnp-tv-z]{25}

Contribution:

Unfortunately I can't contribute an implementation, but would be happy to help with testing.

Examples or references:

Similar request for old protoc-gen-validate (issue 278)

[Question] why buf/validate/validate.proto use optional？

Description

I use protoc to gen code，get an error

buf/validate/validate.proto:37:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:47:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:57:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:72:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:112:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:208:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:403:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:595:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:781:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:967:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:1153:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:1337:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:1522:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:1707:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:1892:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:2077:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:2262:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:2448:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.
buf/validate/validate.proto:2466:12: Explicit 'optional' labels are disallowed in the Proto3 syntax. To define 'optional' fields in Proto3, simply remove the 'optional' label, as fields are 'optional' by default.

Steps to Reproduce

Expected Behavior

How to resolv this?

Actual Behavior

Screenshots/Logs

Environment

• Operating System:
• Version:
• Compiler/Toolchain:
• Protobuf Compiler & Version:
• Protoc-gen-validate Version:
• Protovalidate Version:

Possible Solution

Additional Context

[Feature Request] Swift Support

Feature description:
Extend the capabilities of the protovalidate library to include support for performing runtime validation of Protocol Buffers messages in Swift projects.

Problem it solves or use case:
The current lack of Swift support in the protovalidate library poses challenges for developers who work with Protocol Buffers and Swift. By introducing Swift support, developers can validate Protocol Buffers messages at runtime within their Swift applications, enhancing data integrity and reducing potential issues related to data validation.

Proposed implementation or solution:
The proposed solution involves extending the existing protovalidate library to incorporate Swift bindings and compatibility. This includes defining Swift structures or classes that mirror Protocol Buffers message structures, and implementing runtime validation functions that align with Swift's strong typing.

Challenges with Enforcing Validation on the Swift Client:
Enforcing validation on the Swift client could introduce challenges due to the widespread impact of validation changes. Swift's strong typing and compile-time safety would necessitate strict adherence to validation rules. However, making validation changes would lead to breaking changes on both the client and server sides. Any update to validation rules could result in inconsistencies between the client and server if they're not synced properly. This would require maintaining compatibility across different versions of the Protocol Buffers definitions, which could become a complex and error-prone task.

Importance of a CEL Interpreter for Advanced Validation:
To address these challenges, an integral part of the solution would involve creating a CEL (Common Expression Language) interpreter for Swift. CEL allows for the specification of complex validation rules in an expressive and language-agnostic manner. This interpreter would be instrumental in enabling advanced validation expressions that can be used within the protovalidate library. Given that a CEL interpreter for Swift is not currently under development an effort would need to be undertaken to create this

Contribution:
I am enthusiastic about contributing to this feature by participating in discussions, assisting in defining Swift type structures, collaborating on the implementation of runtime validation functions for Swift, and potentially contributing to the development of the CEL interpreter for Swift. I am also willing to help with writing tests and documentation to ensure comprehensive coverage and guidance.

Examples or references:
While protovalidate does not currently offer Swift support, libraries like connect-swift, SwiftProtobuf and gRPC-Swift can provide insights into integrating Protocol Buffers messages into Swift projects.

Additional context:
Adding Swift support to the protovalidate library will cater to developers who use Protocol Buffers in Swift applications, allowing them to perform runtime validation without the need for external code generation. The creation of a CEL interpreter for Swift would further enhance the validation capabilities of the library, enabling advanced validation expressions to be used seamlessly across languages.

[Feature Request] support for Rust language

Feature description:

It would be really nice to support Rust language. There is a number of Protocol Buffer implementations for Rust, and post, rust-protobuf are one of them.

Problem it solves or use case:

Enables Rust ecosystem to have consistent validation for code written in Rust.

Proposed implementation or solution:

Contribution:

Examples or references:

Additional context:

Linking this issue bufbuild/protoc-gen-validate#457 this is from the PGV project for reference

[Feature Request] support for case insensitivity

Feature description:

I would like to do a case insensitive check on certain string operations, for example, a cel expression to check that a string does not have a prefix regardless of casing:

expression: "this.toLower().startsWith('ci_core') ? 'cannot start with ci_core' : ''"

Problem it solves or use case:

there are certain things that are commonly case-insensitive in operating systems, on macos and windows file paths are for example, and env variables on windows are also case insensitive. It is currently difficult to add this type of validation via buf validate

Proposed implementation or solution:

I don't know much about cel but if its possible to add a toLower or toUpper operator to this when it's a string this would make it possible to implement fairly easily

Contribution:
N/A I don't think I have time to dig into implementing this, and there are some complicated implications with this since strings+unicode is a hairy space