Comments (9)
Thanks for you patience and quick responses. I really appreciate it!
I've just played a bit with Protox
in a repl and realised where my problem lies. That's probably isn't about encode
or decode
but about json serialisation. Maybe I give a bit of context why I asked these questions at all. I have some proto messages which are already in production and I'd like to avoid a situation when I deliver a wrong value just because of a typo I made. Besides, I've noticed that serialising to proto and json formats behave a bit differently. This is an example coming from the production code, although a bit obfuscated:
First case, I deliver a correct value:
iex(15)> %Entity{status: :ENABLED} |> Entity.json_encode
{:ok, ["{", ["\"status\"", ":", "\"ENABLED\""], "}"]}
iex(16)> %Entity{status: :ENABLED} |> Entity.encode
{:ok, [[], "0", <<1>>]}
Both versions return sensible results and I am happy.
Second case, I deliver incorrect value:
iex(17)> %Entity{status: :YOGI} |> Entity.json_encode
{:ok, ["{", ["\"status\"", ":", "\"YOGI\""], "}"]}
iex(18)> %Entity{status: :YOGI} |> Entity.encode
{:error,
%Protox.EncodingError{
field: :status,
message: "Could not encode field :status (invalid field value)"
}}
And here I see some kind of inconsistency, since the json version accepts incorrect values. This worries me a bit, since I use the json serialisation even more often than the binary one. To make things even worse, when I try to decode this wrongly created json I get an error as well:
iex(29)> {:ok, iodata} = %Entity{status: :YOGI} |> Entity.json_encode
{:ok, ["{", ["\"status\"", ":", "\"YOGI\""], "}"]}
iex(30)> iodata |> Entity.json_decode
{:error,
%Protox.JsonDecodingError{
message: "Could not decode JSON payload: YOGI is not value of Elixir.ProtobufSchema.EntityStatus"
}}
As you can see the problem with the incorrectly formed json messages arises only when it is deserialised, which seems a bit too late. I believe, the sender should know that a message includes incorrect data and for that reason shouldn't be sent at all.
I don't know if we can do anything about that, but I hope I managed to point out some potential problems relating to the json serialisation (binary one works as expected)
from protox.
But! I realize that you have :YOGI
(the atom) and not "YOGI"
(the string) in %Entity{status: :YOGI}
, which is not exactly the same scenario as I describe above.
I guess it's possible to check that this atom doesn't belong the enum, I'll see what I can do!
from protox.
So, it seems it was quite easy to implement
from protox.
Thank you for your feedback! I'll release a new version as soon as possible.
from protox.
Hello! Good question :-) In fact, we have to do this to support forward compatibility.
Imagine you have an application that has a more recent version of the protobuf definition, with a new entry in the enum. Then, an application with an older version of the definition (thus without the new entry) must be able to parse the new entry without an error.
Does it answer your question?
from protox.
Ha, this makes sense, you're right. But I am still wondering if we really need this last clause for the encode
function? If we got rid of it then, when you create a new message, you could be sure that you use one of the expected values. And if we kept the last clause of the decode
function, the receiver could parse it without any errors. Wouldn't that be forward compatible?
from protox.
It's the same thing with encode
: any unknown enum entry that is received should be sent as is, without tempering with its content (this is the case if your app with the old definition has to transfer the message to another app).
I hope this is clear enough
I think I'll add a note about this in the README.
from protox.
Aah, yes, I feel the sameβ¦
It fails in the binary version is that because there's no way of converting :YOGI
to an integer (enum are transmitted as integers in the binary format).
However, the JSON specification expects the enum entry names to be outputted as string (thus ENABLED
rather than 0
). This means that YOGI
could have been received from an app with a more recent protobuf definition. And thus, this value must be kept when serializing back to JSON.
from protox.
Great, works as expected
iex(4)> %Entity{status: :ENABLED} |> Entity.encode
{:ok, [[], "p", <<1>>]}
iex(5)> %Entity{status: :ENABLED} |> Entity.json_encode
{:ok, ["{", ["\"status\"", ":", "\"ENABLED\""], "}"]}
iex(6)> %Entity{status: :YOGI} |> Entity.json_encode
{:error,
%Protox.JsonEncodingError{
message: "Could not encode to JSON: invalid enum entry value"
}}
However, there is one thing which must be pointed out: this change isn't backward compatible which means all elixir files must be regenerated. Otherwise an exception will be risen:
%Entity{status: :ENABLED} |> Entity.json_encode
** (UndefinedFunctionError) function EntityStatus.has_constant?/1 is undefined or private. Did you mean one of:
* constants/0
Of course, it refers only to those users who use an option of generating elixir files.
from protox.
Related Issues (20)
- Generating code for services? HOT 7
- Supporting FileOptions HOT 9
- key not found when encoding optional fields HOT 10
- Unexpected list in typespec: [:high, :low] HOT 3
- Feature request: ability to ignore unknown fields and not have __uf__ properties in structs HOT 6
- Not seeing mix tasks HOT 6
- protox and the `package` directive (was: Proposal - Support Namespace definition when use files generation) HOT 4
- [BUG] Fix compile warning when use --keep-unknown-fields=false HOT 1
- Fails to generate module HOT 1
- Enable manual import path specification HOT 2
- Proto3 Field Presence HOT 3
- feature request: ability to inject definition into existing module HOT 6
- protox generates nested lists (was: Strange behavior when encoding complex nested struct) HOT 2
- [BUG] README.md contains misleading examples HOT 1
- [BUG] Compilation of Protox 1.6 fails HOT 6
- Inconsistency in generated functions specs? HOT 1
- [Feature Request] support for decimal 1.9.x HOT 4
- [BUG] The code generated is not deterministic HOT 6
- [BUG] HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from protox.