Coder Social home page Coder Social logo

proto3-json-serializer-nodejs's Introduction

proto3 JSON serializer for TypeScript / JavaScript

This library implements proto3 JSON serialization and deserialization for protobuf.js protobuf objects according to the spec.

Note that the spec requires special representation of some google.protobuf.* types (Value, Struct, Timestamp, Duration, etc.), so you cannot just use .toObject() since the result won't be understood by protobuf in other languages. Hence this module.

JavaScript:

const serializer = require('proto3-json-serializer');

TypeScript:

import * as serializer from 'proto3-json-serializer';

Serialization: protobuf.js object to proto3 JSON

const root = protobuf.loadSync('test.proto');
const Type = root.lookupType('test.Message');
const message = Type.fromObject({...});

const serialized = serializer.toProto3JSON(message);

Serialization works with any object created by calling .create(), .decode(), or .fromObject() for a loaded protobuf type. It relies on the $type field so it will not work with a static object.

Deserialization: proto3 JSON to protobuf.js object

To deserialize an object from proto3 JSON, we must know its type (as returned by root.lookupType('...')). Pass this type as the first parameter to .fromProto3JSON:

const root = protobuf.loadSync('test.proto');
const Type = root.lookupType('test.Message');
const json = {...};

const deserialized = serializer.fromProto3JSON(Type, json);

Complete example

const assert = require('assert');
const path = require('path');
const protobuf = require('protobufjs');
const serializer = require('proto3-json-serializer');

// We'll take sample protos from google-proto-files but the code will work with any protos
const protos = require('google-proto-files');

// Load some proto file
const rpcProtos = protos.getProtoPath('rpc');
const root = protobuf.loadSync([
    path.join(rpcProtos, 'status.proto'),
    path.join(rpcProtos, 'error_details.proto'),
]);
const Status = root.lookupType('google.rpc.Status');

// If you have a protobuf object that follows proto3 JSON syntax
// https://developers.google.com/protocol-buffers/docs/proto3#json
// (this is an example of google.rpc.Status message in JSON)
const json = {
    code: 3,
    message: 'Test error message',
    details: [
        {
            '@type': 'google.rpc.BadRequest',
            fieldViolations: [
                {
                    field: 'field',
                    description: 'must not be null',
                },
            ],
        },
    ],
};

// You can deserialize it into a protobuf.js object:
const deserialized = serializer.fromProto3JSON(Status, json);
console.log(deserialized);

// And serialize it back
const serialized = serializer.toProto3JSON(deserialized);
assert.deepStrictEqual(serialized, json);

Disclaimer

This is not an officially supported Google project.

proto3-json-serializer-nodejs's People

Contributors

alexander-fenster avatar eduardobautista avatar fhinkel avatar gcf-owl-bot[bot] avatar parthea avatar release-please[bot] avatar renovate-bot avatar setsun avatar summer-ji-eng avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

proto3-json-serializer-nodejs's Issues

Packing test: should work in TypeScript code failed

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.


commit: 6e20f58
buildURL: Build Status, Sponge
status: failed

Test output
Process 128 exited with code 1.
Error: Process 128 exited with code 1.
    at ChildProcess. (node_modules/pack-n-play/build/src/utils.js:48:22)
        -> /workspace/node_modules/pack-n-play/src/utils.ts:62:11
    at maybeClose (internal/child_process.js:1022:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)

Browser support

I'm aware of the name of the repo ๐Ÿ˜„ , but it would be nice to have direct support for the browser
without a polyfill for Buffer.

Looking at the code it seems that Buffer is only used for base64 encoding/decoding
of binary arrays, something that browser-friendly libraries like https://www.npmjs.com/package/base64-js do well.

Are there any plans? Is this something that you would be open to?

Support "NaN, Infinity and -Infinity" in double value type

Proto3 float, double maps JSON number, need to support NaN, Infinity and -Infinity.

JSON value will be a number or one of the special string values "NaN", "Infinity", and "-Infinity". Either numbers or strings are accepted. Exponent notation is also accepted. -0 is considered equivalent to 0.

several places where the extra checks need to be added:
1 in fromProto3Json conversion, if a type is double and a value is one of the special values like "NaN", put the corresponding Number value instead of the string as is
2 in 'toProto3Json` conversion, do the same (but the other way around)
3 in google.protobuf.Value toJson conversion, do the same

Dependency Dashboard

This issue provides visibility into Renovate updates and their statuses. Learn more

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

  • chore(deps): update actions/setup-node action to v2

  • Check this box to trigger a request for Renovate to run again on this repository

[Feature] Support generated static code for fromProto3JSON

Is your feature request related to a problem? Please describe.

Currently this serializer takes a reflected message Type as input for type in fromProto3JSON(type, json). However, I want to be able to use generated static code as described in https://github.com/protobufjs/protobuf.js#using-generated-static-code.

Describe the solution you'd like

Provide a feature or instruction for how to take the jspb.Message as input for deserialization: fromProto3JSON().

Describe alternatives you've considered

I found this PR relevant: https://github.com/googleapis/nodejs-ai-platform/pull/22/files

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.