farsounder / protobuf-matlab

Automatically exported from code.google.com/p/protobuf-matlab

protobuf-matlab - FarSounder's Protobuf compiler for Matlab

Copyright 2011 FarSounder, Inc.

http://code.google.com/p/protobuf-matlab/

NOTE(Heath - 04/2024): this is not maintained as we're not really using matlab anymore (we haven't in a long time). Please feel free to use for whatever you want if it helps you get protobuf working with matlab. I'm not sure we'll ever update to latest protobuf version as I don't think I have access to a current matlab license to test any changes. If you're inclined to PR something, feel free to open and issue first and 'at' me (@heathhenley) or email [email protected] to check so you don't waste any effort. It looks like the most recent fork with version updates is here - maybe that will help if you ended up here. 👍

Overview

This package provides a Matlab code generator for version 2.4.1 of Google's Protocol Buffers compiler (protoc) as well as support libraries for the generated Matlab code.

Building protoc with Matlab support

1. Get the protobuf source: svn co http://protobuf.googlecode.com/svn/tags/2.4.1 protobuf

2. Get the protobuf-matlab source: git clone https://code.google.com/p/protobuf-matlab/

3. Add the protobuf-matlab src files to the Google Protobuf src: cp -r protobuf-matlab/src protobuf

4. Compile the modified protobuf project.

This should yield a protoc executable with a --matlab_out option. You can now use protoc to generate Matlab reading and writing code for your .proto file(s).

Matlab support library setup

In order to use the generated Matlab code, you'll need to add the protobuflib directory to your Matlab path. protobuflib is a collection of .m utility files used by the generated code.

protobuf-matlab's Issues

how to avoid: Warning: Required field not set while parsing. This is an error.

running the code sample gives:
Warning: Required field not set while parsing. This is an error. 
upon the 1st call with an empty buffer 

bob = pb_read_Person();

The field is later set. Ideally, the warning shouldn't be there during the 
initial empty construction of bob. Instead there should be a final check on the 
constructed object that checks all required fields are set (could be 
recursive). And then one can throw error instead of warning. 
That should work right?

Original issue reported on code.google.com by [email protected] on 17 Jan 2013 at 5:55

Undefined symbols for architecture x86_64

What steps will reproduce the problem?
1.Download the protobuf src from google code.
2.Download the protobuf-matlab src 
3.Do the src copy as stated in README.txt of protobuf-matlab
4. ./configure
5. make

We will get:

Making check in src
/bin/sh ../libtool --tag=CXX   --mode=link g++ -D_THREAD_SAFE  -Wall 
-Wwrite-strings -Woverloaded-virtual -Wno-sign-compare -O2 -g -DNDEBUG 
-D_THREAD_SAFE   -o protoc main.o  libprotobuf.la libprotoc.la -lz 
libtool: link: g++ -D_THREAD_SAFE -Wall -Wwrite-strings -Woverloaded-virtual 
-Wno-sign-compare -O2 -g -DNDEBUG -D_THREAD_SAFE -o .libs/protoc main.o 
-Wl,-bind_at_load  ./.libs/libprotobuf.dylib ./.libs/libprotoc.dylib 
/Users/mezcal/protobuf/protobuf-2.5.0/src/.libs/libprotobuf.dylib -lz
Undefined symbols for architecture x86_64:
  "farsounder::protobuf::compiler::matlab::MatlabGenerator::MatlabGenerator()", referenced from:
      _main in main.o
  "farsounder::protobuf::compiler::matlab::MatlabGenerator::~MatlabGenerator()", referenced from:
      _main in main.o
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status


What is the expected output? What do you see instead?
Successfully compiled and make.

What version of the product are you using? On what operating system?
My OS is mac os 10.7 64bit.

Please provide any additional information below.

Original issue reported on code.google.com by [email protected] on 19 Oct 2013 at 4:11

README.txt is slightly wrong

In README.txt, it states:

`svn co http://protobuf.googlecode.com/svn/tags/2.4.1 protobuf`

This does not provide an enjoyable environment to build protocol buffers in. 
You still need to run autoconf/automake, and this is documented nowhere (that I 
could find).

To make it easier on your users, change the line to use the provided 2.4.1 .zip 
package: `wget http://protobuf.googlecode.com/files/protobuf-2.4.1.zip && unzip 
protobuf-2.4.1.zip && rm protobuf-2.4.1.zip && mv protobuf-2.4.1 protobuf`

Original issue reported on code.google.com by [email protected] on 19 Jan 2013 at 12:02

Error protoc-gen-matlab not found

Hi, 

It took me a while to compile the modified project. Now I encounter the 
following error whenever I use it:

protoc-gen-matlab: program not found or is not executable          
--matlab_out: protoc-gen-matlab: Plugin failed with status code 1. 

I checked and protoc-gen-matlab does not seem to be in your part of the code? 
In that case it must be generated in the original protobuf. Do you have an idea 
how to solve this?

thanks a lot and best wishes

Original issue reported on code.google.com by [email protected] on 31 Jan 2013 at 3:15

Installation instructions incomplete and ambiguous.

Hello,

First of all, many thanks for producing this library, so that protobuf messages 
can be used as a method to store/transmit data from matlab to other processes.  
We use this library for a matlab simulation with visualization in other 
processes. 

I'd like to raise an issue with the documentation on installation (README.txt) 
and compiling the protoc binary. If it doesn't end up in the actual 
documentation, then at least it will be available as an issue for others who 
may be facing the same problems.


Step 1 is correct and unambiguous.

Step 2 should clarify that src/farsounder/protobuf/compiler/matlab/* should 
actually end up as "src/farsounder" in the protobuf 2.4.0a tree as well, 
because now it can be interpreted as copying files to root or src.

Step 3 is correct, but please be aware that main.cc is in that directory and 
not in 'src' in the 2.4.0a tree. ( I made that mistake ).

Step 4 doesn't work yet, because Makefile.am hasn't been edited.

Explanation:
============
- Open Makefile.am in 'src'.
- There are 2 locations to add "matlab_generator.*" to:
     matlab_generator.h  after "python_generator.h"
     matlab_generator.cc after "python_generator.cc"
  Don't forget to add a backslash character after python_generator.*.

- Now run ./autogen.sh
- Then run ./configure
- Then run make
- Your updated 'protoc' is now in "src" or "src/.libs".

This does give you the required *.m files that you'll need to use protobuf 
messages in matlab. There's another issue with actual use that I hope the 
authors can comment on. This will be highlighted in another issue.

Original issue reported on code.google.com by [email protected] on 13 Oct 2011 at 4:20

Plugin driver to use the compiler without change of protoc

In newer versions of the protobuf compiler plugins can be used to create 
language-specific source code from the parsed .proto files.
By putting the attached driver into the directory with matlab_generator.cc and 
matlab_generator.h, the compiler can be generated simply by linking against 
libprotobuf and libprotoc.

I'm not able to provide an adapted Makefile.am since I'm using CMake to build 
the compiler. 

Torsten

Original issue reported on code.google.com by [email protected] on 15 Mar 2012 at 4:06

Attachments:

• matlab_plugin.cc

Compiling with Visual Studio 2010 (solution)

Compiling with VS2010 (not express), under 64-bits Windows 7, throws several 
errors.

This is how I fixed it. It would be great if it was integrated in the current 
version.

First, you can't compile the solution at once, you must compile the libprotobuf 
and libprotoc projects separately first. You can choose to compile Debug or 
Release; here I chose Release. The Readme could be updated with this.

Then, the solution file that ships with protobuf can be replaced with one that 
incorporates the following changes:

Right-click "protoc" project -> Properties -> Configuration Properties -> 
Linker -> General. Disable Incremental Linking.

Under the same Linker -> Input, add additional dependencies:
Release\libprotobuf.lib
Release\libprotoc.lib

(Or Debug if you chose to compile the release version.)

Finally, the file "matlab_generator.cc" must be added under Source to the 
protoc project.

I've attached files that incorporate these changes (except the readme 
instructions). They should replace the 2 files in "protobuf-2.4.1/vsprojects/". 
Note that they assume you compiled the libraries for Release.

That's it -- hope I didn't forget anything.

Original issue reported on code.google.com by [email protected] on 6 Mar 2014 at 12:20

Attachments:

• protobuf.sln
• protoc.vcxproj

Support for protoc version 3.4.?

The README.md suggests that the protoc compiler needs to be rebuilt to add Matlab support. There is a commit comment to the effect that plugin support was being added.

A plugin could be made available precompiled, which is much easier, and could migrate to newer protoc versions such as 3.4.

Has anyone worked on this?

Is there a working-but-undocumented way to use the Matlab compiler as a plugin?

Thanks.

Add example for complex attributes and repeated fields

Thanks for resolving issues 1 and 2. The only issue remaining from those that I 
have spoken is that the use of repeated fields isn't clear. I gathered that the 
use of complex attributes is resolved.

If repeated is supported (of which I'm not sure yet), could you add an example 
of this in the 'bob' example? This would also immediately demonstrate how 
complex attributes are assigned.

My proposal is to use the same example as protobuf itself and give Bob a couple 
of phone numbers:

http://code.google.com/apis/protocolbuffers/docs/overview.html

Just in case that repeated is not supported, could you make this limitation 
clear on the front page?

Original issue reported on code.google.com by [email protected] on 22 Oct 2011 at 7:06

pb_run_test fails

1. Build according to ReadMe
2. Run MATLAB
3. Switch to the protobuf-matlab/test directory.
    cd ~/Projects/protobuf/protobuf-matlab/test 
4.Compile test.proto
     !/usr/local/bin/protoc --matlab_out=./ ./test.proto
5. Run the test.
    pb_run_test

Expected successful unit test run.

Instead, it looks like some equality tests failed:

optional_float: 23089.235 != 23089.2344
repeated_float: 23089.235 != 23089.2344
repeated_float: 30948.2351 != 30948.2344

ans = 

                   has_field: [66 java.util.HashMap]
              optional_int32: 33
              optional_int64: 234980098
             optional_uint32: 23098
             optional_uint64: 3.5099e+10
             optional_sint32: 25
             optional_sint64: 98482
            optional_fixed32: 2309
            optional_fixed64: 24350809
           optional_sfixed32: 2309
           optional_sfixed64: 24350809
              optional_float: 2.3089e+04
             optional_double: 2.4305e+04
               optional_bool: 130498
             optional_string: 'asldfkj'
              optional_bytes: [97 115 108 107 100 106 108 106]
     optional_nested_message: [1x1 struct]
    optional_foreign_message: [1x1 struct]
     optional_import_message: [1x1 struct]
        optional_nested_enum: 1
       optional_foreign_enum: 4
        optional_import_enum: 7
       optional_string_piece: ''
               optional_cord: ''
              repeated_int32: [23 33]
              repeated_int64: [9823408 234980098]
             repeated_uint32: [23098 23098]
             repeated_uint64: [23408 3.5099e+10]
             repeated_sint32: [-48 25]
             repeated_sint64: [-234080 98482]
            repeated_fixed32: [2309 230489]
            repeated_fixed64: [23408 24350809]
           repeated_sfixed32: [-2309 230489]
           repeated_sfixed64: [-23408 24350809]
              repeated_float: [2.3089e+04 245789 3.0948e+04]
             repeated_double: [2.4305e+04 2.0398e+04]
               repeated_bool: [130498 9038]
             repeated_string: {'asldfkj'  'asdlkfj'}
              repeated_bytes: {[97 115 108 107 100 106 108 106]  [104 103 115 104]}
     repeated_nested_message: [1x2 struct]
    repeated_foreign_message: [1x2 struct]
     repeated_import_message: [1x2 struct]
        repeated_nested_enum: []
       repeated_foreign_enum: []
        repeated_import_enum: []
       repeated_string_piece: ''
               repeated_cord: ''
               default_int32: 41
               default_int64: 42
              default_uint32: 43
              default_uint64: 44
              default_sint32: -45
              default_sint64: 46
             default_fixed32: 47
             default_fixed64: 48
            default_sfixed32: 49
            default_sfixed64: -50
               default_float: 51.5000
              default_double: 52000
                default_bool: 1
              default_string: 'hello'
               default_bytes: [119 111 114 108 100]
         default_nested_enum: 2
        default_foreign_enum: 5
         default_import_enum: 8
        default_string_piece: 'abc'
                default_cord: '123'
              unknown_fields: []
         descriptor_function: @pb_descriptor_test__TestAllTypes

protobuf-matlab pulled 9-Jan-2015
protobuf 2.4.1
MATLAB Version: 8.3.0.532 (R2014a)  Also happens with R2014b. 
Mac OS X Yosemite 8.0.2 (10600.2.5)

Original issue reported on code.google.com by [email protected] on 12 Jan 2015 at 7:37

Little Endian is not enforced for fixed64 formats and the like

No tests done, but examining the code it seems we generally just call typecast 
in matlab, which should use the system's endianness, when writing or 
interpreting fixed64 like types. The protocol buffer standard specifies for 
little endian to be used, so encoding or parsing on a big endian machine would 
produce incorrect results.

Typecast is also used in parsing of several other types, so they might also be 
broken on a big endian system.

Original issue reported on code.google.com by [email protected] on 23 Mar 2012 at 6:14

How to create stub/services files with MATLAB grpc plugin?

Hi,

Im using MatlabWithProtoV3 to create protoc.exe with matlab_out in Windows environment.
I was able to create protoc and when i use

protoc.exe user.proto --matlab_out=./

It is only creating matlab files for messages and it is not creating matlab files for services(client and server)

Then i read about plugins and included the generator and plugin files to gRPC Source to create Matlab plugin and created the grpc_matlab_plugin.exe successfully.

Now, if i execute
protoc.exe user.proto --matlab_out=./ --grpc_out=./ --plugin=protoc-gen-grpc="D:\grpc\cmake\build\Debug\grpc_matlab_plugin.exe

I'm getting
pb_descriptor_LoginRequest.m: Tried to write the same file twice. pb_read_LoginRequest.m: Tried to write the same file twice. pb_descriptor_APIResponse.m: Tried to write the same file twice. pb_read_APIResponse.m: Tried to write the same file twice. pb_descriptor_Empty.m: Tried to write the same file twice. pb_read_Empty.m: Tried to write the same file twice.

error message and no files are getting created.

in gRPC repo, for C++ compiler i could find cpp_plugin.h has some codes to create service related files but similar file is not available here. or here

Can you please let me know how to create matlab files for services?

Attached the files created when i execute the above mentioned commands,
userproto_matlabfiles.zip

Thanks

protoc doesn't parse "option cc_api_version = 2;" and "option py_api_version = 2;"

protoc won't produce any output when cc_api_version or py_api_version are set. 

syntax = "proto2";
option cc_api_version = 2;
option py_api_version = 2;

message MyMessage {
}

Original issue reported on code.google.com by [email protected] on 22 Jan 2013 at 10:36

Installation Problem

What steps will reproduce the problem?

1. svn co http://protobuf.googlecode.com/svn/tags/2.4.1 protobuf or as 
suggested by one of the member I also tried: `wget 
http://protobuf.googlecode.com/files/protobuf-2.4.1.zip && unzip 
protobuf-2.4.1.zip && rm protobuf-2.4.1.zip && mv protobuf-2.4.1 protobuf` to 
avoid autoconf.

2. git clone https://code.google.com/p/protobuf-matlab/

3. cp -r protobuf-matlab/src protobuf

4. cd protobuf

5. ./configure

6. make -j8

main.o: In function `main':
/home/*/Desktop/protobuf/src/google/protobuf/compiler/main.cc:61: undefined 
reference to 
`farsounder::protobuf::compiler::matlab::MatlabGenerator::MatlabGenerator()'
/home/*/Desktop/protobuf/src/google/protobuf/compiler/main.cc:65: undefined 
reference to 
`farsounder::protobuf::compiler::matlab::MatlabGenerator::~MatlabGenerator()'
/home/*/Desktop/protobuf/src/google/protobuf/compiler/main.cc:65: undefined 
reference to 
`farsounder::protobuf::compiler::matlab::MatlabGenerator::~MatlabGenerator()'
collect2: ld returned 1 exit status
make[2]: *** [protoc] Error 1
make[2]: Leaving directory `/home/*/Desktop/protobuf/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/*/Desktop/protobuf'
make: *** [all] Error 2

However If I run ./autogen.sh, then ./configure, make command finish without 
error only saying:
make[3]: Leaving directory `/home/*/Desktop/protobuf/src'
make[2]: Leaving directory `/home/*/Desktop/protobuf/src'
make[1]: Leaving directory `/home/*/Desktop/protobuf'

Then I do sudo make install, again
make[3]: Leaving directory `/home/*/Desktop/protobuf/src'
make[2]: Leaving directory `/home/*/Desktop/protobuf/src'
make[1]: Leaving directory `/home/*/Desktop/protobuf/src'

user@user:/home/*/Desktop/protobuf$ protoc --help
protoc: symbol lookup error: protoc: undefined symbol: 
_ZN10farsounder8protobuf8compiler6matlab15MatlabGeneratorC1Ev


What version of the product are you using? On what operating system?
GNU Make 3.81
Ubuntu 12.04.4 LTS x86_64
autoconf (GNU Autoconf) 2.68
automake (GNU automake) 1.11.3


Please provide any additional information below.

Does anybody have trouble recently to build and install protobuf-matlab ?

Original issue reported on code.google.com by [email protected] on 1 Jul 2014 at 10:00

Instructions for protobuf-matlab use may be incorrect

Hi there,

Goal: After the Makefile.am was edited and the protoc binary was correctly 
built, I attempted to get a protobuf message transmitted correctly over the 
wire. So I followed the example.

1.

Following the example as is in matlab R2011b didn't work. This is because in 
the 
"pb_read_*()" functions, it is expecting a buffer to read values from. If that 
buffer doesn't exist, it will fall into a check that all required fields are 
set. Since the message is just instantiated and no buffer is available, the 
check fails and the object won't be instantiated. It becomes unusable, so 
there's no way to create the actual object this way.

I modified the library code to only perform the check if the length of the 
buffer was > 0. This allowed me to instantiate the object, so that I could 
manipulate it.


2.

The next snag appeared when using pblib_set. Note: I'm by no means a matlab 
expert, I'm actually a first-time user helping someone out. My observations 
here were that the remote process wasn't actually getting the values that I 
thought were being set. Further investigation revealed the following. 

I put a disp( bob ) call before pblib_set calls and after and this demonstrated 
that none of the set calls persisted. The function definition has an "input" 
msg parameter and another "msg" parameter going out. It appears that in this 
version of matlab at least (R2011b), a copy was made of that object and this 
copy was manipulated instead of the "in" object from matlab. 

I think it's helpful if the "out" and "in" parameters into this function refer 
to the same object, but not sure if that can be forced in matlab that way. 
Having additional copies would also raise memory concerns, because every 
attribute being set would copy the entire object by itself.

For the purpose of this demo, I did manage to get this to work by using 
syntaxes like:

bob = pblib_set( bob, 'id', ... );

Greatly appreciate your comments on this issue and whether this should work as 
specified, or whether there are indeed issues with matlab or future versions of 
it after this was released.




I noticed however that whatever value I set on a simple integer, this value was 
not actually transmitted. 

Original issue reported on code.google.com by [email protected] on 13 Oct 2011 at 4:29

pb_run_test.m fails

Reproduction steps:

1. Build according to ReadMe

2. Run MATLAB

3. Switch to the protobuf-matlab/test directory.
   % cd /protobuf-matlab/test

4. Compile test.proto
   % protoc --matlab_out=./ test.proto

5. In MATLAB, Run the test.

   >>pb_run_test
   Undefined function 'pb_read_test_import__ImportMessage' for input arguments of type 'double'.
   
   Error in pb_run_test (line 62)
   msg = pblib_set(msg, 'optional_import_message', pb_read_test_import__ImportMessage([]));
   

When generating the --matlab_out wrappers via protoc, it does not generate "pb_read_test_import__ImportMessage.m file". It is also not present in the protobuf-matlab/protobuflib. Where is this file coming from? And why is it in the pb_run_tests?

Environment information:

• Tested with MATLAB R2017b and R2014a
• protoc --version
  libprotoc 2.4.1
• OS SMP Debian 8 4.4.16 x86_64 GNU/Linux
• The google CPP tests work fine.
• Tested my own application with protobuf CPP API which works fine