telluiot / thingml Goto Github PK

It seems Arduino has changed quite a lot between 0022 and 1.0.1 regarding the way the deal with serial communication... See http://arduino.cc/en/Main/ReleaseNotes

The same ThingML program (using Serial communication on Arduino) compiled with Arduino 0022 and 1.0.1 provides very different results when reading bytes from RxTx on the Java side. (However, it seems some good tools like Putty do not really see any difference... so there might be a way to properly configure RxTx).

For now I am sticking with 0022 for my experiments.

It seems there is a problem with RxTx on this architecture:
osName=Linux, osProc=amd64

It works fine on Windows 7 (64) and I will make some tests on another version of Linux later on...

if (osName.equals("Linux") && (osProc.equals("x86-64") || osProc.equals("amd64"))) {//amd64 was missing
System.out.println("Loading native rxtx libs...");
NativeLibUtil.copyFile(Serial4ThingML.class.getClassLoader().getResourceAsStream("nativelib/Linux/x86_64-unknown-linux-gnu/librxtxSerial.so"), "librxtxSerial.so");
System.out.println("Done!");
}

it copies the so file at the root of the project, but then it raises an exception Caused by: java.lang.UnsatisfiedLinkError: no rxtxSerial in java.library.path

It seems the so file is not copied to the right place... (or maybe it's not the right one)

I am using the RxTx facilities provided by https://github.com/dukeboard/kevoree-extra/tree/master/org.kevoree.extra.osgi.rxtx

We must clarify how events are consumed in Region and in Composite.

For region running in parallel, I think the consuption of an event by a transition should NOT impact the other // regions. However, I think the consumption of an event by an outgoing transition of a sub-state should (recursively) prevent the composite and siblings to consume this event.

Do you agree?

Once we agree on a given semantics, this should be made crystal clear on the wiki (www.ThingML.net) and strictly ensured/tested by our different execution engine: Scala, Arduino, C (currently)

"Maybe we should have annotations which allow generating prints on the serial port or maybe on an LCD screen in some cases."

What is the advised strategy to deal with multiple target languages? Currenly ThingML compiles to C, Scala, etc. ThingML allows to define extern expressions (eg, by directly writing some C, when needed eg, for system calls). It's however quite a pain to duplicate all the models where extern statements are involved (eg, to change "millis()" by "System.currentTimeMillis()"). This goes against the MDE practices ThingML advocates... ;-)

The derived properties on the metamodel significantly helps the implementation of the compilers. However, they are sometimes misleading. For example. calling an opposite property on an element of an allXXXX derived properties still points to the former container object where this element was declared (eg, a fragment).

However, since our different compilers tend to "merge" the ThingML during the compilation process (the C, Scala, etc source code does not contain fragments anymore).

I think the implementation of our compilers would benefit from working on a merged ThingML model.

Maybe a task for Seb? ;-)

The definition of the on_error_callback_client callback in MyThing

function on_error_callback_client(msg: String)
        @c_callback "on_error_callback_client"
do
myport!mymessage(msg)
end

generated code in c

void on_error_callback_client(void *_instance, ...){
        va_list arguments;
        va_start(arguments, _instance);
        const char* msg_blabla = va_arg(arguments, char*);
        va_end(arguments);
       MyThing_send_myport_mymessage((MyThing_Instance *) _instance, msg_blabla );
}

The error statement (the equivalent of System.err.print in Java) is:
1/ not supported in the Arduino compiler (which is not so problematic), and
2/ potentially leads to compilation errors in the resulting code (which is blocking)

For example, the following ThingML code:
if (index == MAX_PACKET_SIZE) do
error("BUFFER OVERFLOW: " + b + " has been ignored. Current index = " + index)
end

compiles to:
if(_instance->PacketManager_index_var == _instance->PacketManager_MAX_PACKET_SIZE_var) {
// ERROR: ("BUFFER OVERFLOW: " + b + " has been ignored. Current index = " + _instance->PacketManager_index_var)}

There should be a \n at the end of the ERROR statement so that the if statement could properly be closed (the closing brace is commented in the generated code

can get it write, on compilation get something like

[java.lang.UNIXProcess@f19d78 ERR] ThingDeSerializer.cpp: In function ‘void f_ThingDeSerializer_forward(ThingDeSerializer_Instance*)’:
[java.lang.UNIXProcess@f19d78 ERR] ThingDeSerializer.cpp:99:6: error: redefinition of ‘void f_ThingDeSerializer_forward(ThingDeSerializer_Instance*)’
[java.lang.UNIXProcess@f19d78 ERR] ThingDeSerializer.cpp:26:6: error: ‘void f_ThingDeSerializer_forward(ThingDeSerializer_Instance*)’ previously defined here

The compilers leverage some annotations in order to adjust how the code is generated. These annotations should be properly documented on the Wiki.

Following discussion with Franck, we should deprecate enqueueing of a message parameter as a pointer to this parameter. Even though it can work when the parameter is allocated in the heap (e.g. the ThingML String type is generated as char*) and things are run on the same machine.

Moreover, ports and messages are concepts which may be used to communicate between things on different machines; thus, a pointer does not make sense.

In addition, please consider the following erroneous example in ThingML:

thing fragment Messages {
    message send_bytes(mbytes : Byte[32]);
}

thing Sender includes Messages {
....
        state SBytes {
            on entry do
                var data : Byte[32]
                data[0] = '0'
                data[1] = '1' 
                datasender!send_bytes(data)
            end         
....
        }

and generated code for the 'on entry' block in C

void Sender_send_datasender_send_bytes(struct Sender_Instance *_instance, uint8_t* mbytes);

void Sender_Behavior_OnEntry(int state, struct Sender_Instance *_instance) {
...
case SENDER_BEHAVIOR_SBYTES_STATE:
{
uint8_t data[32];
data[0] = '0';
data[1] = '1';
cout << "Send bytes "<<data[0]<< data[1] << endl;
Sender_send_datasender_send_bytes(_instance, data);
}
break;
...
}

Since the message handling happens asynchronously, and we pass a pointer to a value (mbytes) on the stack, the pointer contents is changed.

It seems there is a problem with RxTx on this architecture:
osName=Linux, osProc=amd64

It works fine on Windows and I will make some tests on another version of Linux later on...

if (osName.equals("Linux") && (osProc.equals("x86-64") || osProc.equals("amd64"))) {
System.out.println("Loading native rxtx libs...");
NativeLibUtil.copyFile(Serial4ThingML.class.getClassLoader().getResourceAsStream("nativelib/Linux/x86_64-unknown-linux-gnu/librxtxSerial.so"), "librxtxSerial.so");
System.out.println("Done!");
}

Consider the following three files:

testA.thingml

import "../../thingml.thingml"

thing fragment A {

    function a() do
        print "a()\n"
    end
}

testB.thingml

import "../../thingml.thingml"
import "testA.thingml"


thing fragment B includes A {

    function b() do
        print "b()\n"
        a()
    end
}

testC.thingml

import "../../thingml.thingml"
import "testB.thingml"

thing fragment C includes B {

    function c() do
        print "c()\n"
        b()
        a()
    end
}

configuration testSeveralFiles {

 instance c : C
}

The thing C cannot call a() from A. On attempt to compile, a NULL pointer exception throws...

It seems like we need something that allows passing and receiving messages internally. It is possible do it by defining two ports and connecting them together. It looks quite complex and cluttering a thing. We do not use these ports for communication with other things so we should not expose them publicly. Binding these ports looks redundant and we usually forget this.

We suggest having sort of 'trigger' concept which acts as a couple of ports connected together by default. We should be able to generate more efficient code for the internal communication.

Example:

Expected

thing X includes Xmessages {

    trigger trigger_name message_name;

    statechart behavior init Start {
        ...
        state Start{
          on entry do
            print "Srating...\n"
            b = getSomethig()
            trigger_name!message_name(b)
          end

          transition->Something
          event e : trigger_name?message_name
          .....
        }
       ......
    }
}

Actual (current workaround)

thing X includes Xmessages {
    provided port sendportinternal {
           sends message_name
    }
   required port receiveportinternal {
           receives message_name
    }

    statechart behavior init Start {
        ...
        state Start{
          on entry do
            print "Srating...\n"
            b = getSomethig()
            sendportinternal!message_name(b)
          end

          transition->Something
          event e : receiveportinternal?message_name
          .....
        }
       ......
    }
}
configuration Some {
    instance x:X
    connector x.receiveportinternal => x.sendportinternal // we usually forget this
}

Note:
Franck you can reassign to anyone you want if you think this feature makes sense )))

It would be useful to have initialization on properties definition. I am thinking of making it mandatory (like in scala).

It seems messages are sometimes inverted when going through a connector. This is probably due a misalignment between the SMaC framework and ThingML, when it comes to the semantics of ports and connector. In ThingML ports can produce AND receive messages, not in SMaC, where we have to duplicate ports and connectors, which might probably mess up the whole thing.

I will try to align the SMaC framework with ThingML, since it is only use there.

Anyway, the Scala compiler will soon be deprecated and replaced.

The generated Scala code (and any Scala code in general) is a pain to run on Android, as it produces way too large jar files. We'll replace the Scala compiler with a Kotlin compiler, able to generate JVM code with a very contained overhead, and also able to generate JavaScript, so that we can cover more platforms.

getSrv and getCli is annoying and not intuitive. Please refactor to getServer and getClient.

Related to Issue #33, we should have automated tests to ensure the consistency of the different compilers (not just for auto-tranistions) but for all the concepts defined in ThingML

Currently all the properties are dynamic but we could have the equivalent of Java final attributes (or scala val) which value would be fixed at compile time. The value of these properties could be used by the code generators in order to optimize the generated code by, for example, substituting the values.

Currently, a Thing refers to a set of Ports. Each Port is either a RequiredPort or a ProvidedPort. I would suggest having two distinct roles in Thing:

• providedPorts [0..*] : ProvidedPort
• requiredPorts [0..*] : RequiredPort

This way, it would avoid the definition of filters in our compilers, since provided and required ports will typically imply different treatments.

Note: If we apply this refactoring, it may not be useful to keep the RequiredPort and ProvidedPort sub-classes.

It seems the low-level encoding of datatypes (BIG_ENDIAN, LITTLE_ENDIAN) is not uniform. Globally, it seems we favor BIG_ENDIAN (which is BTW the default encoding for Java). It works OK when we exchange data between Arduino and Scala. But the CoAP generated servers seems to use LITTLE_ENDIAN....

The CoAP compiler should be updated.

Also, we should think about how signed/unsigned C types map to Java types

The ANTLR plugin should be bundled with the parse to avoid having to install EMFText runtime to use ThingML. Also, as much as possible we should use package deps and not bundle deps. We should also check that all packages are properly exported (at least the CharacterEscaper of the parser is not).

Java code must be in target/src-gen/java/

Using After in identifiers causes scala compilation errors: generated code contains too many '}'.

Substates should have precedence over superstates to consume events : http://en.wikipedia.org/wiki/UML_state_machine#Event_deferral .

There is currently no precedence rule in C, the test "testCompEventCapture" gives an example of such behaviour.

When trying to compile the blink sample, I get the following error (when compiling the resulting code in the Arduino IDE):
BlinkArduino:12: error: previous declaration of 'void LedUC_LedImpl_OnEntry(int, LedUC_Instance*)' with 'C++' linkage
BlinkArduino:106: error: conflicts with new declaration with 'C' linkage

Te following in ThingML is not compiled properly

var value : Integer = 10
print "some value " + value

When using the LCD_screen, the compilation of:

required port Display
    {
        sends initDisplay, refreshDisplay, setDisplay
  // with initDisplay defined as: message initDisplay (id : UInt8, title : String, unit : String, initValue : Integer, minValue : Integer, maxValue : Integer);
    }

produces compilation error in the generated Arduino code:

// parameter title
ptr_union_t __ptrunion_title; //here is the faulty line
__ptrunion_title.pointer = (void*)title;
_fifo_enqueue( __ptrunion_title.buffer[0] );
_fifo_enqueue( __ptrunion_title.buffer[1] );

the error says:

BlinkArduino.cpp: In function 'void enqueue_Meteo_send_Display_initDisplay(Meteo_Instance*, uint8_t, char*, char*, int, int, int)':
BlinkArduino:1608: error: 'ptr_union_t' was not declared in this scope

Note that it works when we add the @sync_send "true" annotation (which is probably better, but still, it should work without that annotation).

We should have two type of ports to distinguish between required and provided. Maybe we should have "optional" on required ports.

Does ThingML intends to support composite data types and collections? So far, these important concepts are not supported.

should be supported if we want to use third party libs. It is better then carrying along all H files every time

Consider the following code in ThingML

function escape() : Byte[34]
do
    var escaped : Byte[34]
    escaped[0] = 0
    return escaped
end

and C generated code

uint8_t* f_Sender_escape(struct Sender_Instance *_instance) {
{
uint8_t escaped[34];
escaped[0] = 0;
return escaped;
}
}

Pointer to a local array is returned. Should never happen....

I would rather prefer dynamic allocation, but still not quite clear where to free this memory block...see below

uint8_t* f_Sender_escape(struct Sender_Instance *_instance) {
{
uint8_t* escaped = (uint8_t*) malloc(sizeof(uint8_t)*34);
escaped[0] = 0;
return escaped;
}
}

It seems we should (even though I am not expert in C) also declare the ThingML functions in the generated *.h files. In the generated *.c files, it sometimes happen a function is called before it is declared, causing some
"error: conflicting types for ..."
"note: previous implicit declaration of ... was here"

If this is confirmed as an issue by our C expert (Franck), then this would be a blocking issue needing immediate action!

Is the following code valid?

    function charToInt(c : Char) : Int8 do
        return c - 48
    end

    function intToChar(i : Int8) : Char do
        if ( i > -1 and i < 10) return i + 48
        return -1 // Error
    end     

It seems we are doing some numerical algebra mixing numbers and char... While it is no problem for the C compiler (everything is allowed in C...), it of course generate a compilation error in Scala. Though we will probably not develop a complex type system in ThingML, I think this kind of hack should not be allowed in ThingML i.e. it should be an error reflected at the ThingML level.

We should be able to customize the generated Makefile, typically when we wrap existing libraries. For example, the Makefile below integrates a MongoDB component. I had to manually insert 3 elements: -lmongoc, -lbson and -DMONGO_HAVE_STDINT=1

CC = cc
LIBS = -lpthread -lmongoc -lbson 
CFLAGS = -O -w
SRCS = SerialProxy.c ClockTimer.c LinuxSerial.c MessageSerializer.c WeatherStation.c LinuxClock.c MessageDeserializer.c LinuxDB.c RaspiNode.c runtime.c
OBJS = SerialProxy.o ClockTimer.o LinuxSerial.o MessageSerializer.o WeatherStation.o LinuxClock.o MessageDeserializer.o LinuxDB.o RaspiNode.o runtime.o

all : RaspiNode

.c.o :
    ${CC} ${CFLAGS} -c $< -DMONGO_HAVE_STDINT=1

RaspiNode : $(OBJS)
    $(CC) -o $@ $(OBJS) $(LIBS) -lm

clean:
    rm -f *.o *~ RaspiNode

Seems that region are not supported in the GraphML export, as YeD explodes when opening a graphml model containing regions:

y.H.B.B.a: Target node id Start not defined in this graph!
    at y.H.B.B.Y.Ă(Unknown Source)
    at y.H.B.B.Y.ā(Unknown Source)
    at y.H.B.B.Y.Ć(Unknown Source)
    at y.H.B.B.Y.ā(Unknown Source)
    at y.H.B.B.Y.Ą(Unknown Source)
    at y.H.B.B.Y.ā(Unknown Source)
    at y.H.B.B._.ā(Unknown Source)
    at y.H.B.B._.ā(Unknown Source)
    at y.H.B.A$13.ā(Unknown Source)
    at y.H.B.A.ā(Unknown Source)
    at y.H.B.A.ā(Unknown Source)
    at y.H.Q.ā(Unknown Source)
    at B.A.A.B.G.A.F.ā(Unknown Source)
    at B.A.A.B.G.A.F.ā(Unknown Source)
    at B.A.A.B.G.A.D.ā(Unknown Source)
    at y.H.G.ā(Unknown Source)
    at y.B.A.M.ď(Unknown Source)
    at y.B.W.Č(Unknown Source)
    at y.B.W.ā(Unknown Source)
    at y.B.W.ă(Unknown Source)
    at B.A.A.B.Q.Ă(Unknown Source)
    at B.A.A.B.Q.ā(Unknown Source)
    at B.A.A.J.A.ā(Unknown Source)
    at B.A.A.J.A.ā(Unknown Source)
    at B.A.A.J.D.ă(Unknown Source)
    at B.A.A.J.D.Ă(Unknown Source)
    at B.A.A.B$5.run(Unknown Source)
    at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:251)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:727)
    at java.awt.EventQueue.access$200(EventQueue.java:103)
    at java.awt.EventQueue$3.run(EventQueue.java:688)
    at java.awt.EventQueue$3.run(EventQueue.java:686)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:697)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:91)

Refactor the set of projects so that it is easy (as easy as it could be) to release using Maven.

We have a test:

 function sendInt(i : Int16) do
        var hundreds : Int8 = i / 100
        i = i - (100 * hundreds)
        var tens : Int8 = i / 10
        i = i - (10 * tens)
        harness!testOut( intToChar(hundreds) )
        harness!testOut( intToChar(tens) )
        harness!testOut( intToChar(i) )
    end

It seems that ThingML allows i to be modified. But It seems the Scala compiler assumes i cannot be modified, where the C compiler allows it.

Should it really be legal in ThingML? Or is it a but in the Scala compiler?

It should be mapped to a val, to be set when constructing the object in a configuration.

It is currently mapped to a var, which is of course not readonly... even though it will never be re-assigned since ThingML prevent readonly properties to be re-assigned

empty transition in c does not trigger shift to another state while it works for scala. The code bellow works for scala, but does not for c and cpp
....
state Started {
transition -> Stopped
}
state Stopped {
}
....

C and C++ compilers regard an Integer type in ThingML as an unsigned int. Consider the following simple example

import "../../thingml.thingml"


thing fragment DoorWindowMsg{
    //internal messages
    message dw_state(dwstateneg : Integer, dwstatepos : Integer);
}



thing fragment DoorWindow includes DoorWindowMsg {

    provided port dwintsendport {
        sends dw_state
    }

    required port dwintrecport {
        receives dw_state
    }

    statechart behavior init IsClosedOpened {

        state IsClosedOpened {
            on entry do
                print "DoorWindow: finding out the door/window state...\n"
                dwintsendport!dw_state(get_neg(), get_pos())
            end

            transition->Ready
            event e : dwintrecport?dw_state
            action do
                'printf("Received dw_state %d %d\n", '& e.dwstateneg &', '& e.dwstatepos &');'
                'printf("Call directly %d %d \n", '& get_neg() &', '& get_pos() &');'
            end

        }

        state Ready {
            on entry do
                print "DoorWindow: ready ...\n"
            end

        }

    }
}

thing FibaroDoorWindow includes DoorWindow
@c_header "
#include <stdio.h>
"
{

    function get_neg() : Integer do
        return -10
    end


    function get_pos() : Integer do
        return 10
    end
}


configuration SampleTest
  @output_folder "/home/tmp/"
  @debug "true"
  @debug_fifo "true"
  @debug_message_send ".*"
  @debug_message_receive ".*"
{


    instance dw : FibaroDoorWindow

    // Create and connect the test app

    connector dw.dwintrecport => dw.dwintsendport
}

Integer is defined as

datatype Integer    
    @c_type "int"
    @c_byte_size "2"
    @java_type "Short"
    @SenML_type "Double"
    @ros_type "int16";

Actual output:

THINGML: Starting in debug mode...
DoorWindow: finding out the door/window state...
THINGML: -> FibaroDoorWindow_send_dwintsendport_dw_state
THINGML: <- FibaroDoorWindow_handle_dwintrecport_dw_state
Received dw_state 65526 10
Call directly -10 10 
DoorWindow: ready ...

Expected output:

THINGML: Starting in debug mode...
DoorWindow: finding out the door/window state...
THINGML: -> FibaroDoorWindow_send_dwintsendport_dw_state
THINGML: <- FibaroDoorWindow_handle_dwintrecport_dw_state
Received dw_state -10 10
Call directly -10 10 
DoorWindow: ready ...

Note:
I believe the error is here

dispatch_FibaroDoorWindow_send_dwintsendport_dw_state((struct FibaroDoorWindow_Instance*)instance_by_id((mbuf[0] << 8) + mbuf[1]) /* instance */,
(mbuf[2]<<8) + mbuf[3] /* dwstateneg */ ,
(mbuf[4]<<8) + mbuf[5] /* dwstatepos */ );

It seems that local variables are not changeable:

readonly var status : Integer = 0

or

var status : Integer = 0 //should be changeable

will result in myLocalVariable.isChangeable == false

property buffer : Byte[16]

The thing declaring this property will be properly compiled, but its instantiation will not compile. When initializing the thing in the configuration (compiled in a scala main), the array is not declared/initialized and it is thus not passed as argument of the constructor, resulting in a compilation error.

workaround until the bug is fixed
property buffer : Byte[16]
set buffer[0] = 0//explicitly initialize the array

It seems Double is not support by the Arduino compiler.

The following ThingML code:
message report_rotation(t : Double)//report the angle in radians

Compiles to the following Arduino code:
// Enqueue of messages SensorsDisplay::RemoteControl::report_rotation
void enqueue_SensorsDisplay_send_RemoteControl_report_rotation(struct SensorsDisplay_Instance *_instance, double t){
if ( fifo_byte_available() > 8 ) {

_fifo_enqueue( (19 >> 8) & 0xFF );
_fifo_enqueue( 19 & 0xFF );

// ID of the source instance
_fifo_enqueue( (_instance->id >> 8) & 0xFF );
_fifo_enqueue( _instance->id & 0xFF );

// parameter t
_fifo_enqueue( (t>>24) & 0xFF );//error: invalid operands of types 'double' and 'int' to binary 'operator>>
_fifo_enqueue( (t>>16) & 0xFF );//error: invalid operands of types 'double' and 'int' to binary 'operator>>
_fifo_enqueue( (t>>8) & 0xFF );//error: invalid operands of types 'double' and 'int' to binary 'operator>>
_fifo_enqueue( t & 0xFF );//error: invalid operands of types 'double' and 'int' to binary 'operator>>

}
}

Should it be a separate datatype or we need a keyword to declare as static or const. Talk to Franck and Brice

Expression literals to refer to enumerations are missing from the meta-model.

It seems impossible to have multiple connectors targeting the same instance within a group...

We should probably have menus for compilers instead of buttons. We indeed start to have many compilers...