firmata / arduino Goto Github PK

The arduino team is interested in reducing the number of microcontroller-specific #ifdef statements in core library files.

A number of the #ifdef statements in Boards.h can probably be removed for boards that have variant files in the Arduino IDE. What needs to be done here is a comprehensive audit of the variant files in Arduino as they related to the features requiring #ifdef statements in boards.h. The findings should be reported back to the Arduino team.

On compiling Firmata (e.g. when used within a sketch in the Arduino IDE) the following warning is generated:

...\Firmata/Boards.h: In function 'unsigned char writePort(byte, byte, byte)':
...\Firmata/Boards.h:371: warning: no return statement in function returning non-void

This will likely lead to unexpected behavior if writePort() is called and the return value manipulated.

Hi,
not sure where to post this, but the firmata.org website is down with error 500.
greetz!

It seems that current version does not handle interrupts (to manage encoders for example), is it planned in your roadmap?

https://github.com/firmata/arduino/blob/master/examples/StandardFirmata/StandardFirmata.ino#L101

Working on reading a MMA8452Q (triple axis accelerometer), ran the sketch here and it works fine, but no dice over Firmata. Changing this line to Wire.endTransmission(false); (as seen in the sketch) makes it work correctly.

I'm wondering what, if any, implications exist by doing this?

i've done a modification to avoid sending one byte value with 2 serial bytes (not defined by the protocol as far as i guessed) because the vector already contains bytes

void FirmataClass::printFirmwareVersion(void)
{
...
for(i=2; i<firmwareVersionCount; ++i) {
// avoid sending one byte value with 2 serial bytes (not defined by the protocol)
//sendValueAsTwo7bitBytes(firmwareVersionVector[i]); <- removed
sendValueAsOne7bitByte(firmwareVersionVector[i]); <- added
}
endSysex();
}
}

void FirmataClass::sendDigitalPort(byte portNumber, int portData)
{
FirmataSerial->write(DIGITAL_MESSAGE | (portNumber & 0xF));
//FirmataSerial->write((byte)portData % 128); // Tx bits 0-6 <- removed
//FirmataSerial->write(portData >> 7); // Tx bits 7-13 <- removed
FirmataSerial->sendValueAsTwo7bitBytes(portData); <- added
}

When the StandardFirmata (v2.3.1 that comes with arduino v1.0) gets compiled and uploaded to an Uno R1 board then if you try to get the REPORT_FIRMWARE it retuns nothing. The problem is located on line 123 of the Firmata.cpp (filename =
strrchr(name, '/') + 1;) and it is caused because I am compiling the .ino and .cpp under Windows 7 (so the FILE is something like C:\Robot\ ... \Firmata.cpp instead of C:/Robot/ ... /Firmata.cpp as expected).

via Christopher Coleman:

One thing we noticed is that while standard firmata allows for servos on many (pretty much all) digital pins, in Configurable Firmata they fail or act super irregularly above pins 11-15 (depending on the board).

Hi,
Does firmata support stepper motor ? and if yes, there is examples or doc ?
Thanks for all

Create a timer based Stepper motor implementation for ConfigurableFirmata. The current Stepper implementation does not use a timer since all were already in use by StandardFirmata (at least for an Arduino Uno and similar boards). With ConfigurableFirmata a timer-based Stepper implementation should be possible as long as the user does not enable another feature that uses the same timer.

Copying this over from firmata.org.

Extended Configuration Proposal

for special configuration not covered in pinmode-specific messages

/* set analogReference request
 * 0  START_SYSEX (0xF0)
 * 1  CONFIG_EXT command (0x7C)
 * 2  ANALOG_REF command (0x00)
 * 3  analog reference source (0-4, 0=DEFAULT, 1=INTERNAL, 2=INTERNAL1V1, 3=INTERNAL2V56, 4=EXTERNAL
 *    Values supported are device-specific. 
 *    See http://arduino.cc/en/Reference/AnalogReference and Arduino.h for details)
 */

/* set analogReadResolution request
 * 0  START_SYSEX (0xF0)
 * 1  CONFIG_EXT      command (0x7C)
 * 2  ANALOG_READ_RES command (0x01)
 * 3  analog read resolution (1-32, see http://arduino.cc/en/Reference/AnalogReadResolution for details)
 */

/* set analogWriteResolution request
 * 0  START_SYSEX (0xF0)
 * 1  CONFIG_EXT       command (0x7C)
 * 2  ANALOG_WRITE_RES command (0x02)
 * 3  analog write resolution (1-32, see http://arduino.cc/en/Reference/AnalogWriteResolution for details)
 */

• 2 space indentation
• use spaces instead of tabs for indentation
• camel case (for both public and private properties and methods)
• use only wiring types (int, boolean, byte, etc) or include avrgcc types as well (uint8_t, etc)?
• make sure keywords.txt is up to date

void function
{
  // do something
}

Use Doxygen or other doc generator to create documentation for the Firmata library. This will require adding appropriate doc commentes to Firmata.h and Firmata.cpp.

Anyone have a preference for a C++ doc generator?

Copying this over from firmata-devel:

I heard that I2C is already supported.

What are your suggestions about adding one-wire support? How it should be done?

Thanks,
Arthur.

The fact that this repository was named "arduino" is problematic. I understand why it was named as such (because Firmata could be implemented across multiple architectures, not just Arduino) but the issue is that if someone downloads a tag they have to rename it from "arduino" to "Firmata". If someone is cloning the repo they can change the repo name at that time, but it's not so clear to a less experienced user.

The issue is however how to change the name of this repo without being disruptive since there are a number of clones and forks at this point.

I propose changing the name from firmata/arduino to firmata/Firmata. Any thoughts on how to do this without messing up everyone's existing references? One approach would be to create firmata/Firmata and copy over everything from firmata/arduino (including all issues) but keep firmata/arduino until everyone has migrated over to firmata/Firmata.

Hi,

Why not managing Firmata protocol evolutions in a specific Github repository rather than in firmata.org/Protocol and firmata.org/Proposals ?

Pros :

• Different versions managed in specific branches.
• Official version (v2.3 I think) managed in the master branch
• Proposals managed with issues and pull request
• Much more readable
• Encourage contributions

Cons : No idea for now

Of course, If you agree, I intend to contribute.

I have created a branch that includes stepper motor support. Because the Stepper library included with Arduino is blocking and relatively limited, I took the approach of including a library (FirmataStepper) with the StandardFirmata sketch. I created a custom stepper library for this application based on this paper. It is non-blocking and and supports acceleration and deceleration.

A number of requests have been made to add software serial support to Firmata. I think instead of just software serial, it would be better to support both software serial and additional UARTs (on a Mega for example: Serial1, Serial2, etc).

For those who are asking for serial support in Firmata, what are your use cases?

• read only serial devices?
• read/write serial devices
• board to board communication over serial?
• anything else?

I need to know this in order to determine how much of the Stream interface to expose. In the simplest example I assume a read buffer and a write buffer would be sufficient - simply echoing the serial data. However are there also use cases that would need the peek() or flush() methods. There are also limitations in buffer sizes (64 bytes incoming... but you have to split every byte so you only get 32 bytes... even less actually after stripping out the command bytes so more like 29 or 30 bytes of actual data).

Software serial also interferes with some timer-based features. So PWM may not work as expected when using Software serial. There are other potential collisions as well. However this can be avoided by using a Leonardo, Mega or other board that has separate hardware serial ports (Serial1, Serial2, etc). It would use the same protocol as SW serial with the main difference being that you'd configure with a port number rather than a pair of pin numbers.

I've been looking into the RAM usage by the various feature classes (utility directory) in ConfigurableFirmata. Here's what I found.

With all features included:
Available RAM ~ 625 bytes (on an Arduino Uno). My test sketch reports 598 bytes but that includes a 25 char string to send the free ram message.

RAM usage by feature (including any dependencies) on an Arduino Uno:

ServoFirmata, StepperFirmata and OneWireFirmata each use dynamic memory allocation. A stepper instance (create a new stepper in a client application) is fairly large... memory fragmentation may be an issue if using all firmata features and multiple stepper motors. The Servo ram usage is so small (5 bytes per instance) that there's little risk of issues due to memory fragmentation. I'll see if I can make any improvements to the stepper implementation.

I was not able to test the RAM usage per OneWire device instance. @ntruchsess is this something you can look into?

My process for obtaining the RAM usage was to include the FreeMemory class (pulled out of the ArduinoUnit library) with the ConfigurableFirmata sketch and send the free ram amount via sendString to a client application. I then reduced the included features to the bare minimum (FirmataReporting) and then added a single feature class at a time, recorded it's memory usage the removed the feature and repeated for each feature class. I've uploaded my ConfigurableFirmataMemCheck sketch. If you want to run it make sure you install the latest configurable branch from this repo. Then copy ConfigurableFirmataMemCheck to your Arduino Sketch directory, upload and run. You'll need a Firmata client library that implements the FirmataString protocol.

I have been looking at CAPABILITY_QUERY, PIN_STATE_QUERY and ANALOG_MAPPING_QUERY in StandardFirmata and wondering why Serial.write is used instead of Firmata.write? I ask b/c I am working on an adaptation of StandardFirmata in which I have changed the Stream being used by Firmata. I would like these messages to be sent back over my alternative stream as well.

It'd be radical to get this into Standard Firmata.

Thanks for all your hard work so far!

In addition to the firmata test application, we need integration tests and possibly also unit tests.

Integration tests would test the serial interface with Firmata and the StandardFirmata sketch. The tests should be run on a PC and exercise the Firmata API over serial or by using fixtures (data captured from the serial port that can be used to test the serial API entirely on the PC). Tests should be written in an existing, well supported test framework in a language that has good cross-platform serial support (I think python is a good candidate).

The reason for integration testing is that Firmata has become large enough to be difficult to test manually. It's difficult to determine if a change in one part of the code affects another part of the code. Having integration tests in place will also help in expanding Firmata to additional microcontroller architectures as the interface should be the same regardless of microcontroller (aside for any limitations imposed by microcontrollers such as ATTiny).

A new library specification will be introduced with Arduino 1.5.3:

https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification

See proposals: http://firmata.org/wiki/Proposals#ShiftIn.2FOut_Proposal

Need to add Leonardo to the boards.h file. Now could also be a good time to think about using pin definitions from the Arduino core and greatly reducing if not eliminating the boards.h file in the Firmata library.

StandardFirmata 2.3 (one slight modification to use Serial1 instead of Serial, as Im on a Leonardo, specifically the Seeed Studios Xadow)

i2c first write request always gets swallowed

I looked on the firmata arduino side of the board and both writes are coming through fine. I added some logging and the first transaction's wire.endtransmission is returning status 2 --nack.

I tested and its not my i2c slave, as in straight arduino code I can wire begin and send the same payload only once and it works.

I dug all around and in enableI2CPins a comment asks after After wire.begin
" // is there enough time before the first I2C request to call this here?"
I tried adding delays but that wasn't it either.

Its got to be in Wire maybe?

Anyone seen this before or can attempt it and corroborate?

ntruchsess and I began discussing the possibility of a configurable Firmata. This would be an alternative to StandardFirmata that would enable users to select features (such as I2C, Stepper, OneWire, SoftwareSerial, or anything else that is or could be implemented via the Firmata sysex protocol) to use in their Firmata application.

The way it could work is by abstracting features into classes (so you'd have feature classes such as I2C, OneWire, Stepper, SoftwareSerial, etc). These classes could be conditionally added depending on the configuration. Configuration could be done via a simple GUI application that enables users to select which features they need for their application. The configuration application would then output an .ino file (Arduino Sketch) for the user to compile and upload to their board.

These are just some initial thoughts. I'm opening this issue to start a discussion on how a configurable Firmata could be developed.

Maxuino now has i2c capability and I am beta testing it. In the GUI there is an option to select i2c as a pinMode, however I am not sure if you are supposed to select the master device's SDA and SCL or the Slaves (or both). But I also can't find any resources on how to use i2c with Firmata outside of Maxuino, in a sketch for instance.

The idea here is to create a simple site that allows the user to select an assortment of features and the service will auto generate an .ino file (that the user can download) based on the feature selection. The service will need to understand feature dependencies and auto select as needed. It should also report the estimated amount of memory the application will occupy (because as more features are added, including all features may exceed the memory of an Arduino Uno or similar so the user).

This work will make it easier: arduino/Arduino#1595

The general idea would be to add a sysex command to attach or detach an interrupt on a specified pin. The pin must be valid. This is where the referenced pull request comes into play.

Here is one possible approach:

0:  START_SYSEX
1:  ATTACH_INTERRUPT
2:  pin number (implemenation should validate)
3:  mode (low | change | rising | falling | high) *only Arduino Due supports high
4:  type (trigger [default], pulseWidth, other ideas?) *type would resolve to a particular callback function in the firmata implementation
5:  END_SYSEX

0:  START_SYSEX
1:  DETACH_INTERRUPT
2:  pin number
3:  END_SYSEX

The response would depend on the type (byte 4). By default you would get the pin number and the value. However other types such as pulse width could be specified. In this case the response would include the pin number and the pulse width. The type would support various use cases such as rotary encoders, ping type sonar sensors, photo interrupts, button presses, etc.

The interrupt handler needs to execute as quickly as possible so for the default type a flag would be set in the handler and the associated pin value would be recorded. The response would be sent in the next iteration of the main loop after the handler function returns. With something more complex like a pulse width, a change flag would be set in the interrupt handler and the pulse width would be recorded. The response would be sent in the main loop on the next iteration.

// example response
0:  START_SYSEX
1:  PIN_INTERRUPT
2:  pin number
3:  type (trigger, pulseWidth, etc) - may not be necessary since the type would be know on the client side
4:  value bits 0 6 (number of bytes in value depends on response type)
5:  value bits 7-13
n: ...
n+1  END_SYSEX

Another approach is to use a single PIN_INTERRUPT command (byte 1) and then ATTACH, DETACH and RESPONSE sub commands (byte 2).

This change has already been made in the configurable branch but should be applied to master as well. The hardware serial buffer is set to 64 bytes so this should be acceptable: https://github.com/arduino/Arduino/blob/master/hardware/arduino/cores/arduino/HardwareSerial.cpp#L59.

Some feature classes (ServoFirmata, OneWireFirmata, StepperFirmata, in configurable firmata utilize dynamic memory allocation (using the new operator). If a call to malloc (new in this case) fails an error should be reported to the user. This further reinforces the need for a trimmed down error reporting mechanism for Firmata as the current string message solution takes up too much RAM. See #80.

The error messages in StandardFirmata are all stored in SRAM. I'd like to move them out. There are a couple of options.

1 Move the error strings to program memory. The issue here is the PROGMEM macro only applies to AVR micros. However the following (stolen from ArduinoUnit) may work:

#if defined(__arm__) && !defined(PROGMEM)
#define PROGMEM
#define PSTR(s) s
#define memcpy_P(a, b, c) memcpy(a, b, c)
#define strlen_P(a) strlen(a)
#endif

You'd still need a buffer to copy the error string to in order to send it, but one 50 byte buffer takes up much less ram than all of the existing error strings. I implemented this in a test and it saved 93 bytes of ram.

2 Create a new sysexMessage type for error messages and send a numeric error code rather than a string. Client library developers would then need to provide a lookup to the string matching the error code. This would be documented on Firmata.org and within the Firmata code.

0  START_SYSEX
1  ERROR_MSG    (0x68)
2  error code
3  END_SYSEX

Personally I prefer option 2. It makes error messages explicit (so you don't have to weed them out of general string messages) and saves a lot of ram. See proposal on Firmata.org: http://firmata.org/wiki/Proposals#Error_Msg_Proposal
The only issue here would be if any client library is actually doing anything in response to the existing string messages.

Arduino Due support should be added for Arduino 1.5.1+

I've followed the installation instructions in the readme, and when I attempt to compile, I receive the following errors:

ConfigurableFirmata:42: error: 'DigitalInputFirmata' does not name a type
ConfigurableFirmata:45: error: 'DigitalOutputFirmata' does not name a type
ConfigurableFirmata:48: error: 'AnalogInputFirmata' does not name a type
ConfigurableFirmata:51: error: 'AnalogOutputFirmata' does not name a type
ConfigurableFirmata:55: error: 'ServoFirmata' does not name a type
ConfigurableFirmata:59: error: 'I2CFirmata' does not name a type
ConfigurableFirmata:62: error: 'OneWireFirmata' does not name a type
ConfigurableFirmata:65: error: 'StepperFirmata' does not name a type
ConfigurableFirmata:68: error: 'FirmataExt' does not name a type
ConfigurableFirmata:71: error: 'FirmataScheduler' does not name a type

I'm running Arduino 1.5.2 on a mac.

see proposal: http://firmata.org/wiki/Proposals#pulseIn.2FpulseOut_Proposal

Need to create a readme or wiki page that outlines the process for contributing to Firmata.

• Firmata style guide
• Feature proposals
• Testing (what to test...)
• Pull requests

Unlike the other two overloads of begin(), FirmataClass::begin(Stream&) does not call blinkVersion(). It's unclear if this a defect or by design.

Currently the Stream interface is not fully implemented. With the Stream interface implemented, Firmata should be able to work any type of data stream that implements the Stream interface (Ethernet, Wi-Fi, etc).

Any call to Serial.write in StandardFirmata should be replaced by Firmata.write (once Firmata.write is implemented).

An acceptable pull-request must be sufficiently tested to ensure that existing features are not affected. Firmata should also be demonstrated to work with soft-serial, ethernet and wi-fi Streams.

I see Paul has added Teensy 3.0 to Boards.h via the Teensyduino installer. The code should be merged into this repository at some point.

Using analog messages (ANALOG 0xE0 and EXTENDED_ANALOG 0xF0-0x6F) we can control individual PWM and Servo pins. It is not possible to set the state of a digital output pin.

When a client system needs to set a digital output port, it must now first query the state of the port, mask the pin state and write the port's state back to the Arduino.

My proposal is to extend the analog messages to enable setting digital output pins. A 0 value would then map to LOW and any non-zero value would map to HIGH.

This would also be more consistent with the Pin State Query en Set Pin Mode commands.

I realize it's a bit odd writing a digital value using an analog message, but i.m.h.o. this is due to an inconsistency in the Firmata protocol itself: analog messages sent by the Arduino reflect the state of the true analog pins, where analog messages received by the Arduino are mapped to digital (PWM and Servo) pins.

I've tried to set up a NodeJS STRING_DATA proof of concept and I'm having some strange results. Basically, the stringCallback handler is sending back more data than it has received at some point.

Here is my JS code (Arduino code is your EchoString ex):

  board.on('string', function(string){
    console.log('receiving string: "%s"', string);
  });

  var sendString = function(string) {
    console.log('sending string: ' + string);
    var bytes = new Buffer(string, 'utf8');
    var data = [];
    data.push(START_SYSEX);
    data.push(STRING_DATA);
    for (var i = 0, length = bytes.length; i < length; i++) {
        data.push(bytes[i] & 0x7F);
        data.push((bytes[i] >> 7) & 0x7F);
    }
    data.push(END_SYSEX);
    board.sp.write(data);
  }

  setInterval(function() {
    var string = 'Allyourbase';
    sendString(string);
  }, 1000);

Resulting in:

connected!
Firmware: firmata.ino-2.3
sending string: Allyourbase
receiving string: "Allyourbaseknown"
sending string: Allyourbase
receiving string: "Allyourbasen"
sending string: Allyourbase
receiving string: "Allyourbasenknown"
sending string: Allyourbase
receiving string: "Allyourbasewn"
sending string: Allyourbase
receiving string: "AllyourbaseUnknown"
sending string: Allyourbase
receiving string: "Allyourbaseown"
sending string: Allyourbase
receiving string: "Allyourbase"
sending string: Allyourbase
receiving string: "Allyourbasenown"
sending string: Allyourbase
receiving string: "Allyourbase"
sending string: Allyourbase
receiving string: "Allyourbaseknown"

Any ideas?

Servo and Stepper and using dynamic memory allocation in ConfigurableFirmata. It may be better to use static allocation to avoid potential memory issues.

Using this NodeJS implementation (https://github.com/mgcrea/node-firmata) & spamming the device with a ~5 char message every 100ms using a Teensy 3.0. The program will get stuck (memory crash?) after ~300 messages using:

void setup() {
  Firmata.setFirmwareVersion(FIRMATA_MAJOR_VERSION, FIRMATA_MINOR_VERSION);
  Firmata.attach(STRING_DATA, stringCallback);
  Firmata.attach(START_SYSEX, sysexCallback);
  Firmata.attach(SYSTEM_RESET, systemResetCallback);

  Firmata.begin(57600);
  systemResetCallback();  // reset to default config
}

void stringCallback(char *charBuffer)
{
  digitalWrite(13, HIGH);
  digitalWrite(13, LOW);
}

If I comment out // Firmata.attach(STRING_DATA, stringCallback);, the device won't crash anymore.

Looks like somehow attach() may be leaking memory?
Any ideas?

I messed this up in Firmata v2.3.5. This may affect users who are opening a serial connection upon each use (since this generates a hard reset on an Arduino board). This issue is that blinkVersion is blocking so this is blocking communication with the board for about 2 seconds after the initial firmware version and name are sent automatically after a hard reset of the Arduino.

Need to move blinkVersion() from line 71 of Firmata.cpp above the call to begin(Serial) on line 70.

It may also be helpful to create unit tests. These tests will run against individual units of code and are helpful in ensuring that everything is working as expected at a lower level. There is a lot of debate as to whether units tests should run on the microcontroller on the PC. The most interesting approaches I've seen are those that run individual or small collections of tests directly on the microcontroller, but also use a script to automatically load and run tests on the board. So you would create a test suite on the PC that loads and executes unit tests on the actual micocontroller board. Here are a couple of projects taking that approach:

• arduino_testsuite (python)
• ceedling (ruby)

Unit tests will also be specific to the microcontroller architecture whereas the integration tests will be architecture agnostic since they will test against the serial interface rather than the platform-specific implementation.

The intended usage of setFirmwareVersion is (I suspect) that it is called at most once, in which case the memory it allocates will be retained for the lifetime of the sketch (and therefore not leaked, even though it is never deallocated). However, if setFirmwareVersion is called multiple times, previously allocated memory is not released, leading to a memory leak.

Here's a unit test which demonstrates the issue (using ArduinoUnit and the MemoryFree library - from http://playground.arduino.cc/Code/AvailableMemory):

test(setFirmwareVersionDoesNotLeakMemory)
{
  Firmata.setFirmwareVersion(1, 0);
  int initialMemory = freeMemory();

  Firmata.setFirmwareVersion(1, 0);

  assertEquals(0, initialMemory - freeMemory());
}

The output indicates that 20 bytes are leaked:

Equality assertion failed in 'setFirmwareVersionDoesNotLeakMemory' 
    on line 140: expected '0' but was '20'

A change was made to Firmata in Arduino 1.5 directly in the Arduino repo. Need to reflect the same change in this repo.

arduino/Arduino#786

I am writing a hardware extension for Scratch to Arduino intercommunication and have modified Firmata 2.3.6 to accommodate generating tones on a piezo device using the tone library and also have incorporated monitoring and reporting the position of a rotary encoder. Essentially I added some Sysex commands and the code to support it. Please let me know if this is something that you might wish to add to arduino firmata as part of the standard distribution.

Thanks,
Alan Yorinks