what changes are made in the serial library about nefit-buderus-ems-bus-arduino-domoticz

[bbqkees]

Update March 2018:
See the following for a proper answer: https://github.com/bbqkees/Nefit-Buderus-EMS-bus-Arduino-Domoticz/tree/master/Arduino-Code/libraries/Nefitserial

---

If you would have taken a look into the library itself you can see that there is only one major addition (starting at line 259); the function writeEOF().
It is also nicely commented.
Just put this function into a copy of the latest Serial library and name it NefitNewSerial and use it instead of the old library.
Do not forget to make the necessary amendments to the header file too.

[bbqkees]

You can find the latest library here:
https://github.com/arduino/Arduino/blob/master/hardware/arduino/avr/cores/arduino/HardwareSerial.cpp
Please note some dependent files have also changed.

[delchrys]

I saw that part in the library. But thought that part was for writing. And I don't want to use that. And since I'm using your library it should work or not?
Maybe serial is detected by board selection (uno) so your library wouldn't be used?

[delchrys]

How about the frame error part? That's also added in the library

[bbqkees]

writeEOF is used for writing only indeed.
However, there is also the addition of frameError().
This is used in reading as well.
If I recall correctly this is used to signal the end of the data.
It is not that easy to put it back into the new library, as the error part was taken out.

[delchrys]

Is it also not possible to put that in the code instead of the library? Or any other workaround?
I'm now trying to connect my mega to my Uno. And sending the decode data from the mega (that parts works) to the uno and let the uno pass it through wifi. Maybe that works for me. Since i have no other use for them

[delchrys]

how is the frameError() part different from normal, or how is it rewrited so it signals the end of the data?
Could you giveme the line of code, soi know where to look?

[bbqkees]

It's defined in the header file. Although it is basically a simple yes/no statement, it depends on an error status that is not included anymore. 2017-05-16 11:19 GMT+02:00 delchrys <[email protected]>:

…

how is the frameError() part different from normal, or how is it rewrited so it signals the end of the data? Could you giveme the line of code, soi know where to look? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKRsgu3NQrrtZZzYul45Tnve3lbP95HCks5r6WoegaJpZM4NYdhw> .

[delchrys]

Can you think of a reason why it doesn't work when using one hardwareserial and no debugging output?

[delchrys]

Maybe you can see what's wrong?

`#define NEFIT_REG_MAX 17
#include <NefitSerial.h>
#include <avr/pgmspace.h>
#include <stdint.h>
#include<UnoWiFiDevEd.h>

PROGMEM const unsigned char regNefitCoding[]={

0x08,0x18,0x01,0x05, //#1 0 temperature water out (uitgaand) x 10
0x08,0x18,0x04,0x01, //#2 1 burner power
0x08,0x18,0x07,0x00, //#3 2 burner of/off
0x08,0x18,0x07,0x50, //#4 3 heating pump on/off
0x08,0x18,0x07,0x60, //#5 4 tap water heating on/off
0x08,0x18,0x0B,0x05, //#6 5 boiler temperature x 10
0x08,0x18,0x0D,0x05, //#7 6 temperature water in (cvreturn) x 10
0x08,0x18,0x11,0x06, //#8 7 water pressure x 10
0x08,0x18,0x12,0x02, //#9 8 status code 1st letter
0x08,0x18,0x13,0x02, //#10 9 status code 2nd letter
0x08,0x34,0x01,0x05, //#11 10 tap water temperature x 10
0x08,0x34,0x05,0x50, //#12 11 heater boiler on/off (not always available)
0x17,0x91,0x01,0x04, //#13 12 setpoint room temperature x 2
0x17,0x91,0x02,0x05, //#14 13 room temperature x 10
0x17,0xA8,0x17,0x81, //#15 14 setting 0=low, 1=manual, 2=clock
0x17,0xA8,0x1C,0x84, //#16 15 overruled clock setting x 2 ( 0 = not overruled)
0x17,0xA8,0x1D,0x84 //#17 16 manual setpoint temperature x 2
};

void printbuffer(char * buffer, int len );

char buffer[32]; // frame buffer
int nefitRegister[NEFIT_REG_MAX]; // index number see above
unsigned long register_changed =0; // flag if registy changed
byte pollAdres = 0; // if not 0, last polling address
byte nefitRegToVar[NEFIT_REG_MAX]; // Write to which variable
int regToWrite =0 ; // Split execution event handling (normally not needed)
int regValue; // Same as above
char xmitBuffer[7] = {0x0B,0x17,0xA8,0x17,0x00,0x00,0x00}; // load some defaults

long lastTime = 0;
long interval = 5000;

// Calculate CRC for buffer
uint8_t nefit_ems_crc( char * buffer, int len ){
uint8_t i,crc = 0x0;
uint8_t d;
for(i=0;i<len-2;i++){
d = 0;
if ( crc & 0x80 ){
crc ^= 12;
d = 1;
}
crc = (crc << 1)&0xfe;
crc |= d;
crc ^= buffer[i];
}
return crc;
}

// CRC check
boolean crcCheckOK(char * buffer, int len ){
int crc = nefit_ems_crc(buffer, len );
boolean crcOK= (crc==(uint8_t)buffer[len-2]);
return crcOK;
}

// Reas one bus frame, cvreturn number of read bytes
int readBytes(char * buffer){
int ptr = 0;

// nefitIO.flush();
while(nefitSerial.available()){
if((uint8_t)nefitSerial.peek()==0){
uint8_t temp = nefitSerial.read(); // skip zero's
} else { break;}
}

while((!nefitSerial.frameError())&&ptr<32){ // read data until frame-error
if(nefitSerial.available()){
buffer[ptr]=nefitSerial.read();
ptr++;
}
}
nefitSerial.flush();
return ptr;
}

// Read a register value from frame-buffer, decide if necessary and cvreturn as int
int getValue(char * buffer, byte offset, byte vartype){
int result;
uint8_t type = vartype & 0x0F;
switch (type){
case 0: //boolean
result = bitRead(buffer[offset],(vartype&0x70)>>4);
break;
case 1: //byte
// result = (uint8_t)buffer[offset];
// break;
case 2: //ascii
// result = (uint8_t)buffer[offset];
// break;
case 4: //byte x 2
// result = (uint8_t)buffer[offset];
// break;
case 6: //byte x 10
result = (uint8_t)buffer[offset];
break;
case 3: //int
// result = getDoubleByte(buffer,offset);
// break;
case 5: //int x 10
result = buffer[offset]<<8;
result = result + (uint8_t)buffer[offset+1];
break;

}
return result;
}

// Decode frame with help of the table and load register
void nefitFrame2register(char * buffer, int len){
byte sender, message;
int register_data, difference;
uint8_t offset, vartype;
for(uint8_t i=0;i<((NEFIT_REG_MAX)*4);i=i+4){
sender = pgm_read_byte_near(regNefitCoding+i);
message = pgm_read_byte_near(regNefitCoding+1+i);
if ((sender<=(uint8_t)buffer[0]) && (message<=(uint8_t)buffer[2])){
offset = pgm_read_byte_near(regNefitCoding+2+i);
if ((sender==(uint8_t)buffer[0]) && (message==(uint8_t)buffer[2])&&(offset>=(uint8_t)buffer[3])){
offset = offset - (uint8_t)buffer[3]+4; // rekening houden met lange frames (gesplitst)
if (offset<len){
vartype = pgm_read_byte_near(regNefitCoding+3+i);
register_data = getValue(buffer,offset,vartype);
if (register_data != nefitRegister[i/4])
{
nefitRegister[i/4] = register_data;
if (nefitRegToVar[i/4]>0) {
bitSet(register_changed,i/4); // alleen wanneer dit een Nodo var is
}
}
}
}
}else {break;}
}
return;
}

// Read register and convert to float
float nefitRegister2float(char regnr){
float result;
if (regnr>=0)
{
uint8_t format = (pgm_read_byte_near(regNefitCoding+3+(regnr*4)))&0x0F;
switch (format){
case 0: // read type 0, same as 1
// result = nefitRegister[regnr]; break;
case 1: // read type 1, same as 2
// result = nefitRegister[regnr]; break;
case 2:
// result = nefitRegister[regnr]; break; // read type 2, same as 3
case 3: result = nefitRegister[regnr]; break;
case 4: result = nefitRegister[regnr]/2; break;
case 5:
//result = nefitRegister[regnr]/10; break; // read type 5, same as 6
case 6: result = (float)nefitRegister[regnr]/10.0; break;
}
} else { result = -1;}
return result;
}

// -----------------------------------------------------------------------------------------------------------------

void setup(){
nefitSerial.begin(9700);

Ciao.begin();
Ciao.println("Nefit to Domoticz");

for (char i=0;i<=NEFIT_REG_MAX;i++) {
nefitRegister[i]=0;
nefitRegToVar[i]=0;
}
// We migth already be intered in some register values, set a trigger
nefitRegToVar[2] = 1; // burner on/off
nefitRegToVar[4] = 2; // wwvalve on/off

}

// -----------------------------------------------------------------------------------------------------------------

void loop(){
int ptr;
if (nefitSerial.available()){ // check if we need to read
ptr = readBytes(buffer);
if (ptr == 2) {
pollAdres = buffer[0];

}else if (ptr>4){
if (crcCheckOK(buffer,ptr)){
nefitFrame2register(buffer,ptr);
if (register_changed>0) {
if (bool burner = bitRead(register_changed,2)){ // if burner-state changed
if (burner) { // burner turns on
// startTimeBurner = millis();
} else { // burner turns off
//burnerTime = burnerTime + millis() - startTimeBurner;
}
}else if (bool wwvalve = bitRead(register_changed,4)) // wwvalve setting changed
if (wwvalve) {
// startTimeWW = millis();
} else { // wwvalve off
// WWTime = WWTime + millis() - startTimeWW;
}
register_changed = 0; // clear change flags
}
}
}
}

if(millis() - lastTime > interval) {
lastTime = millis(); 

for (int i = 0; i<=NEFIT_REG_MAX; i++){
Ciao.print("nefitRegister["); Ciao.print(i); Ciao.print("]: ");
Ciao.println(nefitRegister[i]);}
Ciao.println("END");
Ciao.println("------------------------");
Ciao.println("------------------------");

} }//end void loop`

[bbqkees]

Well I do not know what your sketch is like, what hardware you used exactly and how you connected everything so I can't tell. If you use the original Arduino library instead of this one the reason is likely the omission of frameError(). If you use this library and it does not work it is likely that other parts of the code expect functions from the original library. You should debug with an Arduino simulator. 2017-05-16 11:40 GMT+02:00 delchrys <[email protected]>:

…

Can you think of a reason why it doesn't work when using one hardwareserial and no debugging output? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AKRsgiPJdA_rMRgSL9dHfKwxr1CiRj_Eks5r6W8SgaJpZM4NYdhw> .

[bbqkees]

Does it even print "Nefit to Domoticz" when using NefitSerial?
I am not familiar with this Ciao framework, but it might rely on the original Serial library.

[delchrys]

I used your hardware circuit. On the arduino mega it works with the same code as above. On mega I connect to rx1 and on uno connect to rx0 pin0. Of course I rename nefitserial1 to nefitserial. And still #include

[delchrys]

It prints everything on my wifi console. So that parts work. Only the registers stay 0

[bbqkees]

So if I understand correctly you try to communicate with both the EMS bus and the Uno via the same serial port?

[delchrys]

Communication with the uno seems to go through the ciao.print part. It seems to send data to the wifi console. There is also a serial2wifi example on arduino.org so it must be possible to use both at the same time

[bbqkees]

So you are not using a regular Uno+WiFi shield, you are using an Uno WiFi.

Well you can try to debug via the web interface.
Add descriptive Ciao.println() statements everywhere in your code, so you can see where it goes wrong.

It is also important that when you run the sketch on the Uno WiFi, that you do not leave the board plugged into the PC via USB.

[delchrys]

everything seems to go well, except the part of displaying the values.
nefitRegister[0]: 0
nefitRegister[1]: 0
nefitRegister[2]: 0
nefitRegister[3]: 0
nefitRegister[4]: 0
nefitRegister[5]: 0
nefitRegister[6]: 0
nefitRegister[7]: 0
nefitRegister[8]: 0
nefitRegister[9]: 0
nefitRegister[10]: 0
nefitRegister[11]: 0
nefitRegister[12]: 0
nefitRegister[13]: 0
nefitRegister[14]: 0
nefitRegister[15]: 0
nefitRegister[16]: 0
nefitRegister[17]: 18734
END

---

This is the output on my ciao wifi console, and it isn't connected to my PC, only USB powering the arduino.

[bbqkees]

So summarized my sketch works als long as you use a Mega with USB serial to pc.
So the initial code and the EMS bus converter hardware is fine.

Therefore the error is likely in the software on the Uno WiFi side.

First try to communicate from PC to Uno WiFi via the USB serial (with another sketch) while using the default Serial library. So open the serial port on the PC and try to send data to the webinterface.
This will likely work.

Now try the same but replace the default Serial library with the NefitSerial library.
If that works as well, the problem could be your sketch itself.

If that does not work, the problem is likely on the library side in the Uno.

[delchrys]

will try that

[delchrys]

found this on the net.
Maybe this is what's happening?
Wifi.write() function only compiles for Wifi.write(val),
refuses to compile for
Wifi.write(str) or
Wifi.write(buf, len)

Are the serial and stream libraries supposed to work for Arduino UnoWifi?
I sure hope so! Perhaps I'm doing something wrong?
It seems that this board works very differently in coding then the Uno with wifi shield. It's very confusing.

Can i write the buffer to a value(maybe int) and then print it????

[bbqkees]

Try converting the array values as float and char values before passing them to the println function (just a few outside of your timed loop, so they print continuously). I believe that could be the problem.
Not that the value is not there, but it is printed incorrectly.
f.i.
Ciao.println((float)nefitRegister[0]/10,1);

[delchrys]

put that in loop but outside the millis timing you mean?

[bbqkees]

Just comment out the last loop in your sketch and below that add Ciao.println((float)nefitRegister[0]/10,1);
So don't execute that loop, just check it for this one register.

[delchrys]

it print 0.2

[bbqkees]

If you try my block on line 533 you should get something like this:

---

nefitRegister[0]: 651
nefitRegister[1]: 0
nefitRegister[2]: 0
nefitRegister[3]: 0
nefitRegister[4]: 1
nefitRegister[5]: -32768
nefitRegister[6]: 628
nefitRegister[7]: 27
nefitRegister[8]: 48
nefitRegister[9]: 72
nefitRegister[10]: 574
nefitRegister[11]: 1
nefitRegister[12]: 0
nefitRegister[13]: 0
nefitRegister[14]: 0
nefitRegister[15]: 0
nefitRegister[16]: 0
nefitRegister[17]: 9

NEFIT EMS bus:
cv water uitgaand: 65.1
brander vermogen: 0.0
brander aan/uit: uit
cv pomp aan/uit: uit
warm watervraag aan/uit: aan
keteltemp: n/a
cv retour temp: 62.8
waterdruk: 2.7
statuscode: 0H
warmwater temp: 57.4
boiler verwarming aan/uit: aan
END

---

I can recall seeing the same output as you have at some point, but I cannot remember what the problem was.
Maybe timing.

[delchrys]

On the mega it looks that way indeed. but not on my UNO.

[delchrys]

since i'm not into hard core coding can you make an example of a value written to a register and the print a register?
that way i can check if it goes wrong

[delchrys]

i renamed my oiginal hardwareserial and now it doesn't compile anymore so it still seems to use hardwareserial instead of nefitserial.

[delchrys]

i'm now rebuilding and making a i2c connectionto my UNO. Can you help me in putting all the values in a string? Don't know how to do that since there are floats and chars.
Maybe you can help me with that so i only need to send one string with all values in it. The uno will split this ad send it over domoticz with ciao.write

[delchrys]

which part in the nefitserial library is exactly responsible for the stop/Break part??
I'm now trying to rewrite the library for softserial

[delchrys]

tested the following:
1- Uno in passthrough mode (reset and round shorted) Data comes in. Works
2- Mega in passthrough mode (reset and round shorted) Data comes in. Works
3- Mega loaded with softserial to ouput to my UNo Data comes in. Works
4- Rewritten step 3 (so softserial) from nefitSerial1 to nefitSerial. No Data empty. so that doesn't work.

So i think it goes wrong at the point of using nefitSerial instead of NefitSerial1, strange somehow the messages aren't decoded properly so the registers stay empty.
Maybe because it's UNO it still uses hardwareserial or there's something in the nefitSerial library that isn't compatible with my UNO (atmega328p)

[bbqkees]

Well step 1 and 2 have nothing to do with the Uno itself, you just basically hardwire Tx and Rx to the PC.
I have no idea what you mean with point 4.

I will have another look at the library, but not anytime soon.
For the time being:
Just use the normal sketch for the Mega and connect Rx0 and Tx0 to the Uno WiFi.
In the Uno just copy what you receive via Rx and Tx to the Ciao module.
In that way you can use NefitSerial on the Mega and the default Serial on the Uno WiFi.

[delchrys]

it seems the editted library is for Atmega8 and a UNO isn't.
So i have to make some adjustments to the new hardwaeserial to make it work. Don't know how yet but i will figure it out eventually.
Called in the help from some arduino.cc guys, maybe they can figure out where it goes wrong.

[delchrys]

hi bbqkees, i have still one question left.
I'm getting close but jus one more thing.
Can you please exlpain me how the offset part works specifically the last code of the regNefitCoding:
so:
0x08,0x18,0x07,0x00, ----- 08-18-12-the first bit?????
0x08,0x18,0x07,0x50, ----- 08-18-12-the fifth bit????

i do not understand the last digits of the coding part so the values 0x00,0x01,0x02,0x05,0x06,0x50 and 0x60.
Do they refer to the different cases?? case 0, case 1, case 2, etc??

It'sthe last piece of the puzzle i'm trying to understand

[bbqkees]

To understand the last part you need to think in binary and hexadecimal.

There are probably easier to understand methods to do what is going on here (I did not create the original sketch) but I think if you use the programmer view of the Windows calculator it will make more sense. In the end it has to do with bit masking etc.

So 1 to 6 likely make sense, it is hexadecimal. So I should have noted in the comments 0x01 to 0x06.
Now the higher numbers:
f.i. on line 102: 0x60 //# 5 4 tap water heating on/off.
In the comments you can already see it's an on/off thing, so a boolean.
0x60=0b01100000. So byte 7-4 is 0110 and byte 3-0 is 0000.
In the comments it is also mentioned that byte 6-4 is the bit number within the byte that indicates the on/off for that function.
Therefore we need to focus on 0b110. This is 6 DEC.

If you check the EMS Wiki you can see that for UBAMonitorFast at 12 DEC (Which is 0x07 + the offset of 5) that this is one byte of which every bit is the on/off indicator of a certain boiler function.
So, if the byte is 0b00000000, everything is off. If this byte is f.i. 0b00000001, the heater is on.
If it is 0b01000000, then the hot tap water is on (indicated by '3-Wege-Ventil auf WW' = 3 way valve set to hot water'). Counting from 0, this is the 6th bit.
(Of course, this byte can also be 0b01000001, which would mean both the heater and hot tap water are on.)

This is the same for for instance 0x08,0x18,0x07,0x50, //# 4 3 heating pump on/off.
Same byte as above, but now it is 0x50, which is 0b 0101 0000. 0b101 =5.
So bit 5 indicates 'Kesselkreispumpe EIN' which is the heating pump.

Now again take 0x60 and use this as input for the function on line 237.
0x60 & 0x0F = 0, so case 0.

Input this in case 0 and it will do bitRead(buffer[offset],(vartype&0x70)>>4).
See https://www.arduino.cc/en/Reference/BitRead
So read byte 0x07, and in that byte bit (0x60&0x70)>>4.
(0x60&0x70)>>4 means the following: 0b0110 0000 AND 0b0111 0000 = 0b0110 000.
Now bitshift this 4 to the right, which will end up in 0b0000 0110 =0b110=6 DEC.
So read bit 6 of byte 0x07.

[delchrys]

ahh so in 0x60 we have:
0 for 0
x for x
6 for the sixth bit
and 0 for indicating boolean so refering for case 0??

so theoreticaly you can have 0x30
so third bit and boolean

[bbqkees]

0x means what follows is in hexadecimal notation.
0b means what follows is in binary notation.

[delchrys]

Still can't figure out why my 0x08.0x34.0x01.0x05 does not work hot water temp.
It is the same parameter setup as 0x08.0x18.0x01.0x05 cv uitgaands using start dez 2 and multiplier 10 according to UBAMONITORWW. they seem the same parameters only different addresses 18 vs 34. And 08 18 01 05 is working properly.

Maybe I have to check 08 34 03 for checking fuhler 2?

Or check the 08 34 08 for checking fuhler defects. Maybe that's why my registers stays -3276.8

[bbqkees]

What boiler do you have?
Maybe you do not have the type 34 message or you do not have the sensor.
If your register value stays 32768 this would mean you receive 0x8000 which indicates you do not have the sensor at all.

[delchrys]

Nefit proline hrc24cw3

[bbqkees]

So that is an OpenTherm model. There must be a converter used if you are able to read the EMS bus. Either there is no message 34 sent through the converter or you don't have the sensor.

[delchrys]

Can I do a nefitserial.print(buffer);
To show the raw untouched incoming data?

[delchrys]

Don't think its opentherm

[delchrys]

Because the other parameters are working correctly.

[bbqkees]

No you are right, I checked the wrong manual (of the Proline NXT). However as you are only missing this one parameter I suggest you just leave it as it is.

[delchrys]

Yeah I also thought of that. Any tips on how to serial print the buffer? Is it just as simple as serial.print(buffer)?

[bbqkees]

In the mean time I have documented the library as well. so all changes are detailed here