hsaturn / tinymqtt

ESP 8266 / 32 / WROOM Small footprint Mqtt Broker and Client

TinyMqtt

TinyMqtt is a small, fast and capable Mqtt Broker and Client for Esp8266 / Esp32 / Esp WROOM

Statuses of all unit tests of TinyMqtt and its dependencies

Project	Unit tests result
TinyMqtt
Dependencies
TinyConsole
EpoxyDuino
EspMock
AUnit
AceRoutine

Features

• Supports retained messages (not activated by default)
• Async Wifi compatible (me-no-dev/ESPAsyncTCP@^1.2.2)
• Very fast broker I saw it re-sent 1000 topics per second for two clients that had subscribed (payload ~15 bytes ESP8266). No topic lost. The max I've seen was 2k msg/s (1 client 1 subscription)
• Act as as a mqtt broker and/or a mqtt client
• Mqtt 3.1.1 / Qos 0 supported
• Wildcards supported (+ # $ and * (even if not part of the spec...))
• Standalone (can work without WiFi) (degraded/local mode)
• Brokers can connect to another broker and becomes then a proxy for clients that are connected to it.
• zeroconf, this is a strange but very powerful mode where all brokers tries to connect together on the same local network.
• small memory footprint (very efficient topic storage)
• long messages are supported (>127 bytes)
• TinyMQTT is largely unit tested, so once a bug is fixed, it is fixed forever

Limitations

• Max of 255 different topics can be stored (change index_t type to allow more)
• No Qos because messages are not queued but immediately sent to clients

Quickstart

• install TinyMqtt library (you can use the Arduino library manager and search for TinyMqtt)
• modify <libraries/TinyMqtt/src/my_credentials.h> (wifi setup)

Examples

Example	Description
client-with-wifi	standalone example
client-without-wifi	standalone example
simple-client	Connect the ESP to an external Mqtt broker
simple-broker	Simple Mqtt broker with your ESP
tinymqtt-test	Complex console example

• tinymqtt-test : This is a complex sketch with a terminal console that allows to add clients publish, connect etc with interpreted commands.

Retained messages

Qos 1 is not supported, but retained messages are. So a new subscription is able to send old messages. This feature is disabled by default. The default retain parameter of MqttBroker::MqttBroker takes an optional (0 by default) number of retained messages. MqttBroker::retain(n) will also make the broker store n messages at max.

Standalone mode (zeroconf)

-> The zeroconf mode is not yet implemented zeroconf clients to connect to broker on local network.

In Zeroconf mode, each ESP is a a broker and scans the local network. After a while one ESP naturally becomes a 'master' and all ESP are connected together. No problem if the master dies, a new master will be choosen soon.

TODO List

• Use Async library
• Implement zeroconf mode (needs async)
• Add a max_clients in MqttBroker. Used with zeroconf, there will be no need for having tons of clients (also RAM is the problem with many clients)
• Why not a 'global' TinyMqtt::loop() instead of having to call loop for all broker/clients instances
• Test what is the real max number of clients for broker. As far as I saw, 1k is needed per client which would make more than 30 clients critical.
• MqttClient auto re-subscribe (::resubscribe works bad on broker.emqx.io)
• MqttClient auto reconnection
• MqttClient user/password
• Wildcards (I may implement only # as I'm not interrested by a clever and cpu consuming matching)
• I suspect that MqttClient::parent could be removed and replaced with a simple boolean (this'll need to rewrite a few functions)

License

Gnu GPL 3.0, see LICENSE.

tinymqtt's Issues

Version 0.9.12 was working but when recompiled throws errors in TinyString.cpp

Error messages for str, size and free
exchange is not a member of std
namespace std has no member exchange

TinyString& TinyString::operator=(TinyString && other)
{
if (this == &other) return *this;
clear();
str = std::exchange(other.str, str);
size_ = std::exchange(other.size_, size_);
free_ = std::exchange(other.free_, free_);
return *this;
}
and the same problem with 0.9.18
Is there a way of not loading TinyConsole?
Using VSCode and PlatformIO

zeroconf

Any idea when zeroconf will be implemented? Its sounds like a great feature!
Also, are there a chance to get websockets supported?
How much clients (on broker) can currently supported/handle by a esp32 ?

Bridged connection from mosquitto broker sets protocol version to 0x84, cannot connect to TinyMqtt

Seems as though the mosquitto broker will set protocol version in the connect header to 0x84, rather than 0x04 for 3.1.1 connections. They use the 0x80 to identify a bridged connection.

I.E.:

*************** Error msg 0x10-------ERROR ------ size(100), state=4
-------ERROR ------ | 10 62 00 04 4D 51 54 54  | .b.. - MQTT |
-------ERROR ------ | 84 2C 00 0A 00 1A 63 61  | �,.. - ..ca |

To be able to accept a connection to the TinyMqtt broker from a mosquitto bridge, one needs to mask header[6] before testing:

if ( (header[6]&0x7F) != 0x04 )
{
	debug("unknown level");
	break;	// Level 3.1.1
}

I cannot locate their use of the spare protocol bits in the MQTT specification. Perhaps they are ahead of the spec.

std::string

I'm opening an issue about Bazmundi comment on #47
and will take care about this this evening.

(copy of Bazmundi comment:)

I would LUV an example of how to set the client ID on the remote client.

On the remote client I am using:

const char* BROKER = "192.168.4.1";
const uint16_t BROKER_PORT = 1883;

static MqttClient pump_client;

...

pump_client.connect(BROKER, BROKER_PORT,60);

What's the trick please? The std::string & is driving me batty 🦇

Message ID

Do you capture the message ID of transmitted messages? I could not find anything hinting on on a message ID, so I did a quick and dirty function to return the ID of a message.

in TinyMqtt.h I have added to class MqttMessage:

const char* PublishID() const { return &buffer[(static_cast<uint16_t>(buffer[3]) << 8) + static_cast<uint16_t>(buffer[4]) + 5];}

in TinyMqtt.cpp I have added to void MqttClient::processMessage(MqttMessage* mesg)
inside

case MqttMessage::Type::Publish:

uint8_t qos = (mesg->flags() / 2) && 3;
if (qos == 1) 
        { 
          auto ID = mesg->PublishID();
          MqttMessage msg(MqttMessage::Type::PubAck);
          msg.add(ID[0]);  // MessageID high
          msg.add(ID[1]);  // MessageID low
          msg.sendTo(this);
        }        

Truncated payload value in binding class when retained messages

I tried to run your mqtt_class_binder example with retained messages and the 2 first characters are truncated in the payload variable in the onPublish method.
However, the value seems correctly retained in the broker.
mqtt_sender.publish(topic_sender, "sent by Sender, message #"+std::string(String(counter++).c_str()), true);

Incorrect message length calc

It looks like message length(function "incoming") should be calculated this way:
case Length:
size += (in_byte & 0x7F) *size_mult;
size_mult *=0x80;
.............

size_mult should initialize with "1" in FixedHeader section

How to see message arriving at MQTT broker

Dear friends,

We are establishing local MQTT communication on esp32 with the help of this library. Basically the message is visible on end node. But it is not visible on local broker. Any help on it, It will be appreciable.

Thanks.

broker without WiFi but "Ethernet.h" connection

Hi,

I am searching for ESP32 MQTT broker library (esp connected with W5500 instead of WiFi) but havent had much luck :)

In feature list you state: Standalone (can work without WiFi) (degraded/local mode) => Degraded? meaning without any net connection? as I couldnt make it work with Ethernet lib.

ESP.getFreeHeap() kill debug on non-esp cores

Please allow some way to ignore ESP.getFreeHeap() in TinyMqtt.cpp line 397 here as it kills using debug on a non-esp board.

My suggested method would be, but you probably have a better method.

was:

#ifdef TINY_MQTT_DEBUG
if (mesg->type() != MqttMessage::Type::PingReq && mesg->type() != MqttMessage::Type::PingResp)
{
	Serial << "---> INCOMING " << _HEX(mesg->type()) << " client(" << (dbg_ptr)client << ':' << clientId << ") mem=" << ESP.getFreeHeap() << endl;
	// mesg->hexdump("Incoming");
}
#endif

to something like:

#ifdef TINY_MQTT_DEBUG
if (mesg->type() != MqttMessage::Type::PingReq && mesg->type() != MqttMessage::Type::PingResp)
{
 #ifdef NOT_ESP_CORE
	Serial << "---> INCOMING " << _HEX(mesg->type()) << " client(" << (dbg_ptr)client << ':' << clientId << ") mem=" << " ESP.getFreeHeap() "<< endl;
#else
	Serial << "---> INCOMING " << _HEX(mesg->type()) << " client(" << (dbg_ptr)client << ':' << clientId << ") mem=" << ESP.getFreeHeap() << endl;
#endif

	// mesg->hexdump("Incoming");
}
#endif

That change will hopefully allow other boards to get this important library running.

TinyConsole.h: No such file or directory

Hello,when I want deloy the library in my ESP32. I use arduino ide to compile the firmware.But the compiler return message "fatal error: TinyConsole.h: No such file or directory" for me.How can I fix the proplem?Thanks!

Platformio update

Hi,

I'm using platformio for my esp32/8266 projects and I would like to use your library for a school project :) But I would like to make use of wildcards and the version of your library on platformio is 0.8.0 (2 years old). Could you update it please? Thank you :)

Martin

Portenta Patch

Hi @hsaturn have you seen this patch suggestion from Martino at Arduino?

arduino/ArduinoCore-mbed#281

diff --git a/src/TinyMqtt.h b/src/TinyMqtt.h
index b1a1cd3..6919c48 100644
--- a/src/TinyMqtt.h
+++ b/src/TinyMqtt.h
@@ -14,6 +14,10 @@
        #ifdef TCP_ASYNC
          #include <AsyncTCP.h> // https://github.com/me-no-dev/AsyncTCP
   #endif
+#elif defined(ARDUINO_PORTENTA_H7_M7)
+  #include <WiFi.h>
+  #include <WiFiClient.h>
+  #include <WiFiServer.h>
 #endif
 #ifdef EPOXY_DUINO
   #define dbg_ptr uint64_t

I am testing it out today.

...

....

Later: I think these changes are for the next Arduino MBED release, so I will have to wait a bit before testing them. I can close this for now.

Invalid CONNACK packet

The CONNACK packet generated by TinyMqtt doesn't look right.

TinyMqtt generated this: 20 82 00 00 00
I was expecting: 20 02 xx xx (Assuming MQTT 3.1.1)

This bug prevents Adafruit_MQTT_Library from connecting to this broker. I'm sure other MQTT clients will have the same issue.

Client life, local to broker, too short?

On my AP host running a tinyMQTT broker, I have the code:

MqttBroker broker(1883);
MqttClient peddle_client(&broker);
MqttClient pump_client(&broker);

The pump_client is acting as a proxy for the remote pump.

There is a connection topic that the pump publishes, to tell the pedal it's up.

The proxy pump client then does a few things, including resetting a count so that the debugging on both devices keeps pedal presses since pump reset in sync. Counting starts from 1 on reset of pump and on receipt of pump connection on pedal.

However, the pump client on the pedal runs out of life early, and so never catches any subsequent resets of the actual pump. I get the inaugural connection topic, and any re-connections, but only up to the life expiry of the pump proxy running on the pedal host.

I have set the default keep_life void connect(std::string broker, uint16_t port, uint16_t keep_alive = 65535 ), along with the hack of onConnect that ensures the top 8 bits of keep_alive is posted; to max out the uint16_t to give me 18 plus hours of connection. This is fine for instruments I am running this on in my lab. I won't have the instruments up for that long (a couple to 6 hours probably).

This works fine for remote clients, but does not work for local clients. A sneaky attempt to hard-code keep_alive = 65535 in void connect(MqttBroker* parent); did not work, not sure then how to max out life of local clients.

I am currently guessing the local clients are only alive for 30 seconds after creation? Or 30 seconds topic processing?

client ceases to respond to topics after a short number of events

I have two ESP32 widgets, one set as an AP and one as a STA. I can get the STA to connect to the AP and I appear to be able to publish a topic on the AP and the STA does receive topics (upwards of 5 but often less), then I get no further responses.

Code for AP

The AP is a WEMOS BAT with a foot pedal

Code for STA

The STA is a WEMOS D1 R32 that will have a UNO relay shield. It will drive an air pump and pneumatic "switch", the code aims to turn on pump then the pneumatic switch, when the pedal is pressed. The air stays on again until the pedal is pressed again, in which case the pneumatic switch is turned off then the pump is turned off.

Since the STA connects to the AP, and then I get through a few responses, I am uncertain whether this is, as yet, a bug in the TinyMQTT code. I will do a little more poking around, but if anyone can see something in the source code (for either AP or the STA) that jumps out at you, thanks ahead of time.

I have snipped out the relay code (in the STA) to leave:

void loop()
{
  pump_client.loop();
}

That seems to allow more topic responses to succeed, but I only get closer to 10 before it stops.

If I connect a terminal to the AP, it happily keeps reporting the foot press code (but that does not guarantee the messages are still going out to the broker). So, without understanding how the TinyMQTT code operates, whether it's the broker or client end amiss, it's a little hard to work out where to start debugging.

Basic design then is AP has a broker and then two clients, one to act as the pedal (to send peddle presses) and one as the proxy for the pump (to catch the pump connection).

I added a callback to the AP and subscribed to the pedal press topic. It does appear the broker stops sending out topics after a few have been sent out. That is, whether the client is on the AP, or on the STA, the messages seem to stop eventually.

The other behaviour seems to be, when using the STA to connect to the AP, if I reset the STA, the STA will sometimes respond to topics for a short time, then stop again.

Any suggestions please?

Memory leak when clients connected and disconnected

I have a simple MqttBroker setup, listening on a socket. I have been doing some testing that requires repeated connection opening/closing and I'm getting weird behavior. I added some more debugging and found there appears to be a memory leak related to TinyMqtt.

To reproduce this I am doing the following:

$ while true
do
mosquitto_pub -h <my_esp> -p 1883 -t foobar -m '{}'

done

I then have my esp periodically dumping memory stats. Effectively:

// every second
Serial.print("free_heap=");
Serial.println(xPortGetFreeHeapSize());

With that, I'm seeing every time I connect I have less memory available.

EDIT: I had some thoughts below on what was causing this, but I now found I was incorrect. I'm still seeing the memory leak, but I'm not clear what's causing it.

My guess is it's not as simple as the clients list not getting purged (I think I understand how that's happening now), but that the MqttClient itself isn't getting cleaned up.

Fail connect ESP8266 client

Tried example simple_broker with a client esp8266 using lib pubsubclient https://github.com/knolleary/pubsubclient
and connection to broker failed.
this the code:

#include <PubSubClient.h>

// Update these with values suitable for your network.

const char* ssid = "-----";
const char* password = "-------";
const char* mqtt_server = "192.168.1.116";

WiFiClient espClient;
PubSubClient client(espClient);
unsigned long lastMsg = 0;
#define MSG_BUFFER_SIZE  (50)
char msg[MSG_BUFFER_SIZE];
int value = 0;

void setup_wifi() {

  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  randomSeed(micros());

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.print("Message arrived [");
  Serial.print(topic);
  Serial.print("] ");
  for (int i = 0; i < length; i++) {
    Serial.print((char)payload[i]);
  }
  Serial.println();

  // Switch on the LED if an 1 was received as first character
  if ((char)payload[0] == '1') {
    digitalWrite(BUILTIN_LED, LOW);   // Turn the LED on (Note that LOW is the voltage level
    // but actually the LED is on; this is because
    // it is active low on the ESP-01)
  } else {
    digitalWrite(BUILTIN_LED, HIGH);  // Turn the LED off by making the voltage HIGH
  }

}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Create a random client ID
    String clientId = "ESP8266Client-";
    clientId += String(random(0xffff), HEX);
    // Attempt to connect
    if (client.connect(clientId.c_str())) {
      Serial.println("connected");
      // Once connected, publish an announcement...
      client.publish("outTopic", "hello world");
      // ... and resubscribe
      client.subscribe("inTopic");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

void setup() {
  pinMode(BUILTIN_LED, OUTPUT);     // Initialize the BUILTIN_LED pin as an output
  Serial.begin(115200);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
  client.setCallback(callback);
}

void loop() {

  if (!client.connected()) {
    reconnect();
  }
  client.loop();

  unsigned long now = millis();
  if (now - lastMsg > 2000) {
    lastMsg = now;
    ++value;
    snprintf (msg, MSG_BUFFER_SIZE, "hello world #%ld", value);
    Serial.print("Publish message: ");
    Serial.println(msg);
    client.publish("outTopic", msg);
  }
} 

Message:
Attempting MQTT connection...failed, rc=-1 try again in 5 seconds

the simple_broker is in a ESP32. I can connect with MQTTX app https://mqttx.app/

thank you

Bad payload received by local client

Steps to reproduce

1 - Compile and run tinymqtt-test.ino
2 - Connect an mqtt explorer (ex: http://mqtt-explorer.com/)
3 - Use the following 'script'
3.1 > ls -> ensure mqtt-explorer is connected
3.2 > client c broker -> create a local client
3.3 > c.pub sensor 128 -> 128 should be received by mqtt-explorer
3.4 > c.sub # -> local client subscribes to all
3.5 > c.pub value 120 -> 120 received by mqtt-explorer but bad payload received by the local client !

Question - How many client handle it? (ESP32-LAN)

Hello there,

What do you think, how many devices able to handle this thing a similar hw?

https://www.olimex.com/Products/IoT/ESP32/ESP32-POE-ISO/open-source-hardware

RP2040 support

Add support for RP2040

suggestion has been made, comprising of adding in TinyMqtt.h:

#elif defined(ARDUINO_ARCH_RP2040)
#include <WiFi.h> // works with Raspberry Pi Pico W, earlephilhower

direct use of WiFiClient ?

Hello, I just forked your nice bit of work (thank you) and modified it to accommodate for W5500 LAN adaptors. Pretty much anything supported by the Ethernet library should work too. But I have not yet included other controllers than the ESP8266.
Thanks to you the adaption was fairly straight forward, mostly just needing a "using TcpClient = EthernetClient;" + server and some initialisation.

I noticed 2 instances of direct use of WiFiClient instead of TcpClient in TinyMqtt.cpp (lines 66 & 214 in the current main). Is there a particular reason? I changed them in my fork and it worked fine so far.

Suggestion:
change
tcp_client = new WiFiClient(*new_client);
to
tcp_client = new TcpClient(*new_client);

and change
WiFiClient client = server->accept();
to
TcpClient client = server->accept();

(Another problem I had, I could neither get the master or the later releases to work. They compiled fine but none of my clients connected to the broker. So I went back to the working release from the Arduino library manager, which I know to be working fine. I must admit, I did not spend a great deal of time yet on fault finding.) <= side note only

My main intention was to get it working on the ESP + W5500 first as I intended to port it to the RP2040 W5500-EVB-Pico at a later point.

Edit: Which I have done now.

where is the my_credentials.h ???

the readme says to modify a file, but it is not in the project

Broker Never Connects

I used the sample broker ino file, but the broker never connects to the router. I tried several ESP8266 devices (all able to connect as mosquitto clients, FWIW), and I tried two different networks/routers. All to no avail. I'm not sure what the issue is.

QoS level greater than 0

Hello and thanks for this library.
I would like to ask if there is any possibility to implement a greater QoS than 0?
Thank you,
Francesco

Memory leak due to removal of MqttClient objects from the clients list

The MqttClient::close() function removes the MqttClient object from the broker's client list. But the object is not destroyed and memory leaks over time. It looks like the parent.removeClient function should be moved from the close() function to the place where the MqttClient object is being deleted.

Cannot install library from IDE

The library cannot be installed from the latest IDE, and can be installed under Arduino IDE 1.8.18 but only by disabling dependencies.
I suspect that AsyncTCP library has disapear.

Suppport for MQTT over Websocket

Hello does this support MQTT over Websocket ?

Write unit test for length message decoding

The length msg is limited to 4k, and has a hard limit to 16k, but yet, no unit tests is verifying the length when decoding messages.

This will involve to simulate some tcp/ip packet send / recceive in EspMock

The goal would be to

• check length (and content) of decoded messages that are shorter than 128bytes
• same for messages that are < 16Kb

Note: messages cannot go beyond the limit if 16K because only two bytes are taken when decoding the message.
This is not a bug, I don't want an Esp to have to handle message of more than that, just because the ram is limited in such devices.

Support large messages

Hi,

at the moment there is no support for messages larger than 127 bytes.
The broker get the correct message with size, tries to publish the message.
The connection to the client will be disconnected after sending.

I´ve added the size calculation from the other issue.

Hive MQ fails to connect

I spent a lot of time debugging my arduino, trying to figure out why my calls were all failing. Turned out to be due to the mqtt cli using v5, while this library only supports v4. By mqtt cli I mean the hivemq one, which is the first one that shows up on google searches.

Being new to mqtt this burned me for awhile, so I'm hoping this helps the next person. Both by having this PR show up if they search, and making the error more informative.

Bind a class function to setCallback function

I would like to bind a class function to the setCallback function. Therefor I declared the callback function like that:

class MyClass
{

  private:
  void callbackFunction(MqttClient::CallBack cb);
  ...
}

and try to pass it like that:

client->setCallback(std::bind(&MyClass::callbackFunction, this, std::placeholders::_1));

I want to do that because I want to access dynamic variables of MyClass.

The problem:
I cannot define my callbackFunction with MqttClient::Callback as function parameter because the type MqttClient::CallBack is inaccessible.

Is there any solution how I can access variables of MyClass within the function body of my callbackFunction?

Arduino Portenta Potential?

This is really cool.

Any chance this could work on the new Arduino Portenta MBED capable board?

multiple retained messages for the same topic

TinyMqtt.cpp, line 965:

auto old = retained.find(topic);

does not work as expected. Consequentially I get multiple retained messages for the same topic.

Arduino 2.1.0

Also Retain seems to store the whole Publish message instead of just the message string, which is quite a bit of overkill (memory consuming) at long topics. In accordance to HiveMQ (https://www.hivemq.com/blog/mqtt-essentials-part-8-retained-messages/) it should just be the QOS and the message string itself.
Would it not be better to just store a reference to the topic?
Then duplicates would be far less likely to occur by design and the load on the memory would be lesser. Also the handling of empty messages (delete/ignore) would be much easier.

To replicate the issue, initialise MqttBroker with a value >1 for retained values and send multiple retained messages to the same topic. Connect a client and subscribe to the topic. Now you should see all retained messages popping up, instead of just one.

I have traced the error to the aforementioned line and a debug output there reveals that [topic] is never found. Using a quoted string as argument on the other side does return a found result.

Warning compiling for ESP8266

Warning compiling for ESP8266

libraries/TinyMqtt/src/TinyMqtt.cpp:173:41: warning: 'WiFiClient WiFiServer::available(uint8_t*)' is deprecated: Renamed to accept(). [-Wdeprecated-declarations]
  173 |   WiFiClient client = server->available();
      |                                         ^
In file included from .arduino15/packages/esp8266/hardware/esp8266/3.1.1/libraries/ESP8266WiFi/src/ESP8266WiFi.h:40,
                 from Arduino/libraries/TinyMqtt/src/TinyMqtt.h:15,
                 fromArduino/libraries/TinyMqtt/src/TinyMqtt.cpp:2:
.arduino15/packages/esp8266/hardware/esp8266/3.1.1/libraries/ESP8266WiFi/src/WiFiServer.h:85:14: note: declared here
   85 |   WiFiClient available(uint8_t* status = NULL) __attribute__((deprecated("Renamed to accept().")));

Incomplete decoding of topic names

I have a device with a MQTT broker and a local client (just as in the example code). On the device side I publish two topics with similar names, where the name of the second is included in the name of the first, as this:
a) sensor/temperature (a floating point value, encoded as string, e.g. "19.8")
b) sensor/temperature_raw (a raw decimal value, e.g. 19384)

When I query the value of "temperature_raw" from a remote machine (e.g. by using mosquitto_sub) it seems that the broker does not properly evaluate the topic name, as it returns the value of "temperature" instead.

This bug is 100% reproducible and can be easily worked around by renaming the topic, in my case it started to work as expected after I renamed the topic to "sensor/raw_temperature".

However, this may be problematic when using nested topics, for example "/actor/xxx" for a read-only value and "/actor/xxx/set" for writing.

Wildcard should not appear in publish

I may close this issue, but Mqtt specification is clear: Wildcards cannot appear in a PUBLISH topic.

I don't really want to parse every publication for performances reason, So I'm pretty sure I won't implement this.
The topic compare function will not interpret the wildcard for the published topic, but only for the subscribed one. So this should not be a real problem.
Anyway, publishing wildcards can leads to stranges behaviour. I consider that the developper won't do such a bad thing so ...

Will it be ESP32 compatible in future?

Nice job. Are you planning to make it compatible with ESP32 in future?

Thanks.

Topic Wildcards

See there lasernav@e61d7ad

I may get this commit and add unit tests for that

Client life, local to broker, too short

Hello

I'm starting this new issue and close others in order to solve this.

I'm running a setup for the day on the main branch (not a release).
If this evening this setup still runs, then I'll close all timeout related issue and will release a new version of TinyMqtt

My setup is

• ESP8266 Broker with one local client, derived from client-with-wifi.ino:
• ESP8266 Client, I'm using a derived code from example/simple-client.ino for this
• MQTT Explorer to see what happens

---

The ESP Broker sketch, derived from client-with-wifi (as alranel said in previous discussion):

#include <TinyMqtt.h>   // https://github.com/hsaturn/TinyMqtt

const char *ssid     = "... my ssid ... ";
const char *password = "... my password ...";

std::string topic="sensor/temperature";

MqttBroker broker(1883);
MqttClient client(&broker);

void onPublish(const MqttClient* /* source */, const Topic& topic, const char* payload, size_t /* length */)
{ Serial << "--> client A received " << topic.c_str() << ", " << payload << endl; }

void setup()
{
  Serial.begin(115200);
  delay(500);
  Serial << "Clients with wifi " << endl;

	if (strlen(ssid)==0)
		Serial << "****** PLEASE EDIT THE EXAMPLE AND MODIFY ssid/password *************" << endl;

  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {   Serial << '-'; delay(500); }

  Serial << "Connected to " << ssid << "IP address: " << WiFi.localIP() << endl;

  broker.begin();

  client.setCallback(onPublish);
  client.subscribe(topic);
}

void loop()
{
  broker.loop();  // Don't forget to add loop for every broker and clients

  client.loop();
}

---

The ESP Client sketch, directly derived from simple-client.ino example:

#include "TinyMqtt.h"    // https://github.com/hsaturn/TinyMqtt
#include "TinyStreaming.h" // https://github.com/hsaturn/TinyConsole

const char* BROKER = "192.168.1.33";  <--- THE IP OF THE ESP 8266 BROKER
const uint16_t BROKER_PORT = 1883;

const char *ssid     = "...";
const char *password = "...";

static float temp=19;
static MqttClient client;

void setup()
{
  Serial.begin(115200);
	delay(500);

	Serial << "Simple clients with wifi " << endl;

  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED)
	{ delay(500); Serial << '.'; }

  Serial << "Connected to " << ssid << "IP address: " << WiFi.localIP() << endl;

	client.connect(BROKER, BROKER_PORT);
}

void loop()
{
	client.loop();	// Don't forget to call loop() for each broker and client

	// delay(1000);		please avoid usage of delay (see below how this done using next_send and millis())
	static auto next_send = millis();
	
	if (millis() > next_send)
	{
		if (not client.connected())
		{
			Serial << millis() << ": Not connected to broker" << endl;
      next_send += 1000;
			return;
		}
		next_send += 1000+random(900*1000);
    Serial << "next send in " << (next_send/1000) << "sec." << endl;

		auto rnd=random(100);

		if (rnd > 66) temp += 0.1;
		else if (rnd < 33) temp -= 0.1;

		Serial << "--> Publishing a new sensor/temperature value: " << temp << endl;

		client.publish("sensor/temperature", String(temp));
	}
}

no way to set the root certificate

Where can we set the root certificate?

Break of communication between broker & MQTT Spy

Thank you for rare software. I have been testing the code with my project. It publishes mqtt messages fine until it receive a message from MQTT Spy. MQTT Spy shows broker is still connected after sending message and it is received fine at broker one time but there are no messages communicated after receiving that message. May be I am doing something wrong. I am using code linked below:

https://github.com/happytm/TinyMqtt/blob/main/examples/ProbeRequestGatewayESP32.ino

Thanks.

Platformio fixes

Check that examples under platformio compile and run.

The Arduino IDE compile options allows things that are not allowed under platformio. I have to refactor a little in order to make the compilation successful

Mosquitto dislikes the default empty client_id ("Client <unknown> disconnected due to protocol error.")

When trying TinyMqtt I got the following errors in my Mosquitto logs:

2022-12-14T14:12:45+0000 (1671027165): New connection from x.x.x.x:55797 on port 1883.
2022-12-14T14:12:46+0000 (1671027166): Sending CONNACK to x.x.x.x (0, 2)
2022-12-14T14:12:46+0000 (1671027166): Client <unknown> disconnected due to protocol error.

It turns out it was because of this:

CONNECT

ACK (with disconnect reason)

This was because I did not set MqttClient.id("something"); -- the example did not suggest so.

---

Possible fix:

--- TinyMqtt/src/TinyMqtt.h.orig	2022-12-14 15:48:05.244851788 +0100
+++ TinyMqtt/src/TinyMqtt.h	2022-12-14 15:48:34.916688801 +0100
@@ -35,6 +35,9 @@
 #include <string>
 #include "StringIndexer.h"
 
+// Default to a non-empty client identifier because some brokers reject those.
+#define TINY_MQTT_DEFAULT_CLIENT_ID "TinyMqtt"
+
 #if TINY_MQTT_DEBUG
 #include <TinyStreaming.h>
 #include <TinyConsole.h>    // https://github.com/hsaturn/TinyConsole
@@ -174,16 +177,16 @@ class MqttClient
   public:
     /** Constructor. Broker is the adress of a local broker if not null
         If you want to connect elsewhere, leave broker null and use connect() **/
-    MqttClient(MqttBroker* broker = nullptr, const std::string& id="");
+    MqttClient(MqttBroker* broker=nullptr, const std::string& id=TINY_MQTT_DEFAULT_CLIENT_ID);
     MqttClient(const std::string& id) : MqttClient(nullptr, id){}
 
     ~MqttClient();
 
     void connect(MqttBroker* local_broker);
-    void connect(std::string broker, uint16_t port, uint16_t keep_alive = 10);
+    void connect(std::string broker, uint16_t port, uint16_t keep_alive=10);
 
     // TODO it seems that connected returns true in tcp mode even if
-    // no negociation occured
+    // no negotiation occurred
     bool connected()
     {
       return (local_broker!=nullptr and client==nullptr) or (client and client->connected());
@@ -195,7 +198,7 @@ class MqttClient
     }
 
     const std::string& id() const { return clientId; }
-    void id(std::string& new_id) { clientId = new_id; }
+    void id(const std::string& new_id) { clientId = new_id; }
 
     /** Should be called in main loop() */
     void loop();

Request for example - receive (get payload) subscribed messages

Dear all,
in the examples, after subscribing to a topic, I didn't find how to receive messages and "use" (get) the payload of subscribed messages.
could you add an example of receiving broadcast/posted messages from another device.

Using mosquitto for debugging I can transmit, but when I transmit messages from mosquitto with ESP8266/TinyMQTT I cannot receive, i.e. print the message.

This is the code I used... and I think I either omitted something relevant or (probably) didn't understand how the code needed to "receive" works.

I realize that the PULL request section is not the right place to ask this question... consider it as a request to update the library user guide :-)

Thanks for your kind reply.

/********************************************/
#include <TinyMqtt.h>

std::string topic_power="energy/power";
std::string topic_timing="energy/timing";

char buffer[200];

MqttBroker broker(1883);
MqttClient mqtt_a(&broker);

void onPublishA(const MqttClient* /* source */, const Topic& topic, const char* payload, size_t /* length */)
{
   Serial << topic.c_str() << ", " << payload << endl;
}

  void setup()
{
   broker.begin();
   mqtt_a.setCallback(onPublishA);
   mqtt_a.subscribe(topic_timing);
    .....
    .....
}

void loop()
{
    .....
    .....
   broker.loop(); // Add loop for every broker and client
   mqtt_a.loop();

   static const int intervalA = 5000; // publishes every 5s (please avoid usage of delay())
   static uint32_t timerA = millis() + intervalA;

   if (millis() > timerA)
   {
      Serial.printf("mqtt_a connected:%d\n",mqtt_a.connected());
      Serial.printf("Broker connected:%d\n",broker.connected());
      timerA += intervalA;
   }
    .....
    .....
}

`mosquitto_pub` fails on 1.0.0 but works on <= 0.9.18

I just updated to the 1.0.0 tag in an attempt to resolve some memory leaks I believe are occurring from TinyMqtt. After updating I can no longer connect to TinyMqtt.

Output under 0.9.18:

$ mosquitto_pub -d -h 192.168.0.70 -p 1883 -t foobar -m '{}'
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending PUBLISH (d0, q0, r0, m1, 'foobar', ... (2 bytes))
Client null sending DISCONNECT

Output under 1.0.0:

$ mosquitto_pub -d -h 192.168.0.70 -p 1883 -t execute/measure_alk/manual/next_stepasdf -m '{}'
Client null sending CONNECT
Error: A network protocol error occurred when communicating with the broker.

I'll debug more, but posting in case others see connectivity issues in the meantime.

Tasmota issue - Error msg 0x10-ERROR - size(69), state=4

I was too quick with my question.

After I set user and password on the client the client is connected.
But removed immediately.

Does anyone have the TinyServer running with TASMOTA clients?
Thanks in advance for every tip.

broker.loop():client
153 New client
---> INCOMING 10 client(1073527448:) mem=279504
Incoming size(72), state=4
Incoming | 10 46 00 04 4D 51 54 54 | .F.. - MQTT |
Incoming | 04 EE 00 1E 00 0C 58 54 | .�.. - ..XT |
Incoming | 36 30 30 5F 47 41 52 41 | 600_ - GARA |
Incoming | 47 45 00 15 74 65 6C 65 | GE.. - tele |
Incoming | 2F 58 54 36 30 30 5F 47 | /XT6 - 00_G |
Incoming | 41 52 41 47 45 2F 4C 57 | ARAG - E/LW |
Incoming | 54 00 07 4F 66 66 6C 69 | T..O - ffli |
Incoming | 6E 65 00 05 67 75 65 73 | ne.. - gues |
Incoming | 74 00 05 67 75 65 73 74 | t..g - uest |

Connected client:XT600_GARAGE, keep alive=30.
815 sending 5 bytes
187 Client XT600_GARAGE Disconnected, parent=1073485520
64 close XT600_GARAGE
137 Remove 1
140 Client removed 0
.
.
broker.loop():client
153 New client
.
.
.
Michael (DE)

Converting to ETH (W5500) supported lib

Hi,

thank you very much for the library (as i believe yours is the only one that out of the box my windows clients managed to connect to and subscribe to topics as they should :) )

So back to topic :) is AsyncTCP not implemented yet? Because my clients cannot connect to ESP32 broker when adding:
#define TINY_MQTT_ASYNC
before including your lib

Also I have a question regarding ethernet implementation.
I'd like to use ESP32 with W5500 ethernet module with stock Ethernet lib v 2.0.1 ( Ethernet=https://github.com/arduino-libraries/Ethernet @ 2.0.1 => am using PIO on VScode) and some MQTT broker lib. And by checking your lib i guess you make only 2 calls to WiFiServer/WiFiClient (stock WiFi lib) so I guess it wouldnt be too much of a hassle to port it on wired connection.

retain is coming

Hello

retain is a wanted feature and is now implemented, not released yet.
I plan to store retained messages in a more compat way (using the String Indexer).

Will be released when these bugs are fixed:

• impossible to remove a retained message
• I think the retain flag is badly sent when the client is online, as far as I've understood, this flag should be set to 0 when msg is send alive, and 1 when this is a retained message.

Broken JSON when sending payload larger than 127 bytes

My PubSubClient send to mosquitto broker topic Light/relay/json_status with payload:
{"r1":1,"r2":1,"r3":1,"r4":1,"r5":1,"r6":1,"r7":1,"r8":0,"r9":1,"r10":0,"r11":0,"r12":0,"r13":0,"r14":0,"r15":0,"r16":0,"r17":0,"r18":0,"r19":0,"r20":0,"r21":0,"r22":0,"r23":0,"r24":0,"r25":0,"r26":0,"r27":0,"r28":0,"r29":0,"r30":1,"r31":0,"r32":1,"freeheap":272172,"uptime":600}
Mosquitto and another client sees same.

If my PubSubClient send this payload to TiniMQTT broker and another client (MQTT Explorer, MQTT Spy) sees other:
{"r1":1,"r2":1,"r3":1,"r4":1,"r5":1,"r6":1,"r7":1,"r8":0,"r9":1,"r10":0,"r11":0,"r12":0,"r13":0,"r14":0,"r15":0,"r16":0,"r17":0,"r18":0,"r19":0,"r20":0,"r21":0,"r22":0,"r23":0,"r24":0,"r25":0,"r26":0,"r27":0,"r28":0,"r29":0,"r30":1,"r31":0,"r32":1,"freeheap":272172,"uptime":600}�

What is the ending symbol?