I've recently dived into MQTT. I got it working without really understanding some core concepts (yay easy-to-use UIs) but felt myself wanting to know more about what it was. I did some reading up on the various parts and decided to document them here for my own reference later on. Maybe it'll be useful to others too, who knows!
That said, I do quite like the Urban Dictionary definition of blogumentation too:
When something that should have documentation in one document is instead documented over blog posts that are incomplete or hard to find and often spread over different blogs.
So, I figured while I'm learning about MQTT, I'd document some findings in here so I have a chance of finding it again sometime. Because of that, I don't claim to be an expert in this, I'm still learning!
Why do I need MQTT?
I used to have some TP Link HS110 plugs for power monitoring. TP Link shut off access to the local API for that in a firmware update (I thought I'd locked those devices off from accessing the Internet, but for some reason the data was no longer available in Home Assistant).
So, I ordered some new smart plugs from Local Bytes that were pre-flashed with Tasmota firmware. (Seriously, £9.50 for a pre-flashed smart plug? Absolute bargain!). These communicate with Home Assistant via MQTT!
I set up an MQTT broker using the Mosquitto Home Assistant addon and configured an authenticated user for it. I added the Tasmota integration in Home Assistant, connected to the plug's hotspot, configured WiFi access, and then configured MQTT. I watched in amazement as it appeared in Home Assistant like magic!
But... what just happened? What's an MQTT broker? What is a 'topic' I saw mentioned in the MQTT configuration of the plug?
If MQTT is for messages, then my thoughts were: the plugs need to send messages to report their status, and respond to messages to instruct them to turn on and off.
I've used other pub-sub systems in the past. I had some experience with the terms used in a lot of places, but what does a topic look like? What does a message look like?
I didn't want to go super low level into the protocol, but wanted to know what all the various bits meant.
Essentially, the MQTT broker is the server that receives messages and routes them to the appropriate clients. For me this is Mosquitto run by the supervisor in Home Assistant (which runs a Docker container under the hood).
Publishers & Subscribers
A publisher is a client that sends messages into the broker.
A subscriber is a client that's connect to the broker to receive messages.
Sounds like other systems I've used!
A topic is a UTF-8 string, that consists of one or more topic levels. It's essentially the 'subject' of a message. A topic doesn't have to be created before it can be used.
Topic levels are separated by a slash
In the above, the topic levels are
Topics are case-sensitive. The following topics are different:
It seems a good practice is to have each device report to a different topic rather than multiple devices to the same topic.
A single slash is also a valid topic
/, and starting a topic with a slash would create a root-level topic that's empty (and that feels weird). So:
Message format seems to be up to you to use what works best for your use case / application.
I wanted to see what sort of messages were being sent from Home Assistant and the plug to the broker.
I used MQTT Explorer to connect to my broker and inspect messages to see what they're like.
The plug seems to report status data in JSON, and commands from Home Assistant to the broker to turn a plug on were just simple
With the integration handling things for me, I don't need to worry about formatting messages myself for this current use case, but could be interesting in future if I wanted to send other data into MQTT and pick it up in Home Assistant!
I saw this mentioned somewhere and wondered what it's for. For IoT, it makes a lot of sense to me.
A retained message is a normal message, but has the
retained flag set to
true. Each topic stores only the latest retained message received. When a client subscribes to that topic it immediately receives that retained message.
For IoT devices, this means that a topic for a particular device can send the status of that device as a retained message. If a new client subscribes to that topic it immediately receives the latest status of the device from that retained message, rather than having to wait until it sends another message which might not be for some time.
My MQTT broker isn't exposed to the internet, it's internal only. It has users setup with passwords so that not just anyone can connect to it.
I found a post on security in MQTT and the issues it faces, and found it very helpful.
The TLDR is: Like most things, MQTT isn't insecure itself, but there are a lot of badly configured servers that have no password protection:
around 49,000 exposed MQTT servers that were accessible from the Internet. The really bad news? Over 32,000 of those servers had no password protection.
This sort of thing happens a lot, default (or no) credentials configured for services exposed to the Internet. If there was a search engine for the Internet of everything that would be scary as they'd be easier to find.