JMRI Hardware Support - MQTT
Author: Bob Jacobsen
Keywords: JMRI MQTT

# Hardware Support: MQTT

MQTT is a "client-server publish-subscribe" protocol, meaning that JMRI connects to an MQTT server (called a "message broker") and subscribes to receive messages on certain "topics." JMRI also publishes messages to the MQTT message broker and any device (on or off the layout) subscribed to the published topic will receive a copy. This approach allows JMRI users to use MQTT communication in developing any number of layout control and automation scenarios. For example, you can use JMRI's MQTT capability to communicate with IOT devices as well as arduinos and other microcontrollers.

MQTT messages consist of two parts: topic and contents. JMRI provides three mechanisms for creating and processing MQTT messages, the specifics of each are provided in sections below:

- MQTT-connected objects: Certain JMRI objects can be defined as being connected via MQTT. As of release 4.26, JMRI implements MQTT-connected turnouts, sensors, lights, and signal masts. Reporters were added in release 5.3.2. Power and throttle control functions were added in release 5.5.4. Other object types and functions may be added in the future based on user requests.
- LogxiNG: As of release 4.99.7, LogixNG has the capability to publish and subscribe MQTT messages on any topic of your choosing.
- Scripting: Advanced Usage: For other objects and devices, you can use a provided script or write your own using JMRI's MQTT Adapter object and methods.

Help Page Contents:

## Getting Started with MQTT

The first step in using MQTT communication with JMRI is to create a connection to an MQTT message broker. This is done via the Connections pane in the Preferences window. (Edit⇒ Preferences⇒Connections)

- Select "MQTT" as the System manufacturer
- Select "MQTT Connection" as the System Connection. [While there is no "MQTT Simulator" as there are for other connection types in JMRI, this may be added in the future to allow testing without direct attachment to an MQTT message broker; see Testing for simple ways to connect to a message broker.]
- The IP Address/Host Name should be either the IP address or host name of an external server, or "localhost" if you are running an MQTT message broker on the same machine as JMRI.
  Since 5.5.7JMRI will use the Preferences Railroad Name when connecting to the broker (Edit⇒Preferences⇒Railroad Name). This is known as the "MQTT Client ID" and will make it easier to identify the JMRI connection in the MQTT broker logs.
- The default Connection Prefix for MQTT is "M". [Advanced Usage: this can be changed when setting up the connection.]
- The default Connection Name is "MQTT." Currently, JMRI can handle only one MQTT connection at a time. [Advanced Usage: The ability to change the connection name will be required at some point in the future when more than one MQTT Connection is allowed.]

You can establish an MQTT Connection in JMRI in addition to having other connection types.

[Go to top of page]

### Additional MQTT Connection Preferences

Many parameters related to MQTT communication can be seen and changed by selecting "Additional Connection Settings" on the MQTT Connection page under Preferences, as shown in the picture below.

- TCP/UDP Port: The MQTT Message Broker listens for traffic on a particular port. The default is 1883. Advanced Usage: This can be changed to suit the message broker instance you are using.
- Since 4.23.6MQTT User Name and Password: MQTT Message Brokers can be initialized to require a user name and password. You can specify those here. See also below under Testing for further information about using a broker without a user name and password.
- MQTT Channel: MQTT Channel is prefixed to the topic of any MQTT messages published or subscribed by JMRI. Prior to JMRI release 5.1.2, MQTT Channel contained "/trains/" as a default.
  Since 5.1.2 MQTT Channel now deaults to blank but you can change it to anything you want.
  Note that current guidance from the MQTT community is not to have a leading slash in the MQTT Channel. A trailing slash should be provided for non-blank strings for convenience in organizing topics.
  [Prior to JMRI Rel. 5.1.2, since the default of "/trains/" was for all users of JMRI, anyone keeping the default and using a public MQTT message broker (such as htttp://test.mosquitto.org, see below) will show their messages (and see messages from) every other JMRI user anywhere in the world who is doing the same thing!]
- MQTT-connected object topics: MQTT-connected objects such as sensors, lights, turnouts, and signal masts use a unique three-part topic (see below). The fields listed here comprise the middle part (MQTT Channel is the first part and the hardware name portion of the object's system name is generally the third part).
  Advanced Usage: While there are fields for specifying separate topics for publishing (the "send" field) and subscribing (receiving back from the layout) for some objects, it is recommended that send and receive topics be set the same except for advanced usage (typically involving custom programmed microcontrollers used on the layout).
  Advanced Usage: If you want the object's system name suffix to be in the topic string at some place other than at the end, you can use a placeholder of "{0}" within the string to put the object's system name suffix at that place. This allows you to set up topics like "track/turnout/{0}/state" so JMRI will generate "/trains/track/turnout/abc123/state" for turnout "MTabc123".
  Advanced Usage: You can also change these fields as part of JMRI start-up using a script as described below.
- Since 4.25.3Last Will: JMRI can specify its "last will" topic and message when it connects to a broker. The broker stores the message and topic and sends it as a normal MQTT message to clients who have subscribed to that topic if and when it finds that JMRI has disconnected in an abnormal manner. Find more information here about Last Will and Testament.

Two other parameters that control the way the MQTT Message Broker interacts with JMRI can be changed via script. See Changing Default MQTT Parameters: QoS and Retained Messages.

Be sure to save the JMRI Preferences if you have changed anything (hit the "Save" button on the Preferences pane) and restart JMRI to put the changes into effect.

[Go to top of page]

## MQTT Topics

JMRI publishes and receives MQTT messages on specific "topics." An MQTT topic is the label or headline for published messages. It is the label that other processes subscribe to to receive messages. MQTT treats a topic as a single string, although topics are typically organized with slashes to create multiple levels. JMRI, however, forms an MQTT topic by concatenating two or three strings, each with or without slashes as desired, as is described under the sections for each type of use (MQTT-connected objects, LogixNG, scripts). MQTT Channel (see above in Preferences) is always the first (prefix) part of the topic sent to the MQTT Broker (but may be blank, as of JMRI relesae 5.1.2).

Topics used in Subscribe messages can optionally have a wild card character. The plus sign ("+") mateches any string only in the level (between slashes) where it appears. The hash sign ("#") must appear at the end of a topic string and will match to any string and any number of levels. See this mosquitto.org page for more information.

Since 5.1.2Setting the MQTT Channel to blank means that the user has complete control over topic strings both for publish and subscribe. This provides great flexibility, especially in communicating with devices that do not have much flexibility to set their topic strings, such as some home automation devices.

Any message topic, defined or custom, can be subscribed to and processed by a LogixNG or a custom script. These messages can further be used to trigger Logix, populate Memories and then display them on panels, or for whatever purposes you find useful.

## MQTT Message Contents

For MQTT-connected objects, MQTT message contents to be published are prescribed and automatically created (see following). For both LogixNG and scripts, MQTT message contents can be any arbitrary string you create. How that message contents is interpreted is up to the processes that subscribe to the topic of the message. See the following sections for more details.

## MQTT Parameters for Quality of Service and Retained Messages

MQTT Quality of Service indicates to the MQTT message broker how to handle message delivery and receipt. Allowable values are 0, 1, and 2. JMRI publishes messages using QOS level 2, the most stringent level, to ensure that messages are received by the broker (this can be changed by script). QoS is between the publisher and the broker. While JMRI sets a QoS level 2, whoever is publishing messages you want JMRI to receive (via subscription) needs to set its own level (which can be the same or different than the level JMRI sets).

The Retained Messages parameter indicates to the MQTT message broker whether to retain the current message to be automatically published whenever a new subscriber to that topic appears. JMRI publishes messages with the retain option on (this can be changed by script). Since JMRI subscribes to the receive topics for the individual objects you have set up in JMRI, subscriptions are made when you load a panel. Any retained messages will be received at that time. For example, JMRI subscribes to messages from MQTT sensors (that is, devices managing sensors and publishing MQTT messages). The publisher of those messages decides whether values are retained by the broker so JMRI will receive them when a panel is loaded or a new sensor created. Turnouts and lights commands are generally published by JMRI so JMRI will not receive anything on startup as it would not be subscribing to its own messages.

[Go to top of page]

## Using MQTT-connected Layout Objects

### Available Objects

JMRI's MQTT capabilities continue to evolve. Older versions of JMRI (4.12 and earlier) directly support Turnouts only. Other objects have been added in later releases as indicated below. Direct support for Signal Heads is not yet available. Any JMRI object can be updated via an MQTT messages using scripts.

Message "contents" for JMRI standard objects are typically single words that represent the state of the object. Specifics are discussed following. Advanced Usage: It is also possible to replace the standard message contents by using a custom version of the message parser (including using JSON messages).

- Turnouts: JMRI sends on the "Turnout send topic" if a user commands a turnout change e.g. by clicking on a panel or table. Additionally, if a Turnout has been created in either the Direct mode or Monitoring mode (see turnout documentation), JMRI expects to receive turnout state on the "Turnout receive topic." The message content to be published by turnouts is "CLOSED" and "THROWN". Turnouts in Direct or Monitoring mode can also receive these as well as the messages "INCONSISTENT" and "UNKNOWN" (sets the Known State). Receiving any other message will generate a Warning in the JMRI System Console and will set the turnout Known state to "UNKNOWN."
- LightsSince JMRI 4.21.1: JMRI sends on the "Light send topic" if a user commands a Light change e.g. by clicking on a panel or table. JMRI also acts on Light state messages sent from the layout on the "Light receive topic." The expected message content for lights is "OFF", "ON" and "INTENSITY dddd" where ddd is a number between 0.00 and 1.00. Receiving any other messages will result in a Warning in the JMRI System Console. and cause the Light state to be set to "UNKNOWN."
- SensorsSince JMRI 4.21.1: JMRI expects to receive sensor state on the "Sensor receive topic" (as specified on the Preferences page). JMRI sends state on the "Sensor send topic" if a user changes sensor state e.g. by clicking on a panel or table. The expected message content for sensors is "ACTIVE" and "INACTIVE". Any other message will generate a Warning in the JMRI System Console but will not change the Sensor state.
- ReportersSince JMRI 5.3.2: MQTT-connected Reporters expect to receive a message like "nnnn text", where nnnn is the locomotive ID and the rest of the message is free form text. The Reporter creates/updates an IdTag with the locomotive ID as the tag ID (e.g. ID1234) and puts the rest of the message in the "content" parameter. Reporters do not publish any MQTT messages. A Reporter can also receive an empty message which clears the current report but doesn't update any IdTags.
  Advanced Usage: Scripts can read from a JMRI reporter and send the content asa MQTT message as needed.
- Signal Heads: MQTT-connected Signal Heads are not yet implemented. You can use e.g. the Three Turnout Signal Head style with MQTT turnouts as a work around. Advanced Usage: You can also write a script that sends the content of a virtual or other Signal Head to the layout.
- Signal MastsSince JMRI 4.26: You can define MQTT-connected Signal Masts which will send their aspect when changed. These do not receive anything from the layout. A JMRI MQTT Signal Mast will send the aspect name, followed by the held status and the lit status. Examples are "Approach Medium; Lit; Unheld" and "Clear; Unlit; Held". The available aspect names are defined in the signal system definition being used.
- Throttles and Power ControlSince JMRI 5.5.4: While not "layout objects," JMRI can also generate and receive MQTT messages for these functions, as described below.

All other messages on these defined topics are ignored by the standard JMRI processing.

### MQTT-connected Object Names

As with all objects within JMRI, MQTT-connected layout objects (turnouts, sensors, lights, and signal masts) have a "system name" and may optionally have a "user name."

If you create objects by "adding" them to their respective tables, first select "MQTT" from the pulldown. Enter a string in the "Hardware address" field. As opposed to certain other JMRI connection types, this string does not have to follow any particular model and can be any alphanumeric characters. For example, MQTT-connected Turnouts can be specified with JMRI hardware addresses such as "abcd" or "123" or ".X$42". JMRI will prefix that string with the MQTT Connection Prefix and Object Type, creating System Names in this example of "MTabcd", "MT123", or "MT.X$42".

Advanced Usage: System Names may also include one or more "/" (slash) characters, causing MQTT to see this as defining another level in the full topic. For example, if sensor "abc" is attached to microcontroller "Ard01", you could name it "Ard01/abc". This allows easy identification of which hardware controls which sensors, turnouts, or other devices.

### Topics for MQTT-connected objects

For MQTT-connected objects, JMRI creates a topic string composed of three parts for communicating with those objects:

1. MQTT Channel (may be blank as of JMRI Release 5.1.2) is prefixed to the...
2. MQTT topic for that object type as defined in the Connection Preferences (Advanced Usage: may be different for publishing and subscribing),
3. followed by the hardware name portion of the object's system name (system name stripped of the connection and object type characters).

An example of MQTT-connected objects: Using the example entries in the JMRI Preferences Connection pane shown in the picture below, JMRI would construct the MQTT topic to publish the status of a Turnout with system name "MTabcd1" by concatenating:

1. the MQTT channel: "trains/"
2. the object-type-specific "send" topic: "jSend/to/"
3. the object's system name suffix: "abcd1"

yielding the complete MQTT message topic: "trains/jSend/to/abcd1".

Advanced Usage: If the object's system name include a "/" (perhaps to separate a hardware designator), say "MTArd01/abcd1", the topic would include that slash and become "trains/jSend/to/Ard01/abcd1"

Advanced Usage: If you want the object's system name suffix to be placed in the topic string other than at the end, you can use a placeholder of "{0}" within the preference string to put the object's system name suffix at that place (see Preference pane diagram above). This allows you to set up topics like "jSend/to/{0}/state" so JMRI will generate "trains/jSend/to/abcd1/state" for turnout "MTabcd1" or "trains/jSend/to/Ard01/abcd1/state" for turnout "MTArd01/abcd1".

[Go to top of page]

## Power and Throttle Control

It is now possible to subscribe to and receive MQTT messages to turn track power on or off and to control engine throttles.

### MQTT Power Control

Since JMRI 5.5.4 JMRI publishes Track Power ON/OFF commands on the "Power send topic" and receives the track power status on the "Power receive topic". These messages will always be sent if you set power via the JMRI Power Control Window.

If you select an MQTT connection as the default for Power Control, MQTT messages will be published when selecting Power ON or OFF from the Power Control Window, the WiThrottle Server window, or any throttle connected via the WiThrottle Server, or any scripts that use "powermanager.setPower(jmri.PpwerManager)". [If multiple command stations types are connecte to JMRI, it is always possible to turn their track power on or off via the Power Control Window.]

### MQTT Throttle Control

Since 5.5.4 JMRI supports controlling locomotives over MQTT. Throttles attached to the specified MQTT connection (either by default or by using Tools⇒Throttles⇒Throttles for connections⇒[select from pulldown]) will publish MQTT messages on the indicated topics. The protocol is controller agnostic, however the internal design of JMRI imposes some compliance with DCC standards.

Throttles will generate and process MQTT messages as follows:

- Throttle: The speed setting as a 0-100 percentage value is sent on the "Throttle send topic" and received on the "Throttle receive topic".
- Direction: The throttle direction is sent on the "Direction send topic" and received on the "Direction receive topic". Direction values are "FORWARD" and "REVERSE", and "STOP" for an emergency stop.
- Function: Function states are sent on the "Function send topic" and received on the "Function receive topic". The placeholder "{1}" within each topic is substituted with the number of the function.Function state values are "ON" and "OFF".
- Consist: MQTT supports Command Station Consists in JMRI using the standard Consist Tool. All of the loco addresses that make up a consist are listed in the "Consist send topic" when a throttle is controlling that consist address. It is up to the controller to determine how it controls the individual locos in the consist.

In the topics listed above, the placeholder "{0}" in the topic name is replaced with the loco or consist address that the throttle is controlling. When the throttle is Released, all of the above topics are removed from MQTT. If a throttle is Dispatched, the topics are retained on MQTT - this allows for locos to be controlled from a script but be able to be taken over and manually controlled as required (ie for shunting operations).

If you select an MQTT connection as the default for Throttle Control (by selecting the "Throttle" button on Edit⇒ Preferences⇒Defaults), any Throttle opened with Tools⇒Throttle⇒New Throttle from the main Panel window or using smartphone applications such as WiThrottle and EngineDriver will generate MQTT messages. You can always select a throttle for any installed connection using Tools⇒Throttles⇒Throttles for connections⇒[select from pulldown].

[Go to top of page]

## Using MQTT via LogixNG

Since 4.99.7 LogixNG provides a significant upgrade to creating Logix procedures to perform a wide variety of functions with JMRI. One of its features is to publish and subscribe to arbitrary MQTT topics and then perform any of the other functions allowed in a LogixNG Conditional.

See Chapter 5 of the LogixNG Reference for information on entering Publish and Subscribe expressions into LogixNG Conditionals.

### LogixNG MQTT Topics

In a LogixNG conditional, any string defined by you as the "topic" is concatenated to the MQTT Channel to form the MQTT topic that will be published or subscribed. If you specify "mytrains/someobject" as the topic, JMRI will publish and subscribe on "[MQTT Channel]mytrains/someobject" using the MQTT Channel specified in the MQTT Connection Preferences. Slashes can be inserted for convenience. JMRI does not automatically insert any slashes.

### LogixNG MQTT Publish

An example of a simple LogixNG conditional that will publish an MQTT message whenever a sensor (Trigger2) is changed is shown in the picture to the right.. Using this LogixNG you can publish the sensor state change using any text message of your choosing (in this example, the contents of JMRI memories MemoryData and MemoryData2), for example, to a display or a sub-panel controlled via MQTT messages. Furthermore, you could do that with any type of sensor, including internal and those that are connected to the layout via other communication methods. [If the sensor Trigger2 is an MQTT-connected sensor, JMRI will have already subscribed to state change messages from the layout. In that case, this LogixNG would additionally publish whatever message you choose on the topic you specify.]

### LogixNG MQTT Subscribe

In the picture below, you can set up LogixNG conditionals to subscribe to any messages coming from any devices on the layout (in this case with topic "test/" followed by anything else; "#" is the wild card character). Note that for Subscribe, LogixNG requires you to specify the topic directly in the conditional expression (it cannot be, for example, in a memory as in the case of Publish). JMRI will put the topic and contents of the message into LogixNG local variables (Topic and Contents) and you can then use them or, as in this example, move them into JMRI memories (TestDataRcvd and TextTopicRcvd) for further use.

[Go to top of page]

## Using MQTT via Scripting

The JMRI MQTT capability is very general and scripts can be used to send and receive arbitrary messages from any MQTT-aware device on or off the layout. In addition, scripts can be used to change standard message processing and default parameters.

In a script, any string defined by you as the "topic" in a reference to an MQTT Adapter object is concatenated to the MQTT Channel to form the MQTT topic that will be published or subscribed. If you define "mytrains/someobject" as the topic, JMRI will publish and subscribe on "[MQTT Channel]mytrains/someobject" using the MQTT Channel specified in the MQTT Connection Preferences. Slashes can be inserted for convenience. JMRI does not automatically insert any slashes.

All the scripts discussed here can be found in the JMRI distribution jython directory under "MQTT." More information about scripting with JMRI is provided under Scripting in the JMRI Help and website. Custom scripts for MQTT will typically use the MQTT Adapter object and methods.

#### Publishing Messages

The SendMqttMessage.py is an example of how to send any message on any topic (automatically prefixed with the JMRI MQTT channel) for processing by other devices.

#### Example: Publishing Object Status (Signal Head Appearance)

Since JMRI 4.99.9

This script provides an example of using JMRI's MQTT "publish" method to publish signal head appearance changes

#### Subscribing to (Receiving) Messages

You can use the ReceiveMqttMessage.py script to subscribe to any topic on the JMRI MQTT channel (both for JMRI objects and custom) to receive message for further processing. Standard MQTT wildcards (# and +) can be used (as of release 4.21.3).

#### Changing the Default Message Parsing

Since JMRI 4.15.5

Advanced Usage: You can use a script to install a new MqttContentParser object to code and decode the content of messages. You might want to use JSON format, or need to adapt to specific messages from some already-existing device; you can create a custom parser to do any of that. See the SetMqttParser.py sample script. For a Java example, see the inner class implementation in MqttTurnout.

Note that you can call `setParser(...)` on the jmri.jmrix.mqtt.MqttTurnoutManager or on an individual jmri.jmrix.mqtt.MqttTurnout object. If you call it on the MqttTurnoutManager, all later created MqttTurnout objects will use the new parser; earlier-created ones will not be changed. This means you should execute a script to change the parser before loading any panel files if you want all MqttTurnouts to be modified.

#### Changing the MQTT Message Topic at JMRI Startup

Since JMRI 4.15.5

The sample script SetMqttPrefix.py shows how to set MQTT-connected object type specific topic components at JMRI startup. Note that any changes should be made at startup time before creating any such objects, i.e. the script must be run before any panel files are loaded. Changing the prefix only affects objects that are created after the change.

#### Changing MQTT QoS and Retain

Since JMRI 4.21.3

MQTT parameters Quality of Service and Retained Message used by JMRI can be reset via scripting. Example script SetMqttOptions.py shows how both of these parameters can be changed .

[Go to top of page]

## Testing

If you want to test using JMRI with the MQTT connection type, you can use a publicly available (non-JMRI) test server or install and run a copy of an MQTT Message Broker locally.

A popular message broker is provided by the mosquito.org foundation. Enter "test.mosquitto.org" as the Host Name on the Preferences Connections page to connect to this message broker (for more information, type test.mosquitto.org into a browser).

You can also install the mosquitto broker and tools on your own machine. After downloading the tools, enter one of the following command lines from the directory where they now reside:

        mosquitto -v

If you are using version 2.0 of mosquitto or later, you will need to provide a "configuration file":

        mosquitto -c "[location of your config file]/mosquitto.conf"

Include at least the following lines in mosquitto.conf:

        log_type all                 #Equivalent to setting -v (verbose mode)
        listener 1883                #To ensure listening on the appropriate port
        allow_anonymous true         #Allows JMRI to subscribe without an ID or password

Now enter "localhost" as the Host Name on the MQTT JMRI Connection Preferences page. You are now connected!

To subscribe and print all JMRI messages as they are published to JMRI's MQTT Channel "trains/", enter this from the directory where you have the mosquitto tools:

        mosquitto_sub -h test.mosquitto.org -v -t 'trains/#'

To remove all retained messages, add the parameter "--remove-retained".

To get a nicely formatted output, add a format string like this following the "-F" parameter:

        mosquitto_sub" -i LocalMonitor -t "trains/# -F "%I  %-38.38t %q %r %l  %-25.25p"

To publish messages for JMRI to subscribe to:

        mosquitto_pub -h test.mosquitto.org -t trains/track/turnout/123 -r -m "CLOSED"
        mosquitto_pub -h test.mosquitto.org -t trains/track/turnout/123 -r -m "THROWN"

These commands can be run on the same machine as JMRI, or from a separate machine.

See the mosquitto documentation for more information.

There are also MQTT tools available for iOS devices and Android devices that can be useful for monitoring, testing and operating your layout.

[Go to top of page]