General Concept

The ThingSet protocol provides a consistent, standardized way to configure, monitor and control ressource-constrained devices via different communication interfaces. It specifies the higher layers (5 to 7) of the OSI (Open Systems Interconnection) modelopen in new window. The payload data is independent of the underlying lower layer protocol or interface, which can be CAN, USB, LoRa, WiFi, Bluetooth, UART (serial) or similar.

The underlying layers have to ensure reliable transfer, correct packet order (if messages are packetized) and error-checking of the transferred data.

Message Types

ThingSet defines three types of messages: Requests, responses and publication messages.

The communication between two specific devices uses a request/response messaging pattern via so-called channels. A communication channel can be established either directly (e.g. serial interface, USB, Bluetooth) or via a network or bus with several devices attached (e.g. CAN, Ethernet, WiFi, LoRa). In case of a network, each device/node has to be uniquely addressable.

The device acts as the server and responds to the requests by a client. The client might be a laptop or mobile phone with a human interface application.

The data transfer is always synchronous: The client sends a request, waits for the response (status code and/or requested data), handles the data of the response and possibly starts with additional requests.

Monitoring data is not intended for only a single device, but could be interesting for several devices (e.g. data logger, display, human interface device, etc.). Thus, the monitoring data is exchanged via a publish/subscribe messaging pattern.

Publication messages are directly broadcast through the network. There is no intermediate broker (like in MQTT) to store the messages and published messages are not confirmed by recipients, so there is no guarantee if the message was received.

Protocol Modes

Similar to Modbus, the ThingSet protocol supports two different modes: A human-readable text-based mode and a binary mode.

In the text-based mode, payload data is encoded in JSON format (RFC 8259open in new window). This mode is recommended when using USB or serial interfaces as the low layer protocol, as it can be easily used directly on a terminal.

The binary protocol uses the CBOR binary encoding (RFC 7049open in new window) instead of JSON payload data in order to reduce the protocol overhead for ressource-constrained devices or low bandwith communication protocols like CAN and LoRa.

Each device must implement the binary encoding of the protocol. The the text-based JSON variant is optional, but recommended.

Data Objects

All accessible data of a device is structured in so-called data objects. A data object might be any kind of measurements (e.g. temperature), device configuration (e.g. setpoint of a controller) or similar data.

Each data object is identified by a unique Data Object ID. The ID can be chosen by the firmware developer. In addition to that, each data object has a unique name. The name is a short ASCII string without blanks, e.g. "vBat" for the battery voltage.

For reduced memory useage, lower camel-case styleopen in new window should be used for the data object names. The first letter(s) should specify the type of value if applicable:

• v for voltage
• e for energy
• i for current
• t for time
• temp for temperature

The numeric IDs are used in the binary protocol for increased efficiency and performance. For all interactions with users and in the text-based mode, only the object name is used.

Data object categories

Each data object belongs to one of the following categories (associated to a category ID of 4 bits):

Category name (JSON)	Category ID	Description	Access
*	0x0	Wildcard ID, representing all IDs
info	0x1	Read-only device information (e.g. manufacturer, software version)
setup	0x2	User-configurable settings	read/write access, may be protected with user password
input	0x3	Input channels (e.g. actuators)	write access
output	0x4	Output channels (e.g. sensor data)	read access
rpc	0x5	remote procedure call	partly access restricted
calibration	0x6	Factory-calibrated settings	read/write access, protected
diagnosis	0x7	Error memory, etc.,	read access, at least partly protected
	0x8-0xE	Reserved for future use	unknown
	0xF	Reserved (termination character in binary protocol)	n/a

Data object IDs are stored as 16-bit unsigned integers. The ID includes the category ID of 4 bits. The remaining 12 bits can be freely chosen. In total, 4095 different values can be associated via an ID per category, with 0 being the wildcard ID again. The wildcard category and data object IDs are used to query the accessible data of a device (see below).

The follwing table describes the bits inside the 2-byte unique data object ID:

Byte 1		Byte 2
b15 ... b12	b11 ... b8	b7 ... b0
Category ID	Object Number
Data Object ID

Examples

For explanation of the protocol functions, the following exemplary device data object structure will be used:

{
    "info" : {
        "manufacturer" : "Test Company Inc."
    },
    "input" : {
        "enableSwitch" : true
    },
    "output" : {
        "vBat"     : 14.2,
        "tAmbient" : 22
    }
}

The above data structure contains 4 data objects in total, grouped into 3 different categories (info, output and input). The device will have an internal map to associate the object name with a unique ID. In C code this might look like the following:

struct data_object_t {
    char* name;
    uint16_t id;
};

const data_object_t data_objects[] = {
    {"manufacturer", 0x1001}, // ID = (0x1 << 12) + 1 = 0x1001
    {"enableSwitch", 0x3001}, // ID = (0x3 << 12) + 1 = 0x3001
    {"vBat",         0x4001}, // ID = (0x4 << 12) + 1 = 0x4001
    {"tAmbient",     0x4002}  // ID = (0x4 << 12) + 2 = 0x4002
};

In addition to that, the programmer must link the name or ID to the actual variable containing the data.

Units

All data communicated with the outside world must use SI units, so it is not necessary to specify the unit of each data object. The data types (see below) are used for scaling of the data.

If the basic SI unit for a given measurement value is not common or not feasible (e.g. use of kWh for energy instead of Ws), the unit must be explicitly defined with an underscore in the name of the data object, e.g. "eBattery_kWh" for the battery energy content in kWh. Units which cannot be derived from the SI basic units (e.g. °F) are not allowed.

ToDo: What about nautical miles and knots...? makes a lot of sense to allow for maritime applications. But it's not at all compatible with SI system...

For temperatures, Kelvin (K) is the official SI unit. However, °C is compatible with K and is allowed, if specified in the data object name.

It is not allowed to publish a voltage in millivolts (mV) instead of volts (V). Instead, the decimal fraction data type of CBOR can be used in the binary protocol, if internal calculation is done in fixed point math.

Functions

Each request fulfills a specified function, e.g. a command to read data from the device. The function is associated to a unique function ID, which defines the layout of the payload and the actions to be performed.

The different functions are encoded using 1 byte, i.e. a number between 0 and 255.

Function IDs 10, 13 and 32-127 are reserved, as they represent the ASCII characters for readable text including CR and LF. Invalid function IDs are ignored by the ThingSet parser, so that other text output (e.g. status information) can be used in parallel to the ThingSet protocol on the same serial interface.

The ASCII characters '!', '#' and ':' (function IDs 33, 35 and 58) are used as identifiers for the text mode protocol (see below). In this way, text-based or binary mode can be automatically detected based on the first byte.

The ID of a response includes a status code which shows if the request could be handled successfully. The ID is calculated as 0x80 + status code. For status codes between 0 and 29, the response was successful. If the status code is greater than or equal to 30, an error occured. The remaining bits are used to specify the error in more detail.

Overview of currently defined request functions

Function ID	Function name	Description
0x01	read	Read data object(s)
0x02	write	Write data object(s)
0x03	list	List all data objects of one category (device-discovery)
0x04	name	Get the name of a function by ID
0x05	pub	Request publication of data object(s)
0x06	auth	Authentication for access to access-restricted data objects

Possible future request functions (not yet specified)

Function ID	Function name	Description
?	ping	Ping a device
?	sync	Time synchronization
?	file	File access
?	reset	Reset the device
?	dfu	Device firmware upgrade.

Text-based protocol

Function ID	ASCII character	Description
0x21 (33)	!	Request
0x3a (58)	:	Response
0x23 (35)	#	Publication message

Response functions (status codes)

Function ID	Status Code	Description
0x80 (128)	0x00 (0)	Success.
0x81 (129)	0x01 (1)	Partial Success. (e.g. not all data values could be changed)
158	30	General Error.
159	31	Unknown/unsupported function.
160	32	Unknown data object.
161	33	Wrong format.
162	34	Wrong data type.
163	35	Device busy.
164	36	Access denied.
165	37	Request too long.
166	38	Response too long.