March 2016 V 0.8 Doug Johnson (finson@whidbey.com)
The document device-driver.md describes the overall architecture of the proposed Arduino Device Driver architecture.
This Appendix C document defines the detailed message formats used to communicate between the client and Firmata and back again.
The Device Driver architecture is intended to be relatively independent of any communications protocol used to control it. However, the initial design was created in order to alleviate some of the pressure on the Firmata protocol to provide individual messages and formats for every new piece of hardware, and so the ability to map Firmata capabilities to device driver functions and vice versa is important, for proof of concept if nothing else.
The translation from Firmata query message to device driver method call and from method result to Firmata response message is handled by DeviceFirmata, a FirmataFeature that can be loaded as part of Configurable Firmata. This module is essentially the Arduino end of a remote procedure call implementation, where the serialization protocol is Firmata, and the other end of the RPC protocol is implemented by a client library.
Two Sysex sub-commands are used by this feature: DEVICE_QUERY and DEVICE_RESPONSE.
There is a small set of action codes that specify what the driver is to do after it receives the message.
The first action is always OPEN. The caller supplies a logical unit name that can be recognized by a device driver, and upon success, a handle is returned for use in future calls. After the handle has been received, the caller can read status and data (READ) and write control and data (WRITE). Once the caller has completed its operations on a device, it can use CLOSE to make the logical unit available for another client.
The detailed message formats for each action are provided below.
##Firmata Message Formats
The arguments provided by the caller of an API method are organized into a DEVICE_QUERY message on the client side by the proxy device driver, encoded in base-64, then transmitted to the server. Firmata dispatches the Sysex message to the DeviceFirmata module, which decodes the body of the message from base-64 and dispatches the API call to the proper device driver via the DeviceTable object. After processing by the device driver, DeviceFirmata captures the results and formats them as a DEVICE_RESPONSE message, then encodes and transmits the message back to the client host where the proxy device driver decodes the message and returns the result to the original caller.
All data values visible to the device drivers are encoded to base-64 immediately before transmission by Firmata and decoded to their true values immediately after receipt. Thus, as far at the device drivers and the clients are concerned, all values are full-width values with no constraints on their magnitude or use of the high order sign bit due to transmission protocols.
The use of Firmata does have some secondary impacts on the device driver parameters. The Device Action Codes and the Device Action Flags are packed in one byte for messaging, and that is why both of those parameters are limited to four bits each. Also the length of the message prologue (byte offsets 0 to 8 in the message body) is fixed at nine bytes so that encoding in base-64 always results in the same number of bytes, regardless of the message being encoded.
###Message Header
The DEVICE_QUERY and DEVICE_RESPONSE messages are Firmata Sysex messages. The first message byte is always START_SYSEX (0xF0). The second byte is the Sysex command byte, in this case either DEVICE_QUERY (0x30) or DEVICE_RESPONSE (0x31). This 2-byte header is transmitted as given with no encoding. The body of the message follows the header and is entirely encoded in base-64. As with all Sysex messages, the final byte of the message is always an unencoded END_SYSEX (0xF7) command byte.
####DEVICE_QUERY header
0 START_SYSEX byte (0xF0).
1 Sysex command byte DEVICE_QUERY (0x30).
####DEVICE_RESPONSE header
0 START_SYSEX byte (0xF0).
1 Sysex command byte DEVICE_RESPONSE (0x31).
###Message Body
The body of the message includes all the values needed to reconstitute the method call and response. These values are marshalled into a single buffer by the sending Firmata method immediately before transmission. The values are unmarshalled by the receiving Firmata method immediately after receipt. Thus the following layouts only exist for a brief moment after marshall and before encode and send, or after receipt and decode and before unmarshall.
Character strings are stored on the server in UTF-8. All eight bits in a UTF-8 byte are significant. A '0' in the high order bit indicates a character in the first group of 127 characters (the ASCII character set). A '1' in the high order bit indicates that the byte is part of a multi-byte sequence. Unfortunately, it might also indicate a Firmata control byte. Encoding in Base-64 avoids this problem.
Remember that all bytes in the body of the message are encoded prior to transmission. The values shown in the layout tables below are the 8-bit values before or after encoding / decoding, they are not the encoded 7-bit quantities that are actually transmitted.
The layouts shown here determine the contents of the bytes visible during transmission, but the actual byte contents will be different due to the base-64 encoding.
Note Since there are 9 raw bytes in the common part of every message body, after encoding and during transmission each message body starts with a 12-byte encoded byte sequence. The encoded data bytes, if any, follow after that.
####DEVICE_QUERY body
The raw DEVICE_QUERY message body has the following format before encoding.
0 [Action flags | Action codes], with values as described below.
1 Options (OPEN) (LSB) or handle value (READ, WRITE, CLOSE) (LSB)
2 Options (OPEN) (MSB) or handle value (READ, WRITE, CLOSE) (MSB)
3 Register (READ, WRITE) (LSB) or 0 (Reserved) (OPEN, CLOSE)
4 Register (READ, WRITE) (MSB) or 0 (Reserved) (OPEN, CLOSE)
5 Requested byte count (READ, WRITE) (LSB) or 0 (Reserved) (OPEN, CLOSE)
6 Requested byte count (READ, WRITE) (MSB) or 0 (Reserved) (OPEN, CLOSE)
7 0 (Reserved)
8 0 (Reserved)
9..n The data bytes supporting the query, if any
####DEVICE_RESPONSE body
The raw DEVICE_RESPONSE message body has the following format before encoding.
0 [Action flags | Action codes] from the associated DEVICE_QUERY.
1 Options (OPEN) (LSB) or handle value (READ, WRITE, CLOSE) (LSB)
2 Options (OPEN) (MSB) or handle value (READ, WRITE, CLOSE) (MSB)
3 Register (READ, WRITE) (LSB) or 0 (Reserved) (OPEN, CLOSE)
4 Register (READ, WRITE) (MSB) or 0 (Reserved) (OPEN, CLOSE)
5 Requested byte count (READ, WRITE) (LSB) or 0 (Reserved) (OPEN, CLOSE)
6 Requested byte count (READ, WRITE) (MSB) or 0 (Reserved) (OPEN, CLOSE)
7 Status: Error code (negative), ESUCCESS (0), handle, or actual byte count read or written (LSB)
8 Status: Error code (negative), ESUCCESS (0), handle, or actual byte count read or written (MSB)
9..n The data bytes supporting the response, if any
####Device Action Codes
These are 4-bit values identifying the device driver action to be invoked. They are stored in the Firmata DEVICE_QUERY and DEVICE_RESPONSE message bodies at offset 0, bits 0-3.
OPEN (0x0)
READ (0x1)
WRITE (0x2)
CLOSE (0x3)
####Device Action Flags
These are 4-bit values that qualify the requested action. They are stored in the Firmata DEVICE_QUERY and DEVICE_RESPONSE message bodies at offset 0, bits 4-7.
There is only one set of device action flags shared among all the action methods (open(), read(), write(), and close()). Presumably the usage of each flag will be similar across the methods, but there is no requirement that it be exactly the same in each method. Similarly, the same numeric value can be used for entirely different meanings in the different methods. In the latter case, the flags should have different names for clarity, even though the actual numeric values are the same.
| Flag Value | OPEN | READ | WRITE | CLOSE |
|---|---|---|---|---|
| 0x0 | NONE | NONE | NONE | NONE |
| 0x1 | FORCE | FORCE | FORCE | FORCE |
| ... | -- | -- | -- | -- |
| 0xC | -- | MILLI_RUN | MILLI_RUN | -- |
| 0xD | -- | MILLI_STOP | MILLI_STOP | -- |
| 0xE | -- | MICRO_RUN | MICRO_RUN | -- |
| 0xF | -- | MICRO_STOP | MICRO_STOP | -- |
####Options or Handle
These are 16-bit values, stored in the Firmata DEVICE_QUERY and DEVICE_QUERY message bodies at offsets 1 and 2. Options are provided to the open() method, the handle is provided to the other methods. The values are stored in the device driver and in the client in a single, wider integer variable (int16_t, int32_t, etc).
Note that handle values are always positive to distinguish them from errors which are always negative and share the same slot in open() response messages.
1 options (LSB)
2 options (MSB)
or
1 handle (LSB)
2 handle (MSB)
####Register numbers
These are 16-bit signed values that identify the register (either real or virtual) to be read or written. See the discussion in the main document and other appendices for more detail about the Device Read and Write Registers.
####Status / Return Value from Methods
Each of the device driver methods returns an int value to the caller. The meaning of the returned int varies depending on the method called. Error return values are negative. Success return values are either 0 or positive, and can be the ESUCCESS status code, a handle value, or a byte count, depending on the method called.
Note that the byte count returned by the various methods is the actual number of bytes read or written by the device driver, it is not the length of the encoded message body. This shouldn't be a problem except as something to remember when debugging and looking at the messages as they are transmitted.
##Detailed Device Driver Message Formats
###Device Driver - Open
####Query
Message Header (Plain text)
0 START_SYSEX (0xF0) 1 DEVICE_QUERY (0x30)Message Body (before base-64 encoding)
0 [Open flags | OPEN action 0x0] 1 Options (LSB) 2 Options (MSB) 3 0 (Reserved) 4 0 (Reserved) 5 0 (Reserved) 6 0 (Reserved) 7 0 (Reserved) 8 0 (Reserved) 9..n name string (UTF-8, null terminated)Message End (Plain text)
k END_SYSEX (0XF7)
####Response
Message Header (Plain text)
0 START_SYSEX (0xF0) 1 DEVICE_RESPONSE (0x31)Message Body (before base-64 encoding)
0 [Open flags | OPEN action 0x0] 1 Options (LSB) 2 Options (MSB) 3 0 (Reserved) 4 0 (Reserved) 5 0 (Reserved) 6 0 (Reserved) 7 Returned handle or error code (LSB) 8 Returned handle or error code (MSB) 9..n name string (UTF-8, null terminated)Message End (Plain text)
k END_SYSEX (0XF7)
###Device Driver - Read
####Query
Message Header (Plain text)
0 START_SYSEX (0xF0) 1 DEVICE_QUERY (0x30)Message Body (before encoding)
0 [Read flags | Read action 0x1] 1 Handle (LSB) 2 Handle (MSB) 3 Register (LSB) 4 Register (MSB) 5 Number of bytes to read (LSB) 6 Number of bytes to read (MSB) 7 0 (Reserved) 8 0 (Reserved)Message End (Plain text)
k END_SYSEX (0XF7)
####Response
Message Header (Plain text)
0 START_SYSEX (0xF0) 1 DEVICE_RESPONSE (0x31)Message Body (before encoding)
0 [Read flags | Read action 0x1] 1 Handle (LSB) 2 Handle (MSB) 3 Register (LSB) 4 Register (MSB) 5 Number of bytes to read (LSB) 6 Number of bytes to read (MSB) 7 Number of bytes actually read or error code (LSB) 8 Number of bytes actually read or error code (MSB) 9..n The requested data bytes, if anyMessage End (Plain text)
k END_SYSEX (0XF7)
###Device Driver - Write ####Query
Message Header (Plain text)
0 START_SYSEX (0xF0) 1 DEVICE_QUERY (0x30)Message Body (before encoding)
0 [Write flags | Write action 0x2] 1 Handle (LSB) 2 Handle (MSB) 3 Register (LSB) 4 Register (MSB) 5 Number of bytes to write (LSB) 6 Number of bytes to write (MSB) 7 0 (Reserved) 8 0 (Reserved) 9..n The data bytes to write, if anyMessage End (Plain text)
k END_SYSEX (0XF7)
####Response
Message Header (Plain text)
0 START_SYSEX (0xF0) 1 DEVICE_RESPONSE (0x31)Message Body (before encoding)
0 [Write flags | Write action 0x2] 1 Handle (LSB) 2 Handle (MSB) 3 Register (LSB) 4 Register (MSB) 5 Number of bytes to write (LSB) 6 Number of bytes to write (MSB) 7 Number of bytes actually written or error code (LSB) 8 Number of bytes actually written or error code (MSB)Message End (Plain text)
k END_SYSEX (0XF7)
###Device Driver - Close
####Query
Message Header (Plain text)
0 START_SYSEX (0xF0) 1 DEVICE_QUERY (0x30)Message Body (before encoding)
0 [Close flags | Close action 0x3] 1 Handle (LSB) 2 Handle (MSB) 3 0 (Reserved) 4 0 (Reserved) 5 0 (Reserved) 6 0 (Reserved) 7 0 (Reserved) 8 0 (Reserved)Message End (Plain text)
k END_SYSEX (0XF7)
####Response
Message Header (Plain text)
0 START_SYSEX (0xF0) 1 DEVICE_RESPONSE (0x31)Message Body (before encoding)
0 [Close flags | Close action 0x3] 1 Handle (LSB) 2 Handle (MSB) 3 0 (Reserved) 4 0 (Reserved) 5 0 (Reserved) 6 0 (Reserved) 7 ESUCCESS or error code (LSB) 8 ESUCCESS or error code (MSB)Message End (Plain text)
k END_SYSEX (0XF7)