INTERNET-DRAFT A. Soloviev
draft-avsolov-dtpdia-03.txt Petrozavodsk State University
Expires: February 28, 2003 September 1, 2003
Data Transfer Protocol
for Distributed Information Acquisition (DTP/DIA)
Status of This Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC 2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress".
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on February 28, 2003.
Copyright Notice
Copyright (C) The Internet Society (2002). All Rights Reserved.
Abstract
The described in this memo protocol was developed for the application
in distributed information measurement systems (IMS) at any stage of
communication. It was designed for transmission of measured data
from measuring devices to processing and storing equipment. Also it
does support common device identification in distributed IMS. Due to
its simplicity it can be easily implemented in firmware of any
digital measuring device.
Soloviev Expires February 28, 2004 [Page 1]
Internet-Draft DTP/DIA September 2003
1. Introduction
The Data Transfer Protocol for Distributed Information Acquisition
(DTP/DIA) was developed for application in distributed information
measurement systems (IMS). DTP/DIA provides the functionality of the
presentation and application layers of the OSI Reference Model for
the specified purposes.
1.1. Concept of Distributed IMS
IMS stands for a bundle of software and hardware which performs
acquisition, processing, storing, and presentation of measured data.
The systems with control function are out of scope. The simplest IMS
consists of measuring device, connected to computer by means of some
instrument interface (such as IEEE 488 (GPIB) or EIA/RS 232).
Typical measuring device contains a sensor and a microcontroller
which performs initial data acquisition and processing. In such a
system the majority of IMS functions are given to computer.
More complicated systems can be developed on the basis of CAMAC or
VXI. But projects with large amount of investigated objects at far
distances from each other require to install distributed IMS. Some
nodes of distributed IMS should perform data transmission to nodes
which process and store data. Sometimes this communication can be
done by measuring device itself if it is rather sophisticated, or
this function may be performed by a computer. That is we deal with
two kinds of data channels: local (1) and network (2).
+---------+ +-----------------------+
|measuring|==============(2)============| | computer ............|
| device | ||==|(collector): database |
+---------+ (1)-local (instrument) bus || +-----------------------+
(2)-network channel || +-----------------------+
+---------+ ||==| computer ............|
|measuring| +--------------+ || |(collector): web-server|
| device |--(1)--| | || +-----------------------+
+---------+ | computer | || +-----------------------+
+---------+ |(retranslator)| ||==| computer :calculating|
|measuring|--(1)--| |==(2)=| |(collector): software |
| device | +--------------+ +-----------------------+
+---------+
In most cases the nodes of distributed IMS are considered to be the
components of the open systems. So OSI Reference Model can be
applied to their communication channels. The upper layers of such
systems are poorly standardized due to the wide range of IMS's
applications. The most of existing standards are vendor-specific.
That's why it's necessary to introduce a simple protocol, suitable
Soloviev Expires February 28, 2004 [Page 2]
Internet-Draft DTP/DIA September 2003
for both kinds of data channel. DTP/DIA matches these requirements.
1.2. Remark about Terms
We avoid using terms "client" and "server" in description of IMS.
Instead we use the term "data source" for the object which transmits
measured data and "data collector" - for the node which receives
measured data. If the node performs both reception and
retransmission this type of nodes will be named "retranslator".
Obviously, in this terminology measuring devices are "data sources",
but a single computer could be either "end-point data collector" or
"retranslator".
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119.
2. General Features
The protocol describes data transmission in small stand-alone
packets. This feature allows using DTP/DIA over both streamed
(connection-oriented) and message-oriented channels.
DTP/DIA provides few data presentation forms. Each form corresponds
to a specific packet type. Some of the forms help to avoid the
implementation of floating point arithmetics in device firmware.
DTP/DIA allows transmission of measured data accompanied with
information about measurement error and its unit measure.
In distributed IMS it is very important to distinguish measuring
devices from each other at the application layer. So the protocol
includes identification mechanism. This feature helps to transmit
data from several data sources over a single communication channel.
For example, one can use a device with several sensors (such device
is represented as multiple data sources).
We suppose to apply DTP/DIA for both network and local channels. Few
types of local channels don't provide reliable data delivery. To
avoid unreliability of local interfaces DTP/DIA has the following
primitive features: detection of packet start (packet leading
sequence), check sum and time stamp.
Soloviev Expires February 28, 2004 [Page 3]
Internet-Draft DTP/DIA September 2003
3. DTP/DIA Packet Format
DTP/DIA packet consists of three logical blocks: Header, Measured
Data and Special Data. Header and Measured Data are REQUIRED. They
have a fixed size (8 bytes and 4 bytes correspondingly). Special
Data block is RECOMMENDED and its size is variable. Here and further
the octets numbering starts from 0.
Header block contains packet leading sequence, version field (VERS),
flags field (L, T, and U bits), Data Source Identifier (ID.1 and ID.2
fields), size field (SIZE), packet type field (TYPE), and device
vendor specific data (DEVINFO).
The content of Measured Data block is determined by the packet type.
Special Data may contain UNIT MEASURE MARK, accuracy fields (PROB and
ERROR), List of Inquiring Identifiers, TIMESTAMP, and CHECKSUM.
A simple measuring device must produce packets, contained at least
Header and one of the forms of Measured Data block. More
sophisticated device should include Special Data block as well.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1 0 0 1 0 0 1 0|0 0 1 0 1 0 1 0| VERS |L|T|U|R| ID.1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID.2 | SIZE | TYPE | DEVINFO |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Measured Data |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
+-+-+-+- . . . Special Data . . . -+-+-+-+
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TIMESTAMP | CHECKSUM |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
3.1. Packet Leading Sequence
The first two octets of DTP/DIA packet contain 0x49 0x54. These
values must be used by software to detect the origin of DTP/DIA
packet. Collector software must ignore any data not started with the
specified packet leading sequence.
3.2. Version Field
Current version code is 0.
Soloviev Expires February 28, 2004 [Page 4]
Internet-Draft DTP/DIA September 2003
3.3. Flags Field
The 20th bit (L) determines the kind of byte order was used in the
packet. 1 corresponds to little-endian format, 0 corresponds to big-
endian format. It's up to IMS developer to choose byte order format.
Obviously, it depends on what kind of microcontroller was used in
measuring device. For example, Intel's MCS 96/296 uses little-endian
byte order, but Motorola's MC68xxx uses big-endian byte order.
The 21st bit (T) determines whether TIMESTAMP field has to be ignored
or not (see Section 3.10). If this bit is set (T=1) TIMESTAMP must
be ignored, otherwise it requires a valid value inside.
The 22nd bit (U) determines whether UTF-8 [10] encoding should be
used for representation of textual information. If this bit is set
(U=1) any text in packet must be encoded with UTF-8, otherwise text
is represented as common ASCII.
The 23rd bit is reserved for future use and must be 0.
3.4. Data Source Identifier
To distinguish data from various data sources DTP/DIA packet contains
Data Source Identifier. To introduce a bit of structurization in
distributed IMS the Identifier was divided into two parts: ID.1 and
ID.2. They correspond to high and low levels of 2-level hierarchy.
Developer of IMS may use ID.1 to specify the group of units in
distributed IMS and ID.2 to appoint a number of device in this group.
The notation "AAA/BBBBB" will be used to specify Data Source
Identifier (where AAA - ID.1 and BBBBB - ID.2). The byte order of
ID.2 field depends on the value of flag L.
It's not recommended to assign a fixed identifier for data source by
device vendor. Developer of IMS should have an opportunity to change
Data Source Identifier, for example, by means of special software or
DIP-switches. Device vendor may apply Data Source Identification
Procedure for assigning identifiers as described in Section 4.
Identifiers "0/0" and "0/65535" are reserved for Data Source
Identification Procedure (see Section 4).
Every retranslator SHOULD keep Data Source Identifier untouched.
3.5. Size of the Packet
The size of the packet specifies amount of 32-bit words occupied by
the packet. The minimal size of DTP/DIA packet is 12 bytes (SIZE=3).
The maximal size of DTP/DIA packet is 60 bytes (SIZE=15).
Soloviev Expires February 28, 2004 [Page 5]
Internet-Draft DTP/DIA September 2003
The values 0, 1, and 2 are illegal. Collector software should scan 7
bytes back for another packet leading sequence or discard received
bytes.
3.6. Packet Types
The following packet types are declared: TYPE_FLOAT (TYPE=0),
TYPE_INT1 (TYPE=1), TYPE_INT2 (TYPE=2), TYPE_INT3 (TYPE=3), TYPE_INFO
(TYPE=14), TYPE_SPEC (TYPE=15). Other values are reserved for future
use. Collector software should ignore the packets of reserved types.
TYPE_FLOAT, TYPE_INT1, TYPE_INT2, and TYPE_INT3 specify different
data presentation forms (as described in Section 3.8).
TYPE_INFO packets are designed to provide additional text information
about data source. Octets from the 8th to the end of the packet
(except for the last 32-bit word) are filled with some text (firmware
version, copyrights, vendor information etc). Encoding of this text
depends on flag U. Such text must be terminated by at least one
octet zero and padding bytes must contain zeros as well. Maximal
length of this text without trailing zeros is 47 bytes. Collector
software may silently ignore such packets.
TYPE_SPEC packets have a special purpose in the Data Source
Identification Procedure (see Section 4). When this packet is used
in Data Source Identification Procedure, Measured Data block must be
filled with zeros. In other cases Measured Data block may contain
some device state information when the packet was issued by data
source, or it may contain some control information when the packet
was issued by collector. These values are vendor specific.
3.7. Device Vendor Specific Data
The DEVINFO field contains device vendor specific information. This
value MUST NOT influence on interpretation of MEASURED DATA.
3.8. Measured Data block
The content of Measured Data block is determined by the packet type.
TYPE_FLOAT packet (TYPE=0)
Measured Data contains a certain value of physical quantity,
represented as 32-bit floating point number ("single" in terms of
IEEE 754). These bits contain a sign bit (S), 8 bits of exponent
(E), and 23 bits of mantissa's fraction (M1...M23). Initial
reference value is calculated in such a way (see details in [6]):
(-1)^S * 2^(E-127) * 1.M1M2M3...M23
Here we use "^" as exponentiating operator.
Soloviev Expires February 28, 2004 [Page 6]
Internet-Draft DTP/DIA September 2003
TYPE_INT1 packet (TYPE=1)
Measured Data contains multiplied by 10 value of physical
quantity, represented as 32-bit signed integer number.
TYPE_INT2 packet (TYPE=2)
Measured Data contains multiplied by 100 value of physical
quantity, represented as 32-bit signed integer number.
TYPE_INT3 packet (TYPE=3)
Measured Data contains multiplied by 1000 value of physical
quantity, represented as 32-bit signed integer number.
Byte order of Measured Data block depends on the value of flag L.
Retranslator may convert measured data presentation from one form to
another.
3.9. UNIT MEASURE MARK and Accuracy Fields
If information about measurement error and unit measure are to be
transmitted DTP/DIA packet should include UNIT MEASURE MARK and
accuracy fields (PROB and ERROR).
+-----------------+-------+-------+
|UNIT MEASURE MARK| PROB | ERROR |
+-----------------+-------+-------+
UNIT MEASURE MARK starts from the 12th octet of the packet. This
field is considered to be a text notation of unit measure. It is
recommended to place generally used notations, based on [9], into
this field. Encoding of this text depends on flag U. Such text must
be terminated by at least one octet zero. The size of this field is
variable but must be divisible by 4 and padding octets must contain
zeros. Maximal length of this text without trailing zeros is 43
bytes. UNIT MEASURE MARK must not be used in TYPE_INFO and TYPE_SPEC
packets. UNIT MEASURE MARK is required if accuracy fields are
present (but may be filled with zeros).
Accuracy fields (PROB and ERROR) start just after the trailing zeros
of UNIT MEASURE MARK. PROB offset must be aligned by 4. ERROR field
contains the measurement relative error and follows PROB. PROB field
contains the probability of physical quantity being outside of the
interval determined by measurement relative error (the complement of
the reliability to 1). In TYPE_FLOAT packets accuracy fields are
represented as 32-bit floating point values and occupy 8 bytes. In
TYPE_INTx packets they are represented as 16-bit unsigned integer
numbers multiplied by 10000 and occupy 4 bytes. Byte order of these
values depends on the value of flag L. Accuracy fields are OPTIONAL.
Soloviev Expires February 28, 2004 [Page 7]
Internet-Draft DTP/DIA September 2003
3.10. TIMESTAMP
To distinguish various packets and so to avoid packet duplication
TIMESTAMP may be included into the packet. TIMESTAMP stands for
measurement time, represented as 24 least significant bits in
seconds, elapsed since 00:00:00 UTC, January 1, 1970. Byte order of
TIMESTAMP is determined by bit L in the flags field.
If collector receives two packets from one data source with the same
TIMESTAMP it should discard one of these packets. This situation may
happen not only because of packets duplication but when developer of
IMS has implemented the opportunity to correct measured data or to
make it more accurate on-the-fly. So collector software may be set
up to discard the first packet with the same TIMESTAMP.
This field is OPTIONAL. In the case TIMESTAMP and the whole Special
Data block are absent (SIZE=3), the bit T in the flags field must be
set (T=1). If Special Data block is included into packet the space
for TIMESTAMP is reserved. In the case time stamp information isn't
required the bit T must be set (T=1), TIMESTAMP must be filled with
zero and will be ignored by collector. If bit T is cleared (T=0)
TIMESTAMP contains a valid value and may be treated.
3.11. CHECKSUM
If Special Data block is present CHECKSUM occupies the last octet of
the packet. CHECKSUM is the residue of division of the sum of
previous bytes by 256 (modulus of 256). This field is required if
the packet is larger than 12 bytes (SIZE>3). The packet with
incorrect CHECKSUM must be discarded.
4. Data Source Identification Procedure
Data Source Identification Procedure is an optional mechanism which
helps sharing a single transport layer connection for data
transmission from several data sources. This feature must not be
applied to connectionless (message-oriented) channels. So data
source can distinguish every collector by a certain transport layer
connection. This mechanism consists of two events: "Offer To
Identify" and "Device Request".
"Offer To Identify" corresponds to TYPE_SPEC packet with
ID="0/65535", sent by data source equipment. This packet may be sent
not only just right after connection establishment but at any moment
(for example, when hardware configuration of data sources has
changed). Expected reaction of collector is "Device Request" packet.
If collector ignores "Offer To Identify" (doesn't reply for specified
Soloviev Expires February 28, 2004 [Page 8]
Internet-Draft DTP/DIA September 2003
time) the action of data source must be one of the following:
- 0..3 retries of "Offer To Identify" then it must break the
transport layer connection;
- 0..3 retries of "Offer To Identify" then it must start
transmission of data packets of data source with the least
existing identifier.
If collector requests both non-existing and existing data sources the
equipment action must be one of the following:
- 0..3 retries of "Offer To Identify" then it must break the
transport layer connection;
- 0..3 retries of "Offer To Identify" then it must ignore the non-
existing data sources request;
If collector requests non-existing data source only the equipment
must break the transport layer connection.
"Device Request" corresponds to TYPE_SPEC packet with ID="0/0", sent
by collector. This packet must contain List of Inquiring Identifiers
in Special Data block. The size of this list is variable but
divisible by 4 (the size of sequence). The list starts from the 12th
octet and lasts to the end of the packet. Each Data Source
Identifier in this list occupies 3 last octets in each sequence.
Leading byte in each sequence is zero. Byte order of ID.2 depends on
the value of flags L.
+0 +1 +2 +3
+----+----+----+----+
| 0 |ID.1| ID.2 |
+----+----+----+----+
List of Inquiring Identifiers may contain up to 11 identifiers of the
data sources, expected to send their data through this communication
channel. If collector needs to request more than 11 data sources it
must distribute their identifiers to several "Device Request"
packets. If data source accepts "Device Request" it starts sending
data packets. Collector may send "Device Request" packet at any time
of communication (not only when it answers to "Offer To Identify").
The difference between stand-alone "Device Request" and the sequence
"Offer To Identify" - "Device Request" is that every time data source
equipment sends "Offer To Identify" packet it clears the stack of
requested identifiers for the corresponding collector, but when data
source receives the next "Device Request" it adds identifiers from
Soloviev Expires February 28, 2004 [Page 9]
Internet-Draft DTP/DIA September 2003
this packet to current stack of requested identifiers.
Also Data Source Identification Procedure may be applied for
assigning identifiers. In this case the first identifier requested
by collector should be used by data source as its own Data Source
Identifier.
Retranslator may act as collector or as data source in this Data
Source Identification Procedure.
5. Protocol Number
IANA has assigned TCP and UDP port number 3489 for DTP/DIA.
6. Security Considerations
DTP/DIA is definitely insecure. To produce security based
applications developer of IMS should provide authentication and data
protection on the transport layer (by means of SSL [5] or TLS [4].
Soloviev Expires February 28, 2004 [Page 10]
Internet-Draft DTP/DIA September 2003
References
[1] ANSI, "Coded Character Set - 7-Bit American Standard Code for
Information Interchange", ANSI X3.4-1986.
[2] Bradner S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, 1997.
[3] Bradner S., "The Internet Standards Process - Revision 3", BCP 9,
RFC 2026, 1996.
[4] Dierks T., Allen C., "The TLS Protocol Version 1.0", RFC 2246,
1999.
[5] Frier A., Karlton P., Kocher P., "The SSL 3.0 Protocol", Netscape
Communications Corp., 1996.
[6] IEEE, "IEEE Standard for Binary Floating-Point Arithmetics", IEEE
754-1985.
[7] Postel J., "Transmission Control Protocol", STD 7, RFC 793, 1981.
[8] Postel J., "User Datagram Protocol", STD 6, RFC 768, 1980.
[9] "Symbols Units and Nomenclature in Physics". Document IUPAP-25,
1987.
[10] Yergeau F., "UTF-8, a transformation format of ISO 10646",
RFC2279, 1998.
Acknowledgements
The author gratefully acknowledges the supervision of A. Moschevikin,
associate professor, PhD, and valuable advice of A. Korolkov.
Soloviev Expires February 28, 2004 [Page 11]
Internet-Draft DTP/DIA September 2003
Author's Address
Alexei V. Soloviev
Chair of IMS & Physical Electronics
Petrozavodsk State University
Lenin St. 33
185640 Petrozavodsk, Karelia
RUSSIA
Phone: +7-8142-711021
E-mail: avsolov@lab127.karelia.ru
Full Copyright Statement
Copyright (C) The Internet Society (2002). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will not be
revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Soloviev Expires February 28, 2004 [Page 12]
Internet-Draft DTP/DIA September 2003
Appendix: Glossary of Acronyms
CAMAC - Computer Automated Measurement And Control
EIA/RS - Electronic Industries Association Recommended Standard
GPIB - General Purpose Interface Bus
IANA - Internet Assigned Numbers Authority
IEC/CEI - International Electrotechnical Commission
IEEE - Institute of Electrical and Electronics Engineers
IETF - Internet Engineering Task Force
IMS - Information Measurement System
ISO - International Organization for Standardization
OSI/RM - Open Systems Interconnection Reference Model
SSL - Secure Sockets Layer
TCP - Transmission Control Protocol
TLS - Transport Layer Security
UDP - User Datagram Protocol
UTC - Universal Coordinated Time
VXI - VME bus eXtension for Instruments
Soloviev Expires February 28, 2004 [Page 13]