IPFIX working group
Internet Draft EDITOR: B. Claise
draft-ietf-ipfix-protocol-07.txt Cisco Systems
Expires: June 2005 December 2004
IPFIX Protocol Specification
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of RFC 3668.
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts. Internet-Drafts are draft documents valid for a maximum of
six months and may be updated, replaced, or obsolete by other
documents at any time. It is inappropriate to use Internet-Drafts
as reference material or to cite them other than as "work in
progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html
Abstract
This document specifies the IPFIX protocol that provides network
operators with access to IP flow information. In order to export
IP flow information to the IPFIX collecting process, a common method
of representing the flow data and a standard means of communicating
them from an exporter to a collector is required. This document
describes how the IPFIX flow record data, options record data and
Claise, et. al Standard Track [Page 1]
IPFIX Protocol Specification December 2004
templates are carried over a congestion-aware transport protocol
from an IPFIX exporting process to an IPFIX collecting process.
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119.
Table of Contents
1. Points of Discussion.........................................4
2. Introduction.................................................5
2.1 IPFIX Documents Overview...................................6
3. Terminology..................................................6
3.1 Terminology Summary Table.................................11
4. Criteria for Flow Expiration and Export.....................12
4.1 Flow Expiration...........................................12
4.2 Flow Export...............................................12
5. Message Format..............................................13
6. IPFIX Message Format........................................15
6.1 Header Format.............................................15
6.2 Field Type Format.........................................16
6.3 Template Set Format.......................................17
6.4 Data Set Format...........................................19
6.5 Options Template Set......................................21
6.5.1 Scope..................................................21
6.5.2 Options Template Set Format............................22
6.5.3 Options Data Record Format.............................24
7. Specific Reporting Requirements.............................26
7.1 The Metering Process Statistics Option Template...........26
7.2 The Metering Process Reliability Statistics Option Template27
7.3 The Exporting Process Reliability Statistics Option Template28
7.4 The Flow Keys Option Template.............................28
8. "Export Time" Computation and Flow Record Time..............29
8.1 Microsecond Precision.....................................29
8.2 Millisecond Precision.....................................30
8.3 Nanosecond Precision......................................31
8.4 Multiple Precisions.......................................31
9. Linkage with the Information Model..........................31
9.1 Reduced Size Encoding of Integer Types....................31
10. Variable Length Information Element........................32
11. Template Management........................................33
12. The Collecting Process's Side..............................36
13. Transport Protocol.........................................38
13.1 Transport Compliance and Transport Usage.................38
13.2 TCP......................................................39
13.2.1 Congestion Avoidance...................................39
Claise, et. al Standard Track [Page 2]
IPFIX Protocol Specification December 2004
13.2.2 Reliability............................................39
13.2.3 MTU....................................................39
13.2.4 Exporting Process......................................39
13.2.4.1 Connection Establishment..............................39
13.2.4.2 Connection Release....................................40
13.2.4.3 IPFIX Message Encoding................................40
13.2.5 Template Management....................................40
13.2.6 Fail-over..............................................41
13.3 SCTP.....................................................41
13.3.1 Congestion Avoidance...................................41
13.3.2 Reliability............................................41
13.3.3 MTU....................................................42
13.3.4 Exporting Process......................................42
13.3.4.1 Association Establishment.............................42
13.3.4.2 Association Shutdown..................................42
13.3.4.3 Source ID.............................................43
13.3.4.4 Stream................................................43
13.3.4.5 Template Management...................................44
13.3.5 Collecting Process.....................................44
13.3.6 Failover...............................................44
13.4 UDP......................................................44
13.4.1 Congestion Avoidance...................................44
13.4.2 Reliability............................................44
13.4.3 MTU....................................................45
13.4.4 Port Numbers...........................................45
13.4.5 Exporting Process......................................45
13.4.6 Template Management....................................45
13.4.7 Collecting Process.....................................46
13.4.8 Failover...............................................47
14. Security Considerations....................................47
14.1 IPsec Usage..............................................47
14.1.1 Selectors..............................................48
14.1.2 Mode...................................................48
14.1.3 Key Management.........................................48
14.1.4 Security Policy........................................48
14.1.5 Authentication.........................................48
14.1.6 Availability...........................................49
14.2 TLS Usage................................................49
14.3 Protection against DoS attacks...........................49
14.4 When IPsec or TLS is not an option.......................50
14.5 Logging an IPFIX Attack..................................50
15. IANA Considerations........................................51
15.1 Numbers used in the Protocol.............................51
15.2 Numbers used in the Information Model....................51
16. Examples...................................................52
16.1 Message Header Example...................................52
16.2 Template Set Examples....................................53
16.2.1 Template Set using IETF specified Information Elements53
Claise, et. al Standard Track [Page 3]
IPFIX Protocol Specification December 2004
16.2.2 Template Set using Enterprise Specific Information
Elements 53
16.3 Data Set Example.........................................54
16.4 Options Template Set Examples............................55
16.4.1 Options Template Set using IETF specified Information
Elements 55
16.4.2 Options Template Set using Enterprise Specific
Information Elements...........................................56
16.4.3 Options Template Set using an Enterprise Specific scope57
16.5 Data Set with Options Data Records Example...............57
17. References.................................................58
17.1 Normative References.....................................58
17.2 Informative References...................................59
18. Acknowledgments............................................60
1. Points of Discussion
This section covers the open issues, still to be resolved/updated in
this draft. Note that the issues starting with PROTO-31 have been
added to this draft version.
PROTO-23: Finalize the time details. The time-related Information
Elements are not defined in [IPFIX-INFO]. We agree that the
definition of timer semantics should be moved to the info model
document (see the http://ipfix.doit.wisc.edu/archive/2588.html email
thread). Now we need:
- to insert some new text in the section 9 of this document
- to insert the timing Information Element in [IFPIX-INFO]
Note: see http://ipfix.doit.wisc.edu/archive/2580.html for a way to
improve the timing-related Information Elements.
PROTO-30: review RFC 3917, to see if we miss any requirements
PROTO-34: Need a security expert to review the security section:
- [TBD] in the security section
- [EDITOR NOTE: the security section may need be adapted to
the revised transport section], [XXX-REFERENCE]
- [XXX-SCTP-BLIND-SPOOFING-REFERENCE] not defined
- TCP Security
1. Handshake: should we introduce a handshake sequence at the
start of the connection? A simple ASCII-based handshake could
be used to request TLS.
2. It would make sense to add TLS support even in the absence
of a handshake. It would be the responsibility of the
Claise, et. al Standard Track [Page 4]
IPFIX Protocol Specification December 2004
collector (connection initiator) to know whether TLS setup is
required.
PROTO-38: [IPFIX-INFO] consistency issue:
- the ipfixOption, time, droppedFUPacketCount,
droppedFUByteCount, droppedFlows, keyList,
timeFirstFUDropped, timeLastFUDropped, droppedFAPacketCount,
droppedFAByteCount, timeFirstFADropped, timeLastFADropped,
and Exporter ID, Source ID Information Elements are not used
in this document but not yet specified in [IPFIX-INFO].
- Review the Options Template example once the Source ID is
defined as an information element (currently the ID 141 is
used)
PROTO-44: UDP, TCP, and SCTP ports XXXX should be replaced once
assigned by IANA. Proposal: 4739
PROTO-46: TCP section update, waiting text for Simon Leinen. See the
thread starting with http://ipfix.doit.wisc.edu/archive/2569.html
PROTO-47: Some ææEDITORÆS NOTEÆÆ in the draft.
PROTO-48: Should the ææsequence numberÆÆ be improved to contains the
number of flow records, like proposed in
http://ipfix.doit.wisc.edu/archive/2587.html?
2. Introduction
A data network with IP traffic, primarily consists of IP Flows
passing through the network elements of the network. It is often
interesting, useful or even a requirement to have access to
information about these flows that pass through the network elements
for administrative or other purposes. The IPFIX collecting process
should be able to receive the flow information passing through
multiple network elements within the data network. This requires
uniformity in the method of representing the flow information and
the means of communicating the flows from the network elements to
the collection point. This document specifies the protocol to
achieve these aforementioned requirements. This document specifies
in detail the representation of different flows, the additional data
required for flow interpretation, packet format, transport
mechanisms used, security concerns, etc.
Claise, et. al Standard Track [Page 5]
IPFIX Protocol Specification December 2004
2.1 IPFIX Documents Overview
The IPFIX protocol provides network administrators with access to IP
flow information. The architecture for the export of measured IP
flow information out of an IPFIX exporting process to a collecting
process is defined in [IPFIX-ARCH], per the requirements defined in
[IPFIX-REQ]. This document specifies how IPFIX flow record data,
options record data, and templates are carried via a congestion-
aware transport protocol from IPFIX exporting process to IPFIX
collecting process. IPFIX has a formal description of IPFIX
information elements (fields), their name, type and additional
semantic information, as specified in [IPFIX-INFO]. Finally [IPFIX-
AS] describes what type of applications can use the IPFIX protocol
and how they can use the information provided. It furthermore shows
how the IPFIX framework relates to other architectures and
frameworks.
3. Terminology
The definitions of the basic terms like IP Traffic Flow, Exporting
Process, Collecting Process, Observation Points, etc. are
semantically identical with that found in the IPFIX requirements
document [IPFIX-REQ]. Some of the terms have been expanded for more
clarity when defining the protocol. Additional terms required for
the protocol has also been defined. Definitions in this document
and in [IPFIX-ARCH] are equivalent, except that definitions which
are only relevant to the IPFIX protocol only appear here. Should
there be any apparent discrepancy in definitions between these two
documents, the definitions defined in this document take precedence.
The terminology summary table in Section 3.1 gives a quick overview
of the relationships between some of the different terms defined.
Observation Point
An Observation Point is a location in the network where IP packets
can be observed. Examples include: a line to which a probe is
attached, a shared medium, such as an Ethernet-based LAN, a single
port of a router, or a set of interfaces (physical or logical) of a
router.
Note that one Observation Point may be a superset of several
other Observation Points. For example, one Observation Point can be
Claise, et. al Standard Track [Page 6]
IPFIX Protocol Specification December 2004
an entire line card. This would be the superset of the
individual Observation Points at the line card's interfaces.
Observation Domain
An Observation Domain is the largest set of Observation Points for
which Flow information can be aggregated by a Metering Process.
Each Observation Domain presents itself using a unique ID to the
Collecting Process to identify the IPFIX Messages it generates. For
example, a router line card may be an observation domain if it is
composed of several interfaces: each of which is an Observation
Point. Every Observation Point is associated with an Observation
Domain.
IP Traffic Flow or Flow
There are several definitions of the term 'flow' being used by the
Internet community. Within the context of IPFIX we use the
following definition:
A Flow is defined as a set of IP packets passing an Observation
Point in the network during a certain time interval. All packets
belonging to a particular Flow have a set of common properties.
Each property is defined as the result of applying a function to the
values of:
1. one or more packet header field (e.g. destination IP address),
transport header field (e.g. destination port number), or
application header field (e.g. RTP header fields [RFC1889])
2. one or more characteristics of the packet itself (e.g. number
of MPLS labels, etc...)
3. one or more of fields derived from packet treatment (e.g. next
hop IP address, the output interface, etc...)
A packet is defined to belong to a Flow if it completely satisfies
all the defined properties of the Flow.
This definition covers the range from a Flow containing all packets
observed at a network interface to a Flow consisting of just a
single packet between two applications. It includes packets
selected by a sampling mechanism.
Claise, et. al Standard Track [Page 7]
IPFIX Protocol Specification December 2004
Flow Key
Each of the fields which
1. Belong to the packet header (e.g. destination IP address)
2. Are a property of the packet itself (e.g. packet length)
3. Are derived from packet treatment (e.g. AS number)
and which are used to define a Flow are termed Flow Keys.
Flow Record
A Flow Record contains information about a specific Flow that was
observed at an Observation Point. A Flow Record contains measured
properties of the Flow (e.g. the total number of bytes of all
packets of the Flow) and usually characteristic properties of the
Flow (e.g. source IP address).
Metering Process
The Metering Process generates Flow Records. Input to the process
are packet headers observed at an Observation Point and packet
treatment at the Observation Point (for example the selected output
interface).
The Metering Process consists of a set of functions that includes
packet header capturing, timestamping, sampling, classifying, and
maintaining Flow Records.
The maintenance of Flow Records may include creating new records,
updating existing ones, computing Flow statistics, deriving further
Flow properties, detecting Flow expiration, passing Flow Records to
the Exporting Process, and deleting Flow Records.
Exporting Process
The Exporting Process sends Flow Records to one or more Collecting
Processes. The Flow Records are generated by one or more Metering
Processes.
Claise, et. al Standard Track [Page 8]
IPFIX Protocol Specification December 2004
IPFIX Device
An IPFIX Device hosts at least one Observation Point, a Metering
Process and an Exporting Process. Typically, corresponding
Observation Point(s), Metering Process(es) and Exporting Process(es)
are co-located at such a device, for example at a router.
Exporter
A device which hosts one or more Exporting Processes is termed an
Exporter.
Collecting Process
A Collecting Process receives Flow Records from one or more
Exporting Processes. The Collecting Process might process or store
received Flow Records, but such actions are out of scope for this
document.
Collector
A device which hosts one or more Collecting Processes is termed a
Collector.
Template
Template is an ordered sequence of pairs (<type,length>), used to
completely identify the structure and semantics of a particular
information that needs to be communicated from the IPFIX Device to
the Collector. Each Template is uniquely identifiable by means of a
Template ID.
IPFIX Message
An IPFIX Message is a message originating at the Exporting Process
that carries the IPFIX records of this Exporting Process and whose
destination is the Collecting Process. An IPFIX Message is
encapsulated within a transport layer.
Message Header
Claise, et. al Standard Track [Page 9]
IPFIX Protocol Specification December 2004
The Message Header is the first part of an IPFIX Message, which
provides basic information about the message such as the IPFIX
version, length of the message, message sequence number, etc.
Template Record
A Template Record defines the structure and interpretation of fields
in a Flow Data Record.
Flow Data Record
A Flow Data Record is a data record that contains values of the Flow
parameters corresponding to a Template Record.
Options Template Record
An Options Template Record defines the structure and interpretation
of fields in an Options Data Record, including defining how to scope
the applicability of the Options Data Record.
Options Data Record
The Options Data Record is a data record that contains values and
scope information of the Flow measurement parameters, corresponding
to an Options Template Record.
Set
Set is a generic term for a collection of records that have a
similar structure. In an IPFIX Message, one or more Sets follow the
Message Header.
There are three different types of Sets: Template Set, Options
Template Set, and Data Set.
Template Set
A Template Set is a collection of one or more Template Records that
have been grouped together in an IPFIX Message.
Options Template Set
Claise, et. al Standard Track [Page 10]
IPFIX Protocol Specification December 2004
An Options Template Set is a collection of one or more Options
Template Records that have been grouped together in an IPFIX
Message.
Data Set
A Data Set is one or more records, of the same type, that are
grouped together in an IPFIX Message. Each record is either a Flow
Data Record or an Options Data Record previously defined by a
Template Record or an Options Template Record.
Information Element
An Information Element is a protocol and encoding independent
description of an attribute which may appear in an IPFIX Flow
Record. The IPFIX information model [IPFIX-INFO] defines the base
set of Information Elements for IPFIX. The type associated with an
Information Element indicates constraints on what it may contain and
also determine the valid encoding mechanisms for use in IPFIX.
3.1 Terminology Summary Table
+------------------+---------------------------------------------+
| | Contents |
| +--------------------+------------------------+
| Set | Template Record | data record |
+------------------+--------------------+------------------------+
| | | Flow Data Record(s) |
| Data Set | / | or |
| | | Options Data Record(s) |
+------------------+--------------------+------------------------+
| Template Set | Template Record(s) | / |
+------------------+--------------------+------------------------+
| Options Template | Options Template | / |
| Set | Record(s) | |
+------------------+--------------------+------------------------+
Figure A: Terminology Summary Table
A Data Set is composed of an Options Data Record(s) or Flow Data
Record(s). No Template Record is included. A Template Record
defines the Flow Data Record, and an Options Template Record defines
the Options Data Record.
Claise, et. al Standard Track [Page 11]
IPFIX Protocol Specification December 2004
A Template Set is composed of Template Record(s). No Flow or
Options Data Record is included.
An Options Template Set is composed of Options Template Record(s).
No Flow or Options Data Record is included.
4. Criteria for Flow Expiration and Export
4.1 Flow Expiration
A Flow is considered as expired under the following conditions:
1. If the Metering Process can deduce the end of a Flow, that
Flow Record should be exported when the end of the Flow is
detected. For example, a Flow generated by TCP [TCP] traffic where
the FIN or RST bits indicate the end of the Flow Record.
2. If no packets belonging to the Flow have been observed for a
certain period of time. This time period should be configurable at
the Metering Process, with a minimum value of 0 seconds for
immediate expiration. Note that a zero timeout would report a Flow
as a sequence of single-packet Flows.
3. If the IPFIX Device experiences resource constraints, a Flow
Record may be prematurely expired (e.g. lack of memory to store Flow
Records).
4. For long-running Flows, the Metering Process should expire the
Flow Record on a regular basis or based on some expiration policy.
This periodicity or expiration policy should be configurable at the
Metering Process. When the Record of a long-running Flow is expired,
that Flow Record may still be maintained by the Metering Process so
that, for further observed packets of the same Flow Record, the
Metering Process does not need to create a new Flow Record.
4.2 Flow Export
The Exporting Process decides when and whether to export an expired
Flow. A Flow can be exported because it expired due to the reasons
mentioned in Flow Expiration section. For example: the Exporting
Process exports a portion of the expired Flows every 'x' seconds.
Claise, et. al Standard Track [Page 12]
IPFIX Protocol Specification December 2004
For long-lasting Flows, the Exporting Process should export the Flow
Records on a regular basis or based on some export policy. This
periodicity or export policy should be configurable at the Metering
Process.
5. Message Format
An IPFIX Message consists of a Message Header followed by one or
more Sets. The Sets can be any of the possible three types:
Template, Data, or Options Template.
The format of the IPFIX Message is shown in Figure B.
+--------+-------------------------------------------+
| | +----------+ +---------+ +----------+ |
|Message | | Template | | Data | | Options | |
| Header | | Set | | Set | | Template | ... |
| | | | | | | Set | |
| | +----------+ +---------+ +----------+ |
+--------+-------------------------------------------+
Figure B: IPFIX Message format
A Set ID is used to distinguish the different types of Sets. Set
IDs lower than 256 are reserved for special Sets, such as the
Template Set (ID 2) and the Options Template Set (ID 3). The Data
Sets have a Set ID greater than 255. The Set ID value of 0 and 1
are not used for historical reasons [NETFLOW9].
The format of the Template, Data, and Options Template Sets will be
discussed later in this document. The Exporter MUST code all binary
integers of the Message Header and the different Sets in network
byte order (also known as the big-endian byte ordering).
Following are some examples of IPFIX Messages:
1. An IPFIX Message consisting of interleaved Template, Data, and
Options Template Sets-A newly created Template is exported as soon
as possible. So if there is already an IPFIX Message with a Data
Set that is being prepared for export, the Template and Option Sets
are also interleaved with this information, subject to availability
of space.
Claise, et. al Standard Track [Page 13]
IPFIX Protocol Specification December 2004
+--------+--------------------------------------------------------+
| | +----------+ +---------+ +-----------+ +---------+ |
|Message | | Template | | Data | | Options | | Data | |
| Header | | Set | | Set | ... | Template | | Set | |
| | | | | | | Set | | | |
| | +----------+ +---------+ +-----------+ +---------+ |
+--------+--------------------------------------------------------+
Figure C: IPFIX Message example 1
2. An IPFIX Message consisting entirely of Data Sets-After the
appropriate Template Records have been defined and transmitted to
the Collecting Process, the majority of IPFIX Messages consist
solely of Data Sets.
+--------+----------------------------------------------+
| | +---------+ +---------+ +---------+ |
|Message | | Data | ... | Data | ... | Data | |
| Header | | Set | ... | Set | ... | Set | |
| | +---------+ +---------+ +---------+ |
+--------+----------------------------------------------+
Figure D: IPFIX Message example 2
3. An IPFIX Message consisting entirely of Template and Options
Template Sets-When UDP is used as the transport protocol, Templates
Sets and Option Template Sets MUST be sent periodically to help
ensure that the Collecting Process has the correct Template Records
and Options Template Records when the corresponding Flow Data
Records are received.
+--------+-------------------------------------------------+
| | +----------+ +----------+ +----------+ |
|Message | | Template | | Template | | Options | |
| Header | | Set | ... | Set | ... | Template | |
| | | | | | | Set | |
| | +----------+ +----------+ +----------+ |
+--------+-------------------------------------------------+
Figure E: IPFIX Message example 3
Claise, et. al Standard Track [Page 14]
IPFIX Protocol Specification December 2004
6. IPFIX Message Format
6.1 Header Format
The format of the IPFIX Message Header format is shown in Figure F.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version Number | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Export Time |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure F: IPFIX Message Header format
Message Header Field Descriptions
Version
Version of Flow Record format exported in this message. The
value of this field is 0x000a for the current version.
Length
Total Length is the length of the IPFIX Message, measured in
octets, including message Header and Set(s).
Export Time
Time in seconds since 0000 UTC 1970, at which the IPFIX
Message Header leaves the Exporter.
Sequence Number
Incremental sequence counter modulo 2exp32 of all IPFIX
Messages sent on this stream from the current Observation
Domain by the Exporting Process. This value SHOULD be used
by the Collecting Process to identify whether any IPFIX
Messages have been missed.
Source ID
Claise, et. al Standard Track [Page 15]
IPFIX Protocol Specification December 2004
A 32-bit value that identifies the Exporter Process
Observation Domain. Collecting Process SHOULD use the
combination of the source IP address and the Source ID field
to separate different export streams originating from the
same Exporting Process.
6.2 Field Type Format
Vendors need the ability to define proprietary Information Elements,
because, for example, they are delivering pre-standards product, or
the Information Element is in some way commercially sensitive. This
section describes the Field Type format for both IETF specified
Information Elements [IPFIX-INFO] and Enterprise Specific
Information Elements, both the Template Set and the Option Template
Set.
The Field Ids used to identify Information Elements are represented
by the Field Type. When the Enterprise Field Type bit is set to 0,
the corresponding Field Type will report an IETF specified
Information Elements. When the Enterprise Field Type bit is set to
1, the corresponding Field Type will report an Enterprise Specific
Information Element. An example of this is shown in section 16.
The Field Type format is shown in Figure G.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|E| Field Type | Field Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure G: Field Type format
Where:
E
Enterprise Field Type. This is the first bit of the Field
Type. If this bit is zero, the Field Type identifies an IETF
specified Information Element, and the four octet Enterprise
Number field MUST NOT be present. If this bit is one, the
Claise, et. al Standard Track [Page 16]
IPFIX Protocol Specification December 2004
Field Type identifies an Enterprise Specific Information
Element, and the Enterprise Number filed MUST be present.
Field Type
A numeric value that represents the type of the field. Refer
to [IPFIX-INFO].
Field Length
The length of the corresponding Field Type, in bytes. Refer
to [IPFIX-INFO].
Enterprise Number
IANA enterprise number [PEN] of the authority defining the
Field Type in this Template Record.
6.3 Template Set Format
One of the essential elements in the IPFIX format is the Template
Set. Templates greatly enhance the flexibility of the Flow Record
format because they allow the Collecting Process to process Flow
Records without necessarily knowing the interpretation of all the
data in the Flow Record. A Template Set MAY exclusively contain
IETF defined Field Types. A Template Set MAY contain Enterprise
Specific Information Elements from one or more vendors.
The format of the Template Set is shown in Figure H.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 2 | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 256 | Field Count 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Type 1.1 | Field Length 1.1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number 1.1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Type 1.2 | Field Length 1.2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Type 1.N | Field Length 1.N |
Claise, et. al Standard Track [Page 17]
IPFIX Protocol Specification December 2004
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number 1.N |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 257 | Field Count 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Type 2.1 | Field Length 2.1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Type 2.2 | Field Length 2.2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number 2.2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Type 2.M | Field Length 2.M |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number 2.M |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Padding (opt) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure H: Template Set Format
Field Types 1.2 and 2.1 are defined by the IETF (bit 0 = 0) and
therefore do not need and Enterprise Number to identify them.
The Template Set Field Definitions are as follows:
Set ID
Set ID value of 2 is reserved for the Template Set.
Length
Total length of this Set. Because an individual Template
Set MAY contain multiple Template Records, the Length value
MUST be used to determine the position of the next Set
record, which could be any type of Set. Length is the sum
of the lengths of the Set ID, the Length itself, and all
Template Records within this Set.
Template ID
Each of the newly generated Template Records is given a
unique Template ID. This uniqueness is local to the
Observation Domain that generated the Template ID.
Claise, et. al Standard Track [Page 18]
IPFIX Protocol Specification December 2004
Template IDs 0-255 are reserved for Template Sets, Options
Sets, and other reserved Sets yet to be created. Template
IDs of Data Sets are numbered from 256 to 65535.
Field Count
Number of fields in this Template Record. Because a
Template Set usually contains multiple Template Records,
this field allows the Collecting Process to determine the
end of the current Template Record and the start of the
next.
Field Type
A numeric value that represents the type of the field.
Refer to [IPFIX-INFO].
Field Length
The length of the corresponding Field Type, in bytes. Refer
to [IPFIX-INFO].
Enterprise Number
IANA enterprise number [PEN] of the authority defining the
Field Type.
Padding
The Exporting Process MAY insert some padding bytes, so that
the subsequent Set starts at an aligned boundary. Padding
MUST be composed of zero (0) bytes. The padding length MUST
be shorter than any allowable Template Record in this
Template Set. It is important to note that the Length field
includes the padding bytes. Because Template Sets are
always 4-byte aligned by definition padding is only needed
in case of other alignments e.g. on 8-byte boundaries.
The Set ID value of 0 and 1 are not used for historical reasons
[NETFLOW9].
6.4 Data Set Format
The format of the Data Set is shown in Figure I.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Claise, et. al Standard Track [Page 19]
IPFIX Protocol Specification December 2004
| Set ID = Template ID | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 1 - Field Value 1 | Record 1 - Field Value 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 1 - Field Value 3 | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 2 - Field Value 1 | Record 2 - Field Value 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 2 - Field Value 3 | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 3 - Field Value 1 | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | Padding (opt) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure I: Data Set Format
Note that not all Field Values do necessarily have a length of 16
bit.
Data Set Field Descriptions are as follows:
Set ID = Template ID
Each Data Set is associated with a Set ID. The Set ID maps
to a (previously generated) Template ID. The Collecting
Process MUST use the Set ID to find the corresponding
Template Record and decode the Flow Records from the Set.
Length
The length of this Set.
Length is the sum total of lengths of Set ID, Length itself,
all Flow Records within this Set, and the padding bytes, if
any.
Record N - Field Value M
The remainder of the Data Set is a collection of Flow Data
Record(s), each containing a set of Field Types and values.
The Type and Length of the fields have been previously
defined in the Template Record referenced by the Set ID or
Template ID.
Padding
Claise, et. al Standard Track [Page 20]
IPFIX Protocol Specification December 2004
The Exporting Process MAY insert some padding bytes, so that
the subsequent Set starts at an aligned boundary. Padding
MUST be composed of zero (0) bytes. The padding length MUST
be shorter than any allowable Flow Data Record in this Data
Set. It is important to note that the Length field includes
the padding bytes.
Interpretation of the Data Set format can be done only if the
Template Set corresponding to the Template ID is available at the
Collecting Process.
6.5 Options Template Set
The Options Template Record (and its corresponding Options Data
Record) is used to supply information about the Metering Process
configuration or Metering Process specific data, rather than
supplying information about IP Flows.
For example, the Options Template Set can report the sample rate of
a specific interface, if sampling is supported, along with the
sampling method used.
6.5.1 Scope
The Options Template Set gives the Exporter the ability to provide
additional information to the Collector which would not be possible
with only Flow Records. The scope, which is only available in the
Options Template Set, gives the context of the reported Information
Elements. One Options Template Set example is the "Metering Process
statistics", which reports the statistics for the Observation
Domain, which is defined as the scope. Another example is the
"Template configuration", which reports the configuration sampling
parameter(s) for the template, which is defined as the scope.
Multiple scope fields MAY be present in the Options Template Set, in
which, the composite scope is the combination of the scopes. For
example, if the two scopes are defined as "cache" and "template",
the combined scope is this template in this cache. The order of the
scope, as defined in the Options Template Set, is in this case
irrelevant. However, if the order of the scopes fields in the Option
Template Set is relevant, the order of the scope fields MUST be
used. For example, if the first scope defines the filtering
function, while the second scope defines the sampling function, the
Claise, et. al Standard Track [Page 21]
IPFIX Protocol Specification December 2004
order of the scope is important. Applying first the sampling
function, followed by the filtering function, would lead to
potential different Flow Records than applying first the filtering
function, followed by the filtering function. In this case, the
Collector deduces the function order by looking at the order of the
scope in the Options Template Set.
Finally, note that the scope length MAY NOT be zero.
6.5.2 Options Template Set Format
An Options Template MAY exclusively contain IETF defined Field
Types. An Options Template MAY contain Enterprise Specific
Information Elements from multiple vendors. An Options Template MAY
contain IETF defined Field Types and Enterprise Specific Information
Elements.
The format of the Options Template Set is shown in Figure J.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID | Field Count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count | Scope 1 Field Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope 1 Field Length | Scope 2 Field Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope 2 Field Length | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | Scope N Field Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope N Field Length | Scope N Entreprise Number ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
... Scope N Enterprise Number | Option 1 Field Type |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option 1 Field Length | Option 1 Entreprise Number ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
... Option 1 Enterprise Number | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | Option M Field Type |
Claise, et. al Standard Track [Page 22]
IPFIX Protocol Specification December 2004
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option M Field Length | Padding (opt) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure J: Option Template Set Format
The Options Template Set Field Definitions are as follows:
Set ID = 3
A Set ID value of 3 is reserved for the Options Template.
Length
Total length of this Set, including the padding bytes, if
any. Each Options Template Set MAY contain multiple Options
Template Records. Thus, the Length value MUST be used to
determine the position of the next Set record, which could
be either a Template Set or Data Set.
Length is the sum total of lengths of Set ID, the Length
itself, and all Options Template Records within this Set
Template ID.
Template ID
Template ID of this Options Template. This value is greater
than 255.
Field Count
Number of all fields in this Option Template Record,
including the Scope Fields. Because a Option Template Set
usually contains multiple Template Records, this field
allows the Collecting Process to determine the end of the
current Option Template Record and the start of the next.
Scope Field Count
Number of scope fields in this Option Template Record.
Scope Field Type
A numeric value that represents the type of the field.
Refer to [IPFIX-INFO].
Scope Field Length
The length (in bytes) of the Scope field, as it would appear
in an Options Data Record.
Claise, et. al Standard Track [Page 23]
IPFIX Protocol Specification December 2004
Scope N Enterprise Number
IANA enterprise number [PEN] of the authority defining
Scope N Field Type. This is 4 bytes long.
Option Field Type
A numeric value that represents the type of field. Refer to
[IPFIX-INFO].
Option Field Length
The length of the corresponding Option Field Type, in bytes.
Refer to [IPFIX-INFO].
Option M Enterprise Number
IANA enterprise number [PEN] of the authority defining the
Option M Field Type. This is 4 bytes long.
Padding
The Exporting Process MAY insert some padding bytes, so that
the subsequent Set starts at an aligned boundary. Padding
MUST be composed of zero (0) bytes. The padding length MUST
be shorter than any allowable Options Template Record in
this Options Template Set. It is important to note that the
Length field includes the padding bytes.
The Set ID value of 0 and 1 are not used for historical reasons
[NETFLOW9].
6.5.3 Options Data Record Format
The Options Data Records are sent in Data Sets.
The format of the Data Set, containing Options Data Records, is
shown in Figure K.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = Template ID | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 1 - Scope 1 Value | Record 1 - Scope 2 Value |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |Record 1 - Option Field 1 Value|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Claise, et. al Standard Track [Page 24]
IPFIX Protocol Specification December 2004
|Record 1 - Option Field 2 Value| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 2 - Scope 1 Value | Record 2 - Scope 2 Value |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |Record 2 - Option Field 1 Value|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Record 2 - Option Field 2 Value| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 3 - Scope 1 Value | Record 3 - Scope 2 Value |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |Record 3 - Option Field 1 Value|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Record 3 - Option Field 2 Value| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | Padding (opt) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure K: Data Set format, containing Options Data Records
Options Data Records of the Data Set Field Descriptions
Set ID = Template ID
A Set ID precedes each group of Options Data Records within
a Data Set. The Set ID maps to a previously generated
Template ID corresponding to this Options Template Record.
The Collecting Process MUST use the Set ID to map the
appropriate type and length to any field values that follow.
Length
The length of this Set.
Length is the sum of the lengths of the Set ID, Length
itself, all the Options Data Records within this Set, and
the padding bytes, if any.
Record N - Option Field M Value
The remainder of the Data Set is a collection of Flow
Records, each containing a set of Scope and Field Values.
The type and length of the fields were previously defined in
the Options Template Record referenced by the Set ID or
Template ID.
Padding
Claise, et. al Standard Track [Page 25]
IPFIX Protocol Specification December 2004
The Exporting Process MAY insert some padding bytes, so that
the subsequent Set starts at an aligned boundary. Padding
MUST be composed of zero (0) bytes. The padding length MUST
be shorter than any allowable Options Data Record in this
Data Set. It is important to note that the Length field
includes the padding bytes.
The Data Set format can be interpreted only if the Options Template
Set corresponding to the Template ID is available at the Collecting
Process.
7. Specific Reporting Requirements
Some specific Options Templates and Options Templates Records are
necessary to provide extra information about the Flow Records and
about the Metering Process.
The ipfixOption Field [IPFIX-INFO], always included in these
specific Options Templates, defines the type of information sent in
the Option Template / Option Template Record pair. For example, if
the ipfixOption [IPFIX-INFO] value is METER_STATS, then the Option
Template will specify information about the Metering Process
statistics.
The Option Template and Option Template Records defined in these
sub-sections are not mandatory to implement as they impose some
constraints about the Metering Process implementation: this document
specifies the protocol to export the records, not the Metering
Process implementation. However, if the specific Option Templates
are implemented, they should ideally be implemented as specified in
these sub-sections. In any case, if the ipfixOption Information
Element is present, it MUST always be the first Information Element
in the Option Template so that the Collector can quickly determine
which specific Option Template Record is received.
The minimum set of Information Elements is always specified in these
Specific IPFIX Options Templates. Nevertheless, extra Information
Elements may be used in these specific Options Templates.
7.1 The Metering Process Statistics Option Template
Claise, et. al Standard Track [Page 26]
IPFIX Protocol Specification December 2004
The Metering Process Statistics Option Template specifies the
Metering Process Statistics. It contains the following Information
Elements [IPFIX-INFO]:
ipfixOption The value is METERING_STATS
exportedOctetCount The number of all octets reported
by the Exporting Process to the
Collecting Process.
exportedPacketCount The number of all packets reported
by the exporting process to the
Collecting Process.
exportedFlowCount The number of all flows records
reported by the Exporting Process
to the Collecting Process.
time The time at which the record was
generated
The Exporting Process should export the Metering Process Statistics
Option Template Record on a regular basis or based on some export
policy. This periodicity or export policy should be configurable.
The Metering Process Statistics Option Template could be extended
with other Information Elements.
The Scope Field specified in the Metering Process Statistics Option
Template Record is the Source ID.
7.2 The Metering Process Reliability Statistics Option Template
The Metering Process Reliability Option Template specifies
information about lack of reliability in the Metering process. It
contains the following Information Elements [IPFIX-INFO]:
ipfixOption The value is METERING_RELIABILITY_STATS
droppedFUPacketCount Packets dropped by Metering Process
droppedFUOctetCount Bytes dropped by Metering Process
timeFirstFUDropped Time of the first packet dropped at the
Specified scope ID
timeLastFUDropped Time of the last packet dropped at the
Specified scope ID
time The time at which the record was
generated
The Exporting Process should export the Metering Process Reliability
Statistics Option Template Record on a regular basis or based on
Claise, et. al Standard Track [Page 27]
IPFIX Protocol Specification December 2004
some export policy. This periodicity or export policy should be
configurable. The Metering Process Reliability Statistics Option
Template could be extended with other Information Elements.
The Scope Field specified in the Metering Process Reliability
Statistics Option Template Record is the Source ID.
7.3 The Exporting Process Reliability Statistics Option Template
The Exporting Process Reliability Option Template specifies
information about lack of reliability in the Exporting process. It
contains the following Information Elements [IPFIX-INFO]:
ipfixOption The value is
EXPORTING_RELIABILITY_STATS
droppedFlows Number of flow records not exported
(due to resources starvation at
Exporting Process or due to some
flow records export policies)
droppedFAPacketCount Packets in the dropped flows
droppedFAByteCount Bytes in the dropped flows
timeFirstFADropped Time of the first packet within the
dropped flows
timeLastFADropped Time of the last packet within the
dropped flows
time The time at which the record was
generated
The Exporting Process should export the Exporting Process
Reliability Statistics Option Template Record on a regular basis or
based on some export policy. This periodicity or export policy
should be configurable. The Exporting Process Reliability Statistics
Option Template could be extended with other Information Elements.
The Scope Field specified in the Exporter Reliability Statistics
Option Template Record is the Exporter ID.
7.4 The Flow Keys Option Template
The Flow Keys Option Template specifies the flow keys used by the
Metering Process for the Template ID definition. It contains the
following Information Elements [IPFIX-INFO]:
Claise, et. al Standard Track [Page 28]
IPFIX Protocol Specification December 2004
ipfixOption The value is FLOW_KEY
keyList Bitmap with the positions of the flow
keys in the template
time The time at which the record was
generated
The Scope Field specified in the Flow Keys Option Template Record is
the Template ID with which the flow keys are associated.
8. "Export Time" Computation and Flow Record Time
8.1 Microsecond Precision
For a Data Set with Flow Records requiring microsecond precision,
the IPFIX Message Header "Export Time" field MUST be calculated so
that each Flow Records flowStartUsec [IPFIX-INFO] and flowEndUsec
[IPFIX-INFO] would contain a 32 bit signed microsecond offset from
the "Export Time" base timestamp. Hereafter some pseudo code to
calculate the Export Time in one pass, which would return an
absolute duration of 35 minutes for all Flow Records contained in
the Data Set. Flow Records MUST be exported in different IPFIX
Message Header if the absolute durations can not fit in those 35
minutes.
// pseudo code for microsecond offset in IPFIX encoded Flow
Records.
//
struct flow{
uint32 tv_sec;
uint32 tv_usec;
uint32 numbytes;
... // other Information Elements...
};
struct flow flowtable [MAX_TABLE_SIZE];
int lastflowindex = -1;
writeflows() {
if (lastflowindex < 0) return;
// simply take the second field from the first available flow
// and make this the base time for this collection of flows.
uint32 base_sec = flowtable[0].tv_sec;
Claise, et. al Standard Track [Page 29]
IPFIX Protocol Specification December 2004
writeheaderToSocket(base_sec); // put 32-bit second value in
header
for (int i=0; i<=lastflowindex; i++){
int32 offset = (flowtable[i].tv_sec - base_sec) * 1000000
+ flowtable[i].tv_usec;
writeint32ToSocket(offset); // put the 32-bit time offset
in the record.
// write other Information Elements...
}
}
A two pass approach calculation for the optimum (center) "Export
Time" base timestamp would allow an absolute duration of 71 minutes
for all Flow Records contained in the Data Set. The two pass
approach MAY be used. The "Export Time" base timestamp calculation
requires that at the IPFIX Message Header exporting time the
Exporting Process MUST run down the list of Flow Records in the Data
Set message and adjust the Flow start and Flow end timestamps.
8.2 Millisecond Precision
For a Data Set with Flow Records requiring a millisecond precision,
the same principles as in section 9.1 "Microsecond Precision" will
be used.
The only difference will be that the Flow start and the Flow end
SHOULD now be represented respectively by the flowStartMsec [IPFIX-
INFO] and flowEndMsec [IPFIX-INFO]. As a consequence of the
millisecond precision, the absolute duration of all Flow Records is
now of about 49 days. The Export header "Export Time" base time
SHOULD be calculated with the algorithm described in the Section 9.1
"Microsecond Precision". In order to reduce the load on the
Exporter, the Export header "Export Time" MAY be the time in seconds
since 0000 UTC 1970 at which the IPFIX Message Header leaves the
Exporter and not the calculated optimum value anymore as described
in section 9.1 "Microsecond Precision".
Alternatively, for a Data Set with Flow Records requiring a
millisecond precision, the microsecond mechanism as described in
section 9.1 MAY be used as such. The Flow Record MAY use the
flowStartUsec [IPFIX-INFO] and flowEndUsec [IPFIX-INFO] rounded at a
millisecond precision.
Claise, et. al Standard Track [Page 30]
IPFIX Protocol Specification December 2004
8.3 Nanosecond Precision
For a Data Set with Flow Records requiring a nanosecond precision,
all Flow Records will contain Flow start flowStartNsec [IPFIX-INFO]
and flowEndNsec [IPFIX-INFO]. The Export header "Export Time" will
be of no use on the Collector side in this case as the flowStartNsec
[IPFIX-INFO] and flowEndNsec [IPFIX-INFO] both have a nanosecond
precision already. Both flowStartNsec [IPFIX-INFO] and flowEndNsec
[IPFIX-INFO] use the NTP time format which is represented as a 64-
bit value which contains a 32-bit specification of seconds since
1900 and a 32-bit "fraction" field. Refer to the NTP
specification, RFC1305, section 3.1 "Data Formats".
8.4 Multiple Precisions
When Flow Records requiring different precisions must be exported,
the Exporting Process SHOULD split the Flow Records in different
Data Set according to the precision: millisecond, microsecond or
nanosecond.
9. Linkage with the Information Model
The Information Elements [IPFIX-INFO] MUST be sent in canonical
format in network byte order.
9.1 Reduced Size Encoding of Integer Types
Information Elements containing integer types in the information
model MAY be encoded using fewer bytes than those implied by their
type in the information model definition [IPFIX-INFO], based on the
assumption that the smaller type is sufficient to carry any value
the Exporter may need to deliver. This reduces the network
bandwidth requirement between the Exporter and the Collector. Note
that the Information Elements definition [IPFIX-INFO] will always
define the maximum encoding size.
For instance the information model [IPFIX-INFO] defines byteCount as
an unsignedLong type, which would require 64-bits. However if the
Exporter will never locally encounter the need to send a value
larger than 4294967295, it may chose to send the value instead as an
unsignedInt. For example, a core router would require an
unsignedLong byteCount while an unsignedInt might be sufficient for
an access router.
Claise, et. al Standard Track [Page 31]
IPFIX Protocol Specification December 2004
This behavior is indicated by the Exporter by specifying a type size
smaller than that associated with the assigned type of the field.
In the example above the Exporter would place a length of 4 versus 8
in the template.
If reduced sizing is used, it MUST be applied only to following
integer types: unsignedLong, long, unsignedInt, int, unsignedShort,
short. In each case the downcasting MUST be to a smaller integer
type. The same signed versus unsigned properties MUST be preserved.
Specifically unsignedLong may be downcast to unsignedInt,
unsignedShort or unsignedByte. A long may be downcast to an int, a
short or a byte. The other downcasts follow the same pattern.
10. Variable Length Information Element
The IPFIX template mechanism is optimized for fixed length
Information Elements [IPFIX-INFO]. Where an Information Element has
a variable length the following mechanism MUST used to carry the
length information, for both the IETF and proprietary Information
Elements.
In the Template Set the length is recorded as 65535. This reserved
length value notifies the Collecting Process that length of the
Information Element will be carried in the Information Element
content itself.
In most cases the length of the Information Element will be less
than 256 bytes. The following length encoding mechanism optimizes
the overhead of carrying the Information Element length in this
majority case.
If the length of the Information Element is less than 255 bytes, the
length is carried in the first byte of the Information Element, as
shown on Figure L.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Length (< 255)| Information element |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... continuing as needed |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure L: Variable Length Information Element (length < 255 bytes)
Claise, et. al Standard Track [Page 32]
IPFIX Protocol Specification December 2004
If the length of the Information Element is greater or equal than
255 bytes, the first byte of the Information Element is 255, and the
length is carried in the second and third bytes of the Information
Element, as shown in Figure M.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 255 | Length (255 to 65535) | IE |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... continuing as needed |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure M: Variable Length Information Element
(length 255 to 65535) bytes
11. Template Management
This section describes Template management when using SCTP and SCTP-
PR as the transport protocol. Any necessary changes to Template
management specifically related to TCP or UDP transport protocols
are specified in section 13.
The Exporting Process assigns and maintains the Template IDs for the
ExporterÆs Observations Domains. A newly created Template Record is
assigned an unused Template ID by the Exporting Process.
Templates Sets and Option Template Sets MUST be only sent once on
SCTP stream zero with full reliability. As such, the Collecting
Process MUST store the Template Record information for the duration
of the association so that it can interpret the corresponding Flow
Data Records that are received in subsequent Data Sets.
New Template Records SHOULD be transmitted as soon as they are
created. The Exporting Process MAY transmit the Template Set and
Options Template in advance of any Data Sets that use that (Options)
Template ID, to ensure that the Collector has the Template Record
before receiving the first Flow or Options Data Record. Flow and
Options Data Records that correspond to a Template Record MAY appear
in the same and/or subsequent IPFIX Message(s).
Claise, et. al Standard Track [Page 33]
IPFIX Protocol Specification December 2004
A Template ID MUST be unique per Observation Domain. Different
Observation Domains from the same Exporter may use the same Template
ID value to refer to different Templates.
Disused Templates SHOULD be deleted. Prior to reuse a Template ID
the disused Template MUST be deleted. In order to delete an
allocated Template, the Template is withdrawn through the use of a
Template Withdraw Message.
The Template Withdraw Message MUST not be sent until sufficient time
has elapsed to allow the Collecting Process to receive and process
the last data record using this Template information.
The Template ID from a withdrawn Template MUST NOT be reused until
sufficient time has elapsed to allow for the Collecting Process to
receive and process the Template withdraw message.
A Template Withdraw Message is Template Record for that Template ID
with a Field Count of 0. The format of the Template Withdrawal
Message is shown in figure O.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = (2 or 3) | Length = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID | Field Count = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure O: Template Withdrawal Message format
The Set ID field MUST contain the value 2 for Template Set
withdrawal, and the value 3 for Options Template Set. Multiple
Template ID MAY be withdrawn with a single Template Withdrawal
Message: in that case, padding MAY be used.
The Template Withdraw Message withdraws the Template ID for the
Source ID specified in the IPFIX Message header.
If the measurement parameters change, the Template MUST be withdrawn
using a Template Withdraw Message or an unused Template ID MUST be
used. Examples of the measurement changes are: a new sampling rate,
a new flow expiration process, a new filtering definition, etc. If a
Claise, et. al Standard Track [Page 34]
IPFIX Protocol Specification December 2004
Template is changed, a Template Withdraw Message MUST be sent to
delete the Template.
If the Exporting Process restarts, the SCTP association MUST be
shutdown and restarted. When the Exporting Process restarts, all
Template assignments are lost and Template IDs MUST be re-assigned.
If the Metering Process restarts, the Exporting Process MUST either
reuse the previously assigned Template ID for each Template, or it
MUST withdraw the previously issued Template IDs by sending Template
Withdraw Message(s) before reusing them. A Template Withdrawal
Message to withdraw all Data Templates for the Source ID specified
in the IPFIX message header MAY be used. Its format is shown in
figure P. A Template Withdrawal Message to withdraw all Options
Templates for the Source ID specified in the IPFIX message header
MAY be used. Its format is shown in figure Q.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 2 | Length = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 2 | Field Count = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure P: All Data Templates Withdrawal Message format
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 3 | Field Count = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure Q: All Options Templates Withdrawal Message format
When the SCTP association restarts, the Exporting Process MUST
resend all the Template Records.
The Exporting Process MUST NOT transmit IPFIX Messages with more
than one Source ID value inside any single stream.
More that one (Option) Template Set MAY be sent in an IPFIX Message.
Claise, et. al Standard Track [Page 35]
IPFIX Protocol Specification December 2004
12. The Collecting Process's Side
This section describes the Collecting Process when using SCTP and
SCTP-PR as the transport protocol. Any necessary changes to the
Collecting Process specifically related to TCP or UDP transport
protocols are specified in section 13.
The Collecting Process SHOULD listen for a new association request
from the Exporting Process. The Exporting Process will request a
number of streams to use for export. A Collecting Process MUST
support at least two inbound streams per association. An Exporting
Process MAY ask for and support more than two streams.
The Collecting Process MUST verify that only one Source ID value is
used inside each stream. If the Collecting Process detects that more
than one Source ID has been received within a stream, it MUST
discard the IPFIX Message, reset the SCTP association, and SHOULD
log the error
If the Collecting Process receives a malformed IPFIX Message, it
MUST reset the SCTP association, discard the IPFIX Message, and
SHOULD log the error.
Templates Sets and Option Template Sets are only sent once. The
Collecting Process MUST store the Template Record information for
the duration of the association so that it can interpret the
corresponding Flow Data Records that are received in subsequent Data
Sets.
Template IDs are unique per Exporting Process and per Observation
Domain. If the Collecting Process receives a Template which has
already been received but which has not previously been withdrawn
(i.e. a Template Record from the same Exporter Observation Domain
with the same Template ID), then the Collecting Process MUST
shutdown the association.
When an SCTP association is closed, the Collecting Process MUST
discard all templates received over that association and stop
decoding IPFIX Messages that use those templates.
The Collecting Process normally receives Template Records from the
Exporting Process, before receiving Flow or Options Data Records.
Claise, et. al Standard Track [Page 36]
IPFIX Protocol Specification December 2004
The Flow Data Records (or Options Data Records) are then decoded and
stored by the Collector. If the Template Records have not been
received at the time Flow Data Records (or Options Data Records) are
received, the Collecting Process MAY store the Flow Data Records (or
Options Data Records) for a short period of time and decode them
after the Template Records are received. A Collecting Process MUST
NOT assume that the Data Set and the associated Template Set (or
Options Template Set) are exported in the same IPFIX Message.
The Collecting Process MUST note the Field ID of any Information
Element that it does not understand and MAY discard that Information
Element from the Flow Record. The Collecting Process MUST note the
size and position of any Vendor Specified Information Element that
it does not understand and discard that Information Element from the
Flow Record.
More that one (Options) Template Set MAY be received in an IPFIX
Message.
The Collector MUST accept padding in Flow Data Records, Options Data
Records and Template Records.
The IPFIX protocol has a Sequence Number field in the Export header
which increases with each IPFIX Message. A Collector may detect out
of sequence, dropped, or duplicate IPFIX Messages by tracking the
Sequence Number. EDITORÆS NOTE: THIS MIGHT CHANGE IF THE SEQUENCE
NUMBER IS PER STREAM AND IF THIS IS THE NUMBER OF FLOW RECORDS. A
collector SHOULD provide a logging mechanism for tracking out of
sequence IPFIX Messages. Such out of sequence IPFIX Messages may be
due to Exporter resource exhaustion where it can not transmit
messages at their creation rate, an Exporting Process reset,
congestion on the network link between the Exporter and Collector,
Collector resource exhaustion where it can not process the IPFIX
Messages at their arrival rate, out of order packet reception,
duplicate packet reception, or an attacker injecting false messages.
If a Collecting Process receives a Template Withdraw Message, the
Collecting Process MUST delete the corresponding Template Record
associated with the specific Exporter and specific Observation
Domain, and stop decoding IPFIX Messages that use those Templates.
A Collecting Process that receives IPFIX Messages from several
Observation Domains from the same Exporter MUST be aware that the
Claise, et. al Standard Track [Page 37]
IPFIX Protocol Specification December 2004
uniqueness of the Template ID is not guaranteed across Observation
Domains.
IPFIX Messages with a Source ID of zero MUST be discarded by the
Collecting Process.
13. Transport Protocol
The IPFIX Protocol Specification has been designed to be transport
protocol independent. Note that the Exporter can export to multiple
Collecting Processes, using independent transport protocols.
The IPFIX Message Header 16-bit LENGTH field limits the length of a
IPFIX Message to 65536 octets including the header. A Collecting
Process MUST be able to handle IPFIX Message lengths of up to 65536
octets.
13.1 Transport Compliance and Transport Usage
We need to differentiate between what must be implemented (so that
operators can interoperably deploy compliant implementations from
different vendors) and what should or could be used in various
operational environments. We must also make sure that ALL
implementations can operate in a congestion-aware and congestion
avoiding mode.
SCTP [RFC2960] and SCTP-PR [RFC3758] MUST be implemented by all
compliant implementations. UDP [UDP] MAY also be implemented by
compliant implementations. TCP [TCP] MAY also be implemented by
compliant implementations.
SCTP-PR SHOULD be used in deployments where Exporters and Collectors
are communicating over links that are susceptible to congestion.
SCTP-PR is capable of providing any required degree of reliability.
TCP MAY be used in deployments where Exporters and Collectors
communicate over links that are susceptible to congestion, but SCTP-
PR is preferred, due to its ability to limit back pressure on
Exporters and its message versus stream orientation.
UDP MAY be used although it is not a congestion aware protocol.
However, the IPFIX traffic between Exporter and Collector MUST
remain wholly within the administrative domains of the operators.
Claise, et. al Standard Track [Page 38]
IPFIX Protocol Specification December 2004
13.2 TCP
This section describes how IPFIX can be transported over TCP [TCP].
13.2.1 Congestion Avoidance
TCP will detect congestion in the end-to-end path between the IPFIX
Exporting Process and the IPFIX Collecting Process, and limit the
transfer rate accordingly. When an IPFIX Exporting Process has
records to export, but detects that transmission by TCP is
temporarily impossible, it can either wait until sending is possible
again, or it can decide to drop the record. In the latter case, the
dropped export data MUST be accounted for, so that the amount of
dropped export data can be reported.
13.2.2 Reliability
TCP provides an intrinsically reliable delivery service from the
IPFIX Exporting Process to the IPFIX Collecting Process.
13.2.3 MTU
TCP provides the required IPFIX Message fragmentation service based
on path MTU discovery.
13.2.4 Exporting Process
The following sections describe how an IPFIX-over-TCP connection is
created, how IPFIX data is transferred over it, and how a connection
is to be terminated.
13.2.4.1 Connection Establishment
The IPFIX Exporting Process initiates a TCP connection to the IPFIX
Collecting Process. By default, the Collecting Process listens for
connections on TCP port XXXX (to be assigned by IANA). It MUST be
possible to configure both the Exporting and Collecting Processes to
use a different TCP port.
An Exporting Process MAY support more than one active connection to
different Collecting Processes (including the case of different
Collecting Processes on the same host).
Claise, et. al Standard Track [Page 39]
IPFIX Protocol Specification December 2004
13.2.4.2 Connection Release
When an Exporting Process has no more IPFIX Messages to send, it
SHOULD close the TCP connection.
When a Collecting Process no longer wants to receive IPFIX messages,
it SHOULD close the TCP connection, but SHOULD continue to receive
and process IPFIX messages until the Exporting Process has closed
its end.
When an Collecting Process detects that the TCP connection to the
Exporting Process is abnormally terminated, it MUST continue to
listen for a new connection.
When an Exporting Process detects that the TCP connection is
abnormally terminated, it SHOULD try to re-establish the connection.
Connection timeouts SHOULD be configurable.
13.2.4.3 IPFIX Message Encoding
TCP provides message boundary marking marking mechanism. When IPFIX
Message are sent over a TCP connection, the LENGTH field in the
IPFIX Message header defines the end of each message and the start
of the next message.
If an Exporting Process exports data from multiple Observation
Domains, it should be careful to choose the IPFIX Message lengths
appropriately to avoid head-of-line blocking between different
Observation Domains.
13.2.5 Template Management
New Template Records SHOULD be transmitted as soon as they are
created on the Metering Process, and preferably before any
associated Flow or Options Data Record is transmitted. The
Collecting Process SHOULD accept Flow and Options Data Records
without the associated Template Record.
A Collecting Process MUST record all Template and Option Template
Records for the duration of the connection, as an Exporting Process
is not required to re-export Template Records.
Claise, et. al Standard Track [Page 40]
IPFIX Protocol Specification December 2004
13.2.6 Fail-over
If the Collecting Process does not acknowledge the attempt by the
Exporting Process to establish a connection, the Exporting Process
should retry. The retry schedule SHOULD be configurable. In the
default configuration, an Exporting Process MUST NOT attempt to
establish a connection more frequently than once per minute.
The Exporter MAY log an alarm if the time to establish the
association exceeds a specified threshold.
If Collecting Process fail-over is supported by the Exporting
Process a second TCP connection MAY be opened in advance.
13.3 SCTP
This section describes how IPFIX can be transported over SCTP
[RFC2960] using the PR-SCTP [RFC3758] extension.
13.3.1 Congestion Avoidance
The SCTP transport protocol provides the required level of
congestion avoidance by design.
SCTP will detect congestion in the end-to-end path between
the IPFIX Exporting Process and the IPFIX Collecting Process,
and limit the transfer rate accordingly. When an IPFIX
Exporting Process has records to export, but detects that
transmission by SCTP is temporarily impossible, it can either
wait until sending is possible again, or it can decide to drop the
record. In the latter case, the dropped export data MUST
be accounted for, so that the amount of dropped export data can be
reported.
13.3.2 Reliability
The SCTP transport protocol is by default reliable, but has the
capability to operate in unreliable and partially reliable modes
[RFC3758].
Using reliable SCTP streams (referred to hereafter as "streams") for
the IPFIX export is not in itself a guarantee that all data records
are delivered. If there is congestion on the link from the
Claise, et. al Standard Track [Page 41]
IPFIX Protocol Specification December 2004
Exporting Process to the Collecting Process, or if a significant
number of retransmissions are required, the send queues on the
Exporting Process may fill up: the Exporting Process MAY either
suspend export or discard IPFIX Messages. If data records are
discarded the sequence numbers used for export MUST reflect the loss
of data.
13.3.3 MTU
SCTP provides the required IPFIX Message fragmentation service based
on path MTU discovery.
13.3.4 Exporting Process
13.3.4.1 Association Establishment
The IPFIX Exporting Process SHOULD initiate an SCTP association with
the IPFIX Collecting Process. By default, the Collecting Process
listens for connections on SCTP port XXXX (to be assigned by
IANA). It MUST be possible to configure both the Exporting
and Collecting Processes to use a different SCTP port.
The Exporting Process MAY establish more than one associations
(connection "bundle" in SCTP terminology) to the Collecting Process.
An Exporting Process MAY support more than one active association
to different Collecting Processes (including the case of different
Collecting Processes on the same host).
13.3.4.2 Association Shutdown
When an Exporting Process has no more IPFIX Messages to send, it
SHOULD shutdown the SCTP association.
When a Collecting Process no longer wants to receive IPFIX
Messages, it SHOULD shutdown its end of the association. The
Collecting Process SHOULD continue to receive and process
IPFIX Messages until the Exporting Process has closed its end.
When a Collecting Process detects that the SCTP association has been
abnormally terminated, it MUST continue to listen for a new
association establishment.
Claise, et. al Standard Track [Page 42]
IPFIX Protocol Specification December 2004
When an Exporting Process detects that the SCTP association to the
Collecting Process is abnormally terminated, it SHOULD try to re-
establish the association.
Association timeouts SHOULD be configurable.
13.3.4.3 Source ID
The IPFIX Message MUST contain a Message Header, which includes a
Source ID. The Exporting Process uses the Source ID to uniquely
identify to the Collecting Process the Observation Domain that
metered the Flows.
13.3.4.4 Stream
An Exporting Process MUST request at least two outbound streams per
association. The first stream (referred to as stream zero in the
rest of this document), is used to send the Template Set and the
Options Template Set. Stream zero MUST be fully reliable. Data
Sets MUST NOT be sent on stream zero.
Depending on the application requirement, the Exporting Process
selects the mode (unreliable, partially reliable, or fully reliable
mode) of the stream, used to send the Data Sets. Unreliable mode
MAY be used where the application does not require reliable
transmission and the use of a retransmission queue is impractical.
An Exporter MAY use multiple streams to export Data Sets, in some
cases different applications will have different requirements in
terms of reliability. In such a case, the Observation Domain MUST
use the same Source ID value on all of the multiple streams it uses.
Data Sets from multiple Observation Domains MUST NOT be transmitted
over the same stream; the Collecting Process should however verify
that the Source ID values are the expected values.
When Data Sets are exported over a partially reliable stream, they
SHOULD be marked for retransmission as long as there is room in the
SCTP send queues. However during times of congestion or other
retransmission events, if the queue overflows, the oldest data
record that has been transmitted and marked as partially reliable
should be freed and marked to be skipped per the PR-SCTP [RFC3758]
Claise, et. al Standard Track [Page 43]
IPFIX Protocol Specification December 2004
specification. The freed buffer space should then be re-used for
the new Data Sets being exported.
13.3.4.5 Template Management
When the transport protocol is SCTP the default Template Management
described in Section 12 is used.
13.3.5 Collecting Process
When the transport protocol is SCTP, the default Collector
processing described in Section 13 is used.
13.3.6 Failover
If the Collecting Process does not acknowledge the attempt by the
Exporting Process to establish an association it will retry using
the SCTP exponential backoff feature. The Exporter MAY log an alarm
if the time to establish the association exceeds a specified
threshold.
If Collecting Process failover is supported by the Exporting Process
a second SCTP association MAY be opened in advance.
13.4 UDP
This section describes how IPFIX can be transported over UDP
[RFC768]
13.4.1 Congestion Avoidance
UDP has no integral congestion avoidance mechanism. Its use
over congestion sensitive network paths is therefore deprecated.
UDP MAY be used in deployments where Exporters and Collectors
always communicate over dedicated links that are not susceptible
to congestion.
13.4.2 Reliability
UDP is not a reliable transport protocol, and cannot guarantee
delivery of messages. IPFIX Messages sent from the Exporting
Process to the Collecting Process using UDP may therefore be lost.
Claise, et. al Standard Track [Page 44]
IPFIX Protocol Specification December 2004
UDP MUST NOT be used unless the application can tolerate some
loss of Messages.
The Collecting Process could deduce the loss and reordering of IPFIX
Messages by looking at the discontinuities in the IPFIX Message
sequence number. These conditions SHOULD be logged.
Templates sent from the Exporting Process to the Collecting
Process using UDP as a transport MUST be resent at regular
intervals in case previous copies were lost. Implementations
MAY send templates using a reliable transport protocol, and
send IPFIX Flow and Option Data Records using UDP as the
transport protocol.
13.4.3 MTU
The maximum size of exported messages MUST be configured such that
the total packet size does not exceed the path MTU.
13.4.4 Port Numbers
By default, the Collecting Process listens on the UDP port XXXX (to
be assigned by IANA). It MUST be possible to configure both the
Exporting and Collecting Processes to use a different UDP port.
13.4.5 Exporting Process
The Exporting Process MAY duplicate the IPFIX Message
to the several Collecting Process.
13.4.6 Template Management
When IPFIX uses UDP as the transport protocol, Template Sets and
Option Template Sets MUST be re-sent at regular intervals. The
frequency of (Options) Template transmission MUST be configurable.
New Template Records SHOULD be transmitted as soon as they are
created, and SHOULD be transmitted before any associated Data
Record is transmitted.
In the event of configuration changes, the Exporting Process SHOULD
send the new template definitions at an accelerated rate. In such a
case, it MAY transmit the changed Template Record(s) and Options
Template Record(s), without any data, in advance to help ensure that
Claise, et. al Standard Track [Page 45]
IPFIX Protocol Specification December 2004
the Collector will have the correct template information before
receiving the first data.
If the Option Template scope is defined in another Template, then
both Templates SHOULD be sent in the same IPFIX Message. For
example: if a Flow Key Option Template (see section 8.3) is sent in
an Option Template, then the associated Template SHOULD be sent in
the same IPFIX Message.
Note that following a configuration change a new Template ID should
be used and the old Template ID SHOULD NOT be reused until its
lifetime has expired.
Template Withdraw Messages SHOULD NOT be sent over UDP.
13.4.7 Collecting Process
The Collecting Process SHOULD accept Flow and Options Data Records
without the associated Template Record. If the Template Records have
not been received at the time Flow Data Records (or Options Data
Records) are received, the Collecting Process SHOULD store the Flow
Data Records (or Options Data Records) for a short period of time
and decode them after the (Options) Template Records are received.
The short period of time MUST be lower than the Template lifetime.
The lifetime of a template at the Collecting Process is limited to a
fixed refresh timeout. The Collecting Process MUST associate a
lifetime with each Template received via UDP. Templates not
refreshed by the Exporting Process within the timeout are expired at
the Collecting Process. If the template is not refreshed by the
Exporting Process before that lifetime has expired, the Collecting
Process MUST discard the Template, and any current and future
associated Flow or Option Data Records. The Collecting Process MUST
NOT decode any further Flow or Option Data Records which are
associated with that expired Template.
At any given time the Collecting Process SHOULD maintain the
following for all the current Template Records and Options Template
Records: <Exporting Process, Observation Domain, Template ID,
Template Definition, Last Received>.
Claise, et. al Standard Track [Page 46]
IPFIX Protocol Specification December 2004
13.4.8 Failover
Because UDP is not a connection oriented protocol, the Exporting
Process is unable to determine from the transport protocol that the
Collecting Process is no longer able to receive the IFPIX Messages.
Therefore, it can not invoke a failover mechanism. However, the
Exporting Process MAY duplicate the IPFIX Message to several
Collecting Processes.
14. Security Considerations
Because IPFIX can be used to collect billing information and network
forensics, confusing or blinding IPFIX must be seen as a prime
objective during a sophisticated network attack.
If an attacker is in a position to inject false messages into an
IPFIX Message stream this will allow them to send forged Flow Data
Records, Options Data Records, or Templates. Forged Templates may
impair the Collectors ability to process any further Flow Records.
Forged Flow Records would have a direct effect on the application
using the Flows, for example a billing system may generate incorrect
billing information. Forged options may be able to alter the
meaning of Flow Records, for example if the sample rate is changed.
The IPFIX Messages themselves may contain information of value to an
attacker, and thus care must be taken to confine their visibility to
authorized users.
IPFIX Messages can be secured using IPsec. Alternatively if IPFIX
runs on top of SCTP or TCP, TLS [TLS] can be used.
When an Information Element containing end-user payload information
is exported, it SHOULD be transmitted to the Collecting Process
using a means that secures its contents against eavesdropping.
Suitable mechanisms include the use of either a direct point-to-
point connection or the use of an encryption mechanism. It is the
responsibility of the Collecting Process to provide a satisfactory
degree of security for this collected data, including, if necessary,
anonymization of any reported data.
14.1 IPsec Usage
To secure messages between the Exporter and the Collector an IPFIX
Claise, et. al Standard Track [Page 47]
IPFIX Protocol Specification December 2004
implementation MAY use IPsec. To ensure interworking between
Exporters and Collectors from different vendors, the following IPsec
profile MUST be supported. This profile is derived from [USEIPSEC].
14.1.1 Selectors
IPFIX runs between manually configured pairs of hosts on the
following transport ports (TBD). The appropriate selector would be
Exporter-Collector pairs and port number.
Note that, if the Exporter is a router, a non-interface ("loopback")
address should be used.
14.1.2 Mode
IPsec MUST be run in transport mode. The AH and ESP MUST be
supported by an IPFIX implementation of IPsec.
The Authentication Header (AH) [RFC2402] MUST be used if
authentication is required. The Security Protocol (ESP) [RFC2406]
must be used if there is a threat to the IPFIX Message content, or
if that content is confidential.
Normally in situations where the ESP was required the AH would also
be required. If ESP only is used, the sender's IP address MUST be
checked against the IP address asserted in the key management
exchange.
14.1.3 Key Management
In many networks, manual key management will be sufficient, and this
reduces the complexity of the Exporter, albeit at a cost of greater
configuration complexity. Manual key management MUST be supported.
If a replay attack is considered likely, an automated key management
such as IKE [IKE] key management system SHOULD be used.
14.1.4 Security Policy
Connections should be accepted only from the designated peer.
14.1.5 Authentication
Claise, et. al Standard Track [Page 48]
IPFIX Protocol Specification December 2004
Given the number of IPFIX capable Exporters that are likely to be
deployed by large ISPs, there will be circumstances where shared key
mechanisms are not adequate. Where an automated key management
system is used, certificate-based IKE SHOULD be supported.
14.1.6 Availability
It is accepted that IPsec will not be universally available in IPFIX
Exporters, and that where it is available, there may be issues of
throughput, which may itself raise security issues. In such
circumstances the other security measures described in this document
provide some threat mitigation.
14.2 TLS Usage
The IPFIX Exporter initiating a connection acts as a TLS client
according to [TLS], and an IPFIX Collector that accepts a connection
acts as a TLS server. If mutual authentication is required the
IPFIX Device acting as TLS server MUST request a certificate from
the IPFIX Device acting as TLS client, and the IPFIX Device acting
as TLS client MUST be prepared to supply a certificate on request.
14.3 Protection against DoS attacks
An attacker may directly mount a DoS attack by generating large
amounts of traffic. If TCP is used for transport, then the Flow to
the Collector would back off due to congestion and eventually stall,
blinding the IPFIX system. An attack could then proceed without
further observation. SCTP-PR will have a different pathology under
such an attack. Stale data at the head of the queue will get
flushed giving some visibility of the attack. In case of UDP, IPFIX
would reduce to some sort of sampling, meaning that some forensics
may be left.
To avoid blinding of the IPFIX system some mechanism for service
differentiation can be used to prioritize IPFIX traffic over user
traffic. An alternative is to use a dedicated network for the
transport of IPFIX Messages. By sending the IPFIX Messages over a
dedicated network, IPFIX Message loss induced by user traffic
congestion is minimized. However an attacker may trigger the
generation of excessive IPFIX Messages, and to avoid information
loss during such an attack the IPFIX network must be adequately
sized.
Claise, et. al Standard Track [Page 49]
IPFIX Protocol Specification December 2004
14.4 When IPsec or TLS is not an option
The use of IPsec or TLS might not be an option because of
performance issues.
Without IPsec or TLS an IPFIX entity has no means to authenticate an
IPFIX entity other than the Source IP address. Useful protection is
gained by allocating Exporter and Collector IP addresses from ranges
that are excluded from use by user traffic and preventing spoofing
attacks by proper ingress filtering. Where large numbers of
Exporters, proxies and Collectors are used in a network, it may be
tempting for the administrator to not impose source IP address
restrictions but this leaves a proxy or Collector open to the
reception of invalid information. Using an open proxy or Collector
is therefore discouraged.
If IP address spoofing can not be prevented some level of protection
against an insertion attack is required. With a modern
implementation of TCP with good ISN randomization [XXX-REFERENCE] or
SCTP insertion such attacks are difficult without the ability to
snoop the packet Flow [XXX-SCTP-BLIND-SPOOFING-REFERENCE]. UDP is
vulnerable to insertion attacks, however, randomization of the IPFIX
Sequence Number might mitigate this problem. In all these cases,
the Sequence Number space is relatively small giving only limited
protection. Therefore a 64 bit cookie [L2TPv3] SHOULD be included
as an element within all messages.
The use of a dedicated network prevents IPFIX Messages from being
inspected by an attacker.
14.5 Logging an IPFIX Attack
A Collector may detect problems by tracking the IPFIX Sequence
Number and therefore SHOULD provide a logging mechanism for tracking
out of sequence messages. Such out of sequence messages may not
only be caused by network congestion or Exporter/Collector resource
exhaustion but also by an attacker injecting false messages.
Note that an attacker may be able to exploit the behavior of the
Collector when it receives an out of sequence message. For example
a Collector that simply reset the expected Sequence Number upon
receipt of a later message would easily be temporarily blinded by
Claise, et. al Standard Track [Page 50]
IPFIX Protocol Specification December 2004
deliberately injecting messages with a much larger Sequence Number.
[EDITOR NOTE: the security section may need be adapted to the
revised transport section]
15. IANA Considerations
The IPFIX Protocol, as set out in this document, has two sets of
assigned numbers. Considerations for assigning them are discussed
in this section, using the example policies as set out in the
"Guidelines for IANA Considerations" document IANA-RFC [RFC2434].
15.1 Numbers used in the Protocol
IPFIX Messages use two fields with assigned values. These are the
IPFIX Version Number, indicating which version of the IPFIX Protocol
was used to export an IPFIX Message, and the IPFIX Set ID,
indicating the type for each set of information within an IPFIX
Message.
Changes in either IPFIX Version Number or IPFIX Set ID assignments
require an IETF Consensus, i.e. they are to be made via RFCs
approved by the IESG.
15.2 Numbers used in the Information Model
Fields of the IPFIX protocol carry information about traffic
measurement. They are modeled as elements of the IPFIX information
model [IPFIX-INFO]. Each Information Element describes a field which
may appear in an IPFIX Message. Within an IPFIX Message the field
type is indicated by its Field Type.
New assignments for IPFIX Field Types will be administered by IANA,
on First Come First Serve basis [RFC 2434] , subject to Expert
Review [RFC 2434], i.e. review by one of a group of experts
designated by an IETF Operations and Management Area Director. The
group of experts must double check the Information Elements
definitions for completeness, accuracy and redundancy with already
defined Information Elements. Those experts will initially be drawn
from the Working Group Chairs and document editors of the IPFIX and
PSAMP Working Groups. The IANA assignments for IPFIX Field Types
will range from 128 to 32767; the values below 128 are reserved or
Claise, et. al Standard Track [Page 51]
IPFIX Protocol Specification December 2004
assigned already; the values ranging from 32768 to 65535 are
allocated for private use by vendors.
16. Examples
Let's consider the example of an IPFIX Message composed of a
Template Set, a Data Set (which contains three Flow Data Records),
an Options Template Set and a Data Set (which contains 2 Options
Data Records).
IPFIX Message:
+--------+---------------------------------------------. . .
| | +--------------+ +-----------------------+
|Message | | Template | | Data |
| Header | | Set | | Set | . . .
| | | (1 Template) | | (3 Flow Data Records) |
| | +--------------+ +-----------------------+
+--------+---------------------------------------------. . .
. . .+-------------------------------------------------+
+------------------+ +--------------------------+ |
| Options | | Data | |
. . .| Template Set | | Set | |
| (1 Template) | | (2 Options Data Records) | |
+------------------+ +--------------------------+ |
. . .--------------------------------------------------+
16.1 Message Header Example
The Message Header is composed of:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version = 0x000a | Length = 152 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Export Time |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Claise, et. al Standard Track [Page 52]
IPFIX Protocol Specification December 2004
16.2 Template Set Examples
16.2.1 Template Set using IETF specified Information Elements
We want to report the following Field Types:
- The source IP address (IPv4), so the length is 4
- The destination IP address (IPv4), so the length is 4
- The next-hop IP address (IPv4), so the length is 4
- The number of bytes of the Flow
- The number of packets of the Flow
Therefore, the Template Set will be composed of the following:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 2 | Length = 28 bytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 256 | Field Count = 5 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP_SRC_ADDR = 0x0008 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP_DST_ADDR = 0x000C | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP_NEXT_HOP = 0x000F | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IN_PKTS = 0x0002 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IN_BYTES = 0x0001 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
16.2.2 Template Set using Enterprise Specific Information Elements
We want to report the following Field Types:
- The source IP address (IPv4), so the length is 4
- The destination IP address (IPv4), so the length is 4
- An Enterprise Specific Field representing proprietary
information, with a type of 0x000F and a length of 4
- The number of bytes of the Flow
- The number of packets of the Flow
Therefore, the Template Set will be composed of the following:
Claise, et. al Standard Track [Page 53]
IPFIX Protocol Specification December 2004
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 2 | Length = 28 bytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 256 | Field Count = 5 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP_SRC_ADDR = 0x0008 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP_DST_ADDR = 0x000C | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| Enterprise Field Type=0x000F| Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IN_PKTS = 0x0002 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IN_BYTES = 0x0001 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
16.3 Data Set Example
In this example, we report the following three Flow Records:
Src IP addr. | Dst IP addr. | Next Hop addr. | Packet | Bytes
| | | Number | Number
------------------------------------------------------------------
192.168.1.12 | 192.168.2.254 | 192.168.1.1 | 5009 | 5344385
192.168.1.27 | 192.168.2.23 | 192.168.1.2 | 748 | 388934
192.168.1.56 | 192.168.2.65 | 192.168.1.3 | 5 | 6534
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 256 | Length = 64 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.168.1.12 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.168.2.254 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.168.1.1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 5009 |
Claise, et. al Standard Track [Page 54]
IPFIX Protocol Specification December 2004
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 5344385 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.168.1.27 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.168.2.23 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.168.1.2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 748 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 388934 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.168.1.56 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.168.2.65 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.168.1.3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 5 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 6534 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Note that padding is not necessary in this example.
16.4 Options Template Set Examples
16.4.1 Options Template Set using IETF specified Information Elements
Per line card (the router being composed of two line cards), we want
to report the following Field Types:
- Total number of IPFIX Messages
- Total number of exported Flows
Each line card is characterized by an unique Observation Domain,
represented by the unique Source ID Information Elements [IPFIX-
INFO]. As a consequence, the Scope Field is the Source ID
Information Element.
Therefore, the Options Template Set will be:
0 1 2 3
Claise, et. al Standard Track [Page 55]
IPFIX Protocol Specification December 2004
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length = 24 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 257 | Field Count = 3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count = 1 | Scope 1 Field Type = 141 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope 1 Field Length = 4 | TOTAL_EXP_PKTS_SENT = 41 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 | TOTAL_FLOWS_EXP = 42 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
16.4.2 Options Template Set using Enterprise Specific Information
Elements
Per line card (the router being composed of two line cards), we want
to report the following Field Types:
- Total number of IPFIX Messages
- An Enterprise Specific number of exported Flows,
with a type of 42 and a length of 4
Each line card is characterized by an unique Observation Domain,
represented by the unique Source ID Information Elements [IPFIX-
INFO]. As a consequence, the Scope Field is the Source ID
Information Element.
The format of the Options Template Set is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length = 24 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 257 | Field Count = 3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count = 1 | Scope 1 Field Type = 141 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope 1 Field Length = 4 | TOTAL_EXP_PKTS_SENT = 41 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 |1| Enterprise Field Type = 42 |
Claise, et. al Standard Track [Page 56]
IPFIX Protocol Specification December 2004
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 | Enterprise number ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
... Enterprise number | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
16.4.3 Options Template Set using an Enterprise Specific scope
In this example, we want to export the same information as in the
example in section 16.4.2:
- Total number of IPFIX Messages
- Total number of exported Flows
But this time, the information pertains to a proprietary scope,
identified by Enterprise Specific Information Element number 123.
The format of the Options Template Set is now as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length = 28 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 257 | Field Count = 3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count = 1 |1| Scope 1 Field Type = 123 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope 1 Field Length = 4 | TOTAL_EXP_PKTS_SENT = 41 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 | TOTAL_FLOWS_EXP = 42 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
16.5 Data Set with Options Data Records Example
In this example, we report the following two Options Data Records:
Line Card ID | IPFIX Message | Exported Flow Records
------------------------------------------------------------------
Line Card 1 (SourceID=1) | 345 | 10201
Line Card 2 (SourceID=2) | 690 | 20402
Claise, et. al Standard Track [Page 57]
IPFIX Protocol Specification December 2004
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 257 | Length = 20 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 345 | 10201 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 690 | 20402 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
17. References
17.1 Normative References
[IPFIX-ARCH] Sadasivan, G, Brownlee, N. "Architecture Model for IP
Flow Information Export" draft-ietf-ipfix-arch-02.txt", October 2003
[IPFIX-INFO] Calato, P, Meyer, J, Quittek, J, "Information Model for
IP Flow Information Export" draft-ietf-ipfix-info-02, November 2003
[UDP] Postel, J., "User Datagram Protocol" RFC 768, August 1980
[TCP] "TRANSMISSION CONTROL PROTOCOL DARPA INTERNET PROGRAM
PROTOCOL SPECIFICATION" RFC 793, September 1981
[RFC1889] Schulzrinne, H., Casner, S., Frederick, R., Jacobson, V.,
"RTP: A Transport Protocol for Real-Time Applications ", RFC 1889,
January 1996
[RFC2434] Alvestrand, H. and T. Narten, "Guidelines for Writing an
IANA Considerations Section in RFCs", RFC 2434, October 1998.
[RFC2402] Kent, S., Atkinson, R., "IP Authentication Header ", RFC
2402, November 1998
[RFC2406] Kent, S., Atkinson, R., "IP Encapsulating Security Payload
(ESP)", RFC 2406, November 1998
Claise, et. al Standard Track [Page 58]
IPFIX Protocol Specification December 2004
[RFC2960] Stewart, R. (ed.) "Stream Control Transmission Protocol",
RFC 2960, October 2000
[RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., Conrad, P.
"Stream Control Transmission Protocol (SCTP) Partial Reliability
Extension", RFC 3758, May 2004
17.2 Informative References
[IPFIX-REQ] Quittek, J, Zseby, T, Claise, B, Zander, S,
"Requirements for IP Flow Information Export" draft-ietf-ipfix-reqs-
15.txt, June 2003
[IPFIX-AS] Zseby, T, Penno, R, Brownlee, N, Claise, B, "IPFIX
Applicability", draft-ietf-ipfix-as-01.txt, October 2003
[IPFIX-EVAL] Leinen, S, "Evaluation of Candidate Protocols for IP
Flow Information Export (IPFIX)", draft-leinen-ipfix-eval-contrib-
02.txt, January 2003
[NETFLOW9] Claise, B, et al "Cisco Systems NetFlow Services Export
Version 9", draft-claise-netflow-9-07.txt, December 2003
[PEN] IANA Private Enterprise Numbers registry
http://www.iana.org/assignments/enterprise-numbers
[USEIPSEC] S. Bellovin, Guidelines for Mandating the Use of IPsec,
draft-bellovin-useipsec-02.txt, October 2003, work
in progress.
[IKE] Harkins, D. and D. Carrel, "The Internet Key Exchange
(IKE)", RFC 2409, November 1998.
[TLS] Dierks, T. and C. Allen, "The TLS Protocol Version
1.0", RFC 2246, January 1999.
[L2TPv3] J. Lau et al. Layer Two Tunneling Protocol (Version 3)
draft-ietf-l2tpext-l2tp-base-11.txt, October 2003, work
in progress.
[XXX-REFERENCE]
[XXX-SCTP-BLIND-SPOOFING-REFERENCE]
Claise, et. al Standard Track [Page 59]
IPFIX Protocol Specification December 2004
18. Acknowledgments
We would like to thank the following persons: Juergen Quittek for
the coordination job; Nevil Brownlee and Dave Plonka for the
thorough reviews; Randall Stewart and Peter Lei for their SCTP
expertise; Martin Djernaes for the first essay on the SCTP section;
Simon Leinen for the first essay on the TCP section Sebastian
Zander, Jeff Meyer, Maurizio Molina, Carter Bullard, Tal Givoly, and
many more, for the technical feedback.
Authors Addresses
Benoit Claise
Cisco Systems
De Kleetlaan 6a b1
1831 Diegem
Belgium
Phone: +32 2 704 5622
E-mail: bclaise@cisco.com
Stewart Bryant
Cisco Systems, Inc.
250, Longwater,
Green Park,
Reading, RG2 6GB,
United Kingdom
Phone: +44 (0)20 8824-8828
Email: stbryant@cisco.com
Ganesh Sadasivan
Cisco Systems, Inc.
170 W. Tasman Dr.
San Jose, CA 95134
USA
Phone: +1 (408) 527-0251
Email: gsadasiv@cisco.com
Mark Fullmer
OARnet
2455 North Star Rd.
Columbus, Ohio 43221
Phone: +1 (614) 728-8100
Claise, et. al Standard Track [Page 60]
IPFIX Protocol Specification December 2004
Email: maf@eng.oar.net
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has made
any independent effort to identify any such rights. Information on the
procedures with respect to rights in RFC documents can be found in BCP
78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this specification
can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary rights
that may cover technology that may be required to implement this
standard. Please address the information to the IETF at ietf-
ipr@ietf.org.
The IETF has been notified of intellectual property rights claimed in
regard to some or all of the specification contained in this document.
For more information consult the online list of claimed rights.
Disclaimer of Validity
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement
Claise, et. al Standard Track [Page 61]
IPFIX Protocol Specification December 2004
Copyright (C) The Internet Society (2004). This document is subject to
the rights, licenses and restrictions contained in BCP 78, and except
as set forth therein, the authors retain all their rights.
Claise, et. al Standard Track [Page 62]