INTERNET DRAFT C. Calabrese
Expires: June, 2001 SANS Institute
<draft-calabrese-requir-logprot-00.txt M. Arpad
Association of Hungarian Linux Users
December 2000
Requirements for a Network Event Logging Protocol
Status of this Memo
This document is an Internet-Draft and is in full
conformance with all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet
Engineering Task Force (IETF), its areas, and its working
groups. Note that other groups may also distribute
working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum
of six months and may be updated, replaced, or obsoleted
by other documents at any time. It is inappropriate to
use Internet- Drafts as reference material or to cite
them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be
accessed at http://www.ietf.org/shadow.html
To learn the current status of any Internet-Draft, please
check the "1id-abstracts.txt" listing contained in the
Internet-Drafts Shadow Directories on ftp.is.co.za
(Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific
Rim), ds.internic.net (US East Coast), or ftp.isi.edu (US
West Coast).
The key words 'MUST', 'MUST NOT', 'REQUIRED', 'SHALL',
'SHALL NOT', 'SHOULD', 'SHOULD NOT', 'RECOMMENDED',
'MAY', and 'OPTIONAL' in this document are to be
interpreted as described in RFC 2119[1].
Copyright Notice
Copyright (C) The Internet Society (2000). All Rights
Reserved.
This document and translations of it may be copied and
furnished to others, and derivative works that comment on
or otherwise explain it or assist in its implementation
may be prepared, copied, published and distributed, in
whole or in part, without restriction of any kind,
provided that the above copyright notice and this
INTERNET DRAFT Event Logging Requirements December 2000
paragraph are included on all such copies and derivative
works. However, this document itself may not be modified
in any way, such as by removing the copyright notice or
references to the Internet Society or other Internet
organizations, except as needed for the purpose of
developing Internet standards in which case the
procedures for copyrights defined in the Internet
Standards process must be followed, or as required to
translate it into languages other than English.
The limited permissions granted above are perpetual and
will not be revoked by the Internet Society or its
successors or assigns.
This document and the information contained herein is
provided on an "AS IS" basis and THE INTERNET SOCIETY AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL
WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN
WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Abstract
This document presents requirements for sending system
event/audit logs over the Internet. In developing these
requirements, it considers the need for log management,
security, high-performance, and compatibility with
existing practice.
Table of Contents
1 Introduction ???
2 Log Reporting Considerations ???
3 Security Considerations ???
4 Resource Utilization ???
5 Existing Practice ???
6 Conclusions ???
7 References ???
8 Author's Addresses ???
1. Introduction
Computer systems and other network devices often need to
"log" or record information about important events. In
many systems, these event logs are recorded locally, in
persistent storage within the system where the event
occurs. These messages may also be transmitted to a
remote system where persistent storage takes place.
Reasons for such remote transmission include event
logging on systems with no local persistent storage and
centralization of log record management.
INTERNET DRAFT Event Logging Requirements December 2000
This memo concerns itself with the requirements for
transmitting log messages over an Internet Protocol
network, including requirements for log management, event
correlation, security, performance, and compatibility
with existing practice. It was originally conceived to
guide the workings of the IETF's Security Issues in
Network Event Logging Working Group, who are dealing with
security issues in the syslog system[2]. However, the
principles apply equally well to other event logging
systems.
2. Log Reporting Considerations
2.1 Introduction and Rational
The ultimate reason for transmitting log messages over
the network is so these messages can be examined later
to detect and analyze "interesting" events (security
incidents, peak server usage, etc.). Although the
transmission of the messages itself is not directly
related to this examination, the systems related to
transmission should aid the process of examination to
the largest extent practical.
2.2 Identification of Event Threads
Since logging system tend to generate and transmit
discrete event messages, one of the most important
reporting requirements for a logging system is to be
able to figure out how those event messages fit
together to form a thread of messages about the same
incident.
Messages in a thread are typically identified by the
originating machine the program or sub-system on the
originating machine, and the instance of that program
or sub-system. Additionally, some kind of sequence
information must be present to determine the order of
event messages in the thread.
In most cases, the originating machine can be
determined by the source IP address of log messages
and the sequence determined by the order of arriving
packets (UDP) or relative ordering in the transmission
stream (TCP). However, the IP address can be forged
and is therefore unreliable. Similarly, an attacker
could reorder packets in a UDP stream. Furthermore,
there is no easy method of determining the
program/sub-system and its instance on the originating
machine without including such identifying information
directly in the message. Therefore, it seems plain
that the logging messages must directly contain all
INTERNET DRAFT Event Logging Requirements December 2000
these pieces of information in order to facilitate the
identification of event threads.
This issue is also discussed in the Security section
below.
2.3 Event Correlation
In addition to identifying event messages within the
same incident thread from the same source, it is also
desirable to correlate messages regarding the same
incident from multiple sources. This implies a
standard mechanism for identifying incidents based on
various criteria such as the time the event occurred,
IP addresses of network entities involved, etc.
2.4 Event Filters
2.4.1 Introduction and Rational
Selecting which events to transmit reduces network
load. Selecting which events to accept when
transmitted reduces the amount of data that must be
processed by higher-level software and which must
be persistently stored. The log transmission
protocols can help these efforts by making it easy
to select events for filtering.
2.4.2 Filtering By Source
The most obvious attribute to filter data on in an
Internet Protocol network is the source IP address
of the data.
2.4.3 Filtering By Priority
Another obvious filter would be by message priority
so that high priority messages can be posted as
alarms, very low priority messages ignored, etc.
2.4.4 Filtering By Facility
Not all high priority messages should necessarily
be handled the same way. Aside from message
priority, the primary method of filtering in the
traditional syslog system[2], is by something
called the message facility (LOG_AUTH, LOG_DAEMON,
LOG_MAIL, etc.).
2.4.5 Filtering By Other Attributes
Source address, priority and facility are by no
means the only pieces of information log messages
will need to be filtered by. Indeed, just about
any attribute of a log message is fair game, and
the potential attributes too numerous to enumerate
here. Some have argued that log messages should be
structured to make such filtering easier (and some
INTERNET DRAFT Event Logging Requirements December 2000
structure is already needed for thread
identification, event correlation, and other
reasons); however, others have argued that rigid
structures are not needed if filters are based on
regular expressions[3].
3. Security Considerations
3.1 Introduction and Rational
According to the U.S. Department of Defense Trusted
Computer System Evaluation Criteria (TCSEC)[4], the
fundamental requirements for computer security are:
1. An explicit and well-defined security policy
enforced by the system.
2. Access control labels associated with objects.
3. Identification of individual subjects.
4. Selective storage and protection of audit
records so that actions affecting security can
be traced to the responsible party.
5. Assurance that the computer system enforces
security requirements.
6. Trusted mechanisms that enforce these basic
requirements that are continuously protected
against tampering and/or unauthorized changes.
In the case of system event logs, the logs themselves
may contain the aforementioned audit records.
3.2 Common Criteria
To put it another way, a logging system should meet
the following Common Criteria for Information
Technology Security Evaluation requirements:[5]
o FAU (Security audit)
o FAU_ARP (automatic response)
o FAU_GEN (generation)
o FAU_SAA (analysis)
o FAU_SAR (review)
INTERNET DRAFT Event Logging Requirements December 2000
o FAU_SEL (selection)
o FAU_STG (storage)
o FCS (Cryptographic support)
o FCS_COP (operation)
o FDP (User data protection)
o FDP_ACC (Access control policy)
o FDP_ACF (Access control functions)
o FDP_DAU (Data authentication)
o FDP_ETC (Export to outside TSF control)
o FDP_IFC (Information flow control policy)
o FDP_IFF (Information flow control functions)
o FDP_ITC (Import from outside TSF control)
o FDP_ITT (Internal TOE transfer)
o FDP_RIP (Residual information protection)
o FDP_ROL (Rollback)
o FDP_SDI (Stored data integrity)
o FDP_UCT (Inter-TSF user data confidentiality
transfer protection)
o FDP_UIT (Inter-TSF user data integrity
transfer protection)
o FPT (Protection of the TSF)
o FPT_FLS (Fail secure)
o FPT_ITA (Availability of exported TSF data)
o FPT_ITC (Confidentiality of exported TSF
data)
o FPT_ITT (Internal TOE transfer)
o FPT_RCV (Trusted recovery)
o FPT_RPL (Replay detection)
INTERNET DRAFT Event Logging Requirements December 2000
o FPT_SSP (State synchrony protocol)
o FPT_STM (Time stamps)
o FPT_TDC (Inter-TSF data consistency)
o FRU (Resource utilization)
o FRU_FLT (Fault tolerance)
o FRU_PRS (Priority of service)
o FRU_RSA (Resource allocation)
3.3 Implications
3.3.1 Policies
The questions of what appropriate policies are for
system logging and how to enforce them are beyond
the scope of this memo. However, any protocol for
transmitting log messages must take into account
the fact that different systems may have different
security policies. Therefore, such protocols
should not require adherence to any particular
policy.
3.3.2 Access Control Labels
Access control labels fall into two categories.
Integrity labels (think file permissions) and
Sensitivity labels (secret, top-secret, etc.).
In the case of system log messages, Integrity
labels could be mapped into the facility tags
present in the existing syslog system[2]. For such
a mapping to be useful, however, there should be be
a larger and more orthogonal set of facility tags
than the small in traditional syslog. The system
should also support the ability to specify multiple
facility tags for a particular message.
As for Sensitivity labels, if we recognize that
logs operating at lower priority also tend to be
much more verbose, then we can allow an inverse
relationship between Sensitivity labels and log
priorities as used in the existing syslog system
(LOG_DEBUG, LOG_INFO, LOG_NOTICE, LOG_WARNING,
etc.)[2]. In other words, higher-priority messages
may be more widely viewed and less widely created.
Conversely, lower-priority messages may be less
widely viewed but more widely created. Of course,
this is a big assumption. It also implies that
message priority must be present in all messages.
INTERNET DRAFT Event Logging Requirements December 2000
Also note that a strictly TCSEC or U.S. National
Security Agency Labeled Security Protection Profile
(LSPP) environment would require the protocol and
the components operating on it to either act as a
multilevel channel or use distinct single-level
channels for each set of Sensitivity labels. A
true multilevel channel would require much closer
mapping to the native Sensitivity labels than the
simple mapping described above. Furthermore, the
any non-labeled to labeled transitions must be done
in a single-level channel only. [4] [6]
3.3.3 Identification/Authentication
Ideally, we want to be able to show by strong
identification mechanisms (i.e., digital
signatures) that a certain log message was
generated by a certain program running with certain
privileges/user-id on a certain machine at a
certain time. This not only allows the
exact/authentic subject originating the message to
be identified, but also allows non-repudiation of
the messages, and easy correlation of events
between machines for purposes such as intrusion
detection and forensics.
The most obvious way of implementing such
identification is to issue a digital signature key
to each program/user/machine combination and to
require all messages to be thusly signed. However,
this runs into two major problems. The first is
one of key management (i.e., there are too many
keys to manage) and the second one of performance
(i.e., not all log messages require such
identification capabilities and not all machines
have the horsepower to supply them). However, if
the logging mechanism on the originating machine is
part of the system's trusted computing base (TCB),
then it is sufficient for TCB to sign the identity
of the log messages on the originating program's
behalf when directed to do so.
Note that it is not necessary for each message to
be digitally signed in its entirity. A common
practice is to negotiate and sign a session key and
then use that key to encrypt a message digest or
message authentication code (MAC) of the individual
messages[7].
3.3.4 Selective Storage
Essentially this boils down to the requirement that
the originator of a log message have the capability
to detect that a particular message has or has not
reached its destination. This does not necessarily
INTERNET DRAFT Event Logging Requirements December 2000
imply that only one copy of the message may reach
its destination or that the sender must block
waiting for delivery notification, which leaves
some room to balance this requirement against
issues such as performance[8].
On the other hand, the TCSEC, the LSPP and the U.S.
National Security Agency Controlled Access
Protection Profile (CAPP) require security
sensitive applications to shut down if they can't
log security-related events, so the capability to
block waiting for delivery notification must be
present in such environments [4] [6] [9].
3.3.5 Assurance
Issues such as protocol and implementation
evaluation against security policies and
requirements are beyond the scope of this memo.
However, we can address this issue at a
requirements level by observing that all security
policies center around confidentially, integrity,
and availability.
3.3.5.1 Confidentiality
It is generally acknowledged that the only
practical way to achieve confidentiality of
messages traveling over an open network is
through the use of cryptography.
3.3.5.2 Integrity
Integrity of log messages boils down to
identification of the sender, immutability of
messages without detection, and assurance that
logs have reached their destination.
Identification and immutability can be addressed
through digital signatures as discussed in the
Identification/Authentication section above.
Assurance of delivery, is discussed in the
Selective Storage section above.
3.3.5.3 Availability
The following seem reasonable requirements for
high-availability:
1. Logging mechanisms should be able to
transmit their log messages to multiple
destinations. Therefore increasing the
probability that at least one will be
available.
2. Logging protocols should make it easy to
filter unwanted messages received.
Therefore reducing the possibility of a
INTERNET DRAFT Event Logging Requirements December 2000
denial of service attack on the recipient
through the generation of bogus messages.
3. Logging protocols should not require log
senders to wait synchronously for positive
acknowledgement of messages sent before
sending any additional messages.
Therefore making denial of service attacks
on log senders more difficult through the
generation of excessive traffic on a
particular network link. Note that not
requiring applications to wait for
positive acknowledgement is not the same
thing as precluding them from waiting[8].
The TCSEC, the LSPP, and the CAPP require
security sensitive applications to shut
down if they can't log security-related
events, so the capability to block waiting
for delivery notification must be present
in such environments [4] [6] [9].
4. Logging protocols should provide the
capability to retransmit unacknowledged
messages. Therefore also making denial of
service attacks on log senders more
difficult through the generation of
excessive traffic on a particular network
link. Note that this facility may be
provided by lower level protocols such as
by the data integrity features of TCP[7].
This implies degraded capabilities when
running over other lower level protocols,
which may be a reasonable trade-off on
many circumstances.
High availability protocol and implementation
issues such as high-availability clustering,
resistance to buffer overrun attacks, etc. are
beyond the scope of this memo.
3.3.6 Trusted Mechanisms
This area is beyond the scope of this memo.
3.4 Protocol Level for Cryptographic Functions
One open issue from the above discussion is whether
cryptographic functions for identification signatures
and confidentiality should be performed at the network
transport layer or at higher-level application-
specific protocols.
Performing these functions at the network transport
layer allows the use of standard transport layer
INTERNET DRAFT Event Logging Requirements December 2000
security mechanisms (TLS, IPsec), making
implementation easier and possibly leading to higher
performance due to hardware assist for standard
mechanisms, smaller memory footprint on devices
already supporting such mechanisms, etc. It also
allows filtering of garbage messages at a lower level,
making denial of service attacks more difficult.
On the other hand, having identification signatures
operate at the network transport level means rejecting
all unsigned or improperly signed messages since
signing information will be lost to the higher
protocol layers. Since not all devices generating log
messages will have sufficient computing abilities to
sign all messages and not all messages will contain
information requiring signature, can be problematic.
Furthermore, signing and encrypting individual
messages facilitates being able to show the
confidentiality and integrity of messages without
needing to show full chain-of-custody or prove that
all links in the chain were appropriately trusted.
Therefore, network logging systems should allow
cryptographic identification functions to be performed
both at the network transport layer and also at higher
layers. Cryptographic confidentiality functions, on
the other hand, are likely to be more appropriate at
the network transport level.
4. Resource Utilization Considerations
From a logging protocol standpoint, the following
resources may be considered:
1. CPU/memory utilization on the sending system to
marshal internal log messages into the on-the-wire
protocol, including any encryption, signing, and/or
compression.
2. On-the-wire bandwidth utilization, including
amenability for the protocol to be compressed by
hardware compressors.
3. CPU/memory utilization on the receiving system
decompress, filter, verify the integrity of, and/or
store incoming messages.
Most of these considerations argue for a simple text
based protocol that is easy to marshal and compress where
application-level compression, encryption, and signatures
are a configurable option. However, the need to filter
messages argues for a more structured protocol to make
INTERNET DRAFT Event Logging Requirements December 2000
filtering easier. Furthermore, there may be interactions
between the encryption mechanism and the ability to
compress the message stream.
5. Existing Practice
5.1 Existing syslog
The syslog system is probably the most widely deployed
mechanism for storing, filtering, and transferring log
message over the network. It is ubiquitous in *nix
environments, being bundled with all *nix flavors. It
is also heavily used by routers, switches, and other
network applicances where its ability to transmit logs
over the network, its simplicity, and the availability
of free implementations make it a natural choice. It
has even crept into usage in the Microsoft Windows
world, where the native logging system does not work
well in large networked environments nor can it
interoperate with the logging mechanisms of other
operating systems.
Unfortunately, syslog also has some very serious and
widely recognized weaknesses.[10]
1. Message authenticity is based on easy-to-spoof
IP addresses.
2. There is no mechanism to assure message
confidentiality or integrity.
3. The protocol does not guarantee delivery of
messages, or ordered delivery of messages.
4. The relatively unstructure nature of syslog
messages makes them difficult to filter and/or
analyze.
Several systems have been devised to improve upon
syslog, and the rest of this section will address
these.
5.2 Schneier and Kelsey
Bruce Schneier and John Kelsey of Counterpane Internet
Security presented a paper at the the Seventh USENIX
Security Symposium on cryptographic mechanisms for
preserving message integrity and confidentiality when
storing log messages on an untrusted system.[11].
They also presented a follow-up paper on accessing
those logs over a network at the Second International
Workshop on the Recent Advances in Intrusion Detection
INTERNET DRAFT Event Logging Requirements December 2000
(RAID '99).[12] To the best of my knowledge, no
practical implementations of this system has been
built.
5.3 Nsyslogd
Nsyslogd is a drop-in syslog replacement developed by
Darren Reed of The Australian National University that
sports improved filtering based on regular
expressions, guaranteed message delivery and ordering
through the uses of TCP as a transport mechanism, and
on-the-wire privacy through SSL/TLS.[13]
5.4 Secure Syslog
Secure Syslog is another drop-in syslog replacement,
this one developed by Core-SDI. It offers message
integrity and confidentiality guarantees through the
use of Core-SDI's PEO-1 cryptographic protocol.[14]
5.5 ULM
ULM is a toolset developed by Herve Schauer
Consultants to query syslog based logs that are stored
in a structured format. The original version of this
tool used a format described in a now-expired Internet
Draft by Abela and Debeaupuis[15]. A newer version
uses an XML schema developed by author C. Calabrese
and based on the original ULM format.[16]
5.6 syslog-ng
Syslog-ng is another drop-in syslog replacement, this
one developed by Balazs Scheidler of BaliBit
Computing. Like nsyslogd, it offers improved
filtering and guaranteed message delivery and
ordering. Unlike nsyslogd it does not offer SSL/TLS
support, but it does support message integrity through
the use of digital signatures.[17]
6. Conclusions
Although several systems have been developed to improve
upon the ubiquitous syslog system, none adequately
addresses all the requirements identified here. It is
hoped that this exercise of identifying these
requirements will provide the designers of these and
other logging systems with the input they need to address
all these issues. In particular, this document and the
process behind its development are playing an important
role in the development of system logging protocols being
carried out by the IETF Security Issues in Network Event
INTERNET DRAFT Event Logging Requirements December 2000
Logging Working Group.
In particular, this process has identified the following
as areas that need to be addressed:
o Log message format structures to support
Identification of Even Threads, Event Correlation,
and Event Filters.
o Security mechanisms to support log message Access
Control Labels, Message Identification, Selective
Storage, Message Confidentiality, Message Integrity,
and System Availability.
7. References
[1] S. Bradner, "Key words for use in RFCs to
Indicate Requirement Levels." The Internet
Society. March 1997. Available from
ftp.ietf.org/rfc/rfc2119.txt.
[2] CSRG, "Unix Programmer's Manual, 4.3 Berkeley
Software Distribution, Virtual VAX-11 Version,"
Six Volumes and an Index Volume. University of
California Computer Systems Research Group, April
1986.
[3] Messages on the syslog-sec@employees.org mailing
list dated October, November and December 1999
from Alex Brown (Alex_Brown@ne.3com.com), Chris
Calabrese (chris_calabrese@yahoo.com), Chris M.
Lonvick (clonvick@cisco.com), Darren Reed
(darrenr@reed.wattle.id.au), and Mark Roth
(roth@feep.net). Mailing list archive available
from www.mail-archive.com/syslog-
sec%40employees.org.
[4] U.S. Department of Defense, Assistant Secretary
of Defense for Command, Control, Communications,
and Intelligence, "Department of Defense Trusted
Computer System Evaluation Criteria" (DoD
5200.28-STD). December 1985. Available from
www.radium.ncsc.mil/tpep/library/rainbow/5200.28-STD.html.
[5] Common Criteria for Information Technology
Security Evaluation Version 2.1 (International
Standard ISO/IEC 15408:1999), August 1999.
Available from
(www.radium.ncsc.mil/tpep/library/ccitse/ccitse.html).
[6] U.S. National Security Agency, Information
Systems Security Organization, "Labeled Security
Protection Profile," October 1999. Available
from
www.radium.ncsc.mil/tpep/library/protection_profiles/index.html.
[7] Private correspondence with Balazs Scheidler
INTERNET DRAFT Event Logging Requirements December 2000
(bazsi@balabit.hu), March 2000.
[8] Private correspondence with Mark D. Roth
(roth@feep.net), March 2000.
[9] U.S. National Security Agency, Information
Systems Security Organization, "Controlled Access
Protection Profile," October 1999. Available
from
www.radium.ncsc.mil/tpep/library/protection_profiles/index.html.
[10] Private correspondence with Chris M. Lonvick
(clonvick@cisco.com) relating to the IETF
Internet Draft he was writing at the time (since
released) on the syslog protocol, July 2000.
[11] B. Schneier and J. Kelsey, "Cryptographic Support
for Secure Logs on Untrusted Machines," The
Seventh USENIX Security Symposium Proceedings,
USENIX Press, January 1998, pp. 53-62.
Available from www.counterpane.com/secure-
logs.html.
[12] B. Schneier and J. Kelsey, "Minimizing Bandwidth
for Remote Access to Cryptographically Protected
Audit Logs," Second International Workshop on the
Recent Advances in Intrusion Detection (RAID
'99), September 1999. Available from
www.counterpane.com/auditlog2.html.
[13] Darren Reed, Nsyslogd home web page,
coombs.anu.edu.au/~avalon/nsyslog.html.
[14] Core-SDI, README file for the Secure Syslog
package, 1998. Available from www.core-
sdi.com/english/slogging/ssyslog-dl.html.
[15] J. Abela and T. Debeaupuis, "Universal Format for
Logger Messages," offered as an Internet Draft,
May 1999.
[16] Private correspondence with Nicolas Jombart of
Herve Schauer Consultants
(Nicolas.Jombart@hsc.fr), May 2000.
[17] BaliBit Computing, Syslog-ng home web page,
www.balabit.hu/products/syslog-ng.
8. Author's Address
C. Calabrese
26 Wellesley Road
Montclair, NJ, USA 07043
Phone: (201) 703-7218
EMail: chris_calabrese@yahoo.com
M. Arpad
EMail: mag@lme.linux.hu
Send comments regarding this memo to
chris_calabrese@yahoo.com and mag@lme.linux.hu.