Internet Draft R. Harrison
Document: <draft-rharrison-bulkldif-00.txt> J. Sermersheim
Intended Category: Proposed Standard Novell, Inc.
June, 1999
LDAP v3 Extensions for Bulk Transfer of LDIF Data
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts. Internet-Drafts are draft documents valid for a maximum of
six months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet- Drafts
as reference material or to cite them other than as "work in
progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
1. Abstract
This document defines an LDAP extension to efficiently import LDIF
into an LDAP server. The extension utilizes a protocol that
maximizes the speed with which LDIF data can be presented to the
LDAP server and yet allows the LDAP client to be kept apprised of
errors and the progress of its request.
2. Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
this document are to be interpreted as described in RFC-2119
[ReqsKeywords].
An LDIF stream is a data stream which conforms to the LDAP Data
Interchange Format (LDIF) [LDIF] with the additional restriction
that a LDIF record which defines an object MUST come before any
LDIF records which define children of that object.
This document describes the transmission of LDIF data from a client
to a server and the server's behavior of accepting or consuming the
LDIF data as well as processing that data. In talking about server
behavior, the terms consume and consuming refer to the server
accepting the LDIF stream. The terms process, and processing refer
to the server applying the changes held in the LDIF records to the
DIT.
Harrison & Sermersheim Expires December, 1999 1
LDAP v3 Bulk LDIF Transmission June, 1999
3. Elements of Protocol
The protocol of this extension is described using Abstract Syntax
Notation 1 (ASN.1) [ASN1] and is typically transferred using a
subset of ASN.1 Basic Encoding Rules [BER].
4. The Bulk LDIF Transmission Extension
Server support of the extension used for the bulk transmission of
LDIF streams is advertised by the presence of its OID,
"2.16.840.1.113719.1.1.102.1.0" in the supportedExtension attribute
of a server's root DSE.
4.1 Bulk LDIF Transmission Request
This extended request is sent in an LDAPMessage as described in 4.12
of [LDAPv3]. It is used by a client to initiate, transmit and
complete the transfer of LDIF data to the server. It optionally
allows specification of progress reports and which errors the server
should halt on.
4.1.1 Bulk LDIF Transmission ASN.1 Definition
The requestName is "2.16.840.1.113719.1.1.102.1.0". The
requestValue is an OCTET STRING, whose value is the BER encoding of
the following type:
BulkLDIFTransmission ::= SEQUENCE {
fileID [0] INTEGER DEFAULT 0,
transactionSizeHint [1] INTEGER OPTIONAL,
progressResponses [2] INTEGER DEFAULT 0,
criticalErrors [3] CriticalErrorMask OPTIONAL,
completedState [4] BOOLEAN DEFAULT TRUE,
data [5] OCTET STRING
}
CriticalErrorMask ::= SEQUENCE {
allExcept BOOLEAN DEFAULT TRUE,
SET OF ENUMERATED {
operationsError (1),
sizeLimitExceeded (4),
strongAuthRequired (8),
referral (10),
adminLimitExceeded (11),
confidentialityRequired (13),
noSuchAttribute (16),
undefinedAttributeType (17),
inappropriateMatching (18),
Harrison & Sermersheim Expires December, 1999 2
LDAP v3 Bulk LDIF Transmission June, 1999
constraintViolation (19),
attributeOrValueExists (20),
invalidAttributeSyntax (21),
noSuchObject (32),
aliasProblem (33),
invalidDNSyntax (34),
inappropriateAuthentication (48),
insufficientAccessRights (50),
busy (51),
unavailable (52),
unwillingToPerform (53),
loopDetect (54),
namingViolation (64),
objectClassViolation (65),
notAllowedOnNonLeaf (66),
notAllowedOnRDN (67),
entryAlreadyExists (68),
objectClassModsProhibited (69),
affectsMultipleDSAs (71),
other (80)
}
}
4.1.2 Bulk LDIF Transmission fileID description
The fileID field is used to synchronize the transmission of LDIF
data when more than one Bulk LDIF Transmission requests are used.
When a client sends the first BulkLDIFTransmission request, this
MUST be set to 0. If subsequent Bulk LDIF Transmission requests are
sent to complete the transmission of the LDIF data, this value MUST
contain the value of the fileID field returned by the server in the
File ID Response. Servers use this value to distinguish between
possible asynchronous Bulk LDIF Transmission requests from the same
client.
4.1.3 Bulk LDIF Transmission transactionSizeHint description
The transactionSizeHint field specifies the number of LDIF records
the client would like applied to the LDAP server's DIT. This option
is provided to allow the server to improve the performance of
loading data. If no value is specified, the server is free to choose
any value it wishes. If the server does not support transactions of
the size requested by the client, it can ignore this value.
4.1.4 Bulk LDIF Transmission progressResponses description
The progressResponses field specifies the frequency with which the
server sends BulkLDIFProgress responses. These are asynchronous
responses, which the client may consume in order to discover the
number of records that the server has processed. If this is set to 0
(the default), no progress responses are sent by the server as it
processes the LDIF data. The value specifies the number of records
Harrison & Sermersheim Expires December, 1999 3
LDAP v3 Bulk LDIF Transmission June, 1999
to be processed between sending BulkLDIFProgress responses. If the
server completes the processing of LDIF data before the non-zero
number has been reached, it sends a final BulkLDIFProgress response.
4.1.5 BulkLDIFTransmission criticalErrors description
The criticalErrors field allows the client to specify which errors
will cause the server to stop processing and consuming LDIF data.
For instance, a client may want the server to ignore
entryAlreadyExists errors and continue to consume and process the
LDIF data. If this field is not specified, all errors will be
treated as critical. If the allExcept field is TRUE it means that
all errors are critical except those listed.
When a critical error occurs, the server MUST stop processing the
LDIF stream and send a Bulk LDIF Error response with the critical
error field set to TRUE. It MUST also refuse any subsequent Bulk
LDIF Transmission requests that use the same fileID value.
There is one error, malformedLdifData (91), which is always treated
as critical.
4.1.6 Bulk LDIF Transmission completedState description
The completedState is set to TRUE if there are to be no more Bulk
LDIF Transmission requests sent for this LDIF stream. If a request
is sent with this value set to FALSE, the server MUST send a Bulk
LDIF FileID Response to the client with a unique value in the fileID
field before processing any LDIF data. If a request is sent with
this value set to TRUE, the server sends a Bulk LDIF Consumed
Response to the client, it finishes processing the LDIF stream and
then sends a Bulk LDIF Processed Response to the client.
4.1.7 Bulk LDIF Transmission data description
The data field contains part or all of the LDIF data as it is
contained in the LDIF file. Each octet represents one 8 bit
character of the file.
Clients may wish to transmit an LDIF stream in small portions to
achieve a performance gain. Consider the scenario where a very large
LDIF file is being processed. The client can send a manageable piece
of data to the server. While the server is processing the first set
of data, the client can asynchronously send remaining portions.
If the data from the LDIF file is not specified in its entirety, the
completedState MUST be set to FALSE.
The data may be broken at any place in the LDIF stream. Server
implementations MUST NOT rely on a client sending a complete LDIF
record or even a complete line of data in a transmission. The
Harrison & Sermersheim Expires December, 1999 4
LDAP v3 Bulk LDIF Transmission June, 1999
resulting concatenation of all transmissions of a set of LDIF data
MUST conform to the definition of an LDIF stream in section 2. This
allows a client to break up and transmit a large LDIF file without
knowledge of the LDIF format.
4.2 Bulk LDIF response
This extended response is sent in an LDAPMessage as described in
4.12 of [LDAPv3]. A server sends this response for several reasons
which are enumerated below.
4.2.1 Bulk LDIF response ASN.1 Definition
BulkLDIFResponse ::= {
fileID INTEGER,
CHOICE {
bulkLdifFileID [0] NULL,
bulkLdifProgress [1] BulkLdifProgress,
bulkLdifError [2] BulkLdifError,
bulkLdifConsumed [3] INTEGER,
bulkLdifProcessed [4] INTEGER
}
}
BulkLdifProgress ::= SEQUENCE {
recordNumber [0] INTEGER,
lastProcessedDN [1] LDAPDN
}
BulkLdifError ::= SEQUENCE {
errorCode [0] BulkLdifErrors,
criticalError [1] BOOLEAN DEFAULT FALSE,
errorRecordNumber [2] INTEGER,
errorDN [3] LDAPDN
}
BulkLdifErrors ::= ENUMERATED {
operationsError (1),
sizeLimitExceeded (4),
strongAuthRequired (8),
referral (10),
adminLimitExceeded (11),
confidentialityRequired (13),
noSuchAttribute (16),
undefinedAttributeType (17),
inappropriateMatching (18),
constraintViolation (19),
attributeOrValueExists (20),
invalidAttributeSyntax (21),
noSuchObject (32),
Harrison & Sermersheim Expires December, 1999 5
LDAP v3 Bulk LDIF Transmission June, 1999
aliasProblem (33),
invalidDNSyntax (34),
inappropriateAuthentication (48),
insufficientAccessRights (50),
busy (51),
unavailable (52),
unwillingToPerform (53),
loopDetect (54),
namingViolation (64),
objectClassViolation (65),
notAllowedOnNonLeaf (66),
notAllowedOnRDN (67),
entryAlreadyExists (68),
objectClassModsProhibited (69),
affectsMultipleDSAs (71),
other (80),
malformedLdifData (91)
}
The fileID field in all responses is a server-generated value that
it uses to uniquely identify this bulk LDIF stream.
4.2.1.1 Bulk LDIF File ID response
This response is sent by the server after the first Bulk LDIF
Transmission request is received. The client must use the file ID
received in this response in all subsequent bulk transmission
requests for this LDIF stream.
4.2.1.2 Bulk LDIF Progress response
This response is sent periodically by the server as requested by the
client in the first Bulk LDIF Transmission request. When this
response is sent by the server, the LDAPResult field MUST contain a
value of success (0).
The recordNumber contains a count of the records processed by the
server, and the lastProcessedDN contains the DN of the last record
processed by the server.
The Bulk LDIF Progress response does not contain error
notifications. Error notifications are sent via the Bulk LDIF Error
response.
4.2.1.3 Bulk LDIF Error response
This response is sent by the server whenever an error in consuming
or processing an LDIF record is encountered. When this response is
sent, the LDAPResult field MUST contain a value of operationsError
(1).
Harrison & Sermersheim Expires December, 1999 6
LDAP v3 Bulk LDIF Transmission June, 1999
The errorCode should contain the actual error value generated by the
server in attempting to consume or process the LDIF record. If the
criticalError flag is FALSE, it indicates that this error was
ignored by the server and the server continued processing the LDIF
stream at the next LDIF record. If the criticalError flag is TRUE,
it indicates that the server has stopped processing LDIF records,
any consumed but unprocessed LDIF data has been discarded, and the
client MUST NOT attempt to send further LDIF data for this LDIF
stream.
The errorRecordNumber is the record number in the LDIF stream in
which the error occurred, and the errorDN is the DN contained in
that record. In the case of data corruption, the DN may be empty.
4.3.1.4 Bulk LDIF Consumed response
This response is sent by the server to acknowledge receipt of all
LDIF records within an entire stream. When this response is sent,
the LDAPResult field MUST contain a value of success (0). The
bulkLdifConsumed field indicates the total number of records counted
while consuming the LDIF stream.
4.3.1.5 Bulk LDIF Processed response
This response is sent by the server to acknowledge that all LDIF
records within an entire stream have been applied to the server's
DIT. When this response is sent, the LDAPResult field MUST contain a
value of success (0). The bulkLdifProcessed field indicates the
total number of records applied to the DIT from the LDIF stream.
5. Security Considerations
The LDIF data format may be used to textually represent all
information in a directory including sensitive information such as
passwords, credit card numbers, etc. This protocol makes no
provision for security of the LDIF stream transmitted to an LDAP
server. Clients SHOULD take appropriate measures to ensure that
sensitive LDIF data is not transmitted in cleartext form by using
encrypted connections or other mechanisms.
6. References
[LDIF] Gordon Good, "LDAP Data Interchange Format (LDIF)", INTERNET-
DRAFT: draft-good-ldap-ldif-04.txt, 22 June 1999.
[ASN1] ITU-T Rec. X.680, "Abstract Syntax Notation One (ASN.1) -
Specification of Basic Notation", 1994.
[BER] ITU-T Rec. X.690, "Specification of ASN.1 encoding rules:
Basic, Canonical, and Distinguished Encoding Rules", 1994.
Harrison & Sermersheim Expires December, 1999 7
LDAP v3 Bulk LDIF Transmission June, 1999
[LDAPv3] Wahl, M., Howes, T., and S. Kille, "Lightweight Directory
Access Protocol (v3)", RFC 2251, December 1997.
[ReqsKeywords] Scott Bradner. "Key Words for use in RFCs to Indicate
Requirement Levels". RFC 2119.
7. Author's Addresses
Roger Harrison
Novell, Inc.
122 E. 1700 S.
Provo, UT 84606
+1 801 861 2642
roger_harrison@novell.com
Jim Sermersheim
Novell, Inc.
122 E. 1700 S.
Provo, UT 84606
+1 801 861 3088
jimse@novell.com
Full Copyright Statement
"Copyright (C) The Internet Society (date). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implmentation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph
are included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into
Harrison & Sermersheim Expires December, 1999 8