2.1.8 LDAP Duplication/Replication/Update Protocols (ldup)

NOTE: This charter is a snapshot of the 47th IETF Meeting in Adelaide, Australia. It may now be out-of-date. Last Modified: 02-Mar-00

Chair(s):

Chris Apple <capple@att.com>
John Strassner <johns@cisco.com>

Applications Area Director(s):

Keith Moore <moore@cs.utk.edu>
Patrik Faltstrom <paf@swip.net>

Applications Area Advisor:

Patrik Faltstrom <paf@swip.net>

Mailing Lists:

General Discussion:ietf-ldup@imc.org
To Subscribe: ietf-ldup-request@imc.org
In Body: subscribe
Archive: http://www.imc.org/ietf-ldup/

Description of Working Group:

As LDAPv3 becomes more widely deployed, replication of data across servers running different implementations becomes an important part of providing a distributed directory service. However, the LDAPv3 community, to date, has focused on standardizing the client-server access protocol. Therefore, this group will standardize master-slave and multi-master LDAPv3 replication as defined below:

o Multi-Master Replication - A replication model where entries can be written and updated on any of several replica copies, without requiring communication with other masters before the write or update is performed.

o Master-Slave, or Single-Master Replication - A replication model that, assumes only one server, the master, allows write access to the replicated data. Note that Master-Slave replication can be considered a proper subset of multi-master replication.

The WG's approach is to first develop a set of requirements for LDAPv3 directory replication and write an applicability statement defining scenarios on which replication requirements are based. An engineering team was formed consisting of different vendors and the co-chairs in order to harmonize the existing approaches into a single standard approach. All of these have been accomplished during the pre-working group stage. It should be noted, however, that replication using heterogeneous servers is dependent on resolving access control issues, which are the domain of other working groups.

The new replication architecture support all forms of replication mentioned above. Six areas of working group focus have been identified through LDUP Engineering Team discussions, each leading to one or more documents to be published:

o LDAPv3 Replication Architecture

This documents a general-purpose LDAPv3 replication architecture, defines key components of this architecture, describes how these key components functionally behave, and describes how these components interact with each other when in various modes of operation.

o LDAPv3 Replication Information Model

Defines the schema and semantics of information used to operate, administer, maintain, and provision replication between LDAPv3 servers. Specifically, this document will contain common schema specifications intended to facilitate interoperable implementations with respect to:

+ replication agreements + consistency models + replication topologies + managing deleted objects and their states + administration and management

o LDAPv3 Replication Information Transport Protocol

LDAPv3 extended operation and control specifications required to allow LDAPv3 to be used as the transport protocol for information being replicated.

o LDAPv3 Mandatory Replica Management

Specifications required to allow administration, maintenance, and provisioning of replicas and replication agreements. These specifications may take the form of definitions for LDAPv3 extended operations, controls, and/or new schema elements.

o LDAPv3 Update Reconciliation Procedures

Procedures for detection and resolution of conflicts between the state of multiple replicas that contain information from the same unit of replication.

o LDAPv3 Profiles

Including the LDAPv3 Replication Architecture, Information Model, Protocol Extensions, and Update Reconciliation Procedures for:

+ LDAPv3 Master-Slave Directory Replication + LDAPv3 Multi-Master Directory Replication

The LDUP WG Chairs will assign to one or two persons to be official LDUP WG liasons to ITU, to monitor X.500 replication work in ITU, and to coordinate the work of the LDUP WG with similar work in ITU.

Goals and Milestones:

Nov 98

  

Submit I-D on LDAPv3 Directory Replication Requirements.

Nov 98

  

Submit Internet-Draft on LDAPv3 Replication Information Model

Feb 99

  

Submit I-D on LDAPv3 Update Reconciliation Procedures.

Feb 99

  

Revise I-D on LDAPv3 Directory Replication Requirements.

Apr 99

  

Revise I-D on LDAPv3 Replication Architecture.

Aug 99

  

Revise I-D on LDAPv3 Replication Architecture.

Aug 99

  

Submit I-D on LDAPv3 Mandatory Replica Management.

Aug 99

  

Submit I-D on LDAPv3 Replication Information Transport Protocol.

Aug 99

  

Revise I-D on LDAPv3 Replication Information Model.

Sep 99

  

LDAPv3 Directory Replication Requirements I-D goes to WG Last Call as Informational.

Sep 99

  

Submit I-D on LDAPv3 Multi-Master Replication Profile.

Sep 99

  

Submit I-D on LDAPv3 Master-Slave Replication Profile.Submit I-D on LDAPv3 Master-Slave Replication Profile.

Nov 99

  

LDAPv3 Replication Information Model I-D goes to WG Last Call as Proposed Standard.

Nov 99

  

LDAPv3 Replication Architecture I-D goes to WG Last Call as Informational.

Nov 99

  

LDAPv3 Update Reconciliation Procedures I-D goes to WG Last Call as Proposed Standard.

Nov 99

  

LDAPv3 Replication Information Transport Protocol I-D goes to WG Last Call as Proposed Standard.

Mar 00

  

LDAPv3 Mandatory Replica Management I-D goes to WG Last Call as Proposed Standard.

Mar 00

  

LDAPv3 Master-Slave Replication Profile I-D goes t WG Last Call as Proposed Standard.

Mar 00

  

LDAPv3 Multi-Master Replication Profile I-D goes to WG Last Call as Proposed Standard.

Internet-Drafts:

No Request For Comments

Current Meeting Report

LDUP Meeting Minutes for the 47th Meeting of the IETF
Minutes recorded by John Strassner

First and most importantly, humble thanks to all working group participants (which was the overwhelming majority) that stayed over 20 minutes late in order to finish our agenda.

The agenda was discussed. We inserted one change to the agenda, the discussion of an alternate proposal to the URP draft. This alternative was presented to the group at the start of the meeting, and the agenda was changed to accommodate it due to its potential ramifications if the working group accepts the work. The revised agenda is below (note that all file names should be prefixed with http://www.ietf.org/internet-drafts):

7. Agenda Bashing
8. Brief Discussion of Alternate URP Proposal

9. General Admonishment for Authors Being Late ;-)
10. LDAP Replication Architecture

11. LDAP V3 Replication Requirements

12. LDUP Replication Information Model

13. LDUP Update Reconciliation Procedures

14. LDAP Subentry Schema

15. The LDUP Replication Update Protocol

16. Drafts that are Missing In Action:

17. Discussion of addition of LBURP to the Charter
18. Discussion of addition of LCUP Work to the Charter

1. Discussion of Agenda Items (John Strassner)
The agenda was discussed. A new item was added: discussion of a new alternate URP proposal and its effects on the
requirements document. The rest of the agenda remained unchanged.

2. Brief Discussion of an alternate URP Proposal (moderated by John)

This item came up because the LDUP Replication Requirement Document completed WG last call, and actually should have gone to IETF Last Call. We had decided to put this document to IETF Last Call at the meeting. However, there was one major objection raised by Mr. Langer. He was concerned that the requirements are too open and subject to interpretation in one specific area (atomicity of updates), and that his alternative method for update resolution would therefore be ruled out. It was decided to do the following:

- since nothing can be formally considered by the working
group until it is in the form of an Internet Draft, we
decided to wait one week (till 7 April) to allow Mr.
Langer to publish his URP alternate approach as an
individual submission
- in addition, Mr. Langer should write a brief e-mail that
explicitly defines the problem with the requirement doc.
Note that this does NOT mean that Mr. Langer has to redo
and/or present an alternate requirements draft.

Once these two steps have been completed, the working group will discuss the points on the mailing list. We would like to decide by the end of the month at the latest whether to go forward with the existing requirements draft and URP proposal, or to delay these while we adjust the requirements document and then either add Mr. Langer's URP document, or to replace our existing document with that of Mr. Langer.

3. General Admonishment ;-) (John)

Your co-chairs are concerned that we're showing the signs of starting to die a flailing death. So, if this work is important for your LDAP implementations, please redouble your efforts to move these drafts along. Your co-chairs are here to help and facilitate this. We will be collecting expected times of updating the drafts today, so that we can update our charter. Working groups have, can, and will be killed for lack of activity, so please try and keep these new sets of dates.

4) LDAP Replication Architecture (Uppilli)
file: draft-ietf-ldup-model-02.txt

(Intro from John)
This draft completed working group last call. The lone item to fix was a removal of references to other WG deliverables (per joint recommendation of the WG Chairs and the Applications Area Directors). This is because this document, being an architecture document, is supposed to stand on its own and not be held up waiting for other deliverables to clear IESG Review, IETF Last Call, and the RFC Editor.

(Uppilli)
The current status is that the above changes were made and the document was resubmitted. The authors tried to decouple references, but in their opinion additional work needs to be done to ensure that those documents are tightly coupled to this architecture document. In addition, Helmut pointed out that there are some errors with respect to naming contexts. He also pointed out that since some things operate fundamentally differently in a multi-master environment, this should be pointed out and affect the architecture document.

A brief call for input was taken, and other people are willing to add requirements. Plus, the other architecture authors should also contribute.

It was decided to solicit general comments up to 24 April, and then reissue this document in mid-May.

5) LDAP V3 Replication Requirements (Ellen)
file: draft-ietf-ldup-replica-req-02.txt

This document also successfully completed working group last call, but has not yet been sent to an IETF last call. And now, we have an objection from Mr. Langer. The basic problem is that Mr. Langer wants to offer up an alternate replication method, one that is different from URP. He is worried that the replication requirements draft, in its current form, would prohibit his document from being considered. This is because, in his view, the document is ambiguous. The ambiguity lies in the fact that although transactions are beyond the scope of the current work, nothing will be done to add in transactions. In addition, the term atomic update as implemented in a single master means that you were reading just that entry, and that no interleaved operations carried out. The replication requirements document (in its present form) allows interleaving of operations, in anticipation of multi-master operation.

Recommendation of the chair and AD: Until an Internet draft is an RFC there is always time to discuss things. However, it is not fruitful to discuss anything until a formal counter-proposal, in the form of an Internet Draft, is submitted (as an individual submission) for consideration by the working group. Thus, it is incumbent on Mr. Langer to produce such an Internet Draft.

Once this new draft is submitted, the working group will consider it. The first order of business is for the replication requirements team and authors to respond on the mail list, and then for the working group to consider whether Mr. Langer's concerns are justified. If they are, then the requirements draft will have to be modified to take this into account, and another working group last call issued. If not, then the requirements draft will go to IETF last call either as is, or with a small editorial modification that further clarifies the requirements draft in response to Mr. Langer's objections. The working group will make a decision by Easter. Note that this is independent of the URP decision.

6) LDUP Replication Information Model (Ed)
file: draft-ietf-ldup-infomod-00.txt

(Chair intro): This was supposed to go through WG Last Call a while ago. However, it is doubtful that it is ready to go in its current form.

(Ed): The Engineering Team met and is simplifying this draft even more. The UUID spec was a dangling reference, but the working group decided (thanks Harald for chipping in here) that we can use the UUID ISO spec. The reference for this spec is:

http://www.opengroup.org/onlinepubs/009629399/apdxa.htm

Ed notes that the process of trying to describe scheduled policy is really hard. We can start to do this by perhaps defining a cron-like process with hysteriesis. However, it isn't clear that this is suitable for all applications. In fact, the Engineering Team is favoring leaving this undefined. If we do this, then there are two classes and a lot of attributes that don't need to be specified in this document. The current proposal is to define a (may) DN attribute that may point to a scheduler class. When absent, the default is to send the replication immediately.

(<Begin Editor's note>
please refer to the subsequent exchange between Harald and Ed regarding particulars of this. Ed points out that the Engineering team are actively trying to avoid the cron(1) directory schema (or at least punt to a separate document). Ed's proposal is thus:

scheduling policy information MAY be described in
published schema and be referenced by
replicationAgreementSubentries governed by entries
(or groups of entries) using such schema descriptions,
but such schema descriptions are outside the scope of
LDUP, and will generally be implementation specific
until concensus develops on what constitutes good
schema design for them.

LDUPers, please comment on the list.
</End Editor's note>)

In Washington, we decided to eliminate ldapAccessPoint in favor of replicaURI. We moved the replicaID translation table to the protocol spec. This is one of the things that we need to ensure is updated in all of the appropriate draft.

The current draft is defined so that the replication agreement subentries use name subordination to define whose replica this is. This brought up a discussion on which naming attributes to use. Here is a summary:

Mark notes that some vendors have objected to using cn for naming ldapSubEntry entries in the directory. The primary objection is that these entries are not normally visible to users and administrators. Thus, you get the rather ugly problem that naming clashes with other entries named by cn will occur, and the administrator or user won't have a clue as to why this is happening.

In addition, many ldapSubEntry entries will be created automatically by software. If this is to succeed, then we must guarantee that name clashes are avoided.

The group defined the following possible solutions:

1) define a new attribute (name TBD) of type DirectoryString, for exclusive use of naming ldapSubEntry entries
2) leave supplying of the naming attribute to developers who use an ldapSubEntry class to hold their information, but this seems to have problems because the ldapSubEntry class is a structural class and therefore probably requires a specific naming rule that will end up interfering with user definitions
3) punt, and leave it the way X.500 defines it, and leave experiments surrounding solving this problem to future innovators.
4) keep cn as the naming attribute but recommend a specific convention for naming subentries that makes a name clash unlikely.

A brief summary of the opinions of the group is:

1) Option 1 moves this farther away from the X.500 definition, which is OK if that is really what we want to do. However, it doesn't seem that we should change just for the sake of changing. So if someone really wants this option, it is incumbent on them to come up with a compelling reason (last sentence from Editor).
2) Option 2 seems an even further departure from X.500, and seems to be recommending that ldapSubEntry be an
AUXILIARY, not a STRUCTURAL, class.
3) Option 3 seems to be a safe decision. If people can't come up with compelling scenarios, then there is no
need to change.
4) This option seems next most feasible, though there is some concern as to how the naming convention can be specified so as to always avoid collisions.

7) LDUP Update Reconciliation Procedures (Steven)
file: draft-ietf-ldup-urp-02.txt

This draft did seem ready for last call, but now we may need to reconsider it given Mr. Langer's alternate proposal. The authors are looking at making a future revision to accommodate the updating of partial replicas. Steven and Alison point out that the main difference between URP and other approaches is whether conflicts are merged or resolved immediately. Steven and Alison also note that this subject keeps coming up. Therefore, Steven and Alison have volunteered to write up a new document that describes why URP ended up the way it did. The working group members that that this was a good idea, and Patrick OKed it. Chairs to revise the charter and send to list.

8 LDAP Subentry Schema (Ed)
file: draft-ietf-ldup-subentry-01.txt

This document is ready for last call, but LDAPEXT wants additional information and guidance stating that auxiliary classes should be used wherever possible instead of structural classes (editor's note: remember that this document is going through a joint last call between the LDUP and LDAPEXT working groups). When that is revised, this doc should be ready for last call.

However, the naming problem discussed above rears its ugly head here too. So if we change the naming convention, then we need to update this document. So at the very least we need some weasel words to allow this.

Conclusion: this last issue needs to be taken to the list. Ed to write mail message, everyone please respond and help, else we will use X.500's definition.

Target date for new publication: end of May

9) The LDUP Replication Update Protocol
file: draft-ietf-ldup-protocol-00.txt and
file: draft-ietf-ldup-framing-00.txt

The main change since the last issue of this draft is to split the ldup-protocol-00.txt draft into 2 drafts:

Ldup-protocol-01.txt and ldup-framing-00.txt

The framing draft defines four framing operations: start framed protocol request and response, and end framed protocol request and response. This is intended to group together a set of related operations. For example, in URP, there would be a series of URP actions. Note that a session is defined as Start framed protocol request contains a protocol OID and a protocol-specific payload. LBURP, for example, specifies this protocol OID and specifies the payload format.

Framing does not address push or pull models that are a function of the protocol implementation.

Open questions:
- Does protocol-01.txt enumerate all necessary error conditions (probably not)
- Anything else? (replica ID translation table, advertisement)

A general question was raised as to why the framing document is necessary (i.e., why not just use an extended response)? The concern is that you might need more semantics than just a Start doing something, and then tell me when you're done. Also, there is concern that since we only have 2 customers, we might not get it right.

Conclusion: take this to the list. Discussion period should go no later than middle of May. Then, we need a damage assessment, and to decide if we merge the drafts back into one, or keep them separate.

10) Drafts that are missing in action

10a) LDAPv3 Mandatory Replica Management I-D (Mark)

The LDAPv3 Mandatory Replica Management I-D is being turned over to Ed Reed because Mark has negative cycles ;-) . Ed's target date for revising this document is the end of June (to ensure time to finish the other important work that he is already committed to).

10b) LDAPv3 Master-Slave Replication Profile I-D (Uppilli/Ed)

These documents will be combined into one, with Uppilli taking the lead. Solicited and received help to fill out some of the sections. New time frame is the end of June to release this document.

10c) LDAPv3 Multi-Master Replication Profile I-D (Uppilli/Ed)

This draft has stagnated somewhat because it is still unclear how applications will make use of this information. The original motivation of this draft was to be able to answer questions on configuration, and guide the implementor in making decisions that support the configuration of the directory server to single or multi-master replication. After some discussion, we decided that this should be tied closely to the requirements document, and to have use cases drive their functionality. Also, we should talk about who should use single- vs. multi-master replication, and how the inherent differences between these two modes affect the applications that use them. Uppilli needs more input as to what type of applications would use this and how this could break an application. Alison will help, as will the co-chairs. Purpose: see what types of operations are different between single- and multi-master. This will be written as one document, not two. Time frame: end of June.

11) Discuss LBURP (Roger)

The latest version of this draft include using the LDUP Framing draft, removing LDUPisms (e.g. replica update vectors), adding sequencing (to help implementation we need to determine which updates were sent in what order), and adding grouping of updates (for better wire efficiency, especially on responses)

Remaining questions:
Need to tighten error handling procedures (what happens if an error occurs in the middle of a set of operations). Also need to consider how much parallelism is desired between LDUP and LBURP.

Should this be added as a WG item? Unclear right now. Roger to revise it again and we will ask again on or before the next IETF meeting.

12) Discuss addition of LCUP to charter (Olga)

The purpose of LCUP is to synchronize LDAP clients with content stored by LDAP servers. As such, there are three main problem areas that are addressed: (1) mobile clients that maintain local data caches, (2) meta-directory applications, and (3) event triggers (both triggered as well as persistent search).

Problem areas that are not addressed include server to server synchronization (addressed by LDUP).

LCUP combines features of DirSync, Persistent search and Triggered search. Its motivation is that the ideal solution needs parts of each of these solutions. So, LCUP is intended to replace persistent and triggered search. DirSync has a number of desirable features that are included in LCUP for convenience sake, but LCUP does not have all of the functionality of DirSync.

The protocol supports one-way synchronization only. Server does not maintain any state info on behalf of the clients. Clients maintain the state information passed to them by the server in a cookie.

A key point is that LCUP does not support pre-defined agreements. Rather, it is up to the client to decide from which server and when to get the information. Thus, the client always initiates the synchronization session, and the client is always responsible for pulling data from the server that it selects.

The specific protocol elements proposed include four extended operations: a client update control, an entry update control, a client update done control, and a stop client update.

There are three basic ways of using LCUP: event triggering, non-persistent, and persistent synchronization. Please see the draft for more specific information.

There are a number of features under discussion. These are features that have been proposed in one or more of the persistent search, triggered search, and dirSync proposals, but which are not currently implemented in LCUP. The reason is that each of these are very difficult to implement, and so we are trying to avoid implementing these unless there is an important need for these. They are as follows:

- Persistent search defines a change type. This is defined in persistent search. It defines a reason for return to each entry sent to the client. Hard to implement for historical reasons.
- Sending changes: present in DirSync; only modified attributes (as opposed to all attributes) are sent.
- Size limit: present in DirSync; specifying amount of data that can be sent to the client. LCUP prefers to use a standard LDAP mechanism instead.
- Data ordering: present in DirSync; guarantees that parent is sent before child for adds and vice-versa for deletes. Very hard to implement, may not catch all problems (e.g., DN pointers).

LCUP is scoped by a single LDUP replica.

One thing that needs better definition is access control (e.g., if access control changes, how does that affect the client?). Note that a number of items are currently unspecified, such as:

- Access control enforcement on the data
- How to restrict the use of the protocol to trusted clients
- Mechanism to identify and disconnect malicious clients
- Server behavior is not specified for the case where data visibility changes due to access control changes
- Proper behavior is not guaranteed if access control on the data is changed from a more restrictive to a less restrictive one

Disposition? Mark would like to see this be done in LDUP, but is willing to accept it as a formal work item if this working group doesn't want to accept it. Poll of the room showed overwhelming majority in favor of accepting this. Chairs to describe this work in new charter, which is to be discussed in the list.

Slides

LDAP Client Update Protocol (LCUP)