NOTE: This charter is a snapshot of the 45th IETF Meeting in Oslo, Norway. It may now be out-of-date. Last Modified: 04-Jun-99
Chair(s):
Chris Apple <capple@att.com>
John Strassner <johns@cisco.com>
Applications Area Director(s):
Keith Moore <moore@cs.utk.edu>
Patrik Faltstrom <paf@swip.net>
Applications Area Advisor:
Patrik Faltstrom <paf@swip.net>
Mailing Lists:
General Discussion:ietf-ldup@imc.org
To Subscribe: ietf-ldup-request@imc.org
In Body: subscribe
Archive: http://www.imc.org/ietf-ldup/
Description of Working Group:
As LDAPv3 becomes more widely deployed, replication of data across servers running different implementations becomes an important part of providing a distributed directory service. However, the LDAPv3 community, to date, has focused on standardizing the client-server access protocol. Therefore, this group will standardize master-slave and multi-master LDAPv3 replication as defined below:
o Multi-Master Replication - A replication model where entries can be written and updated on any of several replica copies, without requiring communication with other masters before the write or update is performed.
o Master-Slave, or Single-Master Replication - A replication model that assumes only one server, the master, allows write access to the replicated data. Note that Master-Slave replication can be considered a proper subset of multi-master replication.
The WG's approach is to first develop a set of requirements for LDAPv3 directory replication and write an applicability statement defining scenarios on which replication requirements are based. An engineering team was formed consisting of different vendors and the co-chairs in order to harmonize the existing approaches into a single standard approach. All of these have been accomplished during the pre-working group stage. It should be noted, however, that replication using heterogeneous servers is dependent on resolving access control issues, which are the domain of other working groups.
The new replication architecture support all forms of replication mentioned above. Six areas of working group focus have been identified through LDUP Engineering Team discussions, each leading to one or more documents to be published:
o Abstract Model of LDAPv3 Replication
This documents a general-purpose LDAPv3 replication architecture, defines key components of this architecture, describes how these key components functionally behave, and describes how these components interact with each other when in various modes of operation
o LDAPv3 Replication Information Model
Defines the schema and semantics of information used to operate, administer, maintain, and provision replication between LDAPv3 servers. Specifically, this document will contain common schema specifications intended to facilitate interoperable implementations with respect to:
+ replication agreements + consistency models + replication topologies + managing deleted objects and their states + administration and management
o LDAPv3 Replication Information Transport Protocol
LDAPv3 extended operation and control specifications required to allow LDAPv3 to be used as the transport protocol for information being replicated
o LDAPv3 Mandatory Replica Management
Specifications required to allow administration, maintenance, and provisioning of replicas and replication agreements. These specifications may take the form of definitions for LDAPv3 extended operations, controls, and/or new schema elements.
o LDAPv3 Conflict Detection and Resolution Procedures
Procedures for detection and resolution of conflicts between the state of multiple replicas that contain information from the same unit of replication.
o LDAPv3 Profiles
Including the Abstract Replication Model, Information Model, LDAPv3 Protocol Extensions, and Conflict Detection and Resolution Procedures for:
+ Master-Slave LDAPv3 Directory Replication + Multi-Master LDAPv3 Directory Replication
The LDUP WG Chairs will assign to one or two persons to be official LDUP WG liasons to ITU, to monitor X.500 replication work in ITU, and to coordinate the work of the LDUP WG with similar work in ITU.
Goals and Milestones:
Nov 98 |
|
LDAPv3 Directory Replication Requirements and Applicability Statement I-D goes to WG Last Call as Informational. |
Nov 98 |
|
Submit Internet-Draft on LDAPv3 Replication Information Model |
Nov 98 |
|
Submit I-D on LDAPv3 Mandatory Replica Management |
Dec 98 |
|
Submit I-D on LDAPv3 Conflict Detection and Resolution Procedures |
Dec 98 |
|
LDAPv3 Abstract Model of Replication goes to WG Last Call as Informational. |
Dec 98 |
|
Submit I-D on LDAPv3 Multi-Master Replication Profile |
Dec 98 |
|
Submit I-D on LDAPv3 Master-Slave Replication Profile |
Dec 98 |
|
Submit I-D on LDAPv3 Replication Information Transport Protocol |
Jan 99 |
|
LDAPv3 Replication Information Model goes to WG Last Call as Proposed Standard. |
Mar 99 |
|
LDAPv3 Mandatory Replica Management goes to WG Last Call as Proposed Standard. |
Mar 99 |
|
LDAPv3 Replication Information Transport Protocol goes to WG Last Call as Proposed Standard. |
Apr 99 |
|
LDAPv3 Conflict Detection and Resolution Procedures goes to WG Last Call as Proposed Standard. |
Apr 99 |
|
LDAPv3 Master-Slave LDAPv3 Replication Profile goes to WG Last Call as Proposed Standard. |
Apr 99 |
|
LDAPv3 Multi-Master LDAPv3 Replication Profile goes to WG Last Call as Proposed Standard. |
Internet-Drafts:
· LDAP Replication Architecture
· LDUP Update Reconciliation Procedures
· LDAP V3 Replication Requirements
No Request For Comments
Meeting Minutes for the LDUP WG Meeting at Oslo, Norway
Minutes taken by John Strassner
Chris Apple - Proposed charter revision status
- Posted to the list, no questions on the list. No questions at the WG meeting.
Chris Apple - LDAPv3 replication requirements
- The document is in good shape, and we're getting ready for working group Last Call. All that remains is tidying up some loose ends.
Q: if you have sensitive info protected by ACLs, and then delete that tree, what happens on the next access? If the access control is done first, the sensitive info might be temporarily available. So we must ensure that the material protected by the access control must disappear before or at the same time as the access control. So the requirements document must be updated to make this more clear. Note that the problem is on full updates, and we might need a flag in the server to say that the information in the entire tree is temporarily unavailable.
This is a good comment, and should be added to the security considerations session, before we go to Last Call. It also looks like this should be added in the architecture specification and the transfer protocol draft.
Estimated to revise the requirements is within 2-4 weeks, followed by WG Last Call.
John Merrells - LDAPv3 replication architecture
- Changes made include the following (please see word attachment): sparse (where only a subset of the entries in a subtree are being replicated) and partial (combination of sparse and fractional) replication deferred - because this was extremely complicated when dealing with update vectors, and querying the list, it was evident that no one needed this right now
Pointed out that by dropping sparse completely, you have less functionality than the base X.500 implementation, since sparse for single-master is different than sparse for multi-master
Read-only replicas were considered, but even this is hard because the information to be replicated is flowing in and out of scope
State-based vs. log-based implementations - this is now TSRs, advisory info added into the document
Consistency model discussion - this is meant to address concerns from IESG. 3 levels of consistency
- single server is tight consistency (read and write serializability)
- single-master (serializability for writes (single-master) but not for reads) is medium consistency
- multi-master where reads and writes are both unserializable is loose consistency
So there is concern that clients will get this wrong. Seems like we need to present more guidance to people.
Paul recommends that any application that updates things that are related to each other (for example) must take explicit mechanisms to deal with this. This is really a paradigm shift for programmers - they must now take explicit measures in their applications. Put another way, LDAP should not be thought of as a single-file database.
LDAP constraints
- Info model constraints and operational behavior constraints The purpose of operational behavior constraints are to enforce the data model constraints. Therefore, every time you process an LDAP operation, you must impose these constraints and return an error. Now, in LDUP, you can't actually do this - you have to accept the operations and then fix it in the URP stage. Note also that an LDAP client is used to get the constraint violation immediately. But in a multi-master case this may not be true.
Glue entries for URP.
- Same as X.500. Entry has a name, and is used as a placeholder for something that will show up later.
Unique identifier: no prefix byte.
- Identifier is a UUID (remove types of identifiers)
Extended ops to protocol draft.
- Start and End operations have been moved out into the protocol draft
Replication start has ordering flag
- Natural for log-, but not for state-based, system. Problem is that in a state-based system, if you have a huge tree (10M entries) and the line goes down after some period (say, 5M), then you might have to start over.
Replica number table
- Traffic will contain CSNs that will contain the replica identifier, which is a lot of data. Hence, we've proposed sending a table that encodes this information.
Replication primitives - these are fully described in the URP doc
Update Resolution Procedures (ed note: it can be a bloodbath ;-):
- Distinguished names - can end up with same RDN twice, so we rename and move to Lost+Found. Note that this is a problem if entry is a UID - we need to think more about this.
- Orphaned entries - entry loses its parent, we recreate the parent as a glue entry, and glue entry's parent is the Lost+Found container
- Distinguished not present - someone has deleted the RDN. We add the value back in and tag it.
- Single valued attributes - could end up with multiple values when there is only supposed to be one
- Required attributes - required could end up flagged as missing
- Extra attributes - attributes that aren't permitted by the class
- Duplicate attribute values - must be resolved
- Ancestry graph cycle - every entry has to have a parent, but doesn't say that an entry can't be its own parent, or that there can't be a cycle in the DIT. We need a formal statement on these rules
ISSUES Section
Could possibly move sparse replica text into a new informational draft. This met with general approval, but it was recommended that this be done AFTER the existing documents are well on their way to last call
Prescriptive Metadata (John Merrells sent a message earlier on the list; would people please take a look at this and reply)
Replica Identifiers - would be nice to have a simple byte checking operation instead of a more complicated string comparison
Update ordering -
- State information representation and new control in order for clients to request state information (e.g., the CSNs). Right now, an ordinary LDAP client can't see the CSNs. Most vendor's clients will, however, have to see this info for admin purposes, so we might consider standardizing it.
Some of the info that we have added for LDUP really should be added as a draft to LDAPEXT
Ordering of updates to meta info - when you replicate, you need to ensure that this occurs at exactly the right time (e.g., all updates that loosen the schema, then updates, then updates that tighten the schema - reverse order for ACLs)
Ed Reed - LDUP replication information model
Ldup-infomod-00.txt was posted, since then an engineering meeting to shrink the schema took place. Shrinking is still continuing ;-)
One request was to not introduce new syntaxes if we could avoid it (because it introduces new matching rules, etc.). Therefore, we dropped ldapAccessPoint in favor of DN to the DSA holding the replica. This will now be an attribute on the replica subEntry class. (DSA entry that has network access of replica that contains this entry)
Drop ldapChangeSequenceNumber syntax and treat it as caseIgnoreString
Update description of CSN to reflect TSRs decision, and to reference the newly added ReplicaID table
Move ldapSubEntry to its own document - proposed to LDAPEXT WG, since it is generally useful
Defer ldapSearchFilter attribute
Still have to define scheduleSubEntry class
Update to follow by end of August
Some concern over storing network address of server in DSA, but this is generally the same problem as that for resolving DNS problems
Alison Payne - LDUP update reconciliation procedures
Major changes from previous draft:
- Previous draft had addition and creation CSNs for entries. This was deleted by simplifying the entry re-add semantics (i.e., add entry implies immediately add it immediately preceding removal). This was originally intended to help resolve conflicts for entries that moved in and out of scope, but didn't help. So we are nuking this.
- Glue entries created under Lost+Found to take the place of missing superiors for orphaned entries. This help simplifies things by no longer requiring explicit moves to Lost+Found for orphaned entries
- Terminology better aligned with the arch draft
- CSNs now include the least significant modification number component, a monotonically increasing modification number
- Pseudo-code descriptions rearranged to follow a more uniform pattern
Deferred to the next draft:
- References to a replication log will be removed when a complete state-based alternative spec is available
- appending of Unique Identifier to RDNs, when required to ensure uniqueness, to be made a local server action. This avoids generating explicit rename primitives
Overview of the basic processing structure
For each process in the URP, a fixup procedure can be invoked. Basically, a series of state checks are followed to determine how to process the primitive and whether there have been any constraint violations.
IF a constraint violation is detected, it is fixed and processing then continues. Once all state checks have been completed, the state change requested by the primitive operation may be carried out and propagated to other DSAs.
Attributes can be changed from not distinguished to distinguished, which means that deletions and additions can happen in one place but not in another place. This is a somewhat pathological case.
Delete Not Present has a number of issues. The basic problem is that an attribute deleted at one DSA becomes part of another DSA (i.e., each not having knowledge of the other). This complicates things a lot.
Alison presented two alternatives, and David Chadwick presented a third - detect conflict through rename. The advantage of David's proposal is that it solves the constraint violation, whereas the other two don't). Allison to post to the list.
Access control - may be present at one site but not at another site - so we need to track the access control work that is going on in LDAPEXT. Need to make this more explicit.
Ellen Stokes - Ldup-protocol (not yet published)
Status: no changes in overall approach described at Minneapolis IETF. There are begin and end operations that allow the receiving server to optimize operations (this is where the bulk LDIF proposal is synergistic). Protocol elements are defined. Currently writing descriptions of protocol semantics.
Timeline: will publish as an I-D within a few weeks. ASN.1 defined, beefing up descriptions and ensuring consistency of semantics. Some things we may want to make optional, which may not be applicable if you aren't doing full LDUP replication
Goal of being implemented with current LDAP APIs, but there may be some feedback needed to the API drafts to ensure that all operations can be implemented. May need to add some operations to the API draft. Two areas of concern are error codes and payload encoding.
Extra LDAP Features to Cope with Consistence Levels. Suggestion - Ed's work seems to be correct place to do this to help you understand differences between single- and multi-master profiles. Note that consistency and constraints are two different things. There are mistakes in underlying LDAP specs. For example, it doesn't say that a delete should fail if the entry doesn't exist. Part of this problem is that we like to refer to X.500, and that produces holes. LDUP has discovered these things, but it may be more appropriate to add them into LDAP to do better general good.
Consistency - some prior work in Bayou (session guarantees). Client can say things like it doesn't want to see anything other than a state that it has already read, etc. This would be a follow-on to the update protocols. LDUP addresses by unique identifier, not by DN. Now, if an LDAP client could also address by identifier, this could bring LDAP and LDUP closer together. But this is a change to the core protocol.
Consistency is different in a single- vs. multi-master deployment, but how would the client know what it is talking to? Perhaps we could handle this through explicit administrative configuration. Problem is that old clients will be broken, since they implicitly expect a tighter consistency than what is available in a multi-master system. Perhaps in the old client system, you could have a "preferred" master. Constraints that are needed by applications are not going to be managed by the directory.
What about support for extended LDAP operations? Example is a recursive delete - how would we replicate it? Our response up to now is to ask the people inventing such features how they would replicate it, and up to now, those people have backed down. This is really more of a process - the chairs of LDAPEXT and LDUP, for example, need to ensure that this doesn't happen
Gordon Good/John Merrells/Roger Harrison - Bulk LDIF
Motivation to move this into LDUP is because bulk operations aren't really suited for LDAP. We will have two new protocols, full update and incremental update. Another motivation is synchronization. Tim concurs. Authors to write up the draft formally and post to the list.
None received.