Last Modified: 2004-10-15
Done | Submit drafts of standard metrics for connectivity and treno-bulk-throughput. | |
Done | Submit a framework document describing terms and notions used in the IPPM effort, and the creation of metrics by the working group to IESG for publication as an Informational RFC. | |
Done | Submit documents on delay and loss to IESG for publication as Informational RFCs. | |
Done | Submit a document on connectivity to IESG for publication as an Informational RFC. | |
Done | Submit a document on bulk-throughput to IESG for publication as an Informational RFC. | |
Done | Submit draft on loss pattern sample metrics to the IESG for publication as an Informational RFC. | |
Done | Submit draft on metrics for periodic streams to the IESG for publication as a Proposed Standard RFC. | |
Done | Submit draft on IP delay variation to the IESG for publication as a Proposed Standard RFC. | |
Done | First draft for AS on one-way delay and loss. | |
Done | Submit draft on One-Way Active Measurement Protocol Requirements to the IESG for consideration as an Informational RFC. | |
Done | Create initial draft on a MIB for reporting IPPM metrics. | |
Done | Create initial draft on a packet reordering metric. | |
Aug 04 | Create initial draft on the definitions of link bandwidth capacity. | |
Done | Create draft on a One-Way Active Measurement Protocol that satisfies the requirements document. | |
Aug 04 | Submit draft on the One-Way Active Measurement Protocol to the IESG for consideration as a PS. | |
Oct 04 | Submit draft on a packet reordering metric to the IESG for Proposed Standard. | |
Dec 04 | Submit draft on a MIB for reporting IPPM metrics to the IESG for Proposed Standard. | |
Dec 04 | Submit link bandwidth capacity definitions draft to the IESG, for consideration as an Informational RFC. | |
Dec 04 | Collect implementation reports for RFCs 2678-2681 | |
Feb 05 | Discuss rechartering or ending working group. |
RFC | Status | Title |
---|---|---|
RFC2330 | I | Framework for IP Performance Metrics |
RFC2678 | E | IPPM Metrics for Measuring Connectivity |
RFC2679 | PS | A One-way Delay Metric for IPPM |
RFC2680 | PS | A One-way Packet Loss Metric for IPPM |
RFC2681 | PS | A Round-trip Delay Metric for IPPM |
RFC3148 | I | A Framework for Defining Empirical Bulk Transfer Capacity Metrics |
RFC3357 | I | One-way Loss Pattern Sample Metrics |
RFC3393 | PS | IP Packet Delay Variation Metric for IPPM |
RFC3432 | PS | Network performance measurement for periodic streams |
RFC3763 | I | A One-way Active Measurement Protocol Requirements |
IP Performance Metrics WG (ippm) Tuesday, November 9 at 0900-1130 ================================= The meeting was chaired by Henk Uijterwaal and Matt Zekauskas. Al Morton acted as scribe with additional notes by Matt; raw notes were edited into these minutes by the chairs. Agenda: 1. Review of Agenda and Status of Milestones 2. One-Way Active Measurements Protocol 3. Reordering drafts 4. Implementation reports for RFC2678-2681. 5. Multiparty Communications Parameters and Metrics. 6. Status of the Link Bandwidth Draft. 7. Discussion on an open network performance testing protocol. 1. Review of Agenda and Status of Drafts and Milestones Henk opened the meeting with a review of the agenda. No one had any suggestions for further changes. He then went on to review working group status. The link bandwidth draft was due in August; although there is no draft to date there has been progress, and Phil Chimento will report on current thinking later. There are four drafts in progress. The MIB draft did not show sufficient interest on the mailing list, so it will be dropped from the charter; it can, however, go on as an individual draft so the work done to date is not lost. Two new work items have been proposed: multi-to-multi measurements and a way to store traceroutes. A personal draft on multiparty communication parameters and metrics has been created and announced to the list; no draft has been created for traceroutes. 2. One-Way Active Measurements Protocol -- Stanislav Shalunov http://www.ietf.org/internet-drafts/draft-ietf-ippm-owdp-11.txt A summary of all the changes is in the slides. There was a substantial wire-format change to robustly differentiate between packets that were sent and not received and packets that were never sent (if, for example, a measurement host had internal sending delays with a relatively small timeout, such that packets would have been sent too late to be received. The ability to start in the past was removed (as the packets are just not sent anyway); there is a new section on timing of packets as sent that clarifies and makes explicit some implications that existed in the old text. The security section has been expanded substantially to clarify requirements and explain design decisions. Return codes in the "ACCEPT field" have been expanded; there was some discussion of what it means to be an internal error versus the catchall failure; internal errors are completely unexpected - something wrong in the underlying implementation. There is now a MUST to remember the list of skipped packets (which can happen for a number of practical reasons). Stanislav ended by noting that we still need a security review; Steve Bellovin asked two people to review; one could not, and the other has not yet completed his review. Jeff Dunn from SI international offered to provide a review, and the chairs will push on the IESG folks again. 3. Reordering drafts -- Al Morton http://www.ietf.org/internet-drafts/draft-ietf-ippm-reordering-07.txt Al Morton reported on the status of the reordering draft. There were a number of substantive comments from last call on the mailing list. See slides for the proposed changes based on the last call comments. One issue that Al brought up that requires closure is that of duplicate packets. Handling duplicate packets has memory implications that vary by implementation. Al did not feel that we could just ignore duplicate packets. The current draft has suggestions on dealing with the memory implications - moreso than in previous IPPM metrics (or in BMWG). These other drafts have been accepted, why is the memory implication now a large issue? Al proposes that if no further comments are received, the extra documentation closes this issue. Randy Presuhn noted (prior to meeting) that we should document the closure of this issue explicitly. There were no objections during the meeting. Al will revise the draft, and we will last call the draft again. There were no comments on the proposed changes from the floor. Al asked if he could get an additional person to read and comment on the draft; Phil Chimento offered to do so. 4. Implementation reports for RFC2678-2681. Henk once again called for implementation reports. We've received three so far. Keynam Hedayat from Brix offered an implementation report; Al Morton said he would try to do one as well. Henk asked that any reports be received by mid-December; this would allow the chairs to compile them over the holidays. 5. Multiparty Communications Parameters and Metrics. draft-liang-multiparty-para-00.txt The author has asked us to discuss the draft and see if it should be picked up as a working group item. This is a personal draft that was announced on the mailing list, but there has been no comment so far. Roughly, this draft concerns the QoS requirements of multiparty communication services, and places them in terms of measurement parameters. It tries to derive a set of parameter metrics from the existing one-way metrics in IPPM. Henk asked if anyone in the audience had read the draft, no one had. Therefore the chairs did not ask if it should be a working group item. Matt noted that this draft is reminiscent of Emile Stephan's spatial metrics draft. Emile said he would read and comment. 6. Status of the Link Bandwidth Draft. --Phil Chimento Mark Allman pointed Phil in the direction of a workshop held at CAIDA last December, and he started with that to take a survey of the current tools and techniques. (See the slides.) There were many tools with three different base techniques, and these techniques (and tools) all measure different things. Phil listed a bunch of difficulties in creating a metric. Phil asked what we should focus on in the draft. We could do a critical analysis of current tools; provide definitions; start at first principles and create definitions, required metrics, and statistical analyses. Matt mentioned that the original intent was to simply attack the definitions. Jeff Dunn supported a rigorous approach; note requirements, which techniques were good, which were questionable, and a statistical analysis. He felt that there was questionable reporting and inferences drawn in what he sees; and he would like to see "solid math" and calculus of probability applied to this area to see what is going on. He also noted that MPLS is another thing that should be added to Phil's difficulty list -- it could add overhead. Matt Mathis noted that there was a general problem with this class of metrics, because operators simply measure utilization with SNMP. He asked if any ISPs are doing this... or would it just be used to check an ISP's service. He also noted that SLA generation based on metrics was not an area the IESG deemed the group should pursue. Cynthia Martin said the DoD uses this to assure they get the bandwidth contracted for. Joeseph Issac said the draft should have definitions and overview of tools, but detailed statistics is probably too much given the state of the art. If we want a document describing statistical analysis, it should be separate. Someone that worked for MCI said that this was a good proposal; we should have definitions and not approach the methodology. What matters is the time measurements are taken, and traffic patterns, these considerations should be documented. Roman Krzanowski from Verizon said the definition is an open issue; and thinks this draft represents something that is needed. Phil asked which of the things the different tools measured would be most useful (link capacity, available capacity, bottleneck/tight link...); Roman said it depends on the part of the network you were interested in, but he would like to have reliable measurements of capacity. He would especially like agreed upon metrics in this area. Henk said that in discussion with ISPS he finds end-to-end parameters are the most important; ISPs know link parameters, but they don't know end-to-end parameters. Roman has attended the IDQ workshop and available BW was of interest there. Summary will be taken to the list. Emile Stephan noted that we have discussed active techniques, but passive techniques are also needed. He said it was important to note what could be measured and with what accuracy. If one operator is sending to another you want to know how the measurement was performed, using what methodology and which accuracy. Roman said he concurred; we have methodology and technology but we do not have agreement. 7. Discussion on an open network performance testing protocol. --Roman Krzanowski Verizon Technologies Roman works on architecture and SLA/monitoring, and he is faced with multiple classes/technologies. Despite all the definitions and dedicated capabilities, there are no standards for conducting measurements. The cost for maintenance of measurement devices far exceeds the cost of the device. Scaling is huge at the customer edge - 1000's of points. What Roman proposes (desires) is a protocol that is implemented in the deployed hardware for latency, packet loss and jitter. This protocol must be open, so it can be implemented by multiple manufacturers. He also sees the need the measurement designs to be standardized. Outline for protocol requirements (see slides): accuracy <1ms,support the IPPM RFCs, etc. Al Morton noted the list of metrics focused on round-trip, which is practical. Roman said yes, because it is easy. You can get accurate time everywhere, but it is not free. Jeff Dunn made a number of observations related to timing and what was possible, including that a system must sample appropriately for the accuracy desired, and these conditions should go into the architecture. Roman said he was interested in the practicality so that we can get something fundamental and basic in everywhere. Jeff also noted you need special treatment of measurement traffic to get accuracy, but that impacts other traffic. Matt Mathis thought we were looking for the holy grail of measurement: you want to measure everything, you want to pay nothing. He said Roman should look at the IPMP draft that has been discussed here previously, and the report from the IRTF measurement group on that protocol. ICMP does work well because it does not get preferred access to resources. IPMP has diverged into two camps, and input from ISPs might help. The IRTF review proposed making this simpler. Just find a way to record a timestamp, and a cookie that the ISP can understand. Can infer everything you are looking for except loss rate from this data. The person from MCI felt that it was very expensive to get core line cards to do the time stamps. It was also important to be sure you define where the edges of what you measure are. Roman will follow up. Keynam from Brix said we shouldn't argue, but be open to all solutions in measurement, both hardware and software. He wanted a clarification if we are looking at a protocol for testing, or a protocol for reporting? Roman said that if you are doing testing, must agree on some way of collecting data, but that is secondary to agreeing on testing methodology and protocol. Al Morton said that how the providers make measurements is another point where we need to agree, and we need comparable measurements to do some inter-domain work. We had an AS draft in IPPM that attempted to narrow the field. That might be a good first step toward what you want. Roman was going to follow up after reading the IPMP report posted to the IMRG and IPPM list, as well as the OWAMP drafts. With that the meeting used the allotted time, and the meeting was closed. |