IETF IP Performance Metrics WG (ippm)
Thursday, November 13, 2003 at 13:00 to 15:00

=============================================

The meeting was moderated by the working group chairs, Matt Zekauskas and 
Henk Uijterwaal.  Al Morton and Ruediger Geib took notes, which were 
edited into these minutes by the chairs.

AGENDA
1. Agenda Bashing
2. Packet Reordering
3. Reordering Density
4. IPPM-MIB
5. OWAMP
6. Status, Milestones & Futures

1. Agenda Bashing
   -- The chairs

There was no bashing, but Matt welcomed Henk Uijterwaal as the new 
co-chair, replacing Merike Kaeo who is now focusing on security work.



2. Packet Reordering
   -- Al Morton

Al Morton presented the changes since the last meeting to the 
reordering draft (see slides).  The major changes are (1) continued 
clarifications started in the last version, including changing the sense of 
the base metric so it reports "true" when a packet has been 
reordered, moving alternate methods of detecting reordering (time or bytes 
instead of sequence number) to a separate section, (2) adding the 
concept of steady-state versus transient reordering, and (3) adding text to 
cover packet fragmentation.  The fragmentation text needs work, and will 
likely have the details moved to a separate section.  The draft is still 
missing an appendix on applicability promised by Greg Wright.  Matt asked 
which appendices, if any, were going to be normative (and we should so 
state).

Jerry Perser asked if we could drop IPPM from the title, since in his view it 
was more general (that is, there was some applicability to BMWG as well).  
There was no objection from the assembled group.

Matt (chair hat off) asked if adding fragmentation added too much 
complexity; none of the other metrics look at individual fragments. Jerry 
felt that either we should specify the metric so fragmentation does not 
occur, or we should check whether fragments are reordered. Al Morton noted 
that he has seen systems "in the wild" that don't handle reordered 
fragments well.  Stanislav Shalunov noted that some versions of Linux 
actually send fragments in the reverse order because they get a slight 
performance improvement during reassembly; a note should be added.


3. Reordering Density
   -- Anura Jayasumana

The next presentation was by Anura Jayasumana on reordering density (an 
individual submission).  This is the second presentation we had on this 
topic, but the first by an author; there was a presentation at the last 
IETF meeting by Jerry McCollum from Agilent, who is familiar with the 
work. One can think of the reorder density function like a normalized 
histogram of buffer occupancy.

The major problem that the group has with this work is that lost packets 
show up in the reorder density, so you cannot completely separate losses 
from reordering events using this technique.

An example showing a lost packet was given.  Stanislav Shalunov noted that it 
appears that loss is counted as reordering.  Anura stated that it was 
correct.  Stanislav asked if a sequence had no reordering but just loss 
would the result be non-zero?  Anura said that was correct, that there is a 
problem with packet loss, the current definition was a compromise, and 
ensured the computation could be done in order n time and constant space.

One interesting claim made by the author is that they have been able to 
characterize the nature of loss based on distributions of reorder 
density. It could also be a basis for TCP adjusting it's sensitivity to 
reordering on the fly, and the distribution could be used when modeling a 
network.  (See also the "Properties" slides.)

One major change from the previous proposal was the addition of "late" and 
"early" reordering density.  Together with the original definition, Anura 
felt that this characterization might also be useful in solving 
concatenation of partial path reordering metrics to give a reordering 
metric for the full path.

This led back to a discussion of packet loss.  Jerry Perser asked if lost 
packets were early or late?  Anura said that was recognized with the help of a 
place label.  Al Morton again brought up that this was the biggest issue in 
this draft; the current working group draft makes reordering 
orthogonal with loss, and there is a clear definition of "early" and 
"late".  The definitions of in and out of order and early and late need to 
merge to add this to the draft.  Anura said that it depends on what you are 
trying to achieve with reordering measurements: perform TCP or network 
diagnosis?  Predict network behavior?  The use of density would add 
value; a simple number describing reordering doesn't provide enough 
information.  The definition of loss would have to be agreed.

Stanislav asserted that n-reordering makes the definition of loss 
orthogonal to reordering. n-reordering metric doesn't change in the 
presence of loss.  Not every router sends a stream of numbers with 
monotonically increasing sequence numbers.  Anura thought there was a 
problem with n-reordering and fast routers. Stanislav said that a linear 
buffer would be sufficient.  Anura said that n squared would hit. 
Stanislav said not for n-reordering.  Anura said that then duplicates are an 
issue. Al said that you either save information on packets which have 
arrived or those missing, usually the latter. Anura thought that a moving 
window is better than keeping information on packets.  Jerry Perser noted 
that test instruments can't track millions of duplicate packets.  Anura 
said that with reordering density you only need to keep track for a fixed 
window.  Stanislav wanted to get back to loss.  Anura said they 
recognize loss within window.  Jerry wondered why it isn't called loss and 
reordering density.  Anura said it isn't a loss density.  Jerry said that he 
had a problem claiming reordered packets when only one is lost.  For 
example, look at slide 6; Jerry said packet 4 is in sequence.  Anura said 
that according to his definition packet 4 is reordered [3 was lost].  
Anura wanted to know about packets arriving much later; suddenly "2" comes in 
and your whole view changes.  Is this appropriate?  Applications would 
presumably treat them as lost.  Are they lost or reordered?  Stanislav said 
the definition of reordering is only based on a sequence of arrivals.

Jerry wanted to know if the authors checked the resources you require to 
implement reordering density (say for an oc192 link) -- what was the 
fastest speed tested?   Anura said that results are forthcoming; they have 
software on Linux for testing, and the fastest speed tested so far was fast 
Ethernet.

The authors would like this draft used as an alternative 
characterization metric in the current draft or published as a separate 
informational draft; publishing elsewhere was mentioned as another 
possibility.  Henk asked if we should merge drafts or proceed 
separately?  Matt said that his sense was that the current authors seem to be 
disinclined to merge with loss mixed with reordering. Al Morton said that 
was true, and there were no other comments.




4. IPPM-MIB
   -- Emile Stephan

Next, Emile Stephan presented the latest version of the MIB.  He wanted to 
know the status of the registry -- it will be presented to the ADs as soon as 
the (apparently now expired) version is refreshed by the I-D editor after 
the meeting.  RMON would like the OIDs for the metrics.  Bob Cole asked 
what happens for new metrics?  Matt said OIDs should be included in those 
drafts.

As far as the MIB goes, Emile presented some recent (minor) changes, and 
then tried to walk the room through a sample setup showing how all the 
tables were organized and how views of the data are consumed (see 
slides).  He tried to do a demo showing the MIB presenting results from a 
commercial measurement system, but the demo hiccupped in the middle.  (He 
later showed it to the chairs.)  He noted that Andy Bierman had sent a 
comment to the list today, and that he hasn't had time to consider it in 
detail, but he will respond.

Matt tried to start a discussion based on Andy's initial paragraph to the 
list, which Matt summarized as that the MIB was "too complex". Henk asked 
who had read the draft and if anyone was interested in implementing it or 
had an alternative.  Only the chairs and Andy admitted to reading it.  
There was therefore no point in an extended discussion, but Andy noted that 
there was a higher-level question: what should be in an SNMP agent.  There 
are roles for an SNMP agent, and roles for the management station.  In his 
mind, there are too many features, and a lot in this draft is out of 
scope.  The draft ideas are basically two years old, and there are no 
other implementations. If no one is going to implement this (or even read 
it) then there must be some action at the WG-level...  Emile noted that an 
operator would not like to get flooded by "unsolicited" measurement 
results; filters are required close to the controller of the 
measurement system, not in the back office.

The feedback from Henk's customers is that the MIB should be simple, 
light-weight, and give users the freedom to do their own aggregation and 
statistics computation.  The draft seems to be too France Telecom 
specific.  Emile noted that this desire was in contradiction with a 
previous concern that the buffer for all results might be too big and 
therefore some aggregation might be required.  There is a 
contradiction between a desire to report only raw data and a need for 
aggregation.  Henk wondered if perhaps the draft could be limited to just 
the two tables it needs to provide raw data and remove everything else?  
Emile noted that the report tables are optional. Bob Cole wondered if the 
compliance section was clear that only two tables were required.  Al said 
that he would read a document limited to core functionality.  Andy felt the 
document itself was too complex?  Does it want to specify a MIB for a 
single aggregation box or 1000 agents doing measurements?  Matt said that he 
would like to see comments on this issue on the mailing list.




5. OWAMP
   -- Stanislav Shalunov

Finally Stanislav Shalunov presented the updates to OWAMP.  The 
approval of the requirements document was announced to the group. 
Updates to the protocol document are based on a sample open source 
implementation, and include test vectors for the random number 
generator used in the generation of packet sending schedules, so that the 
receiver knows accurately when the sender was supposed to send packets 
(which allows losses to be detected at the receiver without having to 
correlate sent packets with the sender).

Matt asked how close Stanislav felt this document was to be forwarded to the 
IESG.  One big issue is that the security has not been reviewed by 
experts. That needs to be done.  Matt: What about a comparison with IPMP? 
Stanislav said that there are two differences: OWAMP includes control, and 
OWAMP tries to be "sneaky" and be not recognizable by routers.  Matt noted 
that there was an ongoing discussion on IPMP on the IRTF/IMRG mailing list if 
any working group members were interested in following IPMP.




6. Milestones and Futures
   -- The Chairs

Matt Zekauskas closed the meeting noting the milestones that have been met, 
those that are in process, and those that will be dropped if no one steps up 
to work on them.  (see slides) These were largely a re-iteration of the 
conclusions from the Vienna meeting; they need to be run through the 
mailing list. Matt and Henk will send email to the list to solicit input.