This session was chaired by Henk Uijterwaal and Matt Zekauskas. Barry Constantine scribed the meeting, and his notes were lightly edited into these minutes by the Chairs.
(see slides)
Henk opened the meeting, and there were no changes to the agenda.
draft-ietf-ippm-spatial-composition was published
draft-ietf-ippm-tcp-throughput-tm-12 passed two WGLCs and now passed onto IESG early next week.
draft-ietf-ippm-reporting-06, ready for second WGLC
Henk then introduced the topic of what is next for ippm, it has been around for over 10 years, and recently perceived energy has been low.
Three options were presented:
draft-ietf-ippm-metrictest-02.txt---Ruediger Geib
Rudeiger Geib presented changes to the advancement draft; there were few, most of the work has been in the next talk, using RFC 2679 as an example.
The same statistical test used to quantify precision for a single metric implementation must be used to compare the implementations.
The precision used to compare the tests is important and may cause unnecessary failures in the ADK test (an example was given to use milliseconds versus microseconds range when milliseconds is sufficient).
draft-ietf-ippm-reporting-metrics-03.txt---Al Morton
Al then took up with the plan and results so far applying the metrictest draft to RFC 2679.
The key discussion that Al wanted to have, and resolve, is a proposal on thresholds of equivalence (used in the Anderson-Darling K-Sample). He felt we needed agreement before we could really review the results.
Recall:
The test configuration was described; The two tools were Perfas (by DT) and Netprobe (by ATT). Perfas is a GPS-synchronized network monitor to capture jitter and packet loss. NetProbe is NTP-synchronized, and measures delay, loss, and manyother IPPM metrics. For the purposes of this discussion, Perfas was on one side (Germany) and Netprobe on the other side (United States). [See slides]
L2TP VLAN tunnels are used which hides the IP addresses. Same paths are nailed down for each test for consistency. A network emulator is inserted in the path to supply additional and controlled impairments.
Tests in the plan were taken from RFC 2679, variations of one-way delay measurements and note that NetProbe and Perfas are synchronized for these one-way delay and variation tests.
Comparing the results of Perfas and NetProbe per RFC 2679 metrics, but what are the equivalence limits? Two outcomes, IPPM reaches consensus on the limits in section 5 before the results are published, or IPPM revises the limits without input from the test team (since the testing is in-progress since section 5 was published).
The proposal: ADK at 95% confidence factor and 1 msec resolution or the smallest confidence factor and results of the same implementation.
A constant time accuracy error of < +/-0.5msec may be removed from one implementation before ADK is the proposal and a constant propagation delay of < + 2msec MAY be removed from one implementation
Matt Z. felt that 1 msec may be too coarse unless distances were long (hence fraction of overall time is small) -- e.g. for cross continent evaluations. Henk felt that 1 msec was reasonable.
95% confidence level was the goal of these tests, using 1 msec and these metrics are for delay variation and not packet loss.
A question was raised, what is the accuracy of the tester's results? Rounding all measurements to 1 msec is key and again Al emphasized that this is a cross continental test, so 1 msec is good enough target.
Ruediger emphasized that ADK is about distribution and not averages and the result distributions were key to compare.
Al pointed out the network emulator as a key part of the experiments and this was used to induce delay, delay variation, and loss of an inter-continental network.
Show of hands for who agrees with the parameters on the slide (0.5 msec accuracy error and 2 msec propagation delay error). The issue was raised concerning the accuracy of the clock, how can we agree on this without notion of the accuracy of the clock. So the vote turned out to be 7-8 hands agreed that these criteria were adequate
If the impairment generator inserted is 100 msec on average (per Martin's clarification request and now this gained consensus)
Test set-up experiences were next discussed; key point was that loopback LB0 was used for the route-able IP address. Preliminary testing had many cases of successful comparison and many instances where the results differed and the results interpretation is on-going.
Should this be a working group document and Henk said "yes".
Steve then had a question "would the test packet characteristics be defined in the document, size, DSCP values, etc.." and Al answered yes.
draft-ietf-ippm-rt-loss-00.txt--Al Morton
Al briefly described the metrics in the document. Al then went into the comments from last review in Beijing and re-ordering, path names forward / reverse ad well as moving the 2-way measurements to TWAMP.
Also identified the processing stages and added to section 4.
Steve B. asked why TWAMP was not specified and Al answered that these metrics were developed before the TWAMP protocol was developed.
Al will do an update to this document was the agreed to course of action.
draft-ietf-ippm-loss-episode-metrics-01.txt--Al Morton
There is IPR associated with this draft ( http://datatracker.ietf.org/ipr/1354/).
Updates included clarifications from Beijing and this was mostly to do with loss episode events.
Al asked if WGLC could occur and Henk agreed.
There is IPR associated with this draft
(
http://datatracker.ietf.org/ipr/1472/).
Steve presented. Introduces a new probing methodology for capacity metrics.
He mentions the current capacity metrics (RFC2544, RFC5136, and ITU-T Y.1540).
Capacity metrics are important and probing is not sufficient.
Capacity measurement principles:
Challenges for TWAMP test protocols:
Proposal of the new TWAMP design, 3 new fields. "Sender
discriminator, last seqno in Train and Desired Reverse Packet
Interval" and these each address the 3 challenges listed above.
Backward compatibility is ensured since there is also a TWAMP mode to
identify the ability of the Server / Session to support the enhanced
TWAMP. For the new mode, both sender and reflector must be
compatible.
Steve then reviewed the new TWAMP protocol format and this was slide 8
which showed the location of the new fields (3). And there are still
9 bits reserved for even further enhancement of future functionality.
Slide 9 went into a description of the behavior of the reflector based
upon the new packet reverse train handling capability. All
requirements followed the same MUST / SHOULD paradigm of RFC5357.
Steve then re-summarized the introductory benefits of the new octet
functionality and added that his colleague would be able to
demonstrate this capability from mobile phone to lab in Stockholm.
Al Morton was surprised to see a control protocol added in -01 and not
in the original draft-00. Steve explained why the control plane was needed
to provide the enhanced capabilities of the reflector and Al agreed.
Henk stated that the draft states that it is backward compatible with
TWAMP, and wanted to know how an older reflector would know.
Matt Z explained that RFC2544 is not to be used in operational
networks and Steve agreed to remove. Matt then stated that capacity
metrics were not defined in RFC5136 nor any specification that he knew
of. Steve's colleague emphasized that this extension provides the
necessary capabilities for vendors to conduct the capacity measurement
and develop the metrics.
The vision here is to standardize on TWAMP for the many test tools
that are used such as PathChirp, Pathload, Spruce, etc.
Steve asked whether a separate document should be pursued to define
the metrics that are to be measured? And Henk stated that yes, that
since the TWAMP extended method allowed for different packet
generation capability for test methods to use, then the measurement
metrics must be defined. Matt Z, was not ready to say yes, but also
added that why should we bother standardizing the method when the
metrics are not measurements.
Matt Z asked why TWAMP versus OWAMP? Costs of implementation but more
importantly TWAMP is superior since carriers want to measure / collect
from one point (versus two control points for OWAMP).
xWAMP nits
Al presented on behalf of Yaakov. This was a discussion of "xWAMP"
which referred to both OWAMP and TWAMP. Al emphasized that the
segregation of the fetch client versus the sender / receiver and
implications. New fetch messages were proposed, "list stopped SID
reuest and list stopped SID response". TWAMP does not need these
since the Sender is both sender and controller, which is exactly why
Steve stated that TWAMP was superior for provider networks.
Why is port 861 re-used for OWAMP fetches? Fetch client should use
the same TCP port as the actual test session.
Steve B. asked why the mailing list was not used to ask these
questions? Al agreed that mailing list should be used.
The main issue is that the fetch client is not synchronized with the
sender / reflector so does not know when the test is complete / when
results can be deleted, etc.
A discussion of source port ranges used occurred and the range is
quite large.
RAD has developed a Wireshark dissector for OWAMP and this will be
extended to TWAMP as well.
Henk and Matt both agreed that Yaakov needs to publish these findings
/ thoughts to the list, so we can have a better discussion.
xWAMP Extensions
The addition of a multiplier and size field was discussed. The
addition of TCP test sessions should be considered for xWAMP.
Matt Z. discussed that BWCTL tool that spawns off different TCP tests
to various locations (point to multi-point) and that the BWCTL tool
was OWAMP based.
Throughput measurement techniques were discussed and the fact that
xWAMP did an excellent job of finding the weakest link (queue
approaching capacity)
Al then re-iterated the list of extensions required from Yaakov's slides.
The basic idea is to only send TWAMP session information in one
direction and payload in the other direction.
Steve B. asked why does it need to be intrusive and how does it
compare with Steve B's TWAMP extension proposal?
Al answered that the deliberate asymmetric sizes of packets in the
test session is a unique feature that does not appear in the value-added
octets proposal. The test could be used by network providers that
control whether the production traffic flows on the measured links.
A new command for the intrusive feature would be added.
Andreas commented that it seems possible that with the TWAMP
extensions this type of testing could be supported; but asymmetric
mode could be considered. Henk indicated that Al, Yaakov and Steve
B. needs to discuss and work out, but the IPR needs to be considered.
Steve B. commented on their IPR and that it was license free.
Problems with RFC5136
The definitions and metrics defined in RFC5136 are not compatible with
RFC2330 (IPPM Framework).
Issue is that RFC5136 defines link capacity and path capacity based on
these newly defined "link" and "path" definitions. So it is ambiguous
just what is getting measured, and what the metrics relate to
The implied reference model in RFC5136 is that the router (or part of
the router) is a part of the link. While RFC2330 clearly indicates
that a router is not part of the link
Xiangsong went over slide 7 that highlighted a GigE Ingress and 100M
Egress router and reviewed the %'s in the slides.
ITU SG12 has already fixed the problem addressed in the RFC5136bis draft.
Y.1540 defines capacity over sections and takes into account that network
hosts affect capacity while RFC5136 does not.
Xiangsong proposed that RFC5136bis be adopted by the IPPM as the
solution. Steve B. felt that RFC5136 could be interpreted properly
and felt it was not worth to open a new RFC. Henk commented that a
short note should be added RFC5136 and felt that it was not worth
starting from scratch. Al commented that the use of the word link
should have been replaced by hop.
Steve B. favored the more general definition of a link which could
include a router if the router is not a bottleneck. This directly
opposed Xiangsong's proposal, which clearly differentiates the link
from any nodes.
The email from the authors of RFC5316 was reviewed.
Henk noted that the flurry of the of emails made this very clear that
there was a lot of contention.
Matt Z. asked what was the driving force? If metrics would be
created, then the RFC5136 document would have to be revisited.
Xiangsong simply stated that RFC5136 was a very confusing document;
Steve B insisted that it was not a confusing document. Steve stated
that some of the new definitions in the "bis" of router capacity etc.,
would cause even more confusion.
Henk prefers to publish this information as errata to RFC5136. Further discussion should go to the mailing list.
There was no time remaining, so the Chairs closed the meeting.
6. TWAMP Value-Added Octets
draft-baillargeon-ippm-twamp-value-added-octets-01.txt
--Steve Baillargeon, presenting
7. OWAMP and TWAMP Measurement Issues/Topics
--Al Morton, presenting for Yaakov Stein
8. Intrusive Raw Capacity Measurement of Asymmetrical Paths
--Al Morton, presenting his own proposal
9. RFC 5136 Issues
draft-cui-ippm-rfc5136bis-00.txt
--Xiangsong Cui, presenting
10. AOB