IETF-77 BMWG Session Minutes
22 March 2010 1300 PDT
WG chair: Al Morton
These
minutes are divided in three sections: Summary, Action Items and Detailed
Notes.
The
BMWG met at IETF-77 with 23 people in attendance, and 10 more using webex conferencing to join remotely (and in one case, to
present slides). This meeting report was prepared by Al Morton, based on
detailed minutes provided by David Newman, as official note taker, and with Mike
Hamilton filling in for David during his presentation and at other times.
SUMMARY
The
IPsec drafts should be updated soon, according to the
lead editor (to resolve remaining DISCUSSes).
The
IGP dataplane convergence drafts have been updated
based on implementation/testing experience via comments in response to the WGLC
in Dec 2009. The drafts have been updated and the will be another short WGLC
before returning these drafts to pub-requested state.
The
Sub-IP drafts completed WGLC in January.
The terms draft will go to to pub-requested
state, while the methodology will wait for IGP-dataplane
IESG review to attempt to derive benefits from comments on the similar methods.
The
SIP device benchmarking drafts have made good progress, and are in WGLC till
March 31st. There are many comments to be incorporated and more readership is needed. The plan is to reach
publication-request by July 2010.
The
RESET performance work proposal did not yet have enough readership
to adopt as a work item, but there is support for the work and no need for a
charter update to add the work.
The
Flow-Monitoring device Benchmarking drafts were updated to address comments,
and there was meeting consensus on the work item paragraph to add to the
charter (already reviewed on the list).
The
Data Center Bridging work has some industry support, but we need to liaise with
IEEE and a letter must be prepared and approved to do that.
Lots
of good discussion on the traffic load characteristics for the Content-Aware
Device Benchmarking proposal, and an updated draft is expected. BMWG has an open
mind w.r.t. new traffic descriptions, with the
proviso that results also need to be interpretable and repeatable.
Work
on BGP Convergence Benchmarking is focusing on the dataplane
for the current work, and may require updates to the terminology RFC.
ACTION ITEMS |
Question to List to pick-up RESET draft on the Charter – small number of readers/supporters present at the meeting (conclusion that no re-chartering is necessary to begin this work). (Done) |
|
Prepare a Liaison to IEEE 802.1, pointing to the Data Center Bridging draft and the proposal to update RFCs 2544 and 2889 to address the Per-Flow Control capabilities of IEEE 802.1Qbb.(Done) |
|
The Sub-IP Protection Terminology Draft has reached WG Consensus – Doc Shepherding Write-up In-Progress. (Done) |
|
Start a WGLC on the IGP Dataplane Convergence Drafts (Done). |
|
Pursue Re-chartering: When Revising Milestones, include the new dates suggested by the SIP benchmarking authors (June/July 2010 AD-review). |
|
WG participants interested in the SIP Device Benchmarking work should schedule some time to do a detailed review of revised drafts in April 2010, during 2nd WGLC. This will be Cross-posted to SIPCORE WG. |
|
Think hard on traffic for content aware devices – Everybody! |
|
No one has been willing to help on chartered items where the drafts have LONG EXPIRED, such as Accelerated Stress Benchmarking, and this lack of progress & interest will be recognized during re-chartering. Anyone with concerns should contact the Chair immediately. |
DETAILED
NOTES
IPR Policy statement covered before all else.
Agenda bashing (none)
WG status
6 WG documents are active (2
each on dataplane convergence, protection and sip
benchmarking; last of these are in wglc)
dataplane convergence applicability ID recently expired
IPSec docs are in IESG
processing, possibly done with tweaks from Merike and
Tim as soon as today (or perhaps this week) ??
No docs in RFC-editor’s
queue (MPLS forwarding is now RFC 5695)
4 documents are proposed
work items
Reminder from Al to use
standard bmwg scope disclaimer in all documents, it
ensures that our BMWG charter limits are accurately reflected in the text.
Link-state
IGP dataplane convergence (Kris Michelsen,
remote presenter)
Summarize main changes in
-20 (from comments during WGLC)
1. Added 2 optional test cases
1-to-N path convergence (local, remote
failure cases)
2, Added accuracy intervals
More specifics on calculations for
loss-derived and route-specific loss-derived methods
3. Added packet delay variation evaluation for the
rate-derived methodologies
Explains calculations for expected packet
counts (uses relationship between PDV and packet sampling interval [PSI])
4. Explains when convergence
occurs in non-ECMP and ECMP cases
Comments:
Al: Got many comments from a
new reader/implementor after end of WGLC, and it
looks like these comments have been addressed. Asks for additional comments
from the room; none from attendees.
Al proposes another short
WGLC, starting today and running two weeks. Kris may have 1 or 2 more additions
after WGLC ends, such as substitution of PDV for jitter. Al believes we can go
to IESG after this last call.
Protection
performance drafts (Al Morton, presenter)
Al: These drafts have completed WGLC at end of
January 2010. Al believes there is working group consensus on moving ahead, but
in terms of schedule also thinks they should move ahead after the IGP drafts.
This is in part because the IGP drafts have some methodologies in common, and
Al would like to hear from IESG on these documents “for safety reasons.” This
order would prevent overloading the IESG with work that may or may not be ready
to go from the IESG’s standpoint.
Remote comment from Rajiv:
Would prefer to see protection IDs go forward independent of IGP convergence
drafts.
Comment from Ron Bonica, as AD: Would prefer that documents be staggered
because what authors learn from one document/documents can be applied to the
other. Authors don’t want to submit a bunch of documents and have them all
bounce at the IESG.
Remote comment from Bavani (and agreed to by Rajiv): Proposes that terminology
ID could proceed independently
Comment from Al: This seems
like a good compromise. We’ll proceed with the terminology. There was nodding
in the room and no further comments.
Comment from Al: In
terminology draft, at least two equations have unbalanced parentheses. Since
this will cause problems at IESG, this should be cleaned up in AD review.
Question from Al for Kris:
Can you quickly make final changes after WGLC to these drafts? Kris says yes.
SIP benchmarking terminology/methodology updates and
status (Carol Davids, presenter)
WGLC in progress, IDs need
review and comments. Masters students at IIT working to implement open-source
SIP servers to test IDs
Presented summary of
terminology contents
Definitions in four sections: Protocol components, test
components, test setup parameters, benchmarks
Fleshed out reporting format and reporting requirements
(comment from Al: be sure to include units of measurement in stating reporting
format)
Update on terminology
changes
Updated definitions and test cases, based on IIT testing
experience
Comment from Al: There is a
diagram to show testers back-to-back in terminology, but that topology is not
included in the methodology. Carol agrees that is missing, and will add it.
Comment from Vijay
(coauthor): There are parts of the methodology that talk about the proxy. For
media-related tests, we might want to look at end-points themselves.
Comment from Al: PMOL draft
is now in IESG Review, and has collected several DISCUSSes
(blocking comments). Vijay is working as document shepherd. Al will steer the
current IESG comments to the authors for incorporation in BMWG drafts, as
appropriate.
Milestones
review
Five sets of docs “in the
red”: Net traffic control, Router accelerated benchmarks, protection
benchmarking, SIP, accelerated EBGP tests, Op Sec. All the Old, overdue, and
expired items have
seem to have entered the DEAD state – they have been expired for a long time.
There has been no response to requests to help-out on these toipics.
Comment from Vijay and Carol:
June-July is reasonable for next versions of SIP IDs
Al: Can move terminology on
protection benchmarking forward now
Al considering rechartering to clean up old work (Net traffic control,
Router accelerated benchmarks, all long expired) and possibly adding new items
Basic BGP convergence is on
the agenda for discussion today, this is a very old work item.
Work proposal summary matrix
has currently has five items
WG has seen Significant
improvement in IP flow export
Content aware devices (needs significant support at
meetings)
LDP convergence (waiting behind IGP, not clear if there
is still interest to do this)
Router reset
Reset
benchmarks (Al Morton, presenter, on behalf of authors)
Review ID’s
motivation: “All resets are not created equal” – R. Bonica
Review of core methodology
for hardware, software resets and reporting formats
Presented changelog. David Newman
provided extensive comments on the list, and also supports the work to
continue.
Al calls for comments and
review
Not enough reviewers at the
meeting to make go/no go determination; Al will ask for input on the email list
(Action).
IP Flow
Information Accounting and Export Benchmarking (Al Morton, presenter, with Jan
Novak’s remote agreement due to audio quality issues)
Review document goals and key concepts of flow
monitoring measurement
Review main changes
Merged
normal and overflow caches into 1
Moved
misc. tests into Appendix B
Clarified
test setup with multiple observation points
Expanded
2544 measurements
Proposed scope items are to develop terms and
methodology, state goal of maximum IP flow rate, and assess forwarding rate
performance of DUT/SUT
Comment from Chris Elliott: Chris volunteers himself
and/or Benoit to help move this forward.
DCB
Benchmarking (David Newman, Presenter)
There is interest among the bridge builders and the test equipment manufacturers to make the proposed measurements. Classic loss-limited Throughput of RFC 2544 will not work. There was agreement to pursue discussing the possible overlap with IEEE 802.1 via liaison. (ACTION)
Content-aware
Benchmarking (Mike Hamilton, presenter)
Characterized 03 as a
maintenance release
Removal of 2544/3511 wording
and “realism” from document, but “underneath the hood” the ID is still aimed at
predicting performance in production networks. “Realism was removed but
underlying concept is still there.”
Explicitly calls out
exclusion of capture/replay test traffic.
Believe they have added
Enhanced repeatability to test.
Comment from Al on “realism
removed”: Traffic characterization beyond fixed packet sizes is there because
it can be well described and understood. This is less true of “real” traffic.
“As long as the tests of a good metric are in your mind, that’s probably the
most we can ask for at this point.” Al asks for presentation of evidence: Some
explanation of test traffic and test results is often compelling and/or
revealing.
Comment from Sarah:
Something to be said about not having fixed packet sizes. So long as you denote
what traffic mixes you use on that day, that’s about the best you can do. There
is for me the notion of having some realistic traffic content. She’s had this
request from multiple employers current and past. She is strongly in favor of
bringing real traffic back.
Response from Al: We’ve only
standardized fixed packet sizes because we understand what these mean, but
don’t prevent vendors of test equipment from doing “Imix” or other kinds of
traffic loads.
Comments from Brett Wolmarans: Offers to help flesh out test cases. Al’s response – please take part on the list.
Comment from Joel Jaeggli: Would love to have an annex with repeatable tests
described. Customers have two devices and want to decide if they’re comparable.
They can do this with RFC 2544, which is straightforward. This is more
nebulous. A benchmark may not reflect operational reality.
Al Morton: So that’s the
thing to focus on: Document repeatable and representative traffic, and then
decide whether that produces repeatable and reproducible results.
Joel: So the point where you
know something is where you know you have repeatable results.
Comment from George Uttice (sp?): As we’re talking about realism, the question
is whether there’s anything that’s closer to realism than a fixed-packet test.
Anything that’s closer to “real” is better. Also, because vendors will tune for
benchmarks, we don’t want to issue a non-meaningful benchmark.
Comment from Brett Wolmarans: Challenge is to isolate the DUT. We all
need to follow a common, good methodology.
Comment from Mike: Less
concern about the methodology itself and more about the offered load. Wonders
if separate offered load effort would make sense first.
Al’s response: Try to keep
them together for now, since methodology has a dependency.
Basic BGP
convergence benchmarking (Al Morton, presenter, with Rajiv’s remote approval)
Review of
background/motivation/current status of ID and review of document scope
Al asks whether route
reflectors should be included in scope, and asks for discussion on the list
Review of test methodologies
(a relative large set)
Review of next steps
(publication of first draft in April, call for review and comments, and call
for input from service providers)
Open question regarding
control vs. data plane benchmarking (David Newman, as attendee is in favor of
adding data-plane component)
Wrapping
up
IGP-Dataplane
drafts to WGLC
Move protection terminology
forward Doc Shepherd Write-up
AM’s comments to SIP authors
SIP docs in June/July
Open question on reset, to
discuss on list
Agree on paragaph
on flow/export list – future Re-charter discussion.
Need liaison letter to IEEE
for DCB
Think hard on traffic for
content aware devices – Everybody!
Meeting concluded at 1500
PDT, on the dot!