NOTE: This charter is a snapshot of the 48th IETF Meeting in Pittsburgh, Pennsylvania. It may now be out-of-date. Last Modified: 17-Jul-00
Chair(s):
Kevin Dubray <kdubray@ironbridgenetworks.com>
Operations and Management Area Director(s):
Randy Bush <randy@psg.com>
Bert Wijnen <bwijnen@lucent.com>
Operations and Management Area Advisor:
Randy Bush <randy@psg.com>
Mailing Lists:
General Discussion:bmwg@ironbridgenetworks.com
To Subscribe: bmwg-request@ironbridgenetworks.com
Archive: http://www.alvestrand.no/archives/bmwg/
Description of Working Group:
The major goal of the Benchmarking Methodology Working Group is to make a series of recommendations concerning the measurement of the performance characteristics of various internetworking technologies; further, these recommendations may focus on the systems or services that are built from these technologies.
Each recommendation will describe the class of equipment, system, or service being addressed; discuss the performance characteristics that are pertinent to that class; clearly identify a set of metrics that aid in the description of those characteristics; specify the methodologies required to collect said metrics; and lastly, present the requirements for the common, unambiguous reporting of benchmarking results.
Because the demands of a class may vary from deployment to deployment, a specific non-goal of the Working Group is to define acceptance criteria or performance requirements.
An ongoing task is to provide a forum for discussion regarding the advancement of measurements designed to provide insight on the operation internetworking technologies.
Goals and Milestones:
Done |
|
Expand the current Ethernet switch benchmarking methodology draft to define the metrics and methodologies particular to the general class of connectionless, LAN switches. |
Done |
|
Edit the LAN switch draft to reflect the input from BMWG. Issue a new version of document for comment. If appropriate, ascertain consensus on whether to recommend the draft for consideration as an RFC. |
Done |
|
Take controversial components of multicast draft to mailing list for discussion. Incorporate changes to draft and reissue appropriately. |
Done |
|
Submit workplan for continuing work on the Terminology for Cell/Call Benchmarking draft. |
Done |
|
Submit workplan for initiating work on Benchmarking Methodology for LAN Switching Devices. |
Done |
|
Submit initial draft of Benchmarking Methodology for LAN Switches. |
Done |
|
Submit Terminology for IP Multicast Benchmarking draft for AD Review. |
Done |
|
Submit Benchmarking Terminology for Firewall Performance for AD review |
Done |
|
Progress ATM benchmarking terminology draft to AD review. |
Done |
|
Submit Benchmarking Methodology for LAN Switching Devices draft for AD review. |
Jul 00 |
|
Submit first draft of Firewall Benchmarking Methodology. |
Jul 00 |
|
First Draft of Terminology for FIB related Router Performance Benchmarking. |
Jul 00 |
|
First Draft of Router Benchmarking Framework |
Jul 00 |
|
Methodology for FIB related Router Performance Benchmarking to AD review. |
Sep 00 |
|
Methodology for IP Multicast Benchmarking to AD Review. |
Sep 00 |
|
Methodology for ATM Benchmarking for AD review. |
Sep 00 |
|
Progress Frame Relay benchmarking terminology draft to AD review. |
Nov 00 |
|
Terminology for ATM ABR Benchmarking for AD review. |
Jan 01 |
|
Firewall Benchmarking Methodology to AD Review |
Jan 01 |
|
Terminology for FIB related Router Performance Benchmarking to AD review. |
Mar 01 |
|
First Draft of Methodology for FIB related Router Performance Benchmarking. |
Mar 01 |
|
Router Benchmarking Framework to AD review. |
Internet-Drafts:
· Benchmarking Methodology for LAN Switching Devices
· Terminology for Frame Relay Benchmarking
· Methodology for IP Multicast Benchmarking
· Methodology for ATM Benchmarking
· Terminology for ATM ABR Benchmarking
Request For Comments:
RFC |
Status |
Title |
RFC1242 |
Benchmarking Terminology for Network Interconnection Devices | |
RFC2285 |
Benchmarking Terminology for LAN Switching Devices | |
RFC2432 |
Terminology for IP Multicast Benchmarking | |
RFC2544 |
Benchmarking Methodology for Network Interconnect Devices | |
RFC2647 |
Benchmarking Terminology for Firewall Performance | |
RFC2761 |
Terminology for ATM Benchmarking | |
The bmwg met Monday morning 7/31/00. Scott Bradner acted as chair due to the normal chair's being unable to attend.
The first item on the agenda was a discussion on a proposal to have the bmwg develop documents about testing of performance in a wireless environment. The decision of the AD and the ad-hoc chair was that at this time it did not look like the proposed work fits within the bmwg area so the offer of a presentation was declined.
The group then discussed draft-khurana-bmwg-diffservm-00.txt. There was a lot of discussion on trying to figure out if testing of a device that implements diffserv also needs to take into account the actions of end devices running TCP for example TCP backing off in the face of packet discard. The consensus seemed to be that any exploration of this should be done after the methodology of testing isolated diffserv performance had been completed.
The final document that the working group discussed was draft-feher-benchresres-00.txt. Feher gave a short presentation explaining what was involved in testing devices implementing RSVP. There was support for continued work on this proposal but the suggestion was made to split the ID into separate terminology and methodology documents.
None received.