Benchmarking Methodology WG (bmwg) Tuesday, Afternoon Session I 1300-1500 Room Name: Spinnaker <<< Note the Change! CHAIR(s): Al Morton AGENDA: 0. Agenda bashing (if we need to shuffle a few items) See https://datatracker.ietf.org/public/meeting_materials.cgi?meeting_num=67 for Agenda updates and Slides 1. Working Group Status (Chair) Topics/Drafts not covered by presentations below: Hash and Stuffing Draft (WGLC completed) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-hash-stuffing-06.txt Change Log for 06: http://www1.ietf.org/mail-archive/web/bmwg/current/msg01273.html Terms and Methods for Benchmarking IPsec Devices (WG review and feedback is needed! Please read and comment...) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-ipsec-term-08.txt http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-ipsec-meth-01.txt (Recently Expired, see http://tools.ietf.org/wg/bmwg/ for these versions) Methodology for Benchmarking Network-layer Traffic Control Mechanisms http://tools.ietf.org/wg/bmwg/draft-ietf-bmwg-dsmmeth/draft-ietf-bmwg-dsmmeth-02.txt IGP Data plane convergence benchmark I-Ds - Publication Requested Revised I-Ds Needed. Standard Introduction/Security Paragraph for BMWG memos Check the BMWG mail archive for comments: http://www.ietf.org/mail-archive/working-groups/bmwg/current/ 2. Techniques for Benchmarking Core Router Accelerated Stress Testing. Discussion GOALS: Review Last Call responses (numerous from many reviewers), updates to new revisions, and open issues Draft file name, or preferably the complete URL: http://www.ietf.org/internet-drafts/draft-ietf-bmwg-acc-bench-term-10.txt http://www.ietf.org/internet-drafts/draft-ietf-bmwg-acc-bench-meth-06.txt draft-ietf-bmwg-acc-bench-meth-opsec-00 Presenter: ??? 3. IPv6 Benchmarking Methodology Discussion GOALS: a) Updates based on BMWG and v6ops feedback - Includes recommendations for benchmarking with traffic that has the Hop-by-Hop Extension Header (as suggested by v6ops and BMWG) - Updated frame size recommendations for Ethernet to include Jumbo frames (Appendix updated to include line rates values for all frame sizes) - Proposal for IPv6 benchmarking prefix assignment by IANA - The Security section contains the new standard text discussed within BMWG (we can take this opportunity to discuss the Security write-up) b) Call for making the document a WG work item and discuss next steps Presenter: Ahmed Hamza Draft: http://tools.ietf.org/wg/bmwg/draft-popoviciu-bmwg-ipv6benchmarking-02.txt 4. Sub-IP Protection Mechanisms Discussion GOALS: Introduce this new official work item and review changes in the -00 terminology and methodology documents different from the individual submittals. Presenter: Jean-Louis Le Roux Drafts: http://www.ietf.org/internet-drafts/draft-ietf-bmwg-protection-term-00.txt http://www.ietf.org/internet-drafts/draft-ietf-bmwg-protection-meth-00.txt 5. Milestone Status (Chair) ******* New Work Proposals ********* 6. SIP Performance Benchmarking Discussion GOALS: Review changes to Terminology, review new Methodology. Ask working group if ready to be an official work item. Proposal TEXT: see http://www1.ietf.org/mail-archive/web/bmwg/current/msg01291.html Presenter: Vijay Gurbani and Carol Davids Related Drafts: draft-poretsky-sip-bench-term-01.txt http://www.ietf.org/internet-drafts/draft-poretsky-bmwg-sip-bench-meth-00.txt 7. Multicast VPN Scalability Benchmarking Discussion GOALS: - Introduce the problem of MVPN scalability benchmarking and need for this draft - Introduce MVPN Metric proposal - Summarize proposed methodology - Request BMWG to take it on as WG work item Statement of proposed work: Multicast VPN (MVPN) is a service deployed by VPN service providers to enable their customers to use IP multicast applications over VPNs. With the increased popularity the scalability of deploying such a service is becoming of a great interest. This document defines standard metric and test methodology for characterizing and comparing control plane MVPN scalability of Provider Edge (PE) devices that implement MVPN service. Presenter: Silvija Dry Related Draft: http://www.ietf.org/internet-drafts/draft-sdry-bmwg-mvpnscale-00.txt 8. Extending the current methodologies to cover wireless LAN switches and wireless LAN meshes TOPIC: WLAN Switch and Mesh Benchmarking GOALS: Provide background Propose specific work items Solicit help towards generating a candidate draft DRAFTS: None PRESENTER: Jerry Perser, Tom Alexander Statement of Proposed Work ------------------------------ Enterprise wireless LANs are now comprised of highly IP-centric devices. Considerable work is being done in various IETF WGs in this area. For example, CAPWAP is defining WLAN switch protocols, and MANET is defining IP over wireless mesh and ad-hoc networks. Modern WLAN switches are Layer 3/4 aware and include many traditional IETF defined functions such as ARP, DHCP and firewalling in combination with wireless functions such as mobility. The lack of industry-standard metrics for benchmarking makes it very difficult for vendors to compare and improve the performance of their devices. We propose that BMWG should take up work in support of these technologies. Metrics are required for roaming and scalability performance (related to CAPWAP) and multi-hop performance and recovery times (related to MANET), as well as general WLAN switch data plane performance. The proposed work item is as follows: - extend existing LAN switch benchmarking terminology and methodology (RFC 1242, RFC 2285 etc.) to wireless LAN switching devices - create new wireless-specific terminology and methodology for mobility, scalability, and mesh networks The IEEE 802.11T Task Group has determined that such work does not fall within their scope, and so there should be no issue with taking it up in BMWG. See Liaison Response: https://datatracker.ietf.org/documents/LIAISON/file369.doc Scope ------- The scope of the proposed work will cover benchmarking terminology and methodology for measuring the performance of wireless LAN switches and wireless LAN meshes, with particular focus on extending RFC 1242, RFC 2544, RFC 2285 and RFC 2889 to such systems. Benchmarking and performance of 802.11 PHY or link layer functions is a non-objective. Status of existing drafts ------------------------- No drafts currently exist in support of this work. For background on related BMWG wireless discussions, see: http://www3.ietf.org/proceedings/04nov/bmwg.html#cmr Please help contribute to a successful meeting by reading the above I-D(s) and references *before* we meet. To offer comments on BMWG work in progress or the agenda itself, please send email to: bmwg@ietf.org Alternatively, to offer potential agenda items, please email: acmorton@att.com