2.4.12 Packet Sampling (psamp)

NOTE: This charter is a snapshot of the 58th IETF Meeting in Minneapolis, Minnesota USA. It may now be out-of-date.

Last Modified: 2003-10-28

Chair(s):
Andy Bierman <abierman@cisco.com>
Juergen Quittek <quittek@ccrle.nec.de>
Operations and Management Area Director(s):
Randy Bush <randy@psg.com>
Bert Wijnen <bwijnen@lucent.com>
Operations and Management Area Advisor:
Randy Bush <randy@psg.com>
Mailing Lists:
General Discussion: psamp@ops.ietf.org
To Subscribe: psamp-request@ops.ietf.org
In Body: subscribe
Archive: https://ops.ietf.org/lists/psamp/
Description of Working Group:
The Packet Sampling working group is chartered to define a standard set of capabilities for network elements to sample subsets of packets by statistical and other methods. The capabilities should be simple enough that they can be implemented ubiquitously at maximal line rate. They should be rich enough to support a range of existing and emerging measurement-based applications, and other IETF working groups where appropriate.

The focus of the WG will be to (i) specify a set of selection operations by which packets are sampled (ii) specify the information that is to be made available for reporting on sampled packets; (iii) describe protocols by which information on sampled packets is reported to applications; (iv) describe protocols by which packet selection and reporting configured.

Packet reports must be communicable in a timely manner, to applications either on-board of off-board the sampling network element. The streams of packet reports produced by a packet sampling must (i) allow consistent interpretation, independent of the particular network element that produced them; (ii) be self-defining, in that their interpretation does not require additional information to be supplied by the network element; (iii) allow robustness of interpretation with respect to missing reports or part of reports;

Network elements shall support multiple parallel packet samplers, each with independently configurable packet selectors, reports, report streams, and export. Network elements must allow easy and secure reconfiguration of these packet samplers by on-board or external applications.

Export of a report stream across a network must be congestion avoiding in compliance with RFC 2914. Unreliable transport is permitted because the requirements at the exporter for reliable transport (state maintenance, addressibilty, acknowledgment processing, buffering unacknowledged data) would prevent ubiquitous deployment. Congestion avoidance with unreliable export is to be accomplished by the following measures, which shall be mandatory to implement and use. The maximum export rate of a report stream must be configurable at the exporter. A report stream must contain sufficient information for transmission loss to be detected a collector. Then the collector must run a congestion control algorithm to compute a new sending rate, and reconfigure the exporter with this rate. In order to maintain report collection during periods of congestion, PSAMP report streams may claim more than a fair share of link bandwidth, provided the number of report streams in competition with fair sharing traffic is limited. Selection of the content of packet reports will be cognizant of privacy and anonymity issues while being responsive to the needs of measurement applications, and in accordance with RFC 2804.

Re-use of existing protocols will be encouraged provided the protocol capabilities are compatible with PSAMP requirements.

Specifically, the PSAMP WG will perform the following tasks, in accordance with the principles stated above:

1. Selectors for packet sampling. Define the set of primitive packet selection operations for network elements, the parameters by which they may be configured, and the ways in which they can be combined.

2. Packet Information. Specify extent of packet that is to be made available for reporting. Target for inclusion the packet's IP header, some subsequent bytes of the packet, and encapsulating headers if present. Full packet capture of arbitrary packet streams is explicitly out of scope. Specify variants for IPv4 and IPv6, extent of IP packet available under encapsulation methods, and under packet encryption.

3. Sampled packet reports. Define the format of the report that is constructed by the network element for each sampled packet for communication to applications. The format shall be sufficiently rich as to allow inclusion in the packet report of (i) IP packet information as specified in paragraph 2 above; (ii) encapsulating packet headers as specified in paragraph 2 above; (iii) interface or channel identifiers associated with transit of the packet across the network element; (iv) quantities computable from packet content and router state, (v) quantities computed during the selection operation. All reported quantities must reflect the router state and configuration encountered by the packet in the network element.

4. Report Streams. Define a format for a stream of packet reports, to include: (i) the format of packet reports in the stream; (ii) the packet reports themselves; (iii) configuration parameters of the selectors of the packets reported on; (iv) configuration parameters and state information of the network element; (v) quantities that enable collectors and applications to infer of attained packet sampling rates, detect loss during samping, report loss in transmission, and correct for information missing from the packet report stream; (vi) indication of the inherent accuracy of the reported quantities, e.g., of timestamps.

5. Multiple Report Streams. Define requirements for multiple parallel packet samplers in one network element, including the allowed degradation of packet reporting when packets are selected by multiple packet samplers.

6. Configuration and Management. Define a packet sampler MIB to reside at the network element, including parameters for packet selection, packet report and stream format, and export. Select or define a communication protocol to configure/read this MIB.

7. Presentation, Export, and Transport of Packet Reports. Define interface for presentation of reports to on-board applications. Select unreliable transport protocol for remote export. Determine rate control algorithms for export.

Initial Internet-Draft: A Framework for Passive Packet Measurement [draft-duffield-framework-papame]

Goals and Milestones:
Done  Submit initial Framework document
Done  Submit initial Packet selector and packet information document
Done  Submit initial Report format and report stream format document
Done  Submit initial Export and requirements for collectors document
Done  Submit initial MIB document
May 03  Submit final Framework document
May 03  Submit final Packet selector and packet information document
Sep 03  Submit final Report format and report stream format document
Oct 03  Submit final Export and requirements for collectors document
Nov 03  Submit final MIB document
Internet-Drafts:
  • - draft-ietf-psamp-framework-04.txt
  • - draft-ietf-psamp-sample-tech-03.txt
  • - draft-ietf-psamp-mib-01.txt
  • - draft-ietf-psamp-protocol-00.txt
  • - draft-ietf-psamp-info-00.txt
  • No Request For Comments

    Current Meeting Report


    Minutes of the PSAMP session at IETF 58
    Tuesday November 11, 9:00 h - 11:30 h

    Minutes taken by Dave Plonka and Jürgen Quittek

    Session summary

    The WG documents on the PSAMP framework and on packet selection are now consistent with respect to each other and both are quite mature. For both documents the editors will send the list of remaining open issue to the mailing list soon. After the next update (planned for mid of December) both documents can enter WG last call together. The protocol and information model documents are available as first versions. They describe how the (upcoming) IPFIX protocol is used for exporting packet samples. They depend on the corresponding IPFIX documents that are expected to be completed by May 2004. The PSAMP MIB definition progressed slowly, because it depends on the packet selection document. The WG discussed how to integrate an individual I-D describing hash functions for packet selection in detail. the WG hesitated to have this as an additional WG document. It rather should be integrated into the packet selection document.


    Presentations and discussions

    Andy Bierman on the WG Status

    Andy reported on the status of the WG documents. The sampling framework document and the packet selection document are quite mature and should enter WG last call soon. The report format and report stream format document (PSAMP protocol) is waiting for progress of the IPFIX protocol. The export protocol and requirements for collectors document (PSAMP information model) is waiting for progress of the IPFIX info model.



    Nick Duffield on the PSAMP framework

    Nick discussed open issues on the framework document. The term 'sampling' was misunderstood by several reader. Therefore it was replaced by 'packet selection'. The framework description was extended to better cover encrypted packets. The description of exporting packet reports now discusses compliancy to and coexistence with IPFIX.

    The term 'emulated selection' will be replaced by 'approximative selection'. The term 'measurement packets' will be replaced by 'export packet'.

    An issue of discussion is the reporting of the sequence counter. Suggestion is to report it regularly, but not with every packet report. The issue will be put on the mailing list. Support of encryption of packet reports will be changed to be REQUIRED.

    Reporting sequence numbers is considered to enable re-normalization of measured bytes.

    He discussed export requirements including confidentiality and timeliness. Is there a need to request that a delay SHOULD not exceed 1 second?
    Q: [Andy] why not make it configurable?
    Q: [Nick] why would an implementation not want to export ASAP?
    A: [Andy] there may be resource issues
    A: [Maurizio Molina] implementer may wish to group packets for export

    There is the general feeling that the export protocol for PSAMP is waiting on IPFIX.
    A: [Andy] coupling with IPFIX is not just to reuse documents, but also implementation code
    A: [?] reminder that SCTP is already RFC defined, PR-SCTP is in IEST queue packet selection



    Tanja Zseby on packet selection

    Tanja presented progress on the packet selection document. Terminology is now almost consistent with the framework document. After recent discussions only small changes are required here. Packet selectors could be part of IPFIX or other metering process. A still open issues is the set of methods for flow state sampling. Further open issues of the document include: Should we include some examples for flow state sampling or just leave it open to vendor-specified parameters based on the info model? Is there a better term for hash-based sampling than "emulation" or "approximation"?
    A: [Jürgen] re: the flow state sampling, since the known current methods are experimental, perhaps we should just say they are vendor-specified (ie. w/o examples)
    A: [Maurizio] perhaps it would help to have one example (such as sample and hold)



    Benoit Claise on the PSAMP protocol

    Benoit discussed the initial version of the PSAMP protocol document describing how the IPFIX protocol is used for exporting packet records. The IPFIX documents only speak about 'flow records'. These will be used as 'packet records' in PSAMP. The information model of IPFIX and PSAMP overlap. The PSAMP information model can be defined as extension of the IPFIX information model.
    A: [Plonka] timeliness of export needs to be mentioned in PSAMP protocol and the other drafts (such as requirements) need to take into consideration that if you choose an existing transport like IPFIX, that it might impose limits on the timeliness
    A: [Andy] doubts that its reasonable to demand some explicit export time, especially as little as one second, perhaps it should just be user configurable [presumably within the implementation limits]
    A: [Benoit] IPFIX does have a timeout, to get timely reporting
    A: [Dave Plonka] in IPFIX, there is currently no explicit bound on the time from flow timeout until export
    A: [Nick] motivation for 1 second "time limit", we want to specify a maximum, so that its not uncontrolled
    A: [Benoit] you can piggyback this on IPFIX



    Benoit Claise on the PSAMP information model

    Benoit presented the initial draft of the PSAMP information model. It is defined as extension of the IPFIX information model and adds a data type of a variable length byte array and it adds several information elements. It needs to be synchronized with the packet selection document.
    Q: [Nick] Do you have fields defined for all the sampling types, such as hash-based?
    A: [Benoit/Jürgen] not yet, yes - that needs to be added. This document is still in a premature state.



    Jürgen Quittek [on behalf of Thomas] on the PSAMP MIB

    Very small changes were committed to the PSAMP MIB since the document depends on the packet selection document that was submitted just before the cut-off date leaving no time to reflect it in the PSAMP MIB. Committed changes concerned the support of multiple receivers of packet samples. We need to determine which portions are mandatory and which optional.
    A: [Andy/Jürgen] regarding the existing drafts, suggest perhaps 4 weeks time to deal with existing open issues, and take them to last call
    A: [Jürgen] document editors should take the open issues to mailing list, try to get them addressed in 4 weeks, to enter last call in december, perhaps to wrap-up in January.



    Maurizio Molina about hash functions

    So far, the WG documents to not precisely describe hash functions. The presented individual I-D suggests to do so and defines some hash functions that have already been tested.
    A: [Andy] re: specification of hash function, I'm relunctant to add more RFCs to the initial PSAMP effort. If this document were an informational RFC (resulting from an individual draft) that would be sufficient.
    A: [Nevil Brownlee] re: hash functions - they're hard to get right, it would be better for PSAMP to settle on a short list (of no more than 3), and concurs with Andy's suggestion for new hash algorithms to be individual drafts specified outside PSAMP.
    A: [Nick] there is a benefit from PSAMP implementations using the same hash functions, for consistency there is a trade-off between strength and complexity when trying to identify the "best" hash functions for PSAMP.
    A: [Andy] if hash functions need to be in their own draft so that they can be easily edited, that probably means that they're not mature enought to be part of PSAMP yet. He suggest taking the discussion to the mailing list.
    A: [Andy] PSAMP could publish detailed functional requirements of a candidate hash function, rather than specifying a specific one with pseudo-code.
    A: [Nick] trying to avoid different vendors using different hash functions - because that's not that useful.
    A: [Andy] OK, well then there needs to be some agreement [on standard hash functions]



    Emile Stephan about spacial metrics for IPPM

    Emile suggests that one use of psamp is to determine the delay and loss of individual packets [and thus would perhaps benefit from IPPM defined spatial metrics for same].
    Q: [Henk Uijterwaal] why can't you apply the existing IPPM metrics (to points H1 to H2 in slide #3)?
    A: [Emile] [something about it not being end-to-end]
    Q: [Andy] Nick, do we have any requirement on time syncronization?
    A: [Nick] there is no such PSAMP requirement specified, but we do say that when timestamps are supplied that the accuracy is indicated.
    Q: [Andy] [to Emile] are you suggestion some change to PSAMP [to accomodate this potential application of PSAMP data]?
    Q: [Nick] Does IPFIX address timestamp issues?
    A: [Nevil] Yes, microsecond resolution is possible in export data type, but the accuracy and resolution are not have to be specified.
    A: [Tanja] QoS monitoring is one of IPFIX' target applications, and that would somewhat depend on timestamps.



    Session conclusion

    Andy asked authors to post their open issues to the mailing list.
    Q: [Randy Bush] how are we doing on progress?
    A: [Andy] plan for Jan. 10 for two main drafts to go to last call.


    Slides

    Agenda
    PSAMP Framework Document
    Sampling and Filtering Technology for IP Packet Selection - Update
    PSAMP Protocol Specifications
    PSAMP Information Model
    PSAMP MIB Status: Document Changes
    Draft-niccolini-hash-descr-00