2.4.10 Packet Sampling (psamp)

Last Modified: 2003-06-27

Chair(s):
Andy Bierman <abierman@cisco.com>
Juergen Quittek <quittek@ccrle.nec.de>
Operations and Management Area Director(s):
Randy Bush <randy@psg.com>
Bert Wijnen <bwijnen@lucent.com>
Operations and Management Area Advisor:
Randy Bush <randy@psg.com>
Mailing Lists:
General Discussion: psamp@ops.ietf.org
To Subscribe: psamp-request@ops.ietf.org
In Body: subscribe
Archive: https://ops.ietf.org/lists/psamp/
Description of Working Group:
The Packet Sampling working group is chartered to define a standard
set of capabilities for network elements to sample subsets of packets
by statistical and other methods. The capabilities should be simple
enough that they can be implemented ubiquitously at maximal line
rate. They should be rich enough to support a range of existing and
emerging measurement-based applications, and other IETF working groups
where appropriate.

The focus of the WG will be to (i) specify a set of selection
operations by which packets are sampled (ii) specify the information
that is to be made available for reporting on sampled packets; (iii)
describe protocols by which information on sampled packets is reported
to applications; (iv) describe protocols by which packet selection and
reporting configured.

Packet reports must be communicable in a timely manner, to
applications either on-board of off-board the sampling network
element. The streams of packet reports produced by a packet sampling
must (i) allow consistent interpretation, independent of the
particular network element that produced them; (ii) be self-defining,
in that their interpretation does not require additional information
to be supplied by the network element; (iii) allow robustness of
interpretation with respect to missing reports or part of reports;

Network elements shall support multiple parallel packet samplers, each
with independently configurable packet selectors, reports, report
streams, and export. Network elements must allow easy and secure
reconfiguration of these packet samplers by on-board or external
applications.

Export of a report stream across a network must be congestion avoiding
in compliance with RFC 2914. Unreliable transport is permitted because
the requirements at the exporter for reliable transport (state
maintenance, addressibilty, acknowledgment processing, buffering
unacknowledged data) would prevent ubiquitous deployment. Congestion
avoidance with unreliable export is to be accomplished by the
following measures, which shall be mandatory to implement and use. The
maximum export rate of a report stream must be configurable at the
exporter. A report stream must contain sufficient information for
transmission loss to be detected a collector. Then the collector must
run a congestion control algorithm to compute a new sending rate, and
reconfigure the exporter with this rate. In order to maintain report
collection during periods of congestion, PSAMP report streams may
claim more than a fair share of link bandwidth, provided the number of
report streams in competition with fair sharing traffic is limited.

Selection of the content of packet reports will be cognizant of
privacy and anonymity issues while being responsive to the needs of
measurement applications, and in accordance with RFC 2804.

Re-use of existing protocols will be encouraged provided the protocol
capabilities are compatible with PSAMP requirements.

Specifically, the PSAMP WG will perform the following tasks, in
accordance with the principles stated above:

1. Selectors for packet sampling. Define the set of primitive packet
  selection operations for network elements, the parameters by which
  they may be configured, and the ways in which they can be combined.

2. Packet Information. Specify extent of packet that is to be made
  available for reporting. Target for inclusion the packet's IP
  header, some subsequent bytes of the packet, and encapsulating
  headers if present. Full packet capture of arbitrary packet
  streams is explicitly out of scope. Specify variants for IPv4 and
  IPv6, extent of IP packet available under encapsulation methods,
  and under packet encryption.

3. Sampled packet reports. Define the format of the report that is
  constructed by the network element for each sampled packet for
  communication to applications. The format shall be sufficiently
  rich as to allow inclusion in the packet report of (i) IP packet
  information as specified in paragraph 2 above; (ii) encapsulating
  packet headers as specified in paragraph 2 above; (iii) interface
  or channel identifiers associated with transit of the packet across
  the network element; (iv) quantities computable from packet content
  and router state, (v) quantities computed during the selection
  operation. All reported quantities must reflect the router state
  and configuration encountered by the packet in the network element.

4. Report Streams. Define a format for a stream of packet reports, to
  include: (i) the format of packet reports in the stream; (ii) the
  packet reports themselves; (iii) configuration parameters of the
  selectors of the packets reported on; (iv) configuration parameters
  and state information of the network element; (v) quantities that
  enable collectors and applications to infer of attained packet
  sampling rates, detect loss during samping, report loss in
  transmission, and correct for information missing from the packet
  report stream; (vi) indication of the inherent accuracy
  of the reported quantities, e.g., of timestamps.

5. Multiple Report Streams. Define requirements for multiple parallel
  packet samplers in one network element, including the allowed
  degradation of packet reporting when packets are selected by
  multiple packet samplers.

6. Configuration and Management. Define a packet sampler MIB to reside
  at the network element, including parameters for packet selection,
  packet report and stream format, and export. Select or define a
  communication protocol to configure/read this MIB.

7. Presentation, Export, and Transport of Packet Reports. Define
  interface for presentation of reports to on-board applications.
  Select unreliable transport protocol for remote export. Determine
  rate control algorithms for export.

Initial Internet-Draft: A Framework for Passive Packet Measurement
[draft-duffield-framework-papame]
Goals and Milestones:
Done  Submit initial Framework document
Done  Submit initial Packet selector and packet information document
Feb 03  Submit initial Report format and report stream format document
Mar 03  Submit initial Export and requirements for collectors document
Done  Submit initial MIB document
May 03  Submit final Framework document
May 03  Submit final Packet selector and packet information document
Sep 03  Submit final Report format and report stream format document
Oct 03  Submit final Export and requirements for collectors document
Nov 03  Submit final MIB document
Internet-Drafts:
  • - draft-ietf-psamp-framework-03.txt
  • - draft-ietf-psamp-sample-tech-02.txt
  • - draft-ietf-psamp-mib-00.txt
  • No Request For Comments

    Current Meeting Report

    
    OPS Area
    PSAMP WG Meeting Minutes
    IETF #57
    July 17, 2003
    Minutes by Andy Bierman
    
    Review Material
    ---------------
    
      A) draft-ietf-psamp-framework-03.txt
      B) draft-ietf-psamp-sample-tech-02.txt
      C) draft-ietf-psamp-mib-00.txt
    
    Agenda
    ------
    
      1) WG Status
      2) Sampling Framework Issues
      3) Packet Selection Issues
      4) Report Format and Export Protocol
      5) PSAMP MIB
    
    Minutes
    -------
    
    1) WG status
    
    The milestones for the working group were presented and briefly 
    discussed: 
    
    Initial   Final Draft     Draft        Description
    
    ----------------------------------------
    ------------------------------
     Done      May 03       PSAMP Framework 
     Done      May 03       Packet Selector and Packet Information 
     Feb 03    Sep 03       Report Format and Report Stream Format
     Mar 03    Oct 03       Export Protocol and Requirements for 
    Collectors 
     Done      Nov 03       PSAMP MIB 
    
    The report format and export protocol documents are behind schedule 
    because the working group decided to use the mechanisms developed by the 
    IPFIX WG for these purposes. The IPFIX WG is not scheduled to complete 
    these documents until the end of the year. 
    
    2) Sampling Framework Issues 
    
    The Framework draft (A) was discussed by the group. 
    
    2.1) Terminology 
    
    Some clarifications are needed in PSAMP terminology:
       - PSAMP device is not a precise enough term 
       - need clarification between filtering and sampling.  Distinction is not 
    always clear (e.g., hashing) 
    
    2.2) Conformance 
    
    It is not clear where specific conformance requirements should be 
    specified.  It is likely that requirements related to a portion of PSAMP 
    covered by another document, such as packet selection or report export, 
    should be moved to that document. 
    
    Some packet report requirements were discussed, such as
       - packet reports must contain:
              - first N bytes
              - report sequence number
              - administrative details (selector ID) 
    
    The congestion avoidance requirement for the export process was 
    discussed briefly.  There is a desire to meet this requirement in a way 
    that is not too burdensome on PSAMP devices. 
    
    The requirements for the PSAMP MIB were briefly discussed. These include 
    some mandatory configuration objects and some optional (additional) 
    complex configuration objects, for such features as multiple selectors and 
    filtering. 
    
    Security requirements were also discussed.  There is a need for secure 
    export, which includes:
        - confidentiality
        - integrity 
        - authentication of sender 
        - secure configuration 
    
    2.3) Open Issues 
    
    There may be a need to make adjustments to the document for better 
    alignment with IPFIX terminology.  There are some minor sections (such as an 
    overview of configuration) that need to be completed.  The group agreed 
    that the current  draft is ready for WG Last Call. 
    
    3) Packet Selection 
    
    The Packet Selection draft (B) was discussed by the group. Updates since the 
    last draft were presented.  Refer to the slides for the details of that 
    presentation.  The significant changes made since the last draft 
    include:
       - better alignment with the framework draft
       - description of packet selection schemes updated
       - conformance changed so that one scheme must be supported but which one 
    is up to the implementor
       - concepts of filtering and sampling aligned with framework
       - listed sampling techniques and parameters 
    
    The group was asked if the current list of packet selection schemes is 
    sufficient and there was agreement it is sufficient. 
    
    3.1) Open Issues 
    
    There are some details that need to be added to the document, such as a 
    description of hash-based sampling.  The information model details will be 
    moved to the report format document. Details about which hash 
    functions also needs to be resolved. A mechanism for defining high level 
    filter specifications is needed.  There is some terminology that could be 
    better aligned related to the IPFIX metering process.  It is expected that 
    the next draft will be ready for WG Last Call. 
    
    4) Report Format and Export Protocol 
    
    A presentation on the IPFIX Report and Export protocol was given.  Refer to 
    the slides for details.  The IPFIX Information Model and IPFIX Protocol 
    documents are relevant to the PSAMP report format and export protocol, and 
    should be read by the working group. 
    
    4.1) Open Issues 
    
    Separate documents are needed for the PSAMP Information Model and PSAMP 
    usage of the IPFIX protocol.  These documents will rely on the IPFIX 
    documents, and only add enough details to support unambiguous 
    conformance for PSAMP purposes.  These documents are expected to be 
    started before the next IETF meeting in November 2003. 
    
    5) PSAMP MIB 
    
    A presentation on the PSAMP MIB was given.  Refer to the slides for 
    details.  The MIB structure follows the structure of the framework and 
    packet selection mechanisms.  There is a group describing packet 
    selection methods.  For each method, there are objects which report the 
    type, capabilities, and configured parameters.   
    
    There is a reporting group used to configure the addresses of the report 
    collectors.  There is an instance group, which identifies the specific 
    active sampling/reporting processes that are active on the PSAMP device.  
    There is a group that identifies the sampling methods supported by the 
    device. This needs to better align with the framework and packet  
    selection terminology and selection methods. 
    
    5.1) Open Issues
    
    The following open issues were identified:
    
      - Information about Filtering and Sampling Reports is currently not 
    included
      - Several sampling methods are not yet defined
      - Packet filtering needs to be added to sampling methods
      - The parameter set definition of sampling methods needs to be checked 
    carefully, some capability definitions are not yet complete.
      - Conformance statement and security considerations needed
      - Row status objects in writable tables needed 
      - Support for reporting to multiple collectors needed
      - Some descriptor name lengths exceed SMI recommendations
    
    Two issues were resolved at the meeting:
      - Should there be support for more than two sampling parameter sets per 
    instance?   The group decided two sets is sufficient.
      - Should there be a single object for capabilities, using BITS syntax, or 
    should there be multiple boolean objects, as in the current draft?  The 
    group decided one BITS object would be better.
    
    A new draft of the PSAMP MIB is expected about six weeks after the IETF 
    meeting. 
    
    6) Proposal on per-packet record export  
    
    There was some time left at the end of the meeting, so a 
    presentation was added which proposes to add per-packet fields to the 
    IPFIX (or PSAMP) information model.  Some new features, such as 
    additional filtering of reports based on packet contents and a global 
    packet ID were requested. 
    
    There was some objection to addressing this work in the PSAMP WG and 
    concern about the significant resources that would be required on a PSAMP 
    device to support these features. 
    
    Note that packet selection is not affected, just the report  process is 
    impacted by this proposal.  Instead of the first N  bytes, an export 
    process would have to extract named fields and maybe apply filters 
    against the payload, to determine which fields to include in the report. 
    
    Nobody in the group said they needed these reporting features. It was 
    suggested that RMON filtering should be used for packet content 
    filtering.  This proposal may be discussed further on the WG mailing list.  
    It is possible that the information model additions can be done 
    independently of the current PSAMP effort.   
    

    Slides

    PSAMP WG
    Current Status of the IPFIX WG
    PSAMP MIB Status
    Sampling and Filtering Techniques for IP Packet Selection