closed.New IETF Standards Track Discussion BOF (newtrk) Thursday, November 13 at 1530-1730 CHAIR: Scott Bradner <sob@harvard.edu> Minutes taken by Spencer Dawkins 1/ Scott presented a description of current IETF Standards Track (see presentation) Scott described the history of the IETF standards process, which has had three formal stages since RFC 1083 by Jon Postel. RFC 1083 had "proposed protocols" as the first stage. Scott did not know why it became "proposed standard" in late 1990 or early 1991. RFC 1310 was the first standalone process description before RFC 1310 the description was part of the IANA standards report. Scott noted that some documents have many more informal stages - for example the current BGP draft is version 22. The requirements for Draft Standard include multiple interoperable implementations to demonstrate document clarity and a requirement for each feature to be implemented to eliminate unused features. Scott reminded us that Draft Standard was changed in RFC 2026 to include a test for IPR licensing. Full Standard is Draft Standard with wide spread use. 2/ observations from problem working group - Elwyn Davies (See presentation) Elwyn described the work of the Problem Working Group relating to the IETF standards track He quoted from 2.4 section of problem statement draft, rev 5 and noted that some people felt that there were some issues relating to the currently implemented IETF standards process. "three-stage process not used properly" - progression was rare higher quality bar for Proposed Standard (at least a perception) quality checked by thought exercises, not running code Proposed Standards deployed as if they were mature this, in turn, raises the quality bar and slows things down, so people sometimes deploy I-Ds instead of waiting for Proposed Standards There is no IETF aftercare for standards including no systematic way to receive bug reports and no periodic reviews Scott Brim: a purpose of Draft Standard is a quality/clarity check of the documentation Bob Hinden specifically, that one can implement from the spec. basically we've lost our running code heritage Dol strom: vendors don't wait for proposed standard 3/ what other SDOs do - Scott Bradner Scott: W3C: W3C uses three-plus stages, pretty much what we use working draft is internal-only (See presentation) Joel Halpern: ISO: ISO process stars with a committee draft with number assigned at the beginning of the process, the formal stages include Draft International Standard and International Standard Ces DeLat: Global Grid Forum (GGF) - modeled after IETF with minor differences four types of documents but no document has reached our second stage have to show interoperability and running code to progress GGF is 80 percent researchers, so specs lag review pretty badly working groups should shut down as soon as possible (sometimes too soon) Steve Woodlock: Open Group (see slides) 10-week timed process after draft iteration company review process and change requests followed by voting board approval Tom Taylor: ITU-T The ITU-T has Study Groups, made up of working parties and questions Recommendations are the outputs of the ITU-T process all decisions on recommendations are made at meetings The document editor just edits the process is driven by written contributions last call issues when recommendation when "finished", last-call started by working parties if consensus on last-call, study group just approves the document if no consensus, try again at a higher level with governments as the voters maintenance - recommendations issued as new versions, identified by date assigned to a question for maintenance Stephen Hayes: 3GPP the 3GPP process is either 1-stage or n-stage, conceptually one type of document - Technical Specification increase in functionality over time - version number increments three-level version numbers major updates every 18 months strong rules about backwards compatibility, but things do change over time Alex Conta: noted that some of these organizations are based on corporate membership this is a radical difference with the IETF IEEE has similar process starts with contributions, goes through drafts, then to ballots in WG and final committee approval membership requirements are very strict (based on individuals) document changes include patches of earlier specs 4/ proposals for alternate standards track processes 4a/ draft-dawkins-pstmt-twostage-01.txt - Dave Crocker (see presentation) Issues include barriers, unused stages, unused process, false advertising, uncoordinated use of drafts, and cruft in the archives. Dave proposes a track that includes 3 stages: Working Group Snapshot, Proposed Standard, Internet Standard PS requires running code, but just one implementation PS is technical vetting, Internet Standard is a popularity contest No requirement for multiple implementations - one widely deployed implementation can be a standard PS goes away after 36 months Working Group Snapshot is same as Scott's, working group synchronization, but IESG comments, doesn't approve totally under control of working group Pete Resnick: for IS, does "community adoption" mean one implementation? Dave: yes, although it sounds crazy... Scott Bradner: "it goes away?" Dave: goes to Historic (RFCs don't go away) John Leslie: can you get from ID to PS in less than 6 months? Dave: it's an ID, it can be handled like an ID 4b/ draft-bradner-ietf-stds-trk-00.txt - Scott Bradner (see presentation) IESG has changed the bar for Proposed Standard - doesn't let documents with nits go to Proposed Standards Vendors don't wait for PS, and implement from random Internet drafts instead. There is no significant difference between draft standard and full standard. The effect is that the whole process has been shifted one step to the left. Scott proposed a 3 stage process starting with a "stable snap shot" (SSS) - level of IESG review is very low - a SSS is specifically a immature prestandard specification with no specific publication requirements. The other two stages are Proposed Standard, largely unchanged from the way the IESG currently implements it, and Internet Standard, which is a combination of the current Draft and Internet Standard. Keith Moore: noted that the "no known technical omissions" requirement in 2026 means a lot more than it used to (security) in addition now there's an IESG provided nits list, etc. Chris Bemondi: noted that I-D nits focuses on normative references, so lengthens publications queue, too Bob Hinden: noted that we now have two stages of IDs, individual and working group draft Christian Huitema: we have seen a creeping insertion of a new step in the process working group draft is now the old PS, modulo some things Aaron Falk: are SSSs ephemeral? Scott Bradner: depends on the proposal, but in my proposal they are not ephemeral Craig White: what's the "big I Internet"? Scott Bradner: A non-isolated IP network 4c/draft-iesg-hardie-outline-00.txt - Leslie Daigle (see presentation) draft-iesg-hardie-category-descriptions-00.txt Leslie Daigle , speaking for Ted Hardie (who was tied up in another session), described Ted's proposed category descriptions. His proposal uses a cooking analogy (ingredients, baking, baked, eaten, boiled) "Boiled" is "cooked by another process" - Experimental, etc. "Baking" could be archival one could also loosen lockstep reference requirements difference in archival series new archival series, "Candidate Specification" new designation, "Full Standard" for documents that prove interoperability Aside - might want to talk about sharing RFC series (includes non-IETF documents) Bob Hinden: observed that Leslie was doing a lot of cooking, but we're starving to death :-}. He noted that one of the problems we have is WG thinks it's finished, IESG doesn't agree Leslie: Ted also has document on document approval Kurt Zeilenga: asked are individual submissions processed different? Leslie: Ted also proposes a variety of working group types Aaron Falk: what about "spoiled" category? Leslie: working group work items are an archival series 4d/ draft-loughney-what-standards-00.txt - John Loughney (see presentation) John observed that the IETF has produced lots of RFCs, but not a lot of standards Mobile Phone TCP team that just used STD 7/RFC 793, and was not sure what RFCs to use. There are bugs in published specs - is there an errata process? new working group? what do you implement? No way to figure out what the best version of a protocol is... We just produce more specifications. He offered some suggestions for things that might help: Better web interface? Maintenance teams? Early STD teams? Enhanced STD numbers? 4e/ Maintaining Standards - Brian Carpenter Brian mentioned that he was trying to deprecate a feature in a draft standard but that the process was not a clean as it might be - How should one document a feature that is deprecated? As a SHOULD NOT or a MUST Not? IETF technologies do not have version numbers which can make it easier to know what text people are referring to. Fortran knew how to deprecate features - a feature was listed in a version of Fortran as deprecated and would not appear in the next version. Vendors would say what version they supported so the user could know what features were present. TCP is the extreme example of the confusion with IETF standards. Brian would love to have a one-step standards process with version numbers and a clear mechanism for identifying interoperability Scott Bradner: remember the IPR licensing "feature" of Draft Standard when we evaluate proposals Spencer: What about conditional IPR statements based on standards status/standards track? Brian: The new IPR procedure is better, but we still need to nail this down. Dave Perkins: Re: versioning - what about references to specific versions? Brian: ASN had that problem - we need to understand this aspect of normative references, we need to be able to do both kinds of normative references 5/ what would define success in a revised IETF Standards track - Scott Bradner (see presentation) Scott listed some things measures that might indicate that a revised standards process was a "success": more advancement along the standards track fewer products based on random I-Ds? better understanding within IETF/WG of document status? trade press understanding of document status? Aaron Falk: documents are in state that corresponds to their status now. Is 43 implementations of a PS is a failure? If it happens all the time, yes. But the effort required for a change is proportional to size of change and this could be a big change. Keith Moore: names of standard types need to coorrespond to reality. Particularly standards must be of high quality. We need to be clear on the kind of endorsement we are providing. And we need to be clear on currency - "relevancy" of the standard. Alan DeCock: define success in terms of failure - should there be a failed documents archive? Harald Alverstrand: the success of the IETF is in producing standards that make the Intenet work. We don't do this for fun. Guessing what to implement, and getting it wrong, is a failure. Our metrics should correspond to this. Mark Handley: harald said that success is that the Internet works better but it works fairly well on Internet Drafts and Proposed Standards - they must have clarity and relevance. Bill Strawn: Te current standards track isn't too difficult to understand and we should not care about the press confusion. We have a lot of problems - how many are the result of slow publication process? Elliot Lear: success is when you take a TCPdump, identify the protocols that are running, and show they are documented in RFCs John Loughney - There is no objective measure of success. There will be problems - success is in the eye of the beholder. There is some problem with the current process - we may not agree on what it is, but relevance of the work we do is what matters. Dave Crocker: This environment is too solution-rich! There are too many possibilities to tweak - what needs change badly? Media silliness doesn't do any damage, and we can't fix it anyway. Cruft in archives is more of a problem. We should minimize the number of changes to minimize risk. Christian Huitema: - He really likes Scott's proposal, which he finds the most realistic, although the other proposals are not very different. Christian observes that there is a danger in defining our work as "preventing harm to the Internet". This is a negative statement that leads to censorship. The IETF pendulum has swung a long way from permitting free experimentation to only allowing publication after extremely careful reviews; we need to swing back the other way. Currently, the "word on the street" is that the IETF spends a lot of time doing nothing. If we don't make it possible for people to agree without two years of censorship, we will fail Bob Hinden - The IETF has stopped building the Internet, but new protocols are still be added independent of what the IETF does. The control being asserted by the IESG has not worked (except to make the IETF less effective). Harald: Making the Internet work better means experiments, things we can do, fun doesn't mean blocking stuff, it means doing stuff. 6/ open discussion Scott asked if there is consensus in the room that we need a change - the hum was clearly "yes" Brian Zorn: "success of the IETF" in my own terms - "less trouble" Two of the protocols I'm working on are I-Ds, there are multiple implementations and shipping products, nobody implemented the RFC. If it's not on time, don't even bother. We can't change protocols between I-D versions because it's already shipping. What about Individual submissions? Brian feels that we need version numbers, with interoperability statements. When a protocol takes years to come out - the people who want to use it have surrendered and left the building. We need to balance too fast and too slow, i.e. implementing from IDs vs. waiting for PS. Melinda Shore: Melinda asked about retrieval - standardization process isn't doing archiving, everything should be archived and retrievable. She thinks we need protocol specification version numbers. She feels that choosing working group documents is full of whimsy today, and should stay that way. What do you need to read to understand what a working group is working on - the charter is insufficient. Building a standards track around what particular documents are working group documents is the wrong thing to do. Keith Moore: Is something wrong with our labeling? probably, but something is wrong with our process - we are iterating toward perfection - too many working groups are in FIN wait. Working groups need to shut down so they don't start doing strange things. The document process needs to reflect this. Don't leave out Individual Submissions - they have to be accommodated. We need some kind of waterfall model for all this stuff - don't make IESG evaluate clean-sheet proposals with no experience. Maybe we should have sketches before snapshots? WGs shoot down incomplete specs and don't read complete ones, so we read powerpoint! Louis: The customers of these documents don't care about labels. IDs used to be "ideas." The market picks what they want to deploy. Deploying Internet Drafts is fine, except for non-archiving. The IETF doesn't make Internet work, but it's where good work gets done. Make some tweaks and don't change much - adding process just won't happen. Elwyn: Re: backwards compatibility - would my 1995 Sun/3 work today? Should we require backwards compatibility for standards at some level? Pre-standard Ethernet and modem vendors offered guaranteed upgrades to standard versions. Scott Bradner: How do we make changes to standards-track protocols when new technology comes along if we focus too much on backward compatibility? Elwyn: We need to be careful about what we standardize. Alex: Two elements need to be addressed - number of stages and moving between stages - everybody has stages, no matter where you go. We need interoperable implementations. Moving between stages doesn't work well at IETF. Labeling could be radically improved - no indication of protocol, standard level, version number. Jonathan Sadler: do we need to change the stages of normative references? Kurt: we should ensure (as RFC 2026 does) the any revised IETF process allow individuals to produce any and all formal IETF outputs (e.g., all classes/categories of RFCs). Brian Zorn: protocols that were individual submissions, but shouldn't have been - they were not WG documents when even though a WG said yes the WG charter said no. But companies are shipping the technology anyway. Aaron: we should have multiple stages with interoperable implementations - we don't do conformance testing. Alex: We should have at most two stages, we need the IPR hook, we need synchronization between IETF formality and Internet reality, there's a huge disconnect here. We need a lower bar with cross-functional review. Mark Handley: How can we make the Internet work better? we should discourage implementation and encourage experimentation, while discouraging deployment. Mark notes that one can't tell stages of a draft - no idea what attributes an I-D has under current process (e.g., reviewed by congestion police). Alex: Alex says that since documents are only advanced in meetings maybe we need more than three meetings per year? Scott Bradner: Work is also done on mailing lists, no need for a meeting to advance a document SUMMARY - Scott asked for a quick show of hands to gage the feeling in the room on a number of topics: strong support were shown for the ideas that: change is needed some type of working group snapshot is needed multiple implementations needed to show document clarity some kind of "IPR hook" is needed that a one stage process was not enough Scott asked Harald, as AD, to take over Harald: Do we have a rough idea of what we want to do? Yes, we need more discussion for WG. Melinda: We really do know what the question is, just not the answer. We need more openness with other changes under consideration. Bob Hinden: go for a WG now, there is a problem to solve Craig White: Should we go back to interoperability testing? What is running code? We have no clarity about this Alex: know what the problem is and have suggestions on how to fix it Mark Handley: We have strong consensus change is needed, need to address it somewhere Harald asked: can we write a WG charter now? the feeling in the room was "yes" Harald asked for volunteers for WG charter team and asked those volunteers to send a note to sob@harvard.edu. He also said that we'll start a mailing |