Subject: Re: [tcpPrague] [tsvwg] FW: I-D Action: draft-black-tsvwg-ecn-experimentation-00.txt

From: John Leslie <[email-masked]>
Date: Wed, 5 Oct 2016 10:27:45 -0400

Black, David <[email-masked]> wrote:
> 
> This is the process-oriented draft on ECN experimentation that I promised
> at the TSVWG meeting in Berlin. Comments are welcome, but please keep
> the process focus in mind - the intent is to leave documentation of the
> actual ECN changes and rationale to the referenced drafts that document
> the ECN experiments.

   Thank you. I have read this draft; and agree it covers what it needs to.

   I note the intended status is "Standards Track". Is this intended for
action by the TSVWG group?

--
John Leslie <[email-masked]>

---

From: Ingemar Johansson S <[email-masked]>
Date: Thu, 6 Oct 2016 08:31:30 +0000

Hi

Thanks for writing up this draft. I have some comments on the contents

Intended status : Standards track. I interpret this that it means that this document becomes standards track. 
Are there any ideas around the intended status for a document such as briscoe-tsvwg-ecn-l4s-id, will it be experimental first or ?

Section 4.2 : " "Unless otherwise specified by an Experimental RFC, routers treat
      the ECT(0) and ECT(1) codepoints as equivalent, and senders are
      free to use either the ECT(0) or the ECT(1) codepoint to indicate
      ECT, on a packet-by-packet basis.""
In some way I find it to contradict the proposed statement in  section 3 " Protocols and senders that only require a single ECT codepoint  SHOULD use ECT(0)."
I see here a risk that the statement in 4.2 can delay deployment of L4S capability based on the use of ECT(1) in the networks ?

Section 5 : I see the main problem with RFC6679 (in this context) in the use of the nonce. Don't however see any major problem to use feedback also for an L4S capable sender/receiver using ECT(1). 
For that purpose it may be better to rewrite :
      Use of ECT(1) and random ECT values is discouraged, as that may
      expose RTP to differences in network treatment of ECT(1) and
      ECT(0), e.g., as proposed in [I-D.briscoe-tsvwg-ecn-l4s-id].
To :
      Use of ECT values is discouraged, as that may
      expose RTP to differences in network treatment of ECT(1) and
      ECT(0), e.g., as proposed in [I-D.briscoe-tsvwg-ecn-l4s-id].

Further ... quote " In support of Alternative Backoff experimentation ...". I would say that this applies to L4S as well. As an example, SCReAM (RMCAT WG congestion control candidate) is easily modified to support L4S, and the RFC6679 feedback can be used with SCReAM. For that reason it may be better to rephrase it like 
" In support of Alternative Backoff experimentation and experiments based on [I-D.briscoe-tsvwg-ecn-l4s-id]....." or something similar

Regards
/Ingemar

> -----Original Message-----
> From: Black, David [mailto:[email-masked]]
> Sent: den 1 oktober 2016 00:42
> To: [email-masked]; [email-masked]; [email-masked]
> Cc: Black, David <[email-masked]>
> Subject: [tcpPrague] FW: I-D Action: draft-black-tsvwg-ecn-experimentation-
> 00.txt
> 
> This is the process-oriented draft on ECN experimentation that I promised at
> the TSVWG
> meeting in Berlin.   Comments are welcome, but please keep the process
> focus in mind -
> the intent is to leave documentation of the actual ECN changes and rationale
> to the referenced drafts that document the ECN experiments.
> 
> Thanks, --David
> 
> -----Original Message-----
> From: I-D-Announce [mailto:[email-masked]] On Behalf Of
> [email-masked]
> Sent: Friday, September 30, 2016 6:24 PM
> To: [email-masked]
> Subject: I-D Action: draft-black-tsvwg-ecn-experimentation-00.txt
> 
> 
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> 
> 
>         Title           : Explicit Congestion Notification (ECN) Experimentation
>         Author          : David Black
> 	Filename        : draft-black-tsvwg-ecn-experimentation-00.txt
> 	Pages           : 10
> 	Date            : 2016-09-30
> 
> Abstract:
>    Multiple protocol experiments have been proposed that involve changes
>    to Explicit Congestion Notification (ECN) as specified in RFC 3168.
>    This memo summarizes the proposed areas of experimentation to provide
>    an overview to the Internet community and updates RFC 3168, a
>    Proposed Standard RFC, to allow the experiments to proceed without
>    requiring a standards process exception for each Experimental RFC to
>    update RFC 3168.  This memo also makes related updates to the ECN
>    specification for RTP in RFC 6679 for the same reason.  Each
>    experiment is still required to be documented in an Experimental RFC.
>    This memo also records the conclusion of the ECN Nonce experiment in
>    RFC 3540, obsoletes RFC 3540 and reclassifies it as Historic.
> 
> 
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-black-tsvwg-ecn-experimentation/
> 
> There's also a htmlized version available at:
> https://tools.ietf.org/html/draft-black-tsvwg-ecn-experimentation-00
> 
> 
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
> 
> _______________________________________________
> I-D-Announce mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/i-d-announce
> Internet-Draft directories: http://www.ietf.org/shadow.html or
> ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>

---

From: "Black, David" <[email-masked]>
Date: Thu, 6 Oct 2016 23:59:33 +0000

John,

>    Thank you. I have read this draft; and agree it covers what it needs to.

Thank you.

> action by the TSVWG group?

Yes, Thanks, --David

> From: John Leslie [mailto:[email-masked]]
> Sent: Wednesday, October 05, 2016 10:28 AM
> To: Black, David
> Cc: [email-masked]; [email-masked]
> Subject: Re: [tsvwg] FW: I-D Action: draft-black-tsvwg-ecn-experimentation-
> 00.txt
>

---

From: "Black, David" <[email-masked]>
Date: Fri, 7 Oct 2016 00:28:12 +0000

Hi Ingemar,

David> Inline

Thanks, --David

> From: Ingemar Johansson S [mailto:[email-masked]]
> Sent: Thursday, October 06, 2016 4:32 AM
> To: Black, David; [email-masked]; [email-masked]; [email-masked]
> Cc: Ingemar Johansson S
> Subject: RE: [tcpPrague] FW: I-D Action: draft-black-tsvwg-ecn-experimentation-
> 00.txt
> 
> Hi
> 

Thanks for carefully reading the draft.

> Are there any ideas around the intended status for a document such as briscoe-
> tsvwg-ecn-l4s-id, will it be experimental first or ?

Possibly - I don' t know.  The ecn-experimentation draft is not intended to
settle that question for the l4s-id draft or any of the other drafts involved
in the described areas of experimentation.

I wrote the ecn-experimentation draft to allow the experiments to be specified
in Experimental RFCs without each experiment also needing a companion Standards
Track RFC just to open up space for the experiment.   If any of the experiments
pan out so well that they deserve to be immediately written up as a Standards
Track RFC without bothering with an Experimental RFC, then that would be
wonderful, IMHO.
 
> In some way I find it to contradict the proposed statement in  section 3 "
> Protocols and senders that only require a single ECT codepoint  SHOULD use
> ECT(0)."
> I see here a risk that the statement in 4.2 can delay deployment of L4S capability
> based on the use of ECT(1) in the networks ?

My understanding is that L4S requires both ECT(0) and ECT(1), as it couples
treatments of traffic marked with those codepoints.  For that reason, the
statement in section 3 (which is quoted from RFC 3168) does not apply
to L4S.

OTOH, I see your concern, namely that "free to use" in the RFC 3168 text quoted
in 4.2 is somewhat at odds with "SHOULD use ECT(0)" in the RFC 3168 text quoted
in section 3.  This is a concern with RFC 3168 itself.

Would it help for this ecn-experimentation draft to update RFC 3168 by
removing the words "and senders are free to use either the ECT(0) or the
ECT(1) codepoint to indicate ECT, on a packet-by-packet basis."  ?

> Section 5 : I see the main problem with RFC6679 (in this context) in the use of the
> nonce. Don't however see any major problem to use feedback also for an L4S
> capable sender/receiver using ECT(1).

Good point - I agree.  I would insert "random" after "Use of" in the "To:" text.
I'll make that change in my working copy that will become -01.

> Further ... quote " In support of Alternative Backoff experimentation ...". I would
> say that this applies to L4S as well. As an example, SCReAM (RMCAT WG
> congestion control candidate) is easily modified to support L4S, and the RFC6679
> feedback can be used with SCReAM. For that reason it may be better to rephrase
> it like
> " In support of Alternative Backoff experimentation and experiments based on
> [I-D.briscoe-tsvwg-ecn-l4s-id]....." or something similar

I'd prefer to do something different.   I think L4S is an example of both the Alternative
Backoff and ECT Differences areas of experimentation, and hence I'd suggest citing
an L4S draft in the description of the Alternative Backoff experiment scope in Section
2.  Which L4S draft would you suggest?

---

From: Ingemar Johansson S <[email-masked]>
Date: Fri, 7 Oct 2016 06:38:06 +0000

Hi

Comments inline [IJ]

> Sent: den 7 oktober 2016 02:28
> To: Ingemar Johansson S <[email-masked]>;
> [email-masked]; [email-masked]; [email-masked]
> Subject: RE: [tcpPrague] FW: I-D Action: draft-black-tsvwg-ecn-
> experimentation-00.txt
> 
> Hi Ingemar,
> 
> David> Inline
> 
> Thanks, --David
> 
> > Are there any ideas around the intended status for a document such as
> > briscoe- tsvwg-ecn-l4s-id, will it be experimental first or ?
> 
> settle that question for the l4s-id draft or any of the other drafts involved in
> the described areas of experimentation.
> 
> I wrote the ecn-experimentation draft to allow the experiments to be
> specified in Experimental RFCs without each experiment also needing a
> companion Standards
> pan out so well that they deserve to be immediately written up as a
> Standards Track RFC without bothering with an Experimental RFC, then that
> would be wonderful, IMHO.
> 
> > Protocols and senders that only require a single ECT codepoint  SHOULD
> > use ECT(0)."
> > I see here a risk that the statement in 4.2 can delay deployment of
> > L4S capability based on the use of ECT(1) in the networks ?
> 
> statement in section 3 (which is quoted from RFC 3168) does not apply to L4S.
[IJ] It is perhaps here I see things differently. 
My interpretation is that L4S is that senders mark packets with some special codepoint (let's call it ECT(1) ) and network queues detect this and send the packets to a dedicated queue which marks these packets to ECN-CE at a very low queue delay. A special case may be that senders are networks are 100% L4S (Data centers), this means that only the code point ECT(1) is in use.
The handling of ECT(0) and not-ECT packets goes outside the L4S definition and definitely requires dualQ solution and classification. 
It is quite possible that my interpretation is flawed. 

> 
> OTOH, I see your concern, namely that "free to use" in the RFC 3168 text
> quoted in 4.2 is somewhat at odds with "SHOULD use ECT(0)" in the RFC 3168
> text quoted in section 3.  This is a concern with RFC 3168 itself.
> 
[IJ] Yes, it probably solves things, mind though that reason why I brought this is up is that I may interpret the definition of L4S differently.

> 
> > Section 5 : I see the main problem with RFC6679 (in this context) in
> > the use of the nonce. Don't however see any major problem to use
> > feedback also for an L4S capable sender/receiver using ECT(1).
> > Further ... quote " In support of Alternative Backoff experimentation
> > ...". I would say that this applies to L4S as well. As an example,
> > SCReAM (RMCAT WG congestion control candidate) is easily modified to
> > support L4S, and the RFC6679 feedback can be used with SCReAM. For
> > that reason it may be better to rephrase it like " In support of
> > Alternative Backoff experimentation and experiments based on
> Backoff and ECT Differences areas of experimentation, and hence I'd suggest
> citing an L4S draft in the description of the Alternative Backoff experiment
> scope in Section 2.  Which L4S draft would you suggest?
[IJ] Good question.. Not sure what is required here?. Our proprietary SCReAM research code has support for L4S, one possibility is to update the SCReAM draft with a description of this functionality. An example was shown at the L4S BoF (page 6-7 in https://www.ietf.org/proceedings/96/slides/slides-96-l4s-2.pdf )

> > > This is the process-oriented draft on ECN experimentation that I
> > > promised at the TSVWG
> > > the intent is to leave documentation of the actual ECN changes and
> > > rationale to the referenced drafts that document the ECN experiments.
> > >
> > > Thanks, --David
> > >
> > > From: I-D-Announce [mailto:[email-masked]] On Behalf
> > > Of [email-masked]
> > > Please note that it may take a couple of minutes from the time of
> > > submission until the htmlized version and diff are available at
> tools.ietf.org.
> > >

---

From: "Black, David" <[email-masked]>
Date: Fri, 7 Oct 2016 16:29:06 +0000

Ingemar,

Summarizing:

[1] Section 4.2 - "free to use" text.  The full quote from RFC 3168 is:

	"Routers treat the ECT(0) and ECT(1) codepoints as equivalent. Senders are free to use either the ECT(0)
	 or the ECT(1) codepoint to indicate ECT, on a packet-by-packet basis."

I've revised my working version of the -01 draft to remove the second sentence as part of this proposed update to RFC 3168.  That's both simpler and clearer than what's in the posted -00.  In addition, doing this obviates the need to say anything in this section about the possible L4S relationships between the ECT(0) and ECT(1) queues.

[2] Section 2 - L4S and description of Alternative Backoff.  If one of the folks directly involved in L4S would like to suggest a draft to cite that describes the L4S backoff for the low latency/higher-CE-marking-probability functionality, I'm happy to cite that as an additional example of Alternative Backoff.

Thanks, --David

> Sent: Friday, October 07, 2016 2:38 AM
> My interpretation is that L4S is that senders mark packets with some special
> codepoint (let's call it ECT(1) ) and network queues detect this and send the
> packets to a dedicated queue which marks these packets to ECN-CE at a very low
> queue delay. A special case may be that senders are networks are 100% L4S (Data
> centers), this means that only the code point ECT(1) is in use.
> The handling of ECT(0) and not-ECT packets goes outside the L4S definition and
> definitely requires dualQ solution and classification.
> [IJ] Yes, it probably solves things, mind though that reason why I brought this is
> up is that I may interpret the definition of L4S differently.
> 
> >
> [IJ] Good question.. Not sure what is required here?. Our proprietary SCReAM
> research code has support for L4S, one possibility is to update the SCReAM draft
> with a description of this functionality. An example was shown at the L4S BoF
> (page 6-7 in https://www.ietf.org/proceedings/96/slides/slides-96-l4s-2.pdf )
> 
> >
> > >
> > > Regards
> > > /Ingemar
> > >

---

From: Michael Welzl <[email-masked]>
Date: Fri, 14 Oct 2016 21:19:34 +0200

Dear all,

I’d like to state, on behalf of the ABE authors, that we’re fine with the draft as it stands, including the update below.

Just a note of clarification, answering a comment from you in a previous email of yours:
"My understanding is that L4S requires both ECT(0) and ECT(1), as it couples treatments of traffic marked with those codepoints.”

=> No.
L4S uses ECT(1), ABE uses ECT(0), and because of that, the two are compatible.
( Maybe you got confused by past discussion that was less clear in that respect. )

I repeat this for others, as I just received an unrelated email from someone else pointing me at L4S possibly competing with ABE:
L4S and ABE are *not* competing proposals. Because L4S uses the ECT(1) codepoint, it is gradually deployable without creating a conflict with ABE (or current RFC3168-style senders).

Cheers,
Michael

> On 7. okt. 2016, at 18.29, Black, David <[email-masked]> wrote:
> 
> Ingemar,
> 
> Summarizing:
> 
> tcpPrague mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpprague

---

From: "De Schepper, Koen (Nokia - BE)" <[email-masked]>
Date: Fri, 14 Oct 2016 21:58:56 +0000

Hi all,

Thanks David, I think this draft is useful to support further L4S 
specifications.

Some excerpts and comments from the mails below: 

I agree with Ingemar that the text:
     "Unless otherwise specified by an Experimental RFC, routers treat
      the ECT(0) and ECT(1) codepoints as equivalent, and senders are
      free to use either the ECT(0) or the ECT(1) codepoint to indicate
      ECT, on a packet-by-packet basis."
Should better become:
     "Unless otherwise specified by an Experimental RFC, routers treat
      the ECT(0) and ECT(1) codepoints as equivalent, and senders should
      use only the ECT(0) codepoint to indicate ECT."
There is no reason anymore to use ECT(1) by senders, unless an Experimental
RFC specifies differently.

> I've revised my working version of the -01 draft to remove the second
> sentence as part of this proposed update to RFC 3168.
I think it is clearer to explicitly state that only ECT(0) should be used.

> statement in section 3 (which is quoted from RFC 3168) does not apply to L4S.
L4S is not changing the meaning of non-ECT or ECT(0) packets. 
It still assumes a response by the sender equal as a response to drop. The 
coupling is only towards ECT(1). Routers will more frequently mark ECT(1)
packets to compensate the more aggressive behavior of L4S. ECT(1) marking
is steered by the non-ECT drop and ECT(0) marking probability such that 
flow rates become equal independent of (non-)ECT(0/1).

> The handling of ECT(0) and non-ECT packets goes outside the L4S definition
Correct. Nothing changes for L4S.

> [2] Section 2 - L4S and description of Alternative Backoff.  If one of the
> folks directly involved in L4S would like to suggest a draft to cite that
> describes the L4S backoff for the low latency/higher-CE-marking-probability
> functionality
Technically speaking L4S has an alternative response to a CE marked packet, 
but only to a packet that was sender-marked as ECT(1) and when the network
marks packets more frequently by the routers.

I think we need to clarify if the text in this section is intended only 
for responses in the sender "without" a compensating change in the network 
between drop and CE marks, or also "with" a compensation. In the first case
we should also specify this, and in the second case we just can also refer
to the L4S draft (where we assume a DCTCP or TCP-Prague response).

Regards,
Koen.

> From: tcpPrague [mailto:[email-masked]] On Behalf Of Black,
> David
> Sent: vrijdag 7 oktober 2016 18:29
> To: Ingemar Johansson S; [email-masked]; [email-masked];
> [email-masked]
> Cc: Black, David
> Subject: Re: [tcpPrague] FW: I-D Action: draft-black-tsvwg-ecn-
> experimentation-00.txt
> 
> Ingemar,
> 
> Summarizing:
> 
> 	"Routers treat the ECT(0) and ECT(1) codepoints as equivalent.
> Senders are free to use either the ECT(0)
> I've revised my working version of the -01 draft to remove the second
> sentence as part of this proposed update to RFC 3168.  That's both simpler
> and clearer than what's in the posted -00.  In addition, doing this obviates
> the need to say anything in this section about the possible L4S
> relationships between the ECT(0) and ECT(1) queues.
> 
> [2] Section 2 - L4S and description of Alternative Backoff.  If one of the
> folks directly involved in L4S would like to suggest a draft to cite that
> describes the L4S backoff for the low latency/higher-CE-marking-probability
> functionality, I'm happy to cite that as an additional example of
> Alternative Backoff.
> 
> Thanks, --David
> 
> experimentation-
> > > > Section 4.2 : " "Unless otherwise specified by an Experimental RFC,
> routers
> > > treat
> > > I'd prefer to do something different.   I think L4S is an example of
> both the
> > > Alternative

---

From: "Black, David" <[email-masked]>
Date: Sun, 16 Oct 2016 18:21:12 +0000

[1] 

> RFC specifies differently.

Ok.

[... snip...]

> I think it is clearer to explicitly state that only ECT(0) should be used.

That's already been done in -00, via section 3 quoting this stronger text
from RFC 3168:

      Protocols and senders that only require a single ECT codepoint
      SHOULD use ECT(0).

That should suffice.

[2] 

> to the L4S draft (where we assume a DCTCP or TCP-Prague response).

Which L4S draft should be referenced?  I believe I agree in general with the
above concern, but I need to know which L4S draft should be referenced on
the "with" approach in order to write the actual text for the ECN Experimentation
draft.

Thanks, --David

> From: De Schepper, Koen (Nokia - BE) [mailto:koen.de_schepper@nokia-bell-
> labs.com]
> Sent: Friday, October 14, 2016 5:59 PM
> To: Black, David; Ingemar Johansson S; [email-masked]; [email-masked];
> [email-masked]

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 24 Oct 2016 18:56:51 +0100

David,

A couple of responses inline...

>        Protocols and senders that only require a single ECT codepoint
[BB] If we're going to rely on that sentence, we need to add your 
exception for experimental RFCs to it. Then it becomes:

       "Unless otherwise specified by an Experimental RFC, when the ECN
       nonce has not already been implemented, protocols and senders MUST
       solely use ECT(0)."

The SHOULD has to turn into a MUST, because all the possible exceptions 
that the original SHOULD allowed are now listed explicitly in the 
sentence, leaving no other exceptions that you want to allow.

> the "with" approach in order to write the actual text for the ECN Experimentation
[BB] draft-briscoe-tsvwg-ecn-l4s-id-01 has the text you need.
It attempts to specify both the difference in network behaviour and 
sender behaviour that is associated with ECT(1).

Specifically, our current attempt to specify the transport behaviour is in:
https://tools.ietf.org/html/draft-briscoe-tsvwg-ecn-l4s-id-01#section-2.3

Cheers

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: "Black, David" <[email-masked]>
Date: Sat, 29 Oct 2016 00:55:20 +0000

Bob,

Thanks for this follow-up.

[1] Should ECT(1) be Reserved now?

At this point, I think this issue ought to go to the Seoul TSVWG meeting, so
the -02 version, will contain the current RFC 3168  "SHOULD use" for ECT(0), and
indicate that whether that ought to become a "MUST use" for ECT(0) is an open
issue.

[2] Relationship of L4S to Alternate Backoff

Thanks for the L4S draft reference, there will be text in the -02 of the ECN
experimentation draft on this.

Thanks, --David

> From: Bob Briscoe [mailto:[email-masked]]
> Sent: Monday, October 24, 2016 1:57 PM
> To: Black, David; De Schepper, Koen (Nokia - BE); Ingemar Johansson S;
> Subject: Re: [tsvwg] [tcpPrague] FW: I-D Action: draft-black-tsvwg-ecn-
> experimentation-00.txt
> 
> David,
> 
> >>>> packets to a dedicated queue which marks these packets to ECN-CE at a
> very
> >>> low