Subject: [spring] Is MSD really a configurable attribute?

From: Muthu Arul Mozhi Perumal <[email-masked]>
Date: Wed, 5 Apr 2017 22:08:21 +0530

draft-ietf-spring-sr-yang seems to describe Maximum SID Depth (MSD) as a
read-write attribute that is configurable on the node, but I really wonder
how many vendors actually support changing the MSD on a node.

Suppose a node is capable of pushing a maximum of K labels in h/w and the
node MSD is configured as K, then a SR-TE tunnel on the node can specify up
to K SIDs. This means the node will not be able to push a VPN label, so
cannot do L3VPN/L2VPN. Given that a miss-configuration like would result in
service failure, is there a real motivation for changing MSD on a node?
Should MSD be a node capability instead, like the
'readable-label-stack-depth' defined in the yang draft?

Regards,
Muthu

---

From: Jeff Tantsura <[email-masked]>
Date: Wed, 05 Apr 2017 09:56:32 -0700

Hi Muthu,

Thanks for your comments!

MSD is a configurable attribute, it is not derived directly from HW capabilities, in fact no vendor today provides an API to query underlying HW for the MSD supported, there’s also dependency on SW support. 

That’s why we have introduced “Type” field, so more than a single MDS type could be signaled, as of now, we have only defined “Base” Type, that describes total number of SID’s supported. I’d expect vendors to provide clear guidance wrt MSD semantics, in disaggregated case, when HW and SW are coming from different vendors, I’d expect HW to be the limiting factor and HW vendors to provide an API to query for the MSD supported and auto-populate the value in IGPs. I have reached out to BCM and Barefoot, plan to discuss with more HW vendors.

Per node vs per LC capability – even on a same generation NPU, depending on revision, MSD supported could vary drastically, routers with 3 generations of line cards are not an exception either, so MSD per adj/interface is a rather valuable information to a PCE if a tunnel could exit over different line cards. Per node MSD limits computation to the lowest value supported by the node.

Hope this helps,

Cheers,

Jeff

From: spring <[email-masked]> on behalf of Muthu Arul Mozhi Perumal <[email-masked]>
Date: Wednesday, April 5, 2017 at 09:38
To: <[email-masked]>
Subject: [spring] Is MSD really a configurable attribute?

draft-ietf-spring-sr-yang seems to describe Maximum SID Depth (MSD) as a read-write attribute that is configurable on the node, but I really wonder how many vendors actually support changing the MSD on a node. 

Suppose a node is capable of pushing a maximum of K labels in h/w and the node MSD is configured as K, then a SR-TE tunnel on the node can specify up to K SIDs. This means the node will not be able to push a VPN label, so cannot do L3VPN/L2VPN. Given that a miss-configuration like would result in service failure, is there a real motivation for changing MSD on a node? Should MSD be a node capability instead, like the 'readable-label-stack-depth' defined in the yang draft?

Regards,

Muthu

_______________________________________________ spring mailing list [email-masked] https://www.ietf.org/mailman/listinfo/spring

---

From: Alexander Vainshtein <[email-masked]>
Date: Wed, 5 Apr 2017 17:04:39 +0000

Jeff, Muthu and all,
I concur with Jeff – MSD is not defined just by HW but also by SW.
Same HW may yield different MSD values with SW defining different data paths thru it.
And it may well be a matter of tradeoff where higher MSD could be achieved at the expense of some other parameters.
This is exactly why MSD should be treated as a configurable attribute.
Of course this does not preclude implementations when exactly one MSD value would be supported.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302
Email:   [email-masked]

From: spring [mailto:[email-masked]] On Behalf Of Jeff Tantsura
Sent: Wednesday, April 05, 2017 7:57 PM
To: Muthu Arul Mozhi Perumal <[email-masked]>; [email-masked]
Subject: Re: [spring] Is MSD really a configurable attribute?

Hi Muthu,

Hope this helps,

Cheers,
Jeff

From: spring <[email-masked]<mailto:[email-masked]>> on behalf of Muthu Arul Mozhi Perumal <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, April 5, 2017 at 09:38
To: <[email-masked]<mailto:[email-masked]>>
Subject: [spring] Is MSD really a configurable attribute?

Regards,
Muthu

_______________________________________________ spring mailing list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/spring

___________________________________________________________________________

This e-mail message is intended for the recipient only and contains information which is 
CONFIDENTIAL and which may be proprietary to ECI Telecom. If you have received this 
transmission in error, please inform us by e-mail, phone or fax, and then delete the original 
and all copies thereof.
___________________________________________________________________________

---

From: Muthu Arul Mozhi Perumal <[email-masked]>
Date: Wed, 5 Apr 2017 23:32:36 +0530

Hi Jeff,

Thanks for your response. Comments inline..

On Wed, Apr 5, 2017 at 10:26 PM, Jeff Tantsura <[email-masked]>
wrote:

> MSD is a configurable attribute, it is not derived directly from HW
> capabilities, in fact no vendor today provides an API to query underlying
> HW for the MSD supported, there’s also dependency on SW support.
>

Since MSD is not derived from h/w capabilities, did you actually mean that
no vendor provides an API to query the underlying h/w label imposition
limit? MSD being a s/w attribute, I believe it can take any value from 1 to
the label imposition limit supported by the h/w?​

>
>
> That’s why we have introduced “Type” field, so more than a single MDS type
> could be signaled, as of now, we have only defined “Base” Type, that
> describes total number of SID’s supported.
>

​Ok, do we expect MSD types to be tied to the application/service, say one
MSD to be used for L3VPN service on a node, one for L2VPN etc?​

> I’d expect vendors to provide clear guidance wrt MSD semantics, in
> disaggregated case, when HW and SW are coming from different vendors, I’d
> expect HW to be the limiting factor and HW vendors to provide an API to
> query for the MSD supported and auto-populate the value in IGPs. I have
> reached out to BCM and Barefoot, plan to discuss with more HW vendors.
>

​Again, guess you meant the label imposition limit of the h/w here?

>
>
> Per node vs per LC capability – even on a same generation NPU, depending
> on revision, MSD supported could vary drastically, routers with 3
> generations of line cards are not an exception either, so MSD per
> adj/interface is a rather valuable information to a PCE if a tunnel could
> exit over different line cards. Per node MSD limits computation to the
> lowest value supported by the node.
>

​Agree, this becomes really tricky with a router supporting different NPU
types/generations, so node MSD is one thing a PCE can rely on for sure..

Regards,
Muthu

>
>
> Hope this helps,
>
>
>
> Cheers,
>
> Jeff
>
>
>
>
>
> *From: *spring <[email-masked]> on behalf of Muthu Arul Mozhi
> Perumal <[email-masked]>
> *Date: *Wednesday, April 5, 2017 at 09:38
> *To: *<[email-masked]>
> *Subject: *[spring] Is MSD really a configurable attribute?
>
>
>
> _______________________________________________ spring mailing list
> [email-masked] https://www.ietf.org/mailman/listinfo/spring
>

---

From: Muthu Arul Mozhi Perumal <[email-masked]>
Date: Wed, 5 Apr 2017 23:42:32 +0530

Hi Saha,

Thanks for your inputs. Comments inline..

> at the expense of some other parameters.

​draft-ietf-isis-segment-routing-msd has the foll:

​    In case, there are additional labels (e.g. service) that are to
    be pushed to the stack - MSD SHOULD be adjusted to reflect that

​If the node needs 2 labels to support L3VPN and the h/w label imposition
limit is 6 (say), then MSD could be set to 4 to support up to 4 SIDs in a
SR-TE tunnel, right? OTOH, if the node is just a 'P' router, MSD could be
set as high as 6. Is that what you mean by trading off higher MSD at the
expense of something else?

Regards,
Muthu​

> Office: +972-39266302 <+972%203-926-6302>
>
> Cell:      +972-549266302 <+972%2054-926-6302>
>
> *From:* spring [mailto:[email-masked]] *On Behalf Of *Jeff
> Tantsura
> *Sent:* Wednesday, April 05, 2017 7:57 PM
> *To:* Muthu Arul Mozhi Perumal <[email-masked]>; [email-masked]
> *Subject:* Re: [spring] Is MSD really a configurable attribute?
>
>
>
> Hi Muthu,
>
>
>
> describes total number of SID’s supported. I’d expect vendors to provide
> clear guidance wrt MSD semantics, in disaggregated case, when HW and SW are
> coming from different vendors, I’d expect HW to be the limiting factor and
> HW vendors to provide an API to query for the MSD supported and
> auto-populate the value in IGPs. I have reached out to BCM and Barefoot,
> plan to discuss with more HW vendors.
>
>
>

---

From: Alexander Vainshtein <[email-masked]>
Date: Wed, 5 Apr 2017 18:24:49 +0000

Muthu hi,
Two points:

1.       My reading of the text in the draft to which you refer is different: from my POV it means that the MSD advertised in the protocol must take into account all labels that can be pushed on a packet (including L3VPN or PW “application” labels, entropy labels/flow labels) and not just the labels that represent the list of SIDs for SR-TE

2.       When I mentioned increase of MSD at expense of some other parameters, I had in mind something else. E.g., if the label stack to be pushed on the packet is stored in a fixed size entry in the  “egress encapsulation” database in the forwarding HW, one option would be to use one such entry (with the resulting limit on the MSD) per LSP, while another option would be to use a linked list of such entries per LSP. This would increase the MSD at the expense of the number of LSP out segments that the device can support.

Hope this helps.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

From: Muthu Arul Mozhi Perumal [mailto:[email-masked]]
Sent: Wednesday, April 05, 2017 9:13 PM
To: Alexander Vainshtein <[email-masked]>
Cc: Jeff Tantsura <[email-masked]>; [email-masked]; Shell Nakash <[email-masked]>; Michael Gorokhovsky <[email-masked]>; Ron Sdayoor <[email-masked]>; Rotem Cohen <[email-masked]>
Subject: Re: [spring] Is MSD really a configurable attribute?

Hi Saha,

On Wed, Apr 5, 2017 at 10:34 PM, Alexander Vainshtein <[email-masked]<mailto:[email-masked]>> wrote:
Jeff, Muthu and all,
I concur with Jeff – MSD is not defined just by HW but also by SW.
Same HW may yield different MSD values with SW defining different data paths thru it.
And it may well be a matter of tradeoff where higher MSD could be achieved at the expense of some other parameters.

​If the node needs 2 labels to support L3VPN and the h/w label imposition limit is 6 (say), then MSD could be set to 4 to support up to 4 SIDs in a SR-TE tunnel, right? OTOH, if the node is just a 'P' router, MSD could be set as high as 6. Is that what you mean by trading off higher MSD at the expense of something else?

Regards,
Muthu​

Regards,
Sasha

Office: +972-39266302<tel:+972%203-926-6302>
Cell:      +972-549266302<tel:+972%2054-926-6302>
Email:   [email-masked]<mailto:[email-masked]>

From: spring [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Jeff Tantsura
Sent: Wednesday, April 05, 2017 7:57 PM
To: Muthu Arul Mozhi Perumal <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [spring] Is MSD really a configurable attribute?

Hi Muthu,

Hope this helps,

Cheers,
Jeff

Regards,
Muthu

---

From: Muthu Arul Mozhi Perumal <[email-masked]>
Date: Thu, 6 Apr 2017 00:23:33 +0530

Hi Saha,

Please see inline..

On Wed, Apr 5, 2017 at 11:54 PM, Alexander Vainshtein <
[email-masked]> wrote:

> Muthu hi,
>
> Two points:
>
> 1.       My reading of the text in the draft to which you refer is
> different: from my POV it means that the MSD advertised in the protocol
> must take into account all labels that can be pushed on a packet (including
> L3VPN or PW “application” labels, entropy labels/flow labels)
>
​That makes MSD same as the label imposition limit supported by the h/w,
right? What is the motivation for s/w configuring the MSD on a node then?

My interpretation was that the label stack has 2 parts -- the service part
and the LSP part. ​MSD is the maximum no. of SIDs that can go into the LSP
part. Now, MSD can be modified by the s/w and advertised in IGP/PCEP/BGP-LS
depending on what MPLS services the node provides..

Regards,
Muthu

> and not just the labels that represent the list of SIDs for SR-TE
>
> 2.       When I mentioned increase of MSD at expense of some other
> parameters, I had in mind something else. E.g., if the label stack to be
> pushed on the packet is stored in a fixed size entry in the  “egress
> encapsulation” database in the forwarding HW, one option would be to use
> one such entry (with the resulting limit on the MSD) per LSP, while another
> option would be to use a linked list of such entries per LSP. This would
> increase the MSD at the expense of the number of LSP out segments that the
> device can support.
>
>
>
>
>
> Hope this helps.
>
>
>
> Regards,
>
> Sasha
>
>
>
> *From:* Muthu Arul Mozhi Perumal [mailto:[email-masked]]
> *Sent:* Wednesday, April 05, 2017 9:13 PM
> *To:* Alexander Vainshtein <[email-masked]>
> *Cc:* Jeff Tantsura <[email-masked]>; [email-masked]; Shell
> Nakash <[email-masked]>; Michael Gorokhovsky <
> [email-masked]>; Ron Sdayoor <[email-masked]>;
> Rotem Cohen <[email-masked]>
>

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 6 Apr 2017 08:16:49 +0000

Muthu,
Two clarifications:

1.       The number of “service-related” labels depends on the service. Flow-aware PWs (RFC 6391<https://tools.ietf.org/html/rfc6391>), entropy labels for IP VPN (RFC 6790<https://tools.ietf.org/html/rfc6790>) and, possibly, using GAL as a VCCV Indicator (RFC 7708<https://tools.ietf.org/html/rfc7708>) give you some examples. To the best of my understanding, the MSD value reflects maximum imposable label stack depth that includes all labels, it is not SR-specific at all.

2.       I believe that I have already explained how the same HW may support different MSD values depending on usage of some HW resources. In the example I’ve given, if a single “egress encapsulation database” entry can contain “N” labels, and “M” such entries are available in the forwarding HW memory, the user may configure MSD to N and expect HW to support “M”LSP  out-segments, or he/she may configure MSD to (2*N) and expect forwarding HW to support only (M/2) LSP out-segments.

Hope this clarifies my position.

Regards,
Sasha

Office: +972-39266302Muthu,

Cell:      +972-549266302

Sent: Wednesday, April 05, 2017 9:54 PM

Hi Saha,

Please see inline..

On Wed, Apr 5, 2017 at 11:54 PM, Alexander Vainshtein <[email-masked]<mailto:[email-masked]>> wrote:
Muthu hi,
Two points:

1.       My reading of the text in the draft to which you refer is different: from my POV it means that the MSD advertised in the protocol must take into account all labels that can be pushed on a packet (including L3VPN or PW “application” labels, entropy labels/flow labels)
​That makes MSD same as the label imposition limit supported by the h/w, right? What is the motivation for s/w configuring the MSD on a node then?

My interpretation was that the label stack has 2 parts -- the service part and the LSP part. ​MSD is the maximum no. of SIDs that can go into the LSP part. Now, MSD can be modified by the s/w and advertised in IGP/PCEP/BGP-LS depending on what MPLS services the node provides..

Regards,
Muthu

Hope this helps.

Regards,
Sasha

From: Muthu Arul Mozhi Perumal [mailto:[email-masked]<mailto:[email-masked]>]
Sent: Wednesday, April 05, 2017 9:13 PM
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: Jeff Tantsura <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>; Shell Nakash <[email-masked]<mailto:[email-masked]>>; Michael Gorokhovsky <[email-masked]<mailto:[email-masked]>>; Ron Sdayoor <[email-masked]<mailto:[email-masked]>>; Rotem Cohen <[email-masked]<mailto:[email-masked]>>

Hi Saha,

Regards,
Muthu​

Regards,
Sasha

Hi Muthu,

Hope this helps,

Cheers,
Jeff

Regards,
Muthu

---

From: Muthu Arul Mozhi Perumal <[email-masked]>
Date: Thu, 6 Apr 2017 15:37:52 +0530

Hi Sasha,

On Thu, Apr 6, 2017 at 1:46 PM, Alexander Vainshtein <
[email-masked]> wrote:

> Muthu,
>
> Two clarifications:
>
> 1.       The number of “service-related” labels depends on the service.
> Flow-aware PWs (RFC 6391 <https://tools.ietf.org/html/rfc6391>), entropy
> labels for IP VPN (RFC 6790 <https://tools.ietf.org/html/rfc6790>) and,
> possibly, using GAL as a VCCV Indicator (RFC 7708
> <https://tools.ietf.org/html/rfc7708>) give you some examples. To the
> best of my understanding, the MSD value reflects maximum imposable label
> stack depth that includes all labels, it is not SR-specific at all.
>
​Isn't it the same as the mplsMaxLabelStackDepth object defined in the MPLS
LSR MIB (RFC 3813), then ​?

​​mplsMaxLabelStackDepth  OBJECT-TYPE
   SYNTAX        Unsigned32 (1..2147483647)
   MAX-ACCESS    read-only
   STATUS        current
   DESCRIPTION
       "The maximum stack depth supported by this LSR."
::= { mplsLsrObjects 11 }

This is a read-only object, so I am wondering why MSD is read-write.
Anyway, we could perhaps name it as MPLS Label Stack Depth (MLSD), instead
of MSD,to indicate that it is not SR specific at all. We should also
clarify this in draft-ietf-isis-segment-routing-msd and other MSD drafts to
avoid misinterpretation, IMHO.

> 2.       I believe that I have already explained how the same HW may
> support different MSD values depending on usage of some HW resources. In
> the example I’ve given, if a single “egress encapsulation database” entry
> can contain “N” labels, and “M” such entries are available in the
> forwarding HW memory, the user may configure MSD to N and expect HW to
> support “M”LSP  out-segments, or he/she may configure MSD to (2*N) and
> expect forwarding HW to support only (M/2) LSP out-segments.
>
​This looks interesting. If I understood you correctly, you are saying that
a higher MSD value could be traded for a lower scale (in terms of LSP out
segments). But, I wonder which h/w vendor currently support it this way. My
understanding is that BCM supports only a fixed maximum impossible label
stack depth on a packet.

​Regards,
Muthu​

> ​​
> *Sent:* Wednesday, April 05, 2017 9:54 PM
>

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 6 Apr 2017 10:37:10 +0000

Muthu,
I may be wrong here, but I think that mplsMaxLabelStackDepth in RFC 3813<https://tools.ietf.org/html/rfc3813> most probably refers to maximum number of labels an LSR can simultaneously look up in its ILM  and not to the maximum number of labels an LSR can impose.

This makes sense to me since the former has been a well-known issue in 2004 (and earlier), e.g., if the LSR in question is an egress LER of an RSVP-TE LSP that uses FRR and is used as a tunnel LSP by a PW or by a L3 VPN), while the latter has mainly become an issue with SR-TE.

Regarding the trade-off between MSD and other HW resources:
Your understanding of my general intention is correct. But what is (or is not) supported by this or that chip vendor is out of scope, and, from my POV,  should not be discussed on the IETF mailing lists.
I can only say that, depending on the specific forwarding HW, there is more than one option for trade-offs, some of them quite ingenious.
Again, the IETF mailing list is not the right place for discussing actual data path implementations IMHO.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Sent: Thursday, April 06, 2017 1:08 PM

Hi Sasha,

On Thu, Apr 6, 2017 at 1:46 PM, Alexander Vainshtein <[email-masked]<mailto:[email-masked]>> wrote:
Muthu,
Two clarifications:

1.       The number of “service-related” labels depends on the service. Flow-aware PWs (RFC 6391<https://tools.ietf.org/html/rfc6391>), entropy labels for IP VPN (RFC 6790<https://tools.ietf.org/html/rfc6790>) and, possibly, using GAL as a VCCV Indicator (RFC 7708<https://tools.ietf.org/html/rfc7708>) give you some examples. To the best of my understanding, the MSD value reflects maximum imposable label stack depth that includes all labels, it is not SR-specific at all.
​Isn't it the same as the mplsMaxLabelStackDepth object defined in the MPLS LSR MIB (RFC 3813), then ​?

This is a read-only object, so I am wondering why MSD is read-write. Anyway, we could perhaps name it as MPLS Label Stack Depth (MLSD), instead of MSD,to indicate that it is not SR specific at all. We should also clarify this in draft-ietf-isis-segment-routing-msd and other MSD drafts to avoid misinterpretation, IMHO.

2.       I believe that I have already explained how the same HW may support different MSD values depending on usage of some HW resources. In the example I’ve given, if a single “egress encapsulation database” entry can contain “N” labels, and “M” such entries are available in the forwarding HW memory, the user may configure MSD to N and expect HW to support “M”LSP  out-segments, or he/she may configure MSD to (2*N) and expect forwarding HW to support only (M/2) LSP out-segments.
​This looks interesting. If I understood you correctly, you are saying that a higher MSD value could be traded for a lower scale (in terms of LSP out segments). But, I wonder which h/w vendor currently support it this way. My understanding is that BCM supports only a fixed maximum impossible label stack depth on a packet.

​Regards,
Muthu​

Regards,
​​
Sasha

Hi Saha,

Please see inline..

Regards,
Muthu

Hope this helps.

Regards,
Sasha

Hi Saha,

Regards,
Muthu​

Regards,
Sasha

Hi Muthu,

Hope this helps,

Cheers,
Jeff

Regards,
Muthu

---

From: Jeff Tantsura <[email-masked]>
Date: Thu, 06 Apr 2017 07:48:27 -0700

Hi Muthu,

Please see inline

Cheers,

Jeff

From: Muthu Arul Mozhi Perumal <[email-masked]>
Date: Wednesday, April 5, 2017 at 11:02
To: Jeff Tantsura <[email-masked]>
Cc: <[email-masked]>
Subject: Re: [spring] Is MSD really a configurable attribute?

Hi Jeff,

On Wed, Apr 5, 2017 at 10:26 PM, Jeff Tantsura <[email-masked]> wrote:

Hi Muthu,

Since MSD is not derived from h/w capabilities, did you actually mean that no vendor provides an API to query the underlying h/w label imposition limit? MSD being a s/w attribute, I believe it can take any value from 1 to the label imposition limit supported by the h/w?​

[jeff] No HW vendor, obviously if your device and SW it runs are vertically integrated (ie cisco/juniper router), there’s no difference, the MSD they should provide is the system/line card MSD as supported by the system

That’s why we have introduced “Type” field, so more than a single MDS type could be signaled, as of now, we have only defined “Base” Type, that describes total number of SID’s supported. 

​Ok, do we expect MSD types to be tied to the application/service, say one MSD to be used for L3VPN service on a node, one for L2VPN etc?​

[jeff] I don’t really see a use case here, however if there’s one, it is easy to realize with a new “Type” 

I’d expect vendors to provide clear guidance wrt MSD semantics, in disaggregated case, when HW and SW are coming from different vendors, I’d expect HW to be the limiting factor and HW vendors to provide an API to query for the MSD supported and auto-populate the value in IGPs. I have reached out to BCM and Barefoot, plan to discuss with more HW vendors.

[jeff] as explained above, it depends, in a vertically integrated system it would be a system capability, if it is HW only (i.e.white box), it would be HW capability. The end result would be the same though, the path, as computed by a PCE must not have MSD larger than communicated  

​Agree, this becomes really tricky with a router supporting different NPU types/generations, so node MSD is one thing a PCE can rely on for sure..

[jeff] Link MSD is optional and provides additional information, in absence of Link MSD, Node MSD (that represents the lowest MSD supported by a system) should be used

Regards,

Muthu

Hope this helps,

Cheers,

Jeff

Regards,

Muthu

---

From: Jeff Tantsura <[email-masked]>
Date: Thu, 06 Apr 2017 07:59:10 -0700

Muthu,

Sasha is right, this list is not a right place to discuss HW details, which is usually under NDA anyway.

Having worked with many different NPU’s – there are ways to program NPU’s differently, depending on scale/ throughput and ability to look below SDK, don’t forget about recirculation too. In a chassis packet processing is divided between ingress and egress NPU’s, which makes end result very implementation dependent. 

Cheers,

Jeff

From: Muthu Arul Mozhi Perumal <[email-masked]>
Date: Thursday, April 6, 2017 at 03:07
To: Alexander Vainshtein <[email-masked]>
Cc: Jeff Tantsura <[email-masked]>, "[email-masked]" <[email-masked]>, Shell Nakash <[email-masked]>, Michael Gorokhovsky <[email-masked]>, Ron Sdayoor <[email-masked]>, Rotem Cohen <[email-masked]>
Subject: Re: [spring] Is MSD really a configurable attribute?

Hi Sasha,

On Thu, Apr 6, 2017 at 1:46 PM, Alexander Vainshtein <[email-masked]> wrote:

Muthu,

Two clarifications:

1.       The number of “service-related” labels depends on the service. Flow-aware PWs (RFC 6391), entropy labels for IP VPN (RFC 6790) and, possibly, using GAL as a VCCV Indicator (RFC 7708) give you some examples. To the best of my understanding, the MSD value reflects maximum imposable label stack depth that includes all labels, it is not SR-specific at all.

   MAX-ACCESS    read-only

   STATUS        current

   DESCRIPTION

​Regards,

Muthu​

Regards,

​​

Sasha

Cell:      +972-549266302

Hi Saha,

Please see inline..

On Wed, Apr 5, 2017 at 11:54 PM, Alexander Vainshtein <[email-masked]> wrote:

Muthu hi,

Two points:

Regards,

Muthu

 

Hope this helps.

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

Hi Saha,

On Wed, Apr 5, 2017 at 10:34 PM, Alexander Vainshtein <[email-masked]> wrote:

Jeff, Muthu and all,

Regards,

Muthu​

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

Hi Muthu,

Hope this helps,

Cheers,

Jeff

Regards,

Muthu

---

From: Muthu Arul Mozhi Perumal <[email-masked]>
Date: Thu, 6 Apr 2017 21:06:23 +0530

While we have been discussing what MSD is, let me rephrase my original
question/problem that made me reach out to the WG in the first place:

Suppose a router supports a maximum imposible label stack depth of 4. For
simplicity, let's also assume that it is a pizza box, so has a single
linecard. Now, suppose the router is configured to do L3VPN with a PCE
(provided by another vendor) initiating SR-TE tunnels on the router to
egress PEs. The questions is, what MSD value should the router advertise to
the PCE?

draft-ietf-isis-segment-routing-msd says:

   MSD of type 1 (IANA Registry) is used to signal the number of SIDs a
   node is capable of imposing, to be used by a path computation
   element/controller *and is only relevant to the part of the stack*
*   created as the result of the computation*.

In this case, the part of the stack created as the result of the
computation is the SR-TE label stack. Apparently, if it advertises a MSD of
4 and the PCE sets up SR-TE tunnels consisting of 4 SIDs, then the router
can't do L3VPN (because it doesn't have room to impose the VPN label).

The draft further says:

   In case, there are
   additional labels (e.g. service) that are to be pushed to the stack -
   MSD SHOULD be adjusted to reflect that.

In this case there is an additional VPN label to be pushed onto to the
stack, so advertising an MSD of 3 should work.

That's the only logical conclusion I can infer from the text in the MSD
drafts. If we mean something different, then we need to improve the text in
the drafts for it to be interpreted in a consistent manner.

BTW, we often discuss what vendors support/implement based on their
publicaly available information in IETF, and I don't see anything wrong
with that (what good are IETF standards that can't be implemented?)

Regards,
Muthu

On Thu, Apr 6, 2017 at 4:07 PM, Alexander Vainshtein <
[email-masked]> wrote:

> Muthu,
>
> I may be wrong here, but I think that *mplsMaxLabelStackDepth* in RFC 3813
> <https://tools.ietf.org/html/rfc3813> most probably refers to maximum
> number of labels an LSR can simultaneously *look up in its ILM * and not
> to the maximum number of labels an LSR can *impose*.
>
>
>
> This makes sense to me since the former has been a well-known issue in
> 2004 (and earlier), e.g., if the LSR in question is an egress LER of an
> RSVP-TE LSP that uses FRR and is used as a tunnel LSP by a PW or by a L3
> VPN), while the latter has mainly become an issue with SR-TE.
>
>
>
> Your understanding of my general intention is correct. But what is (or is
> not) supported by this or that chip vendor is out of scope, and, from my
> POV,  should not be discussed on the IETF mailing lists.
>
> I can only say that, depending on the specific forwarding HW, there is
> more than one option for trade-offs, some of them quite ingenious.
>
> *Sent:* Thursday, April 06, 2017 1:08 PM
>
>    SYNTAX        Unsigned32 (1..2147483647 <02147%20483%20647>)
>
>    MAX-ACCESS    read-only
>
>    STATUS        current
>
>    DESCRIPTION
>
> clarify this in draft-ietf-isis-segment-routing-msd and other MSD drafts
> to avoid misinterpretation, IMHO.
>
> ​This looks interesting. If I understood you correctly, you are saying
> that a higher MSD value could be traded for a lower scale (in terms of LSP
> out segments). But, I wonder which h/w vendor currently support it this
> way. My understanding is that BCM supports only a fixed maximum impossible
> label stack depth on a packet.
>
>
>
> ​Regards,
>
> Muthu​
>
>
>

---

From: Alexander Vainshtein <[email-masked]>
Date: Thu, 6 Apr 2017 16:16:20 +0000

Muthu,
The Type 1 MSD that is signaled by the router to the PCE depends on specific of your L3 application and your network behavior
E.g., if you expect your PCE to return a loose ERO comprised mainly (or completely) of a small number of Node SIDs, your IP VPN packets will employ ECMP between each adjacent pair of nodes in this ERO.

If the routers in our network implement ECMP based on hashing of IP header fields, then you need just one application label in the label stack, so you should signal Type 1 MSD = 3.

However, if the routers in your network rely on entropy labels in the label stack for ECMP, signal the ELC attribute in NRLI for IP-VPN routes they distribute as defined in RFC 6970, and impose these labels at ingress, then you will need two additional label stack entries in your stack (one for the ELI reserved label and one for the entropy label itself). And, of course, you will still need the IP VPN application label. So you can only signal Type 1 MSD=1 to your PCE, i.e., there will be no SR traffic engineering at all.

Hopefully this addresses your concerns.

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

Sent: Thursday, April 06, 2017 6:36 PM

While we have been discussing what MSD is, let me rephrase my original question/problem that made me reach out to the WG in the first place:

Suppose a router supports a maximum imposible label stack depth of 4. For simplicity, let's also assume that it is a pizza box, so has a single linecard. Now, suppose the router is configured to do L3VPN with a PCE (provided by another vendor) initiating SR-TE tunnels on the router to egress PEs. The questions is, what MSD value should the router advertise to the PCE?

   MSD of type 1 (IANA Registry) is used to signal the number of SIDs a
   node is capable of imposing, to be used by a path computation
   element/controller and is only relevant to the part of the stack
   created as the result of the computation.

In this case, the part of the stack created as the result of the computation is the SR-TE label stack. Apparently, if it advertises a MSD of 4 and the PCE sets up SR-TE tunnels consisting of 4 SIDs, then the router can't do L3VPN (because it doesn't have room to impose the VPN label).

The draft further says:

   In case, there are

In this case there is an additional VPN label to be pushed onto to the stack, so advertising an MSD of 3 should work.

That's the only logical conclusion I can infer from the text in the MSD drafts. If we mean something different, then we need to improve the text in the drafts for it to be interpreted in a consistent manner.

BTW, we often discuss what vendors support/implement based on their publicaly available information in IETF, and I don't see anything wrong with that (what good are IETF standards that can't be implemented?)

Regards,
Muthu

On Thu, Apr 6, 2017 at 4:07 PM, Alexander Vainshtein <[email-masked]<mailto:[email-masked]>> wrote:
Muthu,
I may be wrong here, but I think that mplsMaxLabelStackDepth in RFC 3813<https://tools.ietf.org/html/rfc3813> most probably refers to maximum number of labels an LSR can simultaneously look up in its ILM  and not to the maximum number of labels an LSR can impose.

Regards,
Sasha

Hi Sasha,

​​mplsMaxLabelStackDepth  OBJECT-TYPE
   SYNTAX        Unsigned32 (1..2147483647<tel:02147%20483%20647>)
   MAX-ACCESS    read-only
   STATUS        current
   DESCRIPTION
       "The maximum stack depth supported by this LSR."
::= { mplsLsrObjects 11 }

​Regards,
Muthu​

Regards,
​​
Sasha

Hi Saha,

Please see inline..

Regards,
Muthu

Hope this helps.

Regards,
Sasha

Hi Saha,

Regards,
Muthu​

Regards,
Sasha

Hi Muthu,

Hope this helps,

Cheers,
Jeff

Regards,
Muthu

---

From: Muthu Arul Mozhi Perumal <[email-masked]>
Date: Fri, 7 Apr 2017 10:58:31 +0530

Sasha,

I think we are on the same page now. To sum up, Type 1 MSD is *not* always
the maximum imposible stack depth of the hardware/chip -- it is the maximum
imposible stack depth my MPLS applications wants to advertise to the rest
of the world. If there is no such application running on the node (e.g.
bare-metal environment), then it could be the hardware/chip limit, in which
case an external application (i.e PCE) is expected to provide the entire
label stack the node would have to push on to the packets.

This also explains why MSD need to be a configurable attribute -- it needs
to be configured based on the application running on the node. This
configured value is then advertised in IGPs/PCEP/BGP-LS. In the absence of
such a configuration, the node may end up always advertising the
hardware/chip limit, resulting in application/service failure.

Regards,
Muthu

On Thu, Apr 6, 2017 at 9:46 PM, Alexander Vainshtein <
[email-masked]> wrote:

> Muthu,
>
> The Type 1 MSD that is signaled by the router to the PCE depends on
> specific of your L3 application and your network behavior
>
> E.g., if you expect your PCE to return a loose ERO comprised mainly (or
> completely) of a small number of Node SIDs, your IP VPN packets will employ
> ECMP between each adjacent pair of nodes in this ERO.
>
>
>
> If the routers in our network implement ECMP based on hashing of IP header
> fields, then you need just one application label in the label stack, so you
> should signal Type 1 MSD = 3.
>
>
>
> However, if the routers in your network rely on entropy labels in the
> label stack for ECMP, signal the ELC attribute in NRLI for IP-VPN routes
> they distribute as defined in RFC 6970, and impose these labels at ingress,
> then you will need two additional label stack entries in your stack (one
> for the ELI reserved label and one for the entropy label itself). And, of
> course, you will still need the IP VPN application label. So you can only
> signal Type 1 MSD=1 to your PCE, i.e., there will be no SR traffic
> engineering at all.
>
>
>
> *Sent:* Thursday, April 06, 2017 6:36 PM
>

---

From: Jeff Tantsura <[email-masked]>
Date: Fri, 07 Apr 2017 02:20:49 -0700

Sasha, 

Thanks for your help! You were faster than me, 5G transport summit took all the time ☺

Muthu,

Glad all the questions have been clarified and we are on the same page.

Cheers,

Jeff

Date: Thursday, April 6, 2017 at 22:28

Sasha,

I think we are on the same page now. To sum up, Type 1 MSD is *not* always the maximum imposible stack depth of the hardware/chip -- it is the maximum imposible stack depth my MPLS applications wants to advertise to the rest of the world. If there is no such application running on the node (e.g. bare-metal environment), then it could be the hardware/chip limit, in which case an external application (i.e PCE) is expected to provide the entire label stack the node would have to push on to the packets.

This also explains why MSD need to be a configurable attribute -- it needs to be configured based on the application running on the node. This configured value is then advertised in IGPs/PCEP/BGP-LS. In the absence of such a configuration, the node may end up always advertising the hardware/chip limit, resulting in application/service failure.

Regards,

Muthu  

On Thu, Apr 6, 2017 at 9:46 PM, Alexander Vainshtein <[email-masked]> wrote:

Muthu,

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

The draft further says:

   In case, there are

Regards,

Muthu

On Thu, Apr 6, 2017 at 4:07 PM, Alexander Vainshtein <[email-masked]> wrote:

Muthu,

I may be wrong here, but I think that mplsMaxLabelStackDepth in RFC 3813 most probably refers to maximum number of labels an LSR can simultaneously look up in its ILM  and not to the maximum number of labels an LSR can impose. 

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

Hi Sasha,

Muthu,

Two clarifications:

   MAX-ACCESS    read-only

   STATUS        current

   DESCRIPTION

​Regards,

Muthu​

Regards,

​​

Sasha

Cell:      +972-549266302

Hi Saha,

Please see inline..

Muthu hi,

Two points:

Regards,

Muthu

 

Hope this helps.

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

Hi Saha,

Jeff, Muthu and all,

Regards,

Muthu​

Regards,

Sasha

Office: +972-39266302

Cell:      +972-549266302

Hi Muthu,

Hope this helps,

Cheers,

Jeff

Regards,

Muthu

---

From: Alexander Vainshtein <[email-masked]>
Date: Sun, 9 Apr 2017 07:14:39 +0000

Jeff,
You are welcome!

Regards,
Sasha

Office: +972-39266302
Cell:      +972-549266302

From: Jeff Tantsura [mailto:[email-masked]]
Sent: Friday, April 07, 2017 12:21 PM
To: Muthu Arul Mozhi Perumal <[email-masked]>; Alexander Vainshtein <[email-masked]>
Cc: [email-masked]; Shell Nakash <[email-masked]>; Michael Gorokhovsky <[email-masked]>; Ron Sdayoor <[email-masked]>; Rotem Cohen <[email-masked]>
Subject: Re: [spring] Is MSD really a configurable attribute?

Sasha,

Muthu,

Cheers,
Jeff

From: Muthu Arul Mozhi Perumal <[email-masked]<mailto:[email-masked]>>
Date: Thursday, April 6, 2017 at 22:28
To: Alexander Vainshtein <[email-masked]<mailto:[email-masked]>>
Cc: Jeff Tantsura <[email-masked]<mailto:[email-masked]>>, "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>, Shell Nakash <[email-masked]<mailto:[email-masked]>>, Michael Gorokhovsky <[email-masked]<mailto:[email-masked]>>, Ron Sdayoor <[email-masked]<mailto:[email-masked]>>, Rotem Cohen <[email-masked]<mailto:[email-masked]>>
Subject: Re: [spring] Is MSD really a configurable attribute?

Sasha,

Regards,
Muthu

On Thu, Apr 6, 2017 at 9:46 PM, Alexander Vainshtein <[email-masked]<mailto:[email-masked]>> wrote:
Muthu,
The Type 1 MSD that is signaled by the router to the PCE depends on specific of your L3 application and your network behavior
E.g., if you expect your PCE to return a loose ERO comprised mainly (or completely) of a small number of Node SIDs, your IP VPN packets will employ ECMP between each adjacent pair of nodes in this ERO.

Regards,
Sasha

The draft further says:

   In case, there are

Regards,
Muthu

Regards,
Sasha

Hi Sasha,

​Regards,
Muthu​

Regards,
​​
Sasha

Hi Saha,

Please see inline..

Regards,
Muthu

Hope this helps.

Regards,
Sasha

Hi Saha,

Regards,
Muthu​

Regards,
Sasha

Hi Muthu,

Hope this helps,

Cheers,
Jeff

Regards,
Muthu