Subject: Re: [netmod] YANG issues in draft-ietf-netmod-routing-cfg-02

From: Andy Bierman <[email-masked]>
Date: Thu, 01 Mar 2012 00:36:42 -0800

On 02/29/2012 11:59 PM, Ladislav Lhotka wrote:
> On Wed, 29 Feb 2012 13:15:05 -0800, Andy Bierman<[email-masked]>  wrote:
>> Hi,
>>

I will let implementors vote with their feet.
If anyone implements this data model, then that shows that it
is usable just fine and my concerns are unfounded.

I don't agree that unique-stmt in YANG allows list or leaf-list.
I think RFC 6020 is vague on the details.  It certainly does not work
if there is more than 1 node in the tuple.  The text does not call out
the special case where the tuple has only 1 schema-node in it.

Andy

>> I was trying to update the modules on netconfcentral.org, and yangdump
>> is complaining about ietf-routing.yang. 1 fatal error and some warnings.
>>
>>      unique "interfaces/interface/name";
>>
>>      Error: list node (interface) within unique stmt 'interfaces/interface/name' for node 'interfaces/interface/name'
>>      [email-masked]:185.7: error(352): invalid unique-stmt node
>>
>>
>> A unique-stmt for container/list/leaf is not allowed.  Only container and leaf are allowed.
>> RFC 6020 may not be clear on this.  Martin should fix it ;-)
>
> I did check the spec before writing that statement, and it doesn't seem to exclude this case.
> The use case is clear: partition the list of logical interfaces into sublists and make sure the sublists are disjoint.
>
>>
>> It can work if there is only 1 node in the 'uniqueness tuple and only 1 list in the path.
>> But except for that special case, list cannot deterministically produce a set of instances
>> that belong to the same tuple, for comparison to another tuple.
>
> I think it is a matter of definition, and IMO a reasonable definition is in RFC 6110, sec. 12.16. Of course, I could change the 'unique' statement into 'must' along these lines but that would be much less readable.
>
>>
>> If I comment out the unique-stmt, then I get these warnings from ipv4 and ipv6 modules:
>>
>>       Warning: sibling object 'address' already defined in module 'ietf-ipv4-unicast-routing' at line 85
>>       ietf-ipv6-unicast-routing.yang:93.5: warning(425): duplicate sibling node name from external augment
>>
>>       Warning: sibling object 'outgoing-interface' already defined in module 'ietf-routing' at line 132
>>       ietf-routing.yang:132.5: warning(425): duplicate sibling node name from external augment
>>
>>       Warning: sibling object 'dest-prefix' already defined in module 'ietf-ipv4-unicast-routing' at line 63
>>       ietf-ipv6-unicast-routing.yang:71.5: warning(425): duplicate sibling node name from external augment
>>
>>       Warning: sibling object 'next-hop' already defined in module 'ietf-ipv4-unicast-routing' at line 68
>>       ietf-ipv6-unicast-routing.yang:76.5: warning(425): duplicate sibling node name from external augment
>>
>>       Warning: sibling object 'outgoing-interface' already defined in module 'ietf-routing' at line 132
>>       ietf-routing.yang:132.5: warning(425): duplicate sibling node name from external augment
>>
>>       Warning: sibling object 'dest-prefix' already defined in module 'ietf-ipv4-unicast-routing' at line 63
>>       ietf-ipv6-unicast-routing.yang:71.5: warning(425): duplicate sibling node name from external augment
>>
>>       Warning: sibling object 'next-hop' already defined in module 'ietf-ipv4-unicast-routing' at line 68
>>       ietf-ipv6-unicast-routing.yang:76.5: warning(425): duplicate sibling node name from external augment
>>
>> These warnings are because each module does an external augment on
>> the same target, using the same 'route-content' grouping.  So all siblings
>> have duplicate local-names in different namespaces.
>
> I don't see it as a problem. We have next-hop for both IPv4 and for IPv6, named v4ur:next-hop and v6ur:next-hop. Moreover, the augments are conditional so that these two can never become siblings - in fact, they cannot be in the same routing table.
>
>>
>> I really think it sets a bad precedent to intentionally cause local-name
>> collisions -- to be as XML namespace dependent as possible.
>
> For me namespaces is one of the few reasons for using XML rather than, e.g., JSON.
>
>> The user-friendly thing to do would be to always add your own sub-container.
>
> With subcontainers you cannot use leafrefs effectively. That's why we have a flat list of interfaces.
>
>> This is how CLI keeps things straight.  E.g.:
>> (note container insertion before 'uses route-content')
>>
>
> Each 'routing-table' already has 'address-family' and 'safi' as its children, so it makes little sense to carry the redundant address family information in each route:
>
> <rt:routing-table>
>    <rt:address-family>ipV6</rt:address-family>
>    <rt:safi>nlri-unicast</rt:safi>
>    <rt:routes>
>      <rt:route>
>        <v6ur:ipv6-unicast>
>          ...
>        </v6ur:ipv6-unicast>
>      </rt:route>
>      ...
>    </rt:routes>
>    ...
> <rt:routing-table>
>
> With 250K routes this can already make a difference.
>
> Lada
>
>>
>>     augment "/rt:routing/rt:router/rt:routing-tables/rt:routing-table/"
>>           + "rt:routes/rt:route" {
>>       when "../../rt:address-family='ipV4' and "
>>          + "../../rt:safi='nlri-unicast'" {
>>         description
>>           "This augment is valid only for IPv4 unicast.";
>>       }
>>       description
>>         "This augment defines the content of IPv4 unicast routes.";
>>
>>       container ipv4-unicast {
>>          uses route-content;
>>       }
>>     }
>>
>>     augment "/rt:routing/rt:router/rt:routing-tables/rt:routing-table/"
>>           + "rt:routes/rt:route" {
>>       when "../../rt:address-family='ipV6' and "
>>          + "../../rt:safi='nlri-unicast'" {
>>         description
>>           "This augment is valid only for IPv6 unicast.";
>>       }
>>       description
>>         "This augment defines the content of IPv6 unicast routes.";
>>       container ipv6-unicast {
>>         uses route-content;
>>       }
>>     }
>> }
>>
>>
>>
>> Andy
>>
>> _______________________________________________
>> netmod mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 1 Mar 2012 09:47:43 +0100

> is usable just fine and my concerns are unfounded.

I am now working on such an implementation. :-)

> if there is more than 1 node in the tuple.  The text does not call out

The Schematron rule mapped from a 'unique' statement does work even for multiple nodes in the tuple and IMO there is no ambiguity.

Lada

--
Ladislav Lhotka, CZ.NIC Labs
PGP Key ID: E74E8C0C

---

From: Andy Bierman <[email-masked]>
Date: Thu, 01 Mar 2012 01:21:45 -0800

> The Schematron rule mapped from a 'unique' statement does work even for multiple nodes in the tuple and IMO there is no ambiguity.

container top {
   unique "a/b c/d";
   list a {
      key random-string;
      leaf random-string { type string; }
      leaf b { type string; }
   }
   list c {
      key random-int;
      leaf random-int { type int32; }
      leaf d { type string; }
   }

How are instances of 'b' matched with instances of 'd' to form a tuple?

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 1 Mar 2012 11:22:08 +0100

> How are instances of 'b' matched with instances of 'd' to form a tuple?

'unique' is not allowed under 'container', so it must be something like

module unique {
  namespace "http://unique";
  prefix uq;

  list top {
    key "clef";
    unique "a/b c/d";
    leaf clef {
      type uint8;
    }
    list a {
      key "random-string";
      leaf random-string {
        type string;
      }
      leaf b {
        type string;
      }
    }
    list c {
      key "random-int";
      leaf random-int {
        type uint32;
      }
      leaf d {
        type string;
      }
    }
  }
}

OK, let's see - yang2dsdl maps the 'unique' statement to the following Schematron rule:

    <sch:rule context="/nc:data/uq:top">
      <sch:report test="preceding-sibling::uq:top[uq:a/uq:b=current()/uq:a/uq:b
                        and uq:c/uq:d=current()/uq:c/uq:d]">
        Violated uniqueness for "uq:a/uq:b uq:c/uq:d"</sch:report>
    </sch:rule>

which means: IF two different entries E1 and E2 of the "top" list contain both a/b and c/d (each possibly in multiple instances), THEN there must not be a/b anywhere in E1 with the same value as any a/b in E2 AND there must not be c/d anywhere in E1 with the same value as any c/d in E2.

In other words, any combination of (a/b c/d) found in E1 must be different from any combination of (a/b c/d) in E2.

IMO the third paragraph in RFC 6020, sec. 7.8.3, says essentially the same thing.

Lada

---

From: <[email-masked]>
Date: Thu, 1 Mar 2012 11:37:37 -0000

I'm in the process of evaluating two commercial YANG editors/validators:
SegueSoft (formerly ChampNMS) MasterYANG
MG-SOFT Visual YANG Designer

I thought I'd try validating ietf-routing.yang to see whether either of
those tools flagged up a similar error. Unfortunately the former didn't
detect that one of the imported files I had created from the RFC still
had some page headers/footers embedded and claimed no errors were found
(I've notified SegueSoft of the issue). 

That just left YANG Designer. On correcting the file in error
ietfrouting.yang validated with no errors or warnings.

Either their code isn't as tight as yangdump or they come down on the
other side of the ambiguity.

Jonathan Hansford

> -----Original Message-----
> From: Andy Bierman [mailto:[email-masked]]
> Sent: 29 February 2012 21:15
> To: NETMOD Working Group
> Subject: [netmod] YANG issues in draft-ietf-netmod-routing-cfg-02
> 
> Hi,
> 
>     Error: list node (interface) within unique stmt
> 'interfaces/interface/name' for node 'interfaces/interface/name'
>     [email-masked]:185.7: error(352): invalid
unique-stmt
> node
> 
> 
> A unique-stmt for container/list/leaf is not allowed.  Only container
and
> leaf are allowed.
> It can work if there is only 1 node in the 'uniqueness tuple and only
1
> list in the path.
> But except for that special case, list cannot deterministically
produce a
> set of instances
> If I comment out the unique-stmt, then I get these warnings from ipv4
and
> ipv6 modules:
> 
>      Warning: sibling object 'address' already defined in module
'ietf-
> ipv4-unicast-routing' at line 85
>      ietf-ipv6-unicast-routing.yang:93.5: warning(425): duplicate
sibling
> node name from external augment
> 
>      ietf-routing.yang:132.5: warning(425): duplicate sibling node
name
> from external augment
>      ietf-ipv6-unicast-routing.yang:71.5: warning(425): duplicate
sibling
> node name from external augment
> 
>      Warning: sibling object 'next-hop' already defined in module
'ietf-
> ipv4-unicast-routing' at line 68
>      ietf-ipv6-unicast-routing.yang:76.5: warning(425): duplicate
sibling
> node name from external augment
> 
>      ietf-routing.yang:132.5: warning(425): duplicate sibling node
name
> from external augment
>      ietf-ipv6-unicast-routing.yang:71.5: warning(425): duplicate
sibling
> node name from external augment
> 
>      Warning: sibling object 'next-hop' already defined in module
'ietf-
> ipv4-unicast-routing' at line 68
>      ietf-ipv6-unicast-routing.yang:76.5: warning(425): duplicate
sibling
> node name from external augment
> 

This email and any files attached are intended for the addressee and may contain information of a confidential nature. If you are not the intended recipient, be aware that this email was sent to you in error and you should not disclose, distribute, print, copy or make other use of this email or its attachments. Such actions, in fact, may be unlawful. In compliance with the various Regulations and Acts, General Dynamics United Kingdom Limited reserves the right to monitor (and examine for viruses) all emails and email attachments, both inbound and outbound. Email communications and their attachments may not be secure or error- or virus-free and the company does not accept liability or responsibility for such matters or the consequences thereof. General Dynamics United Kingdom Limited, Registered Office: 21 Holborn Viaduct, London EC1A 2DY. Registered in England and Wales No: 1911653.

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 1 Mar 2012 13:00:42 +0100

> ietfrouting.yang validated with no errors or warnings.

So does pyang.

> other side of the ambiguity.

In this case, yangdump is hypercorrect and flags errors that don't exist.
Perhaps RFC 6020 needs a minor clarification but IMO nothing like the CLRs that Andy suggests.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Thu, 01 Mar 2012 06:23:27 -0800

> which means: IF two different entries E1 and E2 of the "top" list contain both a/b and c/d (each possibly in multiple instances), THEN there must not be a/b anywhere in E1 with the same value as an…

Tuples contain only leafs, so the unique-stmt compares a set of leaf values
from one list entry against all the other list entries.

You are expanding the instance tuple into a matrix, so each leaf in the tuple
is not a single value but a complex matrix of keys and instances

   Vi = { top[i]/a[*]/b, top[i]/c[*]/d }

E.g.

   V(22) = { {top[22]/a['fred']/b, top[22]/a['wilma']/b} , { top[22]/c[6]/d, top[22]/c[7]/d, top[22]/c[16]/d } }

So instead of comparing a set of 2 leaf values to another set of 2 leaf values (V(1) to V(2), V(1) to V(3), etc.)
you claim the server is supposed compare a set of 2 arrays of leaf values to another
set of 2 arrays of leaf values.  Do the keys need to exactly match (across 'b' for example)?
What happens if they don't? What constitutes an incomplete tuple?

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 01 Mar 2012 06:45:44 -0800

...
>> IMO the third paragraph in RFC 6020, sec. 7.8.3, says essentially the same thing.

   list router {
          key "name";
          ....

So what this unique-stmt says it that for each router[name='foo'],
compare the array of interface names.  But 'name' is also a key:

    Vi = { router[name=i]/interfaces/interface[name=*]/name }

So 1 entry Vi is a set of logical interface names assigned to router 'i', maybe:

    V1 = { 'eth0', 'eth1' }
    V2 = { 'eth1', 'eth2', 'eth3'}

There is no unique error for 'eth1' because the entire set of interface names per router
must match the exact instances of another entry or the 'tuple' is not
considered a match.  Is this what you want?  I can't tell from the descriptions.

If you intend that a logical interface can appear in zero or 1 routers, that is
not what this unique-stmt does.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 1 Mar 2012 16:03:02 +0100

> you claim the server is supposed compare a set of 2 arrays of leaf values to another

Well, instead of 'unique' I can use 'must' with just the Schematron XPath expression. If the server is able to handle any XPath 1.0 expression in 'must', it should also be able to cope with this interpretation of 'unique'. 

> set of 2 arrays of leaf values.  Do the keys need to exactly match (across 'b' for example)?

If we accept XPath rules, then a/b doesn't care about what the key of a is, so the list keys (of a and c) are not relevant to the validation of 'unique', which can be done exactly as Schematron does.

> What happens if they don't? What constitutes an incomplete tuple?

This is answered in 6020: list entries that don't have all of the nodes from the tuple (including defaults) are not taken into account.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 1 Mar 2012 16:16:58 +0100

> considered a match.  Is this what you want?  I can't tell from the descriptions.

OK, so I should improve the description.

> not what this unique-stmt does.

Yes, that's the idea, and the Schematron rule generated from that 'unique' statement does the right thing, as far as I can tell: Two different router entries cannot have the same value of interfaces/interface/name node. 

Lada

---

From: Andy Bierman <[email-masked]>
Date: Thu, 01 Mar 2012 07:49:19 -0800

> Yes, that's the idea, and the Schematron rule generated from that 'unique' statement does the right thing, as far as I can tell: Two different router entries cannot have the same value of interface…

But each interfaces/interface list within the router list can have a different set
of instances. There may not be just 1 interfaces/interface/name node?
What is the node value if there are 2 instances for router[1] and 3 instances
for router[2]. like in my example above?

   list top {
     key idx;
     unique foo/a;
     leaf idx { type int32; }
     list foo {
       key j;
       leaf j { type int32; ]
       leaf a { type int32; }
     }
   }

  tuple = { node-set } == { node-set values }

  Vi = top[idx=i]/foo[j=*]/a

  V1 = {a[1] = 5}  == { 5 }
  V2 = {a[2] = 10, a[7] = 5}  == { 10, 5 }
  V3 = {a[2] = 5, a[7] = 10, a[9] = 20} == { 5, 10, 20 }
  V4 = { } == { }  (only top entry skipped in test)

So is this no unique-error because none of the 'values' of Vi are the same?
I.e., no tuples actually match?  This seems consistent with the
language in RFC 6020, sec 7.8.3.

Or is it a unique-error because some values (node-sets) matches the equality
test according to XPath rules for comparing node-sets?
This is consistent with XPath 1.0 sec 3.4, but it is not intuitive or
explicitly stated in the YANG RFC.

Andy

---

From: Jernej Tuljak <[email-masked]>
Date: Fri, 02 Mar 2012 08:24:59 +0100

I just tested your example with one of our tools which is based on 
Ladislav's dsdl pyang plugin. The unique constraint is not violated.
But I agree with Ladislav, that this is what should happen in such a 
case..after all that is what you are saying you want when you construct 
such a module: I do not want duplicated value entries in my top list 
when all the referenced leaf instances exist. One could claim, that not 
all referenced leaf instances exist for vectors which are not of the 
length of the maximum sized vector in your instance. Different 
combinations of the same values of course being irrelevant (V1 = {5, 10 
, 20}, V2 = {10, 5, 20}: V1 == V2).

Here's the instance created by our tool, just to check if we are on the 
same wavelength:

<data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0"
xmlns:cu="http://www.mg-soft.com/ns/cartesian-unique">
<cu:top>
<cu:idx>1</cu:idx>
<cu:foo>
<cu:j>5</cu:j>
</cu:foo>
</cu:top>
<cu:top>
<cu:idx>2</cu:idx>
<cu:foo>
<cu:j>10</cu:j>
</cu:foo>
<cu:foo>
<cu:j>5</cu:j>
</cu:foo>
</cu:top>
<cu:top>
<cu:idx>3</cu:idx>
<cu:foo>
<cu:j>5</cu:j>
</cu:foo>
<cu:foo>
<cu:j>10</cu:j>
</cu:foo>
<cu:foo>
<cu:j>20</cu:j>
</cu:foo>
</cu:top>
<cu:top>
<cu:idx>4</cu:idx>
</cu:top>
</data>

I agree that this needs clarification in the RFC, but yangdump is 
currently wrong for this case either way. There is no explicit rule in 
RFC6020 which would make such modules illegal YANG.

Jernej Tuljak

Dne 1.3.2012 16:49, piše Andy Bierman:
>>>> V(22) = { {top[22]/a['fred']/b, top[22]/a['wilma']/b} , { 
>>>> top[22]/c[6]/d, top[22]/c[7]/d, top[22]/c[16]/d } }
>>>>
>>>> So instead of comparing a set of 2 leaf values to another set of 2 
>>>> leaf values (V(1) to V(2), V(1) to V(3), etc.)
>>>>> In other words, any combination of (a/b c/d) found in E1 must be 
>>>>> different from any combination of (a/b c/d) in E2.
>>>>>
>> Yes, that's the idea, and the Schematron rule generated from that 
>> 'unique' statement does the right thing, as far as I can tell: Two 
>> different router entries cannot have the same value of 
>> interfaces/interface/name node.
>>
>

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 Mar 2012 01:07:25 -0800

On 03/01/2012 11:24 PM, Jernej Tuljak wrote:
> I just tested your example with one of our tools which is based on Ladislav's dsdl pyang plugin. The unique constraint is not violated.
> But I agree with Ladislav, that this is what should happen in such a case..after all that is what you are saying you want when you construct such a module: I do not want duplicated value entries in
> my top list when all the referenced leaf instances exist. One could claim, that not all referenced leaf instances exist for vectors which are not of the length of the maximum sized vector in your
> instance. Different combinations of the same values of course being irrelevant (V1 = {5, 10 , 20}, V2 = {10, 5, 20}: V1 == V2).
>
> Here's the instance created by our tool, just to check if we are on the same wavelength:
>
> I agree that this needs clarification in the RFC, but yangdump is currently wrong for this case either way. There is no explicit rule in RFC6020 which would make such modules illegal YANG.
>

I disagree with you.  This is not what the co-editors of the YANG RFC intended
with the text in 7.8.3:

7.8.3.  The list's unique Statement

    The "unique" statement is used to put constraints on valid list
    entries.  It takes as an argument a string that contains a space-
    separated list of schema node identifiers, which MUST be given in the
    descendant form (see the rule "descendant-schema-nodeid" in
    Section 12).  Each such schema node identifier MUST refer to a leaf.

    If one of the referenced leafs represents configuration data, then
    all of the referenced leafs MUST represent configuration data.

    The "unique" constraint specifies that the combined values of all the
    leaf instances specified in the argument string, including leafs with
    default values, MUST be unique within all list entry instances in
    which all referenced leafs exist.  The constraint is enforced
    according to the rules in Section 8.

    The unique string syntax is formally defined by the rule "unique-arg"
    in Section 12.

This text does not say anywhere that XPath node-set comparison rules are
used to determine "MUST be unique within all list entry instances".
The text suggests the opposite to me.

The "descendant-schema-nodeid" is a YANG contrsuct, not an XPath construct.
The plural "in which all referenced leafs exist" refers to 1 instance each of
the nodes in the tuple, not N instances of each node in the tuple.

Unique may or may not include order, so { 5, 10 } and { 10, 5 } together
may or may not be an error, I don't know.

But { 5, 10 } and { 5, 10, 15 } are different values, so they are unique
even by your own notion of stretching the rules to change node to node-set
in each tuple.

Andy

---

From: Jernej Tuljak <[email-masked]>
Date: Fri, 02 Mar 2012 12:34:28 +0100

Dne 2.3.2012 10:07, piše Andy Bierman:
>> case..after all that is what you are saying you want when you 
>> construct such a module: I do not want duplicated value entries in
>> my top list when all the referenced leaf instances exist. One could 
>> claim, that not all referenced leaf instances exist for vectors which 
>> are not of the length of the maximum sized vector in your
>> instance. Different combinations of the same values of course being 
>> irrelevant (V1 = {5, 10 , 20}, V2 = {10, 5, 20}: V1 == V2).
>>
>> currently wrong for this case either way. There is no explicit rule 
>> in RFC6020 which would make such modules illegal YANG.
>>
>

Actually RFC6020 says nothing at all about enforcing the unique 
constraint - it oddly enough just says "it must be satisfied" (Section 
8). But since we are talking about XML instance validation, I believe 
RFC6110 must be taken into account. And there semantic constraints 
(which is exactly what a unique is) are enforced through Schematron 
(therefore relying heavily on XPath assertions). For me it is only 
natural to assume that some sort of XPath evaluation is going to take 
place at one point in the future when validating instances with unique 
constraints. But I am now unsure if Ladislav's implementation is correct.

I agree about "in which all referenced leafs exist". A single 
"referenced leaf" in that text is anything that may be referred to with 
a decendant-schema-nodeid (and is also a leaf). Therefore it may 
consequentially refer to an entire array of instance leafs when a list 
nested within another list is in the path to the referred schema leaf. 
Any schema node identifier identifies only a schema node, not an 
instance node. I now do not understand what "combined values of all the 
leaf instances" means. Perhaps that a union of all of the instance array 
values must not contain duplicates? Would this satisfy the requirement 
of the ietf-routing module?

I do not know what the true intention was, but that's what it may be 
interpreted as.

Jernej Tuljak

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 Mar 2012 07:08:53 -0800

On 03/02/2012 03:34 AM, Jernej Tuljak wrote:
> Actually RFC6020 says nothing at all about enforcing the unique constraint - it oddly enough just says "it must be satisfied" (Section 8). But since we are talking about XML instance validation, I
> believe RFC6110 must be taken into account. And there semantic constraints (which is exactly what a unique is) are enforced through Schematron (therefore relying heavily on XPath assertions). For me
> it is only natural to assume that some sort of XPath evaluation is going to take place at one point in the future when validating instances with unique constraints. But I am now unsure if Ladislav's
> implementation is correct.
>

There is no reference to RFC 6110 at all in RFC 6020, so it is hard to see
how that document has any impact on this one at all.  It doesn't.

> I agree about "in which all referenced leafs exist". A single "referenced leaf" in that text is anything that may be referred to with a decendant-schema-nodeid (and is also a leaf). Therefore it may
> consequentially refer to an entire array of instance leafs when a list nested within another list is in the path to the referred schema leaf. Any schema node identifier identifies only a schema node,
> not an instance node. I now do not understand what "combined values of all the leaf instances" means. Perhaps that a union of all of the instance array values must not contain duplicates? Would this
> satisfy the requirement of the ietf-routing module?
>
> I do not know what the true intention was, but that's what it may be interpreted as.
>

When a standard can be interpreted multiple ways, and the WG then decides
that the text does not clearly support any 1 interpretation, then the
standard is deemed to be silent on the issue, but it will probably
get resolved in a future version.  There were several
of these in RFC 4741 that are now clarified in RFC 6241.

I know the WG never discussed this issue to the detail it is now.
There was never once any discussion or consensus that each unique tuple
to compare included multiple instances of a node.  Since the term 'unique'
is not called out to mean "XPath equality test for node-sets", then it does
not mean that.  The English dictionary meaning of 'unique' is the
intention of the WG.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 02 Mar 2012 08:54:43 -0800

Hi,

....

> Yes, that's the idea, and the Schematron rule generated from that 'unique' statement does the right thing, as far as I can tell: Two different router entries cannot have the same value of interface…

I don't think the normative text actually supports this "XPath node-set equality test"
interpretation at all.  If node-sets are allowed instead of nodes (not clear) then the
"combined values of all the leaf instances..." MUST be unique.  That is not the same at
all as "at least 1 child node from each node-set must have the same string() translation".

So unique-stmt clearly does not provide this literal XPath interpretation you want.
But must-stmt does, and it is intended to enforce referential integrity across nodes.
So use must-stmt instead.

Andy

---

From: "t.petch" <[email-masked]>
Date: Sat, 3 Mar 2012 15:54:10 +0100

----- Original Message -----
From: "Andy Bierman" <[email-masked]>
To: "NETMOD Working Group" <[email-masked]>
Sent: Wednesday, February 29, 2012 10:15 PM

Andy

This did come up on the list 26-aug-08 when Martin replied to you

"In our DML we could only express uniqueness for leafs within a list
entry.  But consider this (simplified) case, where an interface has a
list of addresses:
  list interface {
     ...
     list address {
        ...
        leaf ip { type inet:ipv4address; }
     }
  }

I want to say that all values of 'ip' must be unique across all
interfaces:
  list interface {
    ...
    unique "address/ip";
    ...
  }

"

Tom Petch

>      Warning: sibling object 'address' already defined in module
'ietf-ipv4-unicast-routing' at line 85
>      Warning: sibling object 'next-hop' already defined in module
'ietf-ipv4-unicast-routing' at line 68
>      Warning: sibling object 'next-hop' already defined in module
'ietf-ipv4-unicast-routing' at line 68

---

From: Andy Bierman <[email-masked]>
Date: Sat, 03 Mar 2012 08:56:33 -0800

On 03/03/2012 06:54 AM, t.petch wrote:
> From: "Andy Bierman"<[email-masked]>
> To: "NETMOD Working Group"<[email-masked]>

Martin is the one who recently told me that list was not allowed,
and they did not support it in confd.  I'll wait until he gets
back from vacation to weigh in, before I release yuma 2.2-2.
If the WG consensus is to allow a tuple node to be a node-set
instead of a leaf value, then I'll change yangdump and netconfd
to support that.  If so, then the RFC needs some Errata added.

Even though the unique-stmt is not technically XPath,
I agree it might be confusing if XPath comparison rules
were not used here, but the term 'unique' is not clearly defined
in the current RFC text.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sun, 04 Mar 2012 11:15:14 -0800

...
> This did come up on the list 26-aug-08 when Martin replied to you

FYI,

Message posted to netmod WG in Feb. 2009 from Martin about list within unique-stmt...

http://www.ietf.org/mail-archive/web/netmod/current/msg02083.html

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Sun, 04 Mar 2012 21:00:51 +0100

> back from vacation to weigh in, before I release yuma 2.2-2.

Yes, let's wait for Martin's opinion.

> were not used here, but the term 'unique' is not clearly defined

The advantage of having 'unique' defined via XPath (as in 6110) is that it is a formal expression rather than prose, any particular instance document can be tested against it (using Schematron or just XSLT), and that definition

- works as expected for the no-sublist case,
- works for the use case in the routing module: list of sublists, one node in the tuple.
- is well-defined for all other cases.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Sun, 04 Mar 2012 13:18:55 -0800

On 03/04/2012 12:00 PM, Ladislav Lhotka wrote:
> On Sat, 03 Mar 2012 08:56:33 -0800, Andy Bierman<[email-masked]>  wrote:

I think this could work.
For tuple { A, B, C } there is XPath result Ri = { R(Ai), R(Bi), R(Ci) },
that can be compared to Rj = { R(Aj), R(Bj), R(Cj) }.
A tuple is compared only if non-empty node-sets exist for all nodes (A, B and C).

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Mar 2012 08:50:18 +0100

> A tuple is compared only if non-empty node-sets exist for all nodes (A, B and C).

The definition can state that

unique "A1 A2 … An"

is equivalent to

must "not(preceding-sibling::L [ A1 = current()/A1 and
                                 A2 = current()/A2 and
                                 …
                                 An = current()/An ])"

where L is the parent 'list' node.

Lada

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 Mar 2012 14:39:06 +0100 (CET)

Hi,

Ladislav Lhotka <[email-masked]> wrote:
> > >> and leaf
> > > are allowed.

Andy is correct that we don't support sublists in the unique
expression in our NETCONF server.  However, I do agree with what you
write:

> The advantage of having 'unique' defined via XPath (as in 6110) is
> that it is a formal expression rather than prose, any particular
> instance document can be tested against it (using Schematron or just
> XSLT), and that definition
> 
> - works for the use case in the routing module: list of sublists, one
> - node in the tuple.

The problem is that the text in 6020 is unclear.  The text:

  "the combined values of all the leaf instances [...] MUST be unique"

can be interpreted in different ways.

OTOH, the WG has published two documents related to this, 6020 and
6110.  6020 in itself is unclear, but if you also read 6110, the
interpretation is well-defined.

So, we have two options.  Either we say that with the two documents,
the meaning is well-defined, or we say that since 6020 is unclear, we
cannot publish any module that uses sublists in the unique statement.

[In any case, I will try to fix my implementation to work as defined
in 6110.]

BTW, they got this right in XSD.  We should have done something like:

  unique {
    [selector <xpath>;]  // default is "current()"
    field <xpath>;
    ...
    field <xpath>;
    [error-message <str>;]
    [description <str>;]
  }

Then we could also handle multiple keys in sublists:

  list router {
    unique {
      selector "interfaces/interface";
      // assuming type and location were used instead of name
      field "type";
      field "location";
    }
  }

The drawback is that this looks more complicated...    

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Mar 2012 15:58:35 +0100

> cannot publish any module that uses sublists in the unique statement.

I prefer the first option. I am not sure if it is acceptable, but an erratum could be added to 6020 with a reference to the corresponding section in 6110.

>    [selector <xpath>;]  // default is "current()"

So, essentially, 'unique' in YANG is restricted to having "current()" as the selector. If necessary, constraints involving other selectors can be defined using 'must'. BTW, the possibility of specifying an 'error-message' would indeed be useful.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Mar 2012 08:10:45 -0800

On 03/05/2012 05:39 AM, Martin Bjorklund wrote:
...
> cannot publish any module that uses sublists in the unique statement.

I don't think it is a good idea to declare 6020 "fixed" by 6110.
Tools conforming to 6020 should not be made instantly non-compliant
because the spec is silent on an issue.

A future version of YANG can declare this fixed, but not 6020.

>      [selector<xpath>;]  // default is "current()"
>      field<xpath>;
>      ...
>      field<xpath>;
>      [error-message<str>;]
>      [description<str>;]
>    }
>

Why can't I just do:

    unique "interfaces/interface/type interfaces/interface/location";

The nodes in the unique tuple become completely unrelated in your new interpretation
of unique-stmt.  There is no attempt to correlate keys.  Attempts to
make sure type and location are from the same list entry are not possible
with the YANG unique-stmt.

> The drawback is that this looks more complicated...

It allows more arbitrary placement of the unique-stmt, which is more complicated
and may not provide much more real utility.

IMO the new interpretation of unique-stmt can cause more problems
than it solves.  Potential for garbage-in/garbage-out is very high.
Tools will not be able to analyze the intent, so only XPath experts
will get this right.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Mar 2012 08:46:06 -0800

On 03/05/2012 06:58 AM, Ladislav Lhotka wrote:
...
> I prefer the first option. I am not sure if it is acceptable, but an erratum could be added to 6020 with a reference to the corresponding section in 6110.

I don't think 6020 is clear at all about this, and so nested list should not
be allowed for 6020 implementations.

The reason is that this YANG unique-stmt only works (for nested lists)
when there is 1 node in the tuple.  Users who don't know XPath will
get it wrong (not aware of XPath node-set compare rules) and XPath experts
will assume it works like xsd:unique, but it doesn't.

Selecting 2 siblings from the same nested list entry is far different than
selecting 1 field from entries { 1, 3, 5 } and another field from { 3, 10, 20 }.
This is going to produce garbage results.

>>     [selector<xpath>;]  // default is "current()"

In a future version of YANG, unique-stmt can be changed to allow
lists, and add 1 or more 'field-stmt' to pick siblings from the same target.

> So, essentially, 'unique' in YANG is restricted to having "current()" as the selector. If necessary, constraints involving other selectors can be defined using 'must'. BTW, the possibility of speci…

Only for 1 node.  For 2 or more, the behavior is not the same as XPath.
IMO that problem should not be ignored.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Mar 2012 09:21:16 -0800

..
>> The drawback is that this looks more complicated...

list top {
   description
     "demonstrate that unique-stmt with a nested list is broken
      if there are 2+ nodes instead of 1";

   key a;
   // same pair { x, y } can only be used once
   unique "pair/x pair/y";
   leaf a { type int32; }
   list pair {
     key id;
     leaf id { type int32; }
     leaf x { type int32; }
     leaf y { type int32; }
   }

// a=1: pair 1,1
top {
   a 1
   pair {
     id 1
     x 1
     y 1
   }
}
// a=2: 2 incomplete pair entries combine to form point 1,1
top {
   a 2
   pair {
     id 1
     x 1
   }
   pair {
     id 2
     y 1
   }
}

U(1) = { 1, 1 }
U(2) = { 1, 1 }

The intent is that only complete { x, y } pairs be checked in
the unique-stmt test, but that is not what YANG unique-stmt does.
An error is incorrectly generated in this scenario.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 Mar 2012 18:51:59 +0100 (CET)

> with the YANG unique-stmt.

Exactly.  That's why a new statement like the one above is needed.

> Tools will not be able to analyze the intent, so only XPath experts

Actually, I think it is the other way around.  Without it, you need to
write the explicit low-level XPath expression in a must statement,
instead of the high-level unique statement.

/martin

---

From: "t.petch" <[email-masked]>
Date: Mon, 5 Mar 2012 17:57:45 +0100

----- Original Message -----
From: "Ladislav Lhotka" <[email-masked]>
To: "Andy Bierman" <[email-masked]>; "t.petch" <[email-masked]>
Cc: "NETMOD Working Group" <[email-masked]>
Sent: Sunday, March 04, 2012 9:00 PM
> > >> A unique-stmt for container/list/leaf is not allowed.  Only container and
leaf
> The advantage of having 'unique' defined via XPath (as in 6110) is that it is
a formal expression rather than prose, any particular instance document can be
tested against it (using Schematron or just XSLT), and that definition

Lada

And the disadvantage?  Remember all those lovely discussions we had about the
use of XPath in draft-ietf-netmod-yang, in 2008, all that versioning, contexts,
interpretation ....  I am older and wiser but not sure of reaching a conclusion
any quicker than we did then.

It is a lot of work if we can do it more simply in plain English:-)

Tom Petch

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Mar 2012 10:10:18 -0800

..
> instead of the high-level unique statement.

If only the YANG unique-stmt actually worked for node-sets if N > 1.
The must-stmt would be rather complex to make sure the tuple being
compared came from the same nested list entries.  I challenge you
to find a use-case where it doesn't matter if pairs are incorrectly
formed from different list instances.  The XPath Lada generates
is all very nice, but logical-AND of N unrelated node-set comparisons is
not semantically useful.

There is a reason the xsd:unique statement is constrained to
pick N fields from the same context node.  Doing otherwise
produces garbage results.

IMO, it is not really worth it to have some CLRs for unique-stmt,
like lists are OK for 1 node in the unique-stmt only.

Andy

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 05 Mar 2012 19:44:11 +0100 (CET)

Andy Bierman <[email-masked]> wrote:
> formed from different list instances.

No need to challenge; I agree.

> The XPath Lada generates

Agreed.

> produces garbage results.

Agreed.

> like lists are OK for 1 node in the unique-stmt only.

That was also my opinion earlier, when I told you that sublists were
not allowed.  But Lada's solution is very elegant.  It provides an
unambigous interpretation of the unique statement.  Quoting from his
email:

  - works as expected for the no-sublist case,
  - works for the use case in the routing module: list of sublists,
    one node in the tuple.
  - is well-defined for all other cases.

Note the last bullet.  With Lada's interpretation, we wouldn't need to
have this debates of what the meaning of a certain unique expression
is.

/martin

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Mar 2012 11:06:16 -0800

On 03/05/2012 10:44 AM, Martin Bjorklund wrote:
> Andy Bierman<[email-masked]>  wrote:
>> ..
>> If only the YANG unique-stmt actually worked for node-sets if N>  1.

This is a straw-man, because we could just as easily decide that RFC 6020
does not support nested lists and be done with it.

I am concerned that the only use-case for multiple nodes is to
compare them from the same context node, and that is not what YANG does.
A little bullet in an RFC Errata is not going to remove that confusion.
Being 'well-defined' to do the wrong thing is not that compelling to me.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Mar 2012 20:46:47 +0100

> any quicker than we did then.

Yes, it was difficult but in fact eventually we did solve all the XPath issues back then!

> It is a lot of work if we can do it more simply in plain English:-)

Ideally, we can have both (in accord, of course:-). I think it is pretty much the same as a textual description of syntax versus ABNF.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Mar 2012 20:56:59 +0100

> because the spec is silent on an issue.

Thing is, the text in 6020 is unclear and ambiguous, and apparently there are already different tools that behave differently. This should certainly be avoided.

> with the YANG unique-stmt.

I am not an expert on XSD but my understanding is this is exactly what XSD would do if selector = current().

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Mar 2012 12:14:32 -0800

> Thing is, the text in 6020 is unclear and ambiguous, and apparently there are already different tools that behave differently. This should certainly be avoided.

Yes -- in future versions of the RFC you clarify ambiguous text,
like we just did from NETCONF 1.0 to NETCONF 1.1.
You cannot fix missing normative text with an RFC Errata
when there was no clear intent by the WG for 1 implementation
choice or another.

> I am not an expert on XSD but my understanding is this is exactly what XSD would do if selector = current().

See my example for unique 'pair/x pair/y'.
YANG does not work the way you claim.
There is no requirement to pick a nested current() node in each list entry.
The context node is the list node that contains the unique-stmt.
(E.g., I can have "a/b/c x w/x/y/z" and they all share the same list node
as the context node. Some sub-nodes may be in lists, some not.)

There is no requirement for the compiler or server to correlate nested list entries
and apply a nested context node while evaluating the expression.
I think your XPath is wrong.  YANG does not prevent partial pairs from being
combined in a nodeset:

    pair/x:  { 1, missing }
    pair/y:  { missing, 1 }

This collapses to a single pair in XPath evaluation { 1, 1 } which is wrong.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Mar 2012 21:18:14 +0100

> does not support nested lists and be done with it.

How can you decide this without changing 6020? IMO the current text doesn't forbid lists along the path of descendant node ids.

> Being 'well-defined' to do the wrong thing is not that compelling to me.

The case of a sublist and several nodes in the tuple doesn't seem useful but *if* somebody used it, the result is well-defined and can be verified. Forbidding sublists entirely means eliminating an existing use case with one node in the tuple. Do you prefer a formidable 'must' expression in its place?

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Mar 2012 12:44:42 -0800

> How can you decide this without changing 6020? IMO the current text doesn't forbid lists along the path of descendant node ids.

OK -- at best that means that valid tools can implement either way.
Therefore, it is a MAY implement, and standard YANG modules cannot
use language features that are not required by all implementations.

> The case of a sublist and several nodes in the tuple doesn't seem useful but *if* somebody used it, the result is well-defined and can be verified. Forbidding sublists entirely means eliminating an…

It is less than useful and there's nothing in YANG that says list/leaf1 and list/leaf2
come from the same instance of 'list'.  Leaf1 and leaf2 do not even need to be siblings
in xsd:unique. (field a, field b/c/d).

In the future, unique-stmt can be deprecated and a proper unique2-stmt can replace it
in YANG 1.x, like Martin proposed.

Pretending that RFC 6020 supports the xsd:unique 'field' parameter is not possible.
IMO, high-level user-friendly YANG statements are good, but only
if they prevent the user from doing something that does not work.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 5 Mar 2012 22:05:52 +0100

> when there was no clear intent by the WG for 1 implementation

IMO something like this could be accepted as an erratum:

The textual definition of 'unique' turns out to be ambiguous in certain situations. In case of doubt, the standardized mapping of 'unique' to Schematron (RFC 6110, sec. 12.16) should be used to resolve the ambiguity.
 
> YANG does not work the way you claim.

I can only claim how I understood the text in 6020 and how I implemented it.

> This collapses to a single pair in XPath evaluation { 1, 1 } which is wrong.

How can you tell? My point is: if we DEFINE 'unique' as syntactic sugar for a 'must' statement with my XPath expression, all currently existing use cases for 'unique' will work fine and there will be no ambiguity in other cases, too. As far as I can tell, the only outstanding issue is to change some tools to behave accordingly.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Mon, 05 Mar 2012 14:03:56 -0800

> The textual definition of 'unique' turns out to be ambiguous in certain situations. In case of doubt, the standardized mapping of 'unique' to Schematron (RFC 6110, sec. 12.16) should be used to res…

Then the dangers of nested incomplete pairs are completely obscured.
I don't want any CLRs because that just gives tools more work to do.

>>> I am not an expert on XSD but my understanding is this is exactly what XSD would do if selector = current().

Right, no attempt to correlate nested keys to select nested context nodes.

    k_i=current()/k_i so:

    must "not(preceding-sibling::router [ current()/interfaces/interface/type and
                                     current()/interfaces/interface/location ])"

This picks all instances of type and all instances of location,
not all { type, location } pairs like xsd:unique.

...
> How can you tell? My point is: if we DEFINE 'unique' as syntactic sugar for a 'must' statement with my XPath expression, all currently existing use cases for 'unique' will work fine and there will…

If my intent was to only check complete pairs, it is broken.
You are saying users should know that is not what they get,
so too bad if they do that. YANG Doctors will reject their module
when they try to publish an RFC.

OK, OK, I'll change yuma.  It probably already implements unique the way
you define in RFC 6110.

Andy

---

From: <[email-masked]>
Date: Tue, 6 Mar 2012 10:00:50 -0000

Just a quick note from someone who is still getting to grips with YANG.
RFC6020 section 4.1 states that "YANG provides clear and concise
descriptions of the nodes, as well as the interaction between those
nodes."

The (from my perspective, esoteric) discussion about the current unique
statement is neither a clear nor a concise description of the
interaction between nodes. For those who have been immersed in YANG from
its conception, this discussion may have some (intellectual?) merits.
But if there will be few instances where such a use of unique would be
needed, and if a must statement using an XPath can unambiguously if more
verbosely support those instances (which I think is what has been
stated), then it feels to me that an erratum declaring that 6020's
unique statement does not support nested lists would be the sensible way
to go. It may not even cross the mind of the average user that unique
nested lists could exist.

Further, since valid content MUST abide by constraints, there is again a
risk that your average NETCONF/YANG user could generate invalid
configuration data against a unique nested list and not have a clue what
was wrong or how to correct it.

Jonathan Hansford

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 06 Mar 2012 14:08:58 +0100

> descriptions of the nodes, as well as the interaction between those

This is exactly why a concise 'unique' is preferable to an unwieldy 'must'.

> But if there will be few instances where such a use of unique would be

IMO, this assumption is not necessarily true. The use case is that there is a set of objects that have to be unique and these objects are divided into subsets at two or more levels. This could be quite common.

> nested lists could exist.

An average user needn't bother at all, but it has already crossed the mind of at least two YANG module developers.

> was wrong or how to correct it.

Validating tools should give such a clue, and descriptions should help, too.
At any rate, I don't see any difference between 'unique' and 'must' here.

Lada

---

From: Phil Shafer <[email-masked]>
Date: Tue, 6 Mar 2012 08:26:55 -0500

This sounds like a reasonable solution, and goes along with
the "make easy things easy and hard things possible" mantra.

Thanks,
 Phil

[email-masked] writes:
>This email and any files attached are intended for the addressee and may contain informa
>tion of a confidential nature. If you are not the intended recipient, be aware that this
> email was sent to you in error and you should not disclose, distribute, print, copy or 
>make other use of this email or its attachments. Such actions, in fact, may be unlawful.
> In compliance with the various Regulations and Acts, General Dynamics United Kingdom Li
>mited reserves the right to monitor (and examine for viruses) all emails and email attac
>hments, both inbound and outbound. Email communications and their attachments may not be
> secure or error- or virus-free and the company does not accept liability or responsibil
>ity for such matters or the consequences thereof. General Dynamics United Kingdom Limite
>d, Registered Office: 21 Holborn Viaduct, London EC1A 2DY. Registered in England and Wal
>es No: 1911653.

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 6 Mar 2012 15:06:53 +0100

> the "make easy things easy and hard things possible" mantra.

Why is this better than "make easy things easy, moderately difficult things easy and hard things possible"?

Lada

---

From: "t.petch" <[email-masked]>
Date: Tue, 6 Mar 2012 18:13:07 +0100

----- Original Message -----
From: "Ladislav Lhotka" <[email-masked]>
To: "Andy Bierman" <[email-masked]>
Cc: "Martin Bjorklund" <[email-masked]>; <[email-masked]>;
<[email-masked]>
Sent: Monday, March 05, 2012 10:05 PM
On Mar 5, 2012, at 9:14 PM, Andy Bierman wrote:

>> Thing is, the text in 6020 is unclear and ambiguous, and apparently there are
already different tools that behave differently. This should certainly be
avoided.

The textual definition of 'unique' turns out to be ambiguous in certain
situations. In case of doubt, the standardized mapping of 'unique' to Schematron
(RFC 6110, sec. 12.16) should be used to resolve the ambiguity.

<tp>
Whether or not this would be accepted as an erratum, the problem is that so many
people do not know to look for errata, perhaps expecting there to be a later RFC
instead.

A route that is more likely to be noticed would be to publish an RFC that
updates that section; being listed in the rfc-index as 'updates' is far more
likely to be noticed and acted upon.

The challenge then would be to avoid the pork barrel, "while we are at it we
could also just ..... (Please, no)".

Tom Petch

> This collapses to a single pair in XPath evaluation { 1, 1 } which is wrong.

How can you tell? My point is: if we DEFINE 'unique' as syntactic sugar for a
'must' statement with my XPath expression, all currently existing use cases for
'unique' will work fine and there will be no ambiguity in other cases, too. As
far as I can tell, the only outstanding issue is to change some tools to behave
accordingly.

Lada

---

From: Phil Shafer <[email-masked]>
Date: Tue, 6 Mar 2012 13:21:46 -0500

Ladislav Lhotka writes:
>Why is this better than "make easy things easy, moderately difficult =
>things easy and hard things possible"?

The concern is that you're making rare cases easier at the
cost of our "clear and concise descriptions".  This is what
Jonathan was expressing, and I concur.

Thanks,
 Phil

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 06 Mar 2012 21:22:11 +0100

> Jonathan was expressing, and I concur.

I don't get it. The sentence Jonathan cited from 6020 is about clear and concise description of nodes and this is exactly what 'unique' is in comparison to a long and incomprehensible 'must' expression. And the simple case doesn't change at all.

I understand Andy's concern that somebody might try to use 'unique' with sublists and multiple nodes in the tuple, and expect an outcome different from what the XPath expression in 6110 does.
We haven't seen a use case for this yet but even if one is found - I believe a data modeller doing such nontrivial things should have enough expertise to be able to figure out what that XPath expression means because, in any case, his only option will be to write his own (probably even more complicated) 'must' statement.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Tue, 06 Mar 2012 15:20:04 -0800

On 03/06/2012 12:22 PM, Ladislav Lhotka wrote:
> On Tue, 6 Mar 2012 13:21:46 -0500, Phil Shafer<[email-masked]>  wrote:
>> Ladislav Lhotka writes:

I have a new concern -- the error handling -- it is not always possible
to return the <error-info> as required for lists.

13.1.  Error Message for Data That Violates a unique Statement

    If a NETCONF operation would result in configuration data where a
    unique constraint is invalidated, the following error is returned:

      error-tag:      operation-failed
      error-app-tag:  data-not-unique
      error-info:     <non-unique>: Contains an instance identifier that
                      points to a leaf that invalidates the unique
                      constraint. This element is present once for each
                      non-unique leaf.

                      The <non-unique> element is in the YANG
                      namespace ("urn:ietf:params:xml:ns:yang:1").

It is very difficult to dig into the XPath internals and figure
out the exact instance(s) within 2 node-sets that caused the
comparison to be equal to 'true'.  XPath says true on the first
hit, so a parser will not look for more matches, or indicate which
exact node(s) matched in the unique-test.

So the default (pick first node in the node-set) does not work
for lists, like it does for container or leaf.  The <non-unique>
value returned might be wrong.

Andy

---

From: <[email-masked]>
Date: Wed, 7 Mar 2012 10:22:35 -0000

Lada,

I intended to focus more on the "clear and concise descriptions of the
... interaction between those nodes". Though I concur that unique is
concise, I am not convinced it is clear. If it were, we wouldn't have
two incompatible implementations. 

Ambiguity is the enemy of system management and system integration; we
cannot afford to have situations where client and server implementations
are incompatible. Ultimately what is needed is an unambiguous definition
of what unique means for nested lists.

Since I have yet to see a reason why I would want to use unique for
nested lists, I am ambivalent on whether it should be supported. I leave
that decision to those who can see its utility and those who can see its
current weaknesses. 

Jonathan Hansford

> From: Ladislav Lhotka [mailto:[email-masked]]
> Sent: 06 March 2012 20:22
> To: Phil Shafer
> Cc: Jonathan Hansford; [email-masked]
> Subject: Re: [netmod] YANG issues in draft-ietf-netmod-routing-cfg-02
> 
> I don't get it. The sentence Jonathan cited from 6020 is about clear
and
> concise description of nodes and this is exactly what 'unique' is in
> comparison to a long and incomprehensible 'must' expression. And the
> simple case doesn't change at all.
> 
> I understand Andy's concern that somebody might try to use 'unique'
with
> sublists and multiple nodes in the tuple, and expect an outcome
different
> from what the XPath expression in 6110 does.
> We haven't seen a use case for this yet but even if one is found - I
> believe a data modeller doing such nontrivial things should have
enough
> expertise to be able to figure out what that XPath expression means
> because, in any case, his only option will be to write his own
(probably
> even more complicated) 'must' statement.
> 
> Lada
> 
> >
> > Thanks,
> >  Phil
> 
> --

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 07 Mar 2012 19:18:38 +0100

> exact node(s) matched in the unique-test.

I think it is actually quite easy, you first have to pinpoint the two entries of the top level list for which the uniqueness constraint is violated, and then search again just inside these two entries. I tested it using XSLT a it seems to work. (Another interesting XSLT challenge would be to construct the instance identifiers.)

Anyway, it is not necessary to use XPath for this.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Wed, 07 Mar 2012 19:30:02 +0100

> two incompatible implementations.

This was caused by the ambiguity of the existing 6020 text. I am sure we can find an unambiguous formulation for both the case with sublists and without them.
 
> nested lists, I am ambivalent on whether it should be supported. I leave

One use case is in the routing-cfg draft, this is where this long thread started.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Wed, 07 Mar 2012 22:46:11 -0800

On 03/07/2012 10:18 AM, Ladislav Lhotka wrote:
> On Tue, 06 Mar 2012 15:20:04 -0800, Andy Bierman<[email-masked]>  wrote:
>> to return the<error-info>  as required for lists.
>>
>>
>>        error-info:<non-unique>: Contains an instance identifier that
>>                        The<non-unique>  element is in the YANG

I don't and I don't see any value in implementing this error-info element.
Knowing the list entry (with the unique-stmt) that failed is good enough.

I also think this 'clarification' goes way beyond the bounds of any
RFC Errata I've ever seen, and I strongly object to that publication mechanism.
RFC Errata is for things like the missing YANG ABNF comment that says deviation
sub-statements can appear in any order.  That was always the intent of the WG.
That is clearly indicated in other parts of the draft and nobody disagreed
that the WG intent was to allow sub-statements in any order.

In this case, new uses for the unique-stmt are being invented that the NETMOD WG
never once discussed.  I know (cuz I was there) that the authors intended
unique-stmt to mean 1 leaf instance per unique node.  Unfortunately the language
we thought was precise is ambiguous when nested lists are considered -- something
the NETMOD WG never did consider.

Andy

>> for lists, like it does for container or leaf.  The<non-unique>

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 8 Mar 2012 07:58:38 +0100

> Knowing the list entry (with the unique-stmt) that failed is good enough.

I agree. Schematron gives an error message of this sort:

--- Validity error at "/nc:config/t:foo":
    Violated uniqueness for "t:a/t:b t:c"

This is IMO quite enough for locating the problem.

> the NETMOD WG never did consider.

OK, so let's leave it as it is.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Thu, 08 Mar 2012 08:29:40 -0800

> This is IMO quite enough for locating the problem.

The RFC says list all the non-unique leafs but I'm not going to follow it.
Yuma will still support nested lists for yangdump and netconfd validation though.

> OK, so let's leave it as it is.

What does that mean wrt/ ietf-routing.yang?
I don't mind if that stays as-is I guess, because nobody is
very likely to start putting this obscure feature in their data models.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Thu, 8 Mar 2012 17:45:25 +0100

> What does that mean wrt/ ietf-routing.yang?

I don't know, I guess the WG should decide whether sublists will be allowed in the next revision of the YANG standard or not. I will then follow the suit.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Thu, 08 Mar 2012 17:02:53 -0800

...
> I don't know, I guess the WG should decide whether sublists will be allowed in the next revision of the YANG standard or not. I will then follow the suit.

I already find the intentional use of siblings with duplicate local names
questionable.

I also find the duplication of the /interfaces/interface/name hierarchy within
the /routing subtree to be rather questionable.  This could get very
large.

IMO, the mapping is backwards.
If you want an interface to map to 0 or 1 routers,
then just augment the ietf-interfaces module:

augment /if:interfaces/if:interface {
    leaf router-name {
        type leafref {
           path "/rt:routing/rt:route/rt:name";
        }
        description
           "A reference to the name of the logical router for this interface.";
    }
}

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 09 Mar 2012 10:54:03 +0100

> I already find the intentional use of siblings with duplicate local names

I don't know about any such case in the routing-cfg draft, unless you mean lists and leaf-lists.

>            "A reference to the name of the logical router for this interface.";

No, I think each router instance has to have its *logical* interfaces readily available, together with their routing-specific configuration. Looking them up repeatedly in the flat list of interfaces classified only through the messy IANA types could get very expensive.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 9 Mar 2012 10:56:24 +0100

> No, I think each router instance has to have its *logical* interfaces readily available, together with their routing-specific configuration. Looking them up repeatedly in the flat list of interface…

How are routers doing this today?

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 9 Mar 2012 11:20:45 +0100

> How are routers doing this today?

JUNOS:

set interfaces fe-1/1/3 description "main router interface"
set logical-systems LS1 interfaces fe-1/1/3 unit 0 description "LS1 interface"
set logical-systems LS1 interfaces fe-1/1/3 unit 0 family inet address 10.11.2.2/24

NX-OS:

switch(config)# vdc MyVDC
switch(config-vdc)# allocate interface ethernet 2/11-1

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 09 Mar 2012 10:55:27 -0800

On 03/09/2012 01:54 AM, Ladislav Lhotka wrote:
> On Thu, 08 Mar 2012 17:02:53 -0800, Andy Bierman<[email-masked]>  wrote:
>> ...
>>>>

So the router's logical interfaces -- each router/interfaces/interface entry,
are separate and not contained in the ietf-interfaces 'interface' entry?
I don't like that.  If it mirrors the interfaces module then it seems
like duplication to me, and corner-cases when it is out of sync.

If this was just a mapping between a router and an interface, then why
is the container and list needed?  router/interfaces/interface?
Why not just use a leaf-list router/interface-name:

       leaf-list interface-name {
          type if:interface-ref;
           description
               "A reference to the name of a configured logical
                interface.";
           }
       }

Is this to support vendor-specific augments? Why?
They can just augment the /routing/router or /interfaces/interface entry.

IMO, it is much better to keep 1 /interfaces/interface entry and
have other modules augment that, rather than duplicating this
central data structure for each service-specific need.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 9 Mar 2012 21:47:37 +0100

> are separate and not contained in the ietf-interfaces 'interface' entry?

No, see the module, they must all be under "/if:interfaces", and they are linked from under "/rt:router/rt:interfaces" via leafrefs.

> is the container and list needed?  router/interfaces/interface?

It is prepared for augments, you can already see an example in the module ietf-ipv6-unicast-routing, where router/interfaces/interface is augmented with configuration of IPv6 router advertisements.

> Is this to support vendor-specific augments? Why?

All kinds of augment, not only vendor-specific.

> central data structure for each service-specific need.

This is of course subjective but I prefer this design to having /interfaces/interfaces as a kitchen sink of all interface-related configuration and state data. Another YANG-specific aspect is that every extra level of augments makes the data model less comprehensible. And I think major CLIs follow this design, too, see my previous mail.

Lada

---

From: Andy Bierman <[email-masked]>
Date: Fri, 09 Mar 2012 12:58:27 -0800

...
> This is of course subjective but I prefer this design to having /interfaces/interfaces as a kitchen sink of all interface-related configuration and state data. Another YANG-specific aspect is that…

It's not a kitchen sink.
A common template in networking configs is to have a global
component and a per-interface component (or per circuit, per port-profile, etc.)

  /routing

  /interfaces/interface/routing

The issue is whether it makes sense for an instance of the per-interface
routing config to persist even if the interface is deleted.  If so, then
storing it outside the interface is required.

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 9 Mar 2012 16:02:21 -0500

>  /interfaces/interface/routing

This is a major design issue:  Interfaces will have protocol
information and protocols will have interface information.
The choices of where to put which piece can have a huge impact.

For example, in JUNOS, interfaces have information about
address famiies, but no protocols.  So you have:

   /interfaces/interface/unit/family/inet/address/name
   /interfaces/interface/unit/family/inet6/address/name
   /interfaces/interface/unit/family/mpls

which is mostly nice, but then you have:

   /protocols/bgp/group/neighbor/name
   /protocols/ospf/area/interface/name
   /protocols/isis/level/interface/name
   /protocols/pim/interface/name
   /protocols/ldp/interface/name

which is mostly nice.

But the annoying past is that configuring MPLS requires one turn
on the MPLS address family on the interface, which is simple but
the cause of a shocking number of errors.

It's rather like a table of rows and columns.  Some times it's
useful to have the protocols-oriented view and some times it's
more useful to have an interface-oriented view.  And most of
the time it's nice to have things where you use them the most,
even if it is a bit inconsistent.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Sat, 10 Mar 2012 09:01:02 -0800

> even if it is a bit inconsistent.

I like the JUNOS approach.  I wish we started with /protocols/netconf
instead of /netconf and /netconf-state.

This is what we have (some RFC, some I-D):
      /netconf
      /netconf-state
      /ipfix
      /interfaces
      /nacm
      /routing
      /system

/netconf is misleading because it just contains monitoring info about notifications.
(So we could not use /netconf/nacm as one might expect.)

It looks like there is an interface list under each protocol, as needed.
I guess there is complexity either way, so I will drop my objection
to /routing/router/interfaces/interface.

But I hope we don't end up with a bowl of spaghetti bowls ;-)

Andy