Subject: [netmod] augment and if-feature

From: JOEY BOYD <[email-masked]>
Date: Tue, 14 Mar 2017 14:25:15 +0000

Hi all,

An issue arose recently where a certain tool failed to build the schema tree when a feature was turned off. The problem was that an augmenting module did not have the same if-feature statement as the node being augmented. For example, I have these two modules.

module base-module {
  prefix bmod;

  feature do-things;

  container things {
    if-feature do-things;
    ...
  }
}

module augment-module {
  prefix amod;

  augment "/bmod:do-things" {
    container other-things {
    }
  }
}

If I included both modules and turned off the feature, 'do-things', the tool would complain that the node being augmented did not exist. Forgetting the obvious solution of not including the augmenting module if you don't support the feature (this is a very simplified example), my thought was that the schema tree should first be built including all augments, then the feature is applied.

What are your thoughts on this? Surely, an augment should not have to contain if-feature statements of all parents of the augmented node.

Best regards,
Joey

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 14 Mar 2017 18:39:50 +0100 (CET)

Hi,

JOEY BOYD <[email-masked]> wrote:
> Hi all,
> 
> An issue arose recently where a certain tool failed to build the
> schema tree when a feature was turned off. The problem was that an
> augmenting module did not have the same if-feature statement as the
> node being augmented. For example, I have these two modules.
> 
> module base-module {
>   prefix bmod;
> 
>   feature do-things;
> 
>   container things {
>     if-feature do-things;
>     ...
>   }
> }
> 
> module augment-module {
>   prefix amod;
> 
> If I included both modules and turned off the feature, 'do-things',
> the tool would complain that the node being augmented did not
> exist. Forgetting the obvious solution of not including the augmenting
> module if you don't support the feature (this is a very simplified
> example), my thought was that the schema tree should first be built
> including all augments, then the feature is applied.
> 
> What are your thoughts on this? Surely, an augment should not have to
> contain if-feature statements of all parents of the augmented node.

The spec says:

   When a server implements a module containing an "augment" statement,
   that implies that the server's implementation of the augmented module
   contains the additional nodes.

Compare with a simple augment of a node w/o an if-feature.  In this
case, if the server implements the augmenting module, it MUST also
implement the augmented module.

/martin

---

From: JOEY BOYD <[email-masked]>
Date: Tue, 14 Mar 2017 18:21:15 +0000

What about this scenario?

module base-module {
  prefix bmod;

  feature things;
  feature widgets;

  container things {
    if-feature things;
    ...
    container widgets {
      if-feature widgets;
      ...
    }
  }
}    

module augment-module {
  prefix amod;

  augment "/bmod:things" {
    container other-things {
    }
  }

  augment "/bmod:widgets" {
    container other-widgets {
    }
  }
}

At some point, a product wants to implement the base-module and the augment-module but does not support 'widgets'. How should this be handled? I realize that ideally you would have an augment module for things and another for widgets but let's say that is not the case and you are left with the above. The question then is does the augment need to be conditional via the same feature or does it implicitly inherit the feature of the augmented node?

Regards,
Joey

-----Original Message-----
From: Martin Bjorklund [mailto:[email-masked]] 
Sent: Tuesday, March 14, 2017 12:40 PM
To: JOEY BOYD
Cc: [email-masked]
Subject: Re: [netmod] augment and if-feature

Hi,

JOEY BOYD <[email-masked]> wrote:
> the tool would complain that the node being augmented did not exist. 
> Forgetting the obvious solution of not including the augmenting module 
> if you don't support the feature (this is a very simplified example), 
> my thought was that the schema tree should first be built including 
> all augments, then the feature is applied.
> 

The spec says:

Compare with a simple augment of a node w/o an if-feature.  In this case, if the server implements the augmenting module, it MUST also implement the augmented module.

/martin

---

From: JOEY BOYD <[email-masked]>
Date: Tue, 14 Mar 2017 19:03:25 +0000

Correcting the second augment below.

Joey

-----Original Message-----
From: JOEY BOYD 
Sent: Tuesday, March 14, 2017 1:21 PM
To: 'Martin Bjorklund'
Cc: [email-masked]
Subject: RE: [netmod] augment and if-feature

module base-module {
  prefix bmod;

  feature things;
  feature widgets;

  container things {
    if-feature things;
    ...
    container widgets {
      if-feature widgets;
      ...
    }
  }
}    

module augment-module {
  prefix amod;

  augment "/bmod:things" {
    container other-things {
    }
  }

  augment "/bmod:things/bmod:widgets" {
    container other-widgets {
    }
  }
}

Regards,
Joey

-----Original Message-----
From: Martin Bjorklund [mailto:[email-masked]]
Sent: Tuesday, March 14, 2017 12:40 PM
To: JOEY BOYD
Cc: [email-masked]
Subject: Re: [netmod] augment and if-feature

Hi,

The spec says:

/martin

---

From: Phil Shafer <[email-masked]>
Date: Tue, 14 Mar 2017 16:11:09 -0400

>implement the augmented module.

It implements the module, but it doesn't implement the nodes
since it doesn't express the feature.  IMHO this is a tool
bug and/or an errata, since otherwise one has to carry features
forward, repeating the if-feature using the original modules
prefix:feature-name on every augment of feature-based nodes.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 14 Mar 2017 21:50:41 +0100 (CET)

Phil Shafer <[email-masked]> wrote:
> bug and/or an errata,since otherwise one has to carry features

Well, I agree that it would have been better to state that if a server
doesn't implement the augment target, then it doesn't implement the
augment either.  But the text is pretty clear; this is not how it
works.  This is not appropriate to "fix" in an errata.

/martin

---

From: Phil Shafer <[email-masked]>
Date: Tue, 14 Mar 2017 18:23:51 -0400

>works.  This is not appropriate to "fix" in an errata.

I'm missing the part of the text that's clear.  The above quoted
section certainly doesn't say this.  That text is saying "if you
implement a module that augments a set of nodes, then the server's
schema for that original set of nodes now includes the new set of
nodes".  It's referring to schema nodes.

And if those schema nodes are conditional based on if-feature, then
those nodes are still in the schema, but are not supported by a
server unless the if-feature condition evaluates to true.

I don't see a conflict, it's just a case that we didn't think about
or write about.  It's a case that's not clearly handled in the spec,
for which reasonable implementations can disagree.  That's a bug
in the spec and it that can be clarified via errata.

Thanks,
 Phil

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 15 Mar 2017 08:28:14 +0100 (CET)

> nodes".  It's referring to schema nodes.

It explicitly says that server's *implementation* of the augmented
module contains the additional nodes.

If you don't advertise a certain module, I don't think you can claim
that your implementation contains that module.

And similarly, if you don't advertise a feature, I don't think you can
claim that your implementation implements nodes that are conditional
on that feature.

> I don't see a conflict,

> it's just a case that we didn't think about
> or write about.

This I agree with.

> It's a case that's not clearly handled in the spec,

/martin

---

From: Robert Wilton <[email-masked]>
Date: Wed, 15 Mar 2017 10:12:23 +0000

Hi,

> module contains the additional nodes.
Section "5.6.5.  Implementing a Module", doesn't mention features at all.
Section "5.6.2.  Optional Features" doesn't mention implementation at 
all.  It only writes about portions of a model that are conditional, 
supported, or valid.
Section "7.20.1.  The "feature" Statement" also doesn't mention 
implementation, it only writes about portions of the model being 
conditional.

So, I find the text that you are quoting as ambiguous in respect to its 
applicability to features.

> that your implementation contains that module.
I agree with this.

> claim that your implementation implements nodes that are conditional
I'm not convinced that the RFC text supports this view.  The nodes could 
be implemented but conditionally not supported.

>> in the spec and it that can be clarified via errata.
I also think that this needs to be clarified one way or the other.

I would also prefer it to be allowed to augment a node that is 
conditional on an if-feature without having to restate the if-feature 
condition, in exactly the same way that it is allowed to augment a node 
with a when statement without having to restate the when statement in 
the augmentation.

Rob

>>
>> Thanks,
>>   Phil
>>
>
> /martin
>
> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
> .
>

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 15 Mar 2017 11:20:44 +0100 (CET)

Robert Wilton <[email-masked]> wrote:
> So, I find the text that you are quoting as ambiguous in respect to
> its applicability to features.
> 
> 
> 
> >
> I'm not convinced that the RFC text supports this view.  The nodes
> could be implemented but conditionally not supported.

The same can be said about a whole module; it could be implemented but
temporarily disabled.

> condition, in exactly the same way that it is allowed to augment a
> node with a when statement without having to restate the when
> statement in the augmentation.

Just to be clear - I prefer this as well.  I also think the same thing
applies to normal augment w/o features; if you implement module A that
has an augment of B, and you don't implement B, you should
still be able to state that you implement A.

But I don't think it can be done in an errata.  

/martin

---

From: Robert Wilton <[email-masked]>
Date: Wed, 15 Mar 2017 10:50:57 +0000

> But I don't think it can be done in an errata.
Does this just leave the behaviour as undefined then? I.e. it is up to 
the implementation to decide whether they error the augmentation.

Rob

---

From: Martin Bjorklund <[email-masked]>
Date: Wed, 15 Mar 2017 13:05:55 +0100 (CET)

> >> its applicability to features.

I have to agree.  Reading the quoted text again, it is actually not
clear what it tries to say.  I did some digging, and it turns out that
the text was added very late, as a part of the Gen-ART review (see
https://mailarchive.ietf.org/arch/msg/netmod/rmt_T-lMcFm6MPoo8DwrVuZNUxA).

So what it perhaps should have said was:

    When a server implements a module containing an "augment"
    statement, that implies that if the server implements the
    augmented node, its implementation of the augmented node contains
    the additional nodes.

Also, it turns out that this text is in section 4.2.8 which is marked
as non-normative (whole section 4).

So now I am going to argue that the spec allows the case in the
original question :)

There is no normative text that says that an augment of a target node
with an if-feature statement is illegal.  If it is legal it must mean
something.  And the only reasonable interpretation is that if the
server doesn't implement the feature, then it also doesn't implement
the augmented nodes.

(I still think it is unfortunate that the "doesn't implement a
feature" case is handled differently than the "doesn't implement the
module" case)

/martin

---

From: Phil Shafer <[email-masked]>
Date: Wed, 15 Mar 2017 08:21:53 -0400

>that your implementation contains that module.

The issue is that the module is advertised/implemented/supported
but a feature of that module is not.  If an module is also
advertised/implemented/supported, is that module's author
forced to repeat the if-feature conditionals for the first
module?  I think this is a bad idea/design/precedent, and
makes life harder.

>claim that your implementation implements nodes that are conditional

You certainly can.  Consider the "feature local-storage" from
the spec, where this feature indicates availability of local
storage.  The feature may be conditionally advertised, based
on the presence of an SD card.

>  contains the additional nodes.

The "additional nodes" it's talking about are the _new_ nodes from
the augmentation, not the original modules nodes.

I think this is a case we did not consider previously and we should
find a suitable answer and document it as errata, rather than leave
it hanging in the breeze.

Thanks,
 Phil

---

From: Phil Shafer <[email-masked]>
Date: Wed, 15 Mar 2017 09:51:30 -0400

>the implementation to decide whether they error the augmentation.

Which is an unacceptable outcome.  Errata are an acceptable
means of addressing this.  We are not fixing a protocol design
error, but repairing a missing scenario.

   A savvy implementer of the specification can often, but not always,
   figure out what was intended by the RFC as published, but technical
   errors should be announced somehow.

In this case, the spec says nothing about an odd but interesting
scenario.  I don't think this requires a new version of the protocol,
just a clarification.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Wed, 15 Mar 2017 09:59:13 -0700

Hi,

This thread is not surprising because the YANG conformance model is not
that well defined.

As long as the protocols that access hierarchical YANG data require the
parent
to be implemented in order to access the child, it really doesn't matter
how you
want to spin the augment conformance.

In YANG terms "implement the module" means "implement all base module
constructs".
The server can claim conformance to the module and also choose not to
implement
any of the YANG features. Anything with an if-feature is
optional-to-implement.
I don't see how the text you are debating changes that.

Andy

---

From: [email-masked] (Dale R. Worley)
Date: Wed, 15 Mar 2017 15:02:29 -0400

>   augment "/bmod:do-things" {

First question:  I'm not expert in Yang, but as far as I can figure out,
the augment statement is augmenting "container things", right?  So the
augment statement should be 'augment "/bmod:things"' not 'augment
"/bmod:do-things"'.

But on the important question, I don't see it as at all unreasonable
that the augment needs to be qualified by the same if-feature.  The
reason is that if you're reading the text of module augment-module, it's
helpful to have documented, right there, that the augmentation depends
on the presence of a particular feature in the augmented module.  And
it's helpful to know that the designer did, at least for one moment,
think about the fact that the augmentation is conditional.

Dale

---

From: Robert Wilton <[email-masked]>
Date: Thu, 16 Mar 2017 09:54:14 +0000

Hi Dale,

> augment statement should be 'augment "/bmod:things"' not 'augment
Yes, the augment should be to "things".

> think about the fact that the augmentation is conditional.

It isn't just any if-feature on the container that is being augmented 
that needs to be considered.  You would have to consider all if-feature 
statements by walking up the augmented node's ancestors to the top of 
the tree and combine them, or have multiple if-feature statements.

Further, the 7950 YANG update rules allow for the augmented module to be 
revised and some of those if-feature statements to be subsequently 
removed.  If the augmenting module had restated the if-feature 
conditions then this would probably leave the augmenting module 
unintentionally out of sync with the module that it is augmenting.

In short, I think that if-feature statements work better if they act on 
the given node and all descendant nodes, regardless of which module 
those descendants are defined in.

Rob

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Thu, 16 Mar 2017 11:04:23 +0100

On Thu, Mar 16, 2017 at 09:54:14AM +0000, Robert Wilton wrote:

> In short, I think that if-feature statements work better if they act on the
> given node and all descendant nodes, regardless of which module those
> descendants are defined in.

I think there is something important here. The augment does not really
make a difference. What seems to be the root of the issue is whether
an if-feature on say a container or list applies down the hierarchy.
RFC 7950 is not very specific about this. It only says:

   The "if-feature" statement makes its parent statement conditional.

Of course, if a container or list is conditional, then everything
inside will only exist if the feature is supported, so I think it is
reasonable to assume that the if-feature applies down the hierarchy
but this is not explicitly stated (but it seems to be the only
reasonable way to implement this). Anyway, once we have clarified
this, the augment behaviour falls out of this.

That said, as a service to readers, it might still be useful to repeat
the if-feature definition in augmenting modules but this would purely
be a service to the readers, not something that is required by the
language itself.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: [email-masked] (Dale R. Worley)
Date: Thu, 16 Mar 2017 11:05:45 -0400

> the tree and combine them, or have multiple if-feature statements.

Yes, I would expect that.

> unintentionally out of sync with the module that it is augmenting.

It's an interesting sort of out-of-sync, though, as nothing would be
*incorrect*.  With some combinations of features, the augmented node
would have the agumentation and with some, it would not.  But it seems
to me that is quite OK, since whatever depends on the presence of the
augmentation (e.g., client logic or XPath expressions in the
augmentation) only expects the augmentation to be there if the
additional feature is present.

Dale

---

From: Andy Bierman <[email-masked]>
Date: Thu, 16 Mar 2017 09:17:33 -0700

On Thu, Mar 16, 2017 at 2:54 AM, Robert Wilton <[email-masked]> wrote:

> It isn't just any if-feature on the container that is being augmented that
> needs to be considered.  You would have to consider all if-feature
> statements by walking up the augmented node's ancestors to the top of the
> tree and combine them, or have multiple if-feature statements.
>
> removed.  If the augmenting module had restated the if-feature conditions
> then this would probably leave the augmenting module unintentionally out of
> sync with the module that it is augmenting.
>
> the given node and all descendant nodes, regardless of which module those
"work better"

Please explain which protocol you are using that allows you to manage
descendant nodes of unimplemented nodes.

NETCONF and RESTCONF do not work at all wrt/ accessing such nodes.

Andy

---

From: Andy Bierman <[email-masked]>
Date: Thu, 16 Mar 2017 09:39:17 -0700

> additional feature is present.

YANG does not do a great job of separating conformance from conditional
schema nodes.
The if-feature-stmt does not say false means the conformance requirement is
removed.
It says the schema node is removed.  This means a server cannot pick some
nodes
from a YANG feature. The schema tree has all or none (of the nodes marked
with this if-feature).

This distinction may be important, especially if one tries to reconcile the
"always exists" property
of NP-containers with if-feature. If there are no semantics for an
NP-container then how
can it be included or excluded from the conformance model or the
implementation?
Maybe when-stmt and if-feature-stmt on an NP container really just impacts
the descendants,
not the NP-container itself.

Andy

---

From: Robert Wilton <[email-masked]>
Date: Thu, 16 Mar 2017 17:18:01 +0000

Hi Andy,

On 16/03/2017 16:17, Andy Bierman wrote:
>
>
> On Thu, Mar 16, 2017 at 2:54 AM, Robert Wilton <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     Hi Dale,
>
>
>         JOEY BOYD <[email-masked] <mailto:[email-masked]>>
>         writes:
>
>             module base-module {
>                prefix bmod;
>
>                feature do-things;
>
>                container things {
>                  if-feature do-things;
>                  ...
>                }
>             }
>
>             module augment-module {
>                prefix amod;
>
>     It isn't just any if-feature on the container that is being
>     augmented that needs to be considered.  You would have to consider
>     all if-feature statements by walking up the augmented node's
>     ancestors to the top of the tree and combine them, or have
>     multiple if-feature statements.
>
>     Further, the 7950 YANG update rules allow for the augmented module
>     to be revised and some of those if-feature statements to be
>     subsequently removed.  If the augmenting module had restated the
>     if-feature conditions then this would probably leave the
>     augmenting module unintentionally out of sync with the module that
>     it is augmenting.
>
>     In short, I think that if-feature statements work better if they
>     act on the given node and all descendant nodes, regardless of
>     which module those descendants are defined in.
>
>
> "work better"
>
I think that we may be crossing wires:

It was NETCONF/RESTCONF that I was thinking of, and I am not considering 
managing descendant nodes of unimplemented nodes.

I think that the question is only whether an augmentation of a node that 
has an if-feature statement also requires the same if-feature statement 
on the augment statement.

To quote Joey's example,  I think that both of the following modules are 
valid (presuming that they are both implemented by a device) regardless 
of which features are enabled.  Do you agree?

module base-module {
   prefix bmod;

   feature things;
   feature widgets;

   container things {
     if-feature things;
     ...
     container widgets {
       if-feature widgets;
       ...
     }
   }
}

module augment-module {
   prefix amod;

   augment "/bmod:things" {
     container other-things {
     }
   }

Thanks,
Rob

>         [email-masked] <mailto:[email-masked]>
>         <https://www.ietf.org/mailman/listinfo/netmod>
>         .
>
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/netmod>
>
>

---

From: Andy Bierman <[email-masked]>
Date: Thu, 16 Mar 2017 10:26:08 -0700

On Thu, Mar 16, 2017 at 10:18 AM, Robert Wilton <[email-masked]> wrote:

> has an if-feature statement also requires the same if-feature statement on
> the augment statement.
>
>
no -- I don't think there is any requirement.
As pointed out, the if-features can be anywhere in the ancestor nodes.
If any of them are false for a given server, then the augmenting nodes are
inaccessible.

> valid (presuming that they are both implemented by a device) regardless of
> which features are enabled.  Do you agree?
>
>

The modules are valid.
The conformance decision (i.e., which features are advertised and which are
not by a server implementation)
does not change the compile-time validity of the YANG modules

Andy

---

From: Robert Wilton <[email-masked]>
Date: Thu, 16 Mar 2017 17:34:47 +0000

On 16/03/2017 17:26, Andy Bierman wrote:
>
>
> On Thu, Mar 16, 2017 at 10:18 AM, Robert Wilton <[email-masked] 
>>             JOEY BOYD <[email-masked]
>>             <mailto:[email-masked]>> writes:
>>
>>                 module base-module {
>>                    prefix bmod;
>>
>>                    feature do-things;
>>
>>                    container things {
>>                      if-feature do-things;
>>                      ...
>>                    }
>>                 }
>>
>>                 module augment-module {
>>                    prefix amod;
>>
>>         augmented that needs to be considered.  You would have to
>>         consider all if-feature statements by walking up the
>>         augmented node's ancestors to the top of the tree and combine
>>         them, or have multiple if-feature statements.
>>
>>         Further, the 7950 YANG update rules allow for the augmented
>>         module to be revised and some of those if-feature statements
>>         to be subsequently removed.  If the augmenting module had
>>         restated the if-feature conditions then this would probably
>>         leave the augmenting module unintentionally out of sync with
>>         the module that it is augmenting.
>>
>>         In short, I think that if-feature statements work better if
>>         they act on the given node and all descendant nodes,
>>         regardless of which module those descendants are defined in.
>>
>>
>>     "work better"
>>
>>     Please explain which protocol you are using that allows you to
>>     manage descendant nodes of unimplemented nodes.
>>
>     It was NETCONF/RESTCONF that I was thinking of, and I am not
>     considering managing descendant nodes of unimplemented nodes.
>
>     I think that the question is only whether an augmentation of a
>     node that has an if-feature statement also requires the same
>     if-feature statement on the augment statement.
>
>
> If any of them are false for a given server, then the augmenting nodes 
> are inaccessible.
Agreed.

>
>
>     To quote Joey's example,  I think that both of the following
>     modules are valid (presuming that they are both implemented by a
>     device) regardless of which features are enabled.  Do you agree?
>
>
>
> The modules are valid.
> The conformance decision (i.e., which features are advertised and 
> which are not by a server implementation)

Also agreed.

Thanks,
Rob

---

From: [email-masked] (Dale R. Worley)
Date: Fri, 17 Mar 2017 10:03:14 -0400

>      container other-widgets {

Does it remain valid if base-module is changed to:

    module base-module {
       prefix bmod;

       feature things;
       feature widgets;

       container things {
         if-feature things;
         ...
         }
       }
    }

As I analyze it, the augment statement is unconditional, but the
presence of its target node can be (1) unconditional, (2) conditional,
or (3) *never* present.

My preferred approach is that an augment is only valid if it is
"present" only if the target node is present (a condition I think can be
verified statically).  But if we allow the augment to silently have no
effect if the target node is not present in the current implementation,
do we still require that there is some possible implementation in which
the target node exists?

Dale

---

From: Robert Wilton <[email-masked]>
Date: Fri, 17 Mar 2017 14:40:55 +0000

> Does it remain valid if base-module is changed to:
No, I don't think that it is valid because this augment fails because 
the base node "/bmod:things/bmod:widgets" doesn't exist.

> do we still require that there is some possible implementation in which
I guess that I think that a node with an if-feature stmt is logically 
present in the schema (and hence can always be augmented), but just that 
it (and its descendant nodes) can be supported/unsupported depending on 
what features a device advertises that it is supporting.

But I don't think that means that an augmentation is ever is ignored, 
but it might add nodes to the schema tree that aren't supported because 
a given feature isn't supported.

Phil's example of a feature that relates to whether an SD card is 
present is interesting because it gives an example of when it might be 
useful to dynamically modify the set of supported features whilst a 
device is running.  Likewise, I could see how the features could change 
due to adding/removing software components, or different revisions of 
hardware.

Thanks,
Rob