Subject: [netmod] evaluation of "when" under NMDA

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 04 Dec 2017 17:26:51 +0100

Hi,

if we have

augment "/target/node" {
  when "...";
  ...
}

is the "when" expression supposed to be evaluated separately in each datastore,
and the augment applied only in those datastores where the result is true?

RFC 7950 says in sec. 7.21.5 that the context node for XPath evaluation is "the
augment's target node in the data tree", but with NMDA we have multiple data
trees, hence multiple target nodes.

Lada

-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 04 Dec 2017 17:34:31 +0100 (CET)

> and the augment applied only in those datastores where the result is true?

Yes.

> trees, hence multiple target nodes.

We had multiple datastores even before NMDA.  The when expression
could be true in candidate but false in running.

/martin

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 04 Dec 2017 18:05:58 +0100

On Mon, 2017-12-04 at 17:34 +0100, Martin Bjorklund wrote:
> > is the "when" expression supposed to be evaluated separately in each
> 
> datastore,

But then it cannot be guaranteed that the schema for <operational> is a superset
of the schema of configuration datastores - the when expression can evaluate to
false in <operational> but true in <intended>.

Lada

> 
> > RFC 7950 says in sec. 7.21.5 that the context node for XPath evaluation is
> 
> "the

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 4 Dec 2017 18:22:47 +0100

> false in <operational> but true in <intended>.

For me, its still the same schema - a when expression does not change
my notion of 'schema'.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Robert Wilton <[email-masked]>
Date: Mon, 4 Dec 2017 17:29:35 +0000

Hi Lada,

> false in <operational> but true in <intended>.
I think that comes down to terminology, RFC 7950 section 7.21.5 on when 
statements just talks about making data definition statements 
conditional.  I don't really think that when statements (or choice 
statements) change the schema associated with a datastore, they just 
make parts of it inactive based on the current data.  I.e. you wouldn't 
expect different information to be returned via YANG library depending 
on how particular when statements in the implemented modules get evaluated.

Is seems to me that the when processing falls out quite naturally. Is 
there a scenario where you think that this could cause a problem?

Thanks,
Rob

---

From: Andy Bierman <[email-masked]>
Date: Mon, 4 Dec 2017 09:36:43 -0800

> For me, its still the same schema - a when expression does not change

Agreed, but it changes the validation results against the schema.
As Martin pointed out, we already have this separate validation issue in
<candidate>.
This is not a problem though because only <running> is required to pass
validation tests.

I think NMDA works out fine because the YANG validation of config=true nodes
is done in <intended>, not in <operational>.  Only the config=false nodes
would be validated in <operational>, done by the client, not the server.
(Server implementations do not validate their own output).

> /js

Andy

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Andy Bierman <[email-masked]>
Date: Mon, 4 Dec 2017 09:47:37 -0800

> false in <operational> but true in <intended>.

I see your point now.
The server has to evaluate the when-stmts in operational.
Even though the schema trees are the same, the client has a complex task
comparing
<intended> to <operational>.  However it is no different than the existing
complexity comparing <candidate> to <running>.

Andy

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 04 Dec 2017 19:09:51 +0100

> For me, its still the same schema - a when expression does not change

Well, according to draft-ietf-netmod-revised-datastores-07:

   o  datastore schema: The combined set of schema nodes for all modules
      supported by a particular datastore, taking into consideration any
      deviations and enabled features for that datastore.

And "when" determines whether a given schema node is valid or not. So if a
schema node is invalid in the schema of <operational> but valid in the schema of
<intended>, then the former can hardly be a superset of the latter.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Mon, 04 Dec 2017 19:20:15 +0100

On Mon, 2017-12-04 at 09:36 -0800, Andy Bierman wrote:
> 
> 
> On Mon, Dec 4, 2017 at 9:22 AM, Juergen Schoenwaelder <j.schoenwaelder@jacobs-
> university.de> wrote:

This is not true because "when" constraints are among those that have to be
satisfied in *all* data trees (sec. 8.1 in RFC 7950).

I have argued many many times that it is conceptually wrong to make the schema
dependent on an instance data tree that the schema defines. If "when" was just a
special "must", everything would be fine.

Lada

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 4 Dec 2017 21:00:19 +0100

On Mon, Dec 04, 2017 at 07:09:51PM +0100, Ladislav Lhotka wrote:
> And "when" determines whether a given schema node is valid or
> not. So if a schema node is invalid in the schema of <operational>
> but valid in the schema of <intended>, then the former can hardly be
> a superset of the latter.

I do not follow your notion of 'schema node is valid or not'. Where is
this concept defined? I think RFC 7950 talks about validation of data
trees against a "schema". Perhaps there are places where the wording
is not clear enough? I know that there are places where the RFC 7950
text says just "node" without being explicit about the distinction
between schema node and corresponding data nodes. If this is an issue
here, we may consider posting erratas to clear things up.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Mon, 04 Dec 2017 22:09:16 +0100 (CET)

> > For me, its still the same schema - a when expression does not change

Agreed.

>       deviations and enabled features for that datastore.

This text does not write that nodes with false when expressions are
not considered part of the schema.

> <intended>

They are still part of the schema.

Again, it is a matter of terminology - when we say that the "schema"
of <operational> is a superset of the "schema" of the conventional
datastores, we do not exclude false when expressions from the
"schema".  As per the defintion above we include the modules, their
features and deviations in the term "schema".

/martin

>, then the former can hardly be a superset of the latter.
> 
> Lada
> 
> > 
> > /js
> > 
> -- 
> Ladislav Lhotka
> Head, CZ.NIC Labs

---

From: Alex Campbell <[email-masked]>
Date: Mon, 4 Dec 2017 22:10:46 +0000

Hi,

'when' does not affect schema nodes, only data nodes.
The schema nodes is essentially the YANG itself, without reference to any particular instance data.

Given an example YANG fragment:

    list widgets {
        key name;
        leaf name {type string;}
        leaf can-frob {type boolean;}
        container frob-data {
            when "../frob-data = 'true'";
            leaf frob-count {type uint32; mandatory true;}
        }
    }

The schema could be written like this: (not in any real language)

    widgets [list] [keys: name]
    - name [name]
    - can-frob [boolean]
    - frob-data [container] [when ../frob-data = 'true']
        - frob-count [uint32]

The when statement does not make schema nodes "valid" or "invalid". It only indicates the conditions under which the nodes will be present in the data tree.

A data tree for this schema might look like this: (in the same not-real-language)
    widgets
    - name: Widget1
    - can-frob: false
    widgets
    - name: Widget2
    - can-frob: true
    - frob-data:
       - frob-count: 5

Widget1 doesn't have any frob-data because its can-frob is false, while Widget2 does because its can-frob is true.
However that doesn't mean Widget1 and Widget2 have different schemas. They are both instances of the same schema written above.

________________________________________
From: netmod <[email-masked]> on behalf of Ladislav Lhotka <[email-masked]>
Sent: Tuesday, 5 December 2017 7:09 a.m.
To: Juergen Schoenwaelder
Cc: NETMOD WG
Subject: Re: [netmod] evaluation of "when" under NMDA

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 05 Dec 2017 10:03:07 +0100

> this concept defined? I think RFC 7950 talks about validation of data

Guess what, this notion is used in the definition of "when" statement (sec.
7.21.5):

   The "when" statement makes its parent data definition statement
   conditional.  The node defined by the parent data definition
   statement is only valid when the condition specified by the "when"
   statement is satisfied. ...

This text clearly talks about a schema node, because it is what data definition
statements define.

> between schema node and corresponding data nodes. If this is an issue

Note that data node is just a special type of schema node (sec. 3).

> here, we may consider posting erratas to clear things up.

Right, the text above could say "The schema node defined ..." but it can never
say "an instance of a data node" because "when" can also be used for "choice",
"case" and "augment", and then there is no conditional (parent) instance.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 05 Dec 2017 10:45:49 +0100

> there a scenario where you think that this could cause a problem?

IMO it comes down to the distinction between syntactic (schema) and semantic
constraints. An important syntactic constraint for a container are the children
that a container instance (i) must, and (ii) may contain. Whenever "when" comes
into play, this cannot be determined upfront, we first have to see a particular
(complete) data tree and evaluate an XPath expression on it.

It may seem that this is an academic issue and that the borderline between
syntactic and semantic constraints is arbitrary, but it isn't: it is determined
by the need for having a robust and efficient algorithm for syntactic analysis.
Such an algorithm exists for regular and context-free grammars, as well as for
XSD and RELAX NG, but not for YANG.

As for problems caused by "when", it is by far the trickiest part in YANG and
most complicated implementation-wise. 

Lada

---

From: Robert Wilton <[email-masked]>
Date: Tue, 5 Dec 2017 10:04:55 +0000

On 04/12/2017 17:47, Andy Bierman wrote:
>
>
> On Mon, Dec 4, 2017 at 9:05 AM, Ladislav Lhotka <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     > Ladislav Lhotka <[email-masked] <mailto:[email-masked]>> wrote:
>     > > Hi,
>     > >
>     > > if we have
>     > >
>     > > augment "/target/node" {
>     > >   when "...";
>     > >   ...
>     > > }
>     > >

I think that this is probably down to implementation, but I don't think 
that this is necessarily required.  A server is meant to conform to 
'when' statements in <operational> (e.g. if the system is in a normal 
steady state), but they are allowed to be violated, and I'm not 
expecting that a server would evaluate them (except perhaps to discover 
implementation bugs).  Further, if violations of when statements in 
<operational> are detected then I don't think that there is anything 
that the server can reasonable do.

> Even though the schema trees are the same, the client has a complex 
> task comparing
> <intended> to <operational>.

I don't get why this isn't just a simple diff between the <intended> and 
<operational> data trees.  What is causing the complexity?

Thanks,
Rob

> However it is no different than the existing
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/netmod>
>
>
>
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 5 Dec 2017 11:25:54 +0100

> This text clearly talks about a schema node, because it is what data definition

This is not how I understand the text. Perhaps a clearer phrasing
would have been this:

     The "when" statement makes its parent data definition statement
     conditional.  A node in the data tree corresponding to the parent
     data definition statement is only valid when the condition
     specified by the "when" statement is satisfied. ...

For me, the schema is static, the schema does not change with the
content of the data tree. YANG specifies what a valid data tree is, I
do not thnk it defines schemas that mutate according to some notion of
schema validity.

> Note that data node is just a special type of schema node (sec. 3).

Oops, yes I meant 'node in the data tree' or 'data node instance' or
'data tree node' (it seems we have no term for this and data node is
indeed used for a different purpose).

> "case" and "augment", and then there is no conditional (parent) instance.

The first thing is to settle on what the intention of the YANG
specification was and then we can discuss how to fix any wording
ambiguities. It seems we do not even agree on the intention so far.

/js

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 05 Dec 2017 13:04:08 +0100

>      specified by the "when" statement is satisfied. ...

As I wrote, this doesn't work if the parent statement is "augment", "choice" or
"case" - there is then no such node in the data tree.

> ambiguities. It seems we do not even agree on the intention so far.

What really matters is that "when" is a part of the constraints that have to be
satisfied in all data trees (per sec. 8.1). This is why DSDL schemas (RFC 6110)
never worked exactly according to RFC 6020 (let alone 7950): in RELAX NG there
is no possibility to evaluate XPath expressions, this has to be done in
Schematron.

Moving "when" to semantic constraints (required only for a valid tree) would IMO
remove a lot of complexity at very little cost. For NMDA in particular it would
mean that "when" is only enforced in <intended>.

Lada

---

From: Balazs Lengyel <[email-masked]>
Date: Tue, 5 Dec 2017 15:37:35 +0100

On 2017-12-05 11:04, Robert Wilton
wrote:

I see your point now.

The server has to evaluate the when-stmts in
operational.

I think that this is probably down to implementation, but I don't
think that this is necessarily required.  A server is meant to
conform to 'when' statements in <operational> (e.g. if the
system is in a normal steady state), but they are allowed to be
violated, and I'm not expecting that a server would evaluate them
(except perhaps to discover implementation bugs).  Further, if
violations of when statements in <operational> are detected
then I don't think that there is anything that the server can
reasonable do.

BALAZS: I always thought that if a when statement's argument was
true but becomes false, all instance data that is set/written
according to the schema nodes affected by the when statement shall
be removed by the server.  So IMHO the server can and should do
something about a violated when statement.

Actually I would like a list of statements and constraints that MUST
be satisfied in the different data stores. Speaking about syntactic
versus semantic seems fluffy.

--
Balazs Lengyel Ericsson Hungary Ltd.
Senior Specialist
Mobile: +36-70-330-7909 email: [email-masked]

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 05 Dec 2017 15:38:24 +0100 (CET)

Balazs Lengyel <[email-masked]> wrote:
> On 2017-12-05 11:04, Robert Wilton wrote:
> 
> 
> 
> 
> 
>         I see your point now.
>     conform to 'when' statements in <operational> (e.g. if the system
>     is in a normal steady state), but they are allowed to be violated,
>     and I'm not expecting that a server would evaluate them (except
>     perhaps to discover implementation bugs). Further, if violations
>     of when statements in <operational> are detected then I don't
>     think that there is anything that the server can reasonable do.
> 
> 
> 
> BALAZS: I always thought that if a when statement's argument was true
> but becomes false, all instance data that is set/written according to
> the schema nodes affected by the when statement shall be removed by
> the server. So IMHO the server can and should do something about a
> violated when statement.

Yes.

> versus semantic seems fluffy.

See RFC 7950, section 8 (esp. 8.1 and 8.2).

/martin

---

From: Robert Wilton <[email-masked]>
Date: Tue, 5 Dec 2017 15:31:51 +0000

>> the server. So IMHO the server can and should do something about a
But I still don't think that requires that you have to evaluate when 
constraints on <operational>.

E.g. [implementation dependent]:
  - user sends config to <running>
  - server calculates change to <intended> including implicit deletes 
due to now false when statements.
  - server sends calculated config change to daemons.
  - config daemons late update operational with the updated applied 
configuration.  When statements can be transiently violated in 
operational, but should converge to the behavior defined in the data 
model, all being well.

Thanks,
Rob

> .

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 05 Dec 2017 16:52:55 +0100

> > the server. So IMHO the server can and should do something about a

Even in <operational>?

> See RFC 7950, section 8 (esp. 8.1 and 8.2).

Yes, and I think it is fully appropriate to denote constraints that have to be
satisfied in all data trees as "syntactic" (or "schema constraints") and those
that are needed for validity as "semantic". That said, I think it would be
useful to reconsider the allocations of constraints into these two categories.

Lada

---

From: Ladislav Lhotka <[email-masked]>
Date: Tue, 05 Dec 2017 17:06:57 +0100

> features and deviations in the term "schema".

My understanding was that superset means that the schema tree of
<intended> has to be a subtree of <operational> schema tree. The
paragraph in revised-datastores-07 seems to support this interpretation
as it talks about "YANG nodes omitted from <operational>", not about
modules, features and deviations:

   The datastore schema for <operational> MUST be a superset of the
   combined datastore schema used in all configuration datastores except
   that YANG nodes supported in a configuration datastore MAY be omitted
   from <operational> if a server is not able to accurately report them.

Lada