Subject: [netmod] YANG library structure

From: Martin Bjorklund <[email-masked]>
Date: Fri, 08 Dec 2017 10:47:41 +0100 (CET)

Hi,

There has been quite a lot of discussion about the YANG library
data model on the list.  The authors of draft-ietf-netconf-rfc7895bis
have tried to understand all arguments in the discussion, and provide
a solution.  Below are 3 solution proposals (we have discussed more,
but they are basically just variations on the same themes).

Absolute Requirements
---------------------

o  RFC 7950, Section 5.6.5 says:

     A server MUST NOT implement more than one revision of a module.

o  draft-ietf-netmod-revised-datastores says:

     The conventional configuration datastores [...] share exactly
     the same datastore schema

o  draft-ietf-netmod-revised-datastores says:

     The datastore schema for <operational> MUST be a superset of the
     combined datastore schema used in all configuration datastores
     except that YANG nodes supported in a configuration datastore MAY
     be omitted from <operational> if a server is not able to
     accurately report them.

These requirements (of course) still hold, and we think that the YANG
library document should explain what they imply for the data reported
as part of the library.  (For example, all conventional datastores
MUST have a reference to the same "schema").

Objectives (in no particular order)
-----------------------------------

1. As efficient as possible for a client to consume.

   Since the size of the yang library can be quite large, it should
   be possible for clients to cache the yang library information.

2. A dynamic datastore must be able to implement a module or feature
   that is not implemented in the conventional datastores.

3. It must be possible to NOT implement a module or feature in
   operational, even if it is implemented in some other datastore.

   This is required for transition purposes; a server that wants to
   implement <operational> should not have to implement all modules at
   once.

4. A given module can only be implemented in one revision in all
   datastores.  If a module is implemented in more than one
   datastores, the same revision is implemented in all these
   datastores.

5. Multiple revisions can be used for import, if import-by revision
   is used.

6. Nice to have: make it possible to be used by schema mount

It should be noted that because of 2 and 3 (and 6), the original data
model in RFC 7895 cannot be used.

Use Cases
---------

Here's a set of use cases that must be supported.

  C1. conventional + operational, all have the same schema

  C2. conventional + operational, ietf-hardware is not implemented in
      conventional

  C3. conventional + operational, some modules not yet implemented in
      operational, and some modules are partly implemented in
      operational.

  C4. conventional + operational + ephemeral, ephemeral has its own
      set of modules

Alt. A.
-------

  Each datastore refers to a schema, and each schema contains a flat
  list of all modules, features, etc.

    +--ro yang-library
       +--ro schema* [name]
       |  +--ro name                  string
       |  +--ro checksum              string
       |  +--ro module* [name]
       |  |  +--ro name         yang:yang-identifier
       |  |  +--ro revision?    revision-identifier
       |  |  +--ro namespace    inet:uri
       |  |  +--ro location*    inet:uri
       |  |  +--ro submodule* [name]
       |  |  |  +--ro name        yang:yang-identifier
       |  |  |  +--ro revision?   revision-identifier
       |  |  |  +--ro location*   inet:uri
       |  |  +--ro feature* [name]
       |  |  |  +--ro name    yang:yang-identifier
       |  |  +--ro deviation* [module]
       |  |     +--ro module    -> ../../name
       |  +--ro import-only-module* [name revision]
       |     +--ro name         yang:yang-identifier
       |     +--ro revision     union
       |     +--ro namespace    inet:uri
       |     +--ro location*    inet:uri
       |     +--ro submodule* [name]
       |        +--ro name        yang:yang-identifier
       |        +--ro revision?   revision-identifier
       |        +--ro location*   inet:uri
       +--ro datastore* [name]
       |  +--ro name      identityref
       |  +--ro schema    -> ../../schema/name
       +--ro checksum     string

  How does this solution handle the use cases above?

  C1: One schema, all datastores refer to this schema.

  C2: Two schemas, "conventional" and "operational".  They differ in
      just one element (ietf-hardware).  All other module information
      is entirely duplicated in both.

  C3: Two schemas, "conventional" and "operational".  They differ in
      the modules not implemented in operational, and operational also
      has some deviation modules with "not-implemented".

  C4: Three schemas, "conventional", "ephemeral", "operational".
      "operational" contains the union of all modules in the other
      two.

  Pro: simple on the client, simple on the server

  Con: verbose, since a single difference requires a complete, new,
       schema.

Alt. B.
-------

  Each datastore refers to a schema, and each schema contains a list
  of references to module-sets, and each module-set contains a flat
  list of all modules, features, etc.

  When combining module-sets into a schema there MUST NOT be any
  duplicate module definitions in the module-sets.

    +--ro yang-library
       +--ro module-set* [name]
       |  +--ro name                  string
       |  +--ro checksum              string
       |  +--ro module* [name]
       |  |  +--ro name         yang:yang-identifier
       |  |  +--ro revision?    revision-identifier
       |  |  +--ro namespace    inet:uri
       |  |  +--ro location*    inet:uri
       |  |  +--ro submodule* [name]
       |  |  |  +--ro name        yang:yang-identifier
       |  |  |  +--ro revision?   revision-identifier
       |  |  |  +--ro location*   inet:uri
       |  |  +--ro feature* [name]
       |  |  |  +--ro name    yang:yang-identifier
       |  |  +--ro deviation* [module]
       |  |     +--ro module    -> ../../name
       |  +--ro import-only-module* [name revision]
       |     +--ro name         yang:yang-identifier
       |     +--ro revision     union
       |     +--ro namespace    inet:uri
       |     +--ro location*    inet:uri
       |     +--ro submodule* [name]
       |        +--ro name        yang:yang-identifier
       |        +--ro revision?   revision-identifier
       |        +--ro location*   inet:uri
       +--ro schema* [name]
       |  +--ro name          string
       |  +--ro checksum      string
       |  +--ro module-set*   -> ../../module-set/name
       +--ro datastore* [name]
       |  +--ro name      identityref
       |  +--ro schema    -> ../../schema/name
       +--ro checksum      string

  How does this solution handle the use cases above?

  C1: One module-set, one schema, all datastores refer to this schema,
      the schema refers to the single module-set.

  C2: Two schemas, "conventional" and "operational", and two
      module-sets.  One module-set contains just "ietf-hardware" and
      the other everything else.  The "operational" schema refers to
      both module-sets, and the "conventional" to just the one without
      "ietf-hardware".

  C3: Two schemas, "conventional" and "operational", and three
      module-sets.  One module-set contains all modules fully
      implemented in both conventional and operational, one contains
      the modules implemented only in conventional, and one the
      modules and deviations for the partly implemented modules in
      operational.

  C4: Three schemas, "conventional", "ephemeral", "operational", but
      just two module-sets. "conventional" refers to one of the
      module-sets, and "ephemeral" to the other.  "operational" refers
      to both.

  Pro: less verbose

  Con: the client has to follow extra references and must combine the
       result from the references into a single schema.

Alt. C.
-------

  (This is the draft -02 version with just some name changes)

  Each datastore refers to a schema, and each schema contains a list
  of references to each module it includes.

    +--ro yang-library
       +--ro module* [id]
       |  +--ro id                  string
       |  +--ro name                yang:yang-identifier
       |  +--ro revision?           revision-identifier
       |  +--ro location*           inet:uri
       |  +--ro namespace           inet:uri
       |  +--ro feature*            yang:yang-identifier
       |  +--ro deviation* [module]
       |  |  +--ro module    -> ../../id
       |  +--ro conformance-type    enumeration
       |  +--ro submodule* [name]
       |     +--ro name        yang:yang-identifier
       |     +--ro revision?   revision-identifier
       |     +--ro location*   inet:uri
       +--ro schema* [name]
       |  +--ro name      string
       |  +--ro module*   -> ../../module/id
       +--ro datastore* [name]
       |  +--ro name          identityref
       |  +--ro schema    -> ../../schema/name
       +--ro checksum       string

  How does this solution handle the use cases above?

  C1: One schema, all datastores refer to this schema,
      the schema refers to all modules.

  C2: Two schemas, "conventional" and "operational", and the module
      list contains all modules.  The "operational" schema refers to
      all modules, and "conventional" to all modules except
      "ietf-hardware".

  C3: similar to C2, except there will be two entries in the module
      list for evenry module that is partly implemented in
      operational.

  C4: Three schemas, "conventional", "ephemeral", "operational", and
      the module list contains all modules.
      Each schema refers to the modules it supports.

  Pro: All modules available are listed in one place.

  Con: the client has to follow extra references and must combine the
       result from the references into a single schema.

       the least "direct" solution due to the module "id".

       probably a bit tricky to implement on the server.

/martin

---

From: Lou Berger <[email-masked]>
Date: Fri, 08 Dec 2017 06:10:49 -0500

Martin,

Thanks for raising this, there's definitely some open issues as well as 
confusion on this topic. In the different options listed below you say that 
each datastore refers to a schema, correct? What is the mechanism you would 
use to do this?

In talking to some others on this topic, they suggested using a library per 
datastore. I haven't look into this enough to know if that is a good or bad 
idea, but it seems functionally equivalent to your first option but 
realized in a different way. Just something to add to the mix.

Lou

On December 8, 2017 4:48:15 AM Martin Bjorklund <[email-masked]> wrote:

> _______________________________________________
> netmod mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netmod
>

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 8 Dec 2017 12:15:04 +0100

> each datastore refers to a schema, correct? What is the mechanism you would

This is in the tree diagrams, e.g.

> idea, but it seems functionally equivalent to your first option but realized
> in a different way. Just something to add to the mix.

If people want to put additional options on the table, they should
work out the details and write down a tree diagram and send the result
to the list.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Lou Berger <[email-masked]>
Date: Fri, 8 Dec 2017 07:08:32 -0500

> work out the details and write down a tree diagram and send the result

I guess we have a different philosophical approach here.  I'd prefer to
hear about fresh ideas, even if half baked or asked as a "stupid
question". Sometimes, if not dismissed out of hand, these lead to
something that is a better result than others that have been immersed in
the issue have thought of.

Lou

---

From: Ladislav Lhotka <[email-masked]>
Date: Fri, 08 Dec 2017 13:09:58 +0100

> but they are basically just variations on the same themes).

I believe YANG library should be liberated from the dependence on
servers and protocols and become part of a formal data model
specification. It is not just some state data but rather metadata, and
it is also perfectly conceivable that a server does not publish its YANG
library at all (e.g. for security reasons or constrained devices).

And then, I would suggest to consider dropping the above requirement
for the *general* YANG library, i.e. design it so that it can in
principle support multiple revisions of the same module. This is
important because

- it allows for supporting multiple revisions of some hardware
  (e.g. line cards) in the same device

- the server needn't represent just a single device but may be used for
  configuring a network of devices, and then the above restriction could
  be severely limiting.

This said, it would still be possible for a specific protocol to impose
such a restriction.

...

>   list of all modules, features, etc.

I like this one because different schemas will often need to reuse the
same module sets, and it will also be quite easy to add schema mount
specification to this.

Lada

-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 8 Dec 2017 13:14:34 +0100

> the issue have thought of.

The devil is usally in the details and 'using a library per datastore'
is for my taste a bit too little to understand what is actually being
proposed. I am not dismissing proposals, but I like to be able to
understand what is being proposed. And for that I need a proposal that
is a bit more than 'using a library per datastore'.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Fri, 08 Dec 2017 15:08:31 +0100 (CET)

> is a bit more than 'using a library per datastore'.

I agree.  In this case, I can't understand how it would work at all.
The library is config false, so a client can never get it via
get-config from <running>.  Maybe the proposal is a new operation
<get-yang-library> with the datastore as a parameter?  Or maybe the
idea is to somehow return meta data together with <get-config>?  I can
guess, and dismiss the proposal based on my guesses ;-)  Or someone
can write up a concrete proposal.

/martin

---

From: Vladimir Vassilev <[email-masked]>
Date: Fri, 8 Dec 2017 15:51:42 +0100

> can write up a concrete proposal.
I just posted a followup on a relevant proposal in a thread "[netmod] 
[Netconf] Alternative YANG library structure for 7895bis".

Vladimir

---

From: Kent Watsen <[email-masked]>
Date: Fri, 8 Dec 2017 17:21:45 +0000

CC-ing NETCONF, where the draft is being worked on.

Kent

Vladimir
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=EP64GIX5XS2zO8K1G7A6vXmTlcHV0WBZZw5r5GaBQZU&s=wJW1_1lfeGR1F0ZUiZmfmaKNzE-EpE4DQuLwKYiyRig&e=

_______________________________________________
netmod mailing list
[email-masked]
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=EP64GIX5XS2zO8K1G7A6vXmTlcHV0WBZZw5r5GaBQZU&s=wJW1_1lfeGR1F0ZUiZmfmaKNzE-EpE4DQuLwKYiyRig&e=

---

From: Andy Bierman <[email-masked]>
Date: Sat, 9 Dec 2017 09:43:59 -0800

> but they are basically just variations on the same themes).

Why do all of the alternatives ignore this objective?
All of them are 2X to 3X larger for message encoding than the IETF100
solution.

Why is the schema list needed at all?
Is this for schema-mount?
What does each schema list instance represent in an NMDA server?

The existing RFC 7895 library could be used, and each special feature
(NMDA, licensing, who-knows-what-else) can augment the module list
with additional data.  I guess YANG stability only matters to the people
who already ship code that does it the existing way.  Given that the new
NMDA
version (modules instead of modules-state) is read-only, I fail to see how
deprecating the modules-state subtree solves any real problem.

IMO, all of the new proposals are moving in the wrong direction and force
the client to potentially retrieve lots of conflicting schema information.
An existing client can easily be adapted to work with a single module list
/ schema
tree that now applies to a subset of datastores (instead of all datastores).
If the client now needs to retrieve and resolve potentially overlapping
module lists / schema trees, then deployment may be limited to new
2nd party client implementations only.

Andy