Subject: Re: [nmrg] smi xml document
From: Aiko Pras <[email-masked]>
To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked], [email-masked]
Date: Thu, 29 Jun 2000 16:55:31 +0200

Driemaal scheepsrecht

Juergen Schoenwaelder wrote:
> Aiko> Fixed! The IETF-MIB definitions should now conform to the latest
> Aiko> version. Sorry for the confusion; the developments in this area
> Aiko> go very fast and we were 2 days behind :-)
> 
> Nope. I said CVS version - you probably picked up the latest tar ball.

At this moment you should be able to see all IETF MIBs in the correct
NMRG XML format at http://wwwsnmp.cs.utwente.nl/ietf/mibs/.

As compared to the original DTD we used (1/2 a year back) in Twente to
generate our own XML MIB files , I like the current NMRG DTD better. The
reason for this is that our original DTD resembles the SMIv2, whereas
the NMRG DTD resembles SMIng. The latter one is much clearer, and
provides (in my opinion) a better base for this kind of work.

Bye

Aiko

From: Frank Strauss <[email-masked]>
To: [email-masked] (Aiko Pras)
Cc: [email-masked], [email-masked]
Subject: Re: [nmrg] smi xml document
Date: 28 Jun 2000 18:06:54 +0200
Sender: [email-masked]
Status: O
Content-Length: 216
Lines: 8

Hi Aiko!

Juergen> [Frank is not in town right now and hence there are no new
Juergen> tar balls.]

Aiko> We'll wait till he's back and install the new version a.s.a.p.

I've just put libsmi-0.2.4 on our FTP server.

Date: Mon, 26 Jun 2000 15:40:14 +0200
From: Aiko Pras <[email-masked]>
To: Juergen Schoenwaelder <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: [nmrg] smi xml document
Sender: [email-masked]
Status: O
Content-Length: 1115
Lines: 31

Hi

> Aiko> Fixed! The IETF-MIB definitions should now conform to the 
> Aiko> latest version. Sorry for the confusion; the developments in 
> Aiko> this area go very fast and we were 2 days behind :-)
> 
Juergen> Nope. I said CVS version - you probably picked up the 
Juergen> latest tar ball.
Sorry, you're right. 

Juergen> [Frank is not in town right now and hence there are no new
Juergen> tar balls.]
We'll wait till he's back and install the new version a.s.a.p.

Juergen> Anyway, lets assume I would be using Internet Explorer
hmmm, interesting assumption ;-)

Juergen> Do you provide XSL transformations for the XML files? 
Juergen> Is it possible to look at them
Juergen> (the transformation rules)?
We do not yet have an XSL file. We'll get a student next week and, 
depending on his background, he may start working on this. If he
can not do it, I probably have a good excuse to play with this myself.

Anyway, the Internet Explorer 5.0 presents the XML MIB file with
different indents for the different nestings of XML tags. You can also
click on each level to let it disappear. Quite nice!

Bye

Aiko

Date: Mon, 26 Jun 2000 15:12:00 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked], [email-masked]
Subject: Re: [nmrg] smi xml document
Sender: [email-masked]
Status: O
Content-Length: 537
Lines: 18

>>>>> Aiko Pras writes:

Aiko> Fixed! The IETF-MIB definitions should now conform to the latest
Aiko> version. Sorry for the confusion; the developments in this area
Aiko> go very fast and we were 2 days behind :-)

Nope. I said CVS version - you probably picked up the latest tar ball.

[Frank is not in town right now and hence there are no new tar balls.]

Anyway, lets assume I would be using Internet Explorer. Do you provide
XSL transformations for the XML files? Is it possible to look at them
(the transformation rules)?

/js

Date: Mon, 26 Jun 2000 15:01:09 +0200
From: Aiko Pras <[email-masked]>
To: Juergen Schoenwaelder <[email-masked]>
CC: [email-masked], [email-masked]
Subject: Re: [nmrg] smi xml document
Sender: [email-masked]
Status: O
Content-Length: 619
Lines: 19

Hi all

> Aiko> For those who want to experiment with this: the XML version of
> Aiko> all IETF MIBs can now be downloaded from:
> Aiko> http://www.simpleweb.org/ietf/mibs/
> 
Juergen> Note that these MIBs do not conform to the DTD in the Internet
Juergen> Draft. You need to update your smidump to the latest version in
the
Juergen> anonymous CVS repository in order to get the latest and
Juergen> greatest XML format out of smidump.

Fixed! The IETF-MIB definitions should now conform to the latest
version. Sorry for the confusion; the developments in this area go very
fast and we were 2 days behind :-)

Bye

Aiko

Date: Fri, 23 Jun 2000 16:46:08 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked], [email-masked], [email-masked]
Subject: Re: [nmrg] smi xml document
Sender: [email-masked]
Status: O
Content-Length: 696
Lines: 21

>>>>> Aiko Pras writes:

Aiko> For those who want to experiment with this: the XML version of
Aiko> all IETF MIBs can now be downloaded from:
Aiko> http://www.simpleweb.org/ietf/mibs/

Note that these MIBs do not conform to the DTD in the Internet
Draft. You need to update your smidump to the latest version in the
anonymous CVS repository in order to get the latest and greatest XML
format out of smidump.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

Date: Fri, 23 Jun 2000 15:59:10 +0200
From: Aiko Pras <[email-masked]>
To: Network Management Research Group <[email-masked]>
CC: Bert Helthuis <[email-masked]>, Aiko Pras <[email-masked]>
Subject: Re: [nmrg] smi xml document
Sender: [email-masked]
Status: O
Content-Length: 1572
Lines: 41

Hi all

For those who want to experiment with this: the XML version of all 
IETF MIBs can now be downloaded from: 
http://www.simpleweb.org/ietf/mibs/

Don't forget that you need a browser that supports XML. Things should 
work fine with for example Internet Explorer V5.

Bye

Aiko

Juergen Schoenwaelder wrote:
> 
> Frank and I have written a document (basically the DTD) on how to
> represent SMI MIB module definitions in XML. This DTD basically
> documents the XML output generated by our smidump utility in a format
> DTD. There is much more work needed on the document to define things
> that are hard to describe in an XML DTD. For example, there is no way
> to describe how e.g. a date or a default value looks like in a DTD.
> 
> I believe that the grammar is a good starting point and since I know
> that people are using XML representations of SMI MIB modules, we may
> benefit from finding agreement on a single DTD.
> 
> Any comments on the DTD and the approach taken are highly welcomed. I
> also think that folks in Twente had an XML processor to generate HTML
> pages for SMI modules. Would be nice if you can tell us whether this
> DTD works just fine (with a few minor modifications to your processor)
> or whether there are serious problems with it.
> 
> /js
> 
> --
> Juergen Schoenwaelder      Technical University Braunschweig
> <[email-masked]>  Dept. Operating Systems & Computer Networks
> Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
> Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

Date: Fri, 23 Jun 2000 08:52:35 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked], [email-masked]
Subject: Re: [nmrg] smi xml document
Sender: [email-masked]
Status: O
Content-Length: 1019
Lines: 35

>>>>> J P Martin-Flatin writes:

JP> I suggest you mention this work to George Pavlou. In February,
JP> George and I discussed about SNMP-MIB-to-XML and SMIv2-to-XML
JP> mappings (what the DMTF calls schema-level and metaschema-level
JP> mappings). One of his students was supposed to work on this during
JP> his M.S. thesis. This work might be completed by now.

I will drop him a note.

JP> By the way, I'm glad to see that you're getting interested in
JP> Web-based management.  :-)

I am seriously still waiting for someone to define what "Web-based
management" is all about. Some definitions people seem to be using:

- "everything with Java"

- "everything which has a WEB UI"

- "everything which uses any W3C recommendations"

- "everything that came out of the WBEM M$ initiative"

- "everything that has to do with CORBA"

- "everything that has to do with the Internet and is not SNMP" 

- "..."

I am already interested in your editorial for the JNSM issue since I
expect to find an answer there. ;-)

/js

From: "J.P. Martin-Flatin" <[email-masked]>
To: [email-masked]
Cc: "J.P. Martin-Flatin" <[email-masked]>
Reply-To: "J.P. Martin-Flatin" <[email-masked]>
Subject: Re: [nmrg] smi xml document 
Date: Thu, 22 Jun 2000 23:14:22 +0200
Sender: [email-masked]
Status: O
Content-Length: 599
Lines: 18

On Thu, 22 Jun 2000 15:57:27 +0200, Juergen Schoenwaelder wrote:
> 
> Frank and I have written a document (basically the DTD) on how to
> represent SMI MIB module definitions in XML.

Juergen,

I suggest you mention this work to George Pavlou. In February, George
and I discussed about SNMP-MIB-to-XML and SMIv2-to-XML mappings (what
the DMTF calls schema-level and metaschema-level mappings). One of his
students was supposed to work on this during his M.S. thesis. This work
might be completed by now.

By the way, I'm glad to see that you're getting interested in Web-based
management.  :-)

JP

Date: Thu, 22 Jun 2000 15:58:32 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: Network Management Research Group <[email-masked]>
Subject: [nmrg] smi xml document (addendum)
Sender: [email-masked]
Status: O
Content-Length: 576
Lines: 18

I forgot to give the pointer to the document:

:         Title           : Using XML to Exchange SMI Definitions
:         Author(s)       : J. Schoenwaelder, F. Strauss
:         Filename        : draft-irtf-nmrg-smi-xml-00.txt
:         Pages           : 12
:         Date            : 21-Jun-00
: 
: This memo describes how the Extensible Markup Language (XML) can be
: used to exchange SMIv1, SMIv2 and SMIng definitions between XML
: enabled applications.
: 
: A URL for this Internet-Draft is:
: http://www.ietf.org/internet-drafts/draft-irtf-nmrg-smi-xml-00.txt

/js

Date: Thu, 22 Jun 2000 15:57:27 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: Network Management Research Group <[email-masked]>
Subject: [nmrg] smi xml document
Sender: [email-masked]
Status: O
Content-Length: 1210
Lines: 27

Frank and I have written a document (basically the DTD) on how to
represent SMI MIB module definitions in XML. This DTD basically
documents the XML output generated by our smidump utility in a format
DTD. There is much more work needed on the document to define things
that are hard to describe in an XML DTD. For example, there is no way
to describe how e.g. a date or a default value looks like in a DTD.

I believe that the grammar is a good starting point and since I know
that people are using XML representations of SMI MIB modules, we may
benefit from finding agreement on a single DTD.

Any comments on the DTD and the approach taken are highly welcomed. I
also think that folks in Twente had an XML processor to generate HTML
pages for SMI modules. Would be nice if you can tell us whether this
DTD works just fine (with a few minor modifications to your processor)
or whether there are serious problems with it.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

Date: Mon, 5 Jun 2000 16:53:29 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked], [email-masked], [email-masked]
Subject: Re: [nmrg] SNMP over TCP transport mapping
Sender: [email-masked]
Status: RO

>>>>> Ayers, Mike writes:

Mike> 	I'm wondering if it would be advisable to specify notification
Mike> originator behavior.  Apart from recommending that notification
Mike> receivers listen on port 162, there is no discussion of it.  I'm
Mike> imagining an application that scans its snmpTargetAddrTable at
Mike> startup time and opens TCP connections at that time.  If the
Mike> connections fails or breaks, no attempt is made to reestablish.
Mike> The notification receiver (or another application) must create a
Mike> new snmpTargetAddrEntry (or inactivate/activate its current
Mike> entry) in order to reestablish the connection.  Such an
Mike> implementation would currently violate nothing.  This is a
Mike> deliberately extreme example for illustration of the possibility
Mike> of a conformant application which would not interoperate (or at
Mike> least interoperate poorly).

Mike> 	I suggest a clause that requires notification originator
Mike> applications which support TCP must attempt to connect to any
Mike> targets for which a notification is applicable if a connection
Mike> is not already open to that target.

The question is whether this is really a transport mappings issue or
whether this is a shortcoming in RFC 2573. In fact, RFC 2573 is
generally silent about what happens if an implementation fails to send
a message to a target. Note that this is very well possible with SNMP
over UDP.

I actually think that this should be clarified when RFC 2573 is
revised. I don't think it is the job of the SNMP over TCP document to
fix this.

[The funny think is that RFC 1906 defines snmpCONSDomain any nobody
 ever bothered about connection management and so on. This is Draft
 standard, so there must be interoperable implementations. I am really
 asking myself how this was ever possible to achieve given the many
 problems people see with the SNMP over TCP transport mappings. ;-]

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

Date: Mon, 5 Jun 2000 16:30:45 +0200
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked], [email-masked]
Subject: Re: [nmrg] SNMP over TCP transport mapping
Sender: [email-masked]
Status: RO

>>>>> Lauren Heintz writes:

Lauren> Using TCP and keepalives, I believe it is possible for a
Lauren> notification receiver (NR) to implement a mechanism to be sure
Lauren> that it "knows" about all traps, because in the event the
Lauren> connection goes down, the NR can then sync back up with the
Lauren> notification-generator (NG), perhaps by retrieving all or a
Lauren> portion of an outstanding alarm table.

[...]

Lauren> So, I believe with TCP, keepalives and
Lauren> alarm tables, you can indeed get around appl-level acks and
Lauren> reduce network mgmt traffic.  If that's not true, how so?

The point here is that the document in question defines the SNMP
transport mappings over TCP. And this transport mapping alone is not
sufficient to "reliably" deliver traps since you also need some sort
of an alarm table, as you correctly pointed out above.

Lauren> The doc gave a technical reason why SNMP requests/responses
Lauren> over TCP may be desirable, but it offered no technical reason
Lauren> why traps over TCP was being specified.

This is a _generic_ SNMP transport mapping over TCP. The mapping does
not really care about the protocol operation which sits in your SNMP
message. And it should not.

The only reason why we talk about informs and traps is that many
people just confuse a so called reliable transport with a confirmed
operation. It is not our intention to explain all possible and
legitimate usage scenarios in the transport mapping documents. Of
course, you can argue that the reliable transport vs. confirmed
operation explanation should also not be in this document. Perhaps
this is what you are asking for.

I really think there should be a BCP document on these issues since
these issues come up again and again and a BCP document is IMHO the
right place to outline the various usage scenarios that work. Any
volunteers to write things down?

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

To: [email-masked]
Cc: Wes Hardaker <[email-masked]>, Juergen Schoenwaelder <[email-masked]>, [email-masked], [email-masked]
Subject: Re: [nmrg] SNMP over TCP transport mapping
From: Wes Hardaker <[email-masked]>
Organization: U.C.Davis, Information Technology - D.C.A.S.
Date: 31 May 2000 17:23:23 -0700
Lines: 188
Sender: [email-masked]
Status: RO

>>>>> On Wed, 31 May 2000 09:50:45 -0700, Lauren Heintz <[email-masked]> said:

>> Anyway, consider the following problems with the Notification
>> Responder (NR) opening the TCP connection:
>> 
>> 1) It is the responsible for making sure the connection is up and
>> since it is not the one responsible for sending the data, this
>> like a difficult chore.  How often should it attempt to make
>> connections to the NG when the last attempt failed or was closed?
>> Certainly when the remote NG comes up and tries to send a linkUp
>> trap it'll be forced to use UDP since the NR couldn't have opened a
>> connection to it yet.

Lauren> Keeping a connection open is not more problematic for the NR just
Lauren> because it is not the one sending the traps.  Nevertheless, if the NR
Lauren> was opening connections, then the inability to do so at a given moment
Lauren> raises attention to a potential network problem that requires manual
Lauren> intervention.  This is a feature.  If the NG were istead the TCP client,
Lauren> the problem might go unnoticed for a much longer time, and depending on
Lauren> just how *smart* the connection mgmt module in the NG was, the problem
Lauren> might not be noticed for a VERY long time.

Well, if you relied solely on traps to decide if your network was up,
then yes certainly the only way you'd find out about a problem is if
the NR failed to open a TCP connection to the thing that was supposed
to be generating traps.

I work for a large university (something like 40000+ people including
staff faculty and students).  We have 2 class B networks because we
can't even fit within 1 cleanly.  Our NOC has used numerous network
management systems to do network topology mapping and to create
network status graphs.  It is a nightmare of red and green symbols
interconnected by lines.  If I wanted to receive traps from all of
them then under your setup there is no conceivable way I could easily
configure my NR with the appropriate list of devices that it should
attempt to open a TCP connection to.  The network isn't identical from 
day to day, let alone hour to hour.  Much of the time I don't care if
a machine X is up or down since its frequently an end host (a router
is a different story), but I certainly would care if it wanted to send 
me an authentication failure trap.  Now, granted, all those hosts
could simply send a UDP trap, but that defeats the purpose of using
TCP if I wanted to (I have toyed with the idea of having a *lot* of
host-based intrusion detection boxes notifying a central NR with gobs
of data, in which case I'd really prefer TCP over UDP).

>From simply the standpoint of configuration, I see configuring a NR
with a list of hosts to open a TCP connection to for notification
reception to be a nightmare and impossible to use.  I *only* want to
configure the clients to send the traps, and have the NR receive them
as long as they're deemed authentic.

It must be the end of the day, as I'm rambling...  Sorry...

>> 2) The sheer volume of connections required in a large network is huge
>> problem.

Lauren> If a given NR is managing X number of devices (each device has
Lauren> one or more NGs on it), then X connections are needed no
Lauren> matter who the TCP server is.

I disagree.  I'd rather have the NGs only open the connection if and
only if they had something to send to me.  In that case, the maximum
number of connections might by Y where Y is the number of boxes that
have sent me data out of a possible X.  Further more, if I wanted to
keep the TCP footprint a bit smaller I could limit the number of open
connections on the NR side to a fixed value M such that if more than M 
connections were opened on the NR side it would either:

1) stop accepting new notification streams over TCP (note that M can
   be much less than Y).

2) or what I'd rather do is have the oldest-used TCP connection
   dropped on the NR side and if the NG client ever has to send more
   data it can try to open the connection again, thus limiting the
   number of connections to M/Y.  (appropriate
   minimum-time-limit/thrashing detection would have to be implemented
   as well if M+1 machines were continuously sending data).

Lauren> If the NG were the TCP client, it has to assume a TCP TDomain
Lauren> target entry implies an open connection is mandatory at all
Lauren> times

Why wouldn't the NG merely check to see if a connection existed and
either use it or open a new one?

If a middle network link went down, for instance, either the NG or the 
NR will have to re-establish the connection anyway so dealing with the 
need of sending a notification when the TCP session may not be open
yet will need to be dealt with.  Which ever side is the TCP connection 
originator would have to try every X minutes to open a connection.
I'm arguing that I'd rather not even have the client try to open a
connection until something needed to be sent.

Lauren> Here's my example:  Lets say you have a network with 1000 SNMP devices,
Lauren> each hosting 1 NG, and you have three different NRs in different rooms
Lauren> -- and you're the only network admin person on staff.  If the NG were
Lauren> the TCP client, then you have to keep all three NRs up and
Lauren> running all the time and you would have 3000 connections at
Lauren> all times -- even though you (understaffed, overworked and
Lauren> underpaid) only use one of the NRs at a time.  You'd also have
Lauren> to maintain three target/notify configurations on each of the
Lauren> 3000 devices.

Not at all.  You seem to believe that if the NGs are all sending traps
to all 3 of them that the NGs would fail for all of them if it
couldn't contact a single one?  I fail to see that.

I'd (poor sap) already going to have to configure the 3000 boxes to
set the NR destinations regardless of who was opening the connection,
right?  What I *don't* want to have to do is mirror that configuration 
on the NR side of things in addition to the NG side of things.  Say I
add a new box (#3001).  Under my model, I configure just it to send
stuff to the 3 NRs.  Under your model, I'm required to configure both
it and the 3 NRs making the number of configuration items 4 times as many?

Lauren> Also, I *think* (help me out here) you can get away with using
Lauren> only one set of target/notify configurations on each of the
Lauren> 1000 devices (using subnet mask).

I don't believe so.

>> 3) Many applications are short lived and may wish to send a trap or
>> two while it is up and then shutdown again.  Certainly the NR
>> couldn't predict the window when the application might be up and
>> open a connection to it.

Lauren> The NR would probably use UDP in this situation.  Or if the
Lauren> event was crucial (i.e. reactor core meltdown alarm sent as a
Lauren> trap), a long-lived TCP connection would not be deemed as
Lauren> expensive.  Besides, are you suggesting that TCP connections
Lauren> are established just before a trap is sent and then brought
Lauren> down as soon as it is sent?  Hope not.

Yes I'm suggesting they be brought up when something needs to be sent,
but not necessarily taken down (though if the remote app went down,
then certainly the connection would have to go down as well, but I
don't see that as a bad thing if it came up, fired off 1000
100-varbind notifications (god forbid), and then went down again).

>> 4) Multiple applications on the same host might need to send
>> notifications, and hence multiple ports would have to be opened
>> from the NR side.

Lauren> A given NR would not open a TCP connection to each NG
Lauren> application on a given SNMP device, rather, it would open only
Lauren> one to a well-known port (162?), and the various NG appls on
Lauren> the device could take advantage of a single pool of accepted
Lauren> connections.

You're assuming that an API exists on that box to share a connection
across multiple processes/threads/whatever.  Thats a *much* more
complex situation for the NG side of things to deal with.

Lauren> In the same way the various NG appls share the target/notify
Lauren> tables, they also share the pool of accepted connections.

Why must they share the same tables?  Can't multiple engines with
different tables run on a single network device (I do it all the
time).  Are you suggesting that you couldn't use two different
software products at the same time on the same machine because they
would have to have a common api set to share an open TCP connection?

>> Anyway, due to the above problems I hope you're not suggesting that
>> the NR always be the only one responsible for opening a connection
>> for notifications to be sent through.

Lauren> I am indeed suggesting the NR be the TCP client.

Boy are we on opposite sides of the fence on this one.  I think we're
both sitting in in disbelief that the other person is really saying
what they're saying!

Lauren> I would even argue HTTP works this way (with most of the data
Lauren> being funneled from the server to the client).

I'm afraid that analogy doesn't work well in my mind since the WEB is
a request type situation (GET) and notifications are more push based.

Certainly I'd hate to have the web server be required to open a TCP
connection to any client that might want to POST data to it ;-).

Anyway, certainly time for dinner.
Cheers,

-- 
Wes Hardaker
Distributed Computing Analysis and Support
University of California at Davis