Subject: Re: [nmrg] welcome
From: Wes Hardaker <[email-masked]>
To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked], [email-masked]
Date: 26 Jan 2000 08:17:41 -0800

>>>>> On Wed, 26 Jan 2000 12:13:02 +0100, Juergen Schoenwaelder <[email-masked]> said:

Juergen> This is exactly what I had in mind. At most one interaction
Juergen> with v2c/v3. Potentially multiple interactions in case you
Juergen> use 10-year old technology - but you probably don't expect
Juergen> compression in an SNMPv1 agent anyway.

:-)

-- 
"Ninjas aren't dangerous.  They're more afraid of you than you are of them."

Date: Wed, 26 Jan 2000 15:56:22 +0100
From: Aiko Pras <[email-masked]>
To: [email-masked]
CC: Juergen Schoenwaelder <[email-masked]>, [email-masked]
Subject: Re: [nmrg] next IRTF-NMRG meeting
Sender: [email-masked]
Status: RO

Hi

> David> So I recommend late February for a meeting, or just schedule it
> David> to occur during the week of the IETF meeting. 
Juergen> Aiko, is there a possibility to hold the meeting earlier in 
Juergen> Twente? Can you come up with a concrete proposal?
I'll be away the last 2 weeks of february. You may of course have a
meeting in Twente without me. This could be more productive :-).
If the meeting should be earlier, we can have it 2+3 march, or 9+10
march.

Btw, I like the proposed agenda. These are exactly the points I think we
should discuss!.

Aiko

Date: Wed, 26 Jan 2000 13:09:47 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: Network Management Research Group <[email-masked]>
Subject: [nmrg] next IRTF-NMRG meeting - proposed agenda
Sender: [email-masked]
Status: RO

I want to put the next meeting under the title "Adding Operations to
the Internet Management Framework".

I believe that the lack to formally describe operations (or commands
if you will Aiko) and the lack of an adequate mechanism to invoke
operations is becoming more and more a serious problem. An example
where this issue shows up is IMHO the COPS/PIB vs. SNMP/MIB debate.
(There are also several other places inside and outside the IETF where
the limitations of the Internet Management Framework to adequately
represent and invoke operations on MIB data are a serious topic.)

To break things down a bit, here is a proposal of subtopics that I
would like to address in the next meeting:

(1) SMIng extensions to formally define operations. I expect that we
    have a concrete proposal by the meeting which is based on
    <draft-irtf-nmrg-smi-ops-00.txt>, but fully integrated into SMIng.
    I would like to discuss any open issues where we do not yet have a
    good answer. Perhaps we can resolve some of them. I would also
    like to see a discussion about the relationship between SMIng and
    CIM in case some CIM experts can attend the meeting. The quesion
    here is whether there is something we can do to make mappings
    easier.

(2) Extension of the current SNMP architecture to support the
    invocation of operations via SNMP. We need to identify which new
    blocks (e.g. SNMP applications) need to be added to the current
    architecture (RFC 2571) to support operations and how these new
    blocks interact with existing blocks (e.g. access control).

(3) New SNMP protocol operations that support the invocation of
    operations. <draft-irtf-nmrg-snmp-ops-00.txt> is a first step into
    this direction. We need to discuss and identify what the precise
    invocation semantics are that we need and which are implementable
    with reasonable effort. We also need to discuss how far we can or
    want to go wrt. support of larger "transactions".

I do not want to devote much time on the bulk transfer issues. I think
we can make progress here on the mailing list and by making prototype
implementations to get some real numbers and we do not necessarily
need a face to face meeting.

But I am of course open for suggestions.

/js

Date: Wed, 26 Jan 2000 12:13:02 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] welcome
Sender: [email-masked]
Status: RO

>>>>> Wes Hardaker writes:

Wes> My point was that if you have table indexed by some value X (in
Wes> this case, say an integer) such that the following mappings take
Wes> place:

[...]

Wes> v1: GET(compressionSupported.1, compressionSupported.3) ->
Wes> errorindex = 2 so we have to send a second packet:
Wes> GET(compressionSupported.1) -> compressionSupported.1 = true

Wes> v2c/v3: GET(compressionSupported.1, compressionSupported.3) ->
Wes> compressionSupported.1 = true compressionSupported.3 = No Such
Wes> Instance

This is exactly what I had in mind. At most one interaction with
v2c/v3. Potentially multiple interactions in case you use 10-year old
technology - but you probably don't expect compression in an SNMPv1
agent anyway.

/js

To: Joe Marzot <[email-masked]>
Cc: Wes Hardaker <[email-masked]>, Juergen Schoenwaelder <[email-masked]>, [email-masked]
Subject: Re: [nmrg] welcome
From: Wes Hardaker <[email-masked]>
Organization: U.C.Davis, Information Technology - D.C.A.S.
Date: 25 Jan 2000 10:31:06 -0800
Lines: 37
Sender: [email-masked]
Status: RO

>>>>> On 25 Jan 2000 09:59:45 -0500, Joe Marzot <[email-masked]> said:

>> Then a single get packet could determine which functions were
>> supported.  Of course, I guess SNMPv1 would give you an error for
>> unsupported types requires more guesses.

Joe> what does this last statement mean? what unsupported types would SNMPv1
Joe> encounter?

My point was that if you have table indexed by some value X (in this
case, say an integer) such that the following mappings take place:

int   meaning
1     deflate
2     gzip
3     juergen-oid-compression

And a v1 requset was made to the table to get compressionSupported.INT 
for, say, values 1 and 3 but the agent only implemented values 1 and 2 
then value 3 would be a non-existent request and hence would return an 
error in errorStatus field, where as the v2c/v3 protocols would return 
an error in the varbind and the results of the rest of the query.  The 
v1 request would have to remove the bad varbind in the request, and
resend it to test for the existence of the rest of the compression
types supported.

v1:
    GET(compressionSupported.1, compressionSupported.3) -> errorindex = 2
  so we have to send a second packet:
    GET(compressionSupported.1) -> compressionSupported.1 = true

v2c/v3:
    GET(compressionSupported.1, compressionSupported.3) -> 
      compressionSupported.1 = true
      compressionSupported.3 = No Such Instance
-- 
"Ninjas aren't dangerous.  They're more afraid of you than you are of them."

Date: Tue, 25 Jan 2000 17:50:59 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked], [email-masked]
Subject: Re: [nmrg] next IRTF-NMRG meeting
Sender: [email-masked]
Status: RO

>>>>> David Harrington writes:

David> I have concerns with the proposed dates.

David> 1) The two weeks prior to an IETF meeting are the heaviest
David> publication times for I-Ds and RFCs, and thus require
David> substantial time for document review. I feel it would be better
David> to have the meeting earlier than this to prevent that conflict.

Fair statement.

David> 2) Following the NMRG meeting, it would be good to get the
David> results published so the discussed topics can be further
David> discussed during the IETF meeting. Having the meeting this
David> close to the IETF meeting would make that harder.

Not sure this is so much of a problem. We do not have to align our
schedule with the IETF - at least to an extend that we have to worry
about IETF rules such as the ID cutoff dates.

(I prefer to develop ideas a bit further within the NMRG before we
worry about the question whether "the IETF" wants to discuss the
results or not. At this time, I am more worried that we produce
reasonable "research" results in a timely manner.)

David> So I recommend late February for a meeting, or just schedule it
David> to occur during the week of the IETF meeting.

Aiko, is there a possibility to hold the meeting earlier in Twente?
Can you come up with a concrete proposal?

To all the others: Can you please answer the following questions:

 (a) Would you be able to attend the March meeting?

 (b) Would you be able to attend a meeting at the end of February?
     (Aiko to propose a concrete date.)

 (c) Will you not be able to attend the next meeting regardless of the
     date since e.g. the meeting site is way too far from your home
     base?

 (d) Whether you have suggestions for alternate or future meeting
     places and dates.

Thanks,

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

Date: Tue, 25 Jan 2000 17:17:08 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked], [email-masked]
Subject: Re: [nmrg] welcome
Sender: [email-masked]
Status: RO

>>>>> Joe Marzot writes:

Joe> (btw, why do we think implelemtations will not tolerate extension
Joe> tags - is it supported by the spec?).

I think ASN.1 (not sure which version) defines it, but I have to check
the documents to be sure. All the SNMP code I am familiar with does
not expect that a tag is spread over multiple bytes. Hence my guess
that relying on extension tags is probably a bad idea.

(But perhaps I am just using the wrong code bases. ;-)

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

To: Wes Hardaker <[email-masked]>
Cc: Juergen Schoenwaelder <[email-masked]>, [email-masked]
Subject: Re: [nmrg] welcome
From: Joe Marzot <[email-masked]>
Date: 25 Jan 2000 10:07:31 -0500
Lines: 40
Sender: [email-masked]
Status: RO

Wes Hardaker <[email-masked]> writes:

> >>>>> On Thu, 20 Jan 2000 14:17:05 +0100, Juergen Schoenwaelder <[email-masked]> said:
> 
> Joe> Before I was thinking that the PDU tagged octet string would be
> Joe> one way to tell. Self describing PDUs are good from an
> Joe> administrative point of view.
> 
> Juergen> You can either use the PDU tag or you have a special tag
> Juergen> inside of the compressed PDU. The second option is IMHO the
> Juergen> better choice since the PDU tag space is essentially limited
> Juergen> to 5 bits (unless you use extension tags, which most
> Juergen> implementations will not handle properly, I guess.)

yes, I see the point. (btw, why do we think implelemtations will not
tolerate extension tags - is it supported by the spec?).

I think an integer tag would be better than an oid tag in light of the
desire not to grow the pdu through this conversion. I imagine that part
of the algorithm will be to see if the resultant compressed PDU is
smaller than the original and discard if not so.

-GSM

> 
> Just for anyone keeping tally marks, I too would prefer a separate
> tag.  It's cleaner in my opinion as well.
> 
> -- 
> Wes Hardaker
> Distributed Computing Analysis and Support
> University of California at Davis
> 
> 

-- 
G.S. Marzot                        email: [email-masked]
Nortel Networks                    voice: (978)288-3990
600 Tech Park  M/S E65-60-101      pager: (800)409-6080 ([email-masked])
Billerica, MA  01821                 fax: (978)670-8145

To: Wes Hardaker <[email-masked]>
Cc: Juergen Schoenwaelder <[email-masked]>, [email-masked]
Subject: Re: [nmrg] welcome
From: Joe Marzot <[email-masked]>
Date: 25 Jan 2000 09:59:45 -0500
Lines: 40
Sender: [email-masked]
Status: RO

Wes Hardaker <[email-masked]> writes:

> >>>>> On Mon, 17 Jan 2000 16:51:47 +0100, Juergen Schoenwaelder <[email-masked]> said:
> 
> Juergen> I am looking for a mechanism which allows to select the
> Juergen> compression type with one SNMP get/response interaction. For
> Juergen> a command generator, you can simply generate a get-request
> Juergen> which gets information for all the compression algorithms
> Juergen> supported by the command generator. The command responder
> Juergen> will only respond to those varbinds that he cares about (and
> Juergen> the rest will return an exception). Now the command generator
> Juergen> can pick the algorithm he prefers and which is supported by
> Juergen> the agent.
> 
> Something like a snmp table that was indexed by an oid or integer
> describing the compression algorithm (if it was an integer, doing
> something similar to the assignment range used in SM modules would be
> wise to allow both standardized and enterprise specific compression
> types.  Then a single get packet could determine which functions were
> supported.  Of course, I guess SNMPv1 would give you an error for
> unsupported types requires more guesses.

what does this last statement mean? what unsupported types would SNMPv1
encounter?

-GSM

> 
> -- 
> Wes Hardaker
> Distributed Computing Analysis and Support
> University of California at Davis
> 
> 

-- 
G.S. Marzot                        email: [email-masked]
Nortel Networks                    voice: (978)288-3990
600 Tech Park  M/S E65-60-101      pager: (800)409-6080 ([email-masked])
Billerica, MA  01821                 fax: (978)670-8145

To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: [nmrg] delta oid compression
From: Wes Hardaker <[email-masked]>
Organization: U.C.Davis, Information Technology - D.C.A.S.
Date: 24 Jan 2000 14:48:00 -0800
Lines: 81
Sender: [email-masked]
Status: RO

>>>>> On Mon, 17 Jan 2000 17:55:19 +0100, Juergen Schoenwaelder <[email-masked]> said:

Juergen> Both are possible. The second version requires you to replace
Juergen> all "new" subidentifiers.

No, I was trying to say that the extend-range should be optional.  You 
shouldn't *have* to use it and then replace the values if you're going 
to replace everything anyway.  If you want the default value in
numerous spots, then you could use the extension if it was going to
save you bytes by not having to do the replacement.

Juergen> The first version does not require to replace "new"
Juergen> subidentifiers if the default is already usable (which will
Juergen> rarely be the case, I guess).

I'd bet most commonly you'd use at most 1 of the default value segments.

wjhardaker> Choice 2)

wjhardaker> Similar as #1 but with explicitly 1 trunc, and adding
wjhardaker> ranges into the index space instead.  Explicitly allow a
wjhardaker> trunc of 0x80 if the first part of the encoding needs to
wjhardaker> be a range.  A trunc of 0x80 would essentially mean "skip"
wjhardaker> and don't truncate:

Juergen> I tend to prefer choice 2 since it makes the range a bit more
Juergen> compact, which may pay off if you do multiple range
Juergen> substitutions for a single OID.

This was my preference too.

Juergen> So the highest bit means basically a range follows and we
Juergen> need to decide how we handle truncation. Depending on the
Juergen> fact that the highest bit in the first byte has different
Juergen> meaning than the rest of the bytes (and thus requires pseudo
Juergen> truncation bytes in some cases) certainly works, but it may
Juergen> lead to implementation problems if people do not get it
Juergen> right.

Certainly if people don't get something right, it'll always lead to
implementation problems :-).

However, note that parsing OIDs already requires special attention of
the first byte (since OIDs don't encode the first two segments as a
single byte).

Juergen> I think I should go and implement this stuff in order to
Juergen> figure out how it behaves on real-world data.

:-)

Juergen> I dislike OIDs but I see value in having the compression
Juergen> identification shipped with the data (especially for
Juergen> traps). One way out would be to use an integer and to have
Juergen> rules similar to the SnmpSecurityModel definition in RFC
Juergen> 2571. To quote RFC 2571:

Yep, I obviously didn't read this note before posting the other one I
wrote a few seconds ago mentioning the same suggestion.

Juergen> Another option would be to encode an algorithm in the
Juergen> compressed PDU, which makes it a bit more difficult to
Juergen> satisfy the non-expansion constraint.

wjhardaker> Which is?

Juergen> The non-expansion constraint means that the compressed
Juergen> payload must not be larger than the uncompressed payload. You
Juergen> loose some compression if you encode the compression
Juergen> algorithm in the data. But that is probably OK given the
Juergen> benefits it has.

I'd tend to agree here.  The gains far outweigh the losses.  And,
anyone who encrypts data will have to suffer anyway if their is almost 
no space gain because its uncompressible data, since they will have to 
spend CPU cycles for no reason.

-- 
Wes Hardaker
Distributed Computing Analysis and Support
University of California at Davis

To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: [nmrg] welcome
From: Wes Hardaker <[email-masked]>
Organization: U.C.Davis, Information Technology - D.C.A.S.
Date: 24 Jan 2000 14:31:23 -0800
Lines: 19
Sender: [email-masked]
Status: RO

>>>>> On Thu, 20 Jan 2000 14:17:05 +0100, Juergen Schoenwaelder <[email-masked]> said:

Joe> Before I was thinking that the PDU tagged octet string would be
Joe> one way to tell. Self describing PDUs are good from an
Joe> administrative point of view.

Juergen> You can either use the PDU tag or you have a special tag
Juergen> inside of the compressed PDU. The second option is IMHO the
Juergen> better choice since the PDU tag space is essentially limited
Juergen> to 5 bits (unless you use extension tags, which most
Juergen> implementations will not handle properly, I guess.)

Just for anyone keeping tally marks, I too would prefer a separate
tag.  It's cleaner in my opinion as well.

-- 
Wes Hardaker
Distributed Computing Analysis and Support
University of California at Davis

To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: [nmrg] welcome
From: Wes Hardaker <[email-masked]>
Organization: U.C.Davis, Information Technology - D.C.A.S.
Date: 24 Jan 2000 14:29:37 -0800
Lines: 24
Sender: [email-masked]
Status: RO

>>>>> On Mon, 17 Jan 2000 16:51:47 +0100, Juergen Schoenwaelder <[email-masked]> said:

Juergen> I am looking for a mechanism which allows to select the
Juergen> compression type with one SNMP get/response interaction. For
Juergen> a command generator, you can simply generate a get-request
Juergen> which gets information for all the compression algorithms
Juergen> supported by the command generator. The command responder
Juergen> will only respond to those varbinds that he cares about (and
Juergen> the rest will return an exception). Now the command generator
Juergen> can pick the algorithm he prefers and which is supported by
Juergen> the agent.

Something like a snmp table that was indexed by an oid or integer
describing the compression algorithm (if it was an integer, doing
something similar to the assignment range used in SM modules would be
wise to allow both standardized and enterprise specific compression
types.  Then a single get packet could determine which functions were
supported.  Of course, I guess SNMPv1 would give you an error for
unsupported types requires more guesses.

-- 
Wes Hardaker
Distributed Computing Analysis and Support
University of California at Davis

Date: Mon, 24 Jan 2000 11:01:02 -0500
From: David Harrington <[email-masked]>
To: Aiko Pras <[email-masked]>
CC: Juergen Schoenwaelder <[email-masked]>, Network Management Research Group <[email-masked]>
Subject: Re: [nmrg] next IRTF-NMRG meeting
Sender: [email-masked]
Status: RO

Hi,

I have concerns with the proposed dates. 
1) The two weeks prior to an IETF meeting are the heaviest publication
times for I-Ds and RFCs, and thus require substantial time for document
review. I feel it would be better to have the meeting earlier than this
to prevent that conflict.
2) Following the NMRG meeting, it would be good to get the results
published so the discussed topics can be further discussed during the
IETF meeting. Having the meeting this close to the IETF meeting would
make that harder.

So I recommend late February for a meeting, or just schedule it to occur
during the week of the IETF meeting.

dbh

Aiko Pras wrote:
> 
> Hi all
> 
> Some time back Juergen asked for suggestions and a date for the next
> IRTF-NMRG meeting. Since there has not yet been a reaction from someone
> else, I propose to meet in the middle of March at the University of
> Twente (UT), where we have good meeting facilities.
> 
> If we want a two day meeting, I propose the 16th and 17th of march. If
> we want a three day meeting, we could start a day earlier on wednesday
> the 15th.
> 
> Everyone will be there??? :-)
> 
> Bye
> 
> Aiko

Date: Mon, 24 Jan 2000 13:23:14 +0100
From: Aiko Pras <[email-masked]>
To: Juergen Schoenwaelder <[email-masked]>
CC: Network Management Research Group <[email-masked]>
Subject: [nmrg] next IRTF-NMRG meeting
Sender: [email-masked]
Status: RO

Hi all

Some time back Juergen asked for suggestions and a date for the next
IRTF-NMRG meeting. Since there has not yet been a reaction from someone
else, I propose to meet in the middle of March at the University of
Twente (UT), where we have good meeting facilities.

If we want a two day meeting, I propose the 16th and 17th of march. If
we want a three day meeting, we could start a day earlier on wednesday
the 15th.

Everyone will be there??? :-)

Bye

Aiko

Date: Sun, 23 Jan 2000 13:43:48 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: Network Management Research Group <[email-masked]>
Subject: [nmrg] MIB Object Dependency Expression (MODEX)
Sender: [email-masked]
Status: RO

Someone wrote an interesting ID on formally expressing dependencies
between MIB objects. For more details, please read
<draft-dacosta-snmp-object-dependency-00.txt>.

/js

Date: Thu, 20 Jan 2000 14:17:05 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked], [email-masked]
Subject: Re: [nmrg] welcome
Sender: [email-masked]
Status: RO

>>>>> Joe Marzot writes:

Joe> say a CR handles 3 different compression alg and the CG decidees
Joe> to use alg #2 - how will the CR know which one is used when the
Joe> PDU arrives.

Joe> Before I was thinking that the PDU tagged octet string would be
Joe> one way to tell. Self describing PDUs are good from an
Joe> administrative point of view.

You can either use the PDU tag or you have a special tag inside of the
compressed PDU. The second option is IMHO the better choice since the
PDU tag space is essentially limited to 5 bits (unless you use
extension tags, which most implementations will not handle properly,
I guess.)

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: [nmrg] welcome
From: Joe Marzot <[email-masked]>
Date: 17 Jan 2000 16:30:50 -0500
Lines: 53
Sender: [email-masked]
Status: RO

Juergen Schoenwaelder <[email-masked]> writes:

> >>>>> wjhardaker  writes:
> 
> wjhardaker> Hmm...  How would you negotiate with something that
> wjhardaker> supports multiple compression types?  (I'd like to be able
> wjhardaker> to pick one of the ones the agent supports).  Or are you
> wjhardaker> assuming you MUST use the compression type that the agent
> wjhardaker> dictates (I need to go re-read the draft, its been too
> wjhardaker> long, but I think this is what I remember you wanting to
> wjhardaker> do).
> 
> I am looking for a mechanism which allows to select the compression
> type with one SNMP get/response interaction. For a command generator,
> you can simply generate a get-request which gets information for all
> the compression algorithms supported by the command generator. The
> command responder will only respond to those varbinds that he cares
> about (and the rest will return an exception). Now the command
> generator can pick the algorithm he prefers and which is supported by
> the agent.
> 

say a CR handles 3 different compression alg and the CG decidees to use
alg #2 - how will the CR know which one is used when the PDU arrives.

Before I was thinking that the PDU tagged octet string would be one way
to tell. Self describing PDUs are good from an administrative point of
view.

-GSM

> Of course, this does not work well for a notification originator. So
> we probably need to preconfigure which compression algorithm to use
> (if any). In most cases, compression on notifications won't make much
> sense since notifications itself tend to be small and not bulky.
> 
> /js
> 
> -- 
> Juergen Schoenwaelder      Technical University Braunschweig
> <[email-masked]>  Dept. Operating Systems & Computer Networks
> Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
> Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>
> 
> 
> 
> 

-- 
G.S. Marzot                        email: [email-masked]
Nortel Networks                    voice: (978)288-3990
600 Tech Park  M/S E65-60-101      pager: (800)409-6080 ([email-masked])
Billerica, MA  01821                 fax: (978)670-8145

Date: Mon, 17 Jan 2000 17:55:19 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] delta oid compression
Sender: [email-masked]
Status: RO

>>>>> wjhardaker  writes:

>>>>> On Tue, 11 Jan 2000 18:09:17 +0100, Juergen Schoenwaelder <[email-masked]> said:

wjhardaker> Couple of points to the discussion:

wjhardaker> I'm not sure why you want to truncate in #2 here:

[...]

wjhardaker> Why not let a replacement happen even if there is nothing
wjhardaker> present:

[...]

Both are possible. The second version requires you to replace all
"new" subidentifiers. The first version does not require to replace
"new" subidentifiers if the default is already usable (which will
rarely be the case, I guess).

wjhardaker> Well...  You could use your truncation bit a bit
wjhardaker> differently...

[...]

wjhardaker> Choice 1)

wjhardaker> I think you might want repeated truncation clauses, that a
wjhardaker> special value of 0x80 would indicate a range following...

[...]

wjhardaker> Choice 2)

wjhardaker>   Similar as #1 but with explicitly 1 trunc, and adding
wjhardaker> ranges into the index space instead.  Explicitly allow a
wjhardaker> trunc of 0x80 if the first part of the encoding needs to
wjhardaker> be a range.  A trunc of 0x80 would essentially mean "skip"
wjhardaker> and don't truncate:

[...]

I tend to prefer choice 2 since it makes the range a bit more compact,
which may pay off if you do multiple range substitutions for a single
OID. So the highest bit means basically a range follows and we need to
decide how we handle truncation. Depending on the fact that the
highest bit in the first byte has different meaning than the rest of
the bytes (and thus requires pseudo truncation bytes in some cases)
certainly works, but it may lead to implementation problems if people
do not get it right.

I think I should go and implement this stuff in order to figure out
how it behaves on real-world data.

Juergen> Our thinking so far is to have one compressed PDU (a tagged
Juergen> OCTET STRING) regardless of the compression algorithm in
Juergen> use. You get the compression algorithm from information in
Juergen> the MIB.

wjhardaker> Which causes problems with respect to multiple supported
wjhardaker> algorithms, trap/notification generators, etc.  I think
wjhardaker> I'd prefer a magic number in front of the compressed data
wjhardaker> in the octet string.  Ideally, the magic number should be
wjhardaker> an OID but then you get a long OID which you're trying to
wjhardaker> compress.  Though, we could implement a minor compression
wjhardaker> there by encoding the OID with a couple of pre-defined
wjhardaker> prefixes (standardized mib area, enterprise, etc).

I dislike OIDs but I see value in having the compression
identification shipped with the data (especially for traps). One way
out would be to use an integer and to have rules similar to the
SnmpSecurityModel definition in RFC 2571. To quote RFC 2571:

    "This limits the ability to define new proprietary implementations
    of Security Models to the first 8,388,608 enterprises."

[Some form of limited extensibility.]

Juergen> Another option would be to encode an algorithm in the
Juergen> compressed PDU, which makes it a bit more difficult to
Juergen> satisfy the non-expansion constraint.

wjhardaker> Which is?

The non-expansion constraint means that the compressed payload must
not be larger than the uncompressed payload. You loose some
compression if you encode the compression algorithm in the data. But
that is probably OK given the benefits it has.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

Date: Mon, 17 Jan 2000 16:51:47 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked], [email-masked]
Subject: Re: [nmrg] welcome
Sender: [email-masked]
Status: RO

>>>>> wjhardaker  writes:

wjhardaker> Hmm...  How would you negotiate with something that
wjhardaker> supports multiple compression types?  (I'd like to be able
wjhardaker> to pick one of the ones the agent supports).  Or are you
wjhardaker> assuming you MUST use the compression type that the agent
wjhardaker> dictates (I need to go re-read the draft, its been too
wjhardaker> long, but I think this is what I remember you wanting to
wjhardaker> do).

I am looking for a mechanism which allows to select the compression
type with one SNMP get/response interaction. For a command generator,
you can simply generate a get-request which gets information for all
the compression algorithms supported by the command generator. The
command responder will only respond to those varbinds that he cares
about (and the rest will return an exception). Now the command
generator can pick the algorithm he prefers and which is supported by
the agent.

Of course, this does not work well for a notification originator. So
we probably need to preconfigure which compression algorithm to use
(if any). In most cases, compression on notifications won't make much
sense since notifications itself tend to be small and not bulky.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

From: [email-masked]
To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: [nmrg] delta oid compression
Organization: U.C.Davis, Information Technology - D.C.A.S.
Date: 14 Jan 2000 10:19:23 -0800
Lines: 94
Sender: [email-masked]
Status: RO

>>>>> On Tue, 11 Jan 2000 18:09:17 +0100, Juergen Schoenwaelder <[email-masked]> said:

Couple of points to the discussion:

I'm not sure why you want to truncate in #2 here:

1.3.6.1.2.1.1.8.0	06 08 2B06010201010800	    xx 02 0808
1.3.6.1.2.1.1.9.1.2.1	06 0A 2B060102010109010201  xx 09 8B080909010A020B01

Why not let a replacement happen even if there is nothing present:

1.3.6.1.2.1.1.9.1.2.1	06 0A 2B060102010109010201  xx 08 080909010A020B01

Juergen> I am looking for a way to only do range replacements. Single
Juergen> subid replacements should fall out of this as a degenerated
Juergen> case. Detecting ranges is pretty easy by looking one or two
Juergen> subidentifiers ahead.

Well...  You could use your truncation bit a bit differently...

    [<trunc>]{<index><value>}+

    <trunc> = 0x80 + (0..127)
    <index> = (0..127)
    <value> = BER encoded subidentifier

Choice 1)

  I think you might want repeated truncation clauses, that a special
  value of 0x80 would indicate a range following...

    [<trunc>|<range><start><len><values>+]+{<index><value>}+

    <trunc> = 0x80 + (1..127)
    <range> = 0x80
    <start> = (0..127)
    <len>   = (0..127)
    <index> = (0..127)
    <value> = BER encoded subidentifier

  Old:
    1.3.6.1.2.1.1.8.0	   06 08 2B06010201010800      xx 02 0808
    1.3.6.1.2.1.1.9.1.2.1  06 0A 2B060102010109010201  xx 09 8B080909010A020B01

  New (assuming you want to keep the initial trunc discussed above):
    1.3.6.1.2.1.1.8.0	   06 08 2B06010201010800      xx 02 0808
    1.3.6.1.2.1.1.9.1.2.1  06 0A 2B060102010109010201  xx 08 8B80080409010201

Choice 2)

  Similar as #1 but with explicitly 1 trunc, and adding ranges into
  the index space instead.  Explicitly allow a trunc of 0x80 if the
  first part of the encoding needs to be a range.  A trunc of 0x80
  would essentially mean "skip" and don't truncate:

    [<trunc>]{<index><value>|<range><len><values>+}+

    <trunc> = 0x80 + (1..127)
    <index> = (0..127)
    <value> = BER encoded subidentifier
    <range> = 0x80 + (1..127) -- indicating start position
    <len>   = (0..127)

  Old:
    1.3.6.1.2.1.1.8.0	   06 08 2B06010201010800      xx 02 0808
    1.3.6.1.2.1.1.9.1.2.1  06 0A 2B060102010109010201  xx 09 8B080909010A020B01

  New (assuming you want to keep the initial trunc discussed above):
    1.3.6.1.2.1.1.8.0	   06 08 2B06010201010800      xx 02 0808
    1.3.6.1.2.1.1.9.1.2.1  06 0A 2B060102010109010201  xx 07 80880409010201

Juergen> Our thinking so far is to have one compressed PDU (a tagged OCTET
Juergen> STRING) regardless of the compression algorithm in use. You get the
Juergen> compression algorithm from information in the MIB.

Which causes problems with respect to multiple supported algorithms,
trap/notification generators, etc.  I think I'd prefer a magic number
in front of the compressed data in the octet string.  Ideally, the
magic number should be an OID but then you get a long OID which you're 
trying to compress.  Though, we could implement a minor compression
there by encoding the OID with a couple of pre-defined prefixes
(standardized mib area, enterprise, etc).

Juergen> Another option would be to encode an algorithm in the
Juergen> compressed PDU, which makes it a bit more difficult to
Juergen> satisfy the non-expansion constraint.

Which is?

-- 
Wes Hardaker
Distributed Computing Analysis and Support
University of California at Davis

From: [email-masked]
To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: [nmrg] welcome
Organization: U.C.Davis, Information Technology - D.C.A.S.
Date: 14 Jan 2000 09:26:57 -0800
Lines: 16
Sender: [email-masked]
Status: RO

>>>>> On Tue, 11 Jan 2000 11:44:31 +0100, Juergen Schoenwaelder <[email-masked]> said:

Juergen> I think we know how this can work efficiently in the command
Juergen> generator/command responder case via autodiscovery. Dealing
Juergen> with a notification originators is a bit harder...

Hmm...  How would you negotiate with something that supports multiple
compression types?  (I'd like to be able to pick one of the ones the
agent supports).  Or are you assuming you MUST use the compression
type that the agent dictates (I need to go re-read the draft, its been 
too long, but I think this is what I remember you wanting to do).

-- 
Wes Hardaker
Distributed Computing Analysis and Support
University of California at Davis

Date: Wed, 12 Jan 2000 13:05:45 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: Network Management Research Group <[email-masked]>
Subject: [nmrg] Andrea Westerinen joins the NMRG mailing list
Sender: [email-masked]
Status: RO

Andrea Westerinen <[email-masked]> has just joined the NMRG
mailing list. Andrea has strong backgrounds in CIM. She is interested
to contribute to the SMIng development (and I expect that she is less
interested in things like SNMP performance improvement).

The idea here is to make sure that new SMIng features (like operations
or mechanisms to represent relationships) are defined in such a way
that they easily align with CIM. Andrea's knowledge about CIM might be
very valuable for achieving this goal.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

Date: Tue, 11 Jan 2000 18:09:17 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] delta oid compression
Sender: [email-masked]
Status: RO

>>>>> Joe Marzot writes:

Joe> In the alternate scheme I was thinking, the range and length is
Joe> always everything from the supplied starting point to the end.

Joe> Essentially it re-uses the first M sub-ids from the previous
Joe> varbind and appends the following supplied sub-ids to it. These
Joe> are not replacements to some arbitrarily long sub-section of the
Joe> oid but just the end.

Joe> But I see this does not address the case of large, repeated
Joe> instance identifiers where just some portion is changing.

Exactly. And this can be very valuable if you walk a table row by row
(since the only changing subid is the column number).

Joe> perhaps the are 2 types of deltas one which replaces individual
Joe> sub-ids and another which replaces everyhting from the starting
Joe> point on...but then something would have to determine what the
Joe> optimum delta type is...

I am looking for a way to only do range replacements. Single subid
replacements should fall out of this as a degenerated case. Detecting
ranges is pretty easy by looking one or two subidentifiers ahead.

Joe> the compression technique used, Deflate or DOC, will be
Joe> differentiated based on PDU type, right? If I look at the PDU and
Joe> it is a tagged octet string then the tag tells me what type of
Joe> compression is used, right?  (sorry have not read enough)

Our thinking so far is to have one compressed PDU (a tagged OCTET
STRING) regardless of the compression algorithm in use. You get the
compression algorithm from information in the MIB. Another option
would be to encode an algorithm in the compressed PDU, which makes it
a bit more difficult to satisfy the non-expansion constraint. But I
know we have discussed this point to same extend in the past.

Joe> in anycase, my thought above is that other non-ber compression
Joe> schemes are certainly feasible using different wrappers but a
Joe> simple technique that gets the the majority of the benefit and
Joe> requires little in the way of additional libraries (also may be
Joe> faster since there is less translation) is desireable.

I think there is a need for at least two algorithms. First, an
algorithm which is very efficient and which does a good job to remove
OID redundancies but which leaves payload uncompressed. The second
algorithm should be able to do good compression on complete PDU
(including the payload) and may take more resources to compute. Let
the application decide which is the most effective solution for a
given task. But even if we don't agree on this view, we should at
least design things so that algorithms can be replaced if needed.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked]
Subject: Re: [nmrg] delta oid compression
From: Joe Marzot <[email-masked]>
Date: 11 Jan 2000 11:55:21 -0500
Lines: 97
Sender: [email-masked]
Status: RO

Juergen Schoenwaelder <[email-masked]> writes:

> >>>>> Joe Marzot writes:
> 
> Joe> this could be
> 
> Joe> 1.3.6.1.2.1.1.9.1.2.1 06 0A 2B060102010109010201 xx 04 09010201
> 
> Joe> assuming encoding of varbind N+1 is based on the expansion of
> Joe> varbind N, then the implementation need only remember the last
> Joe> one to encode/decode the next.
> 
> How do you know that it is a range? How do you know that the length of
> the OID changes? How do you recognize the end of the range? I think
> that range encoding requires a bit more work and bits.

perhaps I mis-understood your proposal - on re-reading I see that you
are replacing one sub-id at a time, and there can be multiple single
sub-ids replaced.

In the alternate scheme I was thinking, the range and length is always
everything from the supplied starting point to the end.

Essentially it re-uses the first M sub-ids from the previous varbind and
appends the following supplied sub-ids to it. These are not replacements
to some arbitrarily long sub-section of the oid but just the end.

But I see this does not address the case of large, repeated instance
identifiers where just some portion is changing.

perhaps the are 2 types of deltas one which replaces individual sub-ids
and another which replaces everyhting from the starting point on...but
then something would have to determine what the optimum delta type is...

oh well just thinking out loud.

> 
> I plan to prototype some different compression schemes so that it is
> easy to run test data with various compression schemes. A particularly
> interesting example so far have been the DISMAN-SCRIPT-MIB since it
> have tables indexed by multiple variable length strings. And this is
> where the fun begins.
> 
> Perhaps we need to collect "typical" PDUs that one would like to
> compress so that we can play with the algorithms a bit. Is there
> anyone who has real-world data from RMON and RMON2 probes or other
> interesting MIBs with voluminous data?
> 
> Joe> simplicity of implementation, re-use of existing ber libs get my
> Joe> vote.
> 
> I am not yet sure this is a good choice. With deflate, you put the
> whole compressed PDU into the decoder and you get back the BER input
> for your SNMP engine. This has the advantage that it works with all
> exisiting and future SNMP PDUs. Doing the same (at least conceptually)
> with a less costly compression scheme may actually be preferable since
> you do not make any assumptions on the structure of the PDU. This
> scheme boils down to a table with compression/decompression function
> pointers and you simply call the right function depending on the
> compression algorithm in use. This is easily extensible and you are
> free to actually use non BER conforming encodings in the compressed
> data.

the compression technique used, Deflate or DOC, will be differentiated
based on PDU type, right? If I look at the PDU and it is a tagged octet
string then the tag tells me what type of compression is used, right?
(sorry have not read enough)

in anycase, my thought above is that other non-ber compression schemes
are certainly feasible using different wrappers but a simple technique
that gets the the majority of the benefit and requires little in the way
of additional libraries (also may be faster since there is less
translation) is desireable.

-GSM

> 
> So perhaps my vote is for extensibility and architectural purity
> (which IMHO leads to simplicity of implementation if done right).
> 
> /js
> 
> -- 
> Juergen Schoenwaelder      Technical University Braunschweig
> <[email-masked]>  Dept. Operating Systems & Computer Networks
> Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
> Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>
> 
> 
> 
> 

-- 
G.S. Marzot                        email: [email-masked]
Nortel Networks                    voice: (978)288-3990
600 Tech Park  M/S E65-60-101      pager: (800)409-6080 ([email-masked])
Billerica, MA  01821                 fax: (978)670-8145

Date: Tue, 11 Jan 2000 17:08:57 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] delta oid compression
Sender: [email-masked]
Status: RO

>>>>> Joe Marzot writes:

Joe> this could be

Joe> 1.3.6.1.2.1.1.9.1.2.1 06 0A 2B060102010109010201 xx 04 09010201

Joe> assuming encoding of varbind N+1 is based on the expansion of
Joe> varbind N, then the implementation need only remember the last
Joe> one to encode/decode the next.

How do you know that it is a range? How do you know that the length of
the OID changes? How do you recognize the end of the range? I think
that range encoding requires a bit more work and bits.

I plan to prototype some different compression schemes so that it is
easy to run test data with various compression schemes. A particularly
interesting example so far have been the DISMAN-SCRIPT-MIB since it
have tables indexed by multiple variable length strings. And this is
where the fun begins.

Perhaps we need to collect "typical" PDUs that one would like to
compress so that we can play with the algorithms a bit. Is there
anyone who has real-world data from RMON and RMON2 probes or other
interesting MIBs with voluminous data?

Joe> simplicity of implementation, re-use of existing ber libs get my
Joe> vote.

I am not yet sure this is a good choice. With deflate, you put the
whole compressed PDU into the decoder and you get back the BER input
for your SNMP engine. This has the advantage that it works with all
exisiting and future SNMP PDUs. Doing the same (at least conceptually)
with a less costly compression scheme may actually be preferable since
you do not make any assumptions on the structure of the PDU. This
scheme boils down to a table with compression/decompression function
pointers and you simply call the right function depending on the
compression algorithm in use. This is easily extensible and you are
free to actually use non BER conforming encodings in the compressed
data.

So perhaps my vote is for extensibility and architectural purity
(which IMHO leads to simplicity of implementation if done right).

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

To: Juergen Schoenwaelder <[email-masked]>
Cc: Network Management Research Group <[email-masked]>
Subject: Re: [nmrg] delta oid compression
From: Joe Marzot <[email-masked]>
Date: 11 Jan 2000 10:19:38 -0500
Lines: 99
Sender: [email-masked]
Status: RO

one observation - a new data type should not be problematic since old
implementations or those that do not support the new DOC PDU should
discard it without before ever getting to the new data type.

Juergen Schoenwaelder <[email-masked]> writes:

> Here is a more detailed example for what I have in mind. Lets assume
> we make a "$s getbulk 0 11 SNMPv2-MIB::system" (in scotty notation).
> This will return the following OIDs for ObjectName:
> 
> OID			BER [116 byte]		    COMP BER [57 byte]
> 
> 1.3.6.1.2.1.1.1.0	06 08 2B06010201010100	    06 08 2B06010201010100
> 1.3.6.1.2.1.1.2.0	06 08 2B06010201010200	    xx 02 0802
> 1.3.6.1.2.1.1.3.0	06 08 2B06010201010300	    xx 02 0803
> 1.3.6.1.2.1.1.4.0	06 08 2B06010201010400      xx 02 0804
> 1.3.6.1.2.1.1.5.0	06 08 2B06010201010500      xx 02 0805
> 1.3.6.1.2.1.1.6.0	06 08 2B06010201010600      xx 02 0806
> 1.3.6.1.2.1.1.7.0	06 08 2B06010201010700      xx 02 0807
> 1.3.6.1.2.1.1.8.0	06 08 2B06010201010800	    xx 02 0808
> 1.3.6.1.2.1.1.9.1.2.1	06 0A 2B060102010109010201  xx 09 8B080909010A020B01

this could be

1.3.6.1.2.1.1.9.1.2.1	06 0A 2B060102010109010201  xx 04 09010201

assuming encoding of varbind N+1 is based on the expansion of varbind N,
then the implementation need only remember the last one to encode/decode
the next.

> 1.3.6.1.2.1.1.9.1.3.1   06 0A 2B060102010109010301  xx 02 0A03
> 1.3.6.1.2.1.1.9.1.4.1	06 0A 2B060102010109010401  xx 02 0A04
> 
> The right column shows a simple ASN.1/BER compatible way to encode
> deltas. The value <xx> stands for a yet to be assigned tag value. In
> fact, the encoding is scheme is valid ASN.1/BER based on the following
> replacement definition for ObjectName:
> 
>   ObjectName ::= CHOICE {
>       name OBJECT IDENTIFIER,
>       comp DeltaIdentifier
>   }
> 
>   DeltaIdentifier ::= [APPLICATION xx] IMPLICIT OCTET STRING
> 
> The content of the DeltaIdentifier has the following structure:
> 
>     [<trunc>]{<index><value>}+
> 
>     <trunc> = 0x80 + (0..127)
>     <index> = (0..127)
>     <value> = BER encoded subidentifier
> 
> The first optional byte defines the new length of the OID (truncating
> or expanding by filling up with 0s). The truncation byte has the
> highest bit set in order to identify it. The remaining bytes are a
> sequence of index positions (0..127) followed by a subidentifier value
> which replaces the value at the index position. (The index position is
> the number of the subidentifier and not a position in the BER encoded
> OID value.) This scheme gives already reasonable savings although it
> is not the most efficient version (but very easy and straight-forward
> to compute).
> 
> More compression may be possible:
> 
> - allow subidentifier range substitutions ?

like the variation above? - yes. embedded ranges seems like
overkill. most duplication will be in the first series of sub-ids.

> - a shorthand for substitution in last <index> position ?
> - drop ASN.1/BER compatibility and use xx and <length> fields to hold
>   useful data ?

simplicity of implementation, re-use of existing ber libs get my vote.

-GSM

> 
> Comments?
> 
> /js
> 
> -- 
> Juergen Schoenwaelder      Technical University Braunschweig
> <[email-masked]>  Dept. Operating Systems & Computer Networks
> Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
> Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>
> 
> 
> 
> 

-- 
G.S. Marzot                        email: [email-masked]
Nortel Networks                    voice: (978)288-3990
600 Tech Park  M/S E65-60-101      pager: (800)409-6080 ([email-masked])
Billerica, MA  01821                 fax: (978)670-8145

Date: Tue, 11 Jan 2000 11:56:32 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: Network Management Research Group <[email-masked]>
Subject: [nmrg] delta oid compression
Sender: [email-masked]
Status: RO

Here is a more detailed example for what I have in mind. Lets assume
we make a "$s getbulk 0 11 SNMPv2-MIB::system" (in scotty notation).
This will return the following OIDs for ObjectName:

OID			BER [116 byte]		    COMP BER [57 byte]

1.3.6.1.2.1.1.1.0	06 08 2B06010201010100	    06 08 2B06010201010100
1.3.6.1.2.1.1.2.0	06 08 2B06010201010200	    xx 02 0802
1.3.6.1.2.1.1.3.0	06 08 2B06010201010300	    xx 02 0803
1.3.6.1.2.1.1.4.0	06 08 2B06010201010400      xx 02 0804
1.3.6.1.2.1.1.5.0	06 08 2B06010201010500      xx 02 0805
1.3.6.1.2.1.1.6.0	06 08 2B06010201010600      xx 02 0806
1.3.6.1.2.1.1.7.0	06 08 2B06010201010700      xx 02 0807
1.3.6.1.2.1.1.8.0	06 08 2B06010201010800	    xx 02 0808
1.3.6.1.2.1.1.9.1.2.1	06 0A 2B060102010109010201  xx 09 8B080909010A020B01
1.3.6.1.2.1.1.9.1.3.1   06 0A 2B060102010109010301  xx 02 0A03
1.3.6.1.2.1.1.9.1.4.1	06 0A 2B060102010109010401  xx 02 0A04

The right column shows a simple ASN.1/BER compatible way to encode
deltas. The value <xx> stands for a yet to be assigned tag value. In
fact, the encoding is scheme is valid ASN.1/BER based on the following
replacement definition for ObjectName:

  ObjectName ::= CHOICE {
      name OBJECT IDENTIFIER,
      comp DeltaIdentifier
  }

  DeltaIdentifier ::= [APPLICATION xx] IMPLICIT OCTET STRING

The content of the DeltaIdentifier has the following structure:

    [<trunc>]{<index><value>}+

    <trunc> = 0x80 + (0..127)
    <index> = (0..127)
    <value> = BER encoded subidentifier

The first optional byte defines the new length of the OID (truncating
or expanding by filling up with 0s). The truncation byte has the
highest bit set in order to identify it. The remaining bytes are a
sequence of index positions (0..127) followed by a subidentifier value
which replaces the value at the index position. (The index position is
the number of the subidentifier and not a position in the BER encoded
OID value.) This scheme gives already reasonable savings although it
is not the most efficient version (but very easy and straight-forward
to compute).

More compression may be possible:

- allow subidentifier range substitutions ?
- a shorthand for substitution in last <index> position ?
- drop ASN.1/BER compatibility and use xx and <length> fields to hold
  useful data ?

Comments?

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

Date: Tue, 11 Jan 2000 11:44:31 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] welcome
Sender: [email-masked]
Status: RO

>>>>> wjhardaker  writes:

wjhardaker> I hope you're planning on still implementing the general
wjhardaker> "compressed" PDU type and then simply making this one
wjhardaker> possible compression type?

wjhardaker> (don't get me wrong, it sounds like a great idea, but I'd
wjhardaker> rather see multiple compression types supported.  This
wjhardaker> would be just one of the choices.)

Yes, we plan to support multiple compression algorithms, similar to
what IPComp does (RFC 2393-2395). The tricky part is in fact how you
"negotiate" the compression algorithm you want to use. I think we know
how this can work efficiently in the command generator/command
responder case via autodiscovery. Dealing with a notification
originators is a bit harder...

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

From: [email-masked]
To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: [nmrg] welcome
Organization: U.C.Davis, Information Technology - D.C.A.S.
Date: 10 Jan 2000 11:26:04 -0800
Lines: 19
Sender: [email-masked]
Status: RO

>>>>> On Fri, 7 Jan 2000 18:00:51 +0100, Juergen Schoenwaelder <[email-masked]> said:

Juergen> What is needed to complete the bulk transfer work is to
Juergen> figure out a fast but less costly compression algorithm as
Juergen> deflate. A potential idea is to simply replace OID encodings
Juergen> by deltas to the previous OID iff the delta is smaller than
Juergen> the OID encoding would be.

I hope you're planning on still implementing the general "compressed" 
PDU type and then simply making this one possible compression type?

(don't get me wrong, it sounds like a great idea, but I'd rather see
multiple compression types supported.  This would be just one of the
choices.)

-- 
Wes Hardaker
Distributed Computing Analysis and Support
University of California at Davis

Date: Fri, 7 Jan 2000 22:53:06 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] welcome
Sender: [email-masked]
Status: RO

>>>>> Joe Marzot writes:

Joe> without breaking things...hmmm..what are the guidelines?

I forgot something: Since delta OIDs do not require to completely
restructure PDUs, one could try to see whether it is possible to
define a solution which is valid ASN.1 and has minimal impact on
current PDU encoding. I have some ideas in mind but currently no time
to work them out and to write them down. (And perhaps its better this
way because you or someone else might have a better idea than I
have. ;-)

Final comment: The solution must of course work for all legal OIDs.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

Date: Fri, 7 Jan 2000 22:47:22 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] welcome
Sender: [email-masked]
Status: RO

>>>>> David Harrington writes:

David> I suggest a different PDU format, where the PDU **header**
David> contains the base OID for all varbinds in the varbind list, and
David> **conceptually** the OIDs in each varbind are expanded before
David> they are sent to varbind processing.

A base OID (something like the AgentX encoding) has the important
drawback that it does not help to compress potentially large instance
identifier. It also leads to less optimal results if objects such as
sysUpTime are included in the request (since this reduces the prefix
to mib-2 in most cases).

The DISMAN MIBs are an example I am familiar. They are indexed by two
strings and the amount of redundant instance identifier fragments you
get is interesting. Of course, one could argue that this is bad MIB
design. But I expect that instance identifier tend to get more
complex, regardless of the owner string indexing we use in the DISMAN
MIBs. (Forwarding tables or VACM tables are other nice examples where
you really would like to do compression not only on a common prefix.)

Yes, decompression happens conceptually before you start processing
the PDU so there is no problem here with ordering etc.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

Date: Fri, 07 Jan 2000 15:09:30 -0500
From: David Harrington <[email-masked]>
Organization: Cabletron Systems Inc
To: [email-masked]
Subject: Re: [nmrg] welcome
Sender: [email-masked]
Status: RO

Content-Type: multipart/mixed;
 boundary="------------9C33875E01A01A59CBD4BD67"

This is a multi-part message in MIME format.
--------------9C33875E01A01A59CBD4BD67
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi,

I currently don't have the time to get into a protracted debate on this
issue, but wanted to point out that SNMP varbind processing is not
guaranteed to be in any special order. 

This is important for multi-threaded agents. If the OID in one varbind
is dependent on the OID value in another varbind, then you are
apparently presuming that one OID is recognized before the other, and
that either they are being processed by the same thread, or there is
knowledge-sharing of the base OID between threads. 

I suggest a different PDU format, where the PDU **header** contains the
base OID for all varbinds in the varbind list, and **conceptually** the
OIDs in each varbind are expanded before they are sent to varbind
processing.

dbh

Joe Marzot wrote:
> 
> Juergen Schoenwaelder <[email-masked]> writes:
> > An example: Suppose I do a getbulk with repetitions=7 on the system
> > group. I will get back the following OIDs: 1.3.6.1.2.1.1.1.0,
> > 1.3.6.1.2.1.1.2.0, 1.3.6.1.2.1.1.3.0, 1.3.6.1.2.1.1.4.0,
> > 1.3.6.1.2.1.1.5.0, 1.3.6.1.2.1.1.6.0, 1.3.6.1.2.1.1.7.0.  I am looking
> > for an encoding which ships only 1.3.6.1.2.1.1.1.0, 2@8, 3@8, 4@8,
> > 5@8, 6@8, 7@8, where x@n means replace subidentifier n with x in the
> > previous OID. I think this scheme is pretty nice since it allows to
> > compress complex instance identifier values, which we see more and
> > more as we move to string based indexes.
> >
> > Sure, it should be possible to replace multiple subidentifiers. The
> > question is how we encode 3@8 in a nice compact way without breaking
> > anything.
> 
> without breaking things...hmmm..what are the guidelines?
> 
> no new data types? no new pdus? no msg formats?
> 
> -GSM
> >
> > Another work item is to actually implement this in order to see on
> > real-world agents what the trade-offs are (code size, runtime
> > overhead, compression ratios, ...).
> >
> > Now I could go on with getsubtree PDU details, but I guess this is
> > already enough for a start. ;-)
> >
> > /js
> >
> >
> >
> >
> 
> --
> G.S. Marzot                        email: [email-masked]
> Nortel Networks                    voice: (978)288-3990
> 600 Tech Park  M/S E65-60-101      pager: (800)409-6080 ([email-masked])
> Billerica, MA  01821                 fax: (978)670-8145

-- 
David Harrington         Spectrum for Smart Networking
[email-masked]        Cabletron Systems Inc.
--------------9C33875E01A01A59CBD4BD67
Content-Type: text/x-vcard; charset=us-ascii;
 name="dbh.vcf"
Content-Transfer-Encoding: 7bit
Content-Description: Card for David Harrington
Content-Disposition: attachment;
 filename="dbh.vcf"

begin:vcard 
n:Harrington;David
x-mozilla-html:FALSE
org:Cabletron Systems Inc
adr:;;35 Industrial Way;Rochester;NH;03867;USA
version:2.1
email;internet:[email-masked]
tel;fax:603-337-7370
tel;work:603-337-7357
x-mozilla-cpt:;0
fn:Harrington, David
end:vcard

--------------9C33875E01A01A59CBD4BD67--

Date: Fri, 7 Jan 2000 22:35:24 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] welcome
Sender: [email-masked]
Status: RO

>>>>> Joe Marzot writes:

Joe> without breaking things...hmmm..what are the guidelines?

Joe> no new data types? no new pdus? no msg formats?

We introduce a new tagged PDU (which is basically an OCTET STRING) to
ship compressed PDUs. The receiving SNMP engine detects the compressed
PDU by looking at the tag and decompresses it. Once it is decompressed,
normal PDU processing continues. (Compression is really very close to
encryption. We actually discussed to do compression as a special
encryption algorithm, but we dropped the idea later on for several
reasons. Some of them may be documented in the meeting minutes.

This also answers a question raised by Dave Harrington about the fact
that the SNMP protocol allows for parallel processing of varbinds in
any order. We don't change that since we (at least conceptually)
decompress before PDU processing begins. (I will forward his email in
a minute - his email got blocked by our mailing handler.)

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked]
Subject: Re: [nmrg] welcome
From: Joe Marzot <[email-masked]>
Date: 07 Jan 2000 14:33:04 -0500
Lines: 39
Sender: [email-masked]
Status: RO

Juergen Schoenwaelder <[email-masked]> writes:
> An example: Suppose I do a getbulk with repetitions=7 on the system
> group. I will get back the following OIDs: 1.3.6.1.2.1.1.1.0,
> 1.3.6.1.2.1.1.2.0, 1.3.6.1.2.1.1.3.0, 1.3.6.1.2.1.1.4.0,
> 1.3.6.1.2.1.1.5.0, 1.3.6.1.2.1.1.6.0, 1.3.6.1.2.1.1.7.0.  I am looking
> for an encoding which ships only 1.3.6.1.2.1.1.1.0, 2@8, 3@8, 4@8,
> 5@8, 6@8, 7@8, where x@n means replace subidentifier n with x in the
> previous OID. I think this scheme is pretty nice since it allows to
> compress complex instance identifier values, which we see more and
> more as we move to string based indexes.
> 
> Sure, it should be possible to replace multiple subidentifiers. The
> question is how we encode 3@8 in a nice compact way without breaking
> anything.

without breaking things...hmmm..what are the guidelines?

no new data types? no new pdus? no msg formats?

-GSM
> 
> Another work item is to actually implement this in order to see on
> real-world agents what the trade-offs are (code size, runtime
> overhead, compression ratios, ...).
> 
> Now I could go on with getsubtree PDU details, but I guess this is
> already enough for a start. ;-)
> 
> /js
> 
> 
> 
> 

-- 
G.S. Marzot                        email: [email-masked]
Nortel Networks                    voice: (978)288-3990
600 Tech Park  M/S E65-60-101      pager: (800)409-6080 ([email-masked])
Billerica, MA  01821                 fax: (978)670-8145

Date: Fri, 7 Jan 2000 18:00:51 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] welcome
Sender: [email-masked]
Status: RO

>>>>> Joe Marzot writes:

Joe> what things need the most attention? Is there a FAQ? info on doc
Joe> locations etc.

Our documentation is behind our discussions. I am to blame on this but
I promise to get back to full speed once some other IETF commitments
are from my table.

Reading the minutes <URL:http://www.ibr.cs.tu-bs.de/projects/nmrg/>
may be helpful to understand where we are and how we got there.

What is needed to complete the bulk transfer work is to figure out a
fast but less costly compression algorithm as deflate. A potential
idea is to simply replace OID encodings by deltas to the previous OID
iff the delta is smaller than the OID encoding would be. This should
be simple to compute and it might be pretty effective on large tables
(we need to figure out how effective it is). It also guarantees that
the PDU will never increase. The key here is to find a good encoding
scheme for OID deltas.

An example: Suppose I do a getbulk with repetitions=7 on the system
group. I will get back the following OIDs: 1.3.6.1.2.1.1.1.0,
1.3.6.1.2.1.1.2.0, 1.3.6.1.2.1.1.3.0, 1.3.6.1.2.1.1.4.0,
1.3.6.1.2.1.1.5.0, 1.3.6.1.2.1.1.6.0, 1.3.6.1.2.1.1.7.0.  I am looking
for an encoding which ships only 1.3.6.1.2.1.1.1.0, 2@8, 3@8, 4@8,
5@8, 6@8, 7@8, where x@n means replace subidentifier n with x in the
previous OID. I think this scheme is pretty nice since it allows to
compress complex instance identifier values, which we see more and
more as we move to string based indexes.

Sure, it should be possible to replace multiple subidentifiers. The
question is how we encode 3@8 in a nice compact way without breaking
anything.

Another work item is to actually implement this in order to see on
real-world agents what the trade-offs are (code size, runtime
overhead, compression ratios, ...).

Now I could go on with getsubtree PDU details, but I guess this is
already enough for a start. ;-)

/js

To: Juergen Schoenwaelder <[email-masked]>
Cc: [email-masked]
Subject: Re: [nmrg] welcome
From: Joe Marzot <[email-masked]>
Date: 07 Jan 2000 11:22:13 -0500
Lines: 52
Sender: [email-masked]
Status: RO

Juergen Schoenwaelder <[email-masked]> writes:

> >>>>> Joe Marzot writes:
> 
> Joe> One basic observation is that when retrieving multiple columns
> Joe> from the same table(a common operation), the table prefix is
> Joe> repeated for each column in both request and response. It is not
> Joe> hard to imagine a PDU structure that would specify the prefix and
> Joe> then just which columns. Perhaps "get sub-tree" already adresses
> Joe> this...
> 
> Of course, in some other cases one would prefer a compression scheme
> which is less costly (means more efficient to compute). You can for
> example define a compression algorithm which takes a BER encoded PDU
> and simply does differential OID encodings (that is only encoded to
> difference from the previous OID). This will be very fast to compute.
> That is why the SNMP payload compression proposal supports different
> compression algorithms.

excellent - this approach looks flexible and will satisfy both types of
requirements (general compression, high-speed compression).

> 
> Joe> look forward to learning more on what's been done and how to
> Joe> help.
> 
> And I am looking forward to your input. I am sure it will help us
> to move things forward.

what things need the most attention? Is there a FAQ? info on doc
locations etc.

Thanks, GSM

> 
> /js
> 
> -- 
> Juergen Schoenwaelder      Technical University Braunschweig
> <[email-masked]>  Dept. Operating Systems & Computer Networks
> Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
> Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>
> 
> 
> 
> 

-- 
G.S. Marzot                        email: [email-masked]
Nortel Networks                    voice: (978)288-3990
600 Tech Park  M/S E65-60-101      pager: (800)409-6080 ([email-masked])
Billerica, MA  01821                 fax: (978)670-8145

Date: Fri, 7 Jan 2000 09:30:49 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: [email-masked]
CC: [email-masked]
Subject: Re: [nmrg] welcome
Sender: [email-masked]
Status: RO

>>>>> Joe Marzot writes:

Joe> One basic observation is that when retrieving multiple columns
Joe> from the same table(a common operation), the table prefix is
Joe> repeated for each column in both request and response. It is not
Joe> hard to imagine a PDU structure that would specify the prefix and
Joe> then just which columns. Perhaps "get sub-tree" already adresses
Joe> this...

No. Our thinking was that compression is not necessarily PDU specific.
In other words, it would be nice to have compression for all PDUs and
not just the new get-subtree PDU.

Joe> Also I wonder how practical standard compression is in addressing
Joe> PDU size issues. As we see very large networks O(10000) both PDU
Joe> size and processing delay can matter.

Yes. Using deflate compression is costly but it has benefits in cases
where you also have highly redundant data in the value part of
varbind. For example, when retrieving accounting records, compression
of the accounting data itself is very useful, especially if you have
larger PDU sizes (which is not an issue if you run SNMP over TCP).

Of course, in some other cases one would prefer a compression scheme
which is less costly (means more efficient to compute). You can for
example define a compression algorithm which takes a BER encoded PDU
and simply does differential OID encodings (that is only encoded to
difference from the previous OID). This will be very fast to compute.
That is why the SNMP payload compression proposal supports different
compression algorithms.

Joe> look forward to learning more on what's been done and how to
Joe> help.

And I am looking forward to your input. I am sure it will help us
to move things forward.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>

To: Juergen Schoenwaelder <[email-masked]>
Cc: Network Management Research Group <[email-masked]>
Subject: Re: [nmrg] welcome
From: Joe Marzot <[email-masked]>
Date: 06 Jan 2000 18:15:26 -0500
Lines: 51
Sender: [email-masked]
Status: RO

Juergen Schoenwaelder <[email-masked]> writes:

> Welcome everybody in 2000. I hope you had a good bug-free start.
> 
> (At least the software we use to generate our Web pages seems to have
> a Y2K problem - so we now have the year 19100 on our page :-)
> 
> I want to announce that Joe Marzot <[email-masked]> has
> joined our mailing list. I guess he has some input on bulk retrieval.

Thank you for including me on the list.

First I would like to read what's been written for "get sub-tree" and
check out what's been discussed so far before adding too much.

One basic observation is that when retrieving multiple columns from the
same table(a common operation), the table prefix is repeated for each
column in both request and response. It is not hard to imagine a PDU
structure that would specify the prefix and then just which
columns. Perhaps "get sub-tree" already adresses this...

Also I wonder how practical standard compression is in addressing PDU
size issues. As we see very large networks O(10000) both PDU size and
processing delay can matter.

look forward to learning more on what's been done and how to help.

-GSM

> 
> I also want to make plans for the next NMRG meeting. If you have any
> suggestions where and when we could meet, please post to this list or
> send email to me if you think that is more appropriate.
> 
> /js
> 
> -- 
> Juergen Schoenwaelder      Technical University Braunschweig
> <[email-masked]>  Dept. Operating Systems & Computer Networks
> Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
> Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>
> 
> 
> 
> 

-- 
G.S. Marzot                        email: [email-masked]
Nortel Networks                    voice: (978)288-3990
600 Tech Park  M/S E65-60-101      pager: (800)409-6080 ([email-masked])
Billerica, MA  01821                 fax: (978)670-8145

Date: Thu, 6 Jan 2000 17:55:18 +0100
From: Juergen Schoenwaelder <[email-masked]>
To: Network Management Research Group <[email-masked]>
Subject: [nmrg] welcome
Sender: [email-masked]
Status: RO

Welcome everybody in 2000. I hope you had a good bug-free start.

(At least the software we use to generate our Web pages seems to have
a Y2K problem - so we now have the year 19100 on our page :-)

I want to announce that Joe Marzot <[email-masked]> has
joined our mailing list. I guess he has some input on bulk retrieval.

I also want to make plans for the next NMRG meeting. If you have any
suggestions where and when we could meet, please post to this list or
send email to me if you think that is more appropriate.

/js

-- 
Juergen Schoenwaelder      Technical University Braunschweig
<[email-masked]>  Dept. Operating Systems & Computer Networks
Phone: +49 531 391 3289    Bueltenweg 74/75, 38106 Braunschweig, Germany
Fax:   +49 531 391 5936    <URL:http://www.ibr.cs.tu-bs.de/~schoenw/>