Subject: [Enum] In character

From: "Conroy, Lawrence (SMTP)" <[email-masked]>
Date: Mon, 3 May 2004 17:45:07 +0100

Hi Folks,
  To prove that I have not disappeared and *am* working on the BCP, 
herewith some text
on the choice of RegExp field delimiter character. I expect that I've 
missed something,
but if I read another document on character sets my eyes will fall out.

In short, as '!' is a valid character inside a URI and thus may appear 
in the
replacement (sed-style substitution expression) sub-field, I'm not sure 
if we
should use this.

My current "least bad" choice seems to be '"' (i.e. 0x0022). I had 
thought that this can
appear in a mailto: URI, but according to RFC2396 (which specifies URI 
syntax) it can't.

Comments appreciated.

   Lawrence

--------------------------------------------------------
RegExp field delimiter choice:

There are a number of interacting details in the operation of the 
RegExp field of a NAPTR that define the choice of delimiter, over and 
above the plain specification in RFC3402 ss 3.2.

----

As the match sub-field will be a Posix Regular Expression (PRE), the 
delimiter should not be a character that could reasonablyu appear 
within a PRE that operates on the ENUM Application Unique String.

These characters include the PRE-reserved set:
   |()*+?{}0123456789,.^$\-=<>:
plus the classes defined in ctype:
   alnum   digit       punct
   alpha   graph       space
   blank   lower       upper
   cntrl   print       xdigit
In short, this is every possible character.

However, there are some practical restrictions placed on semantically 
correct PRE within an E2U (ENUM) NAPTR.

The Application Unique String (against which the PRE is matched) 
consists of an .E.164 number excluding any non-digit characters, 
preceded by the character '+', so, in the particular case of the E2U 
DDDS application, one would expect only characters in the PRE-reserved 
set to appear in the RegExp match sub-field.

Thus, it would seem that the RegExp sub-field delimiter SHOULD NOT be 
in the PRE-reserved set of characters.

----

Where used to produce a URI (as in the 'E2U' DDDS application defined 
for ENUM), the replacement sub-field may contain a URI.
To avoid ambiguity, the RegExp sub-field delimiter will need to be 
different from a valid character that may appear in a URI.

Thus the RegExp field delimiter SHOULD be in the list of characters 
excluded from a URI (in section 2.4.3 of RFC2396) - these are defined 
there as control, space, delims, and unwise sets.

Conversely, a URI may have a fragment indicator, so that the character 
'#', whilst defined in the URI delims set, should not be used. 
Similarly, the character '%' may appear in a URI so that even though it 
is defined in the URI delims set, it should not be used.

----
RFC3402 ss 3.2 specifies that the delimiter may be any octet not in 
POS-DIGIT or flags sets, further specifies that POS-DIGIT set is the 
characters 0-9, and that the flags set consists of the character 'i'.

In addition, the delimiter should NOT be the SED-style escape character 
'\', as this may be present and will be processed prior to generation 
of the URI.

----

Finally, whilst RFC3403 (defining the DDDS DNS database and so the 
NAPTR syntax) and RFC3761 (defining ENUM) both specifiy that UTF-8 is 
used as the character set, it is convenient to use a single-octet 
character for the delimiter (if only to simplify implementation of 
initial field parsing). With UTF-8, this means that the character will 
have to be in the US-ASCII equivalent coding space (i.e. a character in 
the range 0x0000 - 0x007F).

Given that many implementations of ENUM clients will use 
null-terminated string processing functions, the choice of 0x0000 as 
the delimiter unnecessarily complicates processing; thus for 
convenience the NUL character (\000) SHOULD NOT be used as the 
sub-field delimiter.

----

Combining these restrictions, as a general rule the delimiter should be:
o	NOT 0x0000
o	in US-ASCII (single octet) range
o	NOT PRE-reserved (which subsumes that SED-reserved range)
   PRE-reserved = |()*+?{}0123456789,.^$\-=[]<>:
o	in URI (control, space, delims, and unwise) sets
   control     = <US-ASCII coded characters 00-1F and 7F hexadecimal>
   space       = <US-ASCII coded character 20 hexadecimal>
   delims      = "<" | ">" | <">
   unwise      = "{" | "}" | "|" | "\" | "^" | "[" | "]" | "`"

Note that the delimiter often used in DDDS/NAPTR/ENUM examples ('!') is 
valid in URIs and so, strictly, should not be used; it is in the mark 
set and so part of the unreserved character set that may appear within 
a URI.

Of these, one of the few reasonable characters that do not appear to 
clash with PRE, URI, or sed-style substitutions is the '"' character.
This is inconvenient, as it is reserved in many zone file processing 
tools. However, it should not clash with the other potential elements 
in a RegExp field, and so it is a valid choice.

_______________________________________________
enum mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/enum

---

From: [email-masked]
Date: Tue, 04 May 2004 03:30:54 +0900 (JST)

Hi, Conroy-san, enum-wg folks,

RFC3402 page 7, 3.2 Substitution Expression Syntax contains:

	Since escaped occurrences of the delimiter character will be
	interpreted as occurrences of that character, digits MUST NOT
	be used as delimiters.

So, My understanding is that a URI which contains '!' character may
appear in the replacement sub-field as '\!' when '!' is a delimiter
character.

# escape is easy to parse.

For example,

  URI = "http://member.wide.ad.jp/~fujiwara/#!"
 then
  REGEXP field may be "!^.*$!http://member.wide.ad.jp/~fujiwara/#\!!"

I think delimiter character should be limited to '!' in ENUM.

'"' should not be used as a delimiter character.  Because many server
softwares and programing languages use '"' as a special character.
RFC3403, RFC3761 use '"' as a string delimiter character.

--
Fujiwara, Kazunori	JPRS

> From: "Conroy, Lawrence (SMTP)" <[email-masked]>
> Hi Folks,
> -- 
> Visit our website at www.roke.co.uk
> 
> Registered Office: Roke Manor Research Ltd, Siemens House, Oldbury, Bracknell,
> Berkshire. RG12 8FZ
> 
> The information contained in this e-mail and any attachments is confidential to
> Roke Manor Research Ltd and must not be passed to any third party without
> permission. This communication is for information only and shall not create or
> change any contractual relationship.
>

---

From: "Conroy, Lawrence (SMTP)" <[email-masked]>
Date: Mon, 3 May 2004 20:11:55 +0100

Hi Fujiwara-san, folks,
I'm happy with this (we use '!' as well :).
My only concern is whether or not use of '!' as a delimiter should be 
mandatory -
my feeling is that it SHOULD be used, not that it MUST be used.

I'd appreciate comments from anyone else "out there" before we commit 
to this in the
BCP, so... three increasingly specific questions (with my notes in 
parentheses):::

(i)   If anyone objects to the requirement that the server/zone
       generator MUST use a SED-style escape if a delimiter character
       appears in the body of the replacement sub-field,
   PLEASE SPEAK NOW

(this one suggests that there may be a conflict, but such a conflict is 
dealt
  with by SED escape).

(ii)  If anyone objects to the statement that a NAPTR RegExp field MUST
       NOT use a character not in the US-ASCII equivalent range of UTF-8,
   PLEASE SPEAK NOW

(!!!note that I propose that this is MANDATORY!!! - I think that this is
  a "needed clarification")

(outside the URI text in the replacement sub-field, the only place I can
  see that a multi-octet character would be used is in the sub-field 
delimiter)

(iii) If anyone has an objection to RECOMMENDing that the '!' character
       SHOULD BE used as the RegExp delimiter,
   PLEASE SPEAK NOW

(note that a client should still be able to process a RegExp that does 
NOT use
  the '!' (0x0021) character as a delimiter - it's recommended, not 
mandatory)

all the best,
   Lawrence

On 3 May 2004, at 7:30 pm, [email-masked] wrote:

>> in the replacement (sed-style substitution expression) sub-field, I'm 
>> not sure
>> if we should use this.
>>
>> thought that this can appear in a mailto: URI, but according to 
>> RFC2396 (which specifies URI
>> syntax) it can't.
>>
>> Comments appreciated.
>>
>>    Lawrence
>> <snip long, involved description of potential conflicts in character 
>> sets for the RegExp content>

---

From: Patrik Fältström <[email-masked]>
Date: Tue, 4 May 2004 08:35:51 +0200

As we decided in Seoul, when you, Lawrence, have an updated version of 
your document, myself and Michael will go through it and come with 
compiled comments.

     paf

On May 3, 2004, at 21:11, Conroy, Lawrence (SMTP) wrote:

> (this one suggests that there may be a conflict, but such a conflict 
> is dealt
>  with by SED escape).
>
>>> on the choice of RegExp field delimiter character. I expect that 
>>> I've missed something,
>>> in the replacement (sed-style substitution expression) sub-field, 
>>> I'm not sure
>>> if we should use this.
>>>