Subject: Re: [saag] encrypted files with UTF-8/16 passwords

From: Nikos Mavrogiannopoulos <[email-masked]>
Date: Mon, 3 Apr 2017 11:21:54 +0200

I have put an initial draft suggesting the utilization of RFC7613 for
UTF-8 password normalization. I'd appreciate comments on the approach
and on the usage of RFC7613 in general.

https://gitlab.com/nmav/ietf-pkcs5

PS. I've started a discussion on the suitability of RFC7613 for
passwords at the precis list:
https://mailarchive.ietf.org/arch/msg/precis/WRFASSjZzb2ddqZJc5bkOlslOLE

On Fri, Mar 24, 2017 at 3:53 PM, Moriarty, Kathleen
<[email-masked]> wrote:
> Hi Nikos,
>
> They are just informational because they were contributed as existing standards.  Change control has been handed over to the IETF, so an update could happen to make them standards track.  Or you could start an updated draft to add what you need and we'll figure out if it has to stay informational or not.
>
> Thanks,
> Kathleen
>
> Sent from my iPhone
>
>> On Mar 24, 2017, at 4:08 AM, Nikos Mavrogiannopoulos <[email-masked]> wrote:
>>
>> Hi,
>> PKCS#8 (rfc8018) and PKCS#12 (rfc7292) can be used to encrypt keys
>> and certificates with a password. In the first case, PKCS#8 utilizes
>> PKCS#5 for converting a password to an encryption key, and PKCS#5
>> requires a password to be in UTF-8. For PKCS#12, a password is input
>> in UTF-16 format (mentioned as BMPString in the document) in some
>> preset schemes, but uses UTF-8 for newer schemes like AES via PKCS#5.
>>
>> However, UTF-8 (and UTF-16) are ambiguous. The same string may have
>> multiple representations, and for that, there are some guidelines in
>> RFC7613 to prepare a unicode string for a password, but they do not
>> update either of these documents.
>>
>> Given that these are informational RFCs, which would be the proper
>> method to propose an update on them based on these lines and requiring
>> RFC7613 processing for passwords entered in UTF-8?
>>
>> regards,
>> Nikos

---

From: 大岩寛 <[email-masked]>
Date: Mon, 3 Apr 2017 14:40:19 +0000

Dear Nikos,

It's interesting.  A few quick comments from me:

1) Because RFC 7613 specifies multiple profiles,
   the OpaqueString profile should be explicitly cited, like:
  "... MUST be prepared using OpaqueString profile (Section 4.2) of
  [RFC7613]."

2) Exception rule for the empty string can be more verbose.
   e.g. "As an exception to the OpaqueString profile, an empty password 
   string MAY be used if and only if the input is also empty;
   an empty string generated from any non-empty input MUST NOT be used."

3) For PKCS #12, the use of the UTF-16 (or UCS-2) encoding is contradictory to
   RFC 7613, so it might be good to explicitly say 
   "use big-endian {UTF-16 or UCS-2}, regardless of Section 4.2.1 of [RFC7613]".

4) Also, the handling of the non-BMP positions seems better if more explicit:
   Choices seems to be "{MUST or MAY} be encoded using UTF-16", 
   or "SHOULD NOT accept any codepoints beyond BMP".
   My preference is "MAY" (although it is a slight extension to original PKCS#12).

-- 
Yutaka OIWA, Ph.D.       Leader, Cyber Physical Architecture Research Group
                                  Information Technology Research Institute
    National Institute of Advanced Industrial Science and Technology (AIST)
                      Mail addresses: <[email-masked]>, <[email-masked]>
OpenPGP: id[440546B5] fp[7C9F 723A 7559 3246 229D  3139 8677 9BD2 4405 46B5]

> -----Original Message-----
> From: saag [mailto:[email-masked]] On Behalf Of Nikos
> Mavrogiannopoulos
> Sent: Monday, April 3, 2017 6:22 PM
> To: Moriarty, Kathleen <[email-masked]>
> Cc: [email-masked]; IETF SAAG <[email-masked]>; [email-masked]
> Subject: Re: [saag] encrypted files with UTF-8/16 passwords
> 
> UTF-8 password normalization. I'd appreciate comments on the approach and on
> the usage of RFC7613 in general.
> 
> PS. I've started a discussion on the suitability of RFC7613 for passwords at
> the precis list:
> > They are just informational because they were contributed as existing
> standards.  Change control has been handed over to the IETF, so an update could
> happen to make them standards track.  Or you could start an updated draft to
> add what you need and we'll figure out if it has to stay informational or not.
> >
> > Thanks,
> > Kathleen
> >
> > Sent from my iPhone
> >
> _______________________________________________
> saag mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/saag

---

From: David Woodhouse <[email-masked]>
Date: Mon, 03 Apr 2017 15:54:08 +0100

> https://gitlab.com/nmav/ietf-pkcs5

The reference to RFC8018 is missing.

Using the methods (especially Normalization Form C) from RFC7613 makes
complete sense. RFC7613 only talks of UTF-8 but we actually need the
equivalent for UTF-16, for at least some forms of PKCS#12.

I'm a bit lost when you talk of using UTF-8 passwords in PKCS#12. Is
that really what people do, dependent on the encryption method in use?
I thought PKCS#12 was *always* BMPString?

I already heckled on IRC your first email in this thread, in which you
stated that "PKCS#5 requires a password to be in UTF-8". It doesn't; it
merely identifies it as a possibility (penultimate paragraph of §3):
   Throughout this document, a password is considered to be an octet
   string of arbitrary length whose interpretation as a text string is
   unspecified.  In the interest of interoperability, however, it is
   recommended that applications follow some common text encoding rules.
   ASCII and UTF-8 [RFC3629] are two possibilities.  (ASCII is a subset
   of UTF-8.)

Given the interoperability concerns with existing files, and the fact
that *trying* a password is free, it does make sense to list a number
of options for attempting to decrypt a file given a password in some
local charset (which might be a legacy non-UTF8 one). I'd attempted
that in
http://david.woodhou.se/draft-woodhouse-cert-best-practice.html#rfc.section.7

Of course, for *creating* such files we can be more prescriptive...

---

From: Nikos Mavrogiannopoulos <[email-masked]>
Date: Tue, 4 Apr 2017 09:01:07 +0200

> It's interesting.  A few quick comments from me:

Thank you, very nice comments. Replying inline too.

>   "... MUST be prepared using OpaqueString profile (Section 4.2) of

Done.

>    an empty string generated from any non-empty input MUST NOT be used."

Done.

>    "use big-endian {UTF-16 or UCS-2}, regardless of Section 4.2.1 of [RFC7613]".

I also like that, however is that possible? Can we override the ASN.1
definition of BMPString and allow UTF-16 to be stored in that field?

regards,
Nikos

---

From: Nikos Mavrogiannopoulos <[email-masked]>
Date: Tue, 4 Apr 2017 09:06:11 +0200

> I thought PKCS#12 was *always* BMPString?

That's partially true. The MAC in PKCS#12 *always* uses the password
as a BMPString. However, the encrypted sections (keys, certificates)
do not have to be. In fact if you intend to use the AES-128 algorithm
in PKCS#12, the only way is to utilize the PKCS#5 encryption method
which uses/suggests UTF-8 (or ASCII).

> Of course, for *creating* such files we can be more prescriptive...

Should/may I copy this section as an informative section for
applications wanting backwards compatibility?

regards,
Nikos

---

From: David Woodhouse <[email-masked]>
Date: Tue, 04 Apr 2017 08:17:41 +0100

> which uses/suggests UTF-8 (or ASCII).

PKCS#5 mentions UTF-8 as a possibility, sure. But it precedes that by
recommending "that applications follow some common text encoding
rules."

The idea of using PKCS#5 within the context of PKCS#12 where a common
text encoding (BMPString) *is* actually specified for some purposes,
and deciding *not* to use the same text encoding as the PKCS#12
container... makes me want to go and smash my head against the wall
until the pain stops.

Do applications *really* use a different encoding for different parts
of the data, like that? Really? Unless it is a well-established
practice, I do not think we should encourage it.

My own best practice document hadn't even listed it as something which
should be *tried* for decoding a file.

> applications wanting backwards compatibility?

Absolutely. It may even make sense to remove that entire section from
my document and just defer to yours, depending on how much you want to
include.

---

From: Nikos Mavrogiannopoulos <[email-masked]>
Date: Wed, 5 Apr 2017 11:12:47 +0200

> recommending "that applications follow some common text encoding

Hopefully the draft I'm proposing clarifies this part by mandating
UTF-8 for all PKCS#5 mechanisms.

> practice, I do not think we should encourage it.

All existing software treats those files like that as far as I know.
PKCS#12 definitions apply to the OIDs/algorithms defined within the
document only, not to other mechanisms.

> should be *tried* for decoding a file.

I do not think there is an ambiguity there given that all software I
have tested is compatible.

> my document and just defer to yours, depending on how much you want to

Done. I have not included the second part on openssl 1.1.0. I believe
assuming input is in UTF-8 is a good practice if it is documented.
Character sets are not something we should be concerned today.

regards,
Nikos

---

From: David Woodhouse <[email-masked]>
Date: Wed, 05 Apr 2017 11:54:06 +0200

> document only, not to other mechanisms.

Hm, what about OpenSSL? OpenSSL doesn't do any real charset conversion,
which is why they've moved from one bug with PKCS#12 in 1.0.x... to a
*different* bug in 1.1.0 instead of actually fixing it.

And yet the above would *require* a proper conversion between BMPString
and UTF-8...