Subject: Re: [Secdispatch] DARE Message and Container formats

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 14 Jun 2018 10:39:34 -0400

After first proposing this, I have become a GDPR stuckee.

One of my prime concerns is the habit that Internet servers of all types
have of creating log files. In many cases log files are generated by
default and in locations that are not necessarily obvious to the operators.
Since it is understood that IP addresses MAY be PII, this is a serious
concern.

And it gets worse: Simply deleting the logs may not help because the logs
may be required to audit compliance with the privacy undertakings made.

The simplest approach is to 'encrypt the log files'. But neither S/MIME nor
OpenPGP is a suitable format for incremental encryption. These data formats
are designed to support encryption of individual records in an append only
log format without losing the capabilities that we demand of logs such as
the ability to read the last 100 records of a million record log.

On Thu, May 24, 2018 at 9:44 PM, Phillip Hallam-Baker <[email-masked]
> wrote:

> ​I have been developing a set of tools and data formats to support
> encryption and authentication of sequenced data. These build on the work in
> JSON and JOSE ​but make certain changes for reasons of efficiency and/or
> simplicity.
>
> The text versions of the documents are to be found here:
>
> https://datatracker.ietf.org/doc/draft-hallambaker-jsonbcd/
> https://datatracker.ietf.org/doc/draft-hallambaker-dare-message/
> https://datatracker.ietf.org/doc/draft-hallambaker-dare-container/
>
>
> The HTML versions have diagrams (converted into data URLs so they are
> still single file objects)
>
> http://mathmesh.com/Documents/draft-hallambaker-jsonbcd.html
> http://mathmesh.com/Documents/draft-hallambaker-dare-message.html
> http://mathmesh.com/Documents/draft-hallambaker-dare-container.html
>
>
> The near term application for this work is to encrypt log files for GDPR
> compliance. An open source reference implementation is available on GitHub.
>
> https://github.com/hallambaker/Mathematical-Mesh
>
>
> The chief difference between this work and earlier work is that the
> formats are designed to support efficient encryption of incremental updates
> to an append only log. So if a server restarts, it can perform a new key
> agreement and keep on writing to the old log.
>
> Use of split decryption key (recryption) functionality is supported but
> not required. So in an enterprise environment log files might be encrypted
> to the site encryption key and individual system administrators granted
> access to specific log files on an as-needed basis by the recryption
> service.
>
> Individual messages posted to the container consist of a body and a series
> of optional headers which may also be encrypted. This feature allows a
> message subject line to be encrypted separately from the message body that
> it applies to.
>
>
> A key innovation in the container format is the use of bidirectional
> frames which allow the file to be read with equal efficiency in either the
> forward or reverse direction. The container may optionally be indexed
> enabling rapid random access.
>
> The container format also provides means for digest authentication
> including static, chained and Merkle tree modes [The astute will notice
> that this provides a functionality similar to one that rhymes with
> clock-train]. Thus the format may be used for applications such as an
> archive format where one signature can be used to validate all the files in
> the archive.
>
> The message format is designed for both signature and encryption. The
> message body is signed first and then encryption is applied to both the
> message body and the signature values. This approach demonstrates that the
> signer had actual knowledge of the message content that was signed,
> defeating certain message substitution attacks.
>
>
> I would like to seek AD Sponsorship of this work.
>