Subject: [saag] How to: SFTP toward RFC?

From: denis bider <[email-masked]>
Date: Fri, 16 Feb 2018 01:11:22 -0600

Hey everyone,

I originally asked this on the Curdle mailing list, where we made some
progress updating SSH. I'm seeking advice about the SFTP situation, and I
was pointed to SECDISPATCH, and hence SAAG where I understand this type of
thing would be discussed.

SFTP is a widely used internet protocol that exists in two main versions:
SFTP v3 with OpenSSH extensions, and SFTP v6. SFTP v4 is also widely
supported, but most v4 implementations now seem to include v6.

A schism happened over a decade ago when OpenSSH refused to adopt SFTP
enhancements to support non-Unix platforms. They argued version 3 has
everything it needs, which it does on Unix. But from a non-Unix
perspective, many others thought it's not good enough, and better support
for other platforms is needed. Because of this, there wasn't consensus, and
SFTP did not become an RFC.

Now, the result is that SFTP is a major internet protocol, and anyone who
wants to implement it needs to follow this for SFTP v3:

https://tools.ietf.org/html/draft-ietf-secsh-filexfer-02

... and this for v6:

https://tools.ietf.org/html/draft-ietf-secsh-filexfer-13

https://tools.ietf.org/html/draft-galb-filexfer-extensions-00

In addition, a number of details are unclear and not documented fully. For
example, OpenSSH encodes some packets differently than SFTP v3 prescribes,
and implementations of check-file extensions are not compatible in practice
due to different implementation restrictions on their usage.

I would think it worthwhile for SFTP to receive better treatment, and I
think practical use justifies documenting both version 3 and version 6.
Since both are widely used, I think it would be reasonable for this to be
Standards track, not Informational.

However, if I write a draft - or perhaps two drafts - I'm not sure who to
turn to. Does documenting existing SFTP practice warrant creating a new
working group? Might this be pursued under the SECDISPATCH or SAAG purview?
Or perhaps another process?

Additionally, I wonder - is this work that can be done without traveling to
London, etc in person? I understand that's how IETF still does things for
the time being, but from my perspective this greatly reduces the number of
people who would be interested in participating in an otherwise useful
endeavor (perhaps to zero).

denis

---

From: Eric Rescorla <[email-masked]>
Date: Fri, 16 Feb 2018 08:41:47 -0800

Hi Denis,

I'm trying to get a handle on how much technical work there is to do here
as opposed to just documentation work. Usually, the IETF expects to have
change control of specifications is standardizes, but that seems perhaps
not wanted here?

-Ekr

On Thu, Feb 15, 2018 at 11:11 PM, denis bider <[email-masked]>
wrote:

> Additionally, I wonder - is this work that can be done without traveling
> to London, etc in person? I understand that's how IETF still does things
> for the time being, but from my perspective this greatly reduces the number
> of people who would be interested in participating in an otherwise useful
> _______________________________________________
> saag mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/saag
>
>

---

From: Theodore Ts'o <[email-masked]>
Date: Fri, 16 Feb 2018 12:01:30 -0500

> supported, but most v4 implementations now seem to include v6.

Is this list of SFTP implementations accurate?  If not, could you
point at more comprehensive list?  Looking at the list I was able to
find, it doesn't appear to support hte claim that SFTP v4 or v6 are
widely deployed.

	https://www.greenend.org.uk/rjk/sftp/sftpimpls.html

Cheers,

					- Ted

---

From: denis bider <[email-masked]>
Date: Tue, 20 Feb 2018 00:06:45 -0600

This list is very incomplete. I see no mention of a variety of commercial
implementations. Tectia (the original) is not even mentioned. There's many
more vendors of SFTP clients, servers and libraries than what's listed here.

Off the top of my mind, SFTP v6 implementations that are not listed include
Bitvise (our own, SFTP 3-6) and VanDyke (SFTP 3-6).

I checked Wikipedia, that's not very useful either, because random people
keep coming and deleting other people's implementations from the articles.

In practical use, SFTP v3 is of course the most common because that's
what's in OpenSSH.

I've heard feedback indicating that the open source people (OpenSSH, PuTTY,
dropbear, etc) are contemplating banding together and making further
extensions to SFTP v3 while ignoring SFTP v6. This is sad. It seems the
schism continues. Group X has needs A, and does not care about addressing
the needs of group Y with needs B. :-/

---

From: denis bider <[email-masked]>
Date: Tue, 20 Feb 2018 00:12:27 -0600

I can confirm that if this work were to be done, it would need to begin by
documenting the existing. In this sense, there is not much leeway in what
to express, but there is leeway in how to express it.

On a protocol level, there would be leeway in new extensions, as well as
existing features that currently don't work well between implementations.
In particular, I think "check-file" extensions have turned out to be
under-specified and further standardization is needed.

On Fri, Feb 16, 2018 at 10:41 AM, Eric Rescorla <[email-masked]> wrote:

>> In addition, a number of details are unclear and not documented fully.
>> For example, OpenSSH encodes some packets differently than SFTP v3
>> prescribes, and implementations of check-file extensions are not compatible
>> in practice due to different implementation restrictions on their usage.
>>

---

From: denis bider <[email-masked]>
Date: Tue, 20 Feb 2018 00:48:47 -0600

Adding to this, it seems clear there continue to be two camps that want to
go in different extension directions while maintaining a compatible core.
We could layer these into:

- "SFTP lite". This is supported by the open source community that wants to
keep things simple, not implement support for platforms they don't care
about, not implement features they don't need. They don't care and don't
want support for Windows ACLs, file attributes, file sharing modes, native
text files on platforms with different text/binary representations. This
community wants to work on more extensions for SFTP v3, but without adding
v4+ features.

- "SFTP full". This is supported by a mixture of open source and commercial
implementations that want to provide features sought by users. If the user
is connecting to a Windows server, these implementations allow manipulating
ACLs, setting file attributes, having control over file sharing modes. When
a platform has distinct text/binary representations, these implementations
allow transferring textual files in a way that's convenient for the user.
This community will want further work done on SFTP v6, but is also likely
to incorporate new work on "SFTP lite".

To the extent these versions develop orthogonally, I think it's fine having
both. However, I worry that at some point the "SFTP lite" camp will add
features that exist in "SFTP full", and will add them in an incompatible
way.

If progress is to be made, it appears it is desirable to:

- Respect the wants of both camps. Work on both forks without sabotaging
the other.

- Clearly determine what work belongs in one fork ("SFTP lite") and what
belongs in the other ("SFTP full").

- Work on both forks in the same standards body, to coordinate what goes
where.

On Tue, Feb 20, 2018 at 12:12 AM, denis bider <[email-masked]>
wrote:

>>> SFTP is a widely used internet protocol that exists in two main
>>> versions: SFTP v3 with OpenSSH extensions, and SFTP v6. SFTP v4 is also
>>> widely supported, but most v4 implementations now seem to include v6.
>>>
>>> Now, the result is that SFTP is a major internet protocol, and anyone
>>> who wants to implement it needs to follow this for SFTP v3:
>>>
>>> However, if I write a draft - or perhaps two drafts - I'm not sure who
>>> to turn to. Does documenting existing SFTP practice warrant creating a new

---

From: Keith Moore <[email-masked]>
Date: Tue, 20 Feb 2018 23:25:31 -0500

I've been wondering for a long time if an approach like this could work.  Though I'd be tempted to call the common base something like "SFTP core" rather than "lite" because "lite" can sound like the work is of lesser importance when it's really of primary importance.   Then I wonder whether "SFTP extensions for Windows" and "SFTP extensions for *IX systems" and perhaps "SFTP extensions for remote file access" could be worthwhile.  "core" should be the mandatory-to-implement set of functions required for basic interoperability.

"core" should seek to minimize incompatible changes from existing server implementations, while also providing a reliable means by which a client can discover and make optimum use of server capabilities.

Keith

On Feb 20, 2018, at 1:48 AM, denis bider wrote:

> Adding to this, it seems clear there continue to be two camps that want to go in different extension directions while maintaining a compatible core. We could layer these into:
> 
> - "SFTP lite". This is supported by the open source community that wants to keep things simple, not implement support for platforms they don't care about, not implement features they don't need. They don't care and don't want support for Windows ACLs, file attributes, file sharing modes, native text files on platforms with different text/binary representations. This community wants to work on more extensions for SFTP v3, but without adding v4+ features.
> 
> - "SFTP full". This is supported by a mixture of open source and commercial implementations that want to provide features sought by users. If the user is connecting to a Windows server, these implementations allow manipulating ACLs, setting file attributes, having control over file sharing modes. When a platform has distinct text/binary representations, these implementations allow transferring textual files in a way that's convenient for the user. This community will want further work done on SFTP v6, but is also likely to incorporate new work on "SFTP lite".
> 
> To the extent these versions develop orthogonally, I think it's fine having both. However, I worry that at some point the "SFTP lite" camp will add features that exist in "SFTP full", and will add them in an incompatible way.
> 
> - Respect the wants of both camps. Work on both forks without sabotaging the other.
> 
> - Clearly determine what work belongs in one fork ("SFTP lite") and what belongs in the other ("SFTP full").
> 
> - Work on both forks in the same standards body, to coordinate what goes where.
>

---

From: denis bider <[email-masked]>
Date: Tue, 20 Feb 2018 23:16:02 -0600

I think that sounds sensible. "Core" sounds fine to me and would likely be
preferred by those who wish to implement that only. :-)

I found another implementation of SFTP v4/v6 that I did not think of
earlier. J2SSH implements SFTP v4 on server side, and SFTP v6 on the
client. It is a very common library in the enterprise world, especially for
custom Java applications that need to interact with SSH. I believe it's the
go-to SSH library in the Java world.

On Tue, Feb 20, 2018 at 10:25 PM, Keith Moore <[email-masked]>
wrote:

> I've been wondering for a long time if an approach like this could work.
> Though I'd be tempted to call the common base something like "SFTP core"
> rather than "lite" because "lite" can sound like the work is of lesser
> importance when it's really of primary importance.   Then I wonder whether
> "SFTP extensions for Windows" and "SFTP extensions for *IX systems" and
> perhaps "SFTP extensions for remote file access" could be worthwhile.
> "core" should be the mandatory-to-implement set of functions required for
> basic interoperability.
>
> "core" should seek to minimize incompatible changes from existing server
> implementations, while also providing a reliable means by which a client
> can discover and make optimum use of server capabilities.
>
> Keith
>
> > Adding to this, it seems clear there continue to be two camps that want
> to go in different extension directions while maintaining a compatible
> core. We could layer these into:
> >
> > - "SFTP lite". This is supported by the open source community that wants
> to keep things simple, not implement support for platforms they don't care
> > - "SFTP full". This is supported by a mixture of open source and
> commercial implementations that want to provide features sought by users.
> If the user is connecting to a Windows server, these implementations allow
> manipulating ACLs, setting file attributes, having control over file
> sharing modes. When a platform has distinct text/binary representations,
> these implementations allow transferring textual files in a way that's
> convenient for the user. This community will want further work done on SFTP
> v6, but is also likely to incorporate new work on "SFTP lite".
> >
> > To the extent these versions develop orthogonally, I think it's fine
> having both. However, I worry that at some point the "SFTP lite" camp will
> add features that exist in "SFTP full", and will add them in an
> incompatible way.
> >