Subject: [saag] draft-kwatsen-reverse-ssh submission for review

From: Kent Watsen <[email-masked]>
Date: Thu, 12 May 2011 16:31:02 -0700

Since the SECSH working group has concluded, the Security Area Directors, Sean and Stephen, recommended posting an announcement regarding this individual submission to the SAAG and IETF-SSH mailing lists.  

   http://tools.ietf.org/html/draft-kwatsen-reverse-ssh-00

   Abstract

      This memo presents a technique for a SSH (Secure Shell) server to
      initiate the underlying TCP connection to the SSH client.  This role
      reversal is necessary in cases where the SSH client would otherwise
      be unable to initiate an SSH connection to the SSH server, such as a
      device "calling home" on its first boot.

I come from the NETCONF and NETMOD working groups, and this submission has been developed primarily to support NETCONF, though it's applicable to any SSH-based protocol and actually has little to do with NETCONF at all, which is why it is brought here for your consideration.

FWIW, Juniper has implemented a variant of this proposal, called "outbound-ssh", on almost all its platforms for nearly 5 years now.  The solution presented in this I-D, being fully transparent to the SSH protocol, has been shown to be easy to implement across various operating systems and programming languages.

PS: I just subscribed to both the SAAG and IETF-SSH lists - cheers!

Thanks,
Kent

--
Kent Watsen
JSBU/DBU Architect
Juniper Networks

---

From: Joe Touch <[email-masked]>
Date: Thu, 12 May 2011 17:37:36 -0700

On 5/12/2011 4:31 PM, Kent Watsen wrote:
> I come from the NETCONF and NETMOD working groups, and this
> submissionhas been developed primarily to support NETCONF, though
> it's applicable to any SSH-based protocol and actually has little to
> do with NETCONF at all, which is why it is brought here for your
> consideration.
>

Any clue what port they're using for this? There doesn't appear to be 
one currently allocated.

Also, there are separate ports for SSH (22) and netconf over SSH (830) - 
does this mean this proposal would need a reverse port for every 
SSH-based service?

Joe

---

From: Kent Watsen <[email-masked]>
Date: Thu, 12 May 2011 18:01:03 -0700

Juniper is using port 7104, I think, but there is no need to maintain that compatibility since the message format presented in this submission is not backward compatible with their's.  For instance, Juniper's existing format only supports one host-key, whereas this this proposal supports all the host-keys the SSH server has.

No, this submission only asks IANA to assign a single port, to bootstrap the SSH protocol.  Once the SSH session is up, the SSH client can open any number of SSH channels for tty, sftp, netconf, port-forwarding, etc.

This solution is NOT expected to work with a standard 'ssh' client.  The reason for why the SSH server has been configure to connect to the SSH client is domain specific.  The expectation is that a custom application is developed (using standard SSH client libraries) for the purpose.

Thanks,
Kent

---

From: Joe Touch <[email-masked]>
Date: Thu, 12 May 2011 18:07:12 -0700

On 5/12/2011 6:01 PM, Kent Watsen wrote:
> Juniper is using port 7104, I think, but there is no need to
> maintain that compatibility since the message format presented in
> this submission is not backward compatible with their's. For
> instance, Juniper's existing format only supports one host-key,
> whereas this this proposal supports all the host-keys the SSH server
> has.

Understood; I was asking because it sounded like Juniper was using a 
port they hadn't registered ;-( That should be fixed...

>> Also, there are separate ports for SSH (22) and netconf over SSH
>> (830) - does this mean this proposal would need a reverse port for
>> every SSH-based service?
>
> No, this submission only asks IANA to assign a single port, to
> bootstrap the SSH protocol. Once the SSH session is up, the SSH client
> can open any number of SSH channels for tty, sftp, netconf,
> port-forwarding, etc.

Netconf over ssh uses a different port, as noted above.

> This solution is NOT expected to work with a standard 'ssh' client.
> The reason for why the SSH server has been configure to connect to the
> SSH client is domain specific. The expectation is that a custom
> application is developed (using standard SSH client libraries) for the
> purpose.

What's the reason for not solving this by having the client just listen 
on the SSH server port?

Joe

---

From: Kent Watsen <[email-masked]>
Date: Thu, 12 May 2011 19:49:53 -0700

> Understood; I was asking because it sounded like Juniper was using 
> a port they hadn't registered ;-( That should be fixed...

Indeed, and that is exactly what we're trying to do with this submission.  Our previous attempt to have IANA assign a port for this ended with their recommendation to bring it to IETF for standardization.  It's taken awhile, but here we are.

> Netconf over ssh uses a different port, as noted above.

The need for the port 830 assignment was only to facilitate filtering.  As RFC 4742 says:

   In order to allow NETCONF traffic to be easily identified and
   filtered by firewalls and other network devices, NETCONF servers MUST
   default to providing access to the "netconf" SSH subsystem only when
   the SSH session is established using the IANA-assigned TCP port
   <830>.  Servers SHOULD be configurable to allow access to the netconf
   SSH subsystem over other ports.

I suppose IETF may feel a similar need to facilitate filtering on a per SSH-based service, but I wouldn't recommend this as port-based filtering is no longer practical (e.g. consider the number of HTTP-based protocols).  Application-based firewalls with deep-inspection capability are needed these days.  In either case, to the original question, there isn’t a *need* for more than one IANA assigned port, since SSH already has a mechanism built into it for the client to select which protocol/subsystem to run on a channel.

> What's the reason for not solving this by having the client just listen 

Because then it would be expected to be the SSH server.  As discussed in the Introduction, a goal of this draft is to ensure the device is always the SSH server and the application is always the SSH client.  We don't want to disturb which peer is which as far as the SSH Transport, Authentication, and Connection protocols are concerned.

Thanks,
Kent

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 13 May 2011 06:33:34 +0200

> I suppose IETF may feel a similar need to facilitate filtering on a per SSH-based service, but I wouldn't recommend this as port-based filtering is no longer practical (e.g. consider the number of…

The subsystem is selected through the encrypted channel, something
difficult to filter on.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Joe Touch <[email-masked]>
Date: Fri, 13 May 2011 08:02:13 -0700

On 5/12/2011 7:49 PM, Kent Watsen wrote:
> Indeed, and that is exactly what we're trying to do with this
> submission. Our previous attempt to have IANA assign a port for this
> ended with their recommendation to bring it to IETF for standardization.
> It's taken awhile, but here we are.

Sure. In the meantime, Juniper shouldn't be using an unassigned port
number in a public distribution, though. ;-)

> The need for the port 830 assignment was only to facilitate filtering.

Yes. And will the need for a reverse port create a similar need for 
every such current SSH-based port assignment to have a corresponding 
reverse-channel port assignment? That would be undesirable...

> Because then it would be expected to be the SSH server.

Well, seems to me that if the server is initiating the connection, then 
it *is* a server (where I define server as "host that listens on a 
registered port").

The particular roles of who checks what certificate should be negotiated 
in-band, IMO.

Joe

---

From: Kent Watsen <[email-masked]>
Date: Fri, 13 May 2011 09:42:00 -0700

> The subsystem is selected through the encrypted channel, something difficult to filter on.

True.  Though some firewalls can be provided the backend server's private key(s) in order to inspect the traffic.  This feature is more prevalent on industrial firewalls than consumer-grade firewalls.

Of course, there is a catch-22 to this is that, at least in our application of this proposal, the device that the "calling home" many times (but not always - see draft for details) is the gateway device having the firewalling functionality.

Still, I concur with your premise that the solution shouldn't discard the ability to do port-level filtering for each SSH-based protocol that has been reversed.  The question remains, though, if there should be a new port-assignment for each SSH-based protocol or if the existing ports are repurposed.

Would repurposing existing ports be too ambiguous, due to them having dual roles?

PS: Per Tom's comment, I've BCC-ed the "ietf-ssh" list to bring this discussion to one list.  I'm choosing the SAAG list primarily because it is an official IETF list.  Hope this is OK with all.

Thanks,
Kent

---

From: der Mouse <[email-masked]>
Date: Fri, 13 May 2011 12:58:45 -0400 (EDT)

> PS: Per Tom's comment, I've BCC-ed the "ietf-ssh" list to bring this
> discussion to one list.  I'm choosing the SAAG list primarily because
> it is an official IETF list.  Hope this is OK with all.

For what it may be worth, I'd say the SSH list is better for discussing
the technical aspects of the draft as it relates to SSH; the SAAG list
(or maybe even some other IETF list; I hardly know them all) is perhaps
better for discussing meta-issues such as what ports to use.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		[email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: Kent Watsen <[email-masked]>
Date: Fri, 13 May 2011 10:04:13 -0700

OK, then let's flip it the other way - sorry about that

Anyone responding to my last email, please put ietf-ssh back into the "To" line and BCC saag

Mea culpa!

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of der Mouse
Sent: Friday, May 13, 2011 12:59 PM
To: [email-masked]; [email-masked]
Subject: Re: [saag] draft-kwatsen-reverse-ssh submission for review

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		[email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B
_______________________________________________
saag mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/saag

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 13 May 2011 14:02:13 -0700

[[ I think both mailing lists should remain on the mail. The issues relate to both. ]]

On May 13, 2011, at 8:02 AM, Joe Touch wrote:

> Yes. And will the need for a reverse port create a similar need for every such current SSH-based port assignment to have a corresponding reverse-channel port assignment? That would be undesirable...

+1. A single reverse-port protocol that has a way to say "and I'm going to be doing X protocol" would be a much better design.

> Well, seems to me that if the server is initiating the connection, then it *is* a server (where I define server as "host that listens on a registered port").

Fully agree. The design of this draft sounds like it can be summarized as "an entity that is normally a client becomes a server for a short period while it gets information from the entity that is normally a server". At the point that the was-a-client becomes a short-term server, why not make it a real SSH server?

> The particular roles of who checks what certificate should be negotiated in-band, IMO.

That sounds right to me.

--Paul Hoffman

---

From: "t.petch" <[email-masked]>
Date: Fri, 13 May 2011 09:16:40 +0200

----- Original Message -----
From: "Kent Watsen" <[email-masked]>
To: "Joe Touch" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Friday, May 13, 2011 4:49 AM

> Because then it would be expected to be the SSH server.  As discussed in the
Introduction, a goal of this draft is to ensure the device is always the SSH
server and the application is always the SSH client.  We don't want to disturb
which peer is which as far as the SSH Transport, Authentication, and Connection
protocols are concerned.

I do :-)

SSH does not authenticate the user or the application, it authenticates SSH.

The proper solution is channel binding in which case it does not matter who
is SSH client and who is SSH server, and the existing SSH technology can
be reused unaltered.

Incidentally, I think that this discussion needs a home, be it ietf-ssh or
whatever;
having more than one makes a mess of mailing lists.

Tom Petch

---

From: Derek Fawcus <[email-masked]>
Date: Fri, 13 May 2011 11:04:46 +0100

> is SSH client and who is SSH server, and the existing SSH technology can

Quite.  I recall looking at this a few years ago.

The ssh protocols are largely symmetrical,  as I recall it should be possible
to flip roles once transport has completed,  but before userauth or connection
are started.

.pdf

---

From: Kent Watsen <[email-masked]>
Date: Fri, 13 May 2011 11:17:49 -0700

> [email-masked] writes: 
>

If the application (using nomenclature from the draft) logs into the device via userauth, then it follows that the device has to be the peer that presents its host-key during the diffie-helman key exchange.  Thus the role-flip has to occur either before the transport protocol begins, as proposed by this draft, or at the very beginning of the transport protocol (i.e. tap into the reserved uint32 in the SSH_MSG_KEXINIT message).  

Whether the roles are negotiated in-band or flipped prior to the start of the SSH protocol, the goal is for the device to always be the SSH server and the application to always be the SSH client.

PS: BCC-ing SAAG list, per der Mouse's suggestion

Thanks,
Kent

---

From: Kent Watsen <[email-masked]>
Date: Mon, 23 May 2011 10:56:52 -0700

> I think both mailing lists should remain on the mail. The issues 
> relate to both.

OK, we'll keep both on, just as the Security ADs originally suggested.

> +1. A single reverse-port protocol that has a way to say "and I'm going 
> to be doing X protocol" would be a much better design.

Sounds a little bit like the tcpmux protocol.  Do you think how SSH opens
channels/subsystems is not good enough?   A proliferation of ports doesn't
sound good, but others expressed a need for ports to support port-based
firewall filters.  How many ssh-based protocols are there currently?  I 
only know of two (SSH/SCP/SFTP:22 and NETCONF:830) - are there any others?  

> The design of this draft sounds like it can be summarized as "an entity
> that is normally a client becomes a server for a short period while it
> gets information from the entity that is normally a server". At the 
> point that the was-a-client becomes a short-term server, why not make
> it a real SSH server?

What does this means at a protocol level?  Does the application present a
SSH host-key during DH key exchange and the device logs into the application
via userauth?  - if so, then please realize that this is exactly what we're
trying to prevent.  We want the device to always be the peer that presents
its SSH host key, whether due to the application connecting to its port 22
or because the device connected to the app's Reverse SSH port.   Likewise,
we always want the app to be the peer that userauth's to the device, 
regardless how the transport was setup.

> > The particular roles of who checks what certificate should be negotiated

This would only be necessary if there are no new port assignments as, 
otherwise, each peer *knows* which role to assume.  For instance, each
peer knows its role if started like this:

  app:     ssh --listen <rssh-port> --handler /usr/local/bin/myapp
  device:  sshd --connect <app>:<rssh-port> --id 234230 --hmackey secret

Otherwise, if there are no new port assignments (i.e. <rssh-port>==22),
then we'd have to have in-band negotiation, perhaps by the device 
sending a special identification string (RFC 4253, section 4.2) like
this?

  SSH-2.0-OpenSSH_5.6 <SP> Reverse SSH <CR> <LF>

If having in-band negotiation, then we should also extend the SSH 
Protocol to support device identification and automatic authentication
of host keys other than PGP and x.509 certificates, this is the 
essence of the REVERSE-SSH-CONN-INFO message in the draft.

Thanks,
Kent

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 23 May 2011 11:41:44 -0700

On May 23, 2011, at 10:56 AM, Kent Watsen wrote:

> firewall filters.  

Firewalls should adapt to good protocol design; protocol design should not be weakened to work with firewalls. (And I say this as someone who likes firewalls.)

> How many ssh-based protocols are there currently?  I 

You say that as if we never expect to have any more, which seems like poor planning. For example, many people run VNC and other such protocols on other ports. The "ssh -L 15901:localhost:5901 example.com" usage is quite common. Making it harder for protocols other than NETCONF to use SSH doesn't seem like a good plan at this point.

> trying to prevent.  

I do realize that; I am proposing that what you are trying to prevent is in fact fine, and a lot more rational than a "I'm a kind-of-server for just a moment" design.

> We want the device to always be the peer that presents

Understood. The draft does a reasonable job of matching that design.

>  SSH-2.0-OpenSSH_5.6 <SP> Reverse SSH <CR> <LF>

This assumes that all SSH use is only on the currently-assigned ports. That ignores the "-L" use case I gave above, as well as future protocols that want to run over SSH.

> essence of the REVERSE-SSH-CONN-INFO message in the draft.

That is one design; another is "when I am acting like a server, I am actually a server". That seems a lot cleaner.

--Paul Hoffman

---

From: Kent Watsen <[email-masked]>
Date: Mon, 23 May 2011 16:17:08 -0700

> Firewalls should adapt to good protocol design; protocol design should
> not be weakened to work with firewalls. (And I say this as someone who
> likes firewalls.)

Agreed - let's focus on a good protocol design ;)

>> How many ssh-based protocols are there currently?  I only know
>> of two (SSH/SCP/SFTP:22 and NETCONF:830) - are there any others?  
>
> You say that as if we never expect to have any more, which seems like 
> poor planning. For example, many people run VNC and other such protocols
> on other ports. The "ssh -L 15901:localhost:5901 example.com" usage is
> quite common. Making it harder for protocols other than NETCONF to use
> SSH doesn't seem like a good plan at this point.

I didn't mean to imply that there wouldn't be more, only that there are
so few now and that new SSH-based protocols can define a reverse-port for
themselves, if it makes sense to them to do so.

I use that port forwarding line with my virtual machines.  It's a good
example because VNC is used to "manage" devices similar to NetConf, so
it's plausible that a device might be configured to init a reverse ssh
connection to support VNC.  In this case, the `ssh` connection goes to
port 22, so reversing it means that the device would connect to the 
application's rssh port and, once the SSH Connection Protocol (RFC 4254)
is running, the application can send a SSH_MSG_CHANNEL_OPEN/direct-tcpip
request to open a tunnel to the VNC server.

> I do realize that; I am proposing that what you are trying to prevent is 
> in  fact fine, and a lot more rational than a "I'm a kind-of-server for 
> just a moment" design.

Thanks, Paul.  I appreciate your endorsement.  At least, I think it is an
endorsement... (I'm not entirely sure per your last comment below)

>This assumes that all SSH use is only on the currently-assigned ports. That ignores the "-L" use >case I gave above, as well as future protocols that want to run over SSH.

FWIW, I personally don't like repurposing the currently-assigned ports 
because it would be confusing to suddenly not be sure that port 22 was
for a SSH Server and port 830 was for a NetConf server.  I only offered
how we might support in-band negotiation with the identification string
because it seemed folks wanted to explore that option.  

I like having a single well-known port (like tcpmux/port 1) that can be
used for any SSH-based protocol, leaving it to the client to select which
ones to start via opening channels/subsystems.  For instance, an app can
have simultaneous channels open for Syslog, SNMP, NetConf, VNC, HTTPS,
and TTY.  Which reminds me that I've always wanted an ability for a SSH
Client to get a listing of all the subsystems available on the server, 
similar to tcpmux's "HELP" command (another draft?)

Another option, that I'm a bit shy to promote, is to have *no* assigned 
ports.  This is technically feasible since the devices MUST be configured
to connect to the applications (i.e. address, port, id, credentials, etc.).
The devices never connect out of the blue, thus no well-known port is 
required.  But this strategy would entail applications using unassigned
ports, the very thing that Joe Touch flagged the day I submitted the draft.

>That is one design; another is "when I am acting like a server, I am 
>actually a server". That seems a lot cleaner.

This is the comment I mentioned above is being unsure if it means you
like the current draft or not.  Is the "I am actually a server" statement
satisfied by having an application listening on a Reverse SSH port, where
the entire "protocol" is merely to bootstrap the SSH protocol?

Thanks again,
Kent

---

From: Joe Touch <[email-masked]>
Date: Tue, 24 May 2011 07:50:40 -0700

On May 23, 2011, at 4:17 PM, Kent Watsen <[email-masked]> wrote:
> ones to start via opening channels/subsystems.  

Why isn't that the current ssh port?

> For instance, an app can

That is what the dns srv records might do, so you could ootentkially use 53 for that. But wouldn't that be a security concern?

> ports, the very thing that Joe Touch flagged the day I submitted the draft.

Using only ports from the dynamic range is fine, but won't appease firewall designers. 

Using the entire port range, or and single port that is from the assignable range that hasn't been assigned to you is a problem (and I was concerned with the latter in your submission).

> the entire "protocol" is merely to bootstrap the SSH protocol?

I'm a server if I listen on the ssh port. On that port you should indicate or negotiate specifics of each side's behavior in-band IMO. 

Joe

---

From: der Mouse <[email-masked]>
Date: Tue, 24 May 2011 11:54:36 -0400 (EDT)

> I'm a server if I listen on the ssh port.  On that port you should
> indicate or negotiate specifics of each side's behavior in-band IMO.

This sounds like a reasonable point of view to me.

If your reversed ssh runs kex with roles reversed (ie, connection
initiator takes the server's role, presenting its host key and such),
then a passive snooper can tell the difference, so you might as well
trigger the role reversal with a pre-kex extension packet.

If your reversd ssh runs kex normally (initiator takes the client's
role) but reverses things after that, then you can do it with an
extension packet immediately after kex (before, and/or instead of, what
would normally be userauth).

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Kent Watsen <[email-masked]>
Date: Wed, 25 May 2011 15:27:11 -0700

>trigger the role reversal with a pre-kex extension packet.

OK, I'll submit an updated draft for this approach

Thanks,
Kent