Subject: LL32 Multihoming

From: Stuart Cheshire <[email-masked]>
Date: Mon, 23 Jun 2003 20:32:18 -0700

>> The phrase "run the algorithm independently on each interface" may
>> seem a simple thing to say, but when you have Ethernet and wireless,
>> and (unknown to you) the wireless base station you're talking to is
>> bridged onto the same Ethernet, the interfaces aren't independent
>> any more, even if you wish they were.
>
>We disagree.  If each interface runs the algorithm independently then
>the hosts wireless and wired interface will select and defend
>properly.  Only if they attempt to select and defend the same address
>will they fail.  In that case they need to choose different addreses.
>Using the algorithm, they are very likely to choose a different
>address that the other interface did not choose, and therefore will
>succeed.

Speaking as someone who has implemented this code, the problem is this:

1. Many IPv4 APIs (setsockopt IP_ADD_MEMBERSHIP, IP_MULTICAST_IF, etc.) 
identify interfaces using IPv4 addresses.

1(b). Therefore: It is helpful to ensure that each local interface has a 
different IPv4 address.

2. If a host sends from interface A to destination address B, and the 
host also has an interface with address B, this is ambiguous and problems 
arise.

2(b). Therefore: It is helpful to ensure that no local interface has an 
IPv4 address that it identical to *any* peer connected via *any* 
interface.

When you put these two requirement together, it means that if a host with 
address A on Ethernet, and address B on AirPort, sees its own AirPort 
packet come back on the Ethernet interface, then it may do the wrong 
thing. It may incorrectly conclude that some other host on the Ethernet 
has the same address as its AirPort interface, and proceed to change its 
AirPort address to something else. This results in an infinite 
"autoimmune" response packet storm. I know. It happened. It was bad.

The fix is that when receiving an apparently conflicting ARP packet, it 
should check *all* of its MAC addresses, to make sure the packet didn't 
come from itself. This does in fact fix the problem.

Stuart Cheshire <[email-masked]>
 * Wizard Without Portfolio, Apple Computer, Inc.
 * www.stuartcheshire.org

---

From: Robert Elz <[email-masked]>
Date: Tue, 24 Jun 2003 17:49:13 +0700

Date:        Mon, 23 Jun 2003 20:32:18 -0700
    From:        Stuart Cheshire <[email-masked]>
    Message-ID:  <[email-masked]>

  | 1(b). Therefore: It is helpful to ensure that each local interface has a 
  | different IPv4 address.

That's fine, and if you want to implement the address choice algorithm
so that addresses assigned to one interface cannot be assigned to another,
go for it.

  | 2. If a host sends from interface A to destination address B, and the 
  | host also has an interface with address B, this is ambiguous and problems 
  | arise.

But this is utter nonsense, and simply illustrates a defect in the
implementation.

When sending to a link local address, you *must* know which interface
you're planning on using, there cannot be any room for flexibility.

Otherwise, you have no idea which (of several) possible owners of the
target address you should communicate with - this is regardless of what
local addresses might be in use, and whether any of those are duplicated
anywhere.

  | 2(b). Therefore: It is helpful to ensure that no local interface has an 
  | IPv4 address that it identical to *any* peer connected via *any* 
  | interface.

This seems to imply that you want to be able to operate in a mode where
you don't care which interface you're sending to.  That is, when a packet
is to be sent to a destination and you have no idea which interface it
is connected to, you'd ARP on all of them.   But for that to work, you
have to assume there will only be one reply.   Otherwise you're simply
lost.   And to make sure there's only one reply, you really have to
do the address claim/defend algorithm on the whole network (each probe
needs to be relayed to all nets).

If we assume that (one day) all nodes support LL addresses on all interfaces,
then that's going to mean that the whole internet will see every probe.

That's absurd.

And of course, there's also nothing in these protocols to provide any
measure of loop control on these relayed probes that would be needed.

So, clearly we cannot do network wide address uniqueness testing, only
link wide.   Thus, it is clearly possible for any address to exist on
multiple links, and some means other than the address itself is going to
be required for disambiguating that case.

As soon as you reach that conclusion, you also see that exactly the same
mechanism suffices to handle local addresses - a packet addressed to a
link local address on interface 1 cannot possibly get caught by any
link local address on interface 2, even where the target address is the
local LL address of interface 2.   At this point it is no longer required
that the local LL address of interface 2 fail to conflict with LL addresses
on the link connected to interface 1.

  | When you put these two requirement together,

We do not do that.

  | it means that if a host with 
  | address A on Ethernet, and address B on AirPort, sees its own AirPort 
  | packet come back on the Ethernet interface, then it may do the wrong 
  | thing. It may incorrectly conclude that some other host on the Ethernet 
  | has the same address as its AirPort interface, and proceed to change its 
  | AirPort address to something else.

It should not, that is broken behaviour.

  | The fix is that when receiving an apparently conflicting ARP packet, it 
  | should check *all* of its MAC addresses, to make sure the packet didn't 
  | come from itself. This does in fact fix the problem.

It is fine to do that, but it should not be necessary.

kre