Network Working Group C. Partridge
Request for Comments: 1546 T. Mendez
Category: Informational W. Milliken
BBN
November 1993
Host Anycasting Service
Status of this Memo
This memo provides information for the Internet community. This memo
does not specify an Internet standard of any kind. Distribution of
this memo is unlimited.
Abstract
This RFCdescribes an internet anycasting service for IP. The
primary purpose of this memo is to establish the semantics of an
anycasting service within an IP internet. Insofar as is possible,
this memo tries to be agnostic about how the service is actually
provided by the internetwork. This memo describes an eXPerimental
service and does not propose a protocol. This memo is prodUCed by
the Internet Research Task Force (IRTF).
Motivation
There are a number of situations in networking where a host,
application, or user wishes to locate a host which supports a
particular service but, if several servers support the service, does
not particularly care which server is used. Anycasting is a
internetwork service which meets this need. A host transmits a
datagram to an anycast address and the internetwork is responsible
for providing best effort delivery of the datagram to at least one,
and preferably only one, of the servers that accept datagrams for the
anycast address.
The motivation for anycasting is that it considerably simplifies the
task of finding an appropriate server. For example, users, instead
of consulting a list of archie servers and choosing the closest
server, could simply type:
telnet archie.net
and be connected to the nearest archie server. DNS resolvers would
no longer have to be configured with the IP addresses of their
servers, but rather could send a query to a well-known DNS anycast
address. Mirrored FTP sites could similarly share a single anycast
address, and users could simply FTP to the anycast address to reach
the nearest server.
Architectural Issues
Adding anycasting to the repertoire of IP services requires some
decisions to be made about how to balance the architectural
requirements of IP with those of anycasting. This section discusses
these architectural issues.
The first and most critical architectural issue is how to balance
IP's stateless service with the desire to have an anycast address
represent a single virtual host. The best way to illustrate this
problem is with a couple of examples. In both of these examples, two
hosts (X and Y) are serving an anycast address and another host (Z)
is using the anycast address to contact a service.
In the first example, suppose that Z sends a UDP datagram addressed
to the anycast address. Now, given that an anycast address is
logically considered the address of a single virtual host, should it
be possible for the datagram to be delivered to both X and Y? The
answer to this question clearly has to be yes, delivery to both X and
Y is permissible. IP is allowed to duplicate and misroute datagrams
so there clearly are scenarios in which a single datagram could be
delivered to both X and Y. The implication of this conclusion is
that the definition of anycasting in an IP environment is that IP
anycasting provides best effort delivery of an anycast datagram to
one, but possibly more than one, of the hosts that serve the
destination anycast address.
In the second example, suppose that Z sends two datagrams addressed
to the anycast address. The first datagram gets delivered to X. To
which host (X or Y) does the second datagram get delivered? It would
be convenient for stateful protocols like TCP if all of a
connection's datagrams were delivered to the same anycast address.
However, because IP is stateless (and thus cannot keep track of where
earlier datagrams were delivered) and because one of the goals of
anycasting is to support replicated services, it seems clear that the
second datagram can be delivered to either X or Y. Stateful
protocols will have to employ some additional mechanism to ensure
that later datagrams are sent to the same host. Suggestions for how
to accomplish this for TCP are discussed below.
After considering the two examples, it seems clear that the correct
definition of IP anycasting is a service which provides a stateless
best effort delivery of an anycast datagram to at least one host, and
preferably only one host, which serves the anycast address. This
definition makes clear that anycast datagrams receive the same basic
type of service as IP datagrams. And while the definition permits
delivery to multiple hosts, it makes clear that the goal is delivery
to just one host.
Anycast Addresses
There appear to be a number of ways to support anycast addresses,
some of which use small pieces of the existing address space, others
of which require that a special class of IP addresses be assigned.
The major advantage of using the existing address space is that it
may make routing easier. As an example, consider a situation where a
portion of each IP network number can be used for anycasting. I.e.,
a site, if it desires, could assign a set of its subnet addresses to
be anycast addresses. If, as some experts expect, anycast routes are
treated just like host routes by the routing protocols, the anycast
addresses would not require special advertisement outside the site --
the host routes could be folded in with the net route. (If the
anycast addresses is supported by hosts outside the network, then
those hosts would still have be advertised using host routes). The
major disadvantages of this approach are (1) that there is no easy
way for stateful protocols like TCP to discover that an address is an
anycast address, and (2) it is more difficult to support internet-
wide well-known anycast address. The reasons TCP needs to know that
an address is an anycast address is discussed in more detail below.
The concern about well-known anycast addresses requires a bit of
explanation. The idea is that the Internet might establish that a
particular anycast address is the logical address of the DNS server.
Then host software could be configured at the manufacturer to always
send DNS queries to the DNS anycast address. In other Words,
anycasting could be used to support autoconfiguration of DNS
resolvers.
The major advantages of using a separate class of addresses are that
it is easy to determine if an address is an anycast address and
well-known anycast addresses are easier to support. The key
disadvantage is that routing may be more painful, because the routing
protocols may have to keep track of more anycast routes.
An intermediate approach is to take part of the current address space
(say 256 Class C addresses) and make the network addresses into
anycast addresses (and ignore the host part of the class C address).
The advantage of this approach is that it makes anycast routes look
like network routes (which are easier for some routing protocols to
handle). The disadvantages are that it uses the address space
inefficiently and so more severely limits the number of anycast
addresses that can be supported.
In the balance it seems wiser to use a separate class of addresses.
Carving anycast addresses from the existing address space seems more
likely to cause problems in situations in which either applications
mistakenly fail to recognize anycast addresses (if anycasts are part
of each site's address space) or use the address space inefficiently
(if network addresses are used as anycast addresses). And the
advantages of using anycast addresses for autoconfiguration seem
compelling. So this memo assumes that anycast addresses will be a
separate class of IP addresses (not yet assigned). Since each
anycast address is a virtual host address and the number of
anycasting hosts seems unlikely to be larger than the number of
services offered by protocols like TCP and UDP, the address space
could be quite small, perhaps supporting as little as 2**16 different
addresses.
Transmission and Reception of Anycast Datagrams
Historically, IP services have been designed to work even if routers
are not present (e.g., on LANs without routers). Furthermore, many
in the Internet community have historically felt that hosts should
not have to participate in routing protocols to operate. (See, for
instance, page 7 of STD 3, RFC1122). To provide an anycasting
service that is consistent with these traditions, the handling of
anycast addresses varies slightly depending on the type of network on
which datagrams with anycast addresses are sent.
On a shared media network, such as an Ethernet and or Token Ring, it
must be possible to transmit an anycast datagram to a server also on
the same network without consulting a (possibly non-existent) router.
There are at least two ways this can be done.
One approach is to ARP for the anycast address. Servers which
support the anycast address can reply to the ARP request, and the
sending host can transmit to the first server that responds. This
approach is reminiscent of the ARP hack (RFC1027) and like the ARP
hack, requires ARP cache timeouts for the anycast addresses be kept
small (around 1 minute), so that if an anycast server goes down,
hosts will promptly flush the ARP entry and query for other servers
supporting the anycast address.
Another approach is for hosts to transmit anycast datagrams on a
link-level multicast address. Hosts which serve an anycast address
would be expected to listen to the link-level multicast address for
datagrams destined for their anycast address. By multicasting on the
local network, there is no need for a router to route the anycast
datagrams. One merit of this approach is that if there are multiple
servers and one goes down, the others will still receive any
requests. Another possible advantage is that, because anycast ARP
entries must be quickly timed out, the multicasting approach may be
less traffic intensive than the ARP approach because in the ARP
approach, transmissions to an anycast address are likely to cause a
broadcast ARP, while in the multicast approach, transmissions are
only to a select multicast group. An obvious disadvantage is that if
there are multiple servers on a network, they will all receive the
anycast message, when delivery to only one server was desired.
On point-to-point links, anycast support is simpler. A single copy
of the anycast datagram is forwarded along the appropriate link
towards the anycast destination.
When a router receives an anycast datagram, the router must decide if
it should forward the datagram, and if so, transmits one copy of the
datagram to the next hop on the route. Note that while we may hope
that a router will always know the correct next hop for an anycast
datagram and will not have to multicast anycast datagrams on a local
network, there are probably situations in which there are multiple
servers on a local network, and to avoid sending to one that has
recently crashed, routers may wish to send anycast datagrams on a
link-level multicast address. Because hosts may multicast any
datagrams, routers should take care not to forward a datagram if they
believe that another router will also be forwarding it.
Hosts which wish to receive datagrams for a particular anycast
address will have to advertise to routers that they have joined the
anycast address. On shared media networks, the best mechanism is
probably for a host to periodically multicast information about the
anycast addresses it supports (possibly using an enhanced version of
IGMP). The multicast messages ensure that any routers on the network
hear that the anycast address is supported on the local subnet and
can advertise that fact (if appropriate) to neighboring routers.
Note that if there are no routers on the subnet, the multicast
messages would simply simply ignored. (The multicasting approach is
suggested because it seems likely to be simpler and more reliable
than developing a registration protocol, in which an anycast server
must register itself with each router on its local network).
On point-to-point links, a host can simply advertise its anycast
addresses to the router on the other end of the link.
Observe that the advertisement protocols are a form of routing
protocol and that it may make sense to simply require anycast servers
to participate (at least partly) in exchanges of regular routing
messages.
When a host receives an IP datagram destined for an anycast address
it supports, the host should treat the IP datagram just as if it was
destined for one of the host's non-anycast IP addresses. If the host
does not support the anycast address, it should silently discard the
datagram.
Hosts should accept datagrams with an anycast source address,
although some transport protocols (see below) may refuse to accept
them.
How UDP and TCP Use Anycasting
It is important to remember that anycasting is a stateless service.
An internetwork has no obligation to deliver two successive packets
sent to the same anycast address to the same host.
Because UDP is stateless and anycasting is a stateless service, UDP
can treat anycast addresses like regular IP addresses. A UDP
datagram sent to an anycast address is just like a unicast UDP
datagram from the perspective of UDP and its application. A UDP
datagram from an anycast address is like a datagram from a unicast
address. Furthermore, a datagram from an anycast address to an
anycast address can be treated by UDP as just like a unicast datagram
(although the application semantics of such a datagram are a bit
unclear).
TCP's use of anycasting is less straightforward because TCP is
stateful. It is hard to envision how one would maintain TCP state
with an anycast peer when two successive TCP segments sent to the
anycast peer might be delivered to completely different hosts.
The solution to this problem is to only permit anycast addresses as
the remote address of a TCP SYN segment (without the ACK bit set). A
TCP can then initiate a connection to an anycast address. When the
SYN-ACK is sent back by the host that received the anycast segment,
the initiating TCP should replace the anycast address of its peer,
with the address of the host returning the SYN-ACK. (The initiating
TCP can recognize the connection for which the SYN-ACK is destined by
treating the anycast address as a wildcard address, which matches any
incoming SYN-ACK segment with the correct destination port and
address and source port, provided the SYN-ACK's full address,
including source address, does not match another connection and the
sequence numbers in the SYN-ACK are correct.) This approach ensures
that a TCP, after receiving the SYN-ACK is always communicating with
only one host.
Applications and Anycasting
In general, applications use anycast addresses like any other IP
address. The only worrisome application use of anycasting is
applications which try to maintain stateful connections over UDP and
applications which try to maintain state across multiple TCP
connections. Because anycasting is stateless and does not guarantee
delivery of multiple anycast datagrams to the same system, an
application cannot be sure that it is communicating with the same
peer in two successive UDP transmissions or in two successive TCP
connections to the same anycast address.
The obvious solutions to these issues are to require applications
which wish to maintain state to learn the unicast address of their
peer on the first exchange of UDP datagrams or during the first TCP
connection and use the unicast address in future conversations.
Anycasting and Multicasting
It has often been suggested that IP multicasting can be used for
resource location, so it is useful to compare the services offered by
IP multicasting and IP anycasting.
Semantically, the difference between the two services is that an
anycast address is the address of a single (virtual) host and that
the internetwork will make an effort to deliver anycast datagrams to
a single host. There are two implications of this difference.
First, applications sending to anycast addresses need not worry about
managing the TTLs of their IP datagrams. Applications using
multicast to find a service must balance their TTLs to maximize the
chance of finding a server while minimizing the chance of sending
datagrams to a large number of servers it does not care about.
Second, making a TCP connection to an anycast address makes perfectly
good sense, while the meaning of making a TCP connection to a
multicast address are unclear. (A TCP connection to a multicast
address is presumably trying to establish a connection to multiple
peers simultaneously, which TCP is not designed to support).
From a practical perspective, the major difference between anycasting
and multicasting is that anycasting is a special use of unicast
addressing while multicasting requires more sophisticated routing
support. The important observation is that multiple routes to an
anycast address appear to a router as multiple routes to a unicast
destination, and the router can use standard algorithms to choose to
the best route.
Another difference between the two approaches is that resource
location using multicasting typically causes more datagrams to be
sent. To find a server using multicasting, an application is
expected to transmit and retransmit a multicast datagram with
successively larger IP TTLs. The TTL is initially kept small to try
to limit the number of servers contacted. However, if no servers
respond, the TTL must be increased on the assumption that the
available servers (if any) were farther away than was reachable with
the initial TTL. As a result, resource location using multicasting
causes one or more multicast datagrams to be sent towards multiple
servers, with some datagrams' TTLs expiring before reaching a server.
With anycasting, managing the TTL is not required and so (ignoring
the case of loss) only one datagram need be sent to locate a server.
Furthermore, this datagram will follow only a single path.
A minor difference between the two approaches is that anycast may be
less fault tolerant than multicast. When an anycast server fails,
some datagrams may continue to be mistakenly routed to the server,
whereas if the datagram had been multicast, other servers would have
received it.
Related Work
The ARPANET AHIP-E Host Access Protocol described in RFC878 supports
logical addressing which allows several hosts to share a single
logical address. This scheme could be used to support anycasting
within a PSN subnet.
Security Considerations
There are at least two security issues in anycasting, which are
simply mentioned here without suggested solutions.
First, it is clear that malevolent hosts could volunteer to serve an
anycast address and divert anycast datagrams from legitimate servers
to themselves.
Second, eavesdropping hosts could reply to anycast queries with
inaccurate information. Since there is no way to verify membership
in an anycast address, there is no way to detect that the
eavesdropping host is not serving the anycast address to which the
original query was sent.
Acknowledgements
This memo has benefitted from comments from Steve Deering, Paul
Francis, Christian Huitema, Greg Minshall, Jon Postel, Ram
Ramanathan, and Bill Simpson. However, the authors are solely
responsible for any dumb ideas in this work.
Authors' Addresses
Craig Partridge
Bolt Beranek and Newman
10 Moulton St
Cambridge MA 02138
EMail: craig@bbn.com
Trevor Mendez
Bolt Beranek and Newman
10 Moulton St
Cambridge MA 02138
EMail: tmendez@bbn.com
Walter Milliken
Bolt Beranek and Newman
10 Moulton St
Cambridge MA 02138
EMail: milliken@bbn.com
