WAN Replicated Service Availability?

[Dave Mack]

How about round robin DNS with short TTLs (a minute) and each machine 
dynamically updating its A record every 30 seconds? A dead machine's 
record stops being handed out fairly quickly and re-enters service 
equally quickly when it comes back.

Rick Reineman wrote:

> I've been grinding this problem around in my head for awhile then it 
> occurred to me that this may be a good forum to pose the question.
>
> How to deploy a series of replicated services (web, ldap and oracle) 
> across a wide geographical area with the goal being survivability in 
> the event of a significant Internet outage?
>
> In other words if (for example) the East coast of the US looses 
> Internet connectivity due to some catastrophe, the rest of the country 
> should still have access to (at least) one of the replicated servers 
> that are still available.
>
> With round robin DNS we can hand out IP's to servers in turn but there 
> is no validation that a service is available.
>
> A load balancer can do some service availability evaluation but we 
> have the possibility of a load balancer in the geographically affected 
> area.
>
> A router can determine availability of a path or paths to an IP, but 
> not a service (as far as I know).  It wouldn't take long for a complex 
> routing issue to be outside of my current ability to understand though.
>
> I know people do replicated services all the time but I get the idea 
> they are not planning for the same sort of network unavailability that 
> I want to.  Most replicated services I am aware of are in one co-lo 
> site with a load balancer, maybe highly available load balancers.
>
> Any ideas or comments?
>
> Thanks,
> Rick