I should point out something...
When you mention 'ARP cache', that says 'two machines on the same LAN'.
In other words, this won't protect you from a silly twit at TP tripping over things or touching power setups inappropriately.
(Point being that this only protects you against an individual node outage, but not against entire data centre outages.)
That type of high availability does work; I've done it on big iron at work (as well as geographic failovers on top of local failovers), though there's a lot of details involved and indeed requires multiple types of physical connections for the reasons you described.
You can probably get away with doing it over IP, but it may not be as reliable. The most dangerous situation is a 'split brain' scenario where both sides are up and working, but thinks the other has failed, and both proceeds to act as a master. Cleaning up after that kind of situation can be non-trivial, unless the setup was designed to make this an impossibility.
For most people with a setup like at Linode, just designing their applications to feed two hosts at same time (one in each data centre) and manually designating one as a master, at any given time, would probably work out to be the most cost effective and reasonable approach.
As always, the devil's in the details.
I'm not sure the original poster's idea for a single IP on two different Linodes is a good idea... because this means that if the data centre goes down, one now has no availability at all. And certain data centres are infamous for outages. *cough*