There isn't really an 'undo'. The only reliable thing (as far as I know) is start again ... or backup your filesystem prior to updating (see below).
The following command will list the pending updates:
Code:
apt-get -u upgrade --assume-no
You can review the list and decide whether or not you want to update them (individually or all).
I've found that Ubuntu 12.04 (and probably other releases, though I have no experience with them) does a good job of not feeding you updates that will break things. That being said, if anything on your VPS relies on a specific version of something you'd better make sure you don't blindly update.
When I'm not sure about an update I make a copy of my filesystem (I try to keep my active filesystem small enough that I can create a copy before I do something that may cause issues). That way I have a system-wide restore point. I do have to shutdown for a short period of time to create the copy, so I have to make sure to time it well (5am seems to be a good time based on our clients and site visitors).