I know I am reviving an old thread, but my problem is I didn't touch my server's config, didn't install anything, didn't do nothing besides uploading files through ftp or subversion.
So, what can this mean?

According to what google tells me, those are the possible reasons
1 - You have re-installed you system and trying to ssh to the newly installed system.
2 - You have assigned the IP address of one system to another system and trying to ssh.
3 - You system is dual boot with different ssh keys in both flavors of linux.
4 - You are using an IP for load balancing and trying to ssh to the same IP.
5 - You generated new ssh keys for your system.
6 - Someone trying to do some nasty things, or you can say man-in-the-middle attack.

I didn't 1,2,3,4, or 5.
There is only 6 left?

There was a dns cache poisoning recently, as in last few days. Maybe your domain was impacted?

Maybe. How can I know?
Should I just add the new ip to my whitelist and forget about it, or could it be dangerous?

If it is not your IP then don't white list it.
Look up who owns the IP.