How to set WordPress to send email?

I probably won't be much help, because I think servers should use CentOS (proper linux server) instead of Ubuntu (desktop-like linux). But just in case, check to see if there is an MTA installed, like postfix or exim. Wordpress usually posts the email via the local MTA, which does the actual transport of the email. If there is, then check for permission restrictions, like SELinux (if that even exists in Ubuntu, not sure).

Also, you mention that you transferred from shared hosting, so check to see if you have any settings relevant to your old hosting, that don't match with the new.

Thanks, I'll check the difference. Appreciated for your advice.

Ubuntu is a perfectly fine server OS. As noted, you probably need to install and configure a Mail Transfer Agent. I tend to recommend Postfix as it is reasonably common, secure, and straightforward to set up. While the previous link gives basic info, this guide walks you through things step by step if you need more guidance.