This is the job of your firewall, not fail2ban. fail2ban just watches your kernel logs for probes on your SSH and then sends commands to the firewall to start blocking those IP addresses.
If you want to whitelist specific IP addresses then you have no need for fail2ban... because all (non-whitelisted) IP addresses will already be blocked by the firewall.
On Ubuntu the easiest firewall choice for admins with rudimentary knowledge is
Uncomplicated Firewall or "ufw". The guide I linked there will give you a few quick lines you can use to setup ufw, but in particular you are interested in is example 6:
Code:
sudo ufw allow proto tcp from 192.168.0.2 to any port 22
... where you replace 192.168.0.2 with an ip address you want to whitelist.
The
Linode Iptables Guide also has an example of whitelisting IP addresses for SSH, but uses iptables directly instead of a beginner friendly syntax like ufw.