What is the correct way to set up standalone postgresql server?

I am on debian 7.5 using postgresql 9.3

I want to connect app server to database server. I have installed posgresql 9.3 on both app server and database server.

On the database server, I have created the database user and database name. I have tried to connect localy and it worked.

Then, I tried to connect from app server to database server, and it did connect again. However, private IP doesnt work, public IP does.

However, when I launched the application it is trying to connect the app server's IP and it is looking for the username and database in the app server.
I have checked pg_hba.conf in both app and database server.

please, advise.

Thank you.