How should I deal with smtp_relay_restrictions in postfix?

Question

I just setup a mail server with postfix and dovecot, now I'm having trouble sending emails with evolution.

e.g When I send an email to to@gmail.com, I got an error:

NOQUEUE: reject: RCPT from unknown[MY_IP_ADDRESS]: 450 4.3.2 <to@gmail.com>: 
Recipient address rejected: Try again later; from=<contact@XX> 
to=<dest@gmail.com> proto=ESMTP helo=<[IP_ADDRESS]>

I googled a little bit and the problems seems to be:

smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination

I can add my current IP address to mynetworks, and it worked. But what if I'm using it from another location? So I need this to work everywhere.

Then I tried to set smtpd_relay_restrictions to an empty value, and smtpd complains

 postfix/smtpd[16217]: fatal: in parameter smtpd_relay_restrictions or 
 smtpd_recipient_restrictions, specify at least one working instance of: 
 reject_unauth_destination, defer_unauth_destination, reject, defer, defer_if_permit or check_relay_domains

I don't really know what should I do now, If I add 0.0.0.0/0 or something similar in mynetworks is it going to be a problem? (Become a SPAM server perhaps?)

P.S I'm running postfix 2.11 with Ubuntu 14.04

postconf -n

#
# I replaced the ip address of my VPS to "VPS_IP"
# MYDOMAIN.com for my email domain
# 

alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases
append_dot_mydomain = no
biff = no
config_directory = /etc/postfix
home_mailbox = Maildir/
inet_interfaces = all
mailbox_size_limit = 0
mydestination = MYDOMAIN.com, localhost.vpsserver.com, localhost
myhostname = MYDOMAIN.com
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 VPS_IP
myorigin = /etc/mailname
readme_directory = no
recipient_delimiter = +
relayhost =
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtpd_banner = $myhostname ESMTP
smtpd_recipient_restrictions = permit_sasl_authenticated
smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
smtpd_tls_cert_file = /etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file = /etc/ssl/private/ssl-cert-snakeoil.key
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_use_tls = yes
virtual_alias_maps = hash:/etc/postfix/virtual