A recent small study by Joy Hawkins showed that if you remove the address from your Google Business Profile listings your rankings in the local pack and in Google Maps will significantly suffer.