Second homes could be contributing to small business’s customer decline