First, make sure your JobScore careers site is hooked up to your corporate website.
Second, turn off your standard jobs email address (usually email@example.com or firstname.lastname@example.org) and/or bounce back the following text:
Thanks for sending us email, but we just wanted to let you know that we won't be reading it.
If you would like to apply for a job at [your company name here] please review and apply for one of our open jobs at [your careers site URL here]
[your name here]