Server :: Postfix Setup To Block Spam From / To Own Address

I have set up a couple of postfix servers for my domains, but the only thing I am missing now is this: How to block the public sending mail from my email to my email? I have managed it with SPF, but surely there must be a better way, that returns "relay not allowed" to the client. The SPF method costs too much, since it must make a dns request for each mail. So far, I have not gotten many of these mails, unless when testing my mailserver, but as I see it, anyone should not be allowed to send mail from abuse@mydomain.com to abuse@mydomain.com.

Somebody must have thought about this a long time ago, and there is simply that little line in main.conf that I'm missing.. My setup is this (virtual): I have a primary mx, with postfix, courier IMAP/POP3 server, a user database, and sasl via saslauthd. I also have a secondary mail server (backup mx) with no sasl auth, but with a copy of the mail users in the virtual tables, but added as relay_* users instead.

Spamassassin and SPF testing is replicated too, so most of the stuff should work, but I simply cannot find a setting in Postfix that denies someone to mail FROM my address TO my address. When mailing from my address and out in the open, they are required to authenticate, but not when using one of my my domain addresses, and also targeting my domain addresses.

As said, It is possible with a strict SPF setting, but that is at a cost for every lookup. It would be quicker for postfix to lookup the sender and the recipient in the relay/virtual tables, and deny if both addresses were in the recipient tables, and sender is not authenticated.