Security :: IPtables Port 25 Connection Limit Without Blocking Barracudas

I am at a loss how to prevent Denial of Service attacks to port 25 and not block legitimate connections from 2 Barracuda 800(s) and block smart phones such as iPhones/Blackberrys/iPhones that use the server smtp.server.com for email.
Presently for port 25
RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 25 -j ACCEPT

The 2 Barracuda 800(s) make port 25 connections all the time, plus users with smart_phones have the incoming server type:
IMAP
pop.server.com
smtp.server.com

Is there a way to keep Denial of Service attacks from happening with iptables rules without causing blocking to the Barracuda(s) that make constant port 25 connections & smart phones that poll? I was thinking if I allowed the Barracuda(s) in these lines
-s (barracuda)24.xx.xx.xx -d (emailserver)24.00.xx.xx -p tcp -m tcp --dport 25 -m state --state NEW -j ACCEPT

Where the source would be the Barracuda going to the email server. It would be allowed, then I am left with how to allow other connections like Smart_Phones that connect via Port 25. I am thinking if I put rules in place doing connection counts in a minute it would result in errors connecting to the server and people would start complaining. Plus any limiting may result in blocking real traffic. Then would I need to allow the ISP range in the above example to accept port 25, I am still left with how to drop a flood/denial of service attack.