Email sucks, and it's incredibly broken!
I mean.... sending an email in Gmail is obviously quite trivial. But working with SMTP and email servers... not so much.
Most people take email for granted. It was invented way before I was born... way before Windows 95, Windows 3.1, or even DOS! What?!?!
Yeah... so not surprisingly, email sucks. Remember... DOS sucked and Windows 95 sucked (not at that time, of course). In fact, email sucks so much, it makes me sad that we still use it. It's almost embarrassing... almost as embarrassing as Internet Explorer.
Don't believe me? Just go to the Postfix mail server documentation page. Their website is straight out of the late-80's. I think there might be one image on the entire website? After all, we can't have those 4.8K modems downloading images... it will take too long, tie up your phone line, and waste too much bandwidth. :)
So, why am I complaining about email? Yeah... so all I wanted to do was send mail from a PHP program out to various people over the Internet. But, email these days has gotten a bit out of hand. I mean... there are A LOT of Viagra emails floating around, and so there are a lot of spam filters. My job is to ensure that emails get delivered without getting trapped by spam filters.
Then, I learned about SPF (Sender Policy Framework), which is basically a DNS record that tells mail servers who is allowed to send email on a domain's behalf. From what I understand, a lot of strict ISP mail servers do SPF record lookups to determine if a sender is valid. If no SPF record is found, and the IP address of the sender does not match an A or MX record on the domain, the email is rejected. In addition, many servers check the PTR record for the sender's domain to ensure that the IP address of the sender points back to it.
My situation looked something like this (IP addresses and domains are made-up):
32.0.0.1 (mail.example.com) was my mail server's Internet IP, running Postfix.
32.0.0.2 (www.foobar.com) was the web server's Internet IP, which was also running Postfix.
PTR records were setup for 32.0.0.1 to point to mail.example.com.
MX records were setup for example.com to point to mail.example.com
A record was setup for foobar.com to point to 32.0.0.2
MX record was setup for foobar.com to point to another mail server (i.e. mail.isp.net)
So, I cannot send mail directly from 32.0.0.2 because it will get rejected. There is no PTR record setup, and the MX records do not point to mail.example.com.
Instead, I thought it might be a good idea to relay the mail to mail.example.com over the LAN. So, I placed the LAN IP address of the web server into 'mynetworks' setting on the Postfix server on mail.example.com. Then, I set 'relayhost' on the Postfix server at www.foobar.com to point to the LAN IP address of the mail server. Great! mail.example.com is now relaying mail on behalf of www.foobar.com.
Now, the issue of the PTR record is taken care of... I need to setup a SPF record for 32.0.0.2. Now, mail.example.com is authorized to send mail on behalf of www.foobar.com. Great! Tested with 'mail' on the command-line! It worked!
Now I try to send mail using PHP on www.foobar.com. SPF did not pass. Why not? PHP inserted a Return-Path header into the email, which was not added by the 'mail' command line tool. Of course, the Return-Path looked something like: "Return-Path: www-data@ip-10-0-23-11.ec2.internal" The SPF validation failed because "ip-10-0-23-11.ec2.internal" is not a valid domain.
I decided to fix it with a PHP-specific configuration directive. In php.ini, I added:
mail.force_extra_parameters = "-f noreply@foobar.com"
Checkout the sendmail man page to see what the -f flag does. :)
This now overrides the Return-Path header. Boom! Now everything works again!
Other viable solutions:
I've done way too much research on the topic, AND I really just want to hang myself!
Anyway, the purpose of this post was mostly to complain about email, but I hope that it might possibly help anyone out there with a similar problem.
Other References:
http://www.php.net/manual/en/function.mail.php
http://ubuntuforums.org/showthread.php?t=804813
I mean.... sending an email in Gmail is obviously quite trivial. But working with SMTP and email servers... not so much.
Most people take email for granted. It was invented way before I was born... way before Windows 95, Windows 3.1, or even DOS! What?!?!
Yeah... so not surprisingly, email sucks. Remember... DOS sucked and Windows 95 sucked (not at that time, of course). In fact, email sucks so much, it makes me sad that we still use it. It's almost embarrassing... almost as embarrassing as Internet Explorer.
Don't believe me? Just go to the Postfix mail server documentation page. Their website is straight out of the late-80's. I think there might be one image on the entire website? After all, we can't have those 4.8K modems downloading images... it will take too long, tie up your phone line, and waste too much bandwidth. :)
So, why am I complaining about email? Yeah... so all I wanted to do was send mail from a PHP program out to various people over the Internet. But, email these days has gotten a bit out of hand. I mean... there are A LOT of Viagra emails floating around, and so there are a lot of spam filters. My job is to ensure that emails get delivered without getting trapped by spam filters.
Then, I learned about SPF (Sender Policy Framework), which is basically a DNS record that tells mail servers who is allowed to send email on a domain's behalf. From what I understand, a lot of strict ISP mail servers do SPF record lookups to determine if a sender is valid. If no SPF record is found, and the IP address of the sender does not match an A or MX record on the domain, the email is rejected. In addition, many servers check the PTR record for the sender's domain to ensure that the IP address of the sender points back to it.
My situation looked something like this (IP addresses and domains are made-up):
32.0.0.1 (mail.example.com) was my mail server's Internet IP, running Postfix.
32.0.0.2 (www.foobar.com) was the web server's Internet IP, which was also running Postfix.
PTR records were setup for 32.0.0.1 to point to mail.example.com.
MX records were setup for example.com to point to mail.example.com
A record was setup for foobar.com to point to 32.0.0.2
MX record was setup for foobar.com to point to another mail server (i.e. mail.isp.net)
So, I cannot send mail directly from 32.0.0.2 because it will get rejected. There is no PTR record setup, and the MX records do not point to mail.example.com.
Instead, I thought it might be a good idea to relay the mail to mail.example.com over the LAN. So, I placed the LAN IP address of the web server into 'mynetworks' setting on the Postfix server on mail.example.com. Then, I set 'relayhost' on the Postfix server at www.foobar.com to point to the LAN IP address of the mail server. Great! mail.example.com is now relaying mail on behalf of www.foobar.com.
Now, the issue of the PTR record is taken care of... I need to setup a SPF record for 32.0.0.2. Now, mail.example.com is authorized to send mail on behalf of www.foobar.com. Great! Tested with 'mail' on the command-line! It worked!
Now I try to send mail using PHP on www.foobar.com. SPF did not pass. Why not? PHP inserted a Return-Path header into the email, which was not added by the 'mail' command line tool. Of course, the Return-Path looked something like: "Return-Path: www-data@ip-10-0-23-11.ec2.internal" The SPF validation failed because "ip-10-0-23-11.ec2.internal" is not a valid domain.
I decided to fix it with a PHP-specific configuration directive. In php.ini, I added:
mail.force_extra_parameters = "-f noreply@foobar.com"
Checkout the sendmail man page to see what the -f flag does. :)
This now overrides the Return-Path header. Boom! Now everything works again!
Other viable solutions:
- Set 'myorigin' variable in Postfix to a valid domain that has a SPF record allowing mail.example.com to send emails
- Similarly, transform local addresses like "www-data@localdomain.net" to "noreply@validdomain.com." Postfix can do this -- check out smtp_generic_maps and this link.
- Use PHP's sendmail_path variable to override the Return-Path header. Check out this page.
I've done way too much research on the topic, AND I really just want to hang myself!
Anyway, the purpose of this post was mostly to complain about email, but I hope that it might possibly help anyone out there with a similar problem.
Other References:
http://www.php.net/manual/en/function.mail.php
http://ubuntuforums.org/showthread.php?t=804813
Comments