Windows DNS fails with SPF/TXT too long

I run my own DNS for some of my domains. My primary DNS is the built-in DNS service in Windows 2003 SP1 and as backup/secondary DNS I’m using a free service from Twisted4Life. I’ve been using SPF for a couple of years and it has been working fine.
(SPF – Sender Policy Framework – is way to prevent spam by entering into your DNS what SMTP servers are allowed to send email for your domain. Some email systems such as Hotmail rely in part on SPF when deciding if your email should be placed in the recipients Inbox or Junk mail folder.)

My ISP has been recommending me to use a new SMTP server when sending out email from one of my email lists. So two days ago I added a few new SMTP servers to my SPF record in my DNS (by updating the DNS file in my \Windows\System32\DNS directory and then doing Reload from the DNS console, dnsmgmt.exe) A quick look and everything seemed just fine.

Normally when sending my email list, which contains a bit over 20k recipients, it takes about 10-20 minutes to actually send the batch and then during the next 10 minutes I get about 4-500 error emails telling me stuff like “user is over quota”, “no such user” (I use these to clean my email list after each batch).
This time I only got about 15 error messages per hour and it took 36 hours before all had arrived in my inbox!
During this 36 hour period I called my ISP and told them there has to be something wrong with the new SMTP server that they had recommended I switch to. I also did send a test batch using the old SMTP server and it was lightning fast as always.

Today I noticed that I had not recieved any emails from a colleague who is using another email system than me. He told me that he sent me several emails yesterday and this morning and had not recieved any error messages. I told him to call his ISP because my email was working fine, I had emailed back and forth with several people and had no problem.

Then this evening it struck me – I had completely forgot that I made changes to my DNS records. Could I have put something wrong in my SPF records? But why would that effect my incoming email?

First I just did a nslookup using centralops.net online nslookup. This tool reported “Name server failed” for my domains! I quickly tried another online nslookup tool from network-tools.com and this tools reported no errors, it worked fine.

I went in to my server and opened the DNS console. And there, on the line showing the TXT record I use for my SPF string there was a blank space about ten characters into the string, in the middle of one of my SMTP server entries. At the end of the string there was some kind of special character, represented by a “square box”. I then looked at the DNS data file at \Windows\system32\DNS but that still looked exactly like it should, without any blank spaces or special characters. Strange…

I then updated the data file from the DNS console (instead of the other way around that I usually do).
When I checked the DNS data file, my TXT record was split into several lines, with line breaks between them, each line starting and ending with a quotation mark. However some characters were missing at the line breaks so I reverted back to the original errornous case, ie where the DNS data file looks fine but in the DNS console there is a blank space and a special character.

Then I went to kitterman.com’s SPF validation tool to validate my SPF string. The actual string validated fine. But then, using the same tool, I asked it to validate my domain using a DNS lookup it failed.

Finally, I deleted a few old SMTP servers from my SPF string in the DNS data files and did Reload on the DNS console. Ran my checks again and everything worked fine. And as I’m typing this blog post emails up to 48 hours old have started to enter my inbox.

My conclusion is that using a string that is too long (I have not done any extensive search on this problem, so I don’t know how long a string is allowed to be) in a TXT record corrupts the string. This is obviously a bug Windows DNS server.
But the thing that is strange to me is that to some DNS clients, the answer my DNS server sent them was ok and to some unacceptable. I also don’t understand why the SMTP servers reacted this way – no delivery I could understand, but extremely slow delivery?

Well, I’m a novice at both DNS and SMTP, so to others this may be crystal clear – please enlighten me!