• SupportEmail

  • SPF record warning from dmarcanalyzer -- anything to worry about?

My SPF record is v=spf1 include:spf.opalstack.com include:_spf.google.com ~all, but I do get a new warning from https://www.dmarcanalyzer.com/spf/checker/ since migrating:

Problems
We've noticed the following problems with your SPF records.

Warning : The maximum amount of 10 lookups exceeded. ISPs could ignore your SPF record

It does seem like the spf.opalstack.com record requires more NS lookups than Google or WF, but I am a bit out of my league to suggest it's a real problem or what the solution is.

I am seeing a lot more of my mail-forwarded-to-gmail being classified as spam, but that may just be an issue until gmail is trained.

Will hopefully migrate away from gmail soon enough with Roundcube looking pretty nice.

Thanks!

  • sean replied to this.

    I've also noticed this issue. It appears we have similar configurations where we have a hosted domain forward e-mail to a gmail account. I think what's happening is that when the forward occurs, the mail picks up the opalstack ip address, which then fails when doing a source lookup on the original e-mail. For instance, if I get an e-mail from reddit, it gets forwarded to my gmail account, but then SPF, DMARC and DKIM fail because the IP address being looked up belongs to opalstack and not reddit. (open the offending e-mail click the three dots and choose "show original") I observed this issue toward the end of my Webfaction days. Feels like this might be due to gmail tightening down on spam. But, I'm a programmer, not a network engineer, so I'll defer to the fine team at opalstack to see if they have additional information.

    I have noticed my outgoing e-mail no longer gets marked as spam ever since I configured gmail to use the opalstack smtp servers for sending mail out.

    Yep, as far as spam goes, I think you're right.

    The email logs indicate that gmail is checking as if the opalstack mx server is sending the email from the origin domain, which it's not authorized to do. Thus, SPF fail. I went back to look at some old emails sent the same way through WF and it had pretty much the same thing in the logs, but maybe gmail had been trained to accept more of those messages. Maybe the situation will improve as I persist in not-spamming emails in gmail.

    I don't know the correct way around this (i.e. having SPF checks succeed for forwarded mail), other than buying into GSuite and having email go there directly.

    db122 Warning : The maximum amount of 10 lookups exceeded. ISPs could ignore your SPF record

    That might be it. If I'm doing this correctly:

    1. include:spf.opalstack.com: lookup SPF for our spf.opalstack.com which contains mx a:smtp2.us.opalstack.com a:smtp1.de.opalstack.com
    2. Lookup the A record for a:smtp2.us.opalstack.com
    3. Lookup the A record for a:smtp1.de.opalstack.com
    4. Lookup the MX for spf.opalstack.com
    5. Lookup the A record for each MX (so that's 4 more lookups)

    That's 8 total lookups for our SPF, Google's is 3 so that puts you up to 11.

    Here's a SPF you can try that includes all of the IPs for our stuff instead of a bunch of lookups, can you try that and let me know if it resolves the issue?

    v=spf1 ip4:178.162.221.165 ip4:46.165.236.26 ip4:178.162.221.137 ip4:23.106.47.103 ip4:23.82.16.244 ip4:108.62.123.121 include:_spf.google.com ~all

      Adding the record definitely addresses the warning issues by dmarcanalyzer. Probably a good change in any case.

      It doesn't change the gmail spam issue for emails sent to my domain and forwarded to my gmail. All my inbound emails have the SPF check=softfail or neutral, depending on the SPF policies of the originating domain. I guess my SPF record has nothing to do with this behavior. I think it's the same behavior as WF but my gmail just isn't trained to accept everything from your MX server IP's. Yet. 🙂 I will persist.

      15 days later

      FYI we've updated our SPF endpoint to eliminate the extra DNS lookups.

      18 days later

      Just wanted to chime into the discussion with some suggestions.

      Forwarding is generally a bad idea exactly due to the issues mentioned above - the forwarding server is often not authorized (via e.g. SPF) to send email on behalf of the actual sender. This can in theory be mitigated using SRS, but in my experience this isn't always 100% reliable, some MTAs don't support it at all, and I personally prefer to avoid rewriting headers when possible.

      A better setup is to not forward emails at all, but rather configure Gmail as a POP3 client using the "Check mail from other accounts" setting in the "Accounts and Import" configuration section.

      Unfortunately, Gmail cannot act as an IMAP client, but it can be configured to leave the messages on the server after downloading, which is still a good solution.

        levacic I reached the same conclusions in the abstract, however in practice, Gmail as a POP3 client was showing ~ 20min latency due to really low and non-configurable polling interval. Some people say it "learns" how often poll over time -- but it's really not good enough.

        The good news is that after switching procmail for forwarding, I'm able to filter most spam before sending to Gmail and the false-positive rate is far, far lower. Unfortunately, it still fails SPF checks in GMail, but ultimately Gmail is just ignoring those now (maybe I've trained it to do so) and using its own heuristics for spam detection. It's back to behaving as well as it was for 10yrs over at WebFaction.

            That's true about the delay, but I'm in the "anything sent via email is not urgent" camp, so I don't mind, and I usually see delays of up to 5 minutes which is fine. If I'm waiting for something "immediate" such as verification emails and similar, I'll just manually refresh (the Gmail refresh button also triggers a check of all the external POP3 accounts as well) until I see what I'm waiting for - then I also don't have to rely on Gmail being trained to ignore SPF fails.

            Gmail acts as a black box in the forwarding setup, because you don't really know exactly how they handle this internally and have no control over the process - does it depend on the sender's SPF configuration (e.g. SOFTFAIL vs. FAIL), does it learn that the forwarding server is trusted by you, etc.

            I use Opalstack (and previously WebFaction as many here) for personal use, but I understand that in a business context you might care about those delays - though I'd generally prefer a more reliable setup in that situation anyway. Happy you've got it setup the way you want though, and hopefully this discussion might help someone else who stumbles onto it!

            db122 it still fails SPF checks in GMail

            For procmail forwarding you might be able to fix that by adding the IP of the IMAP server (ie the machine that does the forwarding) to your SPF, like:

            • USA: v=spf1 include:spf.opalstack.com ip4:209.58.145.201 ~all
            • DE: v=spf1 include:spf.opalstack.com ip4:91.109.16.70 ~all

            (Edit: do not do this. The IMAP server IPs have changed in the time since this post was made, and they are included by default in Opalstack's SPF endpoint so it's not necessary to specify them individually).

              @sean I think we might be talking about a different scenario.

              If alice@example.com (not hosted at Opalstack) sends an email to bob@example.net (hosted at Opalstack, with configured forwarding to bob@gmail.com), this means that when forwarding, Opalstack's servers will be sending the email to Gmail on behalf of example.com - which is not hosted at Opalstack.

              In this scenario we're Bob and we don't control Alice's SPF records, so if Alice configures example.com's SPF records to FAIL (ie. -all), we're relying on Gmail ignoring the SPF rule and accepting the email anyway (which it often does, due to this non-standards-conforming setup being widely used in practice). We only control example.net, but this domain is not the sender domain, so our SPF records shouldn't matter.

              • sean replied to this.

                Ever since Opalstack rejiggered the SPF records, I've noticed less false positives. That could be that the re-training is kicking in.

                My problem is that I've been using Gmail for over a decade. It's convenient, timely and free (but mostly convenient). The only other option I see is getting the Google Suite and changing the MX on all my domains. If anyone has some Gmail alternatives they like, paid or otherwise, I'm listening.

                  adam_w Switched over to fastmail recently and have had nothing but great things to say about it. I feel awkward just randomly sharing an unsolicited referral code, but it gives a 10% discount so if you can't find that elsewhere I can share that here.

                    Mastodon