How to solve SPF alignment issues?

ppo

My configuration seems good and everything seems to work fine except for SPF alignment; not always but most of the time.
Any idea why? And how to fix that?

Here's my config:

example.com: v=spf1 include:spf.opalstack.com include:_spf.google.com ~all
dkim._domainkey.example.com: v=DKIM1; k=rsa; p=… (cf. ℹ️ on example.com in the Dashboard)
_dmarc.example.com: v=DMARC1; p=none; rua=mailto:dmarc@example.com

A few days ago, I added ip4:91.109.16.70 to the SPF record but it doesn't help. (IP for DE as found here)

My main email address (me@example.com) is redirected to my Gmail account (me@gmail.com).
I configured another address and a catch-all that are redirected to my main email (me@example.com, then to Gmail).

In Gmail, I configured my main address to use Opalstack's SMTP (smtp.de.opalstack.com).
I send emails exclusively from Gmail.

Thank you.

sean

ppo what indication of a SPF alignment issue are you seeing, ie what is the exact error message or problem that you see?

ppo

2 examples of errors in the reports…

header_from ≠ envelope_from

  <record>
    <row>
      <source_ip>209.85.167.48</source_ip>
      <count>1</count>
      <policy_evaluated>
        <disposition>none</disposition>
        <dkim>pass</dkim>
        <spf>fail</spf>
      </policy_evaluated>
    </row>
    <identifiers>
      <envelope_to>rtbf.be</envelope_to>
      <envelope_from>gmail.com</envelope_from>
      <header_from>example.com</header_from>
    </identifiers>
    <auth_results>
      <dkim>
        <domain>example.com</domain>
        <selector>dkim</selector>
        <result>pass</result>
      </dkim>
      <spf>
        <domain>gmail.com</domain>
        <scope>mfrom</scope>
        <result>pass</result>
      </spf>
    </auth_results>
  </record>

DKIM & SPF fail. Type: local_policy, Comment: arc=pass

  <record>
    <row>
      <source_ip>209.85.220.69</source_ip>
      <count>1</count>
      <policy_evaluated>
        <disposition>none</disposition>
        <dkim>fail</dkim>
        <spf>fail</spf>
        <reason>
          <type>local_policy</type>
          <comment>arc=pass</comment>
        </reason>
      </policy_evaluated>
    </row>
    <identifiers>
      <header_from>example.com</header_from>
    </identifiers>
    <auth_results>
      <dkim>
        <domain>maxotek-com.20221208.gappssmtp.com</domain>
        <result>pass</result>
        <selector>20221208</selector>
      </dkim>
      <spf>
        <domain>maxotek.com</domain>
        <result>neutral</result>
      </spf>
    </auth_results>
  </record>

I just found the following. So I guess it's normal that Gmail defines its domain in the envelope from, and there's therefore nothing we can do? 🫤

Why SPF alignment fails
If you're using an email marketing service to send emails, the "Envelope From" domain is the domain of the marketing service. It is the domain used to receive bounce error messages from recipient systems. In your case, Zoho Campaigns uses "zcsend.net" as the "Envelope From" domain to collect the bounce error messages so we can display the bounce rate and reasons in your campaign report. During the DMARC check, this "Envelope From" domain will not match with the actual sender's business domain, so SPF alignment will fail. This happens not only with Zoho Campaigns but with any email marketing service.

Is it useful/better to specify ip4:91.109.16.70 in the SPF record?

sean

ppo So I guess it's normal that Gmail defines its domain in the envelope from, and there's therefore nothing we can do? 🫤

I'm afraid so. There's no DNS mechanism that can fix this type of alignment problem.

Is it useful/better to specify ip4:91.109.16.70 in the SPF record?

No. IP 91.109.16.70 is no longer part of our infrastructure, and in the time since that post was made we added the IMAP server IPs to the spf.opalstack.com endpoint so they're already included.

I'll update that post to mention it.

ppo

One more question about DMARC. (also for reference/documentation)

I've been monitoring my config for a few weeks and everything seems to work fine (except the alignment problem that can't be solved). So I guess I can remove the rua=mailto:dmarc@example.com (I won't no more read them anyway 😂).

But what are your advice about the p? Is it better to set it as p=quarantine? And safe or it'll lead to problems (by gradually increasing the pct value)?

Policy (p): The required "p" tag demonstrates the policy for domain (or requested handling policy). It directs the receiver to report, quarantine, or reject emails that fail authentication checks. Policy options are: 1) None 2) Quarantine or 3) Reject. (more info)
Percentage (pct): This DMARC tag specifies the percentage of email messages subjected to filtering. For example, "pct=25" means a quarter of your company’s emails will be filtered by the recipient. (more info)
Note: The "pct" tag's primary use is allow a domain owner to gradually test moving a DMARC policy from a policy of p=none to a policy of quarantine or reject (p=quarantine or reject) by testing a small sampling of messages that fail DMARC so that legitimate messages that have delivery issues can be caught early before impacting all legitimate email from that domain.

Source: MX Toolbox: What is a DMARC DNS Record?

sean

ppo But what are your advice about the p? Is it better to set it as p=quarantine? And safe or it'll lead to problems (by gradually increasing the pct value)?

There is no "better" or "worse" setting for these options. Set p to the policy option that you want and use pct to test your changes as described.

ppo

I got this answer on the Gmail forum:

You will need to check with the other SMTP service, to ask how they generate their envelope From. It is normally the account you use to authenticate with the server, or a system address at the same domain. They should be able to help you.

To be sure to understand correctly… (and maybe other reading this later)
Who's "responsible" for the envelope (or creating it)? The application that sends the mail or the SMTP server? Apparently it's the app (Gmail) that define the Envelope From as my Gmail address.

That said, my Opalstack mailbox (used to authenticate with the SMTP server) is not linked to any email address, so even if it wanted, it couldn't know which address to define as Envelope From.

sean

ppo envelope-from is set by the mail transport agent (ie the SMTP server) which usually sets it based on information from the mail client app (in this case Gmail).

I've set up an Opalstack SMTP account on my own Gmail and I don't see any SPF issues when sending mail. For example here's the header from the SPF check showing that envelope-from was set to my own address (I've replaced the recipient and sending domains here):

Received-SPF: pass (mx.RECIPIENT.com: domain of sean@SENDER.com designates 
23.106.47.103 as permitted sender) receiver=mx.RECIPIENT.com; client-ip=23.106.47.103; 
helo=smtp2.us.opalstack.com; envelope-from=sean@SENDER.com

My account settings in Gmail look like this, with the alias option checked and no different reply address:

ppo

Sorry, I'm still trying to understand but… 🙄

My Gmail config is different. Is the "not an alias" the reason for SPF misalignment?

I unchecked the alias option a long time ago because otherwise people replied on my Gmail address, not on the one used in my initial mail. I tested that now and it doesn't seem to be the case anymore.

I tested with different from addresses: Gmail, domain1 (default, not alias), domain2 (not alias), domain3 (alias). They all (except the Gmail address of course) have the same DNS config on Opalstack and use the same Opalstack SMTP. On the receiving end (Proton), they all have the "same" headers and everything seems to be working fine and as expected[*]. So alias or not, it looks the same?

But these are the mail headers, we need the envelope information to see if they're aligned. Can I find that somewhere?

Return-Path: <ME@EXAMPLE.com>
From: Pascal <ME@EXAMPLE.com>

Authentication-Results: mailin036.protonmail.ch; dkim=pass (Good 1024
    bit rsa-sha256 signature) header.d=EXAMPLE.com header.a=rsa-sha256
Authentication-Results: mailin036.protonmail.ch; dmarc=pass (p=none dis=none)
 header.from=EXAMPLE.com
Authentication-Results: mailin036.protonmail.ch; spf=pass smtp.mailfrom=EXAMPLE.com
Authentication-Results: mailin036.protonmail.ch; arc=none smtp.remote-ip=12.34.56.78
Authentication-Results: mailin036.protonmail.ch; dkim=pass (1024-bit key)
 header.d=EXAMPLE.com header.i=@EXAMPLE.com header.b="d2G6pxgR"

X-Spam-Status: No, score=-0.10
X-Rspamd-Queue-Id: 4RGHMp09s5z9ttBV
X-Rspamd-Server: cp5-mailin-036.plabs.ch
X-Spamd-Result: default: False [-1.00 / 25.00];
 DMARC_POLICY_ALLOW(-0.50)[EXAMPLE.com,none]; R_SPF_ALLOW(-0.20)[+ip4:12.34.56.78];
 R_DKIM_ALLOW(-0.20)[EXAMPLE.com:s=dkim];
 MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_HAM(-0.00)[-0.807];
 FROM_EQ_ENVFROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~];
 RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; TO_DN_ALL(0.00)[]; ASN(0.00)[asn:28753,
 ipnet:46.165.236.0/24, country:DE]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[EXAMPLE.com:+];
 FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[];
 PREVIOUSLY_DELIVERED(0.00)[ME@proton.me]; RCVD_COUNT_TWO(0.00)[2]

PS: I modified my DMARC config to ruf instead of rua for the reports.
rua: reporting URI(s) for aggregate data.
ruf: message-specific forensic information (i.e., comma-separated plain-text list of URIs).

[*] Everything as expected with the exception of that annoying thing… When I reply to all, my custom email is well set as From but it's also always added in Cc. But I guess that's just how Gmail decided to handle things 🤷‍♂️🫤

sean

ppo Is the "not an alias" the reason for SPF misalignment?

I don't know. I know that with the settings described in my previous reply, replies to my messages sent from sean@SENDER.com go to sean@SENDER.com and not to my Gmail address, and that if I "reply all" my Gmail address is not included in the recipients.

ppo So alias or not, it looks the same? But these are the mail headers, we need the envelope information to see if they're aligned. Can I find that somewhere?

In your example "smtp.mailfrom=EXAMPLE.com" is where you see the envelope domain. There's a chart explaining the various pieces of the Authentication-Results header at: https://www.iana.org/assignments/email-auth/email-auth.xml

ppo

Thank you @sean.