Set Up a Mail Server with Postfix

Running your own mail server gives you full control over email routing, storage, and privacy—but it demands careful configuration. A misconfigured mail server becomes a spam relay or lands in everyone's junk folder. This guide builds a production-grade stack: Postfix as the MTA, Dovecot for IMAP, OpenDKIM for signing, and proper DNS records for SPF and DMARC. It assumes you have a dedicated hostname (mail.example.com) pointing to a static public IP.

Prerequisites and Architecture

Before touching packages, confirm these are in place:

• A VPS or dedicated server with a static IPv4 (and ideally IPv6) address.
• Reverse DNS (PTR record) set on your IP to match your mail hostname—contact your provider if needed. This is non-negotiable for deliverability.
• An MX record for your domain pointing to mail.example.com, plus an A record for mail.example.com itself.
• Ports 25, 465, 587, 993 open in your cloud/hardware firewall and not blocked by your ISP (many residential ISPs block port 25; use a VPS).

Step 1 — Install Packages

Debian / Ubuntu

sudo apt update
sudo apt install postfix postfix-pcre dovecot-core dovecot-imapd \
  opendkim opendkim-tools spamassassin -y

The Postfix installer will prompt for mail configuration type. Choose Internet Site and enter your domain (example.com).

Fedora / RHEL 9 / Rocky 9

sudo dnf install postfix dovecot opendkim -y
sudo systemctl enable --now postfix dovecot opendkim

Arch Linux

sudo pacman -S postfix dovecot opendkim

Step 2 — Configure Postfix (main.cf)

Edit /etc/postfix/main.cf. The key directives are shown below—add or update these; leave distro defaults for anything not listed here.

sudo nano /etc/postfix/main.cf

# Identity
myhostname = mail.example.com
mydomain = example.com
myorigin = $mydomain

# Network
inet_interfaces = all
inet_protocols = all
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128

# TLS (certificates from Let's Encrypt — adjust paths)
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file  = /etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_security_level = may
smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1
smtp_tls_security_level = may
smtp_tls_loglevel = 1

# Submission port auth (works with master.cf entry below)
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_recipient_restrictions =
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_unauth_destination

# Mailbox
home_mailbox = Maildir/
mailbox_size_limit = 0
recipient_delimiter = +

Enable Submission (Port 587) in master.cf

Uncomment or add the submission block in /etc/postfix/master.cf:

submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_tls_auth_only=yes
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject

Step 3 — Configure Dovecot for IMAP and SASL

Dovecot handles both IMAP delivery and the SASL socket Postfix uses to authenticate outbound submissions. Edit /etc/dovecot/conf.d/10-mail.conf:

mail_location = maildir:~/Maildir

In /etc/dovecot/conf.d/10-auth.conf, ensure plain auth over TLS is allowed:

disable_plaintext_auth = yes
auth_mechanisms = plain login

In /etc/dovecot/conf.d/10-master.conf, expose the SASL socket to Postfix:

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}

Configure TLS in /etc/dovecot/conf.d/10-ssl.conf:

ssl = required
ssl_cert = </etc/letsencrypt/live/mail.example.com/fullchain.pem
ssl_key  = </etc/letsencrypt/live/mail.example.com/privkey.pem
ssl_min_protocol = TLSv1.2

Create a system user and their Maildir for testing:

sudo useradd -m -s /bin/bash testuser
sudo passwd testuser
sudo mkdir -p /home/testuser/Maildir/{new,cur,tmp}
sudo chown -R testuser:testuser /home/testuser/Maildir

Step 4 — Set Up OpenDKIM

DKIM signs outgoing mail so receivers can verify it wasn't tampered with in transit.

Generate a Key Pair

sudo mkdir -p /etc/opendkim/keys/example.com
sudo opendkim-genkey -b 2048 -d example.com -D /etc/opendkim/keys/example.com \
  -s mail -v
sudo chown -R opendkim:opendkim /etc/opendkim/keys

This creates mail.private (keep secret) and mail.txt (publish in DNS).

Configure OpenDKIM

Edit /etc/opendkim.conf:

Mode                  sv
Subdomain             no
AutoRestart           yes
AutoRestartRate       10/1M
LogWhy                yes
Syslog                yes
UMask                 002
Domain                example.com
KeyFile               /etc/opendkim/keys/example.com/mail.private
Selector              mail
Socket                inet:8891@localhost
TrustedHosts          /etc/opendkim/TrustedHosts

echo -e "127.0.0.1\nlocalhost\nexample.com" | sudo tee /etc/opendkim/TrustedHosts

Tell Postfix to use OpenDKIM as a milter by appending to /etc/postfix/main.cf:

milter_default_action = accept
milter_protocol = 6
smtpd_milters = inet:localhost:8891
non_smtpd_milters = inet:localhost:8891

Publish the DKIM DNS Record

Print the public key:

sudo cat /etc/opendkim/keys/example.com/mail.txt

Create a TXT record at mail._domainkey.example.com with the value shown (the long p= string). DNS propagation takes minutes to hours.

Step 5 — Add SPF and DMARC DNS Records

SPF tells receiving servers which IPs are allowed to send mail for your domain. Add a TXT record on example.com (the bare domain):

# DNS TXT record value — adjust the IP to your server's public IPv4
"v=spf1 ip4:203.0.113.10 -all"

If you also use IPv6:

"v=spf1 ip4:203.0.113.10 ip6:2001:db8::1 -all"

DMARC ties SPF and DKIM together and tells receivers what to do on failure. Add a TXT record at _dmarc.example.com. Start with p=none to observe before enforcing:

"v=DMARC1; p=none; rua=mailto:[email protected]; ruf=mailto:[email protected]; fo=1"

After reviewing a week of aggregate reports, move to p=quarantine, then p=reject.

Step 6 — Firewall Rules

UFW (Debian/Ubuntu)

sudo ufw allow 25/tcp
sudo ufw allow 587/tcp
sudo ufw allow 993/tcp
sudo ufw reload

firewalld (Fedora/RHEL/Rocky)

sudo firewall-cmd --permanent --add-service=smtp
sudo firewall-cmd --permanent --add-service=smtps
sudo firewall-cmd --permanent --add-service=imaps
sudo firewall-cmd --reload

Step 7 — Start and Enable Services

sudo systemctl enable --now postfix dovecot opendkim

Verification

Test SMTP auth and delivery with swaks (Swiss Army Knife for SMTP):

sudo apt install swaks   # Debian/Ubuntu
swaks --to [email protected] --from [email protected] \
  --server mail.example.com --port 587 --auth LOGIN \
  --auth-user testuser --auth-password yourpassword --tls

Check DKIM signing in the received email headers—look for DKIM-Signature and a dkim=pass authentication result. Use MXToolbox or mail-tester.com for a full deliverability score.

Verify Dovecot IMAP with openssl:

openssl s_client -connect mail.example.com:993 -quiet

You should see a * OK Dovecot ready banner.

Troubleshooting

• Mail deferred / connection refused on port 25: Most consumer ISPs and some cloud providers (AWS EC2, Oracle Cloud) block outbound port 25. Check with your provider or use their SMTP relay.
• DKIM signature not applied: Confirm opendkim is running (systemctl status opendkim), the socket matches what Postfix expects, and file permissions on mail.private are 600 owned by opendkim.
• Authentication failure in Postfix: Make sure the Dovecot SASL socket at /var/spool/postfix/private/auth exists and has correct ownership. Run postfix check to surface configuration errors.
• TLS handshake errors: Verify certificate paths are correct and Let's Encrypt has been obtained (sudo certbot certonly --standalone -d mail.example.com). Postfix and Dovecot both need the fullchain.pem, not just cert.pem.
• Landed in spam despite correct records: Check your IP's reputation at MultiRBL. New IPs need a warm-up period of gradually increasing sending volume.