neomutt

neomutt is a mail user agent (MUA). More specifically, it lets you read email from the terminal. It is possible to download and send mail from mutt natively, but I prefer external programs for those functions.

A quick overview:

  • downloading mail: offlineimap
  • sending mail: msmtp
  • reading mail: neomutt + neovim (occasional full-screen reading) + w3m (for html emails)
  • editing mail: neovim
  • indexing mail: notmuch
  • encrypting mail: gpg
  • adding attachments: ranger
    • determining what program to use to open attachments: rifle
  • general selector: fzf

neomutt

neomutt is essentially a superset of regular mutt aiming to fix bugs, collect patches, and in general incite development of mutt. It therefore makes sense to use neomutt rather than mutt.

General notes

My primary email is gmail, which has its quirks. In particular, all emails go into "all mail" (including emails sent by oneself!) and the different "folders" are more like tags --- attributes of the emails in all mail. This mirrors notmuch nicely but IMAP not so much so there will be a few oddities.

I use PGP to sign and encrypt email.

Lastly, I use Google's Advanced Protection program. Surprisingly enough, one can use command-line tools to access mail even with advanced protection on (if you authenticate with OAuth2).

With that in mind, the first step is to get mail.

offlineimap

offlineimap is used to download (and sync!) mail. Note that this is a two-way operation: it will update the local repository of mail if there are changes in the remote and it will also update the remote if there are local changes! There is a risk of deleting email permanently if you delete locally and have offlineimap sync. Run with the --dry-run option to see what offlineimap will do while testing.

offlineimap is configured with Python, unfortunately Python 2. There's a Python 3 fork of offlineimap, and the same author also wrote imapfw, a Python 3 replacement, but the project appears to be dead.

In order to authenticate, we must use OAuth2 as mentioned before. The specific steps depends on the email provider, but in general we need a client id, secret, and refresh token. We can redeem the refresh token for an access token, which is what we actually use to authenticate. I store these credentials in the lightweight PGP-based password manager pass. To generate an access token, we could use offlineimap's built-in oauth2_refresh_token_eval option but for integration with msmtp and caching we might as well use our own program: offlineimap.py.

Google and Microsoft cover all my email accounts, including those which that are not necessarily @gmail.com or @hotmail.com. For example, my Georgia Tech email ending in @gatech.edu is actually provided by Outlook, so I can use Microsoft OAuth to authenticate, without needing to go through Georgia Tech's single sign-on authentication portal.

Google OAuth

We'll be using the gmail-oauth2-tools repository as the client library.

Follow the instructions here. The Google Cloud Console is pretty poorly designed, so it may take some effort to figure out how to create a new project.

If it initially works but after a week there's the error KeyError: 'access_token' it might be that the refresh token is invalid. This is because Google's OAuth policy restricts the lifespan of a refresh token to 7 days if the app is configured for external users and the publishing setting is "Testing", a common situation one would be in for personal use. The solution is to press the "PUBLISH APP" button on the OAuth consent screen. Although it will warn you that "Because you're using one or more sensitive scopes, your app registration requires verification by Google. Please prepare your app to submit for verification", you don't actually need to verify the app, that just removes the warning screen asking the user whether they trust the developer while getting a refresh token.

Microsoft OAuth

We'll be using the msal client library.

Follow the instructions to create a new application and add IMAP and SMTP permissions. These instructions are a bit verbose, so I'll condense them here:

  1. Navigate to the Azure portal
  2. Go to "Azure Active Directory", either by searching or clicking on the icon
  3. Find "App registrations" in the side bar under "Manage" and press "New registration"
  4. Under "Manage", select "Authentication". Use the "Web" platform with a redirect URI of http://localhost.
  5. Select "Certificates & secrets" and press "New client secret". Record the client id and secret.
  6. Select "API Permissions". Press "Add a permission" and use "Microsoft Graph" with "Delegated permissions". The permissions we need are offline_access (under OpenId permissions), User.Read (under User), IMAP.AccessAsUser.all (under IMAP) and SMTP.Send (under SMTP).
  7. Depending on the situation, we might need a tenant. For Georgia Tech, this is gtvault.onmicrosoft.com. This value can be found by going to the "Azure Active Directory" page and looking at the value of "Primary domain". Otherwise, this can be set to common.

Something strange Microsoft does is their refresh tokens: they give a new refresh token back after every access token request, and refresh tokens expire after 90 days. If you were authenticating through offlineimap, you might be passing oauth2_refresh_token so offlineimap can automatically request access tokens. So if you suddenly become unable to request access tokens, it might be because of the refresh token expiration. offlineimap.py will automatically save the refreshed refresh token, but you still need to update the client secrets at least every two years (since 24 months is the longest possible expiration date for client secrets).

msmtp

With offlineimap configured, msmtp works similarly. Set the authentication protocol to oauthbearer and passwordeval to run the above offlineimap.py script, passing in the email. That way both offlineimap and msmtp use the same cached access token.

notmuch

notmuch is an email indexer, tagger, and searcher. Add a postsync hook to offlineimap so tagging happens on new mail. We can also use notmuch as an address book by searching the addresses of previously received emails.