Do you use let's encrypt?





2428 votes ~ 14 comments

 

Performing IMAP queries via curl

Posted by Steve on Thu 14 Jan 2016 at 08:27

Tags: ,

Most people are familiar with curl, the tool that allows you to make HTTP-requests, and FTP-requests, via the command-line. Recently it gained the ability to perform IMAP operations, and this brief article demonstrates how that is done.

The curl command can be installed, if not already present, via:

aptitude install curl

Once installed you can make simple HTTP-fetches like so:

$ curl https://www.debian-administration.org/
..
..
</div>
</body>
</html>

This is probably already familiar to you, but if you have an IMAP or IMAPS server you can do more - you can "read your email". To get started you'll first of all need to know:

  • Your mail-server address.
  • Your mail login.
  • Your mail password.

We'll be calling commands via the shell, so bear in mind this is woefully insecure if you're on a shared system - because other users could see your command, and see the password you've submitted. On a single-user system this is perhaps acceptible though.

Listing Folders

The initial example will connect us to the IMAP server at imap.example.com, using username bobby and password tables:

$ curl --url "imap://mail.example.com/" --user "bobby:tables"
..
..
* LIST (\HasNoChildren) "/" xen-users
* LIST (\HasNoChildren) "/" mentors-debian-org
* LIST (\HasNoChildren) "/" fairshare
* LIST (\HasNoChildren) "/" INBOX

Here we see that we've connected, and received a list of folders, including "xen-users", "fairshare", and the "INBOX".

If your mail-server is running over SSL then instead of using imap:// you should set the schema to imaps://, it may be that you're using a self-signed certificate in that case you'd add --insecure to avoid checking the certificate trust-chain.

This would look like so:

$ curl --insecure  --url "imaps://mail.example.com/" --user "bobby:tables"

Discovering Messages

With the previous example we looked at listing mailboxes. What if we wanted to actually view a message? To fetch a message we need the identifier of the message to fetch - so we need to find out how many messages exist, as message-IDs are sequential.

To see how many messages exist in the folder "People-Steve" we'd run this:

$ curl --insecure \
    --url "imaps://mail.example.com/" \
    --user "bobby:tables" \
    --request "EXAMINE People-Steve"
* OK [PERMANENTFLAGS ()] Read-only mailbox.
* 9465 EXISTS
* 3266 RECENT
* OK [UIDVALIDITY 1373146046] UIDs valid
* OK [UIDNEXT 9468] Predicted next UID

This tells us that there are 9465 messages. So we can probably assume that fetching a message with each of these IDs will work: 1, 2, 3, ... 9465, & 9465.

Fetching A Single Message

Fetching a message is simple if you know both the folder from which it comes and the ID of the message you wish to retrieve.

We've already shown that the folder "People-Steve" contains nearly ten-thousand messages, so this next example will show us fetching the message with ID 512:

$ curl --insecure \
    --url "imaps://mail.example.com/People-Steve;UID=512" \
    --user "bobby:tables"
..
X-HELO: mail.cs.helsinki.fi
X-REMOTE-IP: 128.214.9.1
X-REMOTE-HOST: courier.cs.helsinki.fi
..
..


I just accidentally someone! (see facebook)
..

Fetching the message includes both the headers and the body. If you want to fetch just the headers, or a subset, things are a little fiddlier due to URL-encoding. This example gets the headers "to", "from", "date", and "subject":

$ curl --insecure \
    --url "imaps://mail.example.com/People-Steve;UID=512;SECTION=HEADER.FIELDS%20(DATE%20FROM%20TO%20SUBJECT)" \
    --user "bobby:tables"
Date: Sat, 14 Apr 2012 14:13:41 +0300 (EEST)
From: Steve Kemp <steve@steve.org.uk>
To: Steve Kemp <steve@steve.org.uk>
Subject: Re: Fancy a cake?

We could have avoided the use of URL-encoding and instead sent a custom-request, like so:

$ curl --insecure --verbose \
    --url "imaps://imap.example.com/People-Steve" \
    --user "bobby:tables" \
    --request "fetch 512 BODY.PEEK[HEADER.FIELDS (Subject)]"
..
Subject: Re: Fancy a cake?
..

The reason for avoiding custom-requests where possible is that when you're using the curl API programattically you'll discover that responses are not decoded - which is a known issue.

In the example above we'd have received zero output unless/until we added the --verbose flag. Precisely because curl has received the output from the IMAP-server but not decoded it and presented it to us.

Simple Shell Scripts

As a quick hack the following shell-script will dump the subject of the first ten messages in the given mailbox:

#!/bin/sh
# Dump the subject of the first ten messages in the folder.

for id in `seq 1 10` ; do
    echo "Message ${id}"
    curl --insecure \
        --url "imaps://mail.example.com/People-Steve;UID=${id};SECTION=HEADER.FIELDS%20(SUBJECT)" \
        --user "bobby:tables"
done

This takes no account of the maximum message-ID. You could just keep going indefinitely, to dump all subjects and stop on error:

#!/bin/sh
# Dump the subject of all messages in the folder.

id=1

while true  ; do
    echo "Message ${id}"
    curl --insecure \
        --url "imaps://mail.example.com/People-Steve;UID=${id};SECTION=HEADER.FIELDS%20(SUBJECT)" \
        --user "bobby:tables" || exit
    id=`expr $id + 1`
done

 

 


Re: Performing IMAP queries via curl
Posted by Anonymous (96.127.xx.xx) on Sat 23 Jan 2016 at 19:09
If you leave out the password from the --user argument, curl will prompt you for the password before it preforms the request. This way you're not leaking your password to your shell history or the process list.

[ Parent | Reply to this comment ]

Re: Performing IMAP queries via curl
Posted by xrat (128.130.xx.xx) on Mon 25 Jan 2016 at 17:33
curl also supports .netrc lookup of user:password with -n, if the --user argument is left off.

[ Parent | Reply to this comment ]

curllmail bash script
Posted by celeste (72.253.xx.xx) on Wed 24 Feb 2016 at 17:44
http://crystalfaeries.net/fae/curlmail.html

a bash script providing readonly IMAPS client

[ Parent | Reply to this comment ]