imap2script as daemon - not kicking off script?

Worf

I'm running imap2script using daemonize per the instructions, and I've noticed sometimes it just stops processing mail. The mail stays in the mailbox just fine but it's just not being processed. If I kill it and restart it, it starts processing again. Is this an issue with imap2script? Should I just make a cronjob that restarts it every day?

There are no errors in the log - you just see messages from before it gets stuck.

sean

Worf this first time I've heard of this happening with imap2script.

If your imap2script is using an Opalstack mail user then can you nail down the exact date and time (with time zone) that it stopped working? I'll check our mail logs from that time to see if there were any errors logged around that time.

Worf

Sean, thanks. From the logs I have, the last time imap2script did something was on Feb 6
2021/02/06 04:28:12 [INFO ] (id=0) Processing 0 unread message(s) 2021/02/06 08:51:28 [INFO ] (id=0) Got an IDLE mailbox update 2021/02/06 08:51:28 [INFO ] (id=0) Processing 1 unread message(s) 2021/02/06 08:51:28 [DEBUG] (id=0, i=0) Sending to script /home/worf/run_procmail 2021/02/06 08:51:28 [DEBUG] (id=0) Deleting processed messages

I was doing some testing a little later on Feb 10 and noticed a few hours later that they didn't come through. I haven't done anything until today and I just restarted the daemon and it apparently processed that email from then.

2021/02/13 00:56:39 [INFO ] All instances started 2021/02/13 00:56:39 [INFO ] (id=0) Starting imap2script 2021/02/13 00:56:39 [INFO ] (id=0) Connected to mail.us.opalstack.com:993 2021/02/13 00:56:39 [INFO ] (id=0) Logged into mailuser 2021/02/13 00:56:39 [INFO ] (id=0) Selected folder INBOX 2021/02/13 00:56:39 [INFO ] (id=0) Processing 1 unread message(s) 2021/02/13 00:56:40 [DEBUG] (id=0, i=0) Sending to script /home/worf/run_procmail 2021/02/13 00:56:40 [DEBUG] (id=0) Deleting processed messages

I noticed it when I was testing a few days earlier that it did this, but didn't think much of it since later testing seemed to work just fine.

sean

Worf we've had some recent problems on our DE IMAP server on 9 February and again earlier today (the 12th) so it's possible that when the IMAP server rebooted the script wasn't able to reconnect.

I'll bring this to the attention of the imap2script developer, but in the meantime if this is critical then you should go ahead and make a cron job to restart at whatever interval you need.

Worf

I doubt it's the IMAP server after more investigations over the weekend (time to take the leap from WebFaction so I think I'll cancel the migration and cancel my account there).

It appears to work completely fine if I run imap2script interactively - e.g., through screen - email arrives promptly and is handled promptly.

If I run it through daemonize using the example in the topic guide, it processes email in the inbox at the beginning and then gets stuck. I think if I run it with no unread email, it works until the first one it processes.

Chances are, there may be something my script is doing that is causing imap2script to halt (like writing to stderr? it seems to be caught by imap2script, but maybe?) when running as a daemon but is fine when run interactively through screen?

Of course, running it through screen is less than ideal for many reasons...

sean

Worf the example in the docs includes a log option to catch stderr, but I just noticed it does not have an option to catch stdout. Try running it with an additional option -o like this:

daemonize -p $PWD/imap2script.pid -o $PWD/imap2script.log -e $PWD/imap2script_errors.log -c $PWD /usr/local/bin/imap2script --config config.json

Worf

sean Gave that a try and so far, it looks good. I'll give it a few days to be sure, but that might be the fix.

Thanks.

Worf

It seemed to glitch out last night, but I admit I changed DeleteAfter from "false" to "true" in the meantime and that might have an effect.

I did notice that there was an easy way for me to tell in the imap2script-errors.log if it was working or not - if the last line was

2021/02/17 18:51:07 [INFO ] (id=0) Processing 0 unread message(s)

then it works. If that line isn't there, it seems to get stuck.

This is a log of a successful execution:

2021/02/17 18:51:07 [INFO ] (id=0) Got an IDLE mailbox update 2021/02/17 18:51:07 [INFO ] (id=0) Processing 1 unread message(s) 2021/02/17 18:51:07 [INFO ] (id=0) Processed 1 message(s) 2021/02/17 18:51:07 [INFO ] (id=0) Got an IDLE mailbox update 2021/02/17 18:51:07 [DEBUG] (id=0, i=0) Sending to script /home/worf/run_procmail 2021/02/17 18:51:07 [INFO ] (id=0, i=0) Script Script produced stderr: procmail: [203432] Wed Feb 17 18:51:07 2021 procmail: Assigning "LOGFILE=/home/worf/procmail.log" procmail: Opening "/home/worf/procmail.log" 2021/02/17 18:51:07 [INFO ] (id=0) Processing 0 unread message(s)

The only difference when it gets stuck is the last line goes missing

I'm going to try more controlled experiments to see if I can narrow down the issue.

Worf

Some observations.

If I set DeleteAfter to false, imap2script worked just fine for days on end.
If I set DeleteAfter to true, imap2script got stuck within a few hours of starting as a daemon.

Unconfirmed observation:
If I restart imap2script, if there are emails in the mailbox waiting to be processed, the daemon will process all the email pending and potentially get stuck. (Needs more examination).

sean

Worf If you run this in the foreground (ie without daemonize) in a screen or tmux session, do these problems still occur?

Worf

sean With DeleteAfter set to false, it worked just fine daemonized or within a screen session.

I'll test with DeleteAfter set to true within screen.

Thanks.

Worf

Well, it works fine interactively, however it seems to not be deleting the emails. Is this a bug? It used to delete emails after processing when DeleteAfter was set to true, and now, even though I changed it to true, the mailbox isn't being emptied.

Also, I noticed it got updated - is there a way to get the current version number? Would be useful to figuring out if issues might be fixed...

sean

Worf Just tested this myself, delete seems to work fine under daemonize and when running in the foreground. Do you see any errors in imap2script.log or imap2script_errors.log?

Note that you must kill the imap2script process and restart it any time you modify the configuration file.

Regarding the version, imap2script itself doesn't include a version flag, but you can see the installed version by checking the installed package version with yum or rpm, eg:

$ rpm -q imap2script
imap2script-1.1.0~1-1.x86_64

$ yum list installed | grep imap2script
imap2script.x86_64               1.1.0~1-1                           @opalstack