I would like to monitor my backup services with PRTG, for example, Symantec, Acronis, Norton, Genie, Backup Exec, or many other backup solutions. When something is wrong with my backups, I would like to be warned or alerted respectively. How can I achieve this with an IMAP sensor?
How can I monitor my backup software to be sure backup succeeded last night?
This article applies to PRTG Network Monitor 16.3.26 or later
Update: With PRTG version 16.3.26, we optimized the IMAP sensor settings usage. The look of the settings changed, but the functionalities remain the same. Running IMAP sensors that have already existed in your installation before the update to PRTG 16.3.26 or later will transfer the settings correctly, so you do not have to change anything.
This article refers to the IMAP sensor settings as of PRTG 16.3.26.
Monitoring Backup Solutions via Email
The subject backup is getting more and more complicated for service companies with many customers as well as for end-user companies: there are many kinds of backup solutions, the amount of data rises, etc. Not only in big companies, even in small or medium-sized companies the kinds of backup solutions rather vary:
- Backups in the area of virtualization, for example, Veeam
- Additional backups of the operating system, for example, complete Images, SQL backup, Exchange backups, etc.
- Tape backups, for example, CA ARCserve, Symantec Backup Exec, etc.
- Online Backups
With these various options it is difficult to keep an overview of what is going on with your backups.
Email Notifications as Common Ground
Most backup services can send email notifications to inform you about the status of the last backup. Though, with a certain amount of backups to be checked, it is not practicable to analyze manually every email for malfunctioning backups day by day. You will need a system that displays all backup jobs at a glance and indicates if they succeeded or not.
Monitoring Backups with PRTG
Because of email notifications as common ground for different backup solutions, this is an universal method to centrally receive status messages from almost every backup job. With PRTG you will not need to check any more all individual emails that your backups send to your personal inbox. PRTG will analyze these emails for you and keep track of all your backup jobs.
The PRTG IMAP Sensor has specific filter options and automatically runs through your emails, checking them for keywords in the subject line and in the text. Also it can notify the responsible person if there were not received any emails by a backup during a certain time span. You will know that everything is fine with your backups as long as the IMAP sensors are green.
You only need two things to set up backup monitoring with PRTG:
- Configure your backup software to send emails to a dedicated email account, and
- configure the PRTG IMAP sensor so it checks your backup mails.
Backup Monitoring via Email (enlarge)
Use Case Scenario
You run two different backup solutions. Both send emails to your in-house mail server with information about their respective backup status. Now you would like PRTG to retrieve the emails regularly to check them for keywords in order to show you the backup status, as well as trigger an alert if there has not been any incoming email during the last 24 hours.
Following are listed the necessary steps for this scenario.
Step 1—Configure Your Backup Solution
Your backup software must be enabled to send emails. Furthermore, we recommend that for each of your backup solutions there is particularly one dedicated email account. Thus, in this use case scenario you will need two dedicated email accounts on your in-house email server. These accounts have to be accessible via IMAP.
Emails sent by your backup software most likely look like this:
Email Indicating a Successful Backup
Email Indicating a Partially Successful Backup
Email Indicating a Failed Backup
Step 2—Configure IMAP Sensor in PRTG
Following are listed the steps how to configure IMAP sensors in PRTG for backup monitoring:
- Create a new device in PRTG for your mail server: add a new device to your desired group, for example, by clicking Add Device… in its context menu.
Add a Device to PRTG
- The setup assistant appears. Enter a suitable name and the corresponding IP/DNS of your email server. Click Continue.
Device Settings (enlarge)
- Add two IMAP sensors, one for each of your email accounts on the email server, for example, right-click on the device, and from the context menu, choose Add Sensor…. In the "Add sensor" dialog, filter for Email Server under Target System Type to find the IMAP sensor in a fast way. Click Add This.
Add IMAP Sensor (enlarge)
- The setup assistant appears. Enter a suitable name. You can leave section Connectivity unchanged for the moment (in previous PRTG versions, this was section "IMAP Specific").
IMAP Sensor Connectivity Settings (enlarge)
- Enter the credentials (username/password) for the respective mailbox under Authentication. Enter the name of the corresponding mailbox (this was in section "Identify Email" in previous PRTG versions).
IMAP Sensor Mailbox Credentials (enlarge)
- In the Sensor Behavior section, set General Behavior to Process emails in this mailbox (in previous PRTG versions this was "Process Email Content" in section "Identify Email").
- The Filter Emails in Inbox section appears (section "Identify Emails" in previous PRTG versions). Because you use only one dedicated email account per backup, you can leave most filter settings unchanged: Do not check. You can adjust the Only Include Recent Emails though.
- Choose option Emails must be younger than. Because we would like to trigger an alert if there has not been any incoming backup email for the last 24 hours, enter 24 into the Maximum Age in Hours field and choose Set sensor to "Down" status for If No Email Matches the Above Filters. (In previous PRTG versions, this was setting "Check Last Message Date" with selection "Check for new messages received within the last x hours" entered into the "Error Threshold (Hours)" field.)
IMAP Sensor Message Filter (enlarge)
- In the Validate Latest Email section (in previous versions "Sensor Behavior"), customize your filters for warnings and alarms. In our use case we want green sensors if the backup job succeeded and warnings for backup emails containing both “error” and “partially” in the subject. Additionally, we want a red alarm status for emails containing “error” only. If we get an email with any other text, but without the word “success”, this is an unknown email and indicates an error. We want to have a red alarm status in this case, too.
- For For the Latest Email that Matches, Set Sensor to "Warning" Status (in previous versions "Set to Warning"), choose If subject contains and Regular expression as Check Method. To match subjects like the one in our example (ERROR: Backup backup_xyz is partially successful) use the following regular expression (regex):
- This regex finds matches containing the words error and partially in any order. Enter a Warning Message for this case.
- To get a Warning status for too old messages, choose If emails is older than for Based on Message Age, Set Sensor to "Warning" Status. Enter a number of hours without any backup email into the Message Age in Hours field to receive an additional Warning status.
- For For the Latest Email that Matches, Set Sensor to "Down" Status (in previous versions "Set to Alarm"), choose If subject does not contain and String search as Check Method. Enter success into the Search Text field. This sensor behavior defines that if there is no (sub-)string “success” in the subject line, the sensor is set to a Down status. Enter an Error Message for this case.
IMAP Sensor Email Validation (enlarge)
(Note: In previous PRTG versions, you had to set "No Matching Mail Behavior" to "None", so if none of the defined filter matched, everything would be fine and the sensor would stay green.)
- Click Continue.
- Repeat these configuration steps for your second IMAP sensor.
PRTG will start monitoring your backups immediately!
Note: Of course, you can also define filters for the body of your backup emails.
Step 3: Test Your Configuration
To test your configuration, send some test emails to the accounts you use for your backups. Try several subjects. For example:
- try a subject that simulates a successful backup,
- try a subject for which you want a Warning status,
- try to trigger a Down status, for example, by using an error term without “partially”, and by using some other text, for example, “Acronis True Image Notification from exchanger.johnqpublic.com”.
Using One Mailbox Only for Several Backup Services
For an easier setup, we recommend using separate dedicated mailboxes, one for each backup service. However, it is possible to monitor several backup services within one mailbox. This means that all of your backup solutions will send emails to the same target address. However, each of your solutions will use a different "from" address, or at least a unique identifier in the subject field.
To set up monitoring for this scenario, you need specific criteria in your sensor settings to identify the emails. You can either use a specific sender ("from") address, an ID, or a URL. In any case, you need one IMAP sensor for every backup job, and each of these sensors will query the same IMAP mailbox.
- Configure the IMAP sensor as above.
- Under Filter Emails in Inbox (in previous versions "Identify Email"), configure the criteria how the sensor will discern emails. Use the Only Include Emails with fields. For example, if backup mails from firstname.lastname@example.org are supposed to be analyzed, choose Check using string search in Only Include Emails with Matching "From" and enter this email address. The sensor will now ignore emails from other addresses.
- If the backup emails contain an ID or another distinctive term in the subject, for example, “Backup successful (ID 12345)”, you can use Only Include Emails with Matching "Subject" the same way. (In previous PRTG versions, use the "Identify by" fields.)
If your IMAP sensors do not work as expected, ensure that the following requirements are met:
- Your IMAP mailbox receives emails from your backup software.
- You entered the correct credentials in PRTG for your IMAP mailbox. Ensure that PRTG can log in.
- You set the email content filters correctly. Especially ensure that you use the right syntax for regular expressions! If in any doubt about what they are matching, check your regex with a regex tester like REGex TESTER.
- The error/warning thresholds of your IMAP sensors are set correctly.
- Note: We strongly recommend setting up a mailbox that is only used for monitoring. If a user connects to this mailbox via any email client or web GUI, the IMAP sensor will not be able to correctly calculate the last message date and will therefore likely send false alarms, especially when using error thresholds.
we us the Germany Version of PRTG.
Is name of the Mailbox "Inbox" or "Posteingang" ?
Best regards Chris
the name of you IMAP inbox depends on your email server, not on PRTG. Or did I misunderstand your question?
In the Outlook-Client the name of ther Inboxfolder is language dependent.
I tried some cases but PRTG need sto use "inbox" even if the name in Outlook is displayed as "Posteingang". Th@nx for your help Chris
Great guide thanks Gerald this is just what we have been looking for!
Hi, This looks great - can you also be notified if there is NO email (i.e. the backup service hasn't launched to advise of failure or success)
How about opening up port 25 on the PRTG server to receive the emails directly?
at the moment the option to be notified if there is no mail is not available, sorry. But I will add it to the wish list.
It is unlikely the we will have the PRTG server to receive mail directly any time soon.
Hi, thanks for the great Guide, but i have another Question. Is there also a possibility to check Backup E-Mail in a Subfolder of the Inbox?
Thanks a lot!
Best regards Marco
You can monitor subf olders with this syntax
Best regards, Felix
I have another question. I don't understand how works the censors if they are many emails. Does he look only the last email ?
If I get an email with in the object a string "Failed", my censors change to "ERROR". Then I correct the problem and a new mail with "Success" arrive.
What happend ? Will the censor only check the last mail ?
this is great feature, thanks a lot!!
Please,, two missing features...
Is there a possibility please to mark proceeded/checked emails/messages as "read", move them to different folder, or mark them with any other way please? To be able to recognize parsed and not parsed messages in mailbox/inbox?
Could we store email message or at least its important part/extract in log of sensor, the same way as syslog/snmp traps messages are stored please? It would be really helpful to be able to recheck real message content according to which was check done/status changed.
If these functions are not there, is there a chance you would implemented it please ?
Thank you very much Kind regards Jenda Benda
Thank you for your KB-post.
I am afraid that there are no plans to implement these functions, sorry.
I'm sorry but I have to comment that this seems silly and shortsighted - even foolish - to me.
You specifically "require" a dedicated email account for this. The idea here is (obviously) to automate this process. That means that the process, including maintaining the email account, should be automatic. In that regard once PRTG processes an email, it should process that email all the way - ie. delete it, archive it, move it to a folder.. It seems to me a very basic function.
Frankly I see it as no different than how most ticketing systems handle email - once they retrieve the email for processing, they delete it. I know that to be the case for Spiceworks helpdesk, whmcs helpdesk and Connectwise. I'm sure dozens of others. Sorry if the tone seems harsh but it just seems ridiculous to me that the whole process should be automated but then I need to manually "clean up" the email account behind PRTG. Even writing a cron script can't be as accurate because how can I ensure PRTG has processed a message? I can only make assumptions based on a period of time whereas PRTG itself is perfectly suited to know if it's seen/processed a message.
Please reconsider this position. There is no danger of losing user data. These messages are processed by PRTG, the logs show what's been read and there is a facility to log the email contents to .txt file. That should be plenty but for the ultra paranoid you can have the option for PRTG to move the messages to an archive folder instead of simply delete.
This is a very standard practice for apps that process email incoming to them and PRTG should not be any different. There's no reason that you can't leave options for the user to keep the exisiting behavior if that's desirable but I expect in many cases users would prefer to have the software clean up after itself.
I've got a little problem, the connection of my IMAP sensor times out with the following ERROR :
Connection timed out Socket Error # 10060 Connection timed out. (erreur de socket # 10060)
But I have a doubt that I enter the wrong Mail Server when I create the Device which contains the Mail Server. So my question is if I use a gmail adress which server do I need to indicate that my IMAP sensor can connect to the server ?
Does a mail client work when configured for your Gmail account on the PRTG server?
As good as I like PRTG, I have to agree with JFrost regarding the IMAP sensor. Although the sensor has been reworked since v16, it does not work reliably if one mailbox is used for all notifications. As MSP we have thousands of notifications every day which need to be monitored. To setup a separate mailbox for each source is not acceptable. In addition, this is difficult to automate.
What concerns me is how little importance Paessler attaches to this sensor. It's just a pity.
Thanks for your response. I agree, the sensor has it's quirks and barely has seen any new code in the last 2 years. What kind of backup solution are you using? If it has a decent API, the new REST Custom Sensor might also be a viable solution for you.
Stephan Linke, Tech Support Team
I agree with JFrost and Gee. In an MSP environment it's just not possible to use the IMAP sensor for Backups. We started using a different approach some years ago: Most Backup solutions will write status messages in windows eventlog or allow running a script on success. Our concept is that we have a central share at each customer and every backup solution has to run a script on successful run. Or if this is not possible the windows scheduler will run the script when the event log entry appears. The script will change the timestamp of a file in the central share. PRTG then monitors some or even several dozen files at each customer's central share with a single sensor for each and has warning limits set to the "last changed" timestamp. If possible, we even let the script fill in the last backup log in these files so that it's easier to troubleshoot why the backup failed, you just have to enter the text file you monitor. This setup is working for us with several hundred backups in our environment and has proven to be very reliable.
Thanks for sharing that. Maybe you can post your complete setup in a new thread and link it here for others to find? :)
Stephan Linke, Tech Support Team