Email Scout
This page was last modified 05:42, 29 January 2007.From Documentation
| Revision as of 03:00, 17 January 2007 Daniels (Talk | contribs) (→How we monitor Email) ← Previous diff |
Current revision Daniels (Talk | contribs) (→Event Sink Method) |
||
| Line 5: | Line 5: | ||
| We also offer a "round-trip" email monitoring service. This tests that your mail server is effectively communicating with the rest of the world. | We also offer a "round-trip" email monitoring service. This tests that your mail server is effectively communicating with the rest of the world. | ||
| - | In all instances, a remote testing email address is required for returning our test email. We suggest that this be a specially set up email, so that the return of constant emails does not conflict with day-to-day usage of a "real" email account. | + | In all instances, a remote testing email address is required for returning our test email. For Linux/Unix and Exchange Email monitoring, any mail that is received by the test account is not kept, so using an existing email account is not recommended. |
| + | |||
| + | Functional Software collects three data points in measuring the Round Trip Email time. The start time, the time the email is generated to be sent back to our servers, and the time it is received by our reception script. As a general rule, we only use the total Round Trip time (Receive Time - Start Time) to generate metrics, as we cannot guarantee that all servers will be synchronized with regard to time. | ||
| + | |||
| + | We do attempt to get around Timezone limitations by obtaining either: | ||
| + | ;The time in [http://en.wikipedia.org/wiki/Coordinated_Universal_Time Coordinated Universal Time (UTC)], on platforms where this is an option | ||
| + | or | ||
| + | ;The timezone offset (this is generally the offset from [http://en.wikipedia.org/wiki/Greenwich_Mean_Time Greenwich Mean Time (GMT)] which, while very close to UTC, is not exactly the same thing. | ||
| + | |||
| + | However, neither of these are foolproof methods, as the time on a given server may be given to time slipping, or may simply be incorrectly set at the very start. Therefore, we cannot base our calculations on the time given by a remote machine. In order to achieve the greatest possible time accuracy, all machines on which we run the Email Monitoring Knowledge Base are synchronized with a remote time server every hour. | ||
| + | |||
| + | |||
| + | ---- | ||
| + | ;Note: If we were to include the remote time data, and it were not set correctly, the information would appear to have either a very large outgoing leg, and a negative incoming leg tim, or vice versa | ||
| + | ---- | ||
| + | |||
| + | We do log the data for each of the legs, but have found that the potentially large positive and negative numbers were making things too confusing. Therefore by default it is not displayed in graphs. | ||
| + | |||
| ---- | ---- | ||
| ;Note: Functional Software cannot offer support on how to create a new user, as it varies greatly depending on the type of mail server, security settings and environment required for each site. | ;Note: Functional Software cannot offer support on how to create a new user, as it varies greatly depending on the type of mail server, security settings and environment required for each site. | ||
| ---- | ---- | ||
| + | |||
| + | By default, we measure the time taken for an email Round Trip every 20 minutes. The default timeout is set to 10 minutes, with a slow warning if an email takes between 2 and 10 minutes to arrive. We have found that this suits most customer needs for email monitoring, but as with all of our sentinel3G based products, everything is configurable on a per-item basis. | ||
| + | |||
| + | Similarly, the default notification is to wait for a timeout (10 minutes), then wait 30 seconds and fire another test. If this also times out, the notification is triggered. This would normally be by SMS, as generally email notification of email failure is a bad ides. The details of SMS notification would be covered in your support agreement. These settings mean that the 'maximum' amount of time your email system would be down before notification is less than 41 minutes. This assumes that your email delivery went down immediately following Test1, and was still down for Test2 (20 minutes between Test1 and Test2, plus 10 minute timeout) and Test3 (30 second wait time, plus 10 minute timeout). | ||
| + | |||
| + | As well as the double testing safeguard, to make sure it was not merely a system glitch that caused the missed email, if your monitoring contract supports it, your email will be monitored from multiple remote systems. At least two of which need to be in agreement that there is an issue of mail not being delivered before triggering a notification. | ||
| + | |||
| + | Functional Software currently supports Email Round Trip monitoring under Unix/Linux (using the .forward feature), MS Windows servers running Exchange 2003, and MS Windows servers running the Imail mail server. If you are using something other than the Email systems mentioned here, please contact Functional Software and we may be able to organize something to help you. | ||
| + | |||
| + | |||
| == Email Round Trip Monitoring for Unix/Linux == | == Email Round Trip Monitoring for Unix/Linux == | ||
| Line 23: | Line 50: | ||
| This will run the script, but not retain the original message. This ensures that your users mailbox is not filled with unnecessary messages. If you would like to retain the messages, the <tt>.forward</tt> file becomes: | This will run the script, but not retain the original message. This ensures that your users mailbox is not filled with unnecessary messages. If you would like to retain the messages, the <tt>.forward</tt> file becomes: | ||
| - | <code>\<var>username</var>|/<var><path_to_script></var>/email_return</code> | + | <code>\<var>username</var>|"/<var><path_to_script></var>/email_return"</code> |
| This script <tt>email_return</tt> verifies that the email is originating from Functional Software. If it does come from our monitoring service, it adds a system timestamp to the Subject line of the email, and returns it to the email monitor address. The timestamp is called in UTC format. It is not, however, used in the monitoring metrics, as there is no guarantee that the time on your remote server and the time on our monitoring server will be in synch. | This script <tt>email_return</tt> verifies that the email is originating from Functional Software. If it does come from our monitoring service, it adds a system timestamp to the Subject line of the email, and returns it to the email monitor address. The timestamp is called in UTC format. It is not, however, used in the monitoring metrics, as there is no guarantee that the time on your remote server and the time on our monitoring server will be in synch. | ||
| Line 76: | Line 103: | ||
| There are two possible ways to enable the Functional Software Round Trip Email Monitor for Exchange. Using Agents, or using Event Sinks. The easiest to set up and use is the Event Sink, but many older installations of Exchange do not support this method. These servers must use the Agent Method. | There are two possible ways to enable the Functional Software Round Trip Email Monitor for Exchange. Using Agents, or using Event Sinks. The easiest to set up and use is the Event Sink, but many older installations of Exchange do not support this method. These servers must use the Agent Method. | ||
| + | In both cases, the result and email that is returned are the same. The main difference between the two sinks is merely the way that data structures are called internally. | ||
| - | === Event Sink Method === | + | In order of operation, these scripts: |
| + | #Obtain the local time of the machine | ||
| + | #Attempt to get the timezone offset from the Windows Registry | ||
| + | ##If successful, apply the offset to the date | ||
| + | #Creates a mail message containing required information from the received mail | ||
| + | #Adds the Time stamp obtained above | ||
| + | #Sends the new email to the specified return address | ||
| + | #Halts delivery of the incoming email. | ||
| - | This is the easiest method for installing the Round Trip email monitor. It requires the target machine to have the cscript utility installed. Cscript is the Windows Command Line Script Host command ([http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/cscript_overview.mspx?mfr=true Microsoft Page for Cscript]) | + | Functional Software provides a package (.ZIP archive) containing all the files and scripts necessary to install this method. The package contains the following files: |
| + | *Agent Monitor Script (only used for Agent installs) | ||
| + | *Event Sink Script | ||
| + | *Event Sink Installer | ||
| + | *Event Sink Uninstaller | ||
| + | *Microsoft's Event Register Script. | ||
| + | *Readme file containing more in-depth instructions for installing Event Sink monitoring. | ||
| - | === Mailbox Agent Method === | ||
| - | The easiest way to install the Email Round Trip Monitoring for Exchange (Agent) is to set up a new user to monitor, and automatically forward email for that user to a Public Store that has our Round Trip script installed against it. | + | === Event Sink Method === |
| - | This has the benefit of not requiring any special permissions for an other non-used Exchange user. It does require certain permissions for the Administration account used to create the agents however. | + | This is the easiest method for installing the Round Trip email monitor. It requires the target machine to have the cscript utility installed. Cscript is the Windows Command Line Script Host command ([http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/cscript_overview.mspx?mfr=true Microsoft Page for Cscript]) |
| - | === Setup Account using Public Store === | ||
| - | #. Create a Public Store. | + | If a user already exists to receive the email from the Monitor Script, simply run the Installer File, after reading the instructions. |
| - | ##. In Exchange System Manager (ESM) go to Administrative Groups and select your desired Group. | + | |
| - | ##. Drill down to the Public Folder level, and create a new Public Folder. | + | If a user email account has not already been created for the Event Sink Monitor to use; In Active Directory Users and Computers (U&C): create a new account, specify a password, mark it to never expire and user cannot change. To use the Event Sink method, a mailbox is not required, although it can be created with no harm done to the setup. Then, as Administrator run the Installer file to set it all up. |
| - | ##. Right click on the new folder, go to "All Tasks" and click on "Mail Enable". This allows you to forward email to the given public folder. | + | |
| - | ##. Ensure that the user you are going to use to create the script has "Owner" level abilities in the Permisssions of this folder. | + | |
| - | #. In Active Directory Users and Computers (U&C): create a new account, specify a password, mark it to never expire and user cannot change, create a mailbox | + | |
| - | #. In the Properties of this user, go to the "Exchange General" tab -> Delivery Options Button -> Change the Forwarding Address to the Public Store the Agent is set up against. | + | |
| - | #. Ensure that the user you have chosen has the correct permissions, spelled out in [http://documentation.fs.com.au/index.php/Email_Scout#Outlook_Setup Service Account Setup], below. | + | |
| - | #. Proceed with [http://documentation.fs.com.au/index.php/Email_Scout#Outlook_Setup Outlook Setup], using the assumption that Outlook is set up correctly for collecting mail. | + | |
| ---- | ---- | ||
| - | ;Note | + | ;Note: The VBScript that is used to install the Event Sinks is <strong>NOT</strong> developed or supported by Functional Software |
| - | :At the moment, the Functional Email Round Trip monitor supports only Exchange Server 2003 | + | : The script is available from Microsoft via either the MS Support website, or in the Exchange SDK (also available from the MS website) |
| ---- | ---- | ||
| - | === Service Account Setup === | + | === Mailbox Agent Method === |
| - | The following steps detail how to create a service account which executes event service scripts... | + | The easiest way to install the Email Round Trip Monitoring for Exchange (Agent) is to set up a new user to monitor, and automatically forward email for that user to a Public Store that has our Round Trip script installed against it. |
| - | #. In Active Directory Users and Computers (U&C): create a new account, specify a password, mark it to never expire and user cannot change, create a mailbox | + | This has the benefit of not requiring any special permissions for an other non-used Exchange user. It does require certain permissions for the Administration account used to create the agents however. |
| - | #. In U&C: add this account to the local “Administrators” group. Do not add it to the “Domain Administrators” group, this will enact specific denials on the mailbox store and elevate the permissions unnecessarily within the domain. | + | |
| - | #. In Exchange System Manager (ESM): right click on the organization group, select “Delegate Control…” and add the newly created account as an “Exchange View Only Administrator”. | + | |
| - | #. In U&C: Give the service “Full Mailbox Access” to any script author accounts by right clicking the author’s account, selecting properties, selecting the “Exchange Advanced” tab, and clicking the “Mailbox Rights” button. You may have many authors or want to enable everyone to author scripts. | + | |
| - | #. In ESM: Create a new Public Folder for sending email to. | + | |
| - | #. In ESM: Give “Owner” permissions to this public folder for the new account. | + | |
| - | #. In ESM: Select the “Public Folders” node under the “Folders” node of the Administrative Group, right click and select “View System Folders”. Expand the “Events Root” folder and give “Owner” permissions to the service account on the "EventConfig_<exchange server name>" folder.This folder may not exist if you haven’t tried to start the Exchange Event Service before. Just start the service to create the folder. | + | |
| - | #. At this point you either have to wait for these permissions to replicate which could take up to 2 hours or you can restart the Exchange Information Store service. | + | |
| - | #. In the Services management console, change the Exchange Event Service to “Logon As” the service account and specify the “Startup Type” as “Automatic”. Then start the Exchange Event Service. | + | |
| - | + | ||
| - | === Outlook Setup === | + | |
| - | + | ||
| - | The easiest thing to do here it to simply log on to any machine in the domain as the chosen user. Then starting Outlook will run you through the account wizard to set up an Outlook account. If that is not possible, you will need to edit your current Outlook account and either modify the current Exchange User settings, or add them if you are not already using an Exchange Account. Outlook will only allow one Exchange Server account to be active at a given time. | + | |
| - | + | ||
| - | Once the account is set up: | + | |
| - | #go to Tools -> Options. | + | |
| - | #Click on the "Other" Tab | + | |
| - | #Click the "Advanced Options" button | + | |
| - | #Click the "Add-in Manager" button, and ensure that "Server Scripting" is checked on. | + | |
| - | + | ||
| - | Once that is complete, restart Outlook to make sure changes are saved and loaded. | + | |
| - | + | ||
| - | Back in Outlook: | + | |
| - | #At the bottom of your left-pane, there should be a series of icons. The second from the left should be "Folder List". Click this to view all Folders associated with this account. | + | |
| - | #Expand "Public Folders" -> "All Public Folders" | + | |
| - | #You should see the name of the Public Folder you created earlier. If not, something has gone wrong and you will need to backtrack. | + | |
| - | #Right Click on this folder and go to "Properties" | + | |
| - | ## In this dialog, there should be a tab labelled "Agents". | + | |
| - | ## If not, more than likely the "Server Scripting" option from above is not checked. Also check that you are not using Cached versions of server information. | + | |
| - | ##Create a new Agent with the following information | + | |
| - | ##* Name: Anything Meaningful | + | |
| - | ##* When the Following Events Occur: A new Item is posted in this folder | + | |
| - | ##* Run the Following Action: Script (provided by FS) | + | |
| - | ## Hit the button to Edit the Script, and copy and paste in the script provided by Functional Software. If you do not have a copy of the script, please email support@fs.com.au and we will send you a copy. Please note that a valid Round Trip Email Subscription is required to get access to this script. | + | |
| - | ## Save the script and OK back to the main page. | + | |
| - | That's it. Your exchange server should now be ready for monitoring. Send an email to support@fs.com.au with the details of the email address you would like monitored, so that we can set up the agent from our end, and once we test that it's returning valid data we will be monitoring your exchange site. If the test fails, we will contact you as soon as practiceable to try and fix where it went wrong. | + | As this is no longer a recommended install path for Email Round Trip Monitoring, instructions for installing this method have been moved to: [[Email_Scout:_Install_for_Agent| Agent Installation]] |
| Line 160: | Line 159: | ||
| *Your desired display name: this is a string of up to 50 characters | *Your desired display name: this is a string of up to 50 characters | ||
| *The email address you would like to use for monitoring | *The email address you would like to use for monitoring | ||
| - | *The email services you would like us to monitor | + | *The email services you would like us to monitor, including the names of the servers (mail1.fs.com.au, for example) |
| *Notification and escalation details<sup>*</sup> | *Notification and escalation details<sup>*</sup> | ||
| - | |||
| <sup>*</sup> Please note that notification and escalation details are dependent upon the level of contract you undertake. | <sup>*</sup> Please note that notification and escalation details are dependent upon the level of contract you undertake. | ||
Current revision
Contents |
How we monitor Email
Functional Software ensures that the mail server processes are running on your mail server machine. We currently offer checks for pop3, smtp and imap services.
We also offer a "round-trip" email monitoring service. This tests that your mail server is effectively communicating with the rest of the world.
In all instances, a remote testing email address is required for returning our test email. For Linux/Unix and Exchange Email monitoring, any mail that is received by the test account is not kept, so using an existing email account is not recommended.
Functional Software collects three data points in measuring the Round Trip Email time. The start time, the time the email is generated to be sent back to our servers, and the time it is received by our reception script. As a general rule, we only use the total Round Trip time (Receive Time - Start Time) to generate metrics, as we cannot guarantee that all servers will be synchronized with regard to time.
We do attempt to get around Timezone limitations by obtaining either:
- The time in Coordinated Universal Time (UTC), on platforms where this is an option
or
- The timezone offset (this is generally the offset from Greenwich Mean Time (GMT) which, while very close to UTC, is not exactly the same thing.
However, neither of these are foolproof methods, as the time on a given server may be given to time slipping, or may simply be incorrectly set at the very start. Therefore, we cannot base our calculations on the time given by a remote machine. In order to achieve the greatest possible time accuracy, all machines on which we run the Email Monitoring Knowledge Base are synchronized with a remote time server every hour.
- Note
- If we were to include the remote time data, and it were not set correctly, the information would appear to have either a very large outgoing leg, and a negative incoming leg tim, or vice versa
We do log the data for each of the legs, but have found that the potentially large positive and negative numbers were making things too confusing. Therefore by default it is not displayed in graphs.
- Note
- Functional Software cannot offer support on how to create a new user, as it varies greatly depending on the type of mail server, security settings and environment required for each site.
By default, we measure the time taken for an email Round Trip every 20 minutes. The default timeout is set to 10 minutes, with a slow warning if an email takes between 2 and 10 minutes to arrive. We have found that this suits most customer needs for email monitoring, but as with all of our sentinel3G based products, everything is configurable on a per-item basis.
Similarly, the default notification is to wait for a timeout (10 minutes), then wait 30 seconds and fire another test. If this also times out, the notification is triggered. This would normally be by SMS, as generally email notification of email failure is a bad ides. The details of SMS notification would be covered in your support agreement. These settings mean that the 'maximum' amount of time your email system would be down before notification is less than 41 minutes. This assumes that your email delivery went down immediately following Test1, and was still down for Test2 (20 minutes between Test1 and Test2, plus 10 minute timeout) and Test3 (30 second wait time, plus 10 minute timeout).
As well as the double testing safeguard, to make sure it was not merely a system glitch that caused the missed email, if your monitoring contract supports it, your email will be monitored from multiple remote systems. At least two of which need to be in agreement that there is an issue of mail not being delivered before triggering a notification.
Functional Software currently supports Email Round Trip monitoring under Unix/Linux (using the .forward feature), MS Windows servers running Exchange 2003, and MS Windows servers running the Imail mail server. If you are using something other than the Email systems mentioned here, please contact Functional Software and we may be able to organize something to help you.
Email Round Trip Monitoring for Unix/Linux
The round-trip email monitor is set up under Linux/Unix using the .forward feature of your mail system. The Functional Monitoring server sends a specifically constructed email to an address that the end-user has created for the purpose of monitoring. We recommend using a name that is easily remembered, such as webscout@company.com or something similar.
The mail server machine that is being monitored does not need to have Functional Software's COSmanager suite installed on it, but a service contract for email monitoring is required. Also, at this time, the email monitoring service does not cover IP Address based email systems.
When editing the .forward file of the chosen email user, you will need to add the line below:
|/<path_to_script>/email_return
This will run the script, but not retain the original message. This ensures that your users mailbox is not filled with unnecessary messages. If you would like to retain the messages, the .forward file becomes:
\username|"/<path_to_script>/email_return"
This script email_return verifies that the email is originating from Functional Software. If it does come from our monitoring service, it adds a system timestamp to the Subject line of the email, and returns it to the email monitor address. The timestamp is called in UTC format. It is not, however, used in the monitoring metrics, as there is no guarantee that the time on your remote server and the time on our monitoring server will be in synch.
Once the remote email address is set up, we can start monitoring almost immediately.
When the returned email is received by our monitoring email account, another script builds the information required to measure the round-trip email time, and reports that to the sentinel3G Agent.
Email Round Trip Monitoring for Imail
Many thanks to John Treen from Canning College for this workaround for Windows Imail
The following instruction will guide you through the installation and setup of the Imail Email Return Application for use with our Email Round Trip Monitoring Service
- Place Imail Email Return.exe somewhere on your hard disk
- Copy vbSendMail.dll to your System32 folder
- Load a Command Prompt Window (
cmd) and run:regsvr32 %SYSTEMROOT\System32\vbSendMail.dll
- Load the Imail Administrator
- Expand the folders and navigate to your server name
- Right click on the Aliases folder and click Add Alias
- At the New Alias ID screen enter your desired monitoring email address (for the rest of the example we will assume you used webscout)
- At the Type of Alias screen select Program and click Next
- At the Program Alias screen enter the Program name and Arguments like:
- <Path_to_Application>\IMail Email Return.exe <MAIL_SERVER_ADDRESS> <FROM_ADDRESS>
- For Example:
- C:\Imail\Imail Email Return.exe localhost support@fs.com.au
- At the Create New Alias Screen click the Finish Button
- The address should be set up and ready to go.
Notes
The Imail Email Return.exe program can take up to three command line parameters
- IP/Hostname of the SMTP server (default=localhost)
- E-Mail address for the reply email to originate from (default=unconfigured-from@dummydomain)
- Filename and path to Imail's temp file containing the source email.
The filename and path is automatically appended to the end of the arguments list by Imail when it calls the application. You can also leave the IP/Host and E-Mail address parameters blank and they will be populated with the default values, but it is recommended that you provide them.
The executable can be called in the following three ways:
Imail Return.exe <TEMP_FILE>Imail Return.exe <MAIL_SERVER> <TEMP_FILE>Imail Return.exe <MAIL_SERVER> <FROM_ADDRESS> <TEMP_FILE>
- Note
- The vbSendMail program is NOT developed or supported by Functional Software
- For more information about the vbSendMail program, please visit vbSendMail at www.freevbcode.com
Email Round Trip Monitoring for Exchange
There are two possible ways to enable the Functional Software Round Trip Email Monitor for Exchange. Using Agents, or using Event Sinks. The easiest to set up and use is the Event Sink, but many older installations of Exchange do not support this method. These servers must use the Agent Method.
In both cases, the result and email that is returned are the same. The main difference between the two sinks is merely the way that data structures are called internally.
In order of operation, these scripts:
- Obtain the local time of the machine
- Attempt to get the timezone offset from the Windows Registry
- If successful, apply the offset to the date
- Creates a mail message containing required information from the received mail
- Adds the Time stamp obtained above
- Sends the new email to the specified return address
- Halts delivery of the incoming email.
Functional Software provides a package (.ZIP archive) containing all the files and scripts necessary to install this method. The package contains the following files:
- Agent Monitor Script (only used for Agent installs)
- Event Sink Script
- Event Sink Installer
- Event Sink Uninstaller
- Microsoft's Event Register Script.
- Readme file containing more in-depth instructions for installing Event Sink monitoring.
Event Sink Method
This is the easiest method for installing the Round Trip email monitor. It requires the target machine to have the cscript utility installed. Cscript is the Windows Command Line Script Host command (Microsoft Page for Cscript)
If a user already exists to receive the email from the Monitor Script, simply run the Installer File, after reading the instructions.
If a user email account has not already been created for the Event Sink Monitor to use; In Active Directory Users and Computers (U&C): create a new account, specify a password, mark it to never expire and user cannot change. To use the Event Sink method, a mailbox is not required, although it can be created with no harm done to the setup. Then, as Administrator run the Installer file to set it all up.
- Note
- The VBScript that is used to install the Event Sinks is NOT developed or supported by Functional Software
- The script is available from Microsoft via either the MS Support website, or in the Exchange SDK (also available from the MS website)
Mailbox Agent Method
The easiest way to install the Email Round Trip Monitoring for Exchange (Agent) is to set up a new user to monitor, and automatically forward email for that user to a Public Store that has our Round Trip script installed against it.
This has the benefit of not requiring any special permissions for an other non-used Exchange user. It does require certain permissions for the Administration account used to create the agents however.
As this is no longer a recommended install path for Email Round Trip Monitoring, instructions for installing this method have been moved to: Agent Installation
- Note
- Due to the nature of the Exchange Agent setup, sometimes it can take as long as a minute (60 seconds) to return a result to our monitoring email. This delay should be taken into consideration when looking at your Round Trip metrics.
- We have no way of knowing how long it actually takes between the arrival of the email and the running of the agent.
Information we Require
Once you have entered into an email monitoring service agreement with Functional Software, we will ask you to provide us several pieces of information:
- Your desired CustomerID: this is a string of 8 letters that you will be able to use to log in and view the reporting console
- Your desired display name: this is a string of up to 50 characters
- The email address you would like to use for monitoring
- The email services you would like us to monitor, including the names of the servers (mail1.fs.com.au, for example)
- Notification and escalation details*
* Please note that notification and escalation details are dependent upon the level of contract you undertake.