Integrating with an E-mail Server

iKnowBase E-mail Reader is a stand-alone component that enables you to automatically load mails from a mailbox into the iKnowBase Content Store. It has the following key features:

• Reads e-mails from any POP3-compliant mail server. Enables you to configure e-mail accounts declaratively using the E-mail integration component in iKnowBase Development Studio. It is possible to define multiple e-mail accounts.
• Loads e-mail messages including recipients, body parts, and attachments into a set of database tables, ready for further processing.
• Calls a PL/SQL procedure (message handler) to process the e-mail messages. By default, a procedure exists. However, you can also create your own procedure with different properties.
• Enables you to schedule and run it on the defined e-mail accounts once or at defined intervals.
• Enables you to run it as a service under Windows, Linux, and various Unix platforms using third-party components such as Java Service Wrapper (http://wrapper.tanukisoftware.org/doc/english/download.jsp).

Configuring E-mail Accounts

Use the E-mail integration component in iKnowBase Development Studio to configure e-mail accounts used by the iKnowBase E-mail Reader. For more information on configuring an e-mail account, see the iKnowBase Development Reference.

Process Flow of the E-mail Reader

The iKnowBase E-mail Reader program performs the following tasks, once for each active account during execution:

1. It connects to the mail server using the assigned credentials. The connection is made using the POP3-protocol, supported by most mail servers.
2. It loads a list of e-mails from the inbox of the specified account and processes each message.
   • If the message exists in the E-mail Reader tables in the database, the message is skipped.
   • If the message doesn’t exist in the E-mail Reader tables in the database, the message is downloaded, split into parts and stored in the database.
   • If the account is set up with Delete messages after copying, the message is marked for deletion on the mail server.
3. After copying all messages, the connection to the mail server is closed with a confirmation message with the flag Permanently delete all messages marked for deletion. If the flag is checked, all the messages will be deleted from the mail server.
4. The message handler is called to process the new messages, typically to load them into the iKnowBase Content Store.

E-mail Reader data model

There can be multiple body parts of a single e-mail. This happens because of the e-mail programs that send multiple versions of the body of an e-mail. One version can be a rich HTML-message and the other can be a simple text-message.

To select the body part that you want to use, use the following rules:

• If you want to select a HTML body part, search for the content_type text/html in the body of the e-mail.
• If you want to select a pure text body part, search for the content_type text/plain in the body of an e-mail.
• Other content-types are specific to individual clients, and have priority after earlier mentioned rules.

Message Handler

The E-mail integration accounts can be configured with a message handler (e-mail reader procedure) that is a customizable PL/SQL procedure. You can use the message handler for automatically loading of e-mails from the iKnowBase E-mail Reader tables into the iKnowBase Content Store. Loading e-mails into the iKnowBase Content Store can be useful for various reasons and the actual document structure required varies widely.

When the iKnowBase E-mail Reader program runs, it populates the iKnowBase E-mail Reader tables with e-mail information. After populating these tables, the program activates the message handler, passing the account_id of the account as the only parameter.

The message handler will typically traverse the iKnowBase E-mail Reader tables, using the iKnowBase Service API to create the required iKnowBase documents and objects.

iKnowBase comes with a default message handler TEST_INSERT_MAIL, useful for testing and simple setups. This message handler has the following features:

• Runs through all e-mails where the IKB_EMAILREADER_MESSAGES.USER_STATUS is blank.
• Creates an e-mail document for every new e-mail. It is set up with a few external codes to identify the document type and the attributes for both the sender and receiver.
• For each e-mail message, the message handler runs through all attachments and creates them as sub documents to the main e-mail document.
• If the e-mails are saved without errors, the IKB_EMAILREADER_MESSAGES.USER_STATUS is set to LOADED. If some error occurs while saving, an error message from ikb_integration.integrate_objects appears.

Reliable Messaging

The iKnowBase E-mail Reader guarantees once-and-only-once semantics of message delivery and processing, given that the message handler is built according to certain principles, see the last bullet point of this section.

• The iKnowBase E-mail Reader retrieves all messages from the mail server.
• No messages are deleted from the server until they have been committed to the database in a transaction.
• The iKnowBase E-mail Reader does not duplicate a message in the database (using the MESSAGE_UID from the mail server to identify the message).
• If messages are deleted from the iKnowBase E-mail Reader tables, but still exist in the mail server, a problem occurs. This happens if the mail server fails during message deletion. To avoid this problem, use the field IKB_EMAILREADER_MESSAGES.USER_STATUS to indicate that a message has been processed, rather than deleting the message. The iKnowBase E-mail Reader program can then detect duplication even after the message has been processed.

Running the iKnowBase E-mail Reader

The E-mail Reader program is distributed as a java .jar-archive, which can be run using the following command:

java -jar iKnowBase-<version>-EmailReader.jar

where version is the actual version of iKnowBase Content Services.

The E-mail Reader program accepts the following parameters.

In general, the recommended setup is to use a property file, and enter all the parameters in that file. The syntax of the file has multiple lines, where each line contains a parameter name, an equals sign, and a value. Lines beginning with a hash sign are comments:

# Database connection
url=url
username=user
password=pws
# 3600 seconds = one hour
interval=3600

Configuring logging

The iKnowBase E-mail Reader relies on log4j from the Apache logging project for logging. You can configure the log using a custom log4j properties file.

If you do not configure the log, the program uses a default configuration which logs detailed messages to standard out, and warning messages to the ERROR_TAB table in the iKnowBase database. The actual setup for this logging can be seen in the file log4j.properties which is distributed along with the iKnowBase E-mail Reader program.

To customize the logging, create your own log4j properties file with the parameters that you want and then start the iKnowBase E-mail Reader program with the extra -log4j <filename> command line argument.

For information on log4j configuration, see http://logging.apache.org/log4j/1.2/index.html.

Note: If you want to log to the Windows event log, you need to install the DLL available through the log4j project.