Integrating with Oracle Secure Enterprise Search

The iKnowBase crawler for Oracle Secure Enterprise Search (SES) is an optional component that makes it possible to index the iKnowBase content through the premier standalone search engine of Oracle.

The iKnowBase crawler supports the following features:

• Deliver any content in the iKnowBase Content Store to Oracle SES for indexing.
• Control which documents and attributes are indexed.
• Security with the use of existing ACL-mechanisms in iKnowBase.
• Display of documents in context through the use of Target selectors in iKnowBase.

Use the Crawler definition component in iKnowBase Development Studio to set up an iKnowBase crawler.

Concept

The following are the key concepts around which the iKnowBase crawler for Oracle Secure Enterprise Search is built.

To index data:

• In iKnowBase, you can use a Search source to control which documents and attributes that are indexed.
• In iKnowBase, you can use a Crawler definition to control the state of indexing. The Crawler definition refers to a search source, and adds state information that lets Oracle SES index only new and changed documents.
• In Oracle SES, you can create and run an iKnowBase source to index the data. A source is configured to use a crawler definition in iKnowBase. As part of indexing the data, Oracle SES stores information on which access control list (ACL) is applicable to each document.

Note: You can use different sources to facilitate SES searches in subsets of the available information in iKnowBase.

To search for data:

• In Oracle SES, you can execute a query. Oracle SES asks iKnowBase for a list of ACLs that are accessible to you as a user. Oracle SES caches this list, to reduce the load on the repository.
• The search results that the Oracle SES returns refer to a redirector servlet in iKnowBase. This servlet verifies document access, and decides the context to display the document. This servlet also utilizes the target selector mechanism in iKnowBase to decide the actual page.

Note: Configuration of target selectors is a critical task in using Oracle SES to search for and present the iKnowBase content.

Install the iKnowBase adapter into Oracle SES

To set up an iKnowBase source in SES, perform the following steps:

1. Install the iKnowBase crawler adapter (iKnowBase-7.0.4-SESCrawler.jar) into $ORACLE_HOME/seshome/search/lib/plugins/iknowbase/, and restart Oracle SES.
2. From the Oracle SES Administration Console, under “global settings”, create a new “source type” with these parameters:
   • Set the name to "iKnowBase"
   • Set the description to "iKnowBase Crawler for Oracle SES"
   • Set the plug-in manager class name to "com.iknowbase.search.oracleses.IKBCrawlerPluginManager"
   • Set the plug-in jar file name to "iknowbase/iKnowBase-7.0.4-SESCrawler.jar"

Create a source

To set up an iKnowBase source in SES, perform the following steps:

1. From the iKnowBase Development Studio, define a Search source, selecting the documents and attributes you want to index. Note: The names given to the attributes are set as the search attributes in Oracle SES.
2. From the iKnowBase Development Studio, create a Crawler definition.
   • Specify the Search source you want to use, defining the source data to be crawled.
   • Select the Domain for which to index data. This is used during a search, when the Target selector decides in which context documents should be opened. Note: This crawler definition id is used when defining the iKnowBase source in Oracle SES.
3. In iKnowBase, make sure that there are Target selectors for all relevant documents.
4. From the Oracle SES Administration Console, define a source for the iKnowBase crawler
   • Specify the database connection information (database URL, username and password) to connect to the database.
   • Specify the iKnowBase Crawler definition id, which identifies the crawler definition in iKnowBase.
   • “Delete index on crawl” makes the crawler recreate the index on each crawl. This means that the index is deleted at the beginning of each crawl, and that all documents must be fetched for each crawl. This is not a sustainable idea for production use, but very convenient when setting up and testing a new search source.
   • In the Authorization page, leave defaults enabled.
5. From the iKnowBase Development Studio, after creating the iKnowBase crawler in Oracle SES, edit the crawler definition. Under “crawling parameters”, set the property “number of threads” to 1.

Now you have created an iKnowBase Crawler in Oracle SES.

Schedule a source for indexing

After you have created an iKnowBase source, you are ready to start crawling.

To conduct a test crawl and enable automatic scheduling, perfrom the following steps:

1. On the Oracle SES Administration Console, define a SES crawl schedule.
2. Run the schedule manually, to verify the crawler log.
3. If the crawler log is accurate, enable automatic scheduling.

Security

The iKnowBase crawler supports a secure search based on the Oracle SES ACL-mechanism.

• During a crawl, (ie indexing of data), each document is stamped with its iKnowBase ACL.
• While searching for a document, Oracle SES asks iKnowBase for a list of ACLs that the logged in user is a part of. This list is applied to the query, which returns only the documents with the available ACLs.
• When you click a document link to open it, the regular iKnowBase security mechanisms are used.

The security mechanism has the following limitations:

• Oracle SES stores the ACL on each crawl. If the ACL of a document changes, this change is propagated to the Oracle SES index on the next crawl. Until the next crawl, the original ACL is applied to all search operations.
• Oracle SES caches the list of ACLs for a given user for a specific period of time. During this time, Oracle SES does not apply changes to the ACLs that are available for the user. How long this list is cached, can be configured in the Oracle SES Admin application.
• Oracle SES applies ACLs for a user to each search operation. The number of ACLs per user should be limited, due to performance. The threshold depends on multiple factors, mostly related to the hardware on which Oracle SES is running.

Search User Interface

The iKnowBase crawler is not related to the user interface that is used to search the data. You can use any available mechanism, such as the standalone GUI, predefined portlets, or custom code.

Custom filter plugin

The iKnowBase crawler exposes a plug-in extension point, where it is possible to add a filter which is used to filter documents before indexing.

No default plug-in is provided. Perform the following to implement and configure a plug-in:

• Implement the interface oracle.search.sdk.crawler.DocumentServiceManager.
• Integrate the plug-in into the same jar-file as the iKnowBase crawler (crawler plugin), to avoid class loading issues.
• From the Oracle SES Administration Console, enter the plug-in class name on the parameter page of the iKnowBase crawler (crawler plug-in).

The plug-in will perform much the same job as any filter in a document pipeline. However, since this plug-in is executed before Oracle SES can see the data, it is also possible to replace the document content itself inside this filter.