Event Cache Demystified

The event cache or rules cache as it's sometimes known is one of the most misunderstood areas of the Tivoli Enterprise Console. This tip aims to demystify the Event Cache, it's purpose, it's day to day running and most importantly it's configuration.

What is the Event Cache?

To understand what the event cache is we must first understand how the Event Server as a whole works. The following diagram describes the flow of events through the various components of the Event Server.

As you can see when an event leaves the reception engine, the rules engine takes over. Any event that is processed by the rules engine, and not dropped is placed into memory - into the Event Cache. The rules engine then passes the event to the dispatch engine which places it in the Event Repository and fires it to any open consoles.

What does the Event Cache Do?

The purpose of the event cache is to hold recent events, so that the rule engine when called upon can correlate new events with existing events.

What Events are Cached?

When the Event Server starts it loads events from the Event Repository based on some configurable time and size parameters (see Parameters below). This is the last time events are taken from the Repository and put into the Event Cache.

After the initial load all new events that aren't dropped by the rules engine are cached whether they are in an OPEN, ACK or CLOSED state. These events are written to the cache by the rules engine itself.

What uses the Event Cache?

The event cache is used only by the rules engine for use within rules processing. Contrary to popular belief the Event Consoles do not get there events from the Event Cache, rather Consoles are populated by the dispatch engine which gets the events from either the Event Repository (during startup) or from the rules engine.

Parameters

The event cache determines what to load and what to remove (from the cache only) by way of three Event Server parameters. These can be set either by the GUI (Event Server - Parameters) or from the CLI wsetesvrcfg (and viewed by wlsesvrcfg). The three parameters are as follows:

• The first is the 'Event Cache Size' (wsetesvrcfg -c). This specifies the total number of events in the cache. To get an idea of how much memory this will take up, the maximum size for an event is 4k, but the average is nearer 2k, so you can roughly estimate that 1000 events will require between 2 and 4 Mb of memory.
• The second is the 'Time to keep Closed events'(wsetesvrcfg -k). As this implies this parameter specifies how long a CLOSED event will remain in the cache. After the event has expired, it is automatically removed from the cache (but not the Event Repository). When this happens this event will no longer be included in any of the rule processing, and therefore will never appear as a result of a first_instance, all_instance, first_duplicate or all_duplicate call.
• The final parameter is 'Time to keep Non-closed Events' (wsetesvrcfg -n). This parameter specifies how long all non closed (OPEN, ACK, RESPONSE or any custom status levels you may have defined) are kept in the cache. The default is 180 days. As in the previous case older events are automatically removed from the event cache (but not the Event Repository), and again as above will not be part of any subsequent rules processing.

Perfomance / Size Considerations

When we look at performance of the Event Server we need to look at what exactly uses the Event Cache. The four main rules templates that use the cache are:-

• first_instance
• all_instances
• first_duplicate
• all_duplicates

These templates essentially search the event cache for specific events, in the case of the 'all' templates the whole cache is searched sequentially, with the 'first' templates, the rules engine stops searching the cache when it finds the first match.

There are really two main performance issues to consider. Size and use of rules.

Size

The biggest consideration is the size of the cache. How big should you make it? This is a difficult question to answer. It really depends upon your business requirements. How far back in time do you need to perform event correlation - unfortunately only your organisation can answer that question.

What is the maximum size of the event cache? Well as yet I've not found a concrete answer to this, however in my experience I have found that anything beyond 2,500 events in the cache even on the most powerful servers, the rule processing becomes so slow that the whole Event Server grinds to a halt.

Rules

A secondary consideration is the use of the all and first templates. Searching through the whole cache is obviously less efficient than looking for just the first matching event, therefore performance is impacted using the all templates.

At the end of the day the size of the cache is always going to be a trade off between performance and business requirements, a small cache will perform better than a large cache - but will it meet your requirements of correlation.

Removing Events from the Cache

There are only three methods of removing events from the cache.

• Automatic expiry of events based on the parameters described above
• When the cache becomes full (100%) the Event Server will automatically remove the oldest 5% of events regardless of their current status. When this happens an internal TEC event will be generated - "Cache Full - forced cleaning"
• Deleting events from the Event Repository and then stopping and starting the event server. When the server starts it will then reload events based on the timescales defined above.

The command wtdbclear does not (even with the -f option) remove any events from the Event Cache.