The evolution of Security and Information Event Management systems

A recent study by McKinsey & Company shows that 72 percent of companies use social media in some way. The McKinsey Global Institute (MGI) estimates that, while the marketing and sales impact of social media is significant, up to twice as much potential value lies in using social tools to enhance communications, knowledge sharing, and collaboration within and across enterprises[1].

However, increased usage of social media by employees, “Bring Your Own Device” (BYOD) within the enterprise, and unlocking the potential of social media for employee productivity also brings with it significant risk of improper use, security issues, and information leaks. Security Information and Event Management (SIEM) systems are thus faced with extending their capabilities beyond the physical enterprise in order to identify these new risks.

I had the opportunity to talk with A.N. Ananth, CEO of EventTracker, to talk about the state of the art in SIEM technology and the impact that increased social media and cloud usage within the enterprise has on SIEM systems.

A SIEM backgrounder

SIEM systems are broadly defined and encompass every aspect of enterprise business. However, all these things can be grouped into two main categories:

Operations information: For example, things like gathering various congestion or network usage information, various operational systems within the enterprise, health status, user activity on various machines, file transfers, or what applications are running, and even down to things like identifying when printers are out of paper.

Security: This includes functions such as monitoring firewall probing attempts, user anomalies, and what domains of the network users are attempting to access.

Every computing platform, network device, and application in the modern enterprise network can and does generate log data. These logs tend to contain large amounts of information, much of which is simply informational, and this information may come in many different formats. This creates huge demand on SIEM systems in terms of how information gets to the SIEM system and what format it comes in. Further, access to these systems and the Operating Systems (OSs) running on them makes a difference as well in terms of gathering information. Ananth summarizes all the challenges of these disparate information sources by saying that a “360 degree view is required for SIEM systems”:

The host: Sun Solaris, Windows, Linux, HP-UX, AIX, AS/400 – All these systems have different ways of storing and providing access to logs. The formats might be XML, binary, syslog, or text file.

Network devices: Various intrusion detection and/or prevention systems, routers, and gateways may be managed by Simple Network Management Protocol (SNMP), Remote Monitoring (RMON), or provide custom access to information through web pages.

Software application information used within the enterprise: This may encompass 20-30 different formats, and SIEM systems must support them all in order to be effective. Some systems may be pretty unique, like thumbprint scanners, or contain physical components like racks of servers that provide log files for heat, fan operation, and other platform management information.

A SIEM system must tie all of these elements together into one effective information-gathering environment where information is easily accessible and anomalies are easily detected. That can be a major challenge, and these challenges are currently being addressed in many SIEM systems.

Enterprise SIEM challenges

Employee use of Web and cloud services

In the modern enterprise, the data center has rapidly adopted virtualization. This includes equipment hosted on premises, co-located at a provider, or delivered as a service. The challenge is to provide support for all these delivery models. A software-based SIEM solution has the advantage of being native in the virtual world, as well as highly scalable both up and down to suit the conditions.

“Known bad” and “unknown bad”

In Ananth’s opinion, the behavior aspect is something that is critical for the enterprise of the future. He summarizes this concept as “the known bad” and “the unknown bad.” If behavior is a known bad, it is known when seen and one can make a rule for it. That is easy: Here are all the alarms and we will tell you when the known bad is detected.

Unknown bads, on the other hand, are far more difficult to characterize. For example, things like users logging in are normal – they work there and they are authorized for the system – but maybe that person normally logs in 3 times between 8 a.m. and 5 p.m. and today there were 50 logins. Or there were a number of logins at night or on the weekend. Or maybe there are large file transfers that are unusual. This kind of behavioral tracking and event notification that occurs when something happens outside of the “normal behavioral” pattern is the best defense against “the unknown-bad.” A good SIEM solution needs to incorporate some type of behavioral “normalcy” baseline in order to determine the unknown bad events.


Smart phones and tablet devices are quickly supplanting a variety of tasks within the enterprise. To that end, an employee may want to use personal devices for business purposes. In a competitive universe, companies need to increase productivity by enabling these things, but what about stolen or lost equipment? Do strangers have the potential to access corporate information? What kinds of things will the enterprise IT department be able to do with this device? Can they wipe the memory from a lost or stolen smart phone or tablet? Ananth mentions that the Blackberry Z10 has a work side and a personal side, so that if the phone is lost the business side can be wiped out. These kinds of SIEM-friendly capabilities may be the next big thing for protecting enterprise information while enabling the “bring your own device” paradigm.

Any log standards

For the last 40-50 years there have been no standards for logs: Windows, Microsoft, IBM, Cisco, and VmWare are all vendor standards that are really not standards. MITRE is working on Common Event Expression (CEE), with the objective of having the U.S. government require compliance with CEE ( This ambitious effort is in its early stages, but if everyone were to adhere to this standard, all kinds of back-end capabilities could be possible. A practical problem is the refitting of legacy systems to a new standard of logging, a monumental task.

EventTracker SIEM

Risk-based incident prioritization

Ananth describes how to make sense of the massive amount of information being gathered as a three-tier pyramid shown in Figure 1.

Figure 1: The EventTracker SIEM system is conceptually organized to prioritize risks in a three-tiered pyramid architecture.

At the bottom layer of the pyramid there could be millions of events generated from all the sources being monitored within the enterprise. These events could include people doing things on the network or specific machines, running applications, and embedded device health and status. Ananth also mentions that data center virtualization also impacts these events with the move towards cloud architectures, as well as increased use of services such as social media.

The next level up in the pyramid narrows the large number of daily events to ones that require monitoring. These might involve things that may be an early sign of system failure or when systems report accesses or activities that are not allowed. At the highest level in the pyramid are critical events that require immediate attention.

If you can not enable proper prioritization between these levels with a SIEM solution, the solution becomes ineffective. Rules can also be set up so that activity on the lower levels of the pyramid is logged and stored, while mid-level events in the pyramid show up in summary form on the system health dashboard. The top level of the pyramid represents events that proactively reach out to the administrators by email or some kind of real-time notification.

Another important aspect to a SIEM solution involves performing automated correlation between events. The idea behind automated correlation is that if the SIEM system sees something important, it can “bolt the door” first and send the notification second so no further damage is incurred until the administrator has time to respond.

One-minute manager

The next concept that is a natural extension of the pyramid architecture is the “one-minute-manager.” In one minute, can you show me what I need to know? Figure 2 shows the high-level summary of what is going on in the EventTracker SIEM system. There are three views – dashboard, graph, and tabular.

Figure 2: EventTracker contains a “one-minute manager” user interface that displays risks in a high-level summary so that administrators can quickly react to high-priority threats. Here, the main screen shows a “High” risk incident on the ACCOUNTING machine.

The graphical view shown in Figure 2 is a summary of all the events, filtered and sorted by risk. At any given time, this view allows the administrator to understand the overall health of the enterprise environment at a glance.

Fast search, endless refine

Drilling down on the data from the dashboard, the administrator can focus on the top issues within the environment. Figure 2 displays both a critical event and a serious event, so from there drilling down on these events can begin, as shown in Figure 3.

Figure 3: Once high-risk events have been identified in the “one-minute manager” view, administrators can drill down on specific incidents. A drill down from the main screen shows details of the incident on the ACCOUNTING machine: User cmills has inserted a personal USB at 9PM.

Figure 3 shows the event information for an incident. In this example, cmills inserted a USB stick into the accounting machine. In this case, cmills has the authority to access the machine but did so at 9 p.m., which is after hours and so out of the ordinary. This particular event therefore encompasses time-sensitivity, something that may be simply informational or a serious event depending on what time it is.

From the information in Figure 3, administrators can continue to drill down on not only the fact that a USB media was inserted, but exactly what occurred during the event, as shown in Figure 4.

Figure 4: Once an event has been identified, administrators can investigate the specifics of the incident. For example, this report shows that user cmills was using a personal USB to delete a personal file (football_xls) after working hours. Unfortunately, the USB was infected with a Trojan virus (hacker.exe) that jumped to the machine ACCOUNTING.

The screen in Figure 4 shows that a football stats file was deleted, which is probably why the event happened after working hours. However, along with the delete, there was a file copy of a file called hacker.exe – the USB stick was infected! From this information, further information can be correlated from process events to determine if hacker.exe ever ran on the accounting system and where the file is located. There are even facilities within EventTracker that provide links to information so the administrator can learn about what is known about hacker.exe, what kinds of things it does, and the disinfecting procedures.

While the anomalous event was fairly innocuous, the related information was able to find an inadvertent security issue that the administrator could locate, learn about, and fix.

Incorporating Web application and social media monitoring

Incorporating SIEM with Web and social media usage monitoring is adds another source of log data. In some cases organizations use a proxy server for all Web access. Users access the network via the proxy servers, which in turn record access via logs. These would be captured and reported on by a SIEM system like EventTracker. One notable example of social media is the use of peer-to-peer networks to share music or video and thereby violate copyright. Universities with very high bandwidth pipes and lots of tech savvy students are always concerned about misuse of their resources for this activity. Port monitoring to detect such traffic is one way of managing this.

I asked about integrating new age equipment like Deep Packet Inspection (DPI) systems that may be able to monitor and report not only running web applications, but exactly what each user is doing or saying within the context of that social media platform. Ananth said that integrating these kinds of new age devices involves answering two questions:

How does the information get to the SIEM system?

What format is the information in?

From there, if the protocol to get the information to the SIEM system is non-standard, developing a receiver to collect the information would be required. This is uncommon but possible. A parser to understand the format and incorporate information into the SIEM system would be the next step. So as these new social media and web monitoring devices become available, they should be able to be integrated with SIEM systems that have this kind of modular architecture.

SIEM and the new enterprise

The enterprise environment is a challenging place to collect, manage, and monitor event data. The increased virtualization going on in the enterprise and increased use of social media and web applications within the environment present new challenges that, if addressed effectively, can enable a higher level of employee productivity without security concerns.

For more information, contact Curt at