Meet the SIMs
- By John K. Waters
The SIM market is new enough that it’s not easy to make apples-to-apples
comparisons of these products. Amrit Williams, a Gartner analyst, offers these
criteria to help focus your evaluation.
What information does it collect and what does it report against? Does it capture
data from the network devices, the operating system logs, the database management
system logs, the firewalls, the intrusion detection system sensors—in
other words, all of the security resources that I need to capture data from?
Also look at things that are important for your particular environment. “One
organization may be concerned primarily with correlating firewall log data and
IDS log data,” Williams says. “Another organization may focus on
HIPAA requirements and care only that they are capturing log data from ID and
access management applications.”
How is the data collected? Does the SIM solution require agents to be installed
directly on the monitored system, or is there some type of aggregation point,
such as a syslog server, from which the data is collected? Can you invoke data
collection from a command-line interface with the tool itself when you need
to? When is the data collected? What’s the period of time during which
the data is pulled? Does the product integrate with or have an API that allows
other systems to collect data from it?
“Keep in mind that we’re talking about potentially hundreds of
thousands of events being generated every day and filtering that down to the
one to 20 things that an organization really needs to take action on,”
You’re going to want a SIM solution that operates against current event
data and data that is more static. For example, firewall, IDS and host logs
generate new data into the system frequently; that data needs to be correlated
with things like vulnerability assessment data or asset classification data,
which isn’t changing much in comparison.
How well does the product map information from disparate security sources to
a common classification? A quality taxonomy helps aid in pattern recognition
and improves the scope and stability of the correlation rules.
Incident Management and Workflow
Security organizations don’t usually have the authority or the responsibility
to do very much to the network or the desktops. They capture this data and they
advise the operations team. This is especially common in large companies. So
at some point, the security people are going to need to pass the data collected
by the SIM to another organization in the enterprise.
“There should be some workflow embedded in the product to enable efficient
incident response,” Williams says. “If I’m a security engineer
and I see evidence that the firewall is under attack or I notice that our network
is so out of compliance that a critical event might occur, I need to be able
to be able to let [the operations people] know so that they can act.”
Scalability is generally achieved through a hierarchy of SIM servers. Tiers
of systems aggregate, correlate and store data, which can be reported against
centrally. Then there are servers with specialized functions such as being able
to act as a database for reporting and display or for correlating data.
Does the SIM require you to install agents on the monitored systems to collect
data? The security organization may not be allowed to install agents everywhere.
Does the SIM offer strong authentication and role-based access controls? Does
it support integration with enterprise directories? “That’s been
a pretty strong requirement for a lot of large organizations,” Williams
says, “because they’re usually adopting these enterprise-style directories.”
Does it perform auto discovery? Does it provide asset classification features
to help you to prioritize threats?
“These things aren’t specific to SIMs,” Williams adds, “but
should be considered in the evaluation of many security products.”
Embedded Security Knowledge
How much security know-how does the vendor bring to the table? How sharp is
the support team? How savvy are the tools? These are fair questions to ask during
any product evaluation, but considering the relative youth of this product category,
they’re essential when assessing SIMs.
Look for things like a library of pre-defined correlation rules, some pre-defined
corrective actions and pre-defined threat analysis; some regulator compliance
reports, and threat and vulnerability information displayed in the context of
Back to feature: Traveling
at a Zillion Events Per Second
John K. Waters is a freelance writer based in Silicon Valley. He can be reached
at [email protected].