Traditional measures of security product performance rely on isolating known threats and attempting to infect test AV products in controlled laboratory settings.  This approach has been rendered hopelessly inadequate as cybercriminals have learned to overwhelm AV products with tens of thousands of new threats and variants each day.  As recently as 2005, less than 50 new threat variants per hour were being observed by industry researchers.  Today, over 2000 new threats per hour are being observed.



Historically, the standard metric of AV product security performance has been the detection rate (# of threats detected / # of threats exposed).  As AV product vendors have altered their security strategies and product architectures in response to cybercriminal innovation, new metrics have emerged that are more indicative of the actual security provided to customers in today's threat environment.  Among these are:  (1) ability to detect threats based on their source (URL or IP address) rather than by analysis of malware code;  (2) the time from when a security vendor first sees a new threat to when the vendor is providing customers active protection against it (the so-called, “time-to-protect”). Moreover, AV product testing conditions themselves have been called into question.  To accurately predict actual product performance, some sort of software equivalent to clinical trials for pharmaceuticals is required to assure that products are tested under conditions relevant to customers.

GOALS:

Allow representatives of the leading independent testing labs to explain how they have adapted their security product testing metrics in response to cybercriminal innovations and security vendor responses - with the goal of making the results of independent lab testing more accessible and valuable to security product recommenders and purchasing decision makers.

TAKEAWAYS:

1.  Learn why traditional security product testing methods do not adequately measure the actual security provided to customers by security products.

2.  Learn how cybercriminal innovation has forced the security industry to alter both security product architectures and the methods for measuring their performance.

3.  Learn what new metrics are being adopted to measure the actual security performance in today's threat environment.

4.  Learn how to interpret the metrics and product rankings in security benchmarking tests conducted by independent laboratories.

TARGET ATTENDEE:

Security product recommenders and purchasing decision makers.



Hosted by Trend Micro