Press Center

Malware and antivirus software

News

How Do You Test Anti-Virus For Unknown Virus Detection?

Created: 2006-08-16 00:00:00

Bookmark and Share

Well, there's a right way and a wrong way. Unfortunately ConsumerReports.org didn't know of ESET NOD32 or the right way to test for unknown viruses either.

Here is what happened.

「To pit the software against novel threats not identified on signature lists, we created 5,500 new virus variants derived from six categories of known viruses, the kind you'd most likely encounter in real life.」 -ConsumerReports.org

How do they know that these are the kind of viruses you'd most likely encounter in real life? Do they plan to release them? I doubt it, but how do they know? They don't really.

「Our next round of tests helped us identify superior antivirus software by measuring how well each of the products identified new viruses even before their signatures had been downloaded. The antivirus programs did this by using a technique known as heuristics, in which they seek out behaviors rather than signatures.」 -ConsumerReports.org

There are two problems with their approach. First, writing viruses for this purpose risks the accidental release of new viruses, and second, it doesn't give real world test results.

The proper way to test the heuristics of anti-virus software is called retrospective testing. This means that you take a collection of viruses discovered in a specific period of time and then scan them with scanners that were last updated before that time. For example: I could stop updating a scanner on April 30, then collect samples found between May 1 and July 31. Now when I scan I am testing to see what *real* threats that people are *really* encountering are being detected.

Back in 2001, Igor Muttik from McAfee presented a paper on retrospective testing at the Virus Bulletin Conference. (http://www.mcafee.com/common/media/vil/pdf/imuttik_VB_conf_2001.pdf)

This paper inspired Andreas Clementi (www.av-comparatives.org) to do just that type of testing. Andreas Marx (www.av-test.org) also performs retrospective testing using the same principals as well. There is a good reason these highly respected test organizations test heuristics in this manner. They both know that writing viruses is not proper behavior and that retrospective testing provides scientific, real world results.

It is a pity that ConsumerReports.org didn't know about the safety and scientific advantages of retrospective testing – it would have lent real world credibility to their results.

Why ESET?

ESET has over 25 years' experience of helping people to Enjoy Safer Technology. Our software is light on hardware, but hard on malware.

Our Technology

ESET’s award-winning NOD32® Antivirus technology is at the cutting edge of digital security. It’s updated daily to keep you secure.

Free Support

Enjoy your free, industry-leading customer support locally. For technical, sales and marketing enquires dial (852) 2893 8186.