Eugene's observations

Each year I review the comparative group tests of the major labs to glean the consensus of the ?best? security product.  That is the one I buy many licenses of at wholesale prices and recommend to anyone who?ll listen.

The labs I generally look at are 

• ICSA
• West Coast Labs
• Virus Bulletin
• TopTenReviews
• AV-Test
• AV Comparatives
• Dennis Technology
• Anti-Malware Test Lab
• Checkvir/Veszprog
• Matousec
• MRG Effitas
• NSS Labs
• PassMark
• PC Security Labs

No one lab will give the definitive best product as they

• test different products - there are scores of anti-virus products available so each lab can only test a sub-set
• use different test environments - some test with viruses captured ?in the wild? while others roll their own bespoke test suites; some test on multiple platforms others just one version of Windows
• report results differently - some score 1-5 stars, others use 3 or more groups of words, others just award a pass/fail; so comparing one set of results with another is not straight forward
• record different capabilities - examples of capabilities that are measured are detection, prevention, cleanup, usability, cost, false positives, extra features, and performance
• test different catagories of malware - all will test trojans and worms but not all test keyloggers, screengrabbers, zero-day exploits?

So I look at all and form my own opinion based on the consensus.  Although all the labs are independent, some are commissioned to conduct tests by one of the anti-virus companies and those companies usually get a good score.  This is not because of anything underhand but shows that the issues found have been corrected.  As long as the reader is aware of this it is not a problem as the other products are tested on a level playing field.

I read through many reports and assess the value of the most recent evaluations.  I then form an almost subjective opinion based on objective criteria.