News
AMTSO Membership Approves Major Step Forward in Testing Standards
May 2017 by Marc Jacob
The Anti-Malware Testing Standards Organization (AMTSO) moves the first ever set of
official Standards for anti-malware testing forward to the next level – the
development of a full accreditation and endorsement program.
Several recent controversies in published tests have highlighted the need for
objective standards to measure the quality and fairness of tests conducted on
security products. AMTSO’s Standards project is designed to provide a framework to
enable unbiased evaluation of testing approaches and methods.
“The Draft Standards aim to provide guidance to testers on how best to ensure that
the testing of anti-malware solutions is open, transparent, and fair,” said Dennis
Batchelder, President of AMTSO and CEO of AppEsteem. “Tests should provide useful
and actionable intelligence for everyone who relies on test data, from analysts and
media to purchasers of security products, as well as vendors themselves trying to
improve their products. These Draft Standards are designed to encourage and
facilitate that clarity and fairness.”
“The AMTSO Standards will give testers protection against accusations of unfair
practices and poor testing,” said Simon Edwards, Chairman of the Board at AMTSO
and Director at SE Labs. “By ensuring that all stages of the testing process are
clearly described, well documented and open to comment and feedback, testers are
given a strong position to back up the claims made in their test reports, and AMTSO
is committed to supporting and defending high-quality tests that follow the Draft
Standards”.
Since its formation in 2008, AMTSO has leveraged the combined expertise of
specialist testers, developers of security products and academics working in the
security field to develop a set of guidelines and resources to promote better
testing.
In 2016, the organization formed a Standards Working Group (SWG) to drive the
development of a definitive set of industry standards. The SWG regularly sought
comment from AMTSO’s membership and the wider security community. During the
latest round of feedback in the last few months, over 200 comments and suggestions
were collected and absorbed into the final Draft document.
“The Standards Working Group has put in a huge amount of work building out these
Standards,” said John Hawes, COO of AMTSO. “We’re very proud of how far
we’ve got in little over a year, and we are looking forward to getting the
structures and processes in place to ensure the Standards are implemented and
adopted across the industry.”
At a recent AMTSO meeting in Kraków, Poland, AMTSO members adopted the Draft
Standards and authorized the Standards Working Group to proceed with developing the
procedures and operations to implement the Draft Standards. Over 60 delegates
representing 35 member organizations were present for the discussion and voting.
"The WannaCry outbreak had every security vendor working extra hours to demonstrate
their product stopped it, which shows the importance of independent testing to
verify vendors’ often bold claims,” said Martijn Grooten, Editor of Virus
Bulletin, an AMTSO member organization since 2008. “I was pleased to see the AMTSO
Testing Standards officially receive Draft status, helping testers adopt clear and
transparent procedures."
“In an industry as fast moving and increasingly technology diverse as endpoint
security, there is a strong need for standard test methodologies to help consumers
and businesses determine what products are best for them,” said Norm Laudermilch,
CISO at Sophos. “AMTSO’s vote to approve Draft Standards is a monumental move
toward producing fair, consistent, and quantitative metrics to drive those
decisions.”
