Testing Anti-virus Tools

Would you be alarmed to discover that neither of the leading anti-virus tools could detect some viruses already on your hard drive? We were.

Introduction
Norton AntiVirus and McAfee VirusScan are, respectively, the #1 and #2 anti-virus products in the market and both are regularly placed in the top 10 sales figures of software retailers across Canada. We used to be big fans of McAfee VirusScan, but some "poison pill" updates caused us enough trouble that, back in Y2K, we switched to Symantec Norton AntiVirus 2000. Then -- surprise! -- we found that its successor, Norton AntiVirus 2001, also caused problems on some of our test systems, so we went looking again. This is the tale of the programs we tested and what we discovered. As you'll see, the retail versions of Norton AntiVirus and McAfee VirusScan were unable to detect two different varieties of viruses -- both of which have been around for more than six months -- lurking in our saved email.

Why Anti-virus Software is important
According to IBM researcher Sarah Gordon, author of a number of books examining the mindset of virus authors, there are an estimated 8,000-10,000 virus writers in the world -- the vast majority of which are teenage boys. With over 55,000 viruses in circulation, many of which are transmitted via email systems and can travel embedded in common document types, virtually all computer users are at risk.

How We Tested
For each of the anti-virus utilities, we installed the latest updates, enabled "scan all files" and chose all real-time and web-scanning options. We tested each program with a variety of infected files, including the recently detected SirCam worm. This way, we knew when a program was working. We also attempted to open infected ZIP files, download infected email messages and copy infected folders over a network

Trend PC-cillin
The #2 anti-virus utility is PC-cillin from Trend Micro, which boasts the distinction of being the granddaddy of AV vendors. According to David Perry, Global Education Director for PC-cillin, the company currently focuses primarily on the enterprise market, with anti-virus solutions for servers, email gateways and proxies, holding a 54% share of this server oriented antivirus market. But, says, Perry, "all of us started out on the desktop" and the company remains strong there -- particularly in Asia. In fact, most of the company's 32 million users are in Asia, says Perry. The company claims 88 percent of the market in Japan and a whopping 92 percent in China, thanks primarily to bundling deals with a number of PC motherboard manufacturers

Unfortunately, during our battery of tests of Trend Micro's PC-cillin 2000 (as of July 28, 2001, the latest version available), we uncovered a seemingly serious problem. On a machine NOT protected by PC-cillin, we copied a document infected by the SirCam virus into a folder called "WARNING contains virus." We did NOT compress the file or the folder. Inside, was only a SirCam-infected Excel file.
We then installed PC-cillin 2000, updated it to the latest virus definitions (rebooting when prompted) and performed the following tests.

TEST #1 (SirCam test)

Right-click a folder containing a SirCam infected Excel document and choose to scan with PC-cillin. When we did this on July 28, we received no warnings. (The problem was addressed in a July 29 update -- a full 10 days after Anti-virus forces around the world began issuing warnings about the insidious new worm.)
Open the folder. Again, no warnings.
Right-click the infected file (e.g., infected.xls). There is no option to scan with PC-cillin.

In our estimation, this amounts to a failed test.

TEST #2 (Windows Script Host Shell Object)

Enable PC-cillin
Connect to internet and visit http://astalavista.box.sk
in the search box, type "ebook creator x.x" and click search
Click the first link, leading to a site called "cracks.am"
Click the link to eBook Creator. (You do not need to actually download the software.) WARNING You may see some pornographic images in banners and/or pop-up windows. Please do not perform this step in the company of minors, or if you do not wish to view potentially objectionable material. There is also a possibility that nothing may occur, as such sites come and go frequently, and change banners and pop-ups regularly. You may close the pop-up windows by pressing Alt-F4.
Now, look in your Start Menu. When we performed the above steps, a porno icon (shortcut) and casino icon were deposited there.
Look on your desktop for more evidence of WSH shell execute exploits.
Delete these icons if present.

PC-cillin also failed this test.

Norton AntiVirus 2001, in contrast, passes both tests. All the other antivirus programs reviewed fail test #2.

For more information on the Windows Scripting Host, visit the Sophos site at http://www.sophos.com/support/faqs/wsh.html

McAfee VirusScan for Windows 95/98/Me and Windows 2000
Lightweight Protection for those with slower computers: McAfee VirusScan requires the fewest system resources.

McAfee VirusScan 7.0
Improved Protection - JavasScript exploit blocking and Windows Scripting Host protection have been added. See our report on McAfee Internet Security for details.

About McAfee VirusScan
McAfee has had a rough time since mid-2000, when Windows Me was released to manufacturing. You see, no version of VirusScan available at the time was compatible with it -- a situation that the company didn't effectively address until Feb. 2001. As a consequence, there were a lot of people who dropped McAfee's product in favour of Norton AntiVirus. To make matters worse, the company suffered further consumer unrest over technical difficulties now commonly called the "poison pill" DAT file upgrade later that year that further reduced consumer confidence in its products. (I'll admit to falling off the McAfee nightmare train around this time.)

Version 6.0, released in 2001, didn't do anything to improve the company's reputation. It had the unfortunate propensity to corrupt users' Outlook Express email messages, and it proved to be incompatible with some popular standard Windows components including -- unbelievably -- the decidedly non-invasive Solitaire and FreeCell games. Worse, it had problems scanning long file names, and files inside directories with long path names. It wasn't until Feb 2002 and the release of version 6.02 that McAfee seemed to get its act together in addressing these issues.

As a consequence, McAfee now hold a much smaller share of the lucrative consumer antivirus market. Perhaps most tellingly, Symantec's Norton AntiVirus products regularly figure strongly in the top ten of Amazon's bestsellers list; McAfee in May 2002 was #83. One needs only to read a few dozen user comments to understand why. Problematic installations, poor technical support and an awkward graphical interface are all common complaints.

Sadly, the product was once a best-of-breed contender. Independent studies released around the time of VirusScan 4.0 showed that VirusScan detected the most viruses from floppy disks, Internet downloads, email attachments, Intranets, shared files, CD-ROMs, and online services. It was the industry leader in detection and cleaning of destructive ActiveX and Java applets. More than 60 million people, 40,000 companies, and 80% of the Fortune 1000 used VirusScan to help keep their desktops running smoothly and virus-free. In head-to-head testing conducted by PC Computing, VirusScan beat out Norton AntiVirus on the strengths of its virus and vandal detection rate and its seamless, customizable scanning options. Version 4.0 was also touted by PC World as the best anti-virus product in its "Top Ten AntiVirus Tools" list. The product was awarded PC Computing's prestigious Most Valuable Product (MVP) award and was named "Editors' Choice for the Web Addict" in CNET's utilities round up.

We tested McAfee VirusScan v5.10 and version 5.21 under Windows 2000 and Windows Me. VirusScan is also compatible with Windows 95/98 and NT. (We'll test version 6.02 in a future update to this feature -- Ed.) You should, of course, always use an up-to-date virus checker -- and be sure to update its profiles, sometimes called virus "signatures," to enable the program to recognize the latest threats.

Things worked well when we chose to install VirusScan using the default options, but, when selecting a custom path as we did when we elected to install McAfee VirusScan to the D drive instead, the software couldn't find some necessary components and the scheduling configuration option (allowing you to set the program to run daily, weekly, or at any other predefined time) didn't work properly. This problem persists in version 5.21.1000.1, as recently as June 2001. The solution is easy: just install it to drive C.

We also noticed that some features are missing from Windows 2000, Windows Me and Windows NT (and the forthcoming Windows XP), including the option to scan your system on startup. Thankfully, recent versions have eliminated the software's gratuitous use of sound effects turned on by default when selecting options in its main configuration panel. (They're still available as an optional setting, if this sort of thing appeals to you.) The program does, however, sound an audible alarm when a virus is found -- a more sensible use of the feature, we think. Finally, the software's manually invoked Internet-based update option is less convenient than the automatic "LiveUpdate" feature of Norton AntiVirus 2001 or a similar feature in Kaspersky's AVP. McAfee does the job, but it's not in the same league as its leading competitors anymore.

We did, however, encounter more problems with McAfee VirusScan than with any other anti-virus tool we tested. Most seriously, all three of the test machines on which we installed an update in mid-July to test its ability to detect the the Sircam-A worm crashed when scanning a folder containing the worm. Also, VirusScan was unable to detect the worm in our email, using a standard IMAP server.

What's more, we tested McAfee VirusScan on a test system onto which we placed a I-Worm.KakWorm virus in a saved email message, stored in an Outlook email archive. McAfee did not detect the virus, even when the "Email scan" was enabled and all drives were scanned. When we placed the virus in a Word document on a diskette, the program noticed it, of course, but couldn't repair the document.

Older versions of McAfee's WebScan were even more troublesome. McAfee's WebScan 4.x, for example, had some serious limitations when used with certain browsers. For example, in IE3, the program's WebScan function interfered with the browser's ability to show HTML source. In Netscape and IE, right-clicking a file to Save As... bypassed WebScan's detection routines. The WebScan readme file discussed workarounds for these issues, but should users really have to fiddle around like this? We think not.

And the final straw was the program's repeated failures to successfully update itself via the Internet. We don't know whether Network Associates needs to upgrade its servers or what, but when a version of an antivirus tool doesn't recognize a current threat, such as the SirCam worm, and we can't even get through to to the update servers, it seems like it's time to switch antivirus tools.

Pros: Simple interface, slim resource requirements. Engine versions 4.0.25 or newer detect and protect against Windows Scripting Host viruses such as Melissa, LoveLetter, KAKworm, etc. (which are all on the list of the top ten threats).

Cons: Can't detect viruses in archived email messages; installer incorrectly assumes system drive is always "C:" Can not repair some infected documents (e.g., "W97M.Nono.A") that Norton AntiVirus and AVP can handle. Update servers sometimes too busy to deliver updates. After removal of VirusScan, Outlook still looks for McAfee Add-in files. (The missing scanning extension must be removed with Outlook's Add-in Manager.) Does not protect against WSH "Shell execute" exploits.

Kaspersky AVP
Superior virus detection capabilities for power users.

Kaspersky Anti-Virus Personal Pro, unlike the other programs tested here, successfully detected the virus saved in our test email message, saved in an Outlook Express message database. However, the program's interface for changing settings is awkward and the large number of options makes this program best suited to those who want maximum security and are willing to spend more time configuring their system to have it.

With so many more options that its competitors offer, AVP's interface can seem quite confusing -- a situation that isn't helped by some of the program's design decisions. When the virus scanner is started, several of the program's options and preferences pages become unavailable. Even more confusing is the fact that, if a virus is found but the preferences are set to "deny access," the program seems unable to remove the threat, even when you select the "delete this virus" option. You must stop the scan, change the settings to "disinfect" or "delete" and set any other options you want, such as the "Scan mail databases" option we consider to be this program's defining strength. Without these configuration changes, you may find it seemingly unable to automatically quarantine or disinfect an infected email message. Indeed, it took us quite a while to figure out how to configure the program to automatically delete our "test virus."

The program by default announces the presence of a situation it deems worthy of a "critical alert" with a noise not unlike the squeal of a stuck pig. Oddly, even after choosing to disinfect/delete our test virus and "apply these changes to all future alerts," the program, apparently unable to disinfect the file even when manually selected, would find the same virus day after day, alarming us each time with a hideous squeal. This lasted for about three days before we figured out how to properly configure AVP. Despite its failure to block a Windows Script Host Shell Execute exploit on a web page that placed an annoying "Internet Casino" icon in our start menu and on our desktop, an interface that at times seems overly complex and a disturbing tendency to slow down file system operations on two of the machines on which the program was installed, its ability to find viruses the other programs missed earns Kaspersky AVP (www.kasperskylabs.com) our vote for Editor's Choice. The company also offers a multithreaded version for Unix-based systems running OpenBSD (v2.8) and Solaris 8 for Intel processors.

Pros: Superior virus detection capabilities, including basic WSH protection, speedy scanning.
Cons: Somewhat complex configuration. After removal of AVP, Outlook still looks for its add-in files (as with McAfee, this can be solved with the Outlook Add-in Manager, found in the Tools menu > Options > Advanced Options.) On some systems, for reasons we were not able to determine, AVP slowed down some file system operations, especially renaming or creating new files or folders. On other, similarly configured machines running the same OS, no slowdown was apparent. Does not protect against WSH "Shell Execute" exploits.

Norton AntiVirus 2001
Keep it Simple: Norton AntiVirus 2001's automatic LiveUpdate and wide availability makes it the easiest choice to obtain and install.

In recent years, we've considered Norton AntiVirus (sometimes called NAV) to be the best all-round antivirus utilities for most users -- particularly users wanting reliable, automatic installation of those all-important "virus signature" updates. Indeed, Symantec's antivirus utilities consistently rank as some of the top five selling utilities with PC software buyers.

However, after a couple of years of nearly constant use and extensive testing, we've noticed a few definite weaknesses. For example, Norton AntiVirus 2001 displays a non-fatal error dialog quite regularly on every Windows Me system on which we have tested it. Fortunately, these errors don't seem to affect anything. A dialog pops up periodically, warning that "Winhlp32 has caused an error in Kernel32.dll. Winhlp32 will now close." Everything seems to work fine the next time you restart. Symantec says it is aware of the issue, but admits the cause is not known, and says there is no solution at this time. Oddly, its predecessor, Norton AntiVirus 2000, seems to work fine on the same computers. We encountered no consistent problems win NAV 2001 on machines running Windows 98 or Windows 2000.

We've also noticed that NAV 2000 and 2001 seem to hog more system resources than any other antivirus program we've used. On a Windows Me system with 256MB of RAM, running no applications at all other than a few of the usual task-bar utilities supplied with CD burners, graphics cards and other hardware devices commonly present in PC systems, available system resources ran as low as 30 percent. In our tests, this soon leads to instabilities, particularly when a memory hog like Office XP or Adobe Photoshop is then launched. Removing Norton AntiVirus from our test system and replacing it with McAfee VirusScan or F-PROT made these problems go away, so there clearly seems to be an issue here.

Norton AntiVirus, unlike current versions of Kaspersky AVP or McAfee VirusScan, requires a patch to work with Windows XP. A beta version of a patch to a routine called SYMEVENT is available, but some users may find it complicated to install on Microsoft's newest operating system.

Norton AntiVirus, like McAfee VirusScan and F-PROT, was unable to detect an embedded virus already present on our test computer's hard drive, contained in a saved email "PST" PostBox archive. Symantec has admitted that a bug in Norton AntiVirus allows SirCam to slip through its POP email filter undetected, due to the invalid MIME-header a SirCam infected message creates. (This bug also causes Symantec's enterprise-oriented Norton Antivirus for Gateways v2.x to fail to block the virus.) Symantec says it is working on a fix for the problem, which it will deliver via its LiveUpdate service when it is ready.

Despite this problem, Norton AV, like all of the programs in this survey, will catch most viruses when any attempt is made to download or execute one. The lesson here is that you cannot necessarily trust a program's real-time surveillance of an email inbox to protect against incoming threats. Indeed, in our test, only F-PROT protected an IMAP mail account, while Kaspersky AVP was the only product to detect threats lurking in previously downloaded PST archives.

To its credit, Norton AntiVirus was able to repair the "Nono" infected Word document we placed on a diskette. It also detected the Sircam-A worm. Norton Antivirus provided the best protection against Windows Scripting Host (WSH) exploits that to write to the Windows Registry via the RegWrite command. Without this protection, an unscrupulous hacker can do almost anything from placing an icon on your desktop or in your Start menu (usually to a porno site) to running a program without your knowledge the next time your system starts up. Thus, Norton Antivirus is the best choice if you frequent web pages with "multiple overlapping pop-up" windows, such as you might see if you ever visit websites with "click here to continue" gateways or banners of a pornographic nature. It is a little alarming, though, to know that you might already have a virus on your computer that neither of the two leading anti-virus programs detect.

Norton AntiVirus 2002
Best-of-breed: Norton AntiVirus 2002' improves upon its predecessor's email protection by more effectively trapping and automatically quarantining incoming viruses. Although version 2001 can be made compatible with Windows XP with a manual patch to a file called "symevent," better compatibility with Windows XP is also a highlight in the 2002 edition. As with the 2001 release, it relies upon a subscription plan which must be renewed yearly in order to take advantage of the company's automatic LiveUpdate definition updates. NAC2002 is our editor's choice as the easiest choice to obtain and install.

F-PROT 3.10
From Iceland comes the little-known F-PROT antivirus tool, by Frisk Software International. The program, for Windows 9x/Me/NT/2000, is fairly svelte, at just a little over 5.2MB, but it did a great job at sniffing out suspicious files on our hard drives. Once we configured it to scan "all files," it recognized the Sircam-A worm and other threats and also alerted us to snippets of suspicious code in some web pages we had been reading, which detailed how the ILOVEYOU virus works. If anything, it seems that F-PROT might be the most cautious of all the virus tools we tested. None of the others appeared concerned about a chunk of VBS script virus code embedded in a web page stored in our IE Content cache, but F-PROT warned us about it. It protected our IMAP email inbox, warning us about the Sircam-A virus before we downloaded it. It also warned us when we attempted to copy the file onto a different server. This is the kind of protection we like. No nonsense. It just works. (Remember to turn on the "all files" option, though.) Unfortunately, although it does protect against common WSH-based viruses and worms such as ILOVEYOU, the 3.10 version we tested did not provide WSH "shell execute" protection. Because of this weakness, web pages designed to exploit the WSH shell can write to the Windows Registry, potentially leading to a variety of annoyances. Version 3.10 of F-PROT Antivirus for Windows was released on July 20, 2001. Check out the trial version at http://www.complex.is/f-prot/

Conclusion
We consider each of the programs suitable for certain types of users ... Nahh, face it. A false sense of security sucks. Only one of these programs actually detected the viruses stored in our "Saved Email" databases. Kaspersky Anti-Virus Personal Pro (US$99) delivers the maximum email protection and a superior set of options -- missing only the detection of "virus like behavior" in our test #2, noted in the "how we tested" section of this article. Less expensive AVP options include a Personal edition for US$49 and a "Lite" Workstation edition for US$19. All are available as a free trial. Download a 30-day trial version of AVP at www.kasperskylabs.com.

We also liked F-PROT, although it, too, fails to detect the virus-like behavior of the WSH Shell Object exploit. We remain convinced that protection against WSH Shell Object exploits is important, so this is a concern here, and a vote in favour of Norton Antivirus... the only product tested that protected against these infiltrations.

The one we don't like any more is McAfee VirusScan. It's no longer on our list of trusted products.

Other Virus Tools
There are, of course, many other anti-virus tools on the market. For a list of the top 10 (which, interestingly, rates both McAfee VirusScan and Norton AntiVirus ahead of Kaspersky's offerings), visit http://cws.internet.com/virus.html. Unfortunately, the development of one of the leading free anti-virus tools, Computer Associates' InoculateIT Personal Edition, was discontinued June 11, 2001.

Latest Viruses
See our Virus Alerts page for the latest virus reports, or visit a site such as www.sarc.com for an up-to-date list of new viruses. Notable problems include the unusually destructive Win32-CIH virus that attacks the Flash BIOS on machines running Windows 95 and 98. (This virus activates on the 26th of each month.)

Hoax Alerts
Also problematic are so-called "hoax" viruses. We've received many pieces of e-mail recently from people warning us about some alleged virus. Please: before sending everyone you know a warning, check a site such as http://www.symantec.com/avcenter/hoax.html to see if it is a hoax. One classic example is called "JOIN THE CREW." This has to be the longest living hoax on the net. Please ensure that a virus is real, before forwarding e-mail to your friends or co-workers warning them about it. Thanks.

Macintosh
All the major anti-virus tools have been updated for compatibility with MacOS 9. At the moment, Symantec Antivirus for Mac and Network Associates' Virex are considered the leading Mac antivirus solutions. Symantec also offers anti-virus protection for Mac OS X.

Unfortunately, one of the best-known freeware anti-virus tools, John Norstad's Disinfectant, was discontinued in May 1998. (Version 3.7.1 is the last version.). Norstad cites the burgeoning amount of cross-platform MS Word macro viruses as the reason for discontinuing Disinfectant. Without the resources to keep up with the 1000+ variations of the macro virus, Norstad says the program only creates a false sense of security. He recommends Mac users purchase a commercial anti-virus utility and keep it up to date.

See our previous Virus Tools report for further info on Mac solutions.

Testing your Antivirus Software
How do you know if your anti-virus software is even working unless you endanger your PC to a virus "test attack"?

The following info, from McAfee, can help.

The Eicar Standard AntiVirus Test File is a combined effort by anti-virus vendors throughout the world to come up with one standard by which customers can verify their anti-virus installations. To test your installation, copy the following line into its own file and name it EICAR.COM.

X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*

When done, you will have a 69- or 70-byte file.

When WebScan is applied to this file, Scan will report finding the EICAR-STANDARD-AV-TEST-FILE virus.

It is important to know that THIS IS NOT A VIRUS. However, users often have the need to test that their installations
function correctly. The anti-virus industry, through the European Institute for Computer Antivirus Research, has adopted this standard to facilitate this need.

Please delete the file when installation testing is completed so unsuspecting users are not unnecessarily alarmed.

For more info:

Virex (Mac)