"Viisage at 64 per cent, Imagis at 59 per cent, VisionSphere at 53 per cent, and Dream Mirh bringing up the rear at 34 per cent."
Stockwatch News Item
Imagis Technologies Inc (C-NAB) - Street Wire
Imagis an also-ran in face recognition testing
Imagis Technologies Inc NAB
Shares issued 20,383,371 Mar 14 2003 close $ 0.70
Friday March 14 2003 Street Wire
Also Imagis Technologies Inc (U-IGSTF) Street Wire
FACING REALITY
by Lee M. Webb
Imagis Technologies Inc., a heavily promoted Vancouver-based biometric technology company, finished among the group of distant also-rans in an independently administered face recognition technology evaluation conducted by U.S. government agencies last year. Results from the tests conducted in July and August of last year were finally released on March 13. Imagis's much-hyped face recognition technology did not fare well.
Under the mandate of the USA Patriot Act, the National Institute of Standards and Technology (NIST) was charged with measuring the accuracy of biometric technologies. In co-operation with other U.S. government agencies, NIST conducted the Face Recognition Vendor Test (FRVT) 2002 at the U.S. Naval base in Dahlgren, Va.
FRVT 2002 was open to all developers and providers of core face recognition technology, but in the end only 10 companies participated in the test. The 10 participants included AcSys Biometrics Corp., a Canadian venture 50-per-cent owned by Nexus Group International Inc., a penny dreadful listed on the Toronto Stock Exchange; Cognitec Systems GmbH; C-VIS Computer Vision und Automation GmbH; Dream Mirh Co. Ltd.; Eyematic Interfaces Inc.; Iconquest; Identix Inc., a Nasdaq-listed company; Imagis, which trades on the TSX Venture Exchange; Viisage Technology Inc., another Nasdaq-listed company; and VisionSphere Technologies Inc., based in Ottawa, Ont.
FRVT 2002 consisted of two sub-tests: the high computational intensity test (HCInt); and the medium computational intensity (MCInt) test. According to the test literature, the HCInt was designed to evaluate the performance of state-of-the-art systems on extremely challenging real-world problems. The MCInt was designed to offer an understanding of a participant's capability to perform face recognition tasks with different formats of imagery, still and video, under varying conditions and to help identify promising new technologies not identified in HCInt.
The HCInt, consisting of 121,589 operational images of 37,437 people, was at the heart of FRVT 2002. The images were provided by the U.S. Department of State's Mexican non-immigrant visa archive. According to the report, the images are of good quality and were gathered in a consistent manner; moreover the background is universally uniform. Participants could take the HCInt, MCInt, or both tests. AcSys and Iconquest opted to take only the MCInt, but the rest of the participants took both tests.
The milestone FRVT 2002 examined the performance of the participants in the three primary face recognition tasks that turn on three separate questions: verification, "Am I who I say I am?"; identification, "Who am I?"; and watch list, "Are you looking for me?"
In verification, a person presents their biometric and an identity claim to the face recognition system which then compares the presented biometric with a stored biometric and either accepts or rejects the claim. Performance in the verification task is measured in terms of the rate at which legitimate users are granted access, the verification rate, and the rate at which imposters are granted access, the false accept rate.
In the identification task, an image of an unknown person is provided to the system; that image is then compared to the database of known people. The results of that comparison are presented to an operator in a ranked listing of top candidates, typically anywhere from one to 50 candidates.
The watch list task is considered more difficult than the other two tasks. The face recognition system must first determine whether an individual is on the watch list and then correctly identify the person. Performance is measured by the detection and identification rate and the false alarm rate.
The FRVT 2002 report adds considerably to the literature regarding face recognition technology and makes some significant progress toward answering some open questions. For example, because of the large number of people and images in the data set, FRVT 2002 was able to report the first large-scale results with respect to the question of how database and watch list size affect performance. The tests showed that for every doubling of database size, identification performance decreases by two per cent to three per cent. A similar effect was observed for the watch list task; as the size of the watch list increases, performance decreases.
The complete FRVT 2002 report is presented in three volumes: a 14-page overview and summary; a 56-page evaluation report; and 264 pages of technical appendices. Much of the report makes for some heavy reading, but the overview and summary actually cuts to the chase and offers a clear and understandable synopsis of the results. Those results effectively separate the participating face recognition companies into two groups: the best and the rest.
Cognitec, Identix and Eyematic clearly outperformed the other participants by wide margins in every test. In the verification test for the HCInt participants, Cognitec and Identix each produced a 90-per-cent verification rate at a one-per-cent false accept rate, while Eyematic's verification rate came in at 87 per cent. There was a large gap between the top three and the rest of the field, with C-VIS registering a 65-per-cent verification rate, Viisage at 64 per cent, Imagis at 59 per cent, VisionSphere at 53 per cent, and Dream Mirh bringing up the rear at 34 per cent.
The overview and summary only provides the results for the top three systems for the identification and watch list tests, again dominated by Cognitec, Identix and Eyematic with rather closely grouped results. Results for the rest of the field have to be ferreted out of the 56-page evaluation report or, for those wanting even more detail, the technical appendices. As with the verification testing, the graphs for the identification and watch list tests make it very clear that there is a large gap between the results obtained by the top three companies and the rest of the participants, including Imagis.
Imagis's poor showing in FRVT 2002 is hard to square with all the promotional claims and hype served up about the company's face recognition technology, particularly since the terrorist attacks of Sept. 11, 2001. Like other public biometric companies, Imagis's stock price soared in the wake of the attacks on the World Trade Center and the Pentagon. Just prior to Sept. 11, 2001, Imagis was changing hands for less than 65 cents per share in desultory trading, but within a month the stock price had climbed above $2 per share and was heading higher.
Two months after the attacks, Imagis closed two special warrant financings priced at $2.17 per special warrant for total gross proceeds of $3.6-million. The general market interest in biometric stocks in the wake of the terrorist attacks and a vigorous promotional campaign kept the share price bubbling higher.
On Jan. 7, 2002, analyst Rob Klein of Thomson Kernaghan & Co. Ltd. weighed in with a boosterish research report and strong buy recommendation. Thomson Kernaghan, which had acted as an agent for Imagis's special warrant financing two months earlier, would collapse in a major Bay Street scandal within a few months, after booting its chairman, Mark Valentine. Mr. Valentine was subsequently arrested in the Bermuda Short sting operation. Meanwhile, Mr. Klein was solidly behind Imagis.
According to Mr. Klein, Imagis's face recognition technology was far better than the products offered by its two major competitors, Visionics Corp., which subsequently merged with Identix, and Viisage. "Imagis' technology is superior--it measures more facial data points than either Viisage or Visionics product offering (200 data points vs. 14 for Visionics)," Mr. Klein claimed. "It can better handle low light situations and can recognize a person's image at up to a 25 degree angle--most other systems can only handle a maximum 15 degree deviation. This results in higher recognition accuracy and lower false positives."
Mr. Klein offered his glowing assessment of Imagis and its face recognition product without access to any independent testing of the technology inasmuch as the company had not participated in any such testing prior to FRVT 2002. In that landmark testing, Identix, the successor to Visionics, mopped the floor with Imagis. Viisage, which finished well back in the pack along with Imagis, has since suggested that there is "troubling evidence" that there was something wrong with its test setup, "either due to human error or a problem with the software," accounting for its dismal performance.
The Thomson Kernaghan analyst was not alone in touting Imagis early last year; the frothy stock price also attracted some newsletter touts, including James Dale Davidson. In March of last year, the promotion got another timely boost just as the trading restrictions on millions of Imagis shares expired.
On March 6, 2002, the Pembridge Group, an obscure Boston-based alternative asset management firm hired two months earlier as a financial adviser to Imagis, issued a craftily worded announcement that was widely misinterpreted and misreported as a $90-million (U.S.) buyout offer for the company. Investors piled into the stock in frenzied trading in the wake of the announcement, pushing the share price to a 52-week high of $5.66. While the National Post and CBC Newsworld, among others, jumped on the buyout bandwagon, Stockwatch correctly and consistently reported that Pembridge had not made any offer for Imagis.
With the buyout brouhaha raging, Mr. Davidson ratcheted up his Imagis touting in his subscription newsletter, Vantage Point Investment Advisory, and in a free come-on tout sheet. Another stock tout, Kevin Barker, who offered his strong buy rating in his IT Investor, soon joined Mr. Davidson in the Imagis tout parade. Meanwhile, Imagis representatives were busy beating the promotional drum, too.
On March 13, 2002, Imagis's president and chief executive officer, Iain Drummond, repeatedly told an audience at the Sutton Place Hotel in Vancouver that Pembridge had made a buyout offer of $4.10 (U.S.) per share for Imagis. "The reason Pembridge did that is that they believe we are grossly undervalued," Mr. Drummond said. "Visionics (now Identix) is valued, some believe, at something like $300-million (U.S.) or $400-million (U.S.) and our technology is better and they believe that if Visionics is worth that, then we must be worth that as well."
Mr. Drummond provided his Sutton Place audience with a demonstration of Imagis's technology, a performance that he would repeat right down to a hackneyed joke about cheeseburgers on national television about three months later. In his demonstration, Mr. Drummond presented a photograph of an unidentified individual to the system, which searched the database and returned a ranked list of matches.
The first match was a photograph of a much chubbier individual than the image presented to the system, a testament, Mr. Drummond said, to five or six years of eating American cheeseburgers. In fact, the two images were of the same individual taken approximately six years apart. The individual was Hamza Alghamdi, one of the terrorists who slammed into the World Trade Center on United Airlines Flight 175. Perhaps not surprisingly, many of the members of the Sutton Place audience, and presumably the later national television audience, were very impressed with the demonstration.
Imagis's founder, Howe Street promoter Altaf Nazerali, was also doing some public promotion in early March of last year. On March 14, 2002, the bilingual Mr. Nazerali was featured in a French-language interview on Radio Canada. While Mr. Nazerali focused on a vague deal involving Imagis and Sanyo that had recently been announced, he also took the opportunity to tout the company's wonderful face recognition technology. According to Mr. Nazerali, beards, mustaches and similar disguises could not fool Imagis's face recognition package.
Imagis's stock price began to slide as it became clear to most investors that, contrary to the reports of the National Post and CBC Newsworld, Pembridge had not made any buyout offer for the company. However, the heavy promotion continued.
Earl van As, hired as Imagis's director of marketing on April 11, offered his anecdotal account of the company's amazing face recognition technology to a Vancouver Sun reporter covering a trade show at Whistler, B.C., at the end of April. "I scanned a picture of myself as a two-year-old into a database of 200 people," Mr. van As reportedly told the Vancouver Sun. "It picked me out in the top three."
Roger Henning, hired last March to fill the role of Imagis's Asia Pacific representative, has proven to be no slouch in the touting department, either. Mr. Henning, by his own account an adviser to various heads of state and "best known throughout the world as a Premier Crisis Management Specialist," has been unstinting in his promotional claims on behalf of Imagis.
One of Mr. Henning's more interesting touts was picked up by New Zealand's Computerworld, a specialized information systems newsweekly. On June 10, 2002, Computerworld columnist Darren Greenwood reported that Imagis had been touting its product's "ability to identify a face despite efforts at disguise or changes wrought by age, diet and cosmetic surgery."
Mr. Greenwood is apparently given to some tongue-in-cheek writing. "The ultimate testimony: Imagis Asia Pacific director Roger Henning says it picked 'the little fat kid' from that 70s singing group The Jackson Five as the same person as the emaciated, bleached Michael Jackson we know today," Mr. Greenwood wrote. "You can get a nose job, but you can't change the shape of your skull without radical surgery," Mr. Henning reportedly explained in touting the impressive feat.
While Imagis's promotion in North America has lost much of its steam, Mr. Henning was still doing his bit for the company in Australia as recently as last October. On Oct. 24, Mr. Henning reportedly told the Australian Broadcasting Corporation that Imagis's technology would have picked up suspected terrorists on international watch lists, such as alleged Sept. 11 terrorist Mohammed Atta.
To this point, Imagis has not offered any interpretation of its dismal FRVT 2002 performance in a news release or provided shareholders with any hints on how to reconcile that performance with the company's promotional claims.
FRVT 2002 offered the participants in the landmark test the opportunity to submit comments regarding the test and their individual results. Nine of the 10 participants, including Imagis, did submit comments, which are included in the technical appendices. Some participants, such as the test leaders Cognitec, Identix and Eyematic, submitted rather detailed comments, some running to six pages. Imagis's comments, not submitted on letterhead or even displaying the company's logo, covered less than one page.
Imagis reported that a new release of the company's face recognition software following the FRVT 2002 test offered some significant improvements. "This version not only achieved a substantial increase in accuracy by correcting certain limitations identified during the internal benchmark procedures that preceded the FRVT test itself, it also proved that significant accuracy advancements are achievable on an ongoing basis as a result of the inherently extensible nature of our core imaging algorithms," Imagis advised.
"While laboratory testing provides a good basis for discussion, nothing can substitute for a real-world test deployment using one's own data and imagery," Imagis wrote, somewhat lamely pitching a pilot deployment of its upgraded software. "In such an environment, users can measure the impact that speed and database size have on the operational processes of their own organization."
Imagis and other also-rans may take some small comfort from at least one passage in the FRVT 2002 report. "FRVT 2002 was not designed to be a 'buyer's guide for face recognition'--where one looks at graphs or scores and selects a system for installation," the report carefully notes. "Rather, it is a technology evaluation that should assist decision-makers in determining (1) if face recognition technology could potentially meet the performance requirements for an operational application, and (2) which systems should be selected for application-specific scenario evaluations."
Notwithstanding that careful, if ambivalent, note, the report authors expect that the FRVT 2002 results will provide input into several of the questions that might be asked by purchasers considering implementing face recognition technology. Quite apart from that expectation, it seems likely that the report will strongly influence the decision-making process of potentially major biometric purchasers operating under the auspices of agencies such as the newly-formed U.S. Department of Homeland Security. "FRVT 2002 is the most thorough and comprehensive evaluation of automatic face recognition technology to date," the report notes.
While some Imagis shareholders, at least those who are aware of the release of the landmark FRVT 2002 report, may still be mulling over the details, others may be preoccupied with facing another reality. Imagis's stock price has recently been registering new 52-week lows on both the TSX Venture Exchange and the OTC Bulletin Board. With only 9,500 shares changing hands in TSX Venture trading on March 14, Imagis closed out the week at 70 cents.
Comments regarding this article may be sent to lwebb@stockwatch.com.
(More information regarding Imagis Technologies is available in Stockwatch articles published on March 7, 11, 15, 25, 27 and 28; April 2, 9 and 16; May 17, 23 and 30; June 4, 11, 18, 26 and 28; July 3, 12 and 18; Sept. 12, 13, 16, 20, 23, 24 and 27; Oct. 2, 9, 11, 16, 21, 22, 25 and 29; Nov. 6, 8, 11, 14 and 18; Dec. 12 and 13, 2002; Feb. 20, 24 and 26; and March 3 and 6, 2003.)
(c) Copyright 2003 Canjex Publishing Ltd. |
