How facial recognition technology is creeping into daily life

The Calgary Police Servicebecame the firstforcein Canadato start using facial recognition software to matchsuspects against a mug shot database this week, but it likely won't be the last.

The use of facial recognition technology is growing not just in law enforcement and security fields but also in commerce.

• Calgary police 1st to adopt face recognition software

"One of the reasons face [recognition] is so popular is that face images exist of almost everybody," said KevinBowyer, an expert on biometrics and computer vision andchairof the department of computer science and engineeringat the University ofNotreDame.

"You've got yourdriver's licence photos, you've got your identity badges wherever you work, so youve got this legacy of images that are easily accessible for everyone."

Chances are you've already encountered some form of this technology. Government agencies that issue driver's licencesuse it to verify that you are who your licence says you are. Banks use it when investigating debit card fraud (and lent it to police trying to identify rioters during the 2010 G20 summit).

Smartphoneapps likeFaceCryptandFastAccessAnywhereuse it to grant you access to your mobile devices. Social media sites likeInstagramand Facebook employ it when tagging photos. Google uses it in search and tagging functions and police in Dubai have even incorporated it into the Google Glasseyewearofficers use.

Matching faces not so simple

The algorithms used to match images of faces vary and are largely proprietary but generally employcomputational methods to analyze the pixel values in images and identify patterns and correspondences.

A lot of progress has been made in facial detection and matching in the last decade thanksto the millions of dollars pumped into the field by the U.S. government primarily the Defence Department said JamesWayman, a facial recognition expert who helped allocate that funding and is a research administrator at San Jose University

But facial recognition software can still be stymied by many factors: aperson's pose,lighting, facial expressions,aging, image resolution and obstructions like hats or even hair.

"Identification is a very messy process. It's as messy for computers as it is for humans," said Kelly Gates, author ofOurBiometricFuture: Facial Recognition Technology and the Culture of Surveillanceand an associate professor of communication and science studies at University of California, San Diego.

"People look like each other,people lookdifferent over time people can look very different depending on the lighting conditions, depending on the day.

You can never establish certainties; you can only establish probabilities of matches.- Kelly Gates, author of Our Biometric Future

"You can never establish certainties; you can only establish probabilities of matches."

To make a good match, you need images where people are looking straight into the camera and have similarfacial expressions ideally,a smileand not the neutral expression we've all been told to adopt onIDs and passport photosin thepost-9/11 era.

"That rule is born out of this idea that you don't want to match across expressions,"Bowyersaid. "It turns out itwould be better if everybody smiledbecause your smile is more distinctive than your neutral [expression]."

CCTV images often not useful

Facial recognition is only as strong as the algorithms and image banks driving it. When the U.S. National Institute of Standards and Technologytestedthe six leading suppliersof facial recognition software in 2013, it foundthe best-performing among them (NEC, the provider Calgary police are using)failed to recognize the most likely match inadatabase of 1.6 million mug shots about four per cent of the time. Theworst-performing software missed it about 20 per cent of the time. For webcam images, the failure rates wereroughly11 per centand 67 per cent, respectively.

The biggest obstacle to accurately matching faces isimage quality. Many of the agencies that use facial recognitionrely on CCTV and security cameras, but these produce images thataregrainy, low-resolution and taken from above.

"The worst-possible direction to put a camera to try to recognize somebodysface is up," saidWayman.

He says he doubts thatcasinos, for example, ever really use thefacial recognition software that is supposed to help them keep out problem gamblers and spot card cheats andVIP customers because their cameras are generally onthe ceiling.

The poor quality of CCTVimages also hindered police efforts to identify the two suspected perpetrators after the 2013 Boston Marathon bombings, who were caught on blurry surveillance footage their faces partly obscured by hats taken shortly before two bombs went off. The FBI's facial recognition systemfailed to find a match for either Tamerlanor DzhokharTsarnaeveven though both were in the driver's licence database and Tamerlanwas in agovernment database of known and suspected terrorists.

FBI adopts face recognition

Facebook, on the other hand, is full of high-resolution, front-facing pictures of faces, which is why its more than 250 billion uploaded photos are a veritable gold minefor lawenforcement.

The FBI has said it won't store social network photos in the database of 52 million photos that will be part of its new face recognition system, but groups like the Electronic Frontier Foundationhave raised concernsthat "there are no legal or even written FBI policy restrictions in place to prevent this from occurring."

The FBI has said that by 2015, its database will include at least 4.3 million "civil images" those taken for non-criminal purposes.

"This means you could become a suspect in a criminal case merely because you applied for a job that required you to submit a photo with your background check," the EFF warned in an analysis of the program.

The FBI's Next Generation Identification system will also include fingerprints, palm prints, iris scans andinformation such as ethnicity and immigration status and be shared across agencies and police departments.

Automated passport checks

Facial recognition technology is also becoming a familiar site at airports. Australia, NewZealand, the U.K. and Germanyare among the countries that use automated airport customs gates outfitted withcameras that snap your picture, which gets matched against your passport and potentially a watch list.

Canadian border authorities have so far limited their use ofbiometrictools to iris scans, which are usedto verify theidentities of thosewho useCanPassor Nexus IDs to travel between the U.S. and Canada. New passports are equipped with a digital facial image that can be used in face recognition systems, and Passport Canada does use the technology to check applicants' photos against its database.

But whilefacial recognition might be able to detect passport fraud, it likely won't help authorities pick a knownterrorsuspect out of a crowd at a busy airport.

"How often do terrorists whose picture you have and are looking for really walk through airports? Not very often. A thousand times an hour some poor chump like me walks through the airport so youget thousands upon thousands of false positives," saidWayman, whoadvises Australian customs authorities on theirSmartGatefacialrecognition system.

Experiments in using facial recognition software to identify individuals walking through airports in the U.S. and Japan have failed in recent years because the technology is generally better at the one-to-one type comparisons involved in checking passports or licences than at the one-to-many searches required to match a face in a crowd to one in a database.

Selling you stuff to your face

Security-related usesof facial recognition are to be expected, saysBowyer, but it's the commercial applications that have really picked up in recent years.

Retailers like Reebok andTescohave used cheap webcam-based facial detection softwareto monitor how customers react to store displays or to show them age- or gender-specific ads in real time. (Unlike face recognition, face detection doesn't try to make a match but estimates a person's gender, age and facial expression based on what their face looks like and what thesoftware already knows about faces of a certain age, sex or mood.)

• Face recognition use by Calgary police target of privacy probe

Smart TVs enablecable, video game and marketing companies to gauge audience reaction using face detection while online dating site Match.comwillfind you a matewho looks like your ex with the help offacial recognition.

There's even asmartphoneapp calledSceneTapthat uses cameras and facial detection to tell you if a club or bar is busy and what the average age andgender ratios of the patrons are.

Such applications come at price, says Gates: autonomy.

"Facial recognition plugs into a larger set of practices and problems around predictive analytics and the ways in which all of our online and offline experiences are constantly being modulated using data science and data analytics," she said.

The increasing pervasiveness of facial recognition technologies also raises concerns of privacy.

The Alberta privacy commissioner has already announced she will review the Calgary police's use of face recognition technology. But the fact is, there is little preventing police and others from making use of images taken in public spaces, where there is nopresumption of privacy, or uploaded to networks like Google+ and Facebook, whose privacy policies deem some images to be publicly available.