How Riya Recognition Works

Viswadep wrote here about what might happen if the technology behind Riya was used for evil. Let me address how the Riya recognition system works and how its limitations make it very unlikely that we could use this for security or "evil" applications. (I feel the need to say "one millioooon doooolars" every time I say evil...;-)

The Riya system only looks for people you train it to recognize (with one exception I'll describe later). So if you can't identify a person yourself then it won't recognize that person. Furthermore, the Riya system only looks for the people who you have trained in your account. So even if another user has trained us on Brad Pitt and he is in your photos, he won't get recognized in your photos.

Some people worry that the Riya system will identify them in the background of someones photo and violate their privacy. This just can't happen. Even if you use Riya and trained us on what you look like we wouldn't even look for you in another's album. Now some of you might say, well maybe Riya will change this policy in the future and then my privacy will get violated. It turns out that our algorithms work only because we are typically looking for 100-ish unique people in your albums. If we tried to look for 1 million unique people in your photos our accuracy rate is almost zero (more wrong than right). In fact most of the recognition you see on TV is pure fiction.

However there is one way you can have me recognized in your photos without doing any work. If some else has trained Riya on what Munjal Shah looks like and then added my email address, then if you import your addressbook or add my email address to your address book then you will get to borrow my signature (later we will require my consent first - future relase) and I will be recognized automatically. Again this is subject to the 100-300 unique person limit.

The question I always get after I explain this is:" So does this mean I can't find all of the photos of me?" No it doesn't. If your ten friends use Riya to auto-tag their photos then you can search for your name and find those photos (and see them if they are public). In this case Riya has already locally auto-tagged each photo and is only doing a tag/text search at a global level.

Comments

[Startups.in]

What are all the image formats currently supported by Riya?

Startups.in

[Till]

mmm, people overestimate the state of the art of afce recognition. I really wanna see if it works at all ;) working in the vision community, I know how hard it is to do face detection, even harder is recognition (and thats the tagging part essentially). In particular faces in different scales, from different viewpoints, bad lighting conditions ... To my knowledge, this is all a quite unsolved problem, the best detetction maybe from viola,jones ... I am really anxious to try it out - who can I get to the alpha2 test?

[Woodrow]

Thanks for the clarification Munjal.

This begs a follow-up though, scalability. One of the things I have been most intrigued about since following your launch has been the potential of someday being able to broadly find pictures of anyone I knew personally if they've been digitized on the broader web. It would seem you're suggesting that the facial recognition algorithms simply don't facilitate that.

Am I incorrect?

[Munjal Shah]

Jason, it could facilitate what you are asking in one of two ways

a) If people run Riya locally then the tagged DB gets aggregated and can be searched globally

b) We will shortly have similarity search that will likely find many of the photos of you without needed anyone to run recognition locally - hence it will work globally.

[Munjal Shah]

Currently Riya only supports Jpegs ... however this limitation will be removed by the beta.

[Liski]

Get me in!

[Peter Rothman, Chief Scientist]

I'm sorry, but as someone that works in the field (biometric and other applications in fraud prevention) I have to say that it is simply not true that the current state of the art in face recognition limits you to considering only gallery sizes of 100 or fewer people. Let me suggest that you begin to evaluate the accuracy of this claim by examining http://www.frvt.org/ and the 2002 test where substantially larger databases than this were used by multiple vendors. (http://www.frvt.org/FRVT2002/default.htm)

These tests are almost four years old now and there has been continued development and progress in performance across the industry. There are several existing products and known technical approaches that can deal with much larger databases of moderately unconstrained images than the 100-300 that you claim Riya handles. It all depends on how you approach the problem, and what sort of accuracy and/or error rates you can accept in your application.

However, once a database of images exists, the ability of someone to "use it for evil" depends more on the security and integrity of the database than the capabilities of the built in recognition software. If someone that wants to be "evil" can get into the data, then they can copy the database, run their own software on it, or even alter the contents to incriminate someone or cover up a crime. And public image databases (i.e. Flickr) are already searchable by anyone that wants to search them with their own software. Similarly, any picture that can be viewed from Riya can also be copied and searched with different software later.

Using weak face recognition software doesn't improve the security of the system in any of these regards and won't prevent a large public image database like this from being used for purposes its creators might not intend or which we might find disturbing.

I don't mean to be rude, because I do think this idea is quite interesting. I've delved into some similar ideas as well. But I don't find your argument about "evil" uses very convincing frankly.

[Burak Gokturk]

Thanks for your input Peter. I would like to comment on the 100-300 number that Munjal has mentioned on his post. Our application has different constraints then a security (airport) application.

In a security application, one can not let a terrorist go, i.e. the recall rate needs to be high, although the false hit rate is not as important. As long as the system can show the face of the terrorist within your 10-50 hits, it is satisfactory. Nobody is going to complain that the system has shown many false hits.

Our application has constraints on the amount of false hits. In particular, giving lots of false hits might create a burden on the user since he/she needs to fix them. As an objective, we try to make our first guess correct, and assign a name only when we are confident enough. Faces look statistically similar, and as such, as you add more people to your training cases, the likelihood of your first hit being correct drops. The numbers of 100-300 that Munjal has posted comes mostly through these observations.

As a side note, the conditions under which the consumer photographs are taken is very different from the tests that you mention. The dimensionality of variations in lighting, pose and expressions are much higher than typical test sets reported in technical papers, and hence a quite difficult problem... Oh well, here comes the real world! You are correct that if you constrain these conditions a bit, one would easily get to 10 to 100x of the 100-300 number.

[Startups.in]

Thanks for the clarification.

Startups.in

[Peter Rothman]

Burak,

Thanks so much for your response and clarification. I really appreciate that you took the time to respond.

Unfortunately, I don't agree with your assessment of the state of the art or the difficult issues confronting security oriented applications. In the anti-terrorist example, high false alarm rates are not acceptable as you suggest becasue each of these alarms requires a human response or intervention. Imagine trying to deal with such a high false alarm rate during the current holiday travel season. It would simply be unacceptable. I think you'd be surprised how varied the imaging geometry and behavior is in this scenario as well.

Also, there are other security applications where high false alarm rates are totally unacceptable. For example, consider secure unattended entry to a classified facility. In this case there might be a few people allowed to enter, perhaps only a handful of people, but no false entries can be tolerated. In this case you have a cooperative subject, but very rigid reliability requirements.

I agree that you are dealing with relatively unconstrained images, which is a much more challenging scenario than say the case with processing passport or drivers license photos. Hey you guys signed up to try this! However, I note that most family photographs have the people facing towards the camera and the photographer. From my experience it is possible to deal with pose variations, lighting variations, and more with some degree of reliability. It depends on your application and what error rates you can accept. Moreover, there are some less well known approaches that can be used if you know more about the entire image content. Out of curiosity, have you run Riya on the FERET database? What results do you see there?

Finally, back onto the topic of evil uses, I still don't see how having a classifier that can only recognizes 100-300 people limits or prevents any of the potentially evil uses of your system which some people have been concerned with. If you have 1,000 classifiers each of which can correctly identify 100 people with high confidence and very low false alarm rate, why can't you simply run these classifiers in sequence over the entire image database and find 100,000 different people? Are you saying that the classifer doesn't work well on images outside of its training set? If it does work well outside of its training set, then I don't see any limitations on using sets of these classifiers to create a surveillance application. Now, whether that would in fact be evil would depend on how it was used I suppose, but I think its clearly technically possible. Am I missing something?

[Pankaj Bhagat]

Hi Guys:
Just stumbled upon Riya and Wow!! this looks like a wonderful idea and an amazing team, wish i was part of it. Good luck to you all and wish you great success.
Cheers

[david sutherland]

I'd rather get false hits and a system that can handle around 1,200 people. My plaxo acct has about 1,200 currently.