AI could predict your next move from watching your eye gaze

  • Written by Eduardo Velloso, Lecturer in Human-Computer Interaction, ARC DECRA Fellow, University of Melbourne

Our eyes often betray our intentions. Think of poker players hiding their “tells” behind sunglasses or goalkeepers monitoring the gaze of the striker to predict where they’ll shoot.

In sports, board games, and card games, players can see each other, which creates an additional layer of social gameplay based on gaze, body language and other nonverbal signals.

Digital games completely lack these signals. Even when we play against others, there are few means of conveying implicit information without words.

Read more: What eye tracking tells us about the way we watch films

However, the recent increase in the availability of commercial eye trackers may change this. Eye trackers use an infra-red camera and infra-red LEDs to estimate where the user is looking on the screen. Nowadays, it is possible to buy accurate and robust eye trackers for as little as A$125.

Eye tracking for gaming

Eye trackers are also sold built into laptops and VR headsets, opening up many opportunities for incorporating eye tracking into video games. In a recent review article, we offered a catalogue of the wide range of game mechanics made possible by eye tracking.

This paved the way for us to investigate how social signals emitted by our eyes can be incorporated into games against other players and artificial intelligence.

To explore this, we used the digital version of the board game Ticket to Ride. In the game, players must build tracks between specific cities on the board. However, because opponents might block your way, you must do your best to keep your intentions hidden.

Our studies using Ticket to Ride to explore the roles of social gaze in online gameplay.

In a tabletop setting, if you are not careful, your opponent might figure out your plan based on how you look at the board. For example, imagine that your goal is to build a route between Santa Fe and Seattle. Our natural tendency is to look back and forth between those cities, considering alternative routes and the resources that you have in the cards in your hands.

Read more: A sixth sense? How we can tell that eyes are watching us

In our recent paper, we found that when humans can see where their opponents are looking, they can infer some of their goals – but only if that opponent does not know that their eyes are being monitored. Otherwise, they start employing different strategies to try to deceive their opponent, including looking at a decoy route or looking all over the board.

Can AI use this information?

We wanted to see if a game AI could use this information to better predict the future moves of other players, building upon previous models of intention recognition in AI.

Most game AIs use the player’s actions to predict what they may do next. For example, in the figure below on the left, imagine a player is claiming routes to go from Sante Fe to some unknown destination on the map. The AI’s task is to determine which city is the destination.

When at Santa Fe, all of the possible destinations are equally likely. After getting to Denver, it becomes less likely that they want to go to Oklahoma City, because they could have taken a much more direct route. If they then travel from Denver to Helena, then Salt Lake city becomes much less likely, and Oklahoma City even less.

AI could predict your next move from watching your eye gaze Left: without gaze information, it is difficult to tell where your opponent is going next. Right: by determining that your opponent keeps looking at Helena and Seattle, the AI can make better predictions of the routes the opponent might take.

In our model, we augmented this basic model to also consider where this player is looking.

The idea is simple: if the player is looking at a certain route, the more likely the player will try to claim that route. As an example, consider the right side of the figure. After going to Denver, our eye-tracking system knows that the player has been looking at the route between Seattle and Helena, while ignoring other parts of the map. This tells us that it is more likely that they take this route and end up in Seattle.

Our AI increases the relative likelihood of this action, while decreasing others. As such, its prediction is that the next move will be to Helena, rather than to Salt Lake City. You can read more about the specifics in our paper.

Experimentation

We evaluated how well our AI could predict the next move in 20 Ticket To Ride two-player games. We measured the accuracy of our predictions and how early in the game they could be made.

Read more: Eye tracking is the next frontier of human-computer interaction

The results show that the basic model of intention recognition correctly predicted the next move 23% of the time. However, when we added gaze to the mix, the accuracy more than doubled, increasing to 55%.

Further, the gaze model was able to predict the correct destination city earlier than the basic model, with the AI that used gaze recognising intentions a minute and a half earlier than the one without gaze. These results demonstrate that using gaze can be used to predict action much better and faster than just using past actions alone.

Recent unpublished results show that the gaze model also works if the person being observed knows that they are being observed. We have found that the deception strategies that players employ to make it more difficult for other players to determine their intentions do not fool AIs as well as they fool humans.

Where to next?

This idea can be applied in contexts other than games. For example, collaborative assembly between robots and humans in a factory.

In these scenarios, a person’s gaze will naturally lead to earlier and more accurate prediction by the robot, potentially increasing safety and leading to better coordination.

Authors: Eduardo Velloso, Lecturer in Human-Computer Interaction, ARC DECRA Fellow, University of Melbourne

Read more http://theconversation.com/looks-arent-so-deceiving-ai-could-predict-your-next-move-from-watching-your-eye-gaze-97927

Australia Today

could someone take ownership of a planet or a moon?

Melissa de Zwart, Professor, Adelaide Law School, University of Adelaide

This is an article from I’ve Always Wondered, a series where readers send in questions they’d like an expert to answer. Send your question to alwayswondered@theconversation.edu.auI have always wondered, in a distant future when humanity is able to settle permanently on a distant body (su...

Read more

Fewer Australian university students than ever before believe in creationism

Mike Archer, Professor, Pangea Research Centre, UNSW

The dominant belief among Australian university students now is that God had no hand in creating or developing humans.www.shutterstock.comAustralian university students appear to give far more credit to the science of human evolution and far less to creationism or divine guidance than the previous g...

Read more

Are Australian bridges safe, and can we do better?

Colin Caprani, Senior Lecturer, Structural Engineering, Monash University

There are more than 50,000 public bridges in Australia. (The Sea Cliff bridge is one Australia's newer bridges, built in 2005).from shutterstock.comAfter the tragedy in the Italian city Genoa, where a highway bridge collapsed killing more than 40 people, nations seem to be taking stock of the mainte...

Read more

Research shows 'merit' is highly subjective and changes with our values

Sue Williamson, Lecturer, Human Resource Management, UNSW Canberra, UNSW

Selection panels interrupt women more than men and ask them more follow-up questions, subtly questioning their competence.Andrey Popov/ShutterstockWho is meritorious, what constitutes merit, and how merit and gender targets can operate together are widely misunderstood questions, as our new research...

Read more

Fierravanti-Wells resigns from ministry, accusing Turnbull of ignoring Liberal party's conservative base

Michelle Grattan, Professorial Fellow, University of Canberra

The Minister for International Development and the Pacific, senator Concetta Fierravanti-Wells, has quit the frontbench, delivering a swingeing attack on Malcolm Turnbull for ignoring the Liberals’ conservative base.In her resignation letter to the Prime Minister, Fierravanti-Wells reminded hi...

Read more

Labor now does politics better than the Liberals

Geoffrey Robinson, Senior Lecturer, School of Humanities and Social Sciences, Deakin University

Happy days: Opposition Leader Bill Shorten in parliament.AAP/Mick TsikasThe obverse of the current crisis in Australian conservative politics is the resilience of the Australian left.The Labor Party has twice climbed off the canvas after dramatic political setbacks: the Rudd-Gillard civil war and th...

Read more

Holidays & Travel

Guide to Travel Necessities for 2018

Sure, it sounds like quite an adventure to hit an open road with nothing but a one-way ticket and an open mind, but the harsh reality is – you won’t...

Top Sightseeing Spots in New Zealand

New Zealand is one of the most fascinating locations for anyone to visit in the world. After all, it’s a place with a variety of unique landscapes, but it isn’t...

Must-Pack Items for a Travel Blogging Boss

As a travel blogger, you get to see and explore so many wonderful destinations and meet various interesting people, but before you embark on a new adventure you need to...

Luxury Camping Adventures Around the World

Perhaps in direct response to the dominance of technology in the modern world, many travelers are feeling a strong draw towards nature. While some are able to fully embrace their...

Dredging works underway Plantation Island

Tourists and workers travelling to popular tourist destination Plantation Island, Fiji, are set to benefit from safer access when marine channel dredging is completed at the end of June 2018...

Top 5 Tips for Traveling With Dogs

Travelling with your dog can be thrilling, but sometimes it can turn into a real nuisance. Unless you know how to properly restrain the pooch, whether it’ll be able to...

Top Island Destinations for 2018

When we picture island destinations in our minds, we often imagine a tropical beach of white sand, a sun lounger laid out in the shade of a line of palm...

Why Car Rental Excess Insurance is a Smart Choice

Some car rental agencies include basic damage cover in the standard rental fee, with the option to add on liability reduction, or excess insurance. The addition of excess insurance normally...

More Than a Third of Australians Feel 'Too Busy' With Work to Take a Holiday

New research from InterContinental Hotel & Resorts has unveiled a large number of Australians (32 percent) are simply too busy with work to take some much-needed rest and relaxation. Almost...

Fashion & Beauty

Groom To Do List

Sorry Aussie Men, you no longer just pay for the alcohol, flowers and marriage license. Times have certainly changed. Wedding Expert shares what Australian Grooms are now expected to do before...

Simple Secrets To A More Confident Smile

We all know just how important first impressions are. After a handshake, what’s one of the first things people will notice about you? It’s your smile. A friendly smile is...

Improve Your Body Confidence and Feel Great

No matter how good you feel about yourself, sometimes you just can’t help but see a few little flaws that seem gigantic to you and make you feel self-conscious. Whether...

First-Timer Tips For Your First Lip Filler Appointment

Getting injectable lip fillers can be a quick and effective way to plump up your smile with minimal drama. They’re a cost-effective and safe way to boost the natural shape...

The Most Common Skin Conditions and How to Deal with Them

We often underestimate the importance of our skin, but as the largest organ in our body, and our primary layer of defense, it is constantly affected by many environmental factors...

Beach Babe Basics: Everything You Need to Look Stylish at the Beach

Summer is right around the corner, and this year, we all should make the best of it. Right now, it is not just about looking stylish on the streets, it...

HEALTH

National Action Plan For Endometriosis Is A Game-Changer For Australian Women

Heba Shaheed, The Pelvic Expert, shares her personal story and expert tips for living with endo... First National Action Plan For Endometriosis Is A Game-Changer For Australian Women   The announcement of the first...

Pre - summer pampering: Australian women are detoxifying themselves in the most “kind-Ly” fashion!

In today’s society we are completely exposed to materials, products and chemicals which we unfortunately have no clue if it’s origins, meaning and purpose. Getting back to basics...

Eye Health Dos and Don'ts

Most people take their eyes for granted and spend little to no time thinking about their eye health until they notice something wrong. This leads to plenty of common and...

Non-invasive Cosmetic Treatments

In a world where everyone is chasing perfection, plastic surgeries have become more and more common to the point where it’s odd if you don’t get one during your life...

My Health Yoga

Carrie-Anne Fields of My Health Yoga has taken her Gold Coast yoga teacher training worldwide. She started her health business 20 years ago aged 22 and has grown into a...

Fit Body 101 - Increase Your Fitness Effectiveness

You have to find some space for physical exercise in your weekly schedule in order to live a good life. Embracing this way of life is very beneficial because it...

LIFE STYLE

Add a little colour to your move - and life!

How a little ‘colour’ and planning can take your Move, and make it a more enjoyable one Colour plays a huge role in our lives, it can take us up, or...

6 Simple Rules for Redesigning Your Kitchen

How do you feel about your kitchen? Well, if you feel like your home needs a bit of lifting up, one of the best and most effective places to start...

Dog Flea Allergy

Severe itchy skin disease in dogs causes Flea bite hypersensitivity. The itch is as a result of an allergic reaction to the saliva of the flea which is injected into...

5 Ways to Turn Your Bedroom into a True Zen Retreat

The bedroom should be our little sanctuary where we come to relax after a long day of work. However, in order to be able to unwind all the accumulated stress...

Things to Consider when Deciding on your Wedding Flowers for the Big Day

Choosing the wedding flowers is a complicated process where you have to take into account many elements, from the wedding dress to the wedding theme. You can discover in this...

Renovating tips for your house

There are very few property owners who move into a house where they don’t want to make any changes. Alterations are almost inevitable every time you move-house, but alterations can...

FOOD & DINING

Morning greens: the breakfast skincare regime

It’s safe to assume that most Aussies fail to eat enough greens. Although this was momentarily resolved with the rise and popularity of the green juice movement, the lack of...

Best Hidden Gem Hong Kong Restaurants

You finally find yourself enjoying the endless beauties of China and this fascinating city. In a place with so much variety, all your senses will be wide awake, and your...

JOHNNY DI FRANCESCO NAMED OFFICIAL AMBASSADOR OF EAT STREET MELBOURNE

Johnny Di Francesco, world champion pizzaiolo and owner of Melbourne’s Gradi Group of restaurants, is excited to announce he has been named official ambassador for Eat Street Melbourne 2018. As Ambassador...

The best Mother's Day gift EVER!

Whether it’s to celebrate a special occasion, or just wind down at the end of a long day, there’s nothing much better than pouring yourself a glass (or two…) of...

Sing for Your Supper at Chargrill Charlie’s

To celebrate the final curtain call of the Barden Bellas, Chargrill Charlie’s is giving you the chance to Sing for your Supper! Show off your best acapella vocals to win...

MAKE IT WINE TIME, ANY TIME

Whether it’s to celebrate a special occasion, or just wind down at the end of a long day, there’s nothing much better than pouring yourself a glass (or two…) of...