A sense of purpose

By Chan MY

SMU Office of Research & Tech Transfer – From the temperature around you to how many steps you’ve taken today, your smartphone or smartwatch probably has a sensor that keeps track of it.

These sensors are a treasure trove of information and can be used to help companies and government organisations find better ways of interacting with customers in public spaces – sending them alerts for promotions as they walk past a retail store, for example, or deploying more staff to where they are urgently needed in an airport.

The combination of mobile sensing and computing – relying on data from smartphones and other sensors to inform system designs and efficient algorithms – is at the centre of the work being done at the Singapore Management University’s (SMU) LiveLabs Urban Lifestyle Innovation Platform, or LiveLabs for short.

“I see LiveLabs as producing technology that will help us with important everyday tasks,” says SMU Associate Professor Rajesh Balan, one of the platform’s two directors. “We work on all facets of mobile sensing and analytics and try to generate solutions that will help individuals, venue owners, and society as a whole.”

From sensing to applications

To understand the behaviour of people in physical environments in real time, LiveLabs develops hardware and software that can gather information from sensors in smartphones as well as in the surrounding physical environment.

“We produce solutions that can capture these sensor data, and then process them in real time to generate deep insights about the current situation,” says Professor Balan. These solutions include algorithms and policies that draw upon location-based and contextual data.

“For example, we can identify groups among the visitors to a convention centre, even if there are hundreds of people visiting. This is very useful as groups behave differently from individuals and identifying them allows different types of focused interventions to be applied specifically to them,” he adds.

The advantage of these data capture methods is that they are device-agnostic and do not need any software downloads or technological tweaking. Users who carry Android, iOS, and Windows devices can thus adopt these technologies with minimal effort.

LiveLabs’ researchers are particularly focused on real-world applications of their technology, in key domain areas such as retail and lifestyle spaces, or even in homes.

“In most cases, our research is translated into solutions that are then tested with real users in real environments. In some cases, these solutions are then hardened by LiveLabs into industry-grade solutions that are used 24/7 by commercial companies,” says Professor Balan.

Coming to you live on location

LiveLab’s location-based solutions – consisting of sensors that track smart device movements using Wi-Fi, along with real-time analytics software – have been successfully implemented with the support of its industry partners, says Professor Archan Misra, the co-director of LiveLabs.

With partners – Suntec Singapore International Convention and Exhibition Centre, for example, location tracking and analytics has enabled exhibition organisers to better determine the relative popularity of different booths by tracking how much time exhibition goers spend at each booth.

“The location tracking also shows the booths that an individual visits, so it will also be possible to discover patterns of interest by looking at the thematic similarities among different booths,” explains Professor Misra. For exhibition organisers, these patterns will be important to keep in mind when designing and promoting future exhibitions.

Similar location-tracking and analytics solutions to study the behavioural patterns of visitors have also been deployed by LiveLabs researchers on SMU’s campus, at the National Museum of Singapore, and even to a major North American University, with plans to extend it to Singapore’s Republic Polytechnic in the coming months, Professor Balan says.

If the data collected seems to only help the organisations that roll them out, Professor Balan points out that the beneficiaries are equally likely to be the participants. One such success story, he shares, is LiveLabs’ conference app, which focuses on participant engagement to improve the interactive quality of a conference.

The app uses contextually triggered games and notifications with fun features such as points and badges to engage conference attendees. It also promotes social interaction with a smart ‘friend-finding’ function, and serves as a one-stop guide with information about upcoming programmes as well as activities to take part in during breaks.

“This app has been so well received and it was used in the recent Singapore Fintech Festival at Singapore Expo. The app was used by about 4,000 attendees,” enthuses Professor Balan, adding that the app has also been used by the Association of Computing Machinery, the top professional organisation for computer science researchers, for its flagship mobile computing research conferences.

It’s all in the wrist

Beyond real-time sensing at public spaces, Professor Misra’s team is developing wearable sensing solutions for applications in personal wellness. One of his current projects, an automated food journal system is a smartwatch-based application that can track the eating patterns of the wearer.

This concept relies on the sensors embedded within the watch itself, particularly the gyroscope that tracks how much the wearer’s wrist is rotating and in which direction. “The watch is able to figure out when you are having your meals based on the pattern of your hand activity – for example, the gesture of moving your hand to your mouth – and can also detect how many spoonfuls you eat,” he explains.

But the most interesting piece of the system, he adds, is the smartwatch’s camera, which can automatically capture and curate useful images of the food consumed. “The camera can be programmed to be triggered as your hand is moving down from your mouth to your food to take a picture of the food, and at the end the system can build a personal portal of when you eat your meals, with pictures of the meals you eat.”

Automating the process of building a food diary can make it easier and more convenient to support wellness-related goals, he says, such as losing weight or changing poor eating habits – eating too fast or eating unhealthy foods, for example.

Not only can wearers analyse their own eating patterns, the information in the food diary can also be shared with nutritionists, family and friends to garner support and improve accountability, Professor Misra says. He envisions Annapurna being used by organisations such as the Health Promotion Board to encourage healthy eating.

In the six years since its founding, LiveLabs has made significant progress in developing relevant solutions for a wide range of urban needs; it is next planning to tackle problems in the healthcare and wellness space, says Professor Balan. He adds that all this headway was made possible through sharing a joint purpose with his “awesome colleagues, who are experts in their own areas… to solve problems of real societal need.”

“LiveLabs is a labour of love for myself and my fellow colleagues. Back then, when we first began, it was unclear if we could actually develop solutions to solve deep problems and translate them to industry usage,” Professor Balan muses. “However, today we see that that is possible with the right ideas, solutions, perseverance and partners.” 

Back to Research@SMU Issue 48