Exploiting mobile devices to provide better smart city services

By Grace Segran

SMU Office of Research and Tech Transfer – One vision of smart cities is to exploit mobile devices to collect information, extract knowledge, and make decisions about the citizens and their living environment to provide intelligent personal and urban services. This vision is grounded in the deep penetration of mobile devices (e.g. smartphones) into their users’ daily life. It is expected that millions of such wirelessly connected, sensor-rich, and constantly moving devices will create ubiquitous sensing, computing and interacting fabrics.

SMU Assistant Professor of Information Systems Zhou Zimu’s goal is to develop novel data-driven solutions upon these mobile devices for better smart city services. Professor Zhou teaches Computer Hardware and Embedded Systems. His research and interest areas are Software and Cyber Physical Systems.

“The three key technical challenges to enable ubiquitous sensing, computing, and interacting in mobile systems for smart cities are: empowering low-cost sensors and everyday objects with extended sensing coverage and improved sensing capability, executing computation-intensive inference tasks on resource-constrained mobile devices, and adapting urban services in real-time based on the dynamics of user mobility,” he tells the Office of Research and Tech Transfer.

Professor Zhou’s approach is to explore signal processing, machine learning, optimisation algorithms, and domain sciences in a way that is suited for specific applications. “For example, in air pollution sensing on wearables, the challenge is that the low-cost air pollution sensors are cross sensitive to the emissions from the human body. Hence there will be interference and errors in the sensor readings,” Professor Zhou expounds. “Our solution is to add a complementary sensor to perceive the interference at a signal level, and then design a neural network to calibrate these sensor readings so that we can recover the actual sensor readings from this human interference.”

Multi-task neural network compression

In his research paper ‘Multi-Task Zipping via Layer-wise Neuron Sharing,’ Professor Zhou proposes multi-task zipping to automatically merge correlated pre-trained neural networks for cross-model compressions. So how does this help everyday devices better utilise mobile storage and memory?

“Let's say for example, there is one neural network that can identify the eye colour from a face image, and another neural network that can identify whether a person is wearing glasses from the first image. If we want both functionalities on the smartphone, we need to store two neural networks,” explains Professor Zhou.

But he believes his research can do better than that. “Both neural networks should first recognise the eyes in the image, meaning that they have some common functionalities. So you only need to store one copy of the neurons that has this functionality, and in this way you can save storage. Our algorithm automatically identifies the common functionalities of the neurons and merge multiple neural networks by only sharing one copy of these neurons. Thus saving storage and memory.”

What exactly are these neurons and neural networks to the man in the street?

According to Professor Zhou, the layman can understand a neural network as neurons plus a lot of connections with neurons being units in the software. Different neurons have different functionalities according to their positions or connections in the software. However, the neural network is like a black box. It is very difficult to identify which neurons are performing which functionality. “And so that's the beauty of the algorithm -- we can automatically identify whether two neurons in two neural networks have similar functionalities.”

User privacy and security

By nature of its pervasive reach, ubiquitous computing creates new vulnerabilities in the IT systems that support its functions. This goes into the realm of privacy, IoT, and online security – which may affect physical security.

Let’s take for example, user consent within the context of human body temperature sensing and automatic air-conditioning adjustment – how would a researcher deal with that? Although network security is not Professor Zhou’s main research area, he says that his current research is motivated by user privacy and security in the human body temperature sensing application.

“I’d take a different approach and try to integrate all the sensing, processing, and decision-making locally on the device without transmitting any data to another untrustworthy party like a cloud,” he avers. This way, user privacy is also protected.

“(In principle) I need to improve the sensing capabilities of the device and try to adapt complex data processing algorithms to make them run more efficiently on the device. In the case of air-con control feedback, it is probably still impossible to integrate the air-con onto your device. Then there needs to be extra security or protocols to protect privacy from the control itself.”

On the ubicomp horizon  

Currently most ubiquitous computing applications are even more powerful than before because they are using the latest AI technology. However, because these technologies are data- and computation-intensive, they are uploaded to the cloud. This means that if you want to use applications such as augmented and virtual reality, you need to connect to a network. Professor Zhou believes that future ubiquitous computing applications should be more autonomous.

“Let's say if you want a self-driving car, a personal drone, or a home robot, these applications are expected to perceive, understand, and react to the user and the environment in real time, anywhere and anytime, even when there is no network connection. That is why you need to put all the sensing, computing, and the decision making on one single device,” he argues. “Let's take two areas where my research focus lies – a drone or a robot. I try to contain these complex computations and adapt them to their target application so that they can truly run locally on resource-constrained devices.”

There is also the benefit of real-time interaction because there is no delay from the device to the cloud and back, he concludes.

Back to Research@SMU Feb 2020 Issue