Choreography with computers

By Sim Shuzhen

SMU Office of Research & Tech Transfer – It’s rush hour, and hundreds of cars are taking turns to pass through a busy five-way intersection. But something is different: there are no honking noises, angry words or wasted moments – just smooth-flowing traffic as far as the eye can see.

Sitting comfortably in your vehicle, you don’t even notice this carefully orchestrated ballet, because you are too busy reading something on your phone. You’re not the driver; no one is – all the cars on the road are not only autonomous, but also capable of making decisions in concert with one another to get riders safely and quickly to their destinations.

In the future, a scenario like this is what Assistant Professor Akshat Kumar of the Singapore Management University (SMU) School of Information Systems envisions for our increasingly connected urban societies.

“One of my research aims is to develop computationally efficient techniques to realise the promise of large cooperative teams making decisions towards a common goal,” says Professor Kumar, who uses multiagent planning and automated decision-making to optimise the movement not only of vehicles on land, but also of autonomous ships in maritime ecosystems.  

“Once we have coordination among a large number of agents – autonomous cars or ships, for example – then we would be able to see drastic improvements in safety and productivity,” says Professor Kumar.  

Learning by simulation

Professor Kumar’s computational models are based on real data, some of which is made available to researchers through the Fujitsu-SMU Urban Computing and Engineering Corp Lab (UNiCEN).

For example, to optimise the movement of taxi fleets, Professor Kumar relies on supply and demand data for various regions of Singapore. “Based on that, we can try to find a good dispatching policy such that the revenue of the whole fleet is optimised,” he says. Similarly, aerial location data for ships that enter Singapore waters can be used to understand their navigation patterns, and hence to coordinate their movements.

Because the movement of a large number of independent agents is too complex for hard-coded rules to deal with, Professor Kumar uses multiagent reinforcement learning – a form of machine learning – to discover optimisation strategies.

“One of the main promises of reinforcement learning is that it helps you optimise policies just by using a simulator. This is what enables decision-making in these complex and noisy environments,” he explains. “We build simulators for different urban phenomena – for maritime ecosystems or for how taxis move in urban environments, for example. Using the data coming from these simulators, we can actually optimise agent policies.” 

Professor Kumar’s work on optimising maritime traffic in the Singapore Straits was published in a paper titled ‘Coordinating Vessel Traffic To Improve Safety and Efficiency’ in the 2017 Proceedings of the 16^th International Conference on Autonomous Agents and Multiagent Systems.

“These multiagent scheduling and planning techniques could one day be used by maritime port authorities to coordinate the arrival and departure of ships,” says Professor Kumar. “One important end goal of this work is to reduce congestion and to make it safer to navigate Singapore waters.”

Mix it up, scale it up

It will be some time before our roads and waterways are completely taken over by autonomous entities. Meanwhile, Professor Kumar has to contend with a difficult research challenge – that of how autonomous vehicles or vessels will share their environments with human pilots.

“Robots will follow whatever policies or programmes we give them, but humans are unpredictable and often work more or less in their own self-interest, bounded by some constraints of the system,” says Professor Kumar.

Integrating these two types of decision-makers into computational algorithms is a priority for the field. “Right now it is an open question – how to come up with safe algorithms that can enable such coordination without affecting the safety and productivity of the whole system,” he adds.

A second challenge is scale, or the ability to control thousands of agents while still maintaining computational efficiency. Developing approaches that achieve this and that also show non-trivial improvements in productivity and efficiency remains one of the frontiers of multiagent reinforcement learning, says Professor Kumar.

Towards an autonomous future

Because it is extremely difficult to develop policies that are failsafe under every scenario, Professor Kumar’s algorithms currently play more of a supportive role, where the decision of whether or not to accept their recommendations is left to a human operator.

Ideally, however, autonomous agents should be able to deal with unexpected occurrences. Driverless taxis, for example, should be able to react to accidents or other major disruptions on the road.

“Here, we wouldn’t want taxis to follow their normal policies; instead, we’d want the behaviour of the system to be shaped by these exceptional scenarios. But it’s very hard to train for these because there are so few data points in the real world,” explains Professor Kumar. There is thus a need to build specific capabilities into decision-making algorithms that can behave predictively in these cases, he adds. 

Beyond transportation, Professor Kumar’s work on multiagent decision-making can also be applied to connected urban environments in general, which are increasingly being populated with smart homes, personal digital assistants and Internet-of-Things devices. To benefit from all this connectivity, new computational technologies are needed to achieve coordination amongst the plethora of sensors and devices in the ecosystem, he says.

“Autonomous cars and vessels are just the beginning. In a few years, almost everything is going to be infused with artificial intelligence, sensory capabilities and the ability to take action,” said Professor Kumar. “How we optimise our long-term productivity, efficiency and safety is something I'm very excited to work on in the future.” 

Back to Research@SMU Issue 47