State-of-the-art visual perception models in autonomous vehicles (AV) fail in the physical world when meeting adversarially designed physical objects/environmental conditions. The main reason is that they are trained with discretely-sampled samples and can hardly cover all possibilities in the real world. Although effective, existing physical attacks consider one or two physical factors and cannot simulate dynamic entities (e.g., moving cars or persons, street structures) and environment factors (e.g., weather variation and light variation) jointly. Meanwhile, most defence methods like denoising or adversarial training (AT) mainly rely on single-view or single-modal information, neglecting the multi-view cameras and different modality sensors on the AV, which contain rich complementary information. The above challenges in both attacks and defenses are caused by the lack of a continuous and unified scene representation for the AV scenarios. Motivated by the above limitations, this project firstly aims to develop a unified AV scene representation based on the neural implicit representation to generate realistic new scenes. With this representation, we will develop extensive physical attacks, multi-view & multi-modal defenses, as well as a more complete evaluation framework. Specifically, the project will build a unified physical attack framework against AV perception models, which can adversarially optimize the physical-related parameters and generate more threatening examples that could happen in the real world. Furthermore, the project will build the multi-view and multi-modal defensive methods including a data reconstruction framework to reconstruct clean inputs and a novel ‘adversarial training’ method, i.e., adversarial repairing that enhances the robustness of the deep models with guidance of collected adversarial scenes. Finally, a robust-oriented explainable method will be developed to understand the behaviors of visual perception models under physical adversarial attacks and robustness enhancement.
This project aims to study the implications of Dhaka’s new mass rapid transit system, the Dhaka Metro Rail (DMR), on the distribution of socioeconomic activity and mobility within the city. It will examine the impacts on economic activity, pollution and welfare within the city, and its findings may inform policymakers in Bangladesh and beyond on the impacts of public transportation and importance for sustainable development.
(This is additional funding to SMU for the existing research project.) This project aims to study how better care options can be provided and developed for the local community. The first study will centre around the awareness and preferences of Singapore residents aged 50 – 76 regarding Assisted Living, with a set of survey questions to be designed and fielded through the Singapore Life Panel. Other focus areas will be developed over the course of this 2-year collaboration based on up-and-coming topics as they emerge.
The TAICeN project is focused on developing AI-based solutions for cybersecurity tasks, with two primary research areas: AI for Cybersecurity and Trustworthy AI. Part 1, AI for Cybersecurity, investigates advanced AI technologies for defending against cyber threats such as malware detection, intrusion detection on government cloud systems, crypto attribution, and insider attack attribution. Part 2, Trustworthy AI, ensures the security, robustness, and explainability of the AI-based solutions developed in Part 1. Part 1 will be primarily conducted by NTU, NUS, and BGU, in collaboration with local government agencies. Part 2, led by SMU, explores three key research topics: AI Security, Robustness, and Explanation. AI Security focuses on techniques to mitigate inference attacks, model extraction attacks, adversarial attacks, and poisoning attacks on the solutions. AI Robustness aims to provide quality assurance for AI systems by offering methods to evaluate, debug, and improve AI systems. AI Explainability enables human comprehension and reasoning of the AI decision-making process, which is crucial when predictions have national or safety-critical implications. By addressing both research challenges, the TAICeN project aims to develop effective and trustworthy AI-based solutions for cybersecurity that can keep pace with cybercriminals, automate threat detection, and defend against attacks. Specifically, Prof. Xie Xiaofei and Prof. Sun Jun will focus on the AI Robustness and AI Explainability work packages together with NTU and the relevant partner institutions to demonstrate their research results together with the relevant translation partners.
The project aims to collect data to assess the efficacy of cool paints in mitigating the Urban Heat Island (UHI) effects in schools through the deployment of micro-scale sensors.
This project will pioneer approaches that realize trusted automation bots that act as concierges and interactive advisors to software engineers to improve their productivity as well as software quality. TrustedSEERs will realize such automation by effectively learning from domain-specific, loosely-linked, multi-modal, multi-source and evolving software artefacts (e.g., source code, version history, bug reports, blogs, documentation, Q&A posts, videos, etc.). These artefacts can come from the organization deploying the automation bots, a group of collaborating yet privacy-aware organizations, and from freely available yet possibly licensed (e.g., GPL v2, GPL v3, MIT, etc.) data contributed by many, including untrusted entities, on the internet. TrustedSEERs will bring about the next generation of Software Analytics (SA) – a rapidly growing research area in the Software Engineering research field that turns data into automation – by establishing two initiatives: First, data-centric SA, through the design and development of methods that can systematically engineer (link, select, transform, synthesize, and label) data needed to learn more effective SA bots from diverse software artefacts, many of which are domain-specific and unique. Second, trustworthy SA, through the design and development of mechanisms that can engender software engineers’ trust in SA bots considering both intrinsic factors (explainability) and extrinsic ones (compliance to privacy and copyright laws and robustness to external attacks). In addition, TrustedSEERs will apply its core technologies to synergistic applications to improve engineer productivity and software security.
(This is additional funding to SMU for the existing research project.) MSE and SMU are collaborating to conduct the Public Cleanliness Satisfaction Survey (PCSS), an annual national household survey that aims to measure and track Singaporeans’ satisfaction and perceptions towards public cleanliness and public hygiene. Findings from the survey will aid in identifying key areas of concern and recommendations which are policy or operational in nature, to improve the public’s levels of satisfaction of public cleanliness, public hygiene and/or public cleaning services.
Consumers have widely used conversational AI systems such as Siri, Google Assistant and now ChatGPT. The next generation of conversational AI systems will have visual understanding capabilities to communicate with users through language and visual data. A core technology that enables such multimodal, human-like AI systems is visual question answering and the ability to answer questions based on information found in images and videos. This project focuses on visual question answering and aims to develop new visual question-answering technologies based on large-scale pre-trained vision-language models. Pre-training models developed by tech giants, particularly OpenAI, have made headlines in recent years, e.g., ChatGPT, which can converse with users in human language, and DALL-E 2, which can generate realistic images. This project aims to study how to best utilise large-scale pre-trained vision-language models for visual question answering. The project will systematically analyse these pre-trained models in terms of their capabilities and limitations in visual question answering and design technical solutions to bridge the gap between what pre-trained models can accomplish and what visual question answering systems require. The end of the project will be a new framework for building visual question-answering systems based on existing pre-trained models with minimal additional training.
Data visualisations have been widely used on mobile devices (e.g., smartphones), but they suffer from mobile-friendly issues in terms of their creation and usage. This project aims to develop novel techniques to achieve mobile-friendly data visualisations, including desirable mobile data visualisation creation and effective multimodal interaction design. The research outputs of this project will significantly improve the effectiveness and usability of mobile data visualisations and further promote their applications.
This project aims to improve the scalability of food recognition – to train classifier(s) that recognise a wide range of dishes regardless of cuisines, the amount and type of training examples. Here, “classifier” can be viewed as a “search engine” that retrieves the recipe of a food image. Training such classifiers requires an excessive number of training examples composed of recipes and images, where each recipe is paired with at least an image as visual reference. Training classifiers using paired or parallel data faces several practical limitations – tens of thousands of recipe-image pairs are required for training; other forms of data that are largely available in the public cannot be leveraged for model training; and additional training data is required when the recipes are written in different natural languages. Through the project, these practical limitations will be addressed from the perspective of transfer learning. The aim is to train a generalised classifier that is more adaptable for recognition, by removing the statistical bias, considering the evolving process, and aligning the semantics of different languages in machine learning.
