Full Time Scientist, Centre for Frontier AI Research, IAIC Jobs, Salary up to SGD 4,750 in A*STAR RESEARCH ENTITIES Islandwide (Singapore) - Maukerja

The Trustworthy AI team at the Centre for Frontier AI Research (CFAR), IAIC, is seeking motivated scientists to conduct research on the safety of generative AI (GenAI) and multi-agent systems. Working with regulatory and clinical partners, the team develops the scientific evidence, testing tools and guidance needed for the safe and responsible deployment of AI solutions in real-world.

GenAI and multi-agent systems, such as clinical chatbots built on large language models (LLMs) and vision-language models (VLMs) that interpret medical images, introduce risks that existing regulatory frameworks were not designed to address. These include open-ended outputs, hallucinations, the lack of accepted safety metrics, and vulnerability to jailbreaks, prompt injection and data leakage. You will conduct cutting-edge R&D to assess, test and mitigate these risks, and translate your findings into regulatory-grade toolkits and guidance with real-world meaningful impact.

Key responsibilities

• Design and conduct systematic red-teaming and risk assessments of GenAI and multi-agent systems, covering adversarial attacks, hallucinations, privacy leakage and performance across diverse scenarios
• Develop efficient guardrails and risk-mitigation methods, such as knowledge validation, hallucination and uncertainty detection, adversarial input detection, and automated flagging of high-risk outputs for human review
• Develop methods for continuous post-deployment monitoring, including near real-time performance tracking, drift detection and anomaly alerting

Competencies

1. Integrity

2. Critical Thinking

3. Innovativeness

4. Effective Communication

Achieving Outcomes

IDARE Behaviours - 4. Reach Outward, Tackle Real-World Challenges

Research areas (one or more of the following)

• Red-teaming and adversarial robustness of LLMs, VLMs and LLM-based agents, e.g., jailbreaks, prompt injection, data poisoning, backdoors and model extraction
• Hallucination detection and mitigation, uncertainty quantification and calibration for medical GenAI
• Guardrails and runtime safety mechanisms for clinical conversational agents and multimodal assistants
• Robustness to distribution shift, and fairness across patient populations, clinical sites and imaging devices

Requirements

• PhD degree in Computer Science, Electrical & Electronic Engineering, Computer Engineering, Biomedical Engineering or a related field
• Solid programming skills in Python and hands-on experience with deep learning frameworks such as PyTorch
• Research experience in one or more of the following: AI safety and security, adversarial machine learning, trustworthy machine learning, LLMs or VLMs, and multi-agent systems
• Good communication and teamwork skills, and the ability to work with clinicians, regulators and engineers in a multidisciplinary setting