Ivor Tsang
[introductory/intermediate] Trustworthy Agentic Artificial Intelligence
Summary
This course introduces trustworthy agentic AI, a new approach to building intelligent systems that can plan goals, make long horizon decisions, learn from experience, and act reliably in real-world environments. Instead of treating AI as a single model that reacts to inputs, agentic AI studies how multiple components, such as planning, skills, memory, and control, work together to form an intelligent agent.
The course is organized into four connected modules that follow the full lifecycle of an agentic system:
- how agents plan and optimize workflows,
- how they learn and adapt from experience,
- and how these capabilities are integrated into real applications such as web agents, GUI agents, and embodied systems.
Throughout the course, we emphasize trustworthiness, focusing on safety, correctness, and reliability as core design principles for agentic AI systems.
Syllabus
Module 1: Agentic Planning and Workflow Optimization
This module studies how agentic systems represent and optimize long-horizon decision-making processes as structured workflows rather than flat policies or token sequences. The focus is on constraint-aware planning, where trustworthiness is enforced through explicit representations, verification, and optimization mechanisms.
Key themes:
- Agentic workflows vs. policies and programs
- Symbolic-neural workflow representations
- Safety- and constraint-aware optimization
- Search, evolutionary methods, and workflow verification
Module 2: Learning from Experience: Agentic Memory and Adaptation
This module examines how agentic systems improve over time by accumulating, structuring, and exploiting experience. Trustworthy agency is framed as an adaptive process, where memory and feedback drive continual refinement rather than static behavior.
Key themes:
- Agentic memory: episodic, structured, and symbolic
- Experience-driven planning and adaptation
- Counterfactual reasoning and reflection
- Continual and meta-learning in agentic systems
Module 3: Agentic Systems in Practice
This module integrates planning, skills, and learning into end-to-end agentic systems deployed in real environments. The focus is on grounding abstract agentic representations into concrete actions while ensuring robustness, safety, and verifiability in execution.
Key themes:
- Web agents, GUI agents, and embodied agents
- Execution monitoring and failure handling
- Trust, safety, and reliability in deployed agentic systems
References
Keyi Xiang, Zeyu Feng, Zhuoyi Lin, Yueming Lyu, Boyuan Shi, Yew-Soon Ong, Ivor Tsang, Haiyan Yin. FlowSearcher: Synthesising Memory-Guided Agentic Workflows for Web Information Seeking, ICLR 2026.
Jiejing Shao, Haiyan Yin, Yueming Lyu, Xingrui Yu, Lanzhe Guo, Ivor Tsang, James Kwok, Yufeng Li. Lifting Traces to Logic: Programmatic Skill Induction with Neuro-Symbolic Learning for Long-Horizon Agentic Tasks, ICM 2026.
Haotian Chi, Zeyu Feng, Xingrui Yu, Linibo Luo, Yew-Soon Ong, Ivor Tsang, Hechang Chen, Yi Chang, Haiyan Yin. EvoCF: Multi-Agent Collaboration via Agentic Memory-Driven Evolutionary Counterfactual Planning, ICML 2026.
Chengqi Zheng, Jianda Chen, Yueming Lyu, Wen Zheng Terence Ng, Haopeng Zhang, Yew-Soon Ong, Ivor W. Tsang, Haiyan Yin. MermaidFlow: Redefining Agentic Workflow Generation via Safety-Constrained Evolutionary Programming. https://arxiv.org/abs/2505.22967 (2025).
Haiyan Yin, Hangwei Qian, Yaxin Shi, Ivor W. Tsang, Yew-Soon Ong. Grounding Open-Domain Knowledge from LLMs to Real-World Reinforcement Learning Tasks: A Survey. IJCAI 2025: 10797-10806.
Pre-requisites
Students should have basic knowledge of machine learning, reinforcement learning, planning, foundation models, and agentic AI. Familiarity with LLMs, search, optimization is recommended. Students should be comfortable building simple learning or agentic systems.
Short bio
Professor Ivor W. Tsang is Director of the A*STAR Centre for Frontier AI Research (CFAR), Adjunct Professor at Nanyang Technological University, Singapore, and Honorary Professor of Artificial Intelligence at the University of Technology Sydney, reflecting a strong international academic and research presence.
His research spans transfer learning, deep generative models, weakly supervised learning, and large-scale analytics for ultra-high-dimensional data. He is internationally recognized for foundational contributions to both machine learning theory and real-world applications. His distinctions include the ARC Future Fellowship, the International Consortium of Chinese Mathematicians Best Paper Award, AI 2000 AAAI/IJCAI Most Influential Scholar in Australia, the CVPR Best Student Paper Award, the IEEE TMM Prize Paper Award, and the IEEE TNN Outstanding Paper Award. He was elected IEEE Fellow for his contributions to large-scale machine learning and transfer learning.
Since 2024, Professor Tsang has led Singapore’s national initiative on Trustworthy Foundation Models under the National Multimodal LLM Programme, shaping the country’s strategic direction in AI. He also leads research on Agentic World Models and oversees major national initiatives, including the AI Singapore Materials Design Grand Challenge and the Maritime AI Programme.
He holds prominent editorial and conference leadership roles, including Associate Editor-in-Chief for the Machine Learning track of IEEE TPAMI, and serves as Senior Area Chair or Area Chair for leading conferences such as NeurIPS, ICML, AAAI, and IJCAI.
