Amii is excited to highlight the research that our scientists and students are presenting at the 39th Annual AAAI Conference on Artificial Intelligence, including efforts to make vision-language models more secure against attempts to generate harmful content and improve the ability of AI agents to plan out actions in complex environments.
This year's conference is taking place in Philadelphia from Feb. 25 - March 4th.
AAAI 2025 is hosted by the Association for the Advancement of Artificial Intelligence and is one of the premier international events for AI researchers. The AAAI Conference covers a wide range of topics within AI, including machine learning, computer vision, natural language processing, robotics, and ethical considerations of AI technologies.
This year, Amii’s researchers and their students are presenting papers, workshops and tutorials that advance the science of artificial intelligence. Learn more about the presented papers by reading their abstracts below.
Internal Activation Revision: Safeguarding Vision Language Models without Parameter Update
Qing Li, Jiahui Geng, Zongxiong Chen, Kun Song, Lei Ma, Fakhri Karray
Warning: This paper contains offensive content that may disturb some readers. Vision-language models (VLMs) demonstrate strong multimodal capabilities but have been found to be more susceptible to generating harmful content compared to their backbone large language models (LLMs).
Our investigation reveals that the integration of images significantly shifts the model’s internal activations during the forward pass, diverging from those triggered by textual input. Moreover, the safety alignments of LLMs embedded within VLMs are not sufficiently robust to handle the activations discrepancies, making the models vulnerable to even the simplest jailbreaking attacks. To address this issue, we propose an internal activation revision approach that efficiently revises activations during generation, steering the model toward safer outputs. Our framework incorporates revisions at both the layer and head levels, offering control over the model’s generation at varying levels of granularity. In addition, we explore three strategies for constructing positive and negative samples and two approaches for extracting revision vectors, resulting in different variants of our method. Comprehensive experiments demonstrate that the internal activation revision method significantly improves the safety of widely used VLMs, reducing attack success rates by an average of 48.94%, 34.34%, 43.92%, and 52.98% on SafeBench, SafeUnsafe, Unsafe, and MM-SafetyBench, respectively, while minimally impacting model helpfulness.
An Error-Accumulation-Resistant Ensemble Method for Unsupervised Dependency Parsing
Behzad Shayegh, Hobie H.-B. Lee, Xiaodan Zhu, Jackie Chi Kit Cheung, Lili Mou
We address unsupervised dependency parsing by building an ensemble of diverse existing models through post hoc aggregation of their output dependency parse structures. We observe that these ensembles often suffer from low robustness against weak ensemble components due to error accumulation. To tackle this problem, we propose an efficient ensemble-selection approach that considers error diversity and avoids error accumulation. Results demonstrate that our approach outperforms each individual model as well as previous ensemble techniques. Additionally, our experiments show that the proposed ensemble-selection method significantly enhances the performance and robustness of our ensemble, surpassing previously proposed strategies, which have not accounted for error diversity.
Adaptive Iterative Feedback Prompting for Obstacle-Aware Path Planning via LLMs
Masoud Jafaripour, Shadan Golestan, Shotaro Miwa, Yoshihiro Mitsuka, Osmar R. Zaiane
Planning is essential for agents operating in complex decision-making tasks, particularly in Human-Robot Interaction (HRI) scenarios, which often require adaptability and the ability to navigate dynamic environments. Large LanguageModels (LLMs), known for their exceptional natural language understanding capabilities, hold promise for enhancing planning in HRI by processing contextual and linguistic cues. However, their effectiveness is limited by inherent shortcomings in spatial reasoning. Existing LLM-based planning frameworks often depend on combining with classical planning methods or struggle to adapt to dynamic environments, limiting their practical applicability. This paper examines whether the incorporation of an environmental feedback mechanism and iterative planning can enhance the planning capabilities of LLMs. Specifically, we propose the ”AdaptiveIterative Feedback Prompting” (AIFP) framework for path planning. In AIFP, an LLM generates partial trajectories iteratively, which are evaluated for potential collisions using environmental feedback. Based on the evaluation, AIFP executes the trajectory or re-plans. Our preliminary results show that AIFP increases the success rate of the baseline by 33.3%and generates efficient, appropriately complex paths, making it a promising approach for dynamic HRI scenarios.
Constrained Generative Modeling with Manually Bridged Diffusion Models
Saeid Naderiparizi,Xiaoxuan Liang,Berend Zwartsenberg,Frank Wood
[The abstract for this paper is not yet available]
Tutorial: Advancing Offline Reinforcement Learning: Essential Theories and Techniques for Algorithm Developers
Fengdi Che, Ming Yin
This tutorial will equip empirical reinforcement learning (RL) researchers, including graduate students, early-career researchers and industry practitioners, with a deep theoretical understanding of offline RL. By explaining the necessary and sufficient conditions for theoretical guarantees, participants will gain insights into the challenges of offline RL compared to supervised learning and online RL, including reliance on bootstrapping targets, partial state-action space coverage, and spurious data.
Participants will first explore essential conditions for theoretical guarantees under these challenges and their connection to empirical limitations, such as dataset quality and neural network expressivity. The session will also cover advanced techniques for overcoming the difficulties of Offline RL under more realistic, weaker theoretical assumptions, including pessimism and density ratio estimation. Additionally, Hybrid Reinforcement Learning (Hybrid RL) approaches that integrate offline data with online interactions will be introduced to enhance exploration and data efficiency. This tutorial equips algorithm developers and early-career researchers with the tools to improve offline RL applications by combining theoretical insights with practical algorithmic strategies.
Participants attending this tutorial need to know basic reinforcement learning principles, such as Markov Decision Processes, value functions, and the optimal Bellman operator. Little mathematical knowledge is required since the tutorial will not cover detailed math proofs. Prior knowledge of offline RL algorithms will be beneficial but optional.
Suboptimal Search with Dynamic Distribution of Suboptimality
Mohammadreza Hami, Nathan R. Sturtevant
[The abstract for this paper is not yet available]
Anchor Search: A Unified Framework for Suboptimal Bidirectional Search
Sepehr Lavasani, Lior Siag, Shahaf S. Shperberg, Ariel Felner, Nathan R. Sturtevant
[The abstract for this paper is not yet available]