همایش بین المللی هوش مصنوعی

صفحه اصلی / دومین همایش بین المللی هوش مصنوعی

Stacked Interaction Module for Refining Subject Object Dynamics in Video Captioning

نویسندگان :

Mahdi Khourishandiz¹ Abdollah Amirkhani²

1- دانشگاه علم و صنعت ایران 2- دانشگاه علم و صنعت ایران

کلمات کلیدی :

Video captioning،Interaction،hierarchical،Subject-Object

چکیده :

This Current video captioning models fail to grasp complex Subject-Object interactions, yielding 'entangled' representations where roles are blurred. We introduce SIMCap, a novel hierarchical architecture that disentangles these interactions by explicitly separating roles. Instead of relying on complex external graphs, SIMCap leverages an innovative Stacked Interaction Module (SIM). This module empowers Subjects to actively query a rich memory bank of Objects and video context. This process is stacked iteratively, allowing the model to progressively refine its understanding of the interaction. By intrinsically modeling relationships, SIMCap achieves State-of-the-Art results on the standard MSR VTT and MSVD benchmarks, demonstrating decisive superiority, particularly in the CIDEr semantic accuracy metric.