ADQA Benchmark Submission
(Temporary)
Until the official evaluation website is launched, we are accepting submissions for CMD-AD via email primarily for CVPR 2026 submissions.
MAD-eval submissions are temporarily on hold while we conduct a review to ensure fair evaluation.
Please submit your results in a .csv format. We will evaluate your submission on the ADQA benchmark and return the results.
Before submitting, please carefully read and agree to the terms and conditions below.
The submission format must match the format
here.
We will automatically insert dialogs between ADs before evaluation.
If you would like to submit full transcripts of dialogs and ADs for each CMD-AD video, please contact us separately.
Send your submission to divykala@outlook.com with makarand.tapaswi@iiit.ac.in in CC, with the subject line: [ADQA Eval Submission] (Team Name).
Keep all team members in CC.
Submission Details
Your email should include the following information:
- Team name
- Affiliation (which university, company, etc.)
- Personal, lab, or Google Scholar URL
- Country of affiliation
- Full name of the submitter
- Full names of other team members
Terms and Conditions
-
The training data must not include any movies from the CMD-AD-Eval set, in any form.
Participants are responsible for ensuring proper separation between training and validation data.
-
Please avoid multiple submissions as that can lead to overfitting on the test set.
We will allow a maximum of 1 submission in 10 days per team.
You may test multiple configurations on the public ADQA split,
but only submit the best-performing version for evaluation.
-
Each team should use only one account for submissions.
Do not submit from different email addresses.
Collaborative groups or labs should use a single account.
All submissions must include full participant details and institutional email addresses.
-
Submitted results may be used in future benchmark versions and could be displayed (anonymized on request) on the public leaderboard.
-
We request the participants to follow these terms and conditions.
To maintain fairness, violations may lead to warnings, blacklisting from future submissions, or removal of scores from the public leaderboard.
If you have any questions, please reach out to Divy Kala (divykala@outlook.com) or Makarand Tapaswi (makarand.tapaswi@iiit.ac.in).