Vicente Ordonez's Homepage

Vicente Ordonez

Vicente Ordóñez Román

Associate Professor

Department of Computer Science

Rice University

vicenteor@rice.edu

My research lies at the intersection of Computer Vision, Natural Language Processing and Machine Learning. I am interested in how to develop machine learning models that can understand the real world through multiple modalities and can learn naturally from human guidance. I am generally interested in building efficient visual recognition models that can perform high-level perceptual tasks and doing so in a way that is fair, transparent, and interpretable.

I'm Associate Professor in the Department of Computer Science at Rice University where I lead the Vision, Language, and Learning Lab and a research cluster on Closed-loop Computer Vision as part of the Ken Kennedy Institute. I'm also an Amazon Visiting Academic at the Amazon AGI Foundations team. From 2016-2021 I was an Assistant Professor in the Department of Computer Science at the University of Virginia. In the past, I have also been visiting professor at Adobe Research and visiting researcher at the Allen Institute for Artificial Intelligence (AI2). I received my PhD in Computer Science at the University of North Carolina at Chapel Hill in 2015 advised by Prof. Tamara L. Berg, an MS in Computer Science at Stony Brook University (SUNY) and an engineering degree at the Escuela Superior Politécnica del Litoral in Ecuador. I'm a recipient of a Best -Long- Paper Award at the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), and the Best Paper Marr Prize at the 2013 International Conference in Computer Vision (ICCV). I have also received an NSF CAREER Award, an IBM Faculty Award, a Google Faculty Research Award, and a Facebook Research Award. Here is a link to an official bio, and my curriculum vitae.

News and Updates

12/2024. Invited Talk at the University of Pittsburgh.
09/2024. Keynote Speaker at Pan-American Workshop on Machine Learning (PWML) 2024, Cuzco, Peru.
07/2024. Invited Talk at Air Force Research Lab and Air Force Institute of Technology at the Wright-Patterson Air Force Base, Ohio.
05/2024. Invited Talk at CIMAT, Mexico -- Virtual.
04/2024. Invited Talk at the University of British Columbia.
Serving as Area Chair for EMNLP 2024 (ACL-RR), ECCV 2024, and CVPR 2025.

Teaching

Deep Learning for Vision & Language [Spring 2022] [Spring 2023] [Spring 2024] [Spring 2025]
Computer Vision Seminar [Fall 2022] [Fall 2023] [Fall 2024]
Introduction to Computer Vision [Spring 2018] [Fall 2019] [Spring 2021]
Vision & Language [Spring 2017] [Fall 2020]
Deep Learning for Visual Recognition [Spring 2019] [Spring 2020]
Computational Visual Recognition [Fall 2016] [Fall 2017]

I have also been co-organizing with students in my group an informal Computer Vision seminar, and from 2017-2021 I co-directed with Paul Humphreys† the Human and Machine Intelligence seminar.

Selected Publications

Generative AI for Computer Vision and Beyond.
Vision-Language Models for Visual Grounding.
Generic Visual Representation Learning for Images and Video.
Uncovering and Mitigating Biases in Vision and Language Models.
- Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations, ICCV 2019 [demo].
- Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints, EMNLP 2017 -- Best Paper Award.

Whitepaper

Facial Recognition Technologies in the Wild: A Call for a Federal Office
Facial Recognition Technologies: A Primer [Companion Document]
Erik Learned-Miller, Vicente Ordóñez, Jamie Morgenstern, Joy Buolamwini.
This whitepaper makes the case for a federal office in charge of regulating Face Recognition Technologies (FRTs). We argue that benchmarks are insufficient for determining the appropriateness for FRTs and a more holstic approach is needed that takes into account technical, societal and legal challenges.
May 29th 2020. https://www.ajlunited.org/federal-office-call

Preprints

ParallelSpec: Parallel Drafter for Efficient Speculative Decoding
Zilin Xiao, Hongming Zhang, Tao Ge, Siru Ouyang, Vicente Ordonez, Dong Yu. arXiv:2410.05589 October 2024. [arxiv]
Fairness and Bias Mitigation in Computer Vision: A Survey
Sepehr Dehdashtian, Ruozhen He, Yi Li, Guha Balakrishnan, Nuno Vasconcelos, Vicente Ordonez, Vishnu Naresh Boddeti. arXiv:2408.02464 August 2024. [arxiv]
Taming Data and Transformers for Audio Generation
Moayed Haji-Ali, Willi Menapace, Aliaksandr Siarohin, Guha Balakrishnan, Sergey Tulyakov, Vicente Ordonez. arXiv:2406.19388 June 2024. [project page] [arxiv]
Generative Visual Instruction Tuning
Jefferson Hernandez, Ruben Villegas, Vicente Ordonez. arXiv:2406.11262 June 2024. [github] [arxiv]
FlexEControl: Flexible and Efficient Multimodal Control for Text-to-Image Generation
Xuehai He, Jian Zheng, Jacob Zhiyuan Fang, Robinson Piramuthu, Mohit Bansal, Vicente Ordonez, Gunnar A Sigurdsson, Nanyun Peng, Xin Eric Wang. arXiv:2405.04834 May 2024. [arxiv]
Learning from Models and Data for Visual Grounding
Ruozhen He, Paola Cascante-Bonilla, Ziyan Yang, Alexander C. Berg, Vicente Ordonez. arXiv:2403.13804 March 2024. [project page] [arxiv]

Publications

NEW! PropTest: Automatic Property Testing for Improved Visual Programming
Jaywon Koo, Ziyan Yang, Paola Cascante-Bonilla, Baishakhi Ray, Vicente Ordonez. Conf. on Empirical Methods in Natural Language Processing. EMNLP 2024 (Findings). [project page] [arxiv]
NEW! Zero-Shot Controllable Image-to-Video Animation via Motion Decomposition
Shoubin Yu, Jacob Zhiyuan Fang, Skyler Zheng, Gunnar Sigurdsson, Vicente Ordonez, Robinson Piramuthu, Mohit Bansal. ACM Multimedia. MM 2024. Melbourne, Australia. [project page] [openreview]
NEW! ViC-MAE: Self-Supervised Representation Learning from Images and Video with Contrastive Masked Autoencoders
Jefferson Hernandez, Ruben Villegas, Vicente Ordonez European Conference on Computer Vision. ECCV 2024. Milan, Italy. [project page] [arxiv] [github]
NEW! Grounding Language Models for Visual Entity Recognition
Zilin Xiao, Ming Gong, Paola Cascante-Bonilla, Xingyao Zhang, Jie Wu, Vicente Ordonez. European Conference on Computer Vision. ECCV 2024. Milan, Italy. [github] [arxiv]
NEW! Improved Visual Grounding through Self-Consistent Explanations
Ruozhen He, Paola Cascante-Bonilla, Ziyan Yang, Alexander C. Berg, Vicente Ordonez Conf. on Computer Vision and Pattern Recognition. CVPR 2024. Seattle, WA. [project page] [arxiv]
NEW! ElasticDiffusion: Training-free Arbitrary Size Image Generation through Global-Local Content Separation
Moayed Haji Ali, Guha Balakrishnan, Vicente Ordonez Conf. on Computer Vision and Pattern Recognition. CVPR 2024. Seattle, WA. [project page] [arxiv] [code] [demo]
NEW! SCoRD: Subject-Conditional Relation Detection with Text-Augmented Data
Ziyan Yang, Kushal Kafle, Zhe Lin, Scott Cohen, Zhihong Ding, Vicente Ordonez. Winter Conference on Applications of Computer Vision WACV 2024. Waikoloa, HI. [arxiv] [code] [demo]
Variation of Gender Biases in Visual Recognition Models Before and After Finetuning
Jaspreet Ranjit, Tianlu Wang, Baishakhi Ray, Vicente Ordonez. Workshop on Algorithmic Fairness through the Lens of Time at NeuRIPS 2023. New Orleans, LA. [arxiv] [code]
Going Beyond Nouns With Vision & Language Models Using Synthetic Data
Paola Cascante-Bonilla, Khaled Shehada, James Seale Smith, Sivan Doveh, Donghyun Kim, Rameswar Panda, Gül Varol, Aude Oliva, Vicente Ordonez, Rogerio Feris, Leonid Karlinsky. International Conference on Computer Vision. ICCV 2023. Paris, France. [project page] [arxiv] [code]
Improving Visual Grounding by Encouraging Consistent Gradient-based Explanations
Ziyan Yang, Kushal Kafle, Franck Dernoncourt, Vicente Ordonez Conf. on Computer Vision and Pattern Recognition. CVPR 2023. Vancouver, Canada. [arxiv] [code] [demo]
Estimating and Maximizing Mutual Information for Knowledge Distillation
Aman Shrivastava, Yanjun Qi, Vicente Ordonez Workshop on Fair, Data Efficient and Trusted Computer Vision at CVPR 2023. Vancouver, Canada. [arxiv]
CLIP-Lite: Information Efficient Visual Representation Learning from Textual Annotations
Aman Shrivastava, Ramprasaath R. Selvaraju, Nikhil Naik, Vicente Ordonez International Conference on Artificial Intelligence and Statistics AISTATS 2023. Valencia, Spain (Hybrid). [arxiv]
On the Transferability of Visual Features in Generalized Zero-Shot Learning
Paola Cascante-Bonilla, Leonid Karlinsky, James Seale Smith, Yanjun Qi, Vicente Ordonez arXiv:2211.12494 November 2022. [arxiv] [github]
SimVQA: Exploring Simulated Environments for Visual Question Answering. Paola Cascante-Bonilla, Hui Wu, Letao Wang, Rogerio Feris, Vicente Ordonez Conf. on Computer Vision and Pattern Recognition CVPR 2022. [project page] [arxiv] [bibtex]
Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation. Samhita Honnavalli, Aesha Parekh, Lily Ou, Sophie Groenwold, Sharon Levy, Vicente Ordonez, William Yang Wang Language Resources and Evaluation Conference LREC 2022. [arxiv]
Backpropagation-Based Decoding for Multimodal Machine Translation
Ziyan Yang, Leticia Pinto-Alva, Franck Dernoncourt, Vicente Ordonez. Frontiers in Artificial Intelligence. January 2022. [link] [bibtex]
Evolving Image Compositions for Feature Representation Learning
Paola Cascante-Bonilla, Arshdeep Sekhon, Yanjun Qi, Vicente Ordonez. British Machine Vision Conference. BMVC 2021. November 2021. [project page] [arxiv] [bibtex]
Visual News : Benchmark and Challenges in Entity-aware Image Captioning
Fuxiao Liu, Yinghan Wang, Tianlu Wang, Vicente Ordonez. Empirical Methods in Natural Language Processing. EMNLP 2021. Virtual / Punta Cana, Dominican Republic. November 2021. [arxiv] [code] [bibtex] (~Oral presentation)
Instance-level Image Retrieval using Reranking Transformers
Fuwen Tan, Jiangbo Yuan, Vicente Ordonez.
International Conference on Computer Vision ICCV 2021. [arxiv]
MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning. Sonia Baee, Erfan Pakdamanian, Inki Kim, Lu Feng, Vicente Ordonez, Laura Barnes.
International Conference on Computer Vision ICCV 2021. [project page] [code] [arxiv]
General Multi-label Image Classification with Transformers
Jack Lanchantin, Tianlu Wang, Vicente Ordonez, Yanjun Qi. Conference on Computer Vision and Pattern Recognition CVPR 2021. [arxiv] [bibtex]
Black-box Explanation of Object Detectors via Saliency Maps
Vitali Petsiuk, Rajiv Jain, Varun Manjunatha, Vlad I. Morariu, Ashutosh Mehra, Vicente Ordonez, Kate Saenko.
Conference on Computer Vision and Pattern Recognition CVPR 2021. [arxiv] (~Oral presentation)
Curriculum Labeling: Revisiting Pseudo-Labeling for Semi-Supervised Learning
Paola Cascante-Bonilla, Fuwen Tan, Yanjun Qi, Vicente Ordonez.
The 35th AAAI Conference on Artificial Intelligence. AAAI 2021. February 2021 [arxiv] [code] [bibtex]
Enabling AI at the Edge with XNOR-Networks
Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, Ali Farhadi.
Communications of the ACM. December 2020 (Vol. 62, No. 12). (~Research Highlight)
[link] [bibtex]
Chair Segments: A Compact Benchmark for the Study of Object Segmentation
Leticia Pinto-Alva, Ian K. Torres, Rosangel Garcia, Ziyan Yang, Vicente Ordonez arxiv:2011.14027 Nov 2020. [project page] [code] [arxiv] [bibtex]
Using Visual Feature Space as a Pivot Across Languages
Ziyan Yang, Leticia Pinto-Alva, Franck Dernoncourt, Vicente Ordonez. Findings of the Association for Computational Linguistics: Findings of EMNLP 2020. [pdf] [project page] [code] [bibtex]
Double-Hard Debias: Tailoring Word Embeddings for Gender Bias Mitigation
Tianlu Wang, Xi Victoria Lin, Nazneen Fatema Rajani, Bryan McCann, Vicente Ordonez, Caiming Xiong. Association for Computational Linguistics. ACL 2020. Seattle, Washington. July 2020. [arxiv]
Generative-discriminative Feature Representations for Open-set Recognition
P. Perera, V. Morariu, R. Jain, V. Manjunatha, C. Wigington, V. Ordonez, and V. M. Patel. Conference on Computer Vision and Pattern Recognition CVPR 2020. [pdf] [bibtex]
Testing DNN Image Classifiers for Confusion & Bias Errors
Yuchi Tian, Ziyuan Zhong, Vicente Ordonez, Gail Kaiser, Baishakhi Ray.
International Conference on Software Engineering. ICSE 2020. Seoul, South Korea, October 2020. [arxiv] [bibtex]
Drill-down: Interactive Retrieval of Complex Scenes using Natural Language Queries
Fuwen Tan, Paola Cascante-Bonilla, Xiaoxiao Guo, Hui Wu, Song Feng, Vicente Ordonez. Conf. on Neural Information Processing Systems. NeurIPS 2019. Vancouver, Canada. December 2019. [arxiv] [code] [bibtex]
Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations
Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, Vicente Ordonez. International Conference on Computer Vision. ICCV 2019. Seoul, South Korea. October 2019. [arxiv] [project page] [code] [demo] [bibtex]
Text2Scene: Generating Compositional Scenes from Textual Descriptions
Fuwen Tan, Song Feng, Vicente Ordonez. Intl. Conference on Computer Vision and Pattern Recognition. CVPR 2019. Long Beach, California. June 2019. [arxiv] [code] [demo] [bibtex]
(~Oral presentation + Best Paper Finalist -- top 1% of submissions)
- IBM Research Blog Coverage
- NVIDIA News Coverage
Moviescope: Large-scale Analysis of Movies using Multiple Modalities
Paola Cascante-Bonilla, Kalpathy Sitaraman, Mengjia Luo, Vicente Ordonez.
arXiv:1908.03180. August 2019. [arxiv] [project page] [bibtex]
- TechXplore News Coverage
Gender Bias in Contextualized Word Embeddings
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, Kai-Wei Chang. North American Chapter of the Association for Computational Linguistics. NAACL 2019. short. Minneapolis, Minnesota. June 2019. [arxiv] [bibtex] (~Oral presentation)
Chat-crowd: A Dialog-based Platform for Visual Layout Composition
Paola Cascante-Bonilla, Xuwang Yin, Vicente Ordonez, Song Feng. North American Chapter of the Association for Computational Linguistics. NAACL 2019. System Demonstrations. Minneapolis, Minnesota. June 2019. [arxiv] [project page] [code]
Deep Feature Aggregation and Image Re-ranking with Heat Diffusion for Image Retrieval
Shanmin Pang, Jin Ma, Jianru Xue, Jihua Zhu, Vicente Ordonez.
IEEE Transactions on Multimedia 2019 (Journal). [Accepted October 2018].
[arxiv] [bibtex]
Feedback-prop: Convolutional Neural Network Inference under Partial Evidence
Tianlu Wang, Kota Yamaguchi, Vicente Ordonez. Intl. Conference on Computer Vision and Pattern Recognition. CVPR 2018. Salt Lake City, Utah. June 2018. [pdf] [project page] [arXiv] [code] [bibtex]
Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang.
North American Chapter of the Association for Computational Linguistics. NAACL 2018. short. New Orleans, Louisiana. June 2018. [pdf] [arXiv] [code] [bibtex]
Building Discriminative CNN Image Representations for Object Retrieval using the Replicator Equation
Shanmin Pang, Jihua Zhu, Jiaxing Wang, Vicente Ordonez, Jianru Xue.
Pattern Recognition 2018 (Journal). Volume 83. Pages 150-160.
[link] [code] [bibtex]
Where and Who? Automatic Semantic-Aware Person Composition
Fuwen Tan, Crispin Bernier, Benjamin Cohen, Vicente Ordonez, Connelly Barnes.
Winter Conference on Applications of Computer Vision. WACV 2018. Lake Tahoe, Nevada. March 2018.
[pdf] [arXiv] [supp. material] [code] [bibtex]
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang.
Empirical Methods in Natural Language Processing. EMNLP 2017. Copenhagen, Denmark. September 2017. [pdf] [code] [bibtex] (~Oral presentation + Best Long Paper Award!)
- WIRED News Coverage
- Daily Mail News Coverage
- Times of London News Coverage
Obj2Text: Generating Visually Descriptive Language from Object Layouts
Xuwang Yin, Vicente Ordonez. Empirical Methods in Natural Language Processing. EMNLP 2017. Copenhagen, Denmark. September 2017. [pdf] [arxiv] [code] [bibtex] (~Oral presentation)
Commonly Uncommon: Semantic Sparsity in Situation Recognition
Mark Yatskar, Vicente Ordonez, Luke Zettlemoyer, Ali Farhadi.
Intl. Conference on Computer Vision and Pattern Recognition. CVPR 2017. Honolulu, Hawaii. July 2017. [pdf] [arXiv] [bibtex] [demo]
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, Ali Farhadi.
European Conference on Computer Vision. ECCV 2016. Amsterdam, The Netherlands. October 2016. [arXiv] [project page] [code] [bibtex] (~Oral presentation)
- New York Times News Coverage
- Article on University of Washington News
Stating the Obvious: Extracting Visual Common Sense Knowledge
Mark Yatskar, Vicente Ordonez, Ali Farhadi. North American Chapter of the Association for Computational Linguistics. NAACL 2016. short. San Diego, CA. June 2016. [pdf] [bibtex] (~Oral presentation)
Learning to Name Objects
Vicente Ordonez, Wei Liu, Jia Deng, Yejin Choi, Alexander C. Berg, Tamara L. Berg.
Communications of the ACM. March 2016 (Vol. 59, No. 3). (~Research Highlight)
[pdf] [link] [technical perspective] [bibtex]
Predicting Entry-Level Categories
Vicente Ordonez, Wei Liu, Jia Deng, Yejin Choi, Alexander C. Berg, Tamara L. Berg.
International Journal of Computer Vision - Marr Prize Special Issue. IJCV 2015.
[pdf] [link] [bibtex]
Large Scale Retrieval and Generation of Image Descriptions
V. Ordonez, X. Han, P. Kuznetsova, G. Kulkarni, M. Mitchell, K. Yamaguchi, K. Stratos,
A. Goyal, J. Dodge, A. Mensch, H. Daume III, A.C. Berg, Y. Choi, T.L. Berg.
International Journal of Computer Vision. IJCV 2015. [August 2016 Issue]. [pdf] [link] [bibtex]
Ph.D. Thesis. [pdf] [bibtex]
Language and Perceptual Categorization in Computational Visual Recognition.
Vicente Ordóñez Román. April 2015.
Department of Computer Science. The University of North Carolina at Chapel Hill.
ReferItGame: Referring to Objects in Photographs of Natural Scenes
Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, Tamara L. Berg.
Empirical Methods on Natural Language Processing. EMNLP 2014. Doha, Qatar. October 2014. [pdf] [project page] [game] [bibtex] (~Oral presentation)
Learning High-level Judgments of Urban Perception
Vicente Ordonez, Tamara L. Berg.
European Conference on Computer Vision. ECCV 2014. Zurich, Switzerland. September 2014. [pdf] [project page] [bibtex]
TreeTalk: Composition and Compression of Trees for Image Descriptions
Polina Kuznetsova, Vicente Ordonez, Tamara L. Berg, Yejin Choi.
Transactions of the Association for Computational Linguistics. TACL 2014.
To be presented at EMNLP 2014 in Doha, Qatar. October 2014. [pdf] [bibtex]
Furniture-Geek: Understanding Fine-Grained Furniture Attributes from Freely Associated Text and Tags
Vicente Ordonez, Vignesh Jagadeesh, Wei Di, Anurag Bhardwaj, Robinson Piramuthu. IEEE Winter Conference on Applications of Computer Vision. WACV 2014. Steamboat Springs, CO. March 2014. [pdf] [bibtex]
From Large Scale Image Categorization to Entry-Level Categories
Vicente Ordonez, Jia Deng, Yejin Choi, Alexander C. Berg, Tamara L. Berg.
IEEE International Conference on Computer Vision. ICCV 2013. Sydney, Australia. December 2013. [pdf] [supplemental material] [slides] [project page] [bibtex] (~Oral Presentation + Best Paper Award - Marr Prize!)
Generalizing Image Captions for Image-Text Parallel Corpus
Polina Kuznetsova, Vicente Ordonez, Alexander C. Berg, Tamara L. Berg, Yejin Choi.
Association for Computational Linguistics. ACL 2013. short. Sofia, Bulgaria. August 2013. [pdf] [data+results] [bibtex]
Baby Talk: Understanding and Generating Simple Image Descriptions
G. Kulkarni, V. Premraj, V. Ordonez, S. Dhar, S. Li, Y. Choi, A. C. Berg, T. L. Berg.
IEEE Transactions on Pattern Analysis and Machine Intelligence. PAMI 2013
[pdf] [link] [bibtex]
Collective Generation of Natural Image Descriptions
Polina Kuznetsova, Vicente Ordonez, Alexander C. Berg, Tamara L. Berg, Yejin Choi.
Association for Computational Linguistics. ACL 2012. Jeju, South Korea. July 2012.
[pdf] [data] [bibtex] (~Oral presentation)
Im2Text: Describing Images Using 1 Million Captioned Photographs
Vicente Ordonez, Girish Kulkarni, Tamara L. Berg.
Conf. in Neural Information Processing Systems. NeurIPS 2011. Granada, Spain. December 2011. [pdf] [code+dataset] [poster] [search tool] [bibtex] (~Spotlight presentation)
High Level Describable Attributes for Predicting Aesthetics and Interestingness
Sagnik Dhar, Vicente Ordonez, Tamara L. Berg.
IEEE Computer Vision and Pattern Recognition. CVPR 2011. Colorado Springs, CO. June 2011. [pdf] [related code for saliency + low DoF attributes] [bibtex]
The Ariadne Infrastructure for Managing and Storing Metadata
S. Ternier, G. Parra, B. Vandeputte, K. Verbert, J. Klerkx, E. Duval, V. Ordonez, X. Ochoa. IEEE Internet Computing 2009 . Emerging Internet Technologies and Applications for E-learning. [link]

CURRENT AND PAST SPONSORS