Jyoti Aneja

Member of Technical Staff
Microsoft Research AI Frontiers

Research Interest: Multimodal Reasoning, Computer Vision, NLP

Education
PhD, University of Illinois at Urbana-Champaign, 2021 Advisor: Alex Schwing
MS in Physics, University of Illinois at Urbana-Champaign, 2015
MS in Physics, Indian Institute of Technology, Kanpur, 2012

I am a Member of Technical Staff on the Microsoft Research AI Frontiers team. I co-led Phi-4-reasoning-vision, a 15B multimodal reasoning model, and was a core contributor to Phi-4, Phi-3, Phi-2, and Phi-1.

I graduated with a PhD from the University of Illinois at Urbana-Champaign where I worked with Alex Schwing and also closely collaborated with David Forsyth.

In my (not so) past life, I was a physicist. I have a MS in Physics from UIUC.

If you're interested in working with me, get in touch via email.

Preprints & Publications

	Phi-4-reasoning-vision-15B Technical Report Jyoti Aneja, Michael Harrison, Neel Joshi, Tyler LaBonte, John Langford, Eduardo Salinas Microsoft Research Blog and Tech Report, March 2026 pdf Hugging Face
	Phi-4 Technical Report Marah Abdin, Jyoti Aneja, Harkirat Behl, 24 more authors arXiv 2024 pdf 3,300,000+ 2,100,000+
	Phi-3: A Highly Capable Language Model Locally on Your Phone Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, et al. arXiv 2024 pdf 27,000,000+
	Phi-2: The Surprising Power of Small Language Models Marah Abdin, Jyoti Aneja, Sebastien Bubeck, 23 more authors Microsoft Research Blog 2023 blog post 7,700,000+ Scientific American
	Textbooks Are All You Need Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, Yuanzhi Li arXiv 2023 pdf 132,000+

Explaining Patterns in Data with Language Models via Interpretable Autoprompting
Chandan Singh, John X. Morris, Jyoti Aneja, Alexander M. Rush, Jianfeng Gao
under submission
pdf

ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models
Chunyuan Li, Haotian Liu, Liunian Harold Li, Pengchuan Zhang, Jyoti Aneja, Jianwei Yang, Ping Jin, Houdong Hu, Zicheng Liu, Yong Jae Lee, Jianfeng Gao
Conference on Neural Information Processing Systems (NeurIPS) (Datasets and Benchmarks Track), 2022
pdf

A Contrastive Learning Approach for Training Variational Autoencoder Priors
Jyoti Aneja, Alexander Schwing, Jan Kautz, Arash Vahdat
Conference on Neural Information Processing Systems (NeurIPS), 2021
pdf

	Image Captioning Diversity under the Radar Xiaoming Zhao, Jyoti Aneja, Harsh Agrawal, Alexander Schwing Under Submission pdf
	Sequential Latent Spaces for Modeling the Intention During Diverse Image Captioning Jyoti Aneja, Harsh Agrawal, Dhruv Batra, Alexander Schwing International Conference on Computer Vision (ICCV), 2019 pdf
	Fast, Diverse and Accurate Image Captioning Guided By Part-of-Speech Aditya Deshpande, Jyoti Aneja*, Liwei Wang, Alexander Schwing, David Forsyth Computer Vision and Pattern Recognition (CVPR), 2019 (Oral presentation)* pdf
	Convolutional Image Captioning Jyoti Aneja, Aditya Deshpande, Alexander Schwing Computer Vision and Pattern Recognition (CVPR), 2018 video(CSL Student Conference-2018) \| pdf \| code

Affiliations

News

Mar 2026	Co-led Phi-4-reasoning-vision, a 15B multimodal reasoning model. Featured in Forbes.
Dec 2024	Core contributor to Phi-4, a 14B language model surpassing GPT-4 on STEM benchmarks.
Apr 2024	Phi models featured in the New York Times, Wired, and Wall Street Journal.
Apr 2024	Phi-3 Technical Report released.
Dec 2023	Phi-2 released on Microsoft Research Blog.
June 2023	Textbooks Are All You Need (Phi-1) released.
Oct 2022	Paper accepted at NeurIPS 2022 (Datasets and Benchmarks Track).
Sept 2021	Paper accepted at NeurIPS 2021. Successfully defended my PhD thesis!
May 2021	Recognized as an "outstanding reviewer" at CVPR 2021.

Template credits: Deepak, Jon, Saurabh, Unnat, Abhishek and Jeff