Aug 2022 – Present · 3 yrs 7 mos · Menlo Park, California, United States · On-site

• Working with Facebook AI Research (FAIR) on research on Large scale multimodal foundational models on trillion scale datasets. Particularly interested in Generative AI research and production use cases and exploring the realm of Self supervised learning.
• Published list of papers:
• DINOv2: Learning Robust Visual Features without Supervision (Published at TMLR ) https://dinov2.metademolab.com/
• Chameleon: Mixed-Modal Early-Fusion Foundation Models (https://about.fb.com/news/2024/06/releasing-new-ai-research-models-to-accelerate-innovation-at-scale/)
• Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning (CM3Leon) (https://ai.meta.com/blog/generative-ai-text-images-cm3leon/)
• Demystifying CLIP Data (MetaCLIP) (Published at ICLR 2024): (https://github.com/facebookresearch/MetaCLIP)
• Mavil: Masked audio-video learners (Published at NeurIPS 2023): (https://ar5iv.labs.arxiv.org/html/2212.08071)
• A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions (Published at CVPR 2024): (https://openaccess.thecvf.com/content/CVPR2024/papers/Urbanek_A_Picture_is_Worth_More_Than_77_Text_Tokens_Evaluating_CVPR_2024_paper.pdf)
• Seamless Interaction (https://ai.meta.com/research/publications/seamless-interaction-dyadic-audiovisual-motion-modeling-and-large-scale-dataset/)
• FLAP: Fast Language-Audio Pre-training (Published at ASRU 2023) (https://arxiv.org/abs/2311.01615)
• An Introduction to Vision-Language Modeling (https://arxiv.org/abs/2405.17247)
• Text Quality-Based Pruning for Efficient Training of Language Models (https://arxiv.org/pdf/2405.01582) and DMLR 2025
• Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM (Published at COLM 2024) (https://arxiv.org/abs/2403.07816)

Jan 2022 – Present · 4 yrs 2 mos

• Advised multiple startups and VCs on technical and AI strategies for scaling from ideation to series A/B/C stages.
• Designed AI infrastructure and built AI native products to drive customer acquisition and growth.
• Facilitated connections with VCs and assisted in hiring top talent to support company expansion.

Aug 2021 – Aug 2024 · 3 yrs

Jan 2019 – Aug 2021 · 2 yrs 7 mos

May 2018 – Aug 2018 · 3 mos · Greater Chicago Area

• Working on using Machine Learning/ Deep Learning techniques to better model Financial Time series data and ensure scalability of the algorithms to arbitrary number of input features

Aug 2021 – Aug 2022 · 1 yr · Sunnyvale, California, United States · On-site

• Working with Alexa AI on a variety of problems like:
• A dialog enabled visual-language navigation bot leveraging the multimodal data sources to faithfully navigate a virtual environment based on user instruction. Created a new benchmark for visual language navigation as a part of the Alexa Prize Simbot challenge and designed benchmark models for the same
• Designing efficient multimodal transformers to speed up their training and deployment by improving the computational complexity of the self attention mechanism
• Video processing applications like video action recognition, video question answering, video summarization, moment retrieval etc working directly with compressed video streams
• Created a benchmark for cooperative heterogenous multi agent reinforcement learning platform including open sourcing the collected dataset and it's benchmark models
• Working on creating a massively multimodal transformer pipeline capable of handling a wide range of input modalities with modality agnostic transformer blocks which work well for a several tasks leveraging a multitude of modalities
• Published list of papers:
• Alexa arena: A user-centric interactive platform for embodied ai (Published at Neurips 2023) (https://www.amazon.science/publications/alexa-arena-a-user-centric-interactive-platform-for-embodied-ai)
• Alexa, play with robot: Introducing the first Alexa Prize SimBot Challenge on embodied AI (https://www.amazon.science/alexa-prize/proceedings/alexa-play-with-robot-introducing-the-first-alexa-prize-simbot-challenge-on-embodied-ai)
• CHMARL: A Multimodal Benchmark for Cooperative, Heterogeneous Multi-Agent Reinforcement Learning (Published at RSS 2022) (https://www.amazon.science/publications/chmarl-a-multimodal-benchmark-for-cooperative-heterogeneous-multi-agent-reinforcement-learning)
• ε-ViLM: Efficient Video-Language Model via Masked Video Modeling with Semantic Vector-Quantized Tokenizer (Published at WACV 2024) (https://arxiv.org/abs/2311.17267)

Aug 2018 – Aug 2019 · 1 yr · Multicomp Lab, Language Technologies Institute, School of Computer Science

• I worked with Prof. Loius Phillipe Morency on a wide range of project related to Multi Modal Machine Learning and on building robust, explainable Deep Learning models
• We designed adversarial attack mechanisms on Visual Question Answering models to identify their vulnerabilities and then fix the same by proposing a variety of robust training mechanisms.
• We worked on developing a neural network model which uses a Deep Convolutional Neural Network based pipeline alongside a geometrically conditioned point distribution model for Facial Landmark Detection.
• We alsondeveloped the first fully ecologically validated models of visual perception. We will combine intracranial EEG (iEEG)recordings captured during long stretches of natural visual behavior with cutting-edge computer vision, machine learning, and statistical analyses to understand the neural basis of natural, real-world visual perception.
• We also explored facial expression recognition in extreme face scenarios like profile face views, occluded faces, non centric and rotated faces alongside recognition for gender, age and racially diverse faces.

Aug 2017 – May 2018 · 9 mos · Articulab, Language Technologies Institute, School of Computer Science

• I worked on the SARA and the Yahoo! InMind projects at the ArticuLab which focus on building a socially aware robotic assistant. My primary focus was on trying to combine the user’s visual, vocal and verbal cues to better gauge the ‘rapport’ between the user and the conversational agent and using it to enable the agent to become socially more aware to the user’s emotional needs.

May 2017 – May 2018 · 1 yr · Los Angeles Metropolitan Area · Remote

• Founding team at Localite Inc - a tours and activities marketplace for connecting people with local tour agencies and local tour guides. Raised $200k in pre-seed funding.
• Implemented the core recommendation systems, feed ranking and search retrieval systems.

May 2017 – Jul 2017 · 2 mos · Lausanne, Vaud, Switzerland · On-site

• Worked on learning unsupervised document embeddings using Continuous Bag of Words model implemented in a Deep Convolutional Neural Network framework and using transfer learning techniques to make cross domain use of these generalized embeddings to demonstrate improved performance on a wide array of tasks like similarity matching, sentiment analysis etc.

Aug 2016 – Jul 2017 · 11 mos · Milpitas, California, United States · Remote

• > Worked on building "A Smart E-commerce Virtual Assistant"
• > Implemented features like cloth parsing from images, similar image retrieval from a huge fashion catalogue, a state of the art Deep Recommender system and a multi turn conversational voice agent to facilitate user interaction
• > "Query based document retrieval" by learning rich semantic document embeddings using a deep LSTM pipeline and using these to find the match the queries to relevant documents
• > "Abstractive summarization using Attention based encoder-decoder networks": Worked on building a deep residual LSTM pipeline which used temporal attention over both encoder and decoder networks to generate an abstractive summary of documents.

May 2016 – Jul 2016 · 2 mos · Greater Toronto Area, Canada · On-site

• Research intern with the Computer Vision and Machine Learning group with Raquel Urtasun and Sanja Fidler in Geofffery Hinton's lab.
• Worked on the problem of Instance and semantic segmentation from videos with direct applications in Autonomous driving and video surveillance.
• Implemented a 2 stream network to combine base Segmentation masks generated by Deep Convolutional -
• Deconvolutional Neural Networks and optical flow information obtained via implementing FlowNet based on Deep CNN's to achieve improved performance on video semantic segmentation task.

Jan 2015 – Jan 2015 · 0 mo

• Worked as a research intern with the Computer Vision team at Xerox Research Europe, working on building deep learning frameworks for large scale object recognition.

Jan 2015 – Jan 2015 · 0 mo

• Worked with the Speech and Signal processing team at XRCI to create Deep Neural Network based Speech Recognition systems.
• Worked on 3 projects during this internship: “Application of Deep Learning for Automatic Speech
• Recognition”, “ A comprehensive analysis of Activation Functions in Deep Nets” and “A new
• hashing technique to enhance Deep Net performance ”. Also got the Best Project award for the same.
• The projects primarily focused on constructing Deep Learning frameworks for Speech Recognition. -The Internship provided me extensive research and coding experience of how to efficiently train Deep Nets.

May 2014 – Jul 2014 · 2 mos · Pittsburgh, Pennsylvania, United States · On-site

• Worked on exploring Applications of Deep Learning to Audio and Speech Signal Processing, particularly exploring the use of Gated Recurrent Neural Network for denoising speech signals

Dec 2013 – Dec 2013 · 0 mo

• Completed 2 projects:
• 1.Analyzing Newspaper Crime Reports For Identification Of Safe Transit Paths
• 2.Automatic Image Summarization using Topic Modelling