Oct 2024 – Jul 2025 · 9 mos · Dubai, United Arab Emirates · On-site

• Worked on developing recipes for efficient pre-training. Tech stack involved PyTorch lightning, Ray and Kubernetes.
• Set up job monitoring and evaluations pipelines based on Ray and Kubernetes.

Aug 2023 – Oct 2024 · 1 yr 2 mos · Abu Dhabi Emirate, United Arab Emirates · On-site

• Led the architecture design and distributed training of Falcon3-7B. I performed architecture sweeps and ensured the model scaled across 512 H100 GPUs, running stably for over two months. I also updated the Triton kernels to support H100 GPUs and enabled GQA with TP-agnostic checkpointing.
• Led the distributed training of Falcon2-11B on 1024 A100 GPUs. Main responsibilities were to integrate FA2 for higher throughput and setting up alerts for run maintenance.
• Wrote a distributed implementation of the Sophia optimizer in CUDA C++.
• Enabled access to Falcon models to stakeholders via multiple endpoints.

Oct 2022 – Aug 2023 · 10 mos · Abu Dhabi Emirate, United Arab Emirates

Aug 2021 – Aug 2022 · 1 yr · Sydney, New South Wales, Australia · Remote

• Built end-to-end pipeline for training Transformer/BERT based HTML DOM parser.
• Trained and deployed hierarchical e-commerce product categorisation model.

Oct 2018 – Jul 2021 · 2 yrs 9 mos · Munich Area, Germany

• Developed the Varia Engine which comprised of training Transformer-based DL models for NER, sentence-level sentiment, news category classification, and vector representation of news articles.
• The models were converted to run on the ONNX runtime and deployed in production as docker containers.
• Came up with an algorithm to compute news article relevance without using any user data.
• Added German language support for all the existing DL models (by leveraging XLM-Roberta).
• Implemented logic for recommending news articles for a collection of documents.
• Implemented a simple mechanism to generate an extractive summary of a document.

Jan 2018 – Sep 2018 · 8 mos · Bremen, Bremen, Germany

• Improved the Q&A system that was used both by the web-based chatbot and by the personality installed in the Pepper robots by coding and training a Key-Value memory network in PyTorch that uses a shared LSTM encoder for both the questions and the answers leading to a recall rate of 80%.
• Coded and trained a Neural Question Generation model to help human annotators to come up with questions for a given paragraph of text.

Oct 2017 – Dec 2017 · 2 mos · Bremen

• I helped the professor to evaluate and grade the assignments that were given out throughout the course. I also offered tutorial sessions where students clarified their doubts on both the theory and the practical implementations along with familiarizing themselves with version control using Git.

Jan 2017 – Aug 2017 · 7 mos · San Francisco Bay Area

• Coded and trained an LSTM based siamese network on the Quora question pairs dataset using PyTorch.
• Trained the pointer-generator network on the SQuAD dataset.
• Implemented a Flask-based RESTful service for question answering.
• Implemented a neural ranked retrieval model using the MS Marco dataset. The chosen model was a GRU-based siamese network.
• Worked on some novel ideas to generate contextualized word embeddings for OOV tokens.
• Leveraged neural_qa for performing multistep reasoning for Q&A on HTML tables.

Aug 2015 – Nov 2015 · 3 mos · Bengaluru Area, India

• Visualized job change and hiring patterns across various domains by writing Python scripts that read data from a MongoDB collection and uses NumPy to analyze and matplotlib to visualize the data.
• Worked with the DataOps team to develop simple NLP methods to clean and disambiguate data collected by humans, automatically.
• Built an email classifier using BOW features to predict if a candidate was interested or not interested in a particular job offering. I was responsible for collecting the data by querying multiple SQL tables, cleaning the data to get rid of bad examples and outliers, training and validating a Logistic Regression model, and then deploying it to production as a RESTful service by using Flask.