Oct 2024 – Present · 1 yr 5 mos · United Arab Emirates · On-site

• 1. Contributed to PetalGuard (link: https://petalguard.tii.ae/), a federated learning framework, where I optimized server–client communication using gRPC.
• 2. Worked with Large Language Models (LLMs) including the Falcon and LLaMA series, as well as other models.
• 3. Built and integrated ML components across the full lifecycle -- design, development, deployment, and maintenance of ML workflows.
• 4. Added support for large scale distributed LLM training using DeepSpeed, improving scalability and training efficiency. Deepspeed is a deep learning optimization library that enables faster, more memory-efficient, and highly scalable training for large models (billions or even trillions of parameters).

Jun 2023 – Oct 2024 · 1 yr 4 mos · United Arab Emirates · On-site

Jun 2022 – Jun 2023 · 1 yr · United Arab Emirates · On-site

• 1. Deep Learning:
• ● Worked with various convolutional neural network (CNN) architectures like GoogLeNet, ResNet-56, ResNet-110, VGG-16, DenseNet-40, and EfficientNets using a ranking based pruning technique.
• ● Implemented and trained an efficient CNN having more than 86 % accuracy with just 1.26 million parameters, and 0.13 GMAC operations.
• ● Worked with various language based transformer architectures like BERT, Distilled-BERT, GPT-3.5, and LLaMA.
• ● Worked with various vision transformer architectures like T2T (Tokens-to-Token) ViT and many more.
• ● Trained Distilled-BERT with a large language dataset in a decentralized (federated) manner using Flower (a federated learning framework).
• ● Implemented various research papers.
• ● Datasets used and worked with are CIFAR-10, CIFAR-100, various large scale language datasets, ImageNet-1K, Caltech-256, etc.
• ● Training for above models was carried out on NVIDIA (A100) DGX station.
• 2. Computational neuroscience, neuromorphic computing, and spiking neural networks (SNN):
• ● Implemented a low latency VGG-16, transformed to SNN using LIF (leaky integrate-and-fire) neurons, and simulated using various SNN simulators .
• ● Implemented various variants of ResNet (18, 34, 50, 101, 152) using PyTorch, transformed into SNN, and simulated using SpikingJelly.
• ● Implemented low latency spiking VGG-16 using SpikingRectifiedLinear on Loihi. On-chip training and inference was performed with and without quantization. With further optimization reduced the number of blocks from 9633 to 5438, without compromising performance.
• ● Obtained the energy consumption metircs for spiking VGG-16 and a 5-layered spiking CNN for SpiNNaker, SpiNNaker-2, and Loihi.
• ● Worked on implementing a SNN on BrainScaleS using PyNN and hxtorch, and further worked on implementing it on SpiNNaker.
• Skills: Computer vision, NLP, Deep learning, Docker, Python, Pytorch, Large language models (LLMS), Transformer architectures.