Apr 2024 – Present · 2 yrs 2 mos

• Developed ETL pipelines for pharmaceutical data using PySpark jobs on Kubernetes and AWS Redshift Spectrum. Streamlined multi-source data flow through a Landing–Staging–Curated architecture, enabling downstream analytics for Power BI dashboards.
• Improved sales data quality through accurate mapping of ˜ $250M in previously unmapped sales by automating product mapping in AWS Redshift using Python and ML-based text matching. Deployed the solution via an AWS EC2 and S3 workflow.
• Worked on data pipeline for Power Outage analytics using multiple Azure Cloud services such as Synapse, Eventhub and Functions to develop end-to-end Data pipeline. Optimized Data Streaming and Processing speeds by reducing the data transformation time by 85% from 2hrs to 20 mins.
• Integrated OpenAI GPT-4 model for code context generation using Langchain Custom Prompt Template and utilized inner product(IP) similarity search to retrieve technical context from Milvus vector DB.
• R&D on different Transformer models and architectures such as T5, BART, Flan-T5 for tasks such as question-answer generation, Translation, Summarization and documented the results and key metrics.
• Developed end-to-end ML pipeline on Azure ML Studio to predict power outages for 300 cities in US using real-time weather forecast data and XGBoost model with an accuracy score of 89.7%
• Migrated and Transformed 10 TB of Data from different source systems such as Salesforce, SAP, Bigtable into a structured format using GCP Bigquery, GCS, Pandas, Pyspark, and SQL

Sep 2022 – Mar 2024 · 1 yr 6 mos

• Experienced in Python development including Flask,FastAPI, Machine Learning, PySpark, SQL and Python implementation of AWS services.
• Developed Python Backend services using FastAPI framework and AWS DynamoDB for CRUD operations of Custom Prompts. Achieved code coverage of 90% utilising Pytest and Mockito libraries.
• Focused on Python application dev, Video and Data Processing, Training Machine learning models.
• Handled Critical Tasks of Video chunk processing and JSON data generation.
• Implemented Multi-Processing of backend engine along with Continuous Ping service to enable higher level of flexibility and control from Frontend and Microservices.
• Deployed and Integrated core engine with AWS services such as EC2,S3,SNS etc.

Mar 2022 – Aug 2022 · 5 mos

• Developed an application to detect driver distraction and drowsiness and alert the driver accordingly
• Worked on different POC's for detecting driver distraction using Deep learning models, Computer Vision, Pose estimation.

Apr 2021 – Sep 2021 · 5 mos

• Worked on identifying the influence of social media emotion from Twitter and Reddit on the stock price of Gamestop.
• Developed data extraction and transformation script to create a structured dataset using Python, Pandas, NLTK, Twitter API and Reddit
• API. Performed sentiment and emotion analysis on the textual data using text2emotion and NRCLex packages.
• Visualised the intensity of the different emotions per day using Matplotlib and Plotly visualization libraries.