Jan 2024 – May 2025 · 1 yr 4 mos · Gurugram, Haryana, India · Hybrid

• 1. Designed and implemented ETL pipelines to extract data from diverse sources including data lake tables and parquet files. Processed the data and stored it into Unified Feature Store (UFS) integrated with MongoDB, ensuring optimised data accessibility.
• 2. Collaborated with Machine Learning Scientists (MLS) to facilitate seamless deployment of their models into production environments using Model Registry Service (MRS) and Model Deployment Service (MDS).
• 3. Utilising Apache Airflow for scheduling and monitoring the job workflows by developing the DAGs in Python.
• 4. Optimized a Spark Streaming application, reducing average latency from 45 seconds to 10 seconds by optimizing state management and refining stream processing logic, leading to improved performance and faster data processing capabilities.

Apr 2022 – Jan 2024 · 1 yr 9 mos · Noida, Uttar Pradesh, India

• 1. Designed & Developed multiple batch data pipelines using Spark DataFrame API to process 10 millions Ad-Serving records from
• Kafka on an hourly basis and storing them into HDFS
• 2. Generating multiple performance metrics by aggregating the data on various Ad-Attributes & storing data into the Cassandra Tables which is used by Ad-Engine to update its learning & performance caches
• 3. Integrated Data Lake's extensive dataset with Trino, enabling seamless ad hoc querying, in-depth analysis, and effective debugging of Ad-Serving data.
• 4. Utilized Apache Airflow for scheduling and monitoring the job workflows by developing the DAGs in Python.

Jul 2021 – Apr 2022 · 9 mos · Bengaluru, Karnataka, India

• 1. Developed an end-to-end Azure data engineering pipeline to ingest, process, and transform data for advanced analytics.
• 2. Utilized Azure Data Factory (ADF) for orchestrating data ingestion from Azure SQL DB and CSV files into Azure Data Lake Storage (ADLS) Gen2.
• 3. Applied Medallion Architecture (Bronze, Silver, Gold layers) and used Azure Databricks for data cleansing, deduplication, and standardisation.

Aug 2018 – Jul 2021 · 2 yrs 11 mos · Noida

• 1. Worked on scalable PySpark data ingestion pipelines to ingest and process
• Samsung’s device telemetry and app usage analytics into the data lake,
• enabling efficient downstream analysis and reporting.
• 2. Developed a scalable internal task management web application using PHP,
• Bootstrap, and MySQL, improving operational efficiency and cross-team
• collaboration across Samsung departments.