Dec 2023 – Apr 2025 · 1 yr 4 mos · India · Remote

• Designed and developed real-time and batch Data/ETL pipelines for People Analytics and Experience vertical using SQL, Python, Spark, Airflow, AWS (S3 and Kinesis), DBT, and Databricks, incorporating practices such as SCD.
• Handled PCI/PII data by encrypting required dimensions and attributes and securely storing the data in a protected bucket.
• Migrated real-time and batch processing pipelines from a legacy system to Atlassian’s self-service data platform, utilizing Spark, SQL, DBT, Databricks, and AWS Kinesis.
• Developed a custom data quality check framework to validate key fields in over 100 datasets, ensuring accurate ingestion from source data streams and producers.
• Created a custom real-time stream ingestion framework in collaboration with the data platform team, using Structured Spark Streaming with AWS Kinesis, Python/Scala, and YAML on Databricks clusters to support corporate data engineering business units.
• Optimized data processing workflows, reducing data pipeline runtime by 30% and compute cost by 15%.

Mar 2022 – Sep 2023 · 1 yr 6 mos · Gurugram, Haryana, India

• Designed and developed Data/ETL pipelines and workflows for BI & Product Analytics stakeholders, processing 4-6 TB of Clickstream/events data daily, using SQL, Python/Scala, Spark, AWS, Airflow, Qubole, Trino, and Databricks to support advanced analytics and reporting.
• Designed and developed Funnel Progression Analysis datasets, enabling data science and leadership teams to monitor page-level conversion rates for all LOBs across all brands and identify business chokepoints and facilitate data-driven decision-making for product improvement.
• Led the migration of ETL pipelines from Qubole Presto workflows to SQL, Spark, and Trino using Python/Databricks scripts and Airflow.
• Achieved a 35% reduction in ETL processing time, which included tool evaluation and SQL query optimization.
• Created a critical data pipeline that ingests analytics dataset and computes executive-level summaries shared with leadership, including the
• CEO and CTO. Utilized SQL, Python, Airflow, and in-house deployment tools like Jenkins, Spinnaker, S3, EMR, RDS, and APIs.
• Designed and implemented a Product Data Monitoring BI dashboard using SQL, Python, SparkSQL, PySpark, and Tableau. This reduced metric readiness time by 55% while identifying data anomalies and monitoring metric status (delayed, missing, restatement).

Aug 2021 – Mar 2022 · 7 mos · Chennai, Tamil Nadu, India

• Designed and developed Data/ETL pipelines and workflows for BI & Product Analytics stakeholders, processing 20 TB of transactions data daily, using SQL, Python, Spark, AWS/GCP, Airflow to support credit risk and analysis and reporting.
• Developed OLAP dimensional models and pipelines, handling extensive datasets from the data warehouse. This included crafting measures, dimensions, and critical KPIs and aggregations across various datasets.
• Saved stakeholders over 40 man-hours per month by transitioning from manual analysis to a standardized and automated analytical BI infrastructure. This transformation was achieved through the implementation of Spark, SQL, and BigQuery.

Apr 2019 – Jul 2021 · 2 yrs 3 mos · Pune, Maharashtra, India

• Led a 3-member team in developing ETL pipelines and BI applications to deliver data-driven narratives and insights to pharmaceutical brand teams. This supported field sales analysis and biosimilar impact readiness analysis, employing SQL, AWS, Spark, and Python.
• Assisted a crucial client by enhancing the performance of a critical BI application, resulting in a 65% improvement. This was achieved through
• data model optimization and simplification of calculations and visualizations, effectively avoiding an additional infrastructure cost of $250K.
• Successfully secured and delivered a $200K project focused on automating Oncology brand trackers. This achievement was accomplished by showcasing the essential features and functionalities of Qlik Nprinting, supplemented by POCs for Brand Performance analysis.

Aug 2017 – Apr 2019 · 1 yr 8 mos · Pune, Maharashtra, India

• Created OLAP dimensional models utilizing extensive datasets from a relational data warehouse for development of measures and KPIs.
• Implemented a comprehensive executive scorecard BI solution for leadership, receiving high praise from clients. This accomplishment led
• to the onboarding of new projects from other business units, resulting in an additional revenue of $500K.
• Led an initiative to implement a standalone RPA solution using Talend Studio and the SOAP protocol for OLTP updates, resulting in a monthly reduction of 30 man-hours in daily operations.

Feb 2016 – Jun 2016 · 4 mos · Hyderabad, Telangana, India

• Developed enterprise-level Human Resources Information System with client-level customization using ODOO ERP, Python, and XML.
• Delivered initial stage demo of the product to several clients.