Jul 2025 – Present · 10 mos

• ● Founding Engineer, Data Cloud — responsible for building simple, scalable, and reliable data infrastructure for AI and analytics workloads, with ownership across hiring, architecture, and product definition.

Aug 2023 – Jun 2025 · 1 yr 10 mos · Mumbai, Maharashtra, India

• ● Designed and executed the transition to a headless data architecture using Apache Iceberg, Kafka, and Flink. Led a cross-functional team of 15 engineers to deliver a resilient platform handling 9PB of data, managing a 24k table footprint with 8k active batch and real-time processing workflows
• ● Replaced Redshift with an open compute platform(Trino) on Kubernetes to optimize resource consumption, reducing costs by 25% through elastic scaling. It runs almost 100k queries daily, on frugal compute while maintaining FTE, Performance SLA’s.
• ● Built Pelican gateway for in-house Spark platform with 3 engineers. Implemented authentication, routing, load balancing, and state management with Livy integration for automated deployments and metadata storage.
• ● Actively participating in the open-source data systems community and promoting knowledge-sharing standards within the team.

Jul 2021 – Aug 2023 · 2 yrs 1 mo · Mumbai, Maharashtra, India

• ● Built an in-house streaming platform (Spring, Mysql, Redis, Flink, Kafka) to make streams accessible to everyone without having prior knowledge of stream processing. Reduced real-time pipeline creation by 50% with pre-built operators & 1-click deployment for streams.
• ● Led ELK to Flink/OpenSearch migration with 3 engineers, supporting real-time analytics for Marketing/Product/Revenue teams. Delivered sub-100ms search performance and 50% cost reduction across 98 streams processing 8TB weekly.
• ● Developed a collaborative, drag-and-drop ETL platform (Spring, Mysql, Airflow, Trino) for data warehouses & lakehouses with a team of 5, empowering data analysts/engineers/scientists & eliminating 100% of manual involvement in the data model or feature creation.
• ● Processed TBs of data via Batch + Real Time stateful Jobs to power Reward Shop (on dream11 app can be seen as DreamCoins) with Flink, Spark, S3 and MySQL.

Sep 2018 – Jun 2021 · 2 yrs 9 mos

• ● Created event-driven delivery tracking platform using Kafka, Spark Streaming, and MongoDB. Implemented windowed aggregations with exactly-once processing guarantees, reducing latency from 2 hours to 1 minute. Designed fault-tolerant topology with checkpointing and schema evolution strategies for backward compatibility.
• ● Designed and implemented a scalable Feature engineering service for semi-structured MongoDB data to support machine learning workflows via Airflow. Automated model training, inferencing, and deployment using Docker, FastAPI, S3 and Datadog monitoring, reducing manual intervention and simplifying data flow management for production-grade ML models.
• ● Engineered a self-serve anomaly detection system using FastAPI, MSSQL, Airflow and Snowflake to enable service owners to define dataset rules for anomaly detection. The system delivered batch alerts for breach scenarios, allowing proactive decision-making for new feature launches, improving operational efficiency and reducing downtime.
• ● Worked on implementation of a binlog-based Change Data Capture (CDC) architecture using HVR/Debezium to migrate data from MSSQL to Snowflake, optimizing OLAP workloads. Enabled seamless on-prem to cloud data flow with Snowflake’s elastic scaling, resulting in a 50% reduction in query time for time-sensitive analytical use cases.