Oct 2024 – Jul 2025 · 9 mos · Bangalore Urban, Karnataka, India · Hybrid

Jul 2022 – Sep 2024 · 2 yrs 2 mos · On-site

• Orchestrated and mentored a diverse team of 15+ engineers, including Data Engineers, Frontend Engineers, and Engineering Managers, fostering a culture of collaboration and driving excellence in project delivery.
• Oversaw appraisals, hiring initiatives, roadmap definition, incident reviews, and tech branding initiatives.
• Helped establish best practice standards and guidelines, which increased efficiency by ~30%
• Fostered seamless collaboration among data engineering, data science, analytics, and cross-functional stakeholders to ensure streamlined access to data and actionable insights.
• Provided technical leadership and hands-on implementation role in the areas of data techniques including integration, processing design and implementation.
• Spearheaded multiple transformative data platform initiatives:
• ▪ Streamverse: Pioneered a cutting-edge streaming platform as a service, harnessing technologies such as Apache Flink, Apache Kafka, Apache Pulsar, Kubernetes, and Spring Boot to elevate real-time data management and processing capabilities.
• ▪ Data Highway: Engineered an innovative Events management and collection platform utilising Vert.X, Apache Kafka, and Parquet, optimising event handling and accessibility for enhanced insights.
• ▪ Realtime Warehouse: Pioneered the development of a high-performance real-time warehouse, enabling swift data querying akin to a traditional data warehouse and ensuring timely insights.
• Led efforts to phase out SaaS tools used for user behaviour analysis, such as funnels, traits, and user segmentation, building and championing the adoption of in-house solutions for greater control and efficiency.

Jul 2020 – Jul 2022 · 2 yrs · On-site

• Led a project for clickstream data ingestion using Kafka, Flink and REST APIs, achieving 99%+ uptime at 500M+ RPM.
• Engineered a Real-time user concurrency calculation solution with Kafka and Flink, ensuring 99.9% accuracy for 15M+ concurrent users on the Dream11 platform.
• Architected and implemented a real-time complex event processing (CEP) system using Apache Flink, empowering the generation of dynamic cohorts for enhanced data insights and user targeting.
• Designed and developed a Data Dictionary/Catalog framework, enabling streamlined access to metadata across the Dream11 ecosystem and improving discoverability.
• Innovated in-house funnel analytics product, reducing processing time to 30-50 seconds through optimised SQL queries and storage, surpassing Google Analytics benchmarks.
• Proposed and implemented an improved batch account reconciliation ledger using Apache Spark, reducing processing time from 4+ hours to ~10 minutes.
• Executed migration to an in-house Apache Druid cluster, substantially reducing costs while maintaining performance and scalability.

Dec 2018 – Jul 2020 · 1 yr 7 mos · On-site

• Implemented self-serve data pipeline tool, facilitating seamless movement of data from various sources (MySQL, Cassandra, etc.) to multiple sinks (MySQL, Redshift, etc.).
• Developed multiple batch-processing pipelines for Dream11 product-related tasks.
• Engineered real-time processing pipelines for applications, including operational analytics and anomaly detection for business KPIs.
• Implemented Reverse ETL pipelines to transfer data to third-party platforms securely.

Feb 2017 – Nov 2018 · 1 yr 9 mos · Pune, Maharashtra, India

• Designed and implemented an easily configurable ETL framework written in python.
• Extensively reduced the time taken to onboard new raw datasets by using the python ETL framework. Raw data could be brought into the pipeline, be scheduled and ready for consumption by the Data Science team in a couple of hours depending on the dataset size.
• Worked on designing and implementing of a near real-time data pipeline architecture involving
• Apache Kafka and Spark streaming to make available all recommendations as soon as the data enters the source
• Reached the option to migrate an available tensorflow model on to TensorflowOnSpark to utilize the
• width and capacity of the cluster to train the machine learning models
• Designed a data validator and comparator tool in python and spark to validate and compare data that is pulled into the data pipeline
• Worked on designing a generic and configurable data cleanser as part of the ETL pipeline. (De-dup,
• filtering etc.)
• Worked comprehensively on automating and improving the data processing carried out by CITI Research Analysts on their desktops by converting the process into a streamlined architecture using Spark, Impala, and making the data available to Analysts on Power BI

Feb 2016 – Dec 2016 · 10 mos

• Designed and implemented new software solutions and improvements to existing software products
• Worked on implementing various API’s for the HPE Connected MX product.
• Worked on Implementation of the patented SendOnce technology from Hewlett Packard Enterprise which performs block-level deduplication
• Improved the product usability and helped improve the admin dashboard by extensively working on API’s for generating various reports for the Connected MX product line.
• Researched improvised ETL options and thus implemented a design that reduced the ETL delay in reporting. Designed and implemented data pipelines that transfer data in near real-time to HPE Vertica analytics platform for reporting purposes.
• Achieved a near real-time ETL pipeline after researching and designing an implementation for data transfer from Hbase to Vertica using Apache Spark.
• Investigated and put forward a design that was implemented to schedule ETL jobs using Apache Oozie Scheduler and workflow
• Researched Apache Kafka and put forward a plan to implement kafka to bring near realtime updates for the data pipeline to get real-time reports generated from Vertica.
• Working on Integrating and bringing various HPE Information governance products together under one single framework.
• Comprehensively making functional changes to the product design and ETL functionality to make the complete process of reporting more streamlined.
• Touching base and gaining knowledge of Mesos and Docker to implement the functionality in the current product structure and design.

May 2015 – Jan 2016 · 8 mos

May 2014 – Jan 2015 · 8 mos · Southborough, Massachusetts, United States

• Received formal training and acquired understanding of QA and testing process in an agile environment
• Involved in Planning, developing and administering software test plans to detect and diagnose errors and bugs for the company’s Connected MX product line
• Performed regression, integration, platform and performance testing to certify the stability and usability of software
• Developed and executed test programs that helped improve accuracy of QA results
• Maintained and optimized the environment used for testing the product
• Developed a framework in lieu of the company’s product that is used by the QA team to test the product

Jan 2013 – May 2013 · 4 mos

• Acquired skills in Java programming and had a chance to implement them on a live project.