Jun 2024 – Apr 2025 · 10 mos

• Technical Lead for multiple cross functional projects involving collaboration with
• various teams to grasp the underlying flows and identifying SOTs to fetch data points.
• Owned end to end cluster migration of all the data applications of the organization
• from Cloudera CDP to AWS EMR that reduced the Cost of Compute and Storage by
• nearly 40%, ETL batch runtime reduced from an average of ~4 hours to ~2 hours
• and average Tableau query refresh time reduced from ~550 seconds to ~250 seconds.
• Setting up the platform and infrastructure for Apache Iceberg Adoption, enabling schema evolution, time-travel, and ACID compliance
• Built a multi-tenant Data Lakehouse, with scaling capabilities to unlimited retailers, 3rd-party sources, and e-commerce business processes.

Jan 2022 – Jun 2024 · 2 yrs 5 mos

• Involvement in tech stack selection process by performing various POCs, preparing
• detailed design document and developing custom solutions using open source libs like
• Apache Beam, Apache Sqoop, Maxwell service and Cascading
• Led the project of designing and developing a new generic and technology agnostic
• platform for the data applications with multiple reusable modules by reducing code
• redundancy and refactoring the legacy code using Java.
• Owned the Auditing and Logging Backend Service for the internal applications of the
• organization by designing and building REST APIs using SpringBoot framework.
• Developed a new data pipeline that consumes events from streaming services like
• Apache Kafka using Spark framework and reduced the load on underlying
• infrastructure by 50x.
• Mentoring along with planning and assigning daily tasks of junior team members.

Jul 2019 – Dec 2021 · 2 yrs 5 mos

• Worked in various cross-functional projects to build both batch and realtime data
• pipelines in a distributed system using Java and Big Data technologies including -
• Hadoop, HDFS, Hive, Impala, HBase, Phoenix and Kafka
• Involved in building the reporting platform of the organisation using Microservices
• Observed scalability issues with the former realtime architecture, recommended
• few solutions and led the design review discussions with the architect forum
• Constructed a single Kafka processor capable of processing multiple realtime
• pipelines through ingesting Maxwell events from multiple topics and eventually
• reducing the complexity from O(nk) to O(k) for scalable expansion of the system
• Designed and developed a new advanced framework for Provisioning Auto-Deletion
• of Realtime Events that facilitated smooth flow of realtime pipeline even when few
• records get missed due to infrastructure glitches
• Built a generic platform for masking and deletion of PII data to address privacy
• concerns, helping the organization to become CCPA and GDPR compliant
• Took initiative to create and maintain Monitoring frameworks using Python Django
• and Slack APIs - being a crucial contribution to the team’s load that makes manual
• interventions and monitoring effortless in case of site up issues

Jan 2018 – Jun 2018 · 5 mos · Bengaluru, Karnataka, India

• Indulged in the End-To-End development of data pipelines for Business Intelligence
• team that required gathering requirements, designing, developing, writing functional
• test cases using JUnit, integration testing, deploying and validating resultant data.
• Contributed in migrating the ETL jobs of the organization existing in “Talend” tool
• into the ones implemented through Cascading using Java and Big Data Technologies
• reducing overall time taken by 50%
• Resolved Impala query failure issue that involved understanding basics of Impala
• query mechanism and then developing new utilities to fix the frequent failure of
• Impala queries due to stale metadata fired through Cascading jobs of the company.

Jun 2017 – Jul 2017 · 1 mo

• Remote Internship under Prof. Sameer Mathur, Marketing Professor (IIM Lucknow)
• Learnt Basic Statistics, Business concepts, R-programming and basics of Core ML models like Regression and implemented them using R-
• programming for few projects like analysing factors causing difference between prices of hotel rooms in different cities of India