Oct 2023 – Sep 2024 · 11 mos · Hyderabad, Telangana, India

Mar 2020 – Sep 2023 · 3 yrs 6 mos · Bengaluru, Karnataka, India · Remote

• 1. Engineered data pipeline for calculating market risk metrics processing more than 100 Billion data points daily by leveraging Spark, Kafka and Rest API built using Spring Boot.
• 2. Led implementation of sustainable coding practices, reducing carbon footprint, and promoting eco friendly solutions for project development.
• 3. Optimised Spark jobs by dynamic scaling of clusters, leading to cost reduction of up to 70% through dynamic allocation on Azure HDInsights cluster.
• 4. Designed and installed new modules for Big Data processing adding more checkpoints to pipeline.
• 5. Migrated computing systems from private cloud to Microsoft Azure, reducing operational costs by over 40%.
• 6. Interfaced periodically with downstream and upstream application teams, ensuring seamless workflow and optimal performance.
• 7. Executed tickets promptly, aiding in sprint task prioritisation following Agile methodology for efficient project delivery and team coordination.
• 8. Implemented user access control to delegate incident resolution responsibilities effectively across teams, ensuring targeted response to data anomalies in pipeline.

Jun 2016 – Mar 2020 · 3 yrs 9 mos · Pune, Maharashtra, India · Remote

• Developed NiFi application to consume and enrich transactions from Kafka and data persisting using Lamda architecture.
• Developed Spark Application for migrating business and finance reports from IBM mainframes to new Hadoop based jasper reporting platform.
• Data from different source systems were imported using Sqoop and processed in Hive.

Jan 2013 – May 2016 · 3 yrs 4 mos · Pune/Pimpri-Chinchwad Area

• Developed a web-based tool for ingesting data of RDBMS based sources into Hadoop ecosystem.
• Create Hive table and create partition on the table.
• Defect fixing and closely monitoring and reporting the environmental issues to the concerned platform team.