Oct 2022 – Dec 2024 · 2 yrs 2 mos

• I am currently working as SDE-II in Flipkart Data Platform team.
• I got the opportunity to explore the upcoming lake-house technologies like Iceberg, Hudi. I carried out detailed POCs to explore capabilities of Iceberg and Hudi. As part of exploring capabilities of Iceberg, I contributed to open source community by extending capabilities of Iceberg Flink Connector to support writes to multiple tables (PR: https://github.com/apache/iceberg/pull/10376).
• I explored capabilities of Apache Beam framework for unification of batch and streaming workloads. Apache Flink was used as the underlying runner below the Beam abstractions. After POCs and tech stack comparison, I worked in a team of 7 members to setup a streaming pipeline that used Beam over Flink to read from pubsub and sunk data into iceberg tables.
• I worked closely on Spark Streaming along with GCP provided pubsub. In a team of 4 members, planned and executed an in-house solution to sync data from pubsub and write to GCS buckets. This led to 70% cost reduction in syncing layer.

Jul 2021 – Feb 2023 · 1 yr 7 mos

• I worked in the Flipkart Data Platform Team. FDP provides data warehousing solution to company wide data and provides streaming/batch processing capabilities over it.
• I closely work on batch-processing layer, which supports close to 10000 jobs running daily. Tech stack: Hdfs/GCS, Yarn, Map reduce, Hive, Spark, Big Query.
• One of my significant contribution has been to re-write partitioning logic for few jobs that has brought down storage needs by 80% and execution time by 50% for those jobs.
• I have been a part of the 5 member team that migrated these 10000 jobs to a new cloud platform.
• I did containerisation/dockerisation of services and migrated them to kubernetes. This gave me opportunity to have an in-depth understanding of VMs, k8s, docker, helm charts etc.
• Working in FDP has given me exposure to work with data of petabyte scale. In FDP, I have explored various techniques of optimisations and cost reduction.
• Tech used: Java, Dropwizard, HDFS, Map reduce, spark, Yarn, Hive, Mysql, Python, kubernetes, linux.

Jun 2019 – Jul 2021 · 2 yrs 1 mo · Gurugram, Haryana, India

• I worked on two projects: AirtelXstream and WynkStage. Airtelxstream is an OTT platform (app + website) that provides tv-shows, movies and live-tv channel. Wynkstage, on the other hand, is an online concert streaming platform embedded with Airtelxstream and Wynk music app.
• In a team of 3 people, We managed Airtel Xstream website and implemented features like Continue Watching, Content Filter pages, Subscription Strategy and other product Features.
• We brought down page serve time and page load time to below 2s and 6s respectively. This was achieved as a result of a slew of measures involving pagination of content API, service worker caching, Redis caching etc.
• In a team of 2 people, We developed the MVP of product Wynkstage in 2 weeks time. It involved writing a layer of playback utility from scratch. The platform also had features like live comments, live reactions, recording of old concerts etc.
• Working on these two products gave me exposure to user-facing web applications and web services. Working on them made me have user-first approach in designing features/services.
• Tech Used: Java, Spring, Reactjs, Nosql DBs, Firebase, Redis

May 2018 – Jul 2018 · 2 mos · Bangaon Area, India

• Software intern. I worked in equities data platform team on a big data project. It was a graph based problem. Given the one to one relations between different nodes, the hierarchy had to be constructed in efficient manner. To solve this problem, I worked on spark and scala and explored different open source graph libraries like jGrapht, graphX etc.
• It was a steep learning experience that introduced me to concepts of big data, spark, distributed systems and gave me opportunity to put my factual knowledge of data structures to use.