Apr 2022 – Nov 2023 · 1 yr 7 mos

• As a part of Rakuten Symphony Product suite , I am an integral member of Data Platform product development team. I design, develop & integrate various generic components that will solve the organisational needs in the big data lifecycle and management, such as data generation, data migration, data processing, data discovery, data access control & governance etc., considering the performance at scale. And In the process, making the Open source Tools perform better, by enhancing them to our needs and contributing back to the community.
• Implemented Synchronous Listeners in Apache spark to fix Spark Atlas Connector issues of missing data objects & lineages.
• Extended Apache Ranger for non hadoop stack like S3/MinIO, Cassandra/Yugabyte.
• Enhanced Trino S3(Hive) Tables auto partition sync for registering new partitions.
• Designed and implemented monitoring frameworks and pipelines to capture and calculate KPIs for different components in Data Platform.
• Lead a cross functional team of 5(UI/Backend/DE/QA) ,including myself as an individual contributor, for a Drag Drop UI based Data transformation engine.
• Lead various Critical Data Migration projects with more than 5PB of data to newer Systems of Data Platform and also enhancing the performance of system and applications in the process.

Aug 2021 – Apr 2022 · 8 mos · India

• I am a member of Rakuten Data Platform product development team. My responsibilities include design, develop & integrate various generic components that will solve the organisational needs in the big data lifecycle, such as data generation, data migration, data processing, data discovery, data access control etc., considering the performance at scale.

Apr 2021 – Aug 2021 · 4 mos

• Insights team as a part of FPG (Fintech of Flipkart), aims at predictingcustomers behavior in various aspects like risk, affordability etc., based on which the customers will be lent credit, either with in Flipkart like recommending products by the affordability, or by third party financial firms like banks, insurance companies etc.
• Designed the analytics system (set up the Hadoop cluster and its ecosystem, choosing the right tech stack for the use case, model data for optimized read/writes etc.,) for various insight generation use cases. Flipkart Pay Later(BNPL) product is the primary use case.
• Closely worked with Data Scientists and build ETL + ML pipelines for the needs.
• Developed a Wrapper REST service with Persistence Layer for Apache Livy for remote Spark job submissions.

Sep 2019 – Apr 2021 · 1 yr 7 mos

Apr 2019 – Sep 2019 · 5 mos · Bengaluru Area, India

• ITDSD is VMWare’s Data engineering and analytics solutions platform. It will help in providing efficient data warehouse thereby catering analytics ready data that will help in gaining business insights.
• As a part of this project, I have worked on License Fraud program, which helps in detecting fraudulent licenses’ usage. Also involved in developing ETL pipelines as per the business’s needs.

May 2018 – Mar 2019 · 10 mos

• TCS ACTIVE ARCHIVE
• TCS Active Archive is a high performance, scalable, secure, and cost effective Big Data archive that keeps data ‘warm’ for easy access and analytics. It is a readymade solution developed for the projects/enterprises that face the costly and complex task of managing huge volumes of data and storing it to meet regulatory requirements. It provides solutions in the form of ETLs that helps in the complex data analytics.
• Developed web services using Java that drive Data Ingestions. This involved modules that
• Crawls any Website(Using Apache Nutch) and put the data to MongoDB/ Elastic search.
• Creates/Triggers jobs for RDBMS to HIVE ingestion and streams application logs using flume and saving the same in HFDS for further processing.
• HDFC LIFE TRANSFORMATION
• As a part of middleware layer (IBM SOA) developed Java web services that serves the critical functionality of transformation process. Owner of RMS (Requirement Management Service), a critical module within SOA layer that provides automation for fulfilling the required documents without user's intervention in the Insurance journey

May 2016 – Apr 2018 · 1 yr 11 mos

May 2015 – Apr 2016 · 11 mos