Apr 2022 – Mar 2023 · 11 mos · Bengaluru, Karnataka, India · On-site

• 1. Flex/Auto Debit Team : Responsible for auto debit module and it's smooth integration with entire products line of the organisation.
• 2. Contribution in Strategic/Project Planning for various product lines of the organisation.
• 3. Responsible for Reliable and Timely Tech Support and Data Manipulation Requests for the team.

Oct 2019 – Dec 2021 · 2 yrs 2 mos

• 1. Data Aggregation ETL Pipeline which feeds the analytics team and also powers experiment framework: core decision making tool for rolling out features. Involves discussions with analytics team for changes in data requirements.
• Recently we completed migration from in-house Hadoop cluster to AWS EMR. Single Handedly developed the framework to start/create EMR cluster and run jobs on the same which seamlessly integrates with the current scheduler framework in the organisation using AWS SDK.
• 2. Search indexing logic. Event processing pipeline via Kafka, few other data sources using Apache Storm with pre-processing business logic utilised in search. Involves deep discussions with import team over data flow and with search team for search logic.
• 3. Microservices/Caching (Co-owner) : Data flow from import to platform layer to be used by FE via microservices (thrift) which in turn uses distributed cache (aerospike). Platform layer is powered by various tools including Kafka, apache storm to populate cache and Relational DB. Involves code reviews, design discussions and reviews of the data flow for various feature of the website.
• 4. search thrift service business logic (Co-owner): Refining Apache Solr querying logic for various plenty business use cases including new search features, improving search relevancy. Involves discussions with product management teams over new features.
• 5. Led Initiatives to streamline delivery process of dev community and improve the quality by ensuring best practices. Involves regular discussion with the platform team developers and Tech-ops members and bring them to a common ground for the development practices to be followed in the team.

Sep 2017 – Sep 2019 · 2 yrs

• 1. Data Aggregation ETL Pipeline: Explored Apache Spark for processing Data and Apache Zeppelin for Reporting.
• 2. Apache Solr indexer module for processing Kafka events using Apache Storm. Completed major migration from NoSQL import to Other Data Sources including Cache (Aerospike).
• 3. Billing Module Automation: This module deals directly with billing merchants for sales OR clicks.
• 4. Co-ordinated with other teams and helped various members during
• a) Migration from Rented Data-Centre Servers to AWS Cloud.
• b) Modification for Compliance with GDPR Regulations
• 5. Import jobs (Co-Owner): Maintained Hbase (NoSQL) import ecosystem powered by spring jobs framework and helped MSSQL Team to take over the import while NoSQL retired.
• 6. Coordinated with various teams and worked on issues at many levels, including Networking, JVM optimization, Concurrency issues, Microservices issues etc. to name a few.

Apr 2015 – Feb 2016 · 10 mos · Gurugram, Haryana, India

• 1. Data Aggregation ETL Pipeline:
• a) Independently led Data Aggregation team and made it stable with development of generic data aggregation tool which considerably reduced monitoring hassles also mentored team members to write aggregations on the same.
• b) Documented whole Data Flow for Aggregations i.e. depiction of data flowing from data sources to various processings (all the aggregations) and being delivered to consumers, which gets used to this date for establishing relationships between various aggregations.
• 2. Individual Contributor : Platform team
• a) Developed tools for microservices teams to load test thrift services, log requests etc which then can be correlated to performance metrics during the testing.
• b) Developed script to measure latency of Kafka consumers.
• c) Owned python module with sharp learning curve that generates thrift objects from thrift files in git repository and uploads on nexus to be used by various teams.

Apr 2014 – Mar 2015 · 11 mos · Gurugram, Haryana, India

• Individual Contributor: Data Aggregation team:
• 1) Worked on Data Aggregation with Apache Hadoop/Hive platforms for various metrics including keywords, products, clicks, sales, revenues as per different models etc.
• 2) Contributed to Experiment Framework Aggregation Migration from Java based to Apache Hive. This experiment framework was to absorb the data and come up with metrics which could be used in decision making for changing both front-end or back-end components. Some of these metrics were CTRs, RPVs, COS etc.
• 3) Used Apache Kafka message queue for realtime impressions logs aggregation.

Jun 2013 – Mar 2014 · 9 mos · Gurugram, Haryana, India

• Data Aggregation Team:
• 1. Developed Java programs, Shell scripts to enable data aggregations on Distributed Hadoop/Hive platform (Migrating from Single machine RAM based java applications.)

Jun 2011 – Mar 2012 · 9 mos · India

• CheckoutByAmazon team in amazon.com at Bangalore
• 1) Complex SQL reports concerning revenues from merchants and fixed them to fill the gap of hundreds of dollars.
• 2) Validation of data being inserted in the system for setting up payment modules for merchants.

Jun 2009 – Jun 2011 · 2 yrs · Noida, Uttar Pradesh, India · On-site

• 1 ) June -September 2009: PayTM: Data Validation and Encryption using Java Hibernate.
• 2) Data Aggregation Team:
• a) Voice/IVR data warehousing involved low scale data with multi faceted configuration for processing.
• b) Pre ET - Before extraction of data from various circle servers, compressing and creating meta data for them using python module.
• c) ET - Extract the data from circle servers to Central server using scp and monitor the same using python module. Major achievement included converting daily ETL to almost Real time ETL.
• d) TL (Co-Owner) - Transform and load the data into database and monitor the same.