Jan 2022 – May 2024 · 2 yrs 4 mos

• Working with Payments, UPI Team & Fraud-detection teams.
• Building niche UPI App in Flipkart to handle growing UPI customer base
• Impact: To improve order conversion rate for UPI Mode's by ~20%. To reduce cash handling cost at delivery side by 30% by moving the users to FK UPI.
• Led all the phases starting from resource planning, hiring, understanding audit compliance, high level architecture, cross-team coordinations, way-of-working (agile), naming standards, various guidelines, processes, functional stories design reviews & execution , NPCI Audits, scale testing and finally production launch.
• Used async processing at the scope to improved scalability at reduced hardware footprint.
• Built Domain Driven based services, focussed on future extensiblity. Used Orchestrator saga pattern.
• Tamed HBase NoSQL for this transactional use-case for linear scalability. Used Apache Pulsar + Apache Storm for async message processing.
• Secured all the payment sensitive and Personally Identifiable Information(PII) as per the OWASP Standards and CIS benchmarks using encryption(AES) and data-transmission practices (TLS 1.3).
• Scale: 2000 Peak UPI payment transactions per second. 100K DB QPS
• Tech: Dropwizard, HBase, Apache Pulsar, Zookeeper, HAProxy, Hystrix, Encryption libs.

Apr 2018 – Dec 2021 · 3 yrs 8 mos

• Working in Flipkart Supply Chain Platform Team.
• Record Replay Testing Toolkit - used by multiple teams in Flipkart
• With passion, I have envisioned and built a toolkit which records production traffic and leverages this for both scale-testing and regression testing. This needs zero effort at the application-team level.
• Impact: Used by ~20 teams internally and helped scale-testing since BBD 2020+. Reduces effort from 1-2months to 2-weeks
• Scale: Recorded and replayed 40K API Calls/sec, 350 DB QPS/sec
• Tech: Java agents (+byte-buddy), Java instrumentation Spring Boot, Kafka, , Zookeeper, Async-io, multithreading concurency.
• Enhancements on Open Source Tungsten Replicator (Change Data Capture):
• As we have modelled systems using micro-services architecture, there was a need to provide unified real-time view on the entire business data for operation teams which spanned across multiple microservices.
• Achieved it using Vertica MPP(Massive Parallel Processing) database thru binlog replication from different MySQL stores using fan-in topology. Tungsten replicates from MySQL to Vertica.
• Added DDL Schema-Replication support in Tungsten internal fork for MySQL to Vertica data replication path to provide business continuity & zero downtime.
• Added skip-duplicate-records feature be able to switch the replication stream to any MySQL Server seamlessly (works without GTID support).
• Achieved 20K binlog events QPS per JVM by enhancing the parallel replication & batch support using java Executor Service.
• Tech: Core Java, JDBC, Java Executor, Mozilla Javascript Engine, Velocity, Pentaho, Tomcat

Jul 2016 – Mar 2018 · 1 yr 8 mos

• Working in Big Data Platform Team
• Trinity - Metrics Collection Service for Hadoop batch processing pipelines Aug 16 –Nov 16
• At Flipkart Data Platform ( 5Billion messages, 2.5PBs data processing per day), there was a need to collect scheduled DAG(Directed Acyclic Graph) jobs metrics from different services/stores which are involved in different phases of the batch jobs life cycle such as job definition change-management, auditing, execution, runtime job counters etc.
• Responsibilities:
• Used Pipe-Filter Architecture which can read metrics from any source, enrich and persist to any store.
• Designed as Pull model to decouple from the source systems and implemented Checkpoint State Management ability to recover from commodity hardware failures.
• Implemented to read from YARN Resource Manager, OpenTSDB, Job History Server, Timeline Server, Azkaban & Name Node FSImage
• Also built anomaly detection and reporting module using statistics to prevent any cascading failures or resource crunch at YARN Queues.
• Tech: Java, OpenTSDB, Hadoop components, Azkaban, Dropwizard, Quartz, Hibernate

Sep 2014 – Jun 2016 · 1 yr 9 mos

• Working in Big Data Platform Team.
• Hercules - Multi-tenant, HA, resource-aware, asynchronous data loader to
• realtime consumption stores:
• Hercules is built to decouple Batch-Processing Pipelines from refreshing the final aggregated views on query store (40TBs of data refreshes per day) to scale independently. Also we can apply loading & query optimization techniques, prioritization of pending jobs under resource constrained environments.
• Modelled as Generic Job Scheduler using Event-Driven Architecture (inspired by Hadoop Event Dispatcher)
• Loosely-coupled Singleton components like Request Poller, Job Manager, Strategy based Job Scheduler, Task Orchestrator and executor, Event Logger which register for specific events at Event Dispatcher.
• Used java.util.concurrent and Executor Service extensively, designed multi-threaded concurrent service
• Using Multi-tenant capability, onboarded heterogenous ETL jobs like exporting aggregates to Vertica and analytical Cubes to Azure SQL DataWarehouse.
• Implemented High-Availability with auto fail-over Using LeaderElector of Apache Curator recipe for Zookeeper.
• Extended the design to pre-warm the critical reports cache for better end-user latency.

Apr 2014 – Sep 2014 · 5 mos

• Worked on Learning Management Systems.

May 2012 – Apr 2014 · 1 yr 11 mos

May 2010 – Jun 2012 · 2 yrs 1 mo

• Worked on the CORDYS Service Oriented Orchestration platform. There I developed a connector for the conversion of SOAP protocol messages to REST.