Sep 2021 – Present · 4 yrs 8 mos · Bangalore

• Contributions:
• A scalable, ultra-fast (sub 10 ms), generic rule engine that is flexible enough to support custom scripting yet at the same performant enough to handle 25K RPS on a single instance. Everyday it processes around ~120 million evaluations.
• A real-time storage and retrieval engine that supports snapshotting and aggregation on JSON messages. Provides batch support for ingestion and provides simple APIs to decouple event producers from downstream consumers. It powers high velocity applications at PhonePe such as fraud evaluations and user summary for offer disbursals; provides full support for associating time windows with aggregations. Roughly processes around 1.7 billions queries per day while still serving under 8ms latency (p99).
• A system for storing and processing user profile data that is designed from ground up to be highly performant for real-time ingestion and querying as well as batch analytics. Supports efficient batch writes and keyed reads. Facilitates running custom compute on data to do universal ETL and machine learning on Spark. It supports storing time series data along with raw attributes and cached views. It provides capabilities to encrypt and secure data with data access control. Data could be bulk loaded from Hive, HDFS, CSV and other data sources. Serves roughly 1 billion queries per day.
• Conceptualised, designed and implemented the machine learning platform at PhonePe. I implemented the core batch scoring pipeline over Spark which supports running MLModels developed in Python with support for transformations written in Java, Scala or Python. It uses Apache Arrow for IPC between the Spark worker and Python worker.

Sep 2016 – Jan 2021 · 4 yrs 4 mos