Aug 2015 – Sep 2019 · 4 yrs 1 mo

• Flipkart Plus, Recommendations & Search
• Flipkart Plus
• Designed and built multiple micro-services that power the product
• Building more product features and capabilities
• Building the next-gen Recommendations Platform
• Scaling the backend data pipelines
• Improve and iterate on the ML algorithms
• Real-time ingestion pipeline - Kafka, Storm, Redis, HBase, Solr
• Built and scaled the streaming real-time ingestion pipeline
• Contributions to Apache Storm (storm-kafka-client)
• 10x throughput improvement along with >2x reduction in hardware compared to previous gen
• implemented comprehensive production observability (logging, metrics & alerts)
• made system robust & reliable with negligible impact of instance failures
• significant reduction in ingestion lag & errors
• Architected the custom next gen sharded search serving layer to handle more than 5x growth in index size. Developed & helped implement a comprehensive launch plan which included revamp of infra setup, automated build & deployment, debug tools, metric & alerts, etc.
• Using a data-driven approach convinced stakeholders across functions to remove/reduce data in the serving layer thereby achieving significant reduction in the capacity requirements of the search stack - indexing & serving. Drove the solution end-end to build systems/process for identifying non-critical data at the source, propagate it to all the consumers & remove such data from serving layer of search.
• Kickstarted the long term architecture evolution and tech debt reduction by breaking up a monolithic service.
• Mentor Developers; Evangelize and drive adoption of SDLC best practices - Code review, Design review, Unit/Functional/Perf tests; Ops automation

Aug 2013 – Aug 2015 · 2 yrs

• Building and Scaling a DMP + Online data-serving engine for DemandGen, Marketing & Personalization use-cases
• Built data pipelines to gather raw & processed data (users/merchants/transactions) from various sources
• Dataset of 400M+ User Profiles and 2.2B+ Identities. 40M+ daily updates processed
• Designed the data model for segmentation and targeting use-cases
• Integrated with the Ad Serving platform. Early campaigns confirmed a lift in engagement
• Partnering with Data Science to build & serve models for high-value prospects, shopping propensity, etc.
• Working on extending the platform for a variety of internal personalization use-cases
• Working on building a reporting/analytics component to deliver in-depth insights on the data

Aug 2009 – Aug 2013 · 4 yrs

• Worked on the whole of the Netflix Search Backend powered by Apache Solr and Lucene.
• Designed & Built the new Search Aggregator Service to enable/accelerate planned future product feature testing as well as improve the performance & scalability of the overall search backend architecture
• Developed, Tested (AB testing) and Launched a bunch of successful product updates to the Netflix member search experience on the website and devices partnering closely with Product & Data Science teams
• Designed and built the complete relevance scoring algorithm powering Netflix search on game consoles, iPhone and other devices such as Roku. This was the first-ever launch of Search functionality in Netflix app (PS3 was the initial AB launch platform)
• Evolved the architecture through various progressions - L10N, I18N, Multi-region deployment, Active-Active, etc. as the service expanded to Canada, UK, LATAM, Europe and rest of the world
• Enhanced the search index schema and relevance models to support Internationalization and Localization of search
• Architected and built the systems to run on AWS as part of Netflix's migration to the cloud; Search was among the first set of services shifted to AWS

Jun 2007 – Aug 2009 · 2 yrs 2 mos

• Design and development of Crawler, Webmap, Vertical Suggestion system.
• Crawler: 5B+ docs crawled and indexed
• Webmap: built using OpenMPI and Boost Parallel Graph Lib
• Webmap: ran on 500-node cluster with 2B+ vertices and 4B+ edges
• Vertical Suggester: Dynamic set of Verticals computed and scored at query-time; formed basis of a differentiated UX

Oct 2005 – Jun 2007 · 1 yr 8 mos

• Design, Development and Maintenance of numerous backend servers powering the search engine.