Oct 2023 – May 2025 · 1 yr 7 mos · Kolkata, West Bengal, India · Remote

• Led the platformization of LLMs for various end-user and internal use cases at Coinbase, leveraging multi-cloud solutions (AWS Bedrock, GCP Vertex AI, Azure OpenAI) and in-house models on Databricks and Anyscale. Developed a highly scalable, multi-tenant architecture with a focus on optimal token usage for high-volume applications, and implemented large-scale semantic and full-text search on knowledge bases.
• The platform powers 30+ high-volume use cases, resulting in $40M savings.

Mar 2020 – Oct 2023 · 3 yrs 7 mos · Remote

• AWS EMR Dataplane, optimizing Spark/Trino performance in EMR:
• Optimised EMR Spark query planning, interaction with S3, and JVM memory layout and access patterns.
• Predictive caching and reuse of common query plan fragments
• Optimal Join strategy selection, dynamic partition pruning
• Efficient decimal operations via query operation rewrite and unscaling to fixed points
• Amazon Business Process Automations:
• Designed real-time streaming data platform for Order, Shipment, and Return events, enabling fraud detection and business interventions; standardized solution reduced development time from weeks to day, 40+ use-cases, saved 300Mil USD yr.
• Built serverless DataLake supporting interactive analytics and ML workflows, achieving 3X improvement in operational metrics and 10X cost reduction in primary workloads
• Led implementation of large scale knowledge graphs (+40B vertices) and ML pipelines with automated drift detection, optimizing critical user flows like search and checkout
• Optimized real-time feature store (DynamoDB) performance, achieving 13X throughput improvement and $500K annual cost savings
• Architected and deployed knowledge graph system for AWS customer account detection, enabling automated collection processes and consolidated statements; solution scaled to 40+ Amazon business lines
• Optimized AWS billing and metering platform integrations, reducing pipeline latency from hours to minutes and decreasing infrastructure costs by 70%; implemented intelligent email-account matching to reduce response times to hours from days
• Modernized global AWS enterprise support's CRM and rule engines, significantly improving shared database performance and cost efficiency
• Technologies: AWS (Neptune, DynamoDB, RedShift, Textract, OpenSearch, EMR, RDS, Batch, DataPipeline, Athena, Glue, Lambda, Kinesis, SageMaker, Step Functions, EventBridge, SQS, SNS), Apache Spark, Trino, Apache Iceberg, Apache Parquet, Distributed Graph Neural Networks, HuggingFace.

Nov 2019 – Mar 2020 · 4 mos · Greater Bengaluru Area

• Worked on Common Services, Frameworks and Enterprise team. (Scala, Play, Python)
• Optimised realtime time feature store's operational cost by 40%.
• Implemented public REST apis for model monitoring control plane.
• Implemented config driven ingestion and persistence layers for feature store and model monitors (Kinesis, MongoDb, S3)

Feb 2019 – Nov 2019 · 9 mos · Gurugram, Haryana, India

• Data Platform Development:
• Engineered a scalable platform for ingesting and querying data into and out of a centralized Data Lake using Scala, Spark, Airflow, Python, Pandas, PySpark, and AWS services (EMR, Lambda, S3, Glue, Athena).
• Developed "Vaayu," a Spark-based framework to efficiently process and store data from diverse sources.
• Introduced "Tejas," a serverless feature store that facilitated the sharing and reuse of features required by various ML models.
• Automated the generation of reports and analytics using an in-house Airflow setup, streamlining operations.
• Freight Marketplace Optimization:
• Implemented a queryable application log repository using AWS Glue and Athena, resulting in significant cost savings by reducing ELK stack reliance.
• Achieved a 40% reduction in AWS costs by optimizing service flows, transitioning from real-time processes to batch or Lambda functions, and enhancing SQS utilization through message piggybacking and long polling.
• Prime Relay (Shipment as a Service)
• Developed an automated predictive maintenance alerting system based on sensor data feeds from a truck fleet, using a fully serverless architecture that scales with data volume.
• Created a versatile framework for runtime selection of map vendors, reducing costs and allowing for diverse vehicle profile selection across vendors like Google Maps, Bing Maps, and Map My India.
• Optimized and cached results for microservices that suggest routes and refuel points for relay pilots, leading to improved efficiency and resource management.

Jul 2018 – Feb 2019 · 7 mos · Gurugram, Haryana, India

• As a Staff Engineer at Rivigo, I led the transformation of the Payments and Commerce platform into a robust microservices architecture, delivering significant operational efficiencies and enhanced user experiences.
• Key Achievements:
• Microservices Refactoring:
• Spearheaded the decomposition of monolithic "Payments/Commerce" services into specialized microservices for Payments (pay-in, pay-out), Wallet, Incentives, Fuel, and Fastag, ensuring seamless integration and scalability. (Tech Stack: Java, Scala, SpringBoot, Hibernate, MySQL)
• Real-Time Payment Automation: Engineered a real-time pay-out system to suppliers via IMPS/NEFT, with automated retries and refunds, optimizing transaction success rates through intelligent bank and payment mode selection.
• Wallet Development:
• Created a real-time wallet system for both demand and supply users, integrated with bank accounts, fuel vouchers, and Fastag top-ups, enhancing financial transparency and user engagement.
• Fuel Integration: Successfully integrated with major fuel vendors to automate the generation of fuel vouchers, streamlining fuel management for users.
• Incentive Engine:
• Developed a rule-based incentive engine powered by a custom DSL, allowing business analysts to define and trigger coupon distribution in real-time based on eligibility criteria.
• KYC Service Implementation: Introduced a KYC service utilizing penny drops to verify beneficiary accounts, bolstering security and compliance.
• Ecosystem Testing:
• Designed a comprehensive framework for integration and stress testing, ensuring the reliability and robustness of the entire payment/commerce ecosystem.
• This work resulted in improved system reliability, streamlined operations, and a more responsive and personalized user experience, contributing to Rivigo’s leadership in the logistics sector.

Jan 2016 – Jul 2018 · 2 yrs 6 mos

• Working on ML based extraction of features from print documents, to be translated into responsive and accessible HTML format, for better viewing experience in handheld devices.
• implemented the data and feature extraction pipeline, for training the deep learning networks
• implemented post processing framework for consumption of the deep learning networks output
• implemented feature extraction from pdf documents
• implemented final transformation layer for the HTML output, into AEM forms format.
• Patented process for Conversion quality evaluation for digitised forms (US20190163733A1)
• Launched as part of cloud offerings in Adobe Sense-Ai

Jul 2013 – Jan 2016 · 2 yrs 6 mos

• Worked on feature development & performance optimization in multiple AEM Forms products.
• optimization of XFA PDF data fills and rendering engines
• implemented optimized loading of sections of html forms
• worked on front end representation of form widget controls
• worked on back end representation and manipulation of user data, entered into the form
• worked on extraction of semantics of flat paper forms, to convert into reflowable html forms.