Mar 2023 – Present · 3 yrs · Bengaluru, Karnataka, India · On-site

• Worked on scaling GDPR solution on Datalake holding EB+ data, building Data Quality solution and AI based assistant to discovery the data in data lake.

Dec 2021 – Mar 2023 · 1 yr 3 mos · Bengaluru, Karnataka, India · On-site

• I led a project to optimize storage management for a large Aurora cluster, focusing on differentiating hot and cold data to reduce costs. The challenge involved detecting cold data segments and migrating them to lower-cost storage without affecting customer query performance. I developed a phased migration plan, identified key challenges such as heat management and migration times, and collaborated with the storage team to optimize CPU utilization and improve the efficiency of segment packing.
• As a result, we achieved a incremental improvement towards our cost-saving goal and ensured smooth operation without impacting system performance.

Aug 2017 – Jun 2020 · 2 yrs 10 mos

• Purchase Similarity Detection at Scale
• As a Software Development Engineer, I designed a solution to improve marketing messages. A problem I identified was the recommendation of products similar to those customers had already purchased—like showing another microwave after one was bought, which hurt engagement metrics.
• The existing filters only removed exact product matches but missed visually or functionally similar items across brands or models. I built a similarity detection system using product descriptions and image embeddings to identify similar items across categories like electronics and home appliances.
• The challenge was integrating this system into existing message generation pipelines. I designed it to be scalable, with category-specific similarity thresholds (e.g., stricter for electronics). This reduced redundancy, improved customer experience, and optimized marketing opportunities.The solution improved engagement metrics by offering more diverse, relevant product suggestions.
• Project 2: Redesigned and Built the Next-Generation Marketing Message Generation Platform
• In the second project, I redesigned the marketing message generation platform. The previous system generated multiple messages and selected the best ones, but it lacked personalization. I shifted to a "user-intent-first" model, where only one relevant message was generated based on customer actions.
• I also introduced message-level feedback loops, which allowed the system to adapt messages based on customer interactions, enhancing personalization.
• The architecture used Apache Spark for large-scale data processing and AWS SWF for workflow orchestration. Apache Spark handled large datasets, while AWS SWF ensured efficient workflows.
• I led the design and development with a team of six engineers. Within three months, the new platform significantly outperformed the legacy system, increasing engagement rates.

Oct 2015 – Aug 2017 · 1 yr 10 mos

• Built a scalable framework to generate personalized messages for Amazon customers, delivered via Push and Email channels. The system supports two modes:
• Batch Processing: Leveraging a Spark-based platform, it processes over 100M user actions in batch to generate top-N recommendations per user.
• Real-Time Processing: A high-throughput, low-latency pipeline built using Kafka and an in-memory orchestrator, capable of processing 3M+ real-time events within 5 minutes to trigger push notifications instantly.
• A key challenge was maintaining consistent business logic across both batch and real-time systems. To solve this, we designed a generic interface that abstracts the underlying execution platform—enabling contributors to define processing logic without needing to understand Spark or Kafka. This abstraction empowered 10+ teams across Amazon to build and customize message generation pipelines with ease.

Nov 2013 – Oct 2015 · 1 yr 11 mos

• Working as Software Development Engineer to build solution to process/analyze the tera bytes of Data by Using technologies like Hadoop, AWS EMR, AWS SQS and AWS SWF to design systems
• Worked with BI's to develop the models to predict the customer behavior.