Sep 2023 – Present · 2 yrs 6 mos · Bengaluru, Karnataka, India · Hybrid

• 1. Implemented a Parallel Validation System
• Challenge: Ensuring data consistency across complex pipelines while handling large volumes of
• dynamically changing data.
• Solution:
• Developed a parallel validation system to maintain consistency between source data and
• processed results.
• Ensured the end results accurately matched the source data.
• 2. Led the Migration and Unification of Marketplace Data from MySQL to OpenSearch
• Challenge: Unifying search services across multiple marketplaces (Singapore, Malaysia, Thailand)
• to be powered from a single source, addressing concurrent updates and data
• consistency issues.
• Solution:
• Implemented Redis dynamic locking to handle concurrent updates gracefully.
• Created efficient retry mechanisms to overcome connection drops and ensure the
• latest data is available even with out-of-order updates.
• Utilised a dead letter queue (DLQ) and retry mechanisms in the code to periodically
• process failed updates and retry them automatically.
• Developed a scalable service capable of handling 100,000 document updates per
• minute, with room for further scaling.
• Unification resulted in the 50ms boost in search latency.
• 3. Developing a Real-Time Notification Service
• Current Project: Creating a real-time notification service that alerts users when data falls into their
• selected categories.
• Technologies: Leveraging OpenSearch percolator and RabbitMQ to build a scalable notification
• system.