Mar 2022 – Nov 2024 · 2 yrs 8 mos · Bengaluru, Karnataka, India · On-site

• Designed and implemented scalable and resilient ingestion service on ECS for streaming click events via Java Spring API, utilizing SQS and Kafka for queuing; Java(Camel EIP)/EMR workers for syncing data across multiple databases, efficiently handling 1.5 million RPM.
• Seamlessly integrated Snowflake with Iceberg to optimize workloads across warehouses, significantly reducing costs. Managed end-to-end multi-region, multi-cloud deployments across AWS and Snowflake using Terraform.
• Built an in-house enrichment service powered by LLMs, employing both event-driven and batch using serverless architectures leveraging different AWS services.
• Built task generation and orchestration framework that enabled teams to develop and automate tenant-specific data
• pipeline code with customisations, while abstracting common logic to streamline code management and improve efficiency.
• Developed multi-tenant data pipelines managing over 1100 DAGs using Airflow, orchestrating data transformations, followed by storage into diverse sinks (Snowflake, Hudi, Iceberg) within a lakehouse environment.
• Developed an in-house multi-tenant feature store framework to support delta-based feature generation and feature dependency management, facilitating both offline and online stores, and automating data pipeline feature task creation to enable low-code feature development for the Data Science team.
• Setup low-code transformation setup, supporting multi-tenant e2e transformations, helping team to onboard customers seamlessly. This brought down onboarding time from a week to a day.
• Implemented observability setup using Grafana LGTM stack to monitor different systems health.
• Designed and implemented a dynamic (full segmentation) query generator by parsing JSON inputs, operating on a STAR schema data model with fact and dimension tables.
• Integrated an event-driven reverse ETL system, maintaining various connectors, pushing data to multiple destinations across tenants.

Nov 2020 – Mar 2022 · 1 yr 4 mos · Hyderabad, Telangana, India

Jun 2020 – Sep 2020 · 3 mos

• Worked on multiple product based software.
• My responsibilities included:
• Building/managing multiple microservices(NodeJS based) for different modules of the product.
• Creating/managing REST/GraphQL APIs for product services.
• Creating dynamic/reusable components and state management using frontend libraries such as ReactJS/ReduxJS.
• Defining and structuring database data model for multiple products, involving both SQL and NoSQL types.
• Implementing multi-tenancy architecture for a product, which aims to coordinate between multiple databases through a single microservice.

May 2020 – Jun 2020 · 1 mo

Dec 2018 – Feb 2019 · 2 mos · Banglore

• Worked on developing dynamic web pages using react/redux and different APIs
• Developed a prediction model which will estimate the cost of process, using scikit-learn and different python libraries.
• Contributed to the extensive react components design and development, delivering expected features on time