Jul 2025 – Feb 2026 · 7 mos · Bengaluru, Karnataka, India · Hybrid

• Designed and developed a high-throughput Apache Flink streaming solution (Java DataStream API) to process ~5,000 TPS of realtime events ingested from Apache Kafka, ensuring low-latency and fault-tolerant processing.
• Tuned Flink job performance through checkpointing, parallelism optimization, backpressure handling, and Kafka consumer configuration, achieving stable throughput at scale with minimal processing lag.

Jul 2023 – Jul 2025 · 2 yrs

• Built and maintained 10+ streaming Dataflow jobs, boosting data ingestion efficiency from Kafka topics to GCS and Pub/Sub by 30%.
• Engineered and deployed egress data notification solutions, setting up streaming Dataflow jobs transferring data seamlessly between Pub/Sub and Kafka.
• Designed and implemented batch Dataflow jobs, transforming semi-structured JSON messages into structured BigQuery tables using Apache beam Java SDK.
• Utilized Airflow to orchestrate and manage 20+ batch job workflows, improving task scheduling and execution efficiency by 35%.
• Migrated a machine learning solution from Cloud Functions to Kubeflow, cutting failure rate by 50% and increasing scaling capability by 60%.
• Refined Terraform code, consolidating 10+ modules into a single module for all Dataflow jobs, resulting in a 40% reduction in infrastructure management time.

Jul 2022 – Jun 2023 · 11 mos

• Architected end-to-end TFX-based MLOps solutions leveraging Vertex AI, which elevated model deployment efficiency by 40% and reduced operational costs by 20%.
• Created and optimized 10+ ELT pipelines using SQL in BigQuery (ELT), enhancing data transformation and loading speed by 35%.
• Built an end-to-end data warehouse solution in BigQuery, with staging tables for raw JSON data, nested structured tables for organized data, and analytics datasets with views for Looker dashboard
• Coded 20+ Terraform scripts from scratch to automate infrastructure deployment and management, reducing deployment time by 45%.
• Developed a Python-based automation tool that generated dummy data, cutting manual effort by 60% and boosting overall efficiency by 50%.
• Created a utility in SQL to test ELT queries, reducing manual testing efforts by 70% and streamlining ELT pipeline validation across different environments.

Jan 2022 – Jun 2022 · 5 mos

• Constructed Dataflow jobs for real-time ingestion of over 500,000 events daily from Pub/Sub to BigQuery, enhancing data accessibility and decision-making for DA teams by 40%.
• Implemented Terraform-based infrastructure, enabling scalable and reproducible environments, cutting deployment time by 50%.
• Collected and analyzed 20+ key performance indicators (KPIs) for various features, enhancing data modeling accuracy and supporting more informed business decisions.