Raghavendra Rao

Software Engineer

Berlin, Germany12 yrs experience

Highly Stable

Key Highlights

Designed scalable cloud-native data platforms.
Led transformation of complex data pipelines.
Implemented governance standards for data compliance.

Stackforce AI infers this person is a Data Engineering expert specializing in cloud-native architectures and big data processing.

Contact

Skills

Core Skills

Cloud-native ApplicationsData EngineeringBig Data Processing

Other Skills

ANTLRAirflowAnsibleApache AirflowApache FlinkApache KafkaApache SparkApache ZookeeperAzure Data LakeAzure Event HubsAzure Kubernetes Service (AKS)Batch ProcessingBigQueryClouderaCore Java

About

● Senior Engineer with extensive experience designing cloud-native data platforms, large-scale real-time systems, and enterprise-grade data pipelines across Azure (including Microsoft Fabric), GCP, and AWS. ● Skilled in building reference architectures, semantic and dimensional data models, and standardizing platform-wide practices for lineage, reproducibility, observability, and CI/CD. ● Adept at integrating streaming and batch workloads using Apache Flink, Spark, Kafka, and Airflow, and at shaping scalable infrastructure through Terraform, Kubernetes, and modern cloud services. ● Known for influencing cross-functional teams, enabling analytics and engineering partners, and delivering resilient, high-impact architectures that scale across data volume, complexity, and business domains.

Experience

12 yrs

Total Experience

3 yrs

Average Tenure

Current Experience

Technis

Senior Data Platform Engineer

Jan 2025 – Sep 2025 · 8 mos · Berlin, Germany · Hybrid

Designed a cloud-native real-time analytics architecture on Azure using Apache Flink integrated with Azure Event Hubs, ADLS, and Azure Kubernetes Service, strengthening the foundation for unified batch and streaming patterns across IoT telemetry and reducing fragmentation in downstream analytics and improving cross-team metric consistency.
Defined an enterprise-ready semantic model for core IoT metrics, ensuring consistent interpretation of device events, state transitions, and computed aggregates across downstream analytics and operational systems.
Established governance standards for the IoT data platform covering lineage tracking, PII-safe payload design, and auditability across Azure Data Lake zones, improving compliance and traceability using Azure-native metadata and logging capabilities.
Partnered with engineering and analytics teams to embed architectural patterns into workflows including standardized pipeline templates, semantic model documentation, and Azure resource governance templates, improving onboarding and platform consistency.

Azure Event HubsApache FlinkAzure Kubernetes Service (AKS)Azure Data LakeData GovernanceCloud-Native Applications+1

Wayfair

Senior Data Platform Engineer

Jan 2020 – Jul 2024 · 4 yrs 6 mos · Berlin Area, Germany · Hybrid

Led and managed the transformation of a 50 TB batch data pipeline from a monolithic Java application to decoupled Airflow DAGs, handling over 300 databases and more than 6,000 tables and increasing the success rate of tables meeting the SLO from 60% to 99.99%. This transformation empowered data analysts and financial teams to generate critical reports swiftly, facilitating faster decision-making.
Developed and maintained a Near Real-Time Data Pipeline (NRTDP) with latency under 60 minutes, replicating SQL tables to BigQuery for efficient analytics and decision-making.
Implemented event-driven automation using Kafka and APIs, reducing manual intervention for data source and view management within the pipeline.
Designed and implemented data quality metrics for both batch and near real-time data pipelines using Grafana and Datadog to ensure data completeness, timeliness, and integrity.
Redesigned the entire build and deployment process for the batch data pipeline application using Docker, Kubernetes, and Buildkite, reducing rollout time from days to under an hour.
Led the automation of infrastructure provisioning using Terraform, streamlining the management of Kafka, Spark, Airflow, Kubernetes, and cloud resources across multiple environments. This included defining infrastructure as code (IaC) for Cloud resources, creating Terraform modules to manage clusters, implementing automated scaling policies, overseeing Kubernetes deployments and upgrades ensuring high availability, and maintaining system security and compliance

AirflowKafkaBigQueryDockerKubernetesTerraform+2

Target

Senior Data Enginner

Aug 2014 – Jan 2020 · 5 yrs 5 mos · Bengaluru Area, India · On-site

Implemented custom ORC parser for processing huge volumes of data that utilizes MapReduce programing paradigm to parse nested data from complex JSON/XMLs from multiple Kafka feeds. As part of peak performance testing, processing of 1.1TB volume of data (150M Records) took less than 15mins.
Reduced significant time required for migrating traditional ETL/ELT jobs in Teradata to Hadoop by developing an automated solution that used customized opensource module HPLSQL & ANTLR Parser for converting Teradata SQL to Hive HQL.
Implemented web-based solution for ingesting data from multiple RDBMS Systems including Teradata, IBM DB2 & Oracle to Hadoop. This solution helped reduce the significant time required for analysts to ingest data in Hadoop.
Implemented REST API’s for exposing backend data stored in Postgres and Cassandra database for end user applications.
Orchestrated development, integration & testing of multiple complex data pipelines in Hadoop environment.
Implemented custom Hive UDF’s UDAF’s & UDTF’s for data processing needs.

MapReduceHadoopHiveQLREST APIsPostgreSQLData Engineering+1