Mar 2023 – Apr 2024 · 1 yr 1 mo

May 2022 – Present · 3 yrs 10 mos

• Enabled cross-cloud agent communication by integrating Google Agentspace and AWS Agentcore, and containerizing A2A protocol services in Docker for scalable Kubernetes-based deployment within the enterprise platform.
• Architected and delivered scalable cloud-native data platforms across Azure, AWS, and Databricks, supporting enterprise-grade AI and analytics initiatives.
• Designed and optimised batch, streaming and event-driven pipelines using Kafka, Delta Lake, and Apache Iceberg within Data Lakehouse architectures, alongside Snowflake cloud data warehousing to ensure data reliability, low latency, and scalability.
• Developed real-time feature engineering pipelines and reusable PySpark modules, improving ML model iteration speed by 30% and reducing redundant engineering effort.
• Established secure data governance and lineage tracking frameworks leveraging Unity Catalog and Azure Purview, aligning with enterprise compliance standards.
• Orchestrated complex data pipelines using Apache Airflow, Azure Data Factory, Databricks Workflows, and Argo Workflows established automated CI/CD with GitHub Actions and Azure DevOps for reliable deployment.
• Enhanced deployment consistency and scalability through containerisation and orchestration using Docker and Kubernetes.
• Leveraged Generative AI applications (e.g., Claude, GPT) to automate data summarisation, reducing manual processing by 40% and accelerating insight delivery by 30%.
• Collaborated closely with 10+ data scientists and senior business stakeholders to translate analytical requirements into scalable data architecture solutions.
• Mentored and up-skilled 5+ junior engineers through 1:1 coaching, code reviews, and knowledge-sharing sessions, leading to a 25% increase in team delivery.

Jun 2020 – May 2022 · 1 yr 11 mos

• Developed a reusable data product for pharma clients using Kedro, PySpark, and Python enabling configurable ingestion, feature engineering and modeling via YAML - accelerating use case delivery by 40%.
• Built scalable batch data pipelines for the aluminium manufacturing sector using Azure Data Factory to land multi-source data into ADLS Gen2, transformed data using Databricks with Delta Lake for CDC handling and leveraged Synapse Analytics for enhanced reporting and analytics; orchestrated pipelines via Databricks Workflows and automated CI/CD using Azure DevOps.
• Implemented advanced data governance using Unity Catalog, enabling access controls, lineage tracking and audit logging to ensure compliance and data security.
• Deployed production-grade data pipelines to Kubernetes clusters using Argo Workflows for orchestration following dev-stage validations via Databricks Workflows - enhancing reliability, scalability and observability across environments.

Apr 2019 – Jun 2020 · 1 yr 2 mos

• Built and deployed batch data pipelines using PySpark and Kedro, ingested multi-format data from AWS S3, transforming it and storing curated datasets in PostgreSQL.
• Orchestrated workflows using Apache Airflow ensured reliability through unit and integration testing.
• Containerized workflows with Docker and implemented automated CI/CD using GitHub Actions.
• Collaborated with data scientists and stakeholders to develop reusable feature engineering pipelines, enabling precise HCP targeting and campaign personalization, reducing segmentation time by 30% and improving marketing response rates by 15%.