Rahul Agarwal

VP of Engineering

Bengaluru, Karnataka, India15 yrs 10 mos experience
Most Likely To SwitchAI ML Practitioner

Key Highlights

  • 17+ years in Data Engineering and AI leadership.
  • Drove $2M in annual savings through cloud optimizations.
  • Led global teams to deploy scalable data platforms.
Stackforce AI infers this person is a Data Engineering and AI expert in the SaaS industry.

Contact

Skills

Core Skills

Data EngineeringArtificial Intelligence (ai)Data Quality

Other Skills

AI AutomationAgile MethodologiesApache SparkApache Spark StreamingApache TezBig DataBusiness Intelligence (BI)Cloud PlatformsCore JavaCustomer Relationship ManagementData AnalysisData GovernanceData MigrationData ModelingData Strategies

About

As a senior leader in Data Engineering and AI, I bring 17+ years of experience delivering enterprise-grade data platforms, ML solutions, and GenAI automation across global markets. At Walmart, I’ve led hybrid teams of data engineers and scientists to deploy scalable systems — from Spark-based data lakes to LLM-powered conversational agents. My recent work includes building a RAG-based GenAI chatbot for Walmart’s International Data Lake using LangChain and OpenAI, and an Agentic AI that auto-generates ingestion prompts and discovers data feeds. I also developed a GenAI-powered SQL validator using Spark logs, reducing QA cycles by 70%. With deep expertise in cloud platforms (GCP, Azure), MLOps, Airflow, CI/CD, and data governance, I specialize in scaling systems, optimizing cost ($2M+ savings), and aligning platform roadmaps with business OKRs. I’m passionate about mentoring teams and driving AI innovation that translates into real business impact. Key Highlights from My Profile: Strategic Vision: I've made impactful contributions to annual budgeting, LRAP, and resource planning, while effectively prioritising goals for the International Data organisation. Customer 360 platform (30% CRM targeting improvement) Collaborative Excellence: My collaborations span cross-functional partnerships, professional services, analytics teams, and engineering groups. Together, we've crafted, designed, and executed cutting-edge solutions that drive innovation. Customer-Centric Innovation: Guiding the design and execution of a revolutionary Omni-channel product, the Unified Customer View, across three countries, I've not only elevated technical excellence but also mentored engineering associates in the realms of technology, analytics, and business skills. Cloud Migration Mastery: In devising and implementing intricate GCP and Azure IaaS/PaaS solutions, I've facilitated the seamless migration of data from on-premises systems to GCP/Azure across seven international markets. Streamlined Data Flow: I've engineered a robust streaming framework, ensuring real-time data ingestion into the Data Lake. This framework introduces a multitude of source/target connectors, transformations, reconciliation, and cutting-edge features. combine 2 sections Efficiency Pioneer: My contributions extend to the design and development of an advanced data ingestion CI/CD/DevOps solution, effectively optimising the efficiency International Lake pipelines. GenAI Automation – RAG chatbot + Agentic AI for International Data Lake GenAI Data Validation Assist – SQL auto-generation from Spark logs

Experience

15 yrs 10 mos
Total Experience
5 yrs 3 mos
Average Tenure
10 yrs 2 mos
Current Experience

Walmart global tech

3 roles

Senior Engineering Manager – Data Engineering & AI

Promoted

Jul 2020Present · 5 yrs 10 mos

  • Project - Intl Data Management
  • Led hybrid global teams (30+ engineers, scientists) across 7 international markets to scale data platforms and GenAI capabilities.
  • Built GenAI chatbot using OpenAI + LangChain (RAG pattern) for Walmart’s International Data Lake, enabling conversational access to operational and platform data.
  • Developed Agentic AI to automate ingestion prompt generation and identify missing data feeds, reducing onboarding time by 60%.
  • Created GenAI Data Validation Assist that generates SQL from Spark logs to validate pipeline outputs, reducing QA effort by 70%.
  • Migrated 5000+ data pipelines from Hadoop to GCP + Airflow with full automation.
  • Improved CRM targeting by 30% through unified Customer 360 identity platform.
  • Drove $2M in annual savings via Dataproc and BigQuery cloud cost optimizations.
  • Championed MLOps stack using MLflow, Databricks, and SageMaker for model lifecycle management.
  • Leading a team of 30+ global vendor partners, overseeing L1 and L2 support for 5000+ batch and streaming data pipelines across 7 international markets
  • Led a team of 6 data engineers to create innovative data ingestion frameworks, reducing development efforts by 50%. Developed custom batch and real-time streaming solutions for global data domains with diverse source and target needs
  • Spearheaded the migration of legacy data ingestion pipelines to Google Cloud Platform, resulting in a 40% decrease in infrastructure costs
  • Driving cloud cost efficiency strategy for International Data Lake, resulting in approximate $2M savings through optimisation of storage and compute costs
  • Project - Unified Customer View
  • Led the implementation of customer Omni channel product deployment across ASDA, Mexico, Canada, and Central America, bolstering Walmart’s global single sign-on via "Bill-64" Data Protection Framework
Data EngineeringArtificial Intelligence (AI)MLOpsCloud PlatformsData GovernanceApache Spark+1

Senior Data Engineer

Mar 2018Jul 2020 · 2 yrs 4 mos

  • Intl Data Lake -
  • Built scalable Spark-Scala pipelines to enhance ML models with a dynamic markdown system and international promotion effectiveness.Developed full consumption layer for promotion reports.
SparkScalaData Engineering

Data Engineer

Mar 2016Mar 2018 · 2 yrs

  • Data Quality and Governance
  • Designed and implemented Talend infrastructure, enabling rule migration from Global ID, resulting in $1M license savings at Walmart
TalendData QualityData Governance

Capgemini

Senior Data Engineer

Feb 2014Mar 2016 · 2 yrs 1 mo · bangalore

  • Senior Data Engineer
  • Led a team of 4 data engineers to develop data pipelines (using Spark, Hive, Talend, Informatica) for HPI solutions, established CICD dev Ops, and enhanced data quality/performance, benefiting HP Ink’s TB-scale reporting solution
SparkHiveTalendInformaticaData Engineering

Infosys

2 roles

Senior System Engineer

Promoted

Dec 2012Feb 2014 · 1 yr 2 mos

Software Engineer

Jun 2010Nov 2012 · 2 yrs 5 mos

Education

Babu Banarsi Das University, Lucknow

Bachelor of Technology - BTech — Computer Science

Jun 2006Jun 2010

Babu Banarsi Das University, Lucknow

Bachelor of Technology - BTech — Computer Science

Apr 2006Jun 2010

Stackforce found 100+ more professionals with Data Engineering & Artificial Intelligence (ai)

Explore similar profiles based on matching skills and experience