Ashwatth Nagpal

Senior Software Engineer

Delhi, India6 yrs 10 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Led high-impact data initiatives at Uber.
Designed next-gen data lakes improving SLA freshness.
Spearheaded anomaly detection saving millions annually.

Stackforce AI infers this person is a Full Stack Data Engineer with expertise in Big Data and real-time analytics.

Contact

Skills

Core Skills

Big DataJavaSoftware ArchitecturePlatform DesignPysparkData AnalysisData EngineeringDistributed SystemsSparkFull Stack DevelopmentWeb Development

Other Skills

Apache SparkApache FlinkKafkaHiveIcebergReal-Time StreamingData LakesObservabilityMachine LearningData QualityScalabilitySQLHigh-Volume DataFlinkAirflow

About

Full stack software developer with high analytical and problem solving skills , a polyglot with experience in various frameworks and languages like Java ,Ruby, JavaScript, Rails ,React and Native. Have used Agile Methodology to develop software and have experience with pair programming and test driven development. Believes in developing scalable software that are easy to maintain and easy to change with client needs and demands.

Experience

6 yrs 10 mos

Total Experience

2 yrs 3 mos

Average Tenure

3 yrs 11 mos

Current Experience

Uber

2 roles

Senior Software Engineer

Promoted

Sep 2024 – Present · 1 yr 8 mos

At Uber, I’ve led and delivered multiple high-impact data initiatives powering global mobility and delivery businesses, focusing on data accuracy, scalability, and observability in fare systems processing 50TB+ data daily. My work has enabled real-time insights, system reliability, and cross-org adoption across Pricing, Finance, and Operations.
Key Accomplishments
1. Fares Data Lake (50TB+/day scale):
Designed and built Uber’s next-gen Fares Data Lake, migrating from Wayfare (trip-based) to Quark (order/job-based).
Partnered with 60+ downstream teams (Pricing, Finance, Mobility, Delivery) to prioritize P0 use cases, solve schema complexity, and deliver SLA-compliant datasets.
Architected a layered model (Kafka → Flink → Spark → Hive/Iceberg) balancing scale, freshness, and usability.
Impact: 70% SLA freshness improvement, faster insights (weeks → hours), and 60+ teams migrated smoothly.
2. Unified Reconciliation Framework (Flink + Java + SQL-driven):
Built a real-time reconciliation platform handling 100M+ messages/day with sub-second latency.
Introduced a SQL-based config layer replacing boilerplate Java, reducing development from weeks → hours.
Impact: 10+ teams onboarded, reconciliation errors detected in minutes (vs 24 hrs), boosting developer productivity.
Leadership & Influence:
Spearheaded defect analysis dashboards, cutting investigation time by 50%.
Tech Stack:-
Big Data | Apache Spark | Apache Flink | Kafka | Hive | Iceberg | Real-Time Streaming | Data Lakes | Observability | Machine Learning | Data Quality | Scalability | Java | SQL | PySpark | Distributed Systems | High-Volume Data (50TB+/day)

Big DataApache SparkApache FlinkKafkaHiveIceberg+11

Data Engineer II

Jun 2022 – Sep 2024 · 2 yrs 3 mos

At Uber, I’ve been privileged to lead and contribute to high-impact projects within the Fares Lifecycle Integrity (FLI) program, helping to enhance fare system observability and accuracy across Uber's global mobility and delivery platforms. Here are some key highlights from my work:
1. Founding Engineer for the FLI Accuracy Track: Spearheaded the vision, design, and architecture for the anomaly detection system, covering 95% of global mobility and 80% of delivery fares. This has reduced fare incident detection time by 83% and saved Uber ~$4M annually.
2. System Scalability & Innovation: Designed and implemented a decoupled framework for the FLI Accuracy system, reducing false positives by over 60%, scaling to monitor millions of metrics daily, and directly influencing the redesign of Uber’s D3 system.
3. Collaboration Across Teams: Worked closely with cross-functional teams (CSP, RSP, fares, operations, driver surge, driver pricing) to deliver enhanced monitoring solutions. The integration of FLI with the Panorama dashboard significantly reduced on-call debugging time and improved system visibility.
4. Operational Excellence: Led efforts to create critical dashboards, such as the Fares KPI dashboard, empowering global operations teams with real-time fare performance insights across regions and products.
5. Innovation through Leadership: Played a core role in mentoring fellow engineers, leading the project from inception to deployment, and presenting at Uber Data Con 2024. My work continues to drive operational excellence and impact at scale.
Looking forward to further driving Uber’s technological excellence and leading transformative projects!

Software ArchitecturePlatform Design

Thoughtworks

Data Engineer - Senior Consultant

Jul 2021 – Jun 2022 · 11 mos · Gurugram, Haryana, India

Projects:-
1) IDFC FIRST BANK
Developed frameworks using PySpark to send out different offers campaign to bank customers
Developed Airflow DAG's to automate manual payload generation process.
Lead various spikes and poc’s to find out best possible solution for problem in hand
Developed complex SQL queries to find out insights about bank customer's know as customer dna, which was used to design specific offers for them.
Assist in management of data flows, monitor and update systems as necessary.
Translated ambiguous business requirements into complex analyses and actionable insights.
Developed expertise in handling large data sets, deep-dive data analysis and real-time data analytics.
Tech Stack:- PySpark, Airflow, Docker, EMR, S3 EC2, Aerospike, Athena, GoCD, Python, PyTest, Git, Kafka

PySparkAirflowDockerEMRS3EC2+8

Guavus

2 roles

Software Engineer

Promoted

Mar 2021 – Jul 2021 · 4 mos

Key Contributions:-
1) Work with cross functional teams and stakeholders to drive requirements. Conducted spikes and poc for clients
2) Experience with crafting and building large scale data pipelines in distributed environments with technologies such as Hadoop, Spark, Kafka, Hive, Elastic search etc.
3) Proven skills in designing, tuning & optimizing scalable, highly available distributed systems which can handle high data volumes.
4) Develop and train junior developers and worked with the team manager and PM in estimating scope and story pointing.
5) Participated in design and implementation of the real time analytics architecture.
6) Experience with real-time streaming distributed data processing.
Projects:-
1) Cricket Churn rate prediction POC
Implemented 3 data pipelines in spark scala over Hadoop cluster to process raw edr data for telco service provider CRICKET.
Data pipeline processed session, http and flow data to compute per subscriber per service KPIs that were used by the analytics team for churn rate prediction.
Pipelines process 10 Million events per day in streaming fashion
2) Customer-IQ
Implemented 3 Data pipelines in spark scala over kubernetes to compute service quality metrics.
QoE (Quality of service) score calculation depends on the service and the metrics computed in these jobs.
Job process 100 Million network events per day.
3) IBS
Developed 8 end to end log aggregation pipelines in SqlStream to process logs generated by different software modules of IBS company.
Pipeline worked in streaming fashion and extracted transaction details for each module.
These details were used to compute avg turnaround time per transaction in the last 15 minutes per module.
Information computed is used for making decisions of scaling up and down.
Tech Stack:- Spark , HDFS, Kafka, Kubernetes ,YARN, Hive, Docker, ELK Stack, Scala, Python, Java, Helm, Git , Druid, Ansible

HadoopSparkKafkaHiveElastic searchDocker+9

Associate Software Engineer

Jul 2019 – Mar 2021 · 1 yr 8 mos

Projects
1) JIO
Developed software to analyze JIO cellular data.
Data included call drop rates, voice quality, internet speed at different times of day as well as youtube video quality experience.
By analyzing this data, companies can improve the quality of their service.
2) OPS-IQ
Developed spark scala Pipelines over kubernetes to analyze network and service data to find anomalies.
Jobs digest gigabytes of network traffic important to routing changes that have an impact on applications.
Jobs proactively compute significant network events that can be easily searched in a major trouble-shooting exercise.
Developed 3 end to end pipelines out of 8 pipelines and contributed in development of other pipelines in significant manner.
Pipeline ingested avro data from kafka and stored processed output in kafka and elastic search.
Wrote SQLStream adaptor jobs to transform raw data schema that matches data platform input schema.
Containerised applications with help of docker to deploy them on Kubernetes cluster with the help of helm and ansible.
Job processed 1 Billion network and user events per day.
3) Verizon Service Degradation POC
Implemented 5 spark scala pipelines over kubernetes to process raw edr flow data for telco service provider Verizon.
Pipeline used to process data in batch as well as streaming manner.
Pipelines used to compute KPI’s per service per subscriber level at different data segmentation to find quality of service experienced by the user.
Also implemented spark scala job to compute sessions from raw edr data to analyze session level experience score for user
Computed session using periodicity as well as concurrency of flows.

SparkScalaKubernetesDockerKafkaElastic search+1

Mavenhive technologies pvt ltd

Associate Intern

Jan 2019 – Apr 2019 · 3 mos · Bengaluru, Karnataka, India

1) Contributed to AI customer support platform Tasha as full stack developer.
2) Used React.js and Ruby in Rails to develop dashboard and backend api’
3) Learned Basics of OOD, pattern design, TDD and pair programming.
Tech Stack:- Ruby on Rails, React.js, Docker, Git, RSpec, JavaScript

Ruby on RailsReact.jsDockerGitRSpecJavaScript+2

Twigz technologies private limited

Android Development Intern

Jun 2018 – Jul 2018 · 1 mo · Gurugram, Haryana, India

A mobile development internship , majorly worked upon react native a cross platform mobile app development platform
Tech Stack:- Android, React Native, Java , JavaScript

AndroidReact NativeJavaJavaScript