Shubham Srivastava

Data Engineer

Seattle, Washington, United States12 yrs 6 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Reduced data processing times by 30% at Amazon
Achieved 99.99% data availability for global analytics
Implemented self-service analytics tools for 1,200 users

Stackforce AI infers this person is a Data Engineering expert in SaaS and Fintech industries.

Contact

Skills

Core Skills

Data EngineeringData ArchitectureBusiness IntelligenceIt Business Analysis

Other Skills

AWSAmazon Web Services (AWS)Apache SparkAutomationBusiness AnalyticsBusiness Intelligence ToolsCI/CDData ArchitectsData GovernanceData InfrastructureData ModelingData QualityData StructuresData SystemsData Warehousing

About

At Amazon, I have spearheaded the transformation of data architecture, establishing a robust, data-driven culture that supports strategic decision-making at scale. By architecting and deploying cloud-native AWS data lake solutions, I reduced data processing times by 30% and improved query performance by 45%, significantly advancing analytics capabilities across the organization. Over the past five years, I have specialized in designing and optimizing real-time and batch data pipelines, achieving an exceptional 99.99% data availability that ensures seamless analytics for global operations. My leadership has been instrumental in deploying self-service analytics tools, driving a 25% increase in user adoption among over 1,200 business users. These efforts empower stakeholders to derive actionable insights, streamlining decision-making processes independently. Beyond analytics, I have developed Python-based CI/CD frameworks that automate workflows, reduce deployment time, and elevate engineering standards. I am deeply committed to operational excellence, consistently improving processes, and fostering innovation. By aligning strategic objectives with cutting-edge data solutions, I have enhanced productivity and ensured that the infrastructure scales efficiently to meet Amazon's growing demands. My ability to translate complex technical challenges into impactful solutions has driven measurable success and empowered teams with reliable, high-performance data systems.

Experience

Amazon

4 roles

Principal Data Engineer

Promoted

Oct 2025 – Present · 5 mos

Senior Data Engineer

Promoted

Dec 2019 – Oct 2025 · 5 yrs 10 mos

Data Platform Leadership: Architected scalable AWS data lake solutions, reducing processing times by 30% and enhancing query performance by 45%.
High-Availability Pipelines: Designed real-time and batch data pipelines, achieving 99.99% availability and enabling seamless global analytics.
Empowering Users: Implemented self-service analytics tools, boosting adoption by 25% among 1,200 users and promoting data-driven decision-making.
Collaboration & Strategy: Partnered across teams to translate data requirements into actionable insights and strategic initiatives.
Data Governance: Ensured 100% compliance with rigorous governance frameworks, enhancing data accuracy and security.
Brisk Platform: Led the development of Brisk, reducing data retrieval times by 40% and increasing platform adoption by 70%.
Automation & Optimization: Built automation tools, cutting manual processes by 50% and improving Amazon Flex operations.
Leadership & Innovation: Directed a high-performing team, piloted predictive analytics, and applied machine learning to improve KPIs by 20%, optimizing logistics and capacity planning.

SchemaApache SparkLeadershipTerraformData SystemsData Architects+9

Data Engineer II

Jan 2019 – Dec 2019 · 11 mos

Infrastructure Modernization: Led the migration of Amazon Sponsored Product Ads data platform from Oracle to AWS, reducing costs by 40% and improving scalability.
Distributed Computing: Pioneered the adoption of Spark on EMR, optimizing data pipelines with Parquet and S3, reducing processing times by 35%.
CI/CD Excellence: Built CI/CD pipelines, standardizing deployments and cutting code deployment time by 30%.
Data Quality Innovation: Designed a Data Quality Scoring framework, enhancing revenue analytics accuracy by 15% and enabling reliable insights.
Empowering Teams: Promoted self-service analytics tools, improving efficiency for data consumers by 20%.
Team Enablement: Trained team members in Spark and AWS, ensuring smooth adoption of modern big data technologies.
Collaboration: Partnered with data scientists, analysts, and stakeholders to solve complex data challenges with tailored solutions.

SchemaTerraformData SystemsRole-Based Access Control (RBAC)Data EngineeringPySpark+1

Data Engineer II

Jun 2017 – Dec 2018 · 1 yr 6 mos

Spearheaded the adoption of Spark and distributed computing, optimizing storage and compute through the use of Spark, EMR, and Parquet on S3.
Led the implementation of a Python-based automation framework and drove the adoption of CI/CD practices for data workflow definitions.
Successfully onboarded the team to AWS services, significantly improving engineering standards for data engineering.

Data SystemsData InfrastructureData EngineeringPySpark

Paytm

2 roles

Senior Software Engineer

Promoted

Apr 2016 – Jun 2017 · 1 yr 2 mos

Cloud-Based Data Lake Pioneer: Led the adoption of one of the earliest cloud-based data lake setups using AWS S3 and Parquet, significantly enhancing data accessibility and scalability for analytics.
Scalable Ingestion Framework: Designed and implemented a robust data ingestion framework, enabling seamless processing of hourly datasets and integrating new data sources with minimal manual effort. Real-Time Data Processing: Engineered real-time payment gateway monitoring systems leveraging Spark Streaming, Kafka, and Elasticsearch, enabling a 70% reduction in downtime during transaction anomalies. Cross-Functional Collaboration: Partnered with analytics and product teams to design data platforms that supported diverse use cases, from fraud detection to business performance monitoring.

Data InfrastructureData Engineering

Software Engineer

Jun 2014 – Apr 2016 · 1 yr 10 mos

Hadoop Ecosystem Expertise: Managed the Hortonworks Hadoop-based data platform, optimizing its performance for batch processing and long- term analytics storage.
Data Lake Setup: Spearheaded the migration of analytics workloads from MySQL to Hive, integrating Spark to enable faster, scalable queries. Real-Time Analytics Innovation: Implemented Kafka-based streaming pipelines to capture real-time transactional data, enabling proactive issue resolution and enhanced business intelligence.
Technology Leadership: Introduced and experimented with early versions of Databricks, establishing Spark as a core technology for processing large datasets.
Data Platform Development: Built internal tools for self-service analytics, empowering business teams with direct access to key insights and reducing dependency on engineering resources.

SchemaData SystemsData Engineering

Deloitte u.s. india offices

Business Technology Analyst

Aug 2013 – May 2014 · 9 mos · Hyderabad Area, India

ETL Development Expertise: Designed and implemented complex ETL workflows using Teradata and Informatica PowerCenter to streamline data integration for a leading US-based healthcare client.
Data Warehousing Optimization: Engineered efficient data pipelines, optimizing data extraction, transformation, and loading processes, which reduced processing times by 20%.
Regulatory Compliance Implementation: Ensured strict adherence to US healthcare regulations by designing workflows that met industry compliance standards, enhancing data security and governance.
SQL Query Optimization: Authored and optimized advanced SQL queries for analyzing large datasets, improving the accuracy and performance of critical business insights.

SchemaIT Business Analysis