Kushagra Agarwal

Software Engineer

San Francisco, California, United States11 yrs 5 mos experience
Highly Stable

Key Highlights

  • Achieved 100% GDPR compliance across datasets.
  • Automated workflows saving 20+ engineering hours/month.
  • Developed self-service predictive analytics tools.
Stackforce AI infers this person is a Data Engineering expert in SaaS with a focus on data pipeline automation and compliance.

Contact

Skills

Core Skills

Data EngineeringData Pipelines

Other Skills

Apache SparkPythonSQLDatabasesDatabase DesignBuild Strong RelationshipsAirflowData Build Tool (DBT)DBTRedshiftData ArchitectureData QualityAWS Transfer FamilyAmazon QuickSightAWS Glue

About

As a Data Engineer II at Amazon, I bring over five years of expertise in building and automating scalable data pipelines, leveraging tools like DBT, Airflow, and Redshift. My work has supported critical initiatives, such as achieving 100% GDPR compliance across datasets and automating workflows to empower stakeholders with self-service predictive analytics, significantly enhancing operational efficiency and decision-making flexibility. At Amazon, I contributed to sustainability efforts by automating data transfers for EV incentive programs, showcasing a commitment to impactful, innovative solutions. With a focus on collaboration and process optimization, I am dedicated to enabling teams to enhance their data infrastructure, ensuring alignment with organizational goals while driving measurable results.

Experience

11 yrs 5 mos
Total Experience
2 yrs 2 mos
Average Tenure
6 mos
Current Experience

10x genomics

Senior Data Engineer

Nov 2025Present · 6 mos

Apache SparkPythonSQLDatabasesDatabase DesignBuild Strong Relationships+4

Amazon

Data Engineer II

Jun 2020Nov 2025 · 5 yrs 5 mos · San Francisco Bay Area · On-site

  • Amazon One Medical – Developed the automated the process of updating the internal Data Lake by leveraging
  • DBT documentation, eliminating a manual workflow and ensuring 100% documentation coverage.
  • Implemented and maintained robust monitoring processes for data warehouse infrastructure, ensuring
  • continuous GDPR and OnDemand Data Deletion compliance, increasing compliant Redshift tables from 20%
  • (45) to 100% (185)
  • Developed an automated StarRating Prediction application, allowing stakeholders to directly generate
  • predictions without relying on the team, saving approximately 20 hours per month and increasing flexibility
  • Amazon Sustainability Goals – EV Charging Portfolio
  • o Automated EV incentive program data transfer for utilities via SFTP server (using AWS Transfer Family), saving
  • 20+ engineering hours/month, secure achieving of files, and saving $2MM+ on opex/capex.
  • o Implemented SST, a real-time Amazon QuickSight solution, to provide Siemens charger status updates within
  • Amazon charging network.
  • o Data Architected and built a data warehouse and Spark SQL pipelines capable of handling the increasing data
  • volume from EV chargers, scaling from 20 to 20,000 chargers within 1.5 years.
  • o Automated the Cost Reporting tool, enabling customers to access dashboards instead of relying on email
  • reports, saving 10 hours/week and improving user experience
  • o Implemented data quality/testing framework to detect errors within data received from third-party providers,
  • ensuring the reliability and accuracy of our insights.
  • o Developed Amazon QuickSight dashboards tailored to support Weekly Business Reviews, Monthly Business
  • Reviews, providing leadership with actionable insights to drive strategic decision-making.
DBTAirflowRedshiftPythonSQLData Architecture+5

Facteus (formerly arm insight, inc.)

Data Engineer

Feb 2019Jun 2020 · 1 yr 4 mos · Portland, Oregon Area · On-site

  • Migrated the existing DWH solution of client from Oracle to Snowflake and implemented the data access model of the applications to connect to Snowflake DB
  • Built AWS Glue jobs to output data from Redhsift cluster to S3 datalake and saved 70% of cost of the client
  • Built the data pipelines to ingest data from the S3 buckets to the Snowflake after doing necessary transformations
  • Developed reporting queries and reporting tables in the Google BigQuery for the reports
PythonSQLData ArchitectureData PipelinesAWS GlueBigQuery+1

Maq software

Computer System Analyst

Jul 2018Feb 2019 · 7 mos · Greater Seattle Area

  • ● Connects with stakeholders and users of streams to gather, analyze, and optimize requirements
  • ● Manages the offshore team(s) for the delivery of the solutions in timely delivery
  • ● Evaluating Azure services (Databricks, AML, HDInsights) to leverage them in the projects
  • ● Build, modify Power BI reports which source the data from various Azure resources
SQL

Nyu tandon school of engineering

Graduate Assistant

Feb 2017May 2018 · 1 yr 3 mos · Brooklyn

  • ● Communicate and assist prospective students and maintains their information in the Salesforce CRM
  • ● Evaluate competitors, analyzing business practices and suggesting alternative strategies
Python

Maq software

Software Engineer

Mar 2014Jul 2016 · 2 yrs 4 mos · Hyderabad Area, India

  • ● Awarded as best project of the year 2014 by Client’s leadership; Designed backend architecture and wrote SQL stored procedure to make reporting tables that were consumed by front-end team to show data on web pages
  • ● Built Client Marketing System; Developed schema and created reporting database tables to record purchase transactions of client’s customers and reported revenue generated in Power BI dashboard
  • ● Brainstormed with on-site managers and re-designed the entire data consumption flow; Decreased the data processing latency from 8 hours to 2 hours ~ 75% improvements by using Cosmos scripts
  • ● Actively gathered requirements, tested, deployed and documented all the work done in different projects
  • ● Fostered 15-20 new employees who joined the project teams and guided them in their transitory phase from college to industry
  • ● Conducted 3 company-wide workshops on big data technology Cosmos (Big Data); attended by over 25 people
  • ● Developed, tested, debugged and deployed data flow ETLs from source server to client’s on-premise and cloud (Azure) servers
  • ● Organized and conducted weekend sessions for the Summer Interns hired in 2015 for their boot camp
SQLData Pipelines

Cris; a autonomous society under the ministry of indian railways

Summer Intern

May 2013Jun 2013 · 1 mo · New Delhi Area, India

  • ● Built a Train Detention Report in Control Office Application, to automate the process of a section controller to minimize track delay and maximizing track utilization

Education

NYU Tandon School of Engineering

Master of Science (M.S.) — Computer Science

Jan 2016Jan 2018

Vellore Institute of Technology

Bachelor's degree — Computer Science

Jan 2010Jun 2014

Stackforce found 100+ more professionals with Data Engineering & Data Pipelines

Explore similar profiles based on matching skills and experience