Shubham Gupta

Data Engineer

Pune, Maharashtra, India9 yrs 5 mos experience

Highly Stable

Key Highlights

7+ years of experience in data engineering.
Expert in building scalable ETL pipelines.
Recognized for technical excellence at BP.

Stackforce AI infers this person is a Data Engineering expert in the Energy sector.

Contact

Skills

Core Skills

Data EngineeringData PipelinesData Analytics

Other Skills

Data ModelingAmazon Web Services (AWS)Apache SparkKafkaAWSCCoding LanguagesProgram ManagementSQLSoftware Project ManagementAgile MethodologiesC++JavaSoftware DevelopmentMySQL

About

Data Engineer with 7+ years turning complex data challenges into scalable solutions. I master technologies like Apache Kafka, Spark, Snowflake, and AWS to build pipelines that power real-time analytics. Recently awarded for technical excellence at BP, where I design ETL systems while learning Generative AI to stay ahead of the data revolution.

Experience

9 yrs 5 mos

Total Experience

1 yr 10 mos

Average Tenure

1 yr 10 mos

Current Experience

Bp

Senior Data Engineer

Jul 2024 – Present · 1 yr 10 mos · Pune, Maharashtra, India · Hybrid

Built scalable ETL pipelines using Apache Spark, Kafka, and AWS services to process and transform large-scale energy sector data.
Architected data solutions that enable real-time analytics and reporting for critical business operations
Optimized data workflows to improve processing efficiency and reduce operational costs
Collaborated with cross-functional teams including data scientists and business analysts to deliver actionable insights
Actively upskilling in ML and GenAI technologies including AWS SageMaker and Bedrock to integrate AI capabilities into data pipelines
Recognized with "Play to Win" award from the product team for exceptional technical contribution and delivery excellence

Data EngineeringData PipelinesData ModelingData AnalyticsAmazon Web Services (AWS)

Mindtickle

2 roles

Software Development Engineer 3

Feb 2024 – Jul 2024 · 5 mos

Software Development Engineer 2

Feb 2021 – Feb 2024 · 3 yrs

Designed and developed a platform for real-time analytics
Developed and optimized a lot of ETL jobs for the Data Warehouse Team
Led and delivered multiple projects to provide actionable Analytics
Built a dashboard to check the performance of various Analytics APIs
Achieved a 25% reduction in AWS Cost for the team
Added Automation testing in legacy code using Postman
Mentored 10 Software Developers and increased engineering throughput

Directi

2 roles

SDE 2

Jul 2018 – Jul 2019 · 1 yr

Worked with the Big Data Team to create new and maintain existing pipelines
Developed a big data querying tool in JAVA using Apache Spark
Provided data fetching and visualization for both internal and external users
Developed various Permission and Authentication layers for Big Data products

Platform Engineer

Jul 2016 – Jul 2018 · 2 yrs

Worked on the API Development and Database Design for a B2C email marketing platform
Developed SMTP layer and added 3rd Party integrations for sending 1M+ emails/day

[24]7

Intern, R & D-Insights

Jul 2015 – Dec 2015 · 5 mos · Bengaluru Area, India

Worked on a data indexing project with the Batch Data Platform(BDP) team at [24]7 Innovation Labs.

Infibeam.com

Summer Intern

May 2014 – Jul 2014 · 2 mos · Ahmedabad Area, India

Building a saved item list widget for the Infibeam's website using AngularJS and Ruby on Rails.
Building a chrome extension for Angel API which was a client to Infibeam.com

Bits pilani student union

Student Representative

Aug 2012 – May 2013 · 9 mos

● SPOC for 250+ residents to ensure resolution of all hostel-related problems, and educate hostlers on important communications from institute heads
● Students’ representative for voicing decisions regarding the working of various clubs and departments in the institute
● Helped in organizing the technical and cultural fest for BITS Pilani for the year 2012-13