Shivam Gupta

Data Engineer

Hyderabad, Telangana, India7 yrs 9 mos experience

Highly StableAI ML Practitioner

Key Highlights

Expert in building scalable ETL pipelines for real-time insights.
Strong command of data engineering tools like Spark and AWS.
Passionate about analytical storytelling and data integrity.

Stackforce AI infers this person is a Data Engineer specializing in SaaS and Generative AI solutions.

Contact

Skills

Core Skills

Data EngineeringGenerative Ai

Other Skills

AWS GlueAirflowAlgorithm DesignAmazon Web Services (AWS)Android DevelopmentApache AirflowApache KafkaBIRT ProjectBackend DeveloperBig Data AnalyticsCC++ChatGPTCloud ComputingComputer Vision

About

Shivam Gupta is a Data Engineer with 7 years of industry experience, including 5+ years specializing in building scalable ETL pipelines and modern data platforms. He has a strong command of Spark/PySpark, Python, AWS, Snowflake, Airflow, and Databricks, along with hands-on expertise in optimizing data workflows, improving data quality, and architecting cost-efficient solutions at scale. His work spans ingesting and transforming millions of records, orchestrating complex pipelines, enabling analytics for 100K+ users, and integrating data with LLM-powered systems for real-time insights. Key Projects: Built large-scale ETL pipelines (10M+ records) powering LLM-driven real-time insights through QBot, improving context-aware responses for enterprise users. Designed and transformed 5M+ row datasets to fix missing values, outliers, and duplicates, boosting downstream data integrity and reporting accuracy. Developed an ad-break detection system using OpenCV, a RASA chatbot, and a genetic algorithm–based timetable scheduler, showcasing versatility across AI, automation, and optimization. Beyond engineering, Shivam enjoys solving algorithmic challenges, experimenting with computer vision, and building automation scripts. He loves exploring side projects - ranging from ad-break detection using OpenCV to AI chatbots with RASA, and even genetic-algorithm-based scheduling systems. He’s also passionate about analytical storytelling, dashboard design, and bringing clarity to messy datasets.

Experience

Jpmorganchase

Data Engineer III

Nov 2025 – Present · 4 mos · Hyderabad, Telangana, India · Hybrid

Alter domus

Data Engineer II

Oct 2025 – Nov 2025 · 1 mo · Hyderabad, Telangana, India · On-site

Product Development

Quantum strides llc

Data Engineer II

Apr 2022 – Oct 2025 · 3 yrs 6 mos · Herndon, Virginia, United States · Remote

As part of the innovative team at QS, I am leading data engineering efforts to integrate Generative AI with TRIRIGA through QBot, our intelligent assistant designed to enhance user interactions within integrated workplace management systems. I have developed scalable data pipelines that ingest, process, and structure large volumes of TRIRIGA data, enabling seamless integration with Large Language Models (LLMs) for real-time insights and support. These pipelines power QBot’s ability to understand complex queries and deliver context-aware responses tailored to user needs. Additionally, I designed algorithms to optimize the TRIRIGA upgrade process by packaging thousands of objects into compressed, efficient deployments. I have also implemented REST API integrations to connect TRIRIGA with external systems such as financial platforms and contributed to UX-driven tools that improve data accessibility and system usability for workplace operations

Generative AITRIRIGALarge Language Models (LLM)Data PipelinesREST APIData Integration+1

Cmex management private limited

2 roles

Technical Consultant

Promoted

Jan 2021 – Apr 2022 · 1 yr 3 mos

Collaborated with business stakeholders and accountants to develop a plan for integrating multiple ERP software systems, allowing for seamless financial flow across Scandinavia and Poland.
Presented product demos to senior leadership to showcase the proposed architecture, gaining buy-in and support for the project.
Led a Scrum team throughout the software development life cycle, ranging from requirement analysis to issue debugging, ensuring successful project delivery.
Developed and implemented relevant algorithms to migrate financial data between two ERP software systems using MS Excel and VBScript, resulting in improved data accuracy and integrity.

Associate Consultant

Jan 2020 – Jan 2021 · 1 yr

Tata consultancy services

2 roles

Assistant System Engineer

Jun 2019 – Feb 2020 · 8 mos · On-site

Designed and developed customized Integrated Workspace Management Systems' data models to meet client requirements, resulting in improved system functionality and data accuracy.
Utilized a business intelligence reporting tool to streamline data tabulation, reducing delivery time by 75%.
Developed integrations to configure demographic data using various web services, improving data accuracy and efficiency.
Collaborated with multidisciplinary teams to ensure the timely delivery of financial data to clients, achieving scheduled milestones and ensuring customer satisfaction.

Assistant System Engineer Trainee

Jun 2018 – Jun 2019 · 1 yr · On-site

Customized synchronous and asynchronous workflows, forms, and queries to meet business requirements, improving overall system functionality and efficiency.
Performed data migration using data integrator, ensuring the accuracy and integrity of migrated data.
Customized various out-of-the-box (OOTB) BIRT disclosure reports and developed new BIRT reports from scratch, utilizing both TRIRIGA queries and SQL queries, resulting in improved reporting capabilities.
Led proof-of-concept (POC) efforts to convert classic lease creation modules and other dependencies to mobile apps, improving system accessibility and user experience.