Shivam Gupta

Senior Software Engineer

Hyderabad, Telangana, India5 yrs 11 mos experience
Most Likely To SwitchHighly Stable

Key Highlights

  • Expert in developing cloud-agnostic data pipelines.
  • Automated cloud infrastructure deployment, saving significant time.
  • Successfully mentored junior engineers, enhancing team performance.
Stackforce AI infers this person is a Data Engineering and Cloud Infrastructure specialist in the SaaS industry.

Contact

Skills

Core Skills

Data EngineeringCloud InfrastructureSoftware Development

Other Skills

AWS CDKAWS Redshift ServerlessAirflowApache BeamBig Data File FormatCI/CDCloud FormationDelta TablesEMR on EKSStaging Environment SetupVPC Infrastructure

About

Software Engineer with 4+ years of professional experience. Skilled in Software Development and Data Engineering.

Experience

5 yrs 11 mos
Total Experience
5 yrs 11 mos
Average Tenure
5 yrs 11 mos
Current Experience

Qualcomm

3 roles

Senior Software Engineer

Nov 2022Present · 3 yrs 6 mos

  • Pioneering the development of versatile, cloud-agnostic, and data-agnostic data pipelines by researching tools such as Airflow, Apache Beam, EMR on EKS, Delta Tables.
  • Successfully Deployed over 20+ pipeline instances for internal customers, significantly enhancing team recognition for data infrastructure provider.
  • Collaborating with AWS Solution Architect team to investigate the integration of Redshift Serverless with our system, resulting in successful conclusion & acceptance
  • Facilitating effective testing for the team by establishing a dedicated staging environment for pipeline testing.
  • Successfully mentored and led a team of three junior members, providing comprehensive training, guidance, and support. Led them to execute tasks with precision, fostering their growth and contributing to the team's overall success.
AirflowApache BeamEMR on EKSDelta TablesAWS Redshift ServerlessStaging Environment Setup+2

Software Engineer

Jun 2020Nov 2022 · 2 yrs 5 mos

  • Performed Cost analysis & resource optimization for AWS cloud, Saved approx $5K/month.
  • Mentored a Junior Engineer.
  • Performed Disaster recovery drill. AWS CDK implementation allowed us to deploy a complete data pipeline under 20 minutes which takes weeks or months if done manually.
  • Solved a Major bottleneck in scaling due to IP starvation, Solved it by reimplementing VPC Infrastructure.
  • Completely automated Cloud Infrastructure Deployment by adopting AWS CDK to achieve true "Infrastructure is code" status. Infrastructure includes a Data Pipeline that handles data of Petabytes scale ( Billion+ Devices ) along with all existing Infrastructure.
  • Developed a Customized CI/CD from Scratch & integrated 3+ highly customized jobs that are active in production. Saved hours of manual work.
  • Did Comprehensive comparison-based research on Big data file formate selection and got AVRO adopted as primary storage formate.
  • Received appreciation for developing a Cloud Formation script for a notifier based on Lambda for a confidential gov entity.
  • Single-handedly developed, deployed and maintained Fullstack of a web portal for more than 1 year with <1% downtime. The success of the project led to its adaptation to external customers.
AWS CDKCloud FormationCI/CDVPC InfrastructureBig Data File FormatCloud Infrastructure+1

Interim Engineering Intern

Jan 2020Jun 2020 · 5 mos

  • - Built and deployed the initial Phase of a web portal from concept to the production stage. Enabled it for Team wide usage.

Education

Netaji Subhas Institute of Technology

Master of Technology - MTech — Computer Science

Jan 2018Jan 2020

Delhi University

Master of Technology - MTech — Computer Science

May 2018May 2020

Jaypee University of Information Technology

Bachelor of Technology (B.Tech.) — Computer Science

Jan 2013Jan 2017

Stackforce found 100+ more professionals with Data Engineering & Cloud Infrastructure

Explore similar profiles based on matching skills and experience