Shivam Dubey

Software Engineer

Varanasi, Uttar Pradesh, India8 yrs 2 mos experience
Most Likely To SwitchHighly Stable

Key Highlights

  • Experienced in building data pipelines at Goldman Sachs.
  • Currently developing products at Google Search.
  • Strong background in data management and engineering.
Stackforce AI infers this person is a Data Engineer with experience in Fintech and software development.

Contact

Skills

Core Skills

Data EngineeringData Management

Other Skills

APIsAlgorithmsCompetitive ProgrammingDB2Data PipelineFlinkFootballHTMLIntegration Testing FrameworkLinuxParquetProblem SolvingProgrammingPythonSpark

About

I'm working as a SWE in Google Search, building the most used product on planet. Prior to this, I worked for 2 years at Goldman Sachs as part of Data Platforms. Apart from work, I love playing football, badminton, table tennis. I enjoy writing too, do checkout https://bit.ly/shivam-dubey for my work. Connect with me on https://topmate.io/shivam07 for mentorship!

Experience

Google

Software Engineer

Aug 2021Present · 4 yrs 7 mos · Bengaluru, Karnataka, India

Goldman sachs

2 roles

Analyst

May 2019Aug 2021 · 2 yrs 3 mos · Bengaluru Area, India

  • Key contributor in developing a Flink based data pipeline, which acts as the only source of data inflow to our stack in the new world. Challenges
  • were to ensure fault tolerance, recoverability, hierarchy buildup for the stream of events wherein a child may appear before the parent event.
  • Developed a user friendly Integration Testing Framework for the Flink EDM Pipeline discussed above, which acts as a crucial piece for a quick
  • debugging and certifying any change across the complete pipeline.
  • In complete ownership of an EOD Spark job which generates optimized Parquet files, exposed to end users via Dremio and used in critical
  • reporting. Challenges were to design an efficient way to retrieve overall latest versions, where parquet files are internally partitioned based on
  • rundate & an order may span for multiple days.
  • Designed an architecture across our DB2 store to enforce Fine Grained Acces Control by tweaking data model, setting up jobs & APIs to get
  • credentials per user, adding views/indices as per use case
FlinkData PipelineIntegration Testing FrameworkSparkParquetDB2+3

Summer Analyst

May 2018Jul 2018 · 2 mos · Bengaluru Area, India

  • Worked as a part of Equity Tech Data platform team.

Indian institute of technology(bhu),varanasi

Teaching Assistant

Aug 2017Nov 2018 · 1 yr 3 mos · Varanasi Area, India

  • Involves assisting freshers with the Introductory Programming course on C and Linux.
  • Conducting Lab sessions and examinations.

Education

Indian Institute of Technology (Banaras Hindu University), Varanasi

Bachelor’s Degree — Computer Science

Jan 2015Jan 2019

Delhi Public School Varanasi

All India Senior School Certificate Examination — CBSE

Jan 2013Jan 2015

Delhi Public School Varanasi

All India Secondary School Examination — CBSE

Jan 2013Jan 2013

Stackforce found 100+ more professionals with Data Engineering & Data Management

Explore similar profiles based on matching skills and experience