Shivam Singh

Data Engineer

Pune, Maharashtra, India5 yrs 4 mos experience

Highly Stable

Key Highlights

Expert in developing scalable data pipelines using Python and PySpark.
Strong background in data reliability and efficiency improvements.
Proficient in hybrid cloud management and data integration strategies.

Stackforce AI infers this person is a Data Engineering specialist with expertise in Big Data solutions.

Contact

Skills

Core Skills

Data EngineeringBig Data

Other Skills

AlgorithmsAmazon Web Services (AWS)Apache KafkaC (Programming Language)C++Cascading Style Sheets (CSS)Data AnalysisData CleaningData EfficiencyData EngineerData IntegrationData LoadingData ModelingData QualityData Reliability

About

Python || AWS || ETL Tools || Big Data || Spark || Apache Spark || Hive || SQL || Scala || Apache Kafka || Apache Airflow || Data Integration !!! Data Engineer develop best in class pipelines using Python and PySpark for consumption layers to ensure the best capacity utilization and meet SLAs that effectively reflects business needs and service level requirements Identify and recommended solutions to improve data reliability ,quality and efficiency for new and existing data pipelines. Collaborating with developers and fellow Data engineers to design, builds and deliver a Hybrid cloud managements platform. Develop & deploy connectors to data sources of interest to established end-to-end pipelines including data cleaning and modeling to potentially span in multiple public cloud, private cloud or on premises infrastructure. Collaborating with different teams, develop keys deliverable for data engineering strategy — Scalability Optimization, Operations, Availability, Roadmap .

Experience

Ey global services limited

Senior Analyst

Oct 2024 – Present · 1 yr 5 mos · Pune, Maharashtra, India

Tata consultancy services

Data Engineer

Apr 2021 – Oct 2024 · 3 yrs 6 mos · India

Big Data developer best in class pipelines using Python and PySpark for consumption layers to ensure the best capacity utilization and meet SLAs that effectively reflects business needs and service level requirements Identify and recommended solutions to improve data reliability, quality and efficiency for new and existing data pipelines.
Collaborating with developers and fellow Data engineers to design, builds and deliver a Hybrid cloud managements platform.
Develop & deploy connectors to data sources of interest to established end-to-end pipelines including data cleaning and modeling to potentially span in multiple public cloud, private cloud or on premises infrastructure.
Collaborating with different teams, develop keys deliverable for data engineering strategy — Scalability Optimization, Operations, Availability, Roadmap.

PythonPySparkData ReliabilityData QualityData EfficiencyHybrid Cloud Management+4