Piyush Singh

Data Engineer

Pune, Maharashtra, India4 yrs 7 mos experience

Key Highlights

3 years of experience in Azure Big Data Services
Recognized with Engineering Excellence Award for ETL Migration
Proficient in PySpark and Azure Databricks

Stackforce AI infers this person is a Data Engineer specializing in Fintech and Big Data solutions.

Contact

Skills

Core Skills

Data EngineeringEtlMachine Learning

Other Skills

Python (Programming Language)DatabasesAzure Data FactoryData TransformationData IntegrationHadoopApache SparkAlgorithmsDelta TableHiveData WarehousingData ModelingBig DataData PipelinesExtract, Transform, Load (ETL)

About

Data Engineer with 3 years of experience in Azure Big Data Services including Azure Data Lake Gen2, Azure DataFactory, Azure Synapse, and Azure Databricks. Proficient in PySpark, Spark SQL, and Python for transformation logic.Recognized for exemplary performance in critical projects. Certified in Azure (Az-900, DP-900) and skilled in leveraging AI to optimize team workload optimization and innovation distinctive solutions

Experience

4 yrs 7 mos

Total Experience

1 yr 9 mos

Average Tenure

1 yr

Current Experience

Hashedin by deloitte

Software Engineer 2

May 2025 – Present · 1 yr · Pune, Maharashtra, India · Hybrid

Infosys

3 roles

Specialist Programmer

May 2023 – Apr 2025 · 1 yr 11 mos

🎯 Contributed to Leading Indian Bank Data Warehouse ETL Migration project, focusing on Data ingestion , Data Transformation to align with business requirements and Data Integration for downstream consumption
🎯Developed reusable Python utilities to automate common tasks across Azure Databricks Notebooks, employing a compute-optimized and scalable methodology
🎯 Developed Automation Framework to reproduce 4000 Informatica PowerCenter Workflows Logic onto Azure Databricks Notebooks, significantly expediting delivery timelines by 80% and reducing manual efforts by 95%
🎯 Analysed and developed an efficient, parallel data flow for an existing Bank’s Loan Application Data Warehouse. Simplified complex data transformations and integrated additional business requirements
🎯 Supervising daily job triggers in Azure Data Factory to minimize failures, and swiftly resolving issues
🎯 Handled the Release of deliverable to multiple Azure Environment through Azure DevOps and CI/CD
Pipelines
🏅Appreciated with Engineering Excellence Award (Sept 2023) for the work in ETL Migration Project

Python (Programming Language)DatabasesData EngineeringETL

Specialist Programmer Trainee

Feb 2023 – Apr 2023 · 2 mos

HadoopApache Spark

Digital Specialist Engineer

Jun 2022 – Feb 2023 · 8 mos

End to end implementation of automated methods for machine learning use-case
Transforming modular code into object oriented way and implemented various automated analysis methods
Trained in Big Data Technologies and cleared required certification

Python (Programming Language)Machine Learning