Shubham D.

Data Engineer

Jaipur, Rajasthan, India1 yr 3 mos experience

AI EnabledAI ML Practitioner

Key Highlights

Managed 15+ production Databricks pipelines.
Achieved 99% data availability through troubleshooting.
Executed data quality validation on over 1M records.

Stackforce AI infers this person is a Data Engineer with expertise in Azure and Databricks, focusing on data quality and pipeline optimization.

Contact

Skills

Core Skills

Data EngineeringCloud

Other Skills

Azure DatabricksData Quality ValidationSQLPySparkDelta LakeCreative Content CreationCreative Content ProductionDatabricksContent ServicesGenerative AIMachine LearningMicrosoft Power BIAzure SQLazure synapsisMySQL

About

Data Engineer with 1+ year of production experience in Azure data pipeline operations and support. Skilled in monitoring and troubleshooting Databricks jobs, data quality validation using SQL and PySpark, and Delta Lake optimization. 🎓 Microsoft Certified: Azure Data Engineer Associate (DP-203) - December 2024 💼 Professional Experience: - Managed 15+ production Databricks pipelines processing 50GB+ daily data - Performed data quality validation on 1M+ records using SQL and PySpark - Troubleshot Spark job failures maintaining 99% data availability - Optimized Delta Lake storage operations (VACUUM/OPTIMIZE) - Executed data reconciliation ensuring 100% source-to-target consistency 🔧 Technical Skills: Cloud: Microsoft Azure, Azure Databricks, Azure Data Factory, Azure Data Lake Storage Data Engineering: PySpark, Spark SQL, Delta Lake, ETL Pipelines, Data Quality Validation Programming: Python, SQL, Shell Scripting Databases: MySQL, MS SQL Server, Azure SQL Database Concepts: Data Warehousing, Medallion Architecture, Schema Design, Star Schema 📍 Working on Azure Big Data and EDW solutions. Databricks Elite Partner environment. 📧 Reach me at: shubham19dangi@gmail.com

Experience

1 yr 3 mos

Total Experience

1 yr 1 mo

Average Tenure

1 mo

Current Experience

Celebal technologies

Data Engineer

Apr 2026 – Present · 1 mo · Jaipur, Rajasthan, India · On-site

Azure DatabricksData Quality ValidationSQLPySparkDelta LakeData Engineering+1

Career break

Professional development

Apr 2025 – Mar 2026 · 11 mos · Bhilwara, Rajasthan, India

I took a planned career break to strengthen my technical skills and transition deeper into data engineering. During this period, I focused on hands-on learning of Python, MySQL, PySpark, data warehousing concepts, Delta Lake, Azure, and Databricks. I practiced SQL-based data validation, worked with Spark transformations, and studied modern data architectures to build a strong foundation for core data engineering roles.

Tata consultancy services

Assistant System Engineer ( Data Engineer )

Jan 2024 – Mar 2025 · 1 yr 2 mos · Ahmedabad, Gujarat, India · On-site

Electronic Components Project | Data Engineer
Managed 15+ production Databricks pipelines processing 50GB+ daily data across Bronze/Silver layers in medallion architecture
Developed and executed data quality validation scripts on 1M+ records using SQL and PySpark to ensure data integrity
Resolved 50+ Spark job failures through log analysis, reducing pipeline downtime and maintaining 99% SLA
Optimized Delta Lake storage operations by running VACUUM and OPTIMIZE commands on 2TB+ data warehouse, improving query performance
Executed data reconciliation checks ensuring 100% consistency between source and target tables during historical loads
Supported schema modification activities for 5+ core tables under senior engineer guidance
Created technical documentation and runbooks to streamline team operations