Vishal Chauhan

Senior Software Engineer

Indore, Madhya Pradesh, India4 yrs 11 mos experience

Highly Stable

Key Highlights

Refactored 900+ workflows for enhanced efficiency
Reduced IP lookup pipeline runtime by 25x
Migrated 360 TB of data to Databricks successfully

Stackforce AI infers this person is a Data Engineer specializing in cloud-based data solutions and pipeline optimization.

Contact

Skills

Core Skills

Big DataDatabricksPython

Other Skills

Python (Programming Language)PySparkSQLData GovernanceDelta LakeApache SparkAmazon Web Services (AWS)ScalaMicrosoft Power BIHBaseHiveApache SqoopMySQLHadoopSQL Server Integration Services (SSIS)

About

I'm a results-driven Data Engineer with over 4+ years of experience designing scalable, high-performance data solutions across cloud ecosystems. As a 4x Databricks Certified professional, I specialize in building optimized data pipelines, implementing Lakehouse architectures, and managing large datasets with Python, PySpark, SQL, and Azure technologies. 📌 Highlights from my journey: ✅ Refactored 900+ workflows replacing Synapse objects with Databricks equivalents 📉 Reduced IP lookup pipeline runtime by 25x, boosting performance and efficiency 💾 Migrated 360 TB of data from Snowflake to Databricks 🛠️ Developed internal accelerator tools to automate Databricks job creation 🏅 Recognized with Koantek's Excellence Award & Mindtree's Mastermind Award 🧰 Tech Stack & Tools: 💻 Databricks (SQL, Unity Catalog, Workflows, API) ☁️ Azure (ADF, ADLS Gen2, DevOps), AWS S3 📂 Delta Lake, Streaming Pipelines, Data Governance 🔄 GitHub, Jira, CI/CD I thrive in data-driven environments where innovation, ownership, and continuous learning are core values. I'm passionate about solving complex data challenges and enabling organizations to unlock the full potential of their data. 📩 Contact: vichauhan446@gmail.com 🤝 Let’s connect if you’re working on cloud-first, data-intensive problems or just want to chat about building smart, scalable data platforms!

Experience

4 yrs 11 mos

Total Experience

2 yrs

Average Tenure

10 mos

Current Experience

Impetus

Senior Software Engineer

Aug 2025 – Present · 10 mos · Indore, Madhya Pradesh, India · Hybrid

Python (Programming Language)Big Data

Koantek

Data Engineer

Jul 2022 – Jul 2025 · 3 yrs · Hyderabad, Telangana, India · Remote

● Developed and optimized structured streaming pipelines that process data from external S3 buckets and REST APIs through databricks medallion architecture.
● Integrated new data sources into existing pipelines, ensuring seamless data flow and consistency.
● Improved the runtime for a demographic IP lookup function by 25x, enhancing overall pipeline efficiency.
● Refactored 900+ Informatica workflows by replacing Synapse object references with Databricks equivalents.
● Validated migrated workflows and data pipelines by comparing outputs from Synapse and Databricks
environments to ensure data accuracy and integrity.
● Worked on the development of an internal accelerator tool, automating the creation of Databricks jobs with best practices, leading to streamlined processes and reduced manual effort.

DatabricksPythonPySparkSQLData Governance

Mindtree

Data Engineer

Jun 2021 – Jul 2022 · 1 yr 1 mo · Pune, Maharashtra, India · Remote

● Migrated 360 TB data including SQL queries, tables, and views from Snowflake to Databricks, ensuring
compatibility and performance optimization.
● Validated data across both platforms post-migration to ensure accuracy, completeness, and consistency.
● Leveraged Delta Lake and Unity Catalog in Databricks to enhance data governance and scalability.

DatabricksDelta LakeSQL