Shaurya Rathi

Data Engineer

Bengaluru, Karnataka, India8 yrs 9 mos experience

Highly Stable

Key Highlights

Over 8.5 years of IT experience in data engineering.
Certified Microsoft Azure Data Engineer Associate.
Expert in developing optimized ELT pipelines.

Stackforce AI infers this person is a Data Engineering professional with expertise in Azure technologies and data pipeline development.

Contact

Skills

Core Skills

Data EngineeringAzure Data FactoryAzure DevopsDatabricksIt Administration

Other Skills

Agile MethodologiesAzure SynapseBig DataCode MigrationsCommunicationData AnalyticsData LoadingDecision-MakingGitHubLeadershipLinuxMaintenance ManagementMentoringMicrosoft OfficeOracle E-Business Suite

About

Over 8.5 years of IT experience Worked in CPG Domain, Finance Domain. Microsoft Azure Data Engineer Associate certified (DP-203) Developed dynamic and optimized ELT pipelines to load data from different sources to Azure using Azure Data Factory , Databricks and Azure Synapse • Working experience in Data Factory Activities • Understands and interprets data to derive meaningful conclusions and insights. • Build analytics tools that utilize the data pipeline to provide actionable insights using Databricks notebook. (Language: ScalaSpark and SQL). • Worked with stakeholders to assist with data-related technical issues and support their data infrastructure needs. • Advance knowledge of Azure DevOps & Github • Excellent interpersonal skills with strong analytical and problem-solving skills.

Experience

Shell

2 roles

Senior Data Engineer

Dec 2025 – Present · 3 mos

Data Engineer

Jun 2022 – Nov 2025 · 3 yrs 5 mos

Developed dynamic and optimized ELT pipelines to load data (MC /Non-MC) from different sources to Azure
I understood how to Utilize Azure DevOps and GitHub for CI/CD pipeline setup and management.
Working closely with Technical SMEs and Business Analysts on requirement gathering and providing Technical estimates
for the module.
I worked with Azure Data Factory to orchestrate data pipelines, connecting to multiple sources like SharePoint, On--
Premises .NET Application, GitHub that worked on Pull mechanism.
Transitioned from a Tactical Solution that included Databricks to a Strategic Solution featuring Azure Synapse Analytics
where primary responsibility was notebook conversion, end-to-end Integration, and developing code for new use cases.
Used Scala Spark code for data extraction, transformation, and aggregation.
Worked within Databricks to transform multiple data sources to create a fully functioning delta Lakehouse and worked with
source file types such as CSV, Excel, and Images.
Actively participate in all phases of the software development lifecycle: analysis, technical design, planning, development,
testing, CICD, release, postproduction/escalation support
Guided and mentored new team members to accelerate their development and integration within the team.
Proactively identified opportunities for process improvements, addressing inefficiencies through innovative solutions and
worked on multiple technical Debt items.
Worked on Continuous Improvement to Automate Hierarchy in adding Historical Business Elements with their latest
information which saved $40,000/year on average.
I Worked on Agile projects, updating tasks daily in Azure DevOps. Strictly followed sprint planning and timelines,
assigning and tracking PBIs and tasks.

Azure Data FactoryDatabricksAzure SynapseScalaSparkAzure DevOps+4

Ibm

3 roles

Data Engineer

Promoted

Jun 2021 – May 2022 · 11 mos

Responsibilities:
Worked on different Data Factory Activities
Build analytics tools that utilize the data pipeline
Worked on Tools like Azure Databricks, Azure Data lake, and technologies like Spark, Scala, and SQL.
Worked with stakeholders to assist with data-related technical issues and support their data infrastructure needs.
Reskilled and got Certified as an Azure Data Engineer and got the opportunity to work in a Data Engineering Project.

Azure Data FactoryDatabricksSparkScalaSQLData Engineering