Mahasweta Pal — Associate Consultant

With over 6.8 years of experience in the IT industry, I specialize in Azure Databricks, data engineering, and Unix/Linux-based automation. I have strong expertise in Databricks notebook development using PySpark, Scala, Python, SQL, and Spark SQL, building scalable data pipelines and processing large datasets using Apache Spark. During my tenure at Accenture, I leveraged my Unix development and SQL expertise to design automation solutions, develop shell scripts, and support data processing workflows. I built and maintained 100+ Unix shell scripts for data ingestion, transformation, monitoring, and operational automation, significantly reducing manual effort and improving system efficiency. Currently at Deloitte, I work as an Azure Databricks Engineer, developing Databricks notebooks and Azure Data Factory (ADF) pipelines to build scalable data processing solutions. My work focuses on distributed data processing, ETL pipeline development, and optimizing Spark workloads to ensure reliable and high-performance data platforms. My B.Tech in Information Technology from the Institute of Engineering and Management, Kolkata provides a strong foundation in computing and data systems, which I apply to design efficient, scalable solutions and manage complex data workflows across enterprise environments. I am passionate about leveraging modern cloud data technologies and automation to deliver impactful data solutions that support business insights and operational excellence.

Stackforce AI infers this person is a Data Engineering specialist with strong expertise in cloud technologies and automation in the IT sector.

Location: Bengaluru, Karnataka, India

Experience: 6 yrs 9 mos