Pulastya Das

Product Manager

Kolkata, West Bengal, India4 yrs 7 mos experience

AI EnabledAI ML Practitioner

Key Highlights

4+ years of experience in Azure Cloud technologies.
Expert in designing scalable data pipelines and real-time processing.
Certified Azure Data Engineer with a strong focus on data solutions.

Stackforce AI infers this person is a Data Engineer specializing in Azure Cloud solutions for the Pharmaceutical and Food & Beverage industries.

Contact

Skills

Core Skills

Azure Data FactoryEtl PipelinesData ProcessingAzure Synapse AnalyticsData Migration

Other Skills

Ab InitioAnacondaApache SparkArtificial Intelligence (AI)Azure Cosmos DBAzure Data LakeAzure DatabricksAzure SQLAzure SQL DatabaseAzure StorageCC++Cloud Data WarehousingData AnalysisData Engineering

About

Data Engineer with 4+ years of experience specializing in Azure Cloud technologies. I have successfully delivered data solutions for industry-leading clients, including a major American pharmaceutical company and a world-leading beverage company in Northwest Europe. My expertise lies in designing, developing, and optimizing scalable data pipelines, real-time data processing solutions, and data warehousing architectures using Azure tools. With a strong foundation in SQL, PySpark, Databricks, Azure Data Factory, Azure Synapse Analytics, Spark Streaming and Python, I am passionate about driving data-driven decision-making through innovative cloud solutions. ------------------------------------------------------------------------------ • Key Skills & Technologies: Azure Data Factory (ADF) Azure Databricks Azure Synapse Analytics Apache Spark & Spark Streaming SQL (T-SQL, PL/SQL) PySpark Python Data Warehousing and Business Intelligence (BI) Real-Time Data Processing Data Lakes & Lakehouses Power BI Data Modeling & ETL Pipelines Azure DevOps & CI/CD Git Version Control API & Kafka Integration ------------------------------------------------------------------------------ • Certifications: DP-203: Microsoft Certified : Azure Data Engineer Associate PL-300: Microsoft Certified : Power BI Data Analyst Associate DP-900: Microsoft Certified : Azure Data Fundamentals Snowflake SnowPro Core Certified Feel free to reach out for collaboration or opportunities in data engineering, cloud solutions, and real-time analytics projects!

Experience

Ey

Consultant - Data Engineer

Dec 2024 – Present · 1 yr 3 mos · Kolkata, West Bengal, India · On-site

Proficient in Azure Data Factory, Snowflake, Python, Scala, Oracle Cloud, and Shell Scripting with expertise in ETL pipelines, big data processing, cloud-based data warehousing, and job scheduling using Tidal. Experienced in processing diverse file formats to ensure efficient data ingestion, transformation, and analysis.
Processing complex insurance claims JSON files using Scala. Also Handling other data formats such as Parquet, PDF, and Excel to capture a variety of data sources.
Utilizing Scala and Apache Spark to transform and clean raw data. Storing structured data into Snowflake for efficient querying and analytics.
Designing and optimizing robust ETL pipelines using Azure Data Factory and custom scripts. Automating the data ingestion process to manage high-volume, multi-format datasets.
Leveraging cloud platforms like Oracle Cloud and Snowflake to manage large-scale data warehouses. Ensuring data architecture scalability and performance optimization for business insights.
Integrating Tidal job scheduling tool to automate and orchestrate data workflows. Coordinating complex batch processes and ensuring timely execution of ETL pipelines.
Implementing modern data architectures that support real-time analytics. Enabling data-driven decision-making through efficient data processing and visualization.

Azure Data FactorySnowflakePythonScalaOracle CloudShell Scripting+4

Tata consultancy services

2 roles

System Engineer - Azure Data Engineer

Promoted

Aug 2023 – Dec 2024 · 1 yr 4 mos · Kolkata, West Bengal, India · On-site

As a Azure Data Engineer, I played a key role in migrating enterprise data for a world-renowned food and beverage company across Canada and Northwestern Europe division by:
Migrating Excel sheets from SharePoint and on-prem warehouse management databases to Azure Synapse Analytics using Azure Databricks, PySpark, and SQL stored procedures.
Building and optimizing ETL pipelines for seamless data transformation, ensuring high performance and scalability.
Automating job scheduling and execution through Azure Data Factory, streamlining data workflows.
Designing data models and mappings to enable Power BI visualizations, providing actionable business insights and enhancing decision-making.

Azure Synapse AnalyticsAzure DatabricksPySparkSQL stored proceduresETL pipelinesjob scheduling+2

Assistant System Engineer - Azure Data Engineer

Aug 2021 – Aug 2023 · 2 yrs · Kolkata, West Bengal, India · On-site

As an Azure Data Engineer, I worked in the migration of an on-premises Data Engineering solution to Azure for a leading American pharmaceutical company, ensuring a scalable and modern cloud architecture by:
Migrating Hadoop, Spark, and Oracle databases (fact and dimension tables) to Azure, leveraging Azure Databricks, PySpark, and Delta Lake as the data warehousing solution.
Refactoring existing Shell Scripts and SQL Stored Procedures used for data transformation into PySpark and optimized SQL Stored Procedures, improving performance and maintainability.
Designing and optimizing ETL pipelines for seamless data ingestion, transformation, and efficient data processing at scale.
Implementing Delta Lake for enhanced data reliability, versioning, and ACID-compliant transactions, ensuring data consistency and enabling real-time analytics.
Automating job scheduling and data workflows using Azure Data Factory, streamlining operations and reducing manual intervention.

HadoopSparkOracle databasesAzure DatabricksPySparkDelta Lake+3