Shivam Dutta

Software Engineer

Jamshedpur, Jharkhand, India3 yrs 10 mos experience

Key Highlights

Implemented cloud migration reducing data processing time by 87.5%.
Designed automation reducing manual effort for DataScience team by 90%.
Specialized in scalable data pipelines and architectures.

Stackforce AI infers this person is a SaaS-focused Data Engineer with expertise in cloud solutions and data processing optimization.

Contact

Skills

Core Skills

Azure DatabricksEtlData Processing

Other Skills

ADFAlgorithmsAutomationAzure Data FactoryAzure Data LakeAzure Tech StackBack-End Web DevelopmentC (Programming Language)C++Data StructuresDatabase DesignDatabase Management System (DBMS)GitGitHubHTML

About

Software Engineer with 2+ years of experience implementing data solutions on cloud platforms, specialising in designing, building, and maintaining scalable data pipelines and architectures; led daily development team meetings, ensuring timely project delivery by prioritising tasks based on capacity and stakeholder feedback;

Experience

Sigmoid

SDE - Data

Jan 2025 – Present · 1 yr 2 mos

Maq software

3 roles

Software Engineer 2 - Data

Promoted

Sep 2024 – Jan 2025 · 4 mos · On-site

Worked on Azure Tech Stack to migrate from on-prem to cloud.
Worked on Azure Databricks for notebook creation and ADF for orchestration of the ETL process.
Accelerated data processing time in Databricks by 87.5%, reducing from 48 hours to approximately 6 hours by implementing workload distribution and multi-threading.
Reduced the manual effort for the DataScience team by 90% by designing and implementing an automated logical flow to streamline data collection and distribution.

Azure Tech StackAzure DatabricksADFETLData ProcessingWorkload Distribution+2