Shubham Kumar

Data Engineer

Kolkata, West Bengal, India4 yrs 4 mos experience

Most Likely To SwitchAI ML Practitioner

Key Highlights

Designed scalable data pipelines on Azure Databricks.
Achieved 65% runtime reduction in data processing.
Developed automated frameworks ensuring 99.4% data accuracy.

Stackforce AI infers this person is a Data Engineer specializing in cloud-based data solutions and analytics.

Contact

Skills

Core Skills

Data EngineeringCloud ComputingWeb Development

Other Skills

Amazon Web Services (AWS)Analytical SkillsAnalyticsAndroid Data BindingAndroid StudioArtificial Intelligence (AI)AzureAzure CloudAzure Data Lake StorageAzure DatabricksBig DataBootstrapC (Programming Language)CI/CDCSS

About

I am a Data Engineer with over 2.8+ years of experience designing, building, and optimizing scalable data pipelines using Python, SQL, and cloud platforms such as AWS and Azure. My expertise covers the full data lifecycle, from ETL development and data modeling to transforming raw data into actionable business insights. I have hands-on experience with tools like SSIS, Databricks, PySpark, and Spark UI, and am skilled in advanced SQL, including window functions and set operations. I am passionate about solving complex data challenges, ensuring data quality, and enabling data-driven decision-making. I thrive in collaborative environments, enjoy sharing knowledge, and am always open to connecting with fellow data professionals to drive innovation and business values.

Experience

4 yrs 4 mos

Total Experience

2 yrs 2 mos

Average Tenure

3 yrs 10 mos

Current Experience

Cognizant

3 roles

Programming Analyst

Feb 2024 – Present · 2 yrs 4 mos · Kolkata, West Bengal, India · On-site

Designed and deployed end-to-end data solutions on Azure Databricks, processing over 2TB of data daily from SQL and NoSQL sources.
Optimized PySpark jobs, achieving a 65% reduction in runtime through effective partitioning and caching strategies.
Implemented Delta Lake architecture, ensuring ACID compliance and reducing cloud costs by 40%.
Developed automated data validation frameworks, improving data accuracy to 99.4%.
Built CI/CD pipelines, reducing deployment cycles from 2 weeks to 2 days.

Azure DatabricksPySparkDelta LakeCI/CDdata validationData Engineering+1

Programmer Analyst Trainee

Feb 2023 – Feb 2024 · 1 yr · Kolkata, West Bengal, India · On-site

Big Data Cloud Intern

Feb 2022 – Aug 2022 · 6 mos · Remote · Remote

Project Overview:
∆ Managed and processed large datasets to enhance business intelligence and analytics.
∆ Developed scalable data solutions for seamless integration, transformation, and storage, supporting data-driven decision-making.
✓ Built and optimized ETL pipelines for large-scale data processing.
✓ Enhanced SQL performance with advanced queries and stored procedures.
✓ Managed and analyzed structured and unstructured data using Hadoop, Spark, Data 360.
✓ Ensured high data quality through cleansing and deduplication.
✓ Developed data visualizations and reports for actionable insights.
✓ Collaborated with teams to refine data solutions.
✓ Utilized GCP and Azure for scalable cloud-based data storage and analytics.
> Applied skills in Python, Java, PySpark, Databricks, SQL, Hadoop, GCP, and Azure.

PythonJavaPySparkDatabricksSQLHadoop+4

The sparks foundation

Web Development Intern

Jun 2021 – Jul 2021 · 1 mo

Worked on a website which is used to transfer money between two users and keep a record of their all transaction and activities.
Tools and Technology used:
Editor used : Apache NetBeans
Frontend : HTML, CSS
Backend : JavaScript
Github Link: https://lnkd.in/gBKiS_2

HTMLCSSJavaScriptWeb Development