Shayan Dutta 🇮🇳

Associate Consultant

Kolkata, West Bengal, India5 yrs 9 mos experience

AI ML PractitionerHighly Stable

Key Highlights

5+ years of experience in data engineering.
Expertise in PySpark and AWS cloud services.
Proven ability to enhance data processing efficiency.

Stackforce AI infers this person is a Big Data Engineer with expertise in cloud data solutions.

Contact

Skills

Core Skills

Data EngineeringBig Data

Other Skills

AWS GlueAWS S3Advance Data StructureAgile EnvironmentAirflowAmazon AthenaAmazon CloudWatchAmazon Web Services (AWS)Batch processingCI/CD DeploymentCore JavaData ModelingData StructuresData ingestionData transformation

About

Data Engineer with 5+ years of experience in pyspark development and maintaining Big Data sets by designing Data pipelines and having knowledge on data migration and batch processing job. Having work experience in pyspark, sql, Databricks, AWS cloud services like S3, Glue, Athena, Redshift etc. Proven to offer cost effective solution with continuous learning, exploring and best practices.

Experience

Ey

Consultant - Cloud Data Engineer

Aug 2023 – Present · 2 yrs 7 mos · Kolkata, West Bengal, India · Hybrid

Tata consultancy services

Systems Engineer-Data Engineer

Jul 2020 – Aug 2023 · 3 yrs 1 mo · Kolkata, West Bengal, India · Hybrid

Implemented data ingestion and batch processing pipeline to structure unstructured/semi-structured data, transform data, handled historic/incremental load.
Solved multiple prod defects and enhanced the existing job for process improvement and fast execution.
Having sound knowledge in Unit Testing, Linux, CI/CD Deployment, Project Process flow.

Data ingestionBatch processingData transformationUnit TestingLinuxCI/CD Deployment+3