Niraj Mane

Data Engineer

Pune, Maharashtra, India4 yrs 9 mos experience

Key Highlights

Proven expertise in building scalable data pipelines.
Hands-on experience with AWS services and data engineering.
Strong background in data transformation and management.

Stackforce AI infers this person is a Data Engineer specializing in AWS and data pipeline automation.

Contact

Skills

Core Skills

Data EngineeringAws

Other Skills

AWS DMSAWS LambdaAWS Step FunctionsAmazon CloudWatchAmazon EC2Amazon S3Amazon SQSAmazon Simple Notification Service (SNS)Amazon VPCAmazon Web Services (AWS)AthenaC (Programming Language)C++Data LakeData Lakes

About

Proven 3.5+ years of experience working as data engineer . Programming Language - Python, pyspark Cloud- AWS Work flow management - Stepfunction Database - MYSQL, SQL server DevOps -Terraform, Git,Jira You can always reach out to me on my mail - nirajmane2@gmail.com

Experience

Accenture in india

Data engineering analyst

Apr 2024 – Present · 1 yr 11 mos · Pune, Maharashtra, India · Hybrid

Created automated, robust and scalable data pipeline using various AWS services, pyspark and SQL per business logic.
Extracted data from SNS message for creation of initial json file using lambda
Created glue job to transform initial json to final json structure and then to XML
Created components like Glue, Sns, Sqs, lambda,trigger,crawlers using terraform scripts

TerraformAWSPySparkSQLData Engineering

Tata consultancy services

3 roles

System Engineer

Apr 2023 – Mar 2024 · 11 mos

Created automated data pipeline using aws Stepfunction.
Handled production incidents across all use case.
Created glue job to transform data

AWS Step FunctionsData PipelineGlueData EngineeringAWS

Assistant System Engineer

Apr 2022 – Apr 2023 · 1 yr

worked in agile team of 3 with 15 days of sprint
Extracted data from different source engines like IBM db2, Mysql,oracle etc.
transformed data using complex sql queries and Pyspark script.
maintained raw, trusted, curated layers in datalake.
have hands-on experience on aws stacks such as aws dms, lambda,glue,cloudwatch,eventbridge,EC2,RDS,Stepfunction,S3,athena
Involved in Cost cutting for DMS replication instance.
Created automated data pipeline via aws Stepfunction

AWS DMSPySparkSQLData LakeData EngineeringAWS

assistant system engineer trainee

Apr 2021 – Mar 2022 · 11 mos

completed handson training on hadoop ecosystem
completed python and pyspark training
completed training on BI tool - motionboard
member of datalake team primarily rersponsible for data migration from On premises to aws cloud

HadoopPythonPySpark