Shivam kaushal

Associate Consultant

Gurgaon, Haryana, India9 yrs 1 mo experience

Highly Stable

Key Highlights

Over 5 years of specialized Data Engineering experience.
Expert in building scalable data pipelines and optimizing ETL processes.
Proven ability to translate technical solutions into user-centric products.

Stackforce AI infers this person is a Data Engineer specializing in scalable data solutions and ETL processes.

Contact

kaushalshivam2013@gmail.com LinkedIn

Skills

Core Skills

Data EngineeringEtl ProcessesData AnalysisFrontend Development

Other Skills

AWSAlgorithmsAngular 6Angular MaterialBootstrapCSSCascading Style Sheets (CSS)Data QualityData StructuresETLEclipseFlaskHTMLHTML5JavaScript

About

I am a Data Engineer with 8+ years of total experience, relevant 5 years of specialized expertise in Data Engineering, and Product Development. I have a strong foundation in building scalable data pipelines, optimizing ETL processes, and ensuring data quality across diverse environments. Additionally, my experience in product development and application programming has honed my ability to translate technical solutions into actionable products, collaborating cross-functionally to deliver innovative, user-centric outcomes.

Experience

Pwc india

Senior Consultant

Jan 2022 – Present · 4 yrs 2 mos · Gurugram, Haryana, India

Developed a robust ETL pipeline using PySpark to ingest, clean, and integrate cost data from multiple
vendor sources, processing over 5GB of data daily.
◦ The pipeline was designed to handle cost files in various formats (CSV, Parquet, JSON), ensuring data
consistency and quality across diverse sources.Reduced data processing time by 35%.
◦ Developed and deployed an application that automates the conversion of SQL scripts to PySpark, streamlining
the job creation process on Azure and AWS in Databricks.
◦ Implemented the layers of lambda function in aws which help us to keep all the old functionality.
◦ Automate the process of report sharing of around 1000+ users, among the team members and stack-holders.
which led to cutting down 15% of user intervention. It helped Stack-holder to take decision on time.
◦ Developed the python standalone scripts and schedule them in midnight for each day as result increase
application faster by 80%.
◦ Used aws S3 services to store the csv files, and access the files when required for application using the aws
library boto3.
◦ Successfully implemented email notification feature in aws cloud using the SNS and SES Services.

PySparkETLAWSSQLPythonData Engineering+1

Tata consultancy services

3 roles

Assistant System Engineer

Feb 2018 – Jan 2019 · 11 mos

Provide the solution to create UI and did data analysis from scratch and make proper report.
Design the High Level Design (HLD) and Low Level Design (LLD) documents.
Develop the frontend from scratch using angular 6, angular matrial, Bootstarp3, Html, LESS, CSS3, jQuery.
Integrate the frontend with the backend and check the functionality.
Resolve the bugs.

Angular 6HTMLCSSJavaScriptFrontend Development

Senior Software Engineer

Jan 2017 – Dec 2021 · 4 yrs 11 mos

Developed and managed data pipelines for collecting, processing, and analyzing data from call center systems
using PySpark , ETL & Kafka.
◦ The pipeline will integrate with the call center platforms to capture raw call data such as call start and end
times, wait times, call dispositions (resolved, escalated, etc.), customer satisfaction survey responses, and
agent performance metrics.
◦ Create the SQL query for the application and use them in Python.
◦ Design the APIs in flask for all the operations and build up the logic.
◦ Develop and maintain project documentation for functionalities, APIs, and internal tool process.
◦ Improve existing or inefficient code written earlier without changing its functionality using new concepts leading to increasing performance.

PySparkETLKafkaFlaskSQLData Engineering+1

Assistant System Engineer Trainee

Jan 2017 – Jan 2018 · 1 yr

Work as to collect the data from API's and analyse it and filter the data on the basis of requirement.
Create UI and show the data on it and in table as well.