VIVEK KUMAR

Data Engineer

Delhi, India7 yrs 6 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Expert in data pipeline creation and maintenance.
Proven track record in cloud migration and optimization.
Strong analytical skills with experience in real-time reporting.

Stackforce AI infers this person is a Data Engineer with expertise in Fintech and cloud-based data solutions.

Contact

vivek.k@coinswitch.co LinkedIn

Skills

Core Skills

Data EngineeringCloud ComputingData OptimizationData AnalysisBusiness Intelligence

Other Skills

AWS EMRAWS GlueAirflowAmazon Web Services (AWS)Apache RangerApache SparkAzkabanHadoopHiveJenkinsMapReduceMicrosoft ExcelPythonPython (Programming Language)Report Generation

About

Big Data Engineer with over 4 years of experience in providing data warehousing solutions, creating & maintaining data pipelines and reporting. Strong knowledge of data modelling and storage principles, ETL and data integration. Extensive knowledge in data analysis, product sense and documentation.

Experience

Coinswitch

Data Engineer ll

Jun 2022 – Present · 3 yrs 9 mos

PythonScalaSparkJenkinsAzkabanAWS EMR+2

Paytm

Data Engineer

Mar 2019 – Jun 2022 · 3 yrs 3 mos · Noida Area, India

## PACE (Paytm’s Advanced Compute Engine)
Tech Stack – Python, Scala, Spark, Jenkins, Azkaban -
Migration from On-premise Hadoop Cluster to AWS EMR
Delivering data to our business users 30 percent faster and at 70 percent the cost of our on-premises solutions
Created a common driver which fires spark applications through Livy on EMR
Handles registering schema, manifest and freshness of the registered dataset on Dataset API (Internal Framework)
Supports all types of datasets i.e. Full, Incremental and Append Merge
Used AWS S3 to store and retrieve the data with schema present on Dataset API
Created job flows on Azkaban and CI/CD using Jenkins and job alerts using incoming web hook on slack
Case Study by AWS on our infra migration - https://aws.amazon.com/solutions/case-studies/paytm/
## CDO-Compute
Tech Stack – Python, Scala, Spark, Hive, Jenkins, Azkaban
Migrated all OLAPS/FACTS written in HiveQL into Spark jobs
Optimised existing spark jobs and reduced run time to great extent
Handled Data skewness by implementing salting technique
Used HDFS to store the computed data
Created job flows on Azkaban and CI/CD using Jenkins and job failure/success metrics and push it to Prometheus using statsD for Pager Duty alerts
## T-EMR
Tech Stack – Python, Spark, Hive, Jenkins, Airflow
Migrated all MIS written in HiveQL into Spark jobs
Optimised existing spark jobs and reduced run time to great extent
Handled Data skewness by implementing salting technique
Used HDFS to store the computed data
Created job flows on Airflow and CI/CD using Jenkins and job failure/success metrics and push it to
Prometheus using statsD for Pager Duty alerts

Technical AnalysisReport GenerationTableauData AnalysisBusiness Intelligence

Nearbuy.com

Data Specialist

Sep 2018 – Mar 2019 · 6 mos · Gurgaon, Haryana, India

Systems Update, Technical Analysis, Report Generation & Charting. Analyze the client’s requirements and processes through document analysis and feedback.
Demonstrate excellent troubleshooting and problem-solving skills.
Liaise between business and technical personnel to ensure a mutual understanding of processes and applications.
Created real time dashboard using Tableau for the business team.

TableauData Analysis