Apr 2018 – Nov 2024 · 6 yrs 7 mos · India

• Being an initial member of the organization, Responsible for the development of the data platform including the Data Lake, ETL/Data pipeline, Data warehousing, Dashboarding, Reporting & Data Analytics.
• Responsible for the Design, Development, Deployment & monitoring of the Data Platform.
• Roles & Responsibilities are following.
• 1. Responsible for delivering data engineering, data analyzing & dashboarding solutions to the Media and Entertainment industry.
• 2. Developed a data pipeline that processes more than TBs of compressed data. Initially, we were running workloads on EMR, but the cost and performance were not meeting our expectations. so migrated to the latest tech stacks such as Kubernetes, airflow, docker, and CI/CD. that results in saving the cost of the AWS and performance improvements for spark workloads.
• 3. Design & Data modeling of the redshift data warehousing. I have applied the best practices and optimization techniques of the redshift to reduce the redshift cost, and we were able to achieve better query performance and save the costs of the 2 nodes of ra3.16xlarge.
• 4. Having a good knowledge of the development, optimizing, and tuning of spark applications. The tuning and optimization helped to reduce the required spark resources(Memory & cores).
• 5. Used AWS services extensively such as EMR, EKS, SQS, Redshift, EC2, S3, Kinesis, and other big data services.
• 6. Design and Developed Dashboarding solution for the batch processing and real-time system, using Kibana, Sisense, and Superset.

Nov 2016 – Apr 2018 · 1 yr 5 mos · India

• Responsible for implementing the data pipeline for the product i.e Data shop - the operating system that powers the healthcare, and provides end-to-end solutions to all healthcare organizations like ACOs, HIEs, Health Systems, and Payers.
• 1. Responsible for the development of Data pipelines using Apache Spark
• 2. Responsible for reading and writing data to Hbase/Elasticsearch.
• 3. Responsible for writing queries on Hbase/Elasticsearch
• 4. Responsible for writing a code in Scala

Aug 2015 – Oct 2016 · 1 yr 2 mos · mumbai

• Newgen Software is a globally recognized provider of Low Code Automation Platform for Digital Transformation. Measure work in finance, banking, and BPM.
• Roles & Responsibilities are following.
• 1. Developed the web application using java, spring, and hibernate.
• 2. Design and developed the data models.
• 3. Gathered the requirement of the client and worked on the change request.
• 4. Interacted and coordinated with the client.

Jun 2014 – Aug 2015 · 1 yr 2 mos · mumbai

• Nucsoft Ltd: Works in the Banking domain.
• Roles & Responsibilities are following.
• 1. Developed the web application using java, and servlet.
• 2. Gathered the requirement of the client and worked on the change request.
• 3. Interacted and coordinated with the client.