Mridul Pant

Data Engineer

Bengaluru, Karnataka, India8 yrs 1 mo experience

Key Highlights

  • Reduced BigQuery costs by 40%.
  • Built scalable data pipelines for real-time analytics.
  • Optimized data workflows for improved system performance.
Stackforce AI infers this person is a Data Engineer specializing in cloud-based data solutions and real-time data processing.

Contact

Skills

Core Skills

Data EngineeringAwsMicroservicesCost OptimizationReal-time Data Processing

Other Skills

AWS GlueAWS Step FunctionsAlgorithm AnalysisAmazon ECSAmazon Elastic MapReduce (EMR)Amazon RedshiftAmazon S3Android DevelopmentAndroid StudioApache AirflowApache KafkaApache SqoopCC (Programming Language)CDC

About

Mridul is an accomplished Data Engineer with over 6 years of experience designing, developing, and optimizing ETL pipelines, data platforms, and real-time data workflows. He possesses a strong command of data engineering tools and frameworks, including Pyspark, Airflow, Kafka, Hive, Hadoop, and AWS cloud services (S3, Glue, EMR, ECS, MSK). His expertise spans multiple aspects of the data engineering lifecycle — from data ingestion and transformation to data warehousing and analytics. He has demonstrated his ability to build scalable data pipelines that process data from diverse sources (APIs, databases, SFTP, and streaming platforms) into modern data warehouses like Redshift, BigQuery, and Snowflake. He has a proven track record of automating workflows and enhancing system performance. He notably optimized BigQuery costs by 40%, reduced SLA times from 24 hours to real-time, and built event-driven microservices to enable seamless data flow. His work has improved system reliability, security, and operational efficiency. Mridul is also skilled in real-time data streaming using Kafka and Debezium to enable CDC (Change Data Capture) pipelines, ensuring low-latency data availability. He has experience in event-driven microservices development using Spring Boot, Docker, and AWS ECS, enabling seamless service integration. With hands-on experience in data security, incident management, and system monitoring (using Loki, Grafana, and Tableau dashboards), he ensures continuous system availability and data integrity. His experience with reverse ETL, CDC, and self-serve ETL frameworks makes him a well-rounded engineer capable of simplifying complex data workflows for analytics and reporting. His technical toolkit includes Python, SQL, Java, Shell scripting, and DevOps tools like Git, Jenkins, and Docker. He has contributed to high-impact projects at companies like British Petroleum, Bukuwarung, Rupeek, and Enquero Global, earning recognition like the Spot Award for extraordinary service.

Experience

8 yrs 1 mo
Total Experience
2 yrs
Average Tenure
--
Current Experience

Bp

Data Engineer

Jan 2024Oct 2024 · 9 mos · Pune, Maharashtra, India · Remote

  • Custom data pipeline to migrate over 200+ tables on AWS
  • Writing AWS Data Pipelines leveraging glue, Lake formation, s3 bucket
  • Automated Migration of around 2k+ tables on aws for ingestion purposes.
  • Responsible for optimising of the pyspark jobs.
  • Incidents and resolutions for the Castrol data globally as part of the Castrol Dataworx SRE team..
AWS GlueAWS Step FunctionsPythonPySparkSoftware Design PatternsAmazon Redshift+3

Bukuwarung

SDE-2(Data Platform)

Jul 2022Nov 2023 · 1 yr 4 mos · Bengaluru, Karnataka, India

  • 1. Writing Airflow Code to perform ETL from varied sources.
  • 2. Implemented Basic Authentication using Java for the microservices and unit test cases.
  • 3. Implemented BigQuery Cost Optimisation by 40% and implemented the Best Practices across the org.
  • 4. Implemented platform architecture for Data Science model Deployment by using Flask, Docker and deployment on ECS. Reduced the KYB SLA from 1 day to ~30 seconds.
  • 5. Zoho/ Zohodesk and Big Query Integration by using Airflow
  • 6. Created Event-Driven Microservices using Spring Boot, Docker, Kafka, and ECS
  • 7. Contributed on the backend for the Payment System
  • 8. Mentoring juniors
MockitoAmazon ECSJavaPythonApache KafkaFlask+7

Rupeek

Data Platform Engineer

May 2021Jul 2022 · 1 yr 2 mos · Bengaluru, Karnataka, India

  • 1. Design and Build a CDC pipeline to ingest real-time data from the source system(MongoDB, Mysql, Postgresql) to the data lake using Kafka and Kafka connect.
  • 2. Develop the ETL and ELT code using pyspark and Airflow.
  • 3. Optimise the Redshift data Load.
  • 4. Develop Python/Pyspark Modules for ingesting data.
  • 5. Python Module to perform CRUD Operations on Mongodb
  • 6. Resiliency dashboard using Tableau.
  • 7. Mentoring Interns
GrafanaPySparkTableauAmazon RedshiftMongoDBApache Kafka+7

Enquero

3 roles

Senior Data Engineer

Oct 2020Apr 2021 · 6 mos

HivePySparkInformaticaSQLREST APIsHadoop+3

Big Data Engineer

Jun 2018Oct 2020 · 2 yrs 4 mos

Apache SqoopAmazon S3HiveSQLSnowflake CloudPython (Programming Language)+1

Intern

Jan 2018Mar 2018 · 2 mos · Bengaluru, Karnataka, India

  • Data Engineering Internship, worked on Python,Apache Spark ,kafka and hive.

Bharat electronics

Intern

Jun 2017Jul 2017 · 1 mo · Kotdwara Area, India

  • Worked on the Voice control Communication System Module of the BEL

Birla institute of applied sciences

2 roles

Executive

Oct 2015Nov 2016 · 1 yr 1 mo

  • 1-.College Representative Of E-cell IIT Bombay.
  • 2-Participated in the National Entrepreneurship Challenge (NEC) of E-Cell IIT Bombay and scored a national rank 13 (college rank) among the 250 colleges who participated in the Advance Track of the NEC.
  • 3- Developed my own Business Model "Navigating Young Minds By Experienced Birds" to participate in the the Business Pitching Event 'IDEATHON' conducted by the GREYMETER.
  • https://www.facebook.com/biasecell/?fref=ts

Placement Coordinator

Apr 2015Jan 2018 · 2 yrs 9 mos

Education

Indian Institute of Technology Hyderabad

Master of Technology - MTech — Data Science

Jul 2024Jun 2027

Birla Institute of Applied Sciences

Bachelor of Technology (B.Tech.) — Computer Science

Jan 2014Jan 2018

Central Board of Secondary Education

Class 12

Jan 2013Jan 2014

Central Board of Secondary Education

Class 10

Jan 2011Jan 2012

Stackforce found 100+ more professionals with Data Engineering & Aws

Explore similar profiles based on matching skills and experience