May 2024 – Present · 2 yrs 1 mo · India

• I create content on data.

Dec 2023 – Aug 2025 · 1 yr 8 mos · Pune, Maharashtra, India

• Developed an end-to-end data pipeline for real-time and batch data ingestion, transformation, and analytics using cloud services (Azure, AWS), Kafka, and PySpark. Automated workflows with Apache NiFi and Airflow, and enabled data transformation via DBT, ensuring high-quality data for machine learning and analytics.
• Key Contributions:
• Ingested real-time data with Kafka and managed batch ETL workflows with Apache Airflow and NiFi.
• Processed large datasets using PySpark on Azure Databricks, storing results in AWS S3 and Azure Data Lake.
• Transformed data using DBT for SQL-based modular transformations.
• Built interactive dashboards in Tableau for business insights.
• Used Jira for task management and Bitbucket for version control.
• This pipeline enhanced data availability and accuracy, supporting real-time decision-making and analytics

Sep 2021 – Nov 2023 · 2 yrs 2 mos

• Designing data architecture: I work to create an architecture that enables efficient and effective data storage, processing, and analysis.
• Developing data pipelines: I design and implement pipelines to extract, transform, and load (ETL) data from various sources into data warehouses or other data storage systems.
• Designing data models and schemas: I create the schemas and data models that allow data to be organized and analyzed effectively.
• Optimizing performance: I identify and address performance bottlenecks in the data infrastructure and systems to ensure that data processing and analysis is as efficient as possible.
• Ensuring data quality and governance: I am responsible for ensuring that data is accurate, consistent, and meets the needs of the organization, and that proper governance practices are followed.
• Collaborating with data scientists and analysts: I work closely with data scientists and analysts to understand their needs and ensure that data is accessible and available for analysis.
• Selecting tools and technology: I evaluate and select the appropriate tools and technologies to build and maintain the data infrastructure and systems.
• Communicating with stakeholders: I document my work and communicate with stakeholders to ensure that they understand the data engineering process and the systems and infrastructure that I have built.

Sep 2021 – Nov 2023 · 2 yrs 2 mos

• Led the migration from Sap to Mongodb using amazon athena,lambda and s3.resulting in an annual cost saving of $678,000 and increase performance by 14 %.
• Designed and implemented a real-time data pipeline to process structured data by integrating 150 million raw records from 30+ data source using kafka and pyspark.
• Build data visualization using Power-Bi that reduced manual report by 10 hours weekly.
• Designed the pipeline architecture for new product that quickly scale from 0 to 125,000 active users used iics.
• Worked on collections and database creation process also Worked on mongodb-atlas and created cluster where all the collections and database stores.
• Worked on azure synapses, storage,blob etc.

Jul 2021 – Sep 2021 · 2 mos · Bengaluru, Karnataka, India · On-site

• Worked on creating data pipeline,data analysis, Data modelling
• Build out the data and reporting infrastructure from the ground using SQL and python,to provide real time insights into the product.
• Received,cleaned ,and prepped the data using python,sql to build marketing mix-models that result in lift in ROI of 8 basic points.
• Error resolving and make code optimized for production ready.
• Compiled studied and inferred large amount of data,modeling information to drive auto policy pricing.
• Analysed model and interpret data and created dashboard by using power-bi and also IICS for etl work.

Feb 2021 – Jun 2021 · 4 mos · Indore, Madhya Pradesh, India

• Settled up processes and systems to make working with data more efficient.
• Communicated with business department to understand need and requests to build 20+ pipelines by analyzinh technical issues.
• Automated ETL Processes across billions of rows of data,which reduces manual workload.

Jan 2021 – Feb 2021 · 1 mo · United States

• Collected and compiled data from various sources, including databases, spreadsheets, and surveys.
• Utilized SQL queries to extract and organize data from relational databases.
• Developed and implemented data collection plans to ensure data accuracy and completeness.

Aug 2020 – Nov 2020 · 3 mos · Indore, Madhya Pradesh, India

• Demonstrated strong interpersonal and analytical skills.
• Worked in Panda,numpy,matplotlib,Machine learning algorithm,Python language.
• Used tools to extract the data needed.
• Analysed model and interpret ,collects,cleaning of the data sets in order to answer a question or solve a problem.

Aug 2020 – Oct 2020 · 2 mos · India

• Data-analysis intern worked as Data-mapping,Data-cleaning,data-processing,Scheduling by airflow.