Oct 2024 – Present · 1 yr 7 mos · Remote

• As a Databricks preferred partner we guide some clients on their migration strategy with databricks end to end deployment.

Sep 2024 – Feb 2025 · 5 mos · Remote

Jan 2020 – Oct 2025 · 5 yrs 9 mos · Remote

• As a Databricks Community Contributor, I actively assist data professionals in resolving complex challenges related to Databricks. I regularly engage with community members, share solutions, and contribute to discussions to support the broader data engineering community.
• You can view my profile here:
• https://community.databricks.com/t5/user/viewprofilepage/user-id/53460
• I am also recognized as a featured member in the Databricks Community:
• https://community.databricks.com/s/feed/0D58Y00009oyBOmSAM

Aug 2024 – Present · 1 yr 9 mos · Bengaluru, Karnataka, India · Remote

• I am actively contributing to the open source community, focusing on projects that drive innovation in data engineering and cloud technologies. My GitHub profile showcases my ongoing work and collaborations: github.com/aviral-bhardwaj.
• Some of the notable projects I contribute to include:
• MarkItDown by Microsoft: A popular Python package that converts a wide range of file formats-including Office documents, images, audio, and web data-into Markdown, making it a powerful tool for content conversion and LLM training.
• Unity Catalog by Databricks: The industry’s first open source catalog for unified data and AI governance, enabling organizations to manage and secure data assets across clouds and data platforms. Unity Catalog is now open-sourced, fostering collaboration and integration within the community.
• Open Source Projects by AWS: I engage with AWS-driven open source initiatives, which include a variety of developer tools and cloud-native projects such as the AWS Cloud Development Kit, Bottlerocket, Firecracker, and OpenSearch, all aimed at improving cloud development and operational excellence.
• I am passionate about building and collaborating on cloud-based data solutions, and I am always open to new opportunities to contribute and learn within the open source ecosystem

Jul 2024 – Oct 2025 · 1 yr 3 mos · Hyderabad, Telangana, India · On-site

• I played a key role in implementing advanced data engineering solutions at Coforge Limited, focusing on Databricks and AWS technologies.
• Developed data pipelines that provided cost usage notifications, enhancing stakeholder awareness.
• Collaborated with the Fivetran team to integrate audit logs into S3 storage for improved data governance.
• Successfully migrated Databricks jobs across various environments, ensuring smooth deployment and operational continuity.

Jul 2024 – Oct 2025 · 1 yr 3 mos · Hyderabad, Telangana, India · Remote

• Implemented Databricks products including Unity Catalog, Service Principals, and Cluster Policies.
• Developed data pipelines to provide cost usage notifications to key stakeholders.
• Collaborated with Fivetran team to integrate audit logs into S3 storage.
• Successfully migrated Databricks jobs across development, test, and production workspaces, ensuring seamless deployment.
• Collaborating with a major HR client (Trient) to implement Databricks solutions.

Jan 2024 – Jul 2024 · 6 mos · Gurugram, Haryana, India · Remote

• Collaborated with the Drivewealth team to migrate the Hive Metastore data to Unity Catalog.
• Analyzed the existing architecture and recommended a new approach for deploying solutions using Terraform.
• Migrated Databricks jobs across development, testing, and production workspaces for smoother deployment.

Jan 2024 – Jul 2024 · 6 mos · Gurugram, Haryana, India · Remote

• Successfully completed the GMG Retail Order Management System (OMS) project within deadlines, utilizing Fluent Commerce API.
• Developed data pipelines in Unity Catalog across bronze, silver, and gold layers, ensuring data integrity and accessibility.
• Pioneered the Data Engineering division at GMG's Technology Center, leading a team of five to enhance data infrastructure.

Jan 2024 – Feb 2024 · 1 mo · Gurugram, Haryana, India · Remote

• Book Review: Databricks Certified Associate Developer for Apache Spark Using Python -I recently reviewed the book Databricks Certified Associate Developer for Apache Spark Using Python: The Ultimate Guide to Getting Certified in Apache Spark Using Practical Examples with Python. This comprehensive guide is designed to help readers prepare for the Databricks certification exam by providing practical examples and clear explanations using Python. The book covers essential Spark concepts, hands-on exercises, and exam-focused tips, making it a valuable resource for anyone aiming to become a certified Spark developer.
• For more details, you can find the book at Packt Publishing:
• https://www.packtpub.com/en-in/product/databricks-certified-associate-developer-for-apache-spark-using-python-9781804619780

Nov 2023 – Jan 2024 · 2 mos · Bengaluru, Karnataka, India · Remote

• Worked with the FanDuel Insight Engineering team to deploy Databricks products such as Unity Catalog, Service Principals, Cluster Policies, and Data Pipelines for cost usage notifications to stakeholders.
• Collaborated with the Fivetran team to ingest Fivetran audit logs into S3.
• Worked closely with the Tableau team on data ingestion and multiple token-related issues.
• Extensively used Terraform to simplify and deploy Databricks products and data pipelines.

Nov 2023 – Jan 2024 · 2 mos · Bengaluru, Karnataka, India · Remote

Apr 2021 – Jan 2024 · 2 yrs 9 mos · Remote

• In my role at Knowledge Lens, I spearheaded the transition of on-premise databases to cloud solutions for major clients like Coca-Cola and Amgen Inc. I developed robust data pipelines using Azure Data Factory and AWS services, ensuring efficient data transformation and validation across various medical streams. My work significantly improved data processing capabilities while maintaining high security and cost-efficiency.

Sep 2019 – Oct 2023 · 4 yrs 1 mo · Remote

Mar 2020 – Jan 2023 · 2 yrs 10 mos · Bengaluru, Karnataka, India · Remote

• Developed robust data pipelines for oncology, respiratory, and COVID-19 medical streams.
• Designed ETL workflows utilizing AWS Glue and Databricks with PySpark and Python.
• Implemented open-source Airflow on AWS EC2 for efficient orchestration and validation using SQL.
• Automated AWS infrastructure management through Lambda functions, enhancing security policies.

Jan 2020 – Mar 2020 · 2 mos · Bengaluru South, Karnataka, India · Remote

• I have recently started teaching on Udemy, dedicating my weekends to creating courses that help me enhance my skills while sharing knowledge with others. My current offerings include:
• Python for Data Engineering: This course covers Python from the basics to advanced concepts, focusing on logic building, data structures, object-oriented programming, exception handling, and more. It is designed for engineering students and software professionals aiming to strengthen their Python skills for data engineering roles.
• Databricks Associate Developer for Apache Spark: This course is tailored for those preparing for the Databricks certification, providing practical insights and hands-on examples with Apache Spark.
• You can explore these courses here:
• Python Course: https://www.udemy.com/course/python-for-data-engineering/
• Spark Course: https://www.udemy.com/course/databricks-associate-developer-for-apache-spark/
• I look forward to helping learners advance their careers in data engineering and big data technologies.

Sep 2019 – Mar 2020 · 6 mos · Bengaluru, Karnataka, India · Remote

• Developed and maintained secure, cost-efficient data pipelines utilizing AWS services including EC2, S3, and EMR.
• Conducted thorough data validation to ensure accuracy and reliability of data loads.
• Leveraged Databricks functionalities to enhance data processing efficiency and performance.

Jul 2016 – Jan 2019 · 2 yrs 6 mos · India

• Worked As a Website Developer, Search Engine Optimization Head And Content Writer at FreeJobAware.
• Managed 8 People in Team For 2 Years.
• Worked in Area on On-page and Off-Page Optimization.
• Developed Website For NGO Astha Seva Sansthan Kotdwar.
• Developed Website For IBharatNews a News Portal for Uttrakhand.