Harman Bhatia — Data Engineer

I am a Data Engineer with specialized expertise in Generative AI and Large Language Models (LLMs). With a strong background in data research, analysis, and transformation, I have successfully led and executed multiple AI projects and hold certifications in LLMs and Prompt Engineering. Key Achievements: ● AI & LLM Expertise: Developed and implemented state-of-the-art solutions using Generative AI and LLMs to drive innovation and efficiency in various projects. ● Automation & Efficiency: Reduced testing time by 70% per metric (handled 240+ metrics) by developing a testing automation framework using web scraping and Python. ● Data Pipeline Development: Built a real-time pipeline to migrate historical data (>1TB) and incremental data (2 million records per day) from RDS Postgres to Snowflake using AWS DMS, Amazon S3, and AWS Lambda. ● Data Warehouse Design: Designed a Snowflake Data Warehouse framework with a star schema for the platform migration of 280 tables. ● ETL Implementation: Implemented an ETL pipeline to migrate 40+ TB of data across 1000+ tables from Teradata to Snowflake using TPT, AWS S3, and AWS EMR. ● Redshift to Databricks Migration: Migrated data from Redshift to Databricks, leveraging DBT and Airflow for efficient data transformation and orchestration. ● Data Ingestion: Created an ingestion module to process historical data (>10 GB) into HDFS from various heterogeneous sources. ● Data Transformation: Developed PySpark code to transform 67 million records using Spark SQL. ● Code Debugging & Testing: Enhanced data accuracy for decision-making by 90% through rigorous unit testing, system integration testing, and performance benchmarking. Leadership & Mentorship: ● Team Leadership: Led a team of 5 associates, effectively delegating tasks and delivering projects ahead of deadlines through strong communication and teamwork. ● Mentorship: Trained and mentored over 20 new colleagues, providing them with essential knowledge of tools and technologies such as Python, SQL, Snowflake, AWS, and Big Data stack. Publications: Check out my blogs on Medium: > Python Series: "Just Python" > Data Migration: "Postgres to Snowflake — Migrate Real-time and Historical Data" > Salesforce to Snowflake: "Migrate Data from Salesforce to Snowflake" I am open to working as an independent contractor or in a full-time role, with flexibility across multiple time zones and a willingness to travel. This adaptability ensures that I can meet the diverse needs of global clients and projects. Feel free to contact me at - imharmanbhatia@gmail.com

Stackforce AI infers this person is a Data Engineering expert specializing in AI and Big Data solutions.

Location: Bengaluru, Karnataka, India

Experience: 7 yrs 8 mos