Harshit Goyal

Software Engineer

Bengaluru, Karnataka, India11 yrs 6 mos experience
Highly Stable

Key Highlights

  • Expert in building scalable data-driven platforms.
  • Proven track record in optimizing big data solutions.
  • Strong background in cloud infrastructure and microservices.
Stackforce AI infers this person is a SaaS-focused Software Engineer with expertise in cloud computing and big data solutions.

Contact

Skills

Core Skills

Cloud ComputingBig DataMicroservicesSoftware DevelopmentData EngineeringProgramming

Other Skills

Apache KafkaApache AmbariMicrosoft Azure HDInsightData AnalyticsApache AirflowReal-time ProcessingBig QueriesGCP HiveJavaSpringAWS AppSyncDynamoDBLambdaAWSReal-time Monitoring

About

I am an experienced Software Engineer specializing in high-scale, data-driven platforms, with expertise spanning cloud infrastructure, big data analytics, and microservices architecture. Currently, at Microsoft, I play a key role in the development of Microsoft Fabric, a unified data analytics platform that integrates diverse Azure technologies to streamline data processing and generate actionable insights. Additionally, I have contributed to the evolution of Azure HDInsight, enabling scalable cloud-based processing through Apache Spark, Hive, Kafka, and HBase to meet complex enterprise demands. Previously, at Walmart, I was instrumental in optimizing the Market Intelligence (MINT) platform, scaling it to process over 20TB of data daily and efficiently manage 60 million requests per day. My work strengthened competitive pricing strategies, operational excellence, and cloud efficiency by leveraging Apache Airflow, distributed systems, and orchestration tools. I also contributed to Walmart’s Product Matching Platform, automating data pipelines and implementing robust scraping mechanisms for market analysis. Before Walmart, at Get My Parking, I developed a global B2C parking automation solution, designing and deploying microservices, serverless architectures, and real-time monitoring systems using AWS, Prometheus, and Grafana. My efforts significantly enhanced API performance and optimized caching mechanisms, ensuring seamless scalability and a superior user experience. Beyond industry roles, I am the creator of a successful YouTube channel, where I guide thousands of students and professionals through tech career preparation, covering algorithms, data structures, system design, coding interviews, and emerging technologies. I bring deep expertise in Java, Python, Big Data, Cloud Architecture (AWS, Azure, GCP), Data Pipelines, C, C++, C#, System Design, Distributed Computing, AI & Machine Learning, Microservices, SQL & NoSQL databases, and orchestration tools such as Apache Airflow. Passionate about driving innovation, scaling systems, and unlocking business potential through data, I am always eager for opportunities that push boundaries and create lasting impact.

Experience

11 yrs 6 mos
Total Experience
1 yr 11 mos
Average Tenure
2 yrs
Current Experience

Microsoft

Software Engineer - IC3

Jun 2024Present · 2 yrs · On-site

  • Building Fabrics Live Tables, Auto-Generated DAGs, Materialised Lake Views for Microsoft Fabric.
  • Owning Apache Kafka and Apache Ambari on Microsoft Azure HDInsight.
Apache KafkaApache AmbariMicrosoft Azure HDInsightCloud ComputingBig Data

Walmart

2 roles

Senior Software Engineer - IN4

Promoted

Oct 2022May 2024 · 1 yr 7 mos · Bengaluru, Karnataka, India

  • Led the development of scalable big data and orchestration pipelines to drive item and seller assortment growth on Walmart Marketplace, including key seasonal events like Black Friday.
  • Played a pivotal role in scaling Walmart’s Market Intelligence Platform to handle 60 M+ daily requests and process 20TB of data per day.
  • Enhanced competitive pricing strategy by building systems for real-time competitor product matching and price optimization.
  • Improved overall platform architecture by implementing efficient streaming and batch processing systems, and integrating tools like Apache Airflow and Camunda.
Apache AirflowBig DataReal-time ProcessingCloud Computing

Software Engineer - IN3

Feb 2021Sep 2022 · 1 yr 7 mos · Bengaluru, Karnataka, India

  • Working with ECommerce teams to bridge the market gaps aggressively. The team is helping the Product Matching Platform to find similar products across Walmart catalogs and competitors’ catalogs to drive pricing and lead the assortment intelligence initiatives.
  • Major Contributions -
  • Wrote Batch Processing Pipelines single-handedly.
  • Utilised Apache Airflow, Big Queries, and GCP Hive to write Big Data Pipelines.
  • Worked on distributed systems to write orchestration frameworks.
  • Researched on various orchestration tools and created POCs
  • Python libraries are used to write data pipelines.
  • Scalable scrapping tools were used to get some competitive data for Walmart.
  • Utilized Java and Spring to write microservices.
Apache AirflowBig QueriesGCP HiveBig DataCloud Computing

Youtube

Megaverse | Tech Instructor

Apr 2021Present · 5 yrs 2 mos · Remote

  • Helping thousands of students/working professionals out there who are preparing for tech jobs
  • Upload Algorithms and DSA Concepts
  • System Design Concepts (Low Level and High Level) - System Design Interview Problems
  • Live Coding
  • Tech Related Talks with Industry Experts

Get my parking

Software Development Engineer

Jun 2019Jan 2021 · 1 yr 7 mos · Bengaluru, Karnataka, India · On-site

  • I worked on a highly scalable product to automate the parking industry globally, and interacted with the largest parking space providers in the European and American regions to write the product for them. Worked with tight SLA so that the end user's experience would be seamless.
  • Some Major contributions are -
  • Built a Permit (pass) Management System and multiple microservices end-to-end, including a custom monitoring tool with alerting.
  • Designed and developed highly scalable distributed systems in a B2C environment.
  • Implemented full-stack serverless solutions using AWS AppSync, DynamoDB, and Lambda functions.
  • Set up monitoring and alerting infrastructure with Prometheus, Grafana, and AlertManager; worked extensively with ELK stack and New Relic for real-time observability.
  • Boosted API performance through optimized architecture, caching mechanisms, and a custom cache library.
  • Integrated AmazonMQ for reliable message queuing in distributed systems.
  • Led client interactions for application demos, technical feedback, and solution proposals.
  • Contributed to architecture design, project planning, and code reviews across AWS-based projects.
AWS AppSyncDynamoDBLambdaCloud ComputingMicroservices

Evive | goevive.com

2 roles

Software Developer

Jun 2018May 2019 · 11 mos · On-site

  • Participated in Sprint Planning, Teamwork, and Code Analysis.
  • Wrote ETL (Extract-Transform-Load).
  • Deep hands-on unit test cases and code quality.
  • Participated in various Production Code Reviews, Unit Test Cases, and Performance Issues.
  • Improved command on Data Structures by using it in my code.
  • Establish and stimulate software development standards.
  • Added processes along with best practices for the delivery of scalable and high-quality Software.
  • Used JavaScript to handle front-end views
  • Worked on RxJava to enhance the quality of ETL implementations.
  • Started using logging tools like Kibana and Elastic Search to see the real-time logs.
  • Worked on Cassandra so got a good understanding of a NoSQL database.
ETLJavaScriptRxJavaSoftware DevelopmentData Engineering

Software Developer Intern

Mar 2018May 2018 · 2 mos · On-site

  • Understood the work and team flow.
  • Played with some build tools(Maven, Gradle)
  • Project development tools(Redmine, Jira)
  • Logging tools(Elastic Search/Logstash/Kibana)
  • Got a chance to deep dove into various IDE, databases (SQL and NoSQL)
  • Software development cycles.
  • Deep drove into Git and other Version Control Systems.
  • Worked on Sprintboot microservices to enhance the code quality, and scalability.
  • Unit test case coverage.

Thinking machines

Software Programmer

Jun 2017Feb 2018 · 8 mos · Ujjain, Madhya Pradesh, India · On-site

  • Wrote some services and good frameworks single-handedly in a plain text editor.
  • By using the same technique I have worked on J2EE to understand servlets.
  • Learn core and advanced Java, Java-based web applications.
  • Some nice frameworks including
  • Web Services Framework - the core of Spring boot
  • ORM framework - the core of Hibernate framework
  • Socket Programming-based server -the core of Java RMI.
  • Implementations from scratch helped me deeply understand the existing frameworks.
  • Worked on C and C++
  • Wrote an Image recognition application in Python using supervised learning.
  • Wrote visualization tool to understand sorting algorithms using JavaScript.
JavaC++PythonSoftware DevelopmentProgramming

Self-employed

Academic Tutor

Aug 2014May 2017 · 2 yrs 9 mos · On-site

  • Taught Maths and Science to students from 5th to 10th grade students.
MathematicsPhysicsAlgebra

Freelance

Computer Operator

Jun 2013Jul 2014 · 1 yr 1 mo · Ujjain, Madhya Pradesh, India · On-site

  • I was learning computer and internet skills and using someone else's computer to fill out other students' exam forms and printouts for free.

Education

Rajiv Gandhi Prodyogiki Vishwavidyalaya

Bachelors of Computer Science and Engineering

Central Board of Secondary Education (CBSE)

Matriculation and High School

Stackforce found 100+ more professionals with Cloud Computing & Big Data

Explore similar profiles based on matching skills and experience