Vaibhav Aggarwal

Software Engineer

Delhi, India4 yrs 7 mos experience

Key Highlights

3.5+ years of experience in data engineering.
Proficient in Apache Spark and ETL tools.
Strong foundation in data structures and algorithms.

Stackforce AI infers this person is a Data Engineer with expertise in backend development and cloud data solutions.

Contact

Skills

Core Skills

Data EngineeringBackend DevelopmentCloud ComputingTechnical Writing

Other Skills

AlgorithmApache SparkApache Spark StreamingData StructuresData WarehousingDatabase Management System (DBMS)ETLExtract, Transform, Load (ETL)GitJDBCJavaMicroservicesMySQLProblem SolvingSQL

About

i am a working as a Member of Technical staff in Salesforce.com i am versatile Data Engineer and Backend Engineer with 3.5+ years of experience in developing robust data solutions and backend systems. With a strong foundation in data structures and algorithms. Specializing in data pipeline development, i am skilled in technologies such as Apache Spark, Hadoop, and various ETL tools. I have proven track record of optimizing data workflows and ensuring data integrity, enabling teams to harness the power of data for strategic decision-making. On the backend side,i am also proficient in languages such as Python, Java, and springboot, coupled with in-depth knowledge of RESTful APIs and microservices architecture. This combination of skills allows me to build high-performance applications that meet user needs and business requirements.

Experience

Salesforce

Member of Technical Staff

Mar 2024 – Present · 2 yrs · Hyderabad, Telangana, India · Hybrid

Apache SparkAlgorithmData EngineeringBackend Development

Tata consultancy services

Software Developer

Aug 2021 – Mar 2024 · 2 yrs 7 mos · India

Data Migration
Developed a Framework Data Migration to migrate the data to Cloud Storage from Existing Oracle Database.
Data processing jobs are running in parallel fashion because of Extraction of Billions of Records. Plus to achieve
real-time data processing we use SparkStreaming , AutoLoader , Upsert merge, SparkUI , SparkSQL
and identify the Latest data of the user that can be store in Azure cloud Storege for transformation.
This framework will help the organisation to perform analytics on extracted data on daily basis that will help
Business to provide better user experience to Approx 10 million customers.
Immunization Data Feed
The objective of the Framework is provide the data of the users who have taken and vaccination.
The job is scheduled to run on daily to extract the data of New only users get vaccinated we are using Spark
Streaming and SparkSql for identification of not processed records.
Auto-Refill Data Feed
The objective of the Framework is to identify the list of users opted services Auto-refill .
The job is scheduled to run once in a day to extract details of users opted this facility we are using Spark
Executers, Spark read and write function and jdbc connections to write the user details in Mysql
DataBase.

Database Management System (DBMS)Spring BootData WarehousingJavaExtract, Transform, Load (ETL)Apache Spark+7

Geeksforgeeks

Technical Content Writer

May 2020 – Jul 2020 · 2 mos · India

Data Structures & Algorithm Tutorials - Individually contributed 15+ articles on Data Structures (Arrays, Linked list, Tree) and Algorithms (Dynamic Programming, Optimised Searching).
Articles live on their website; received 12,000+ views.

Problem SolvingJavaData StructuresApache SparkAlgorithmTechnical Writing