Vishal Kumar — Associate Consultant

Results driven Data Engineer with expertise in building scalable, cloud-native data platforms on Azure and AWS. Passionate about designing end-to-end data pipelines, Lakehouse architectures, and ETL/ELT frameworks that deliver clean, reliable, and business-ready data. Azure Data Engineering : Proven experience in Azure Data Factory (ADF), Azure Databricks, and Azure Synapse Analytics to architect and operationalize enterprise-grade data platforms. Skilled in implementing the Medallion Architecture (Bronze–Silver–Gold layers) to standardize data flow, improve governance, and enable high-quality analytics. Leverage Delta Lake for ACID transactions, schema evolution, incremental loads, and time travel, ensuring data reliability for both batch and real-time use cases. AWS Serverless Data Engineering : Hands-on experience designing event-driven, serverless pipelines using AWS Glue, Lambda, S3, Athena, and Step Functions. Adept at integrating real-time streaming data from Kinesis and Kafka into downstream analytics, enabling cost-efficient and auto-scaling data solutions. Big Data & Distributed Processing : Proficient in Apache Spark (PySpark), Hadoop, Hive, and Apache NiFi for large-scale data ingestion, transformation, and orchestration. Skilled in optimizing Spark workloads, managing cluster performance, and implementing streaming and batch data processing. Programming, Automation & DevOps : Strong command of Python and SQL for data manipulation, automation, and validation. Implement CI/CD pipelines using Git, Azure DevOps, and Jenkins to ensure automated testing, deployment, and version control across environments. Data Governance & Quality : Experienced in data governance frameworks, metadata management, and data quality enforcement across the Medallion layers. Focused on delivering trustworthy, discoverable, and compliant data to drive strategic insights. Business Domain Expertise : Hands-on experience in Banking & Payments and E-commerce, managing mission-critical data flows for customer analytics, fraud detection, and operational efficiency. Skilled in translating complex data architectures into actionable insights that fuel business growth.

Stackforce AI infers this person is a Data Engineer specializing in Fintech and cloud-native data solutions.

Location: Bengaluru, Karnataka, India

Experience: 11 yrs 10 mos

Skills

Azure Data Engineering
Data Governance

Career Highlights

Expert in building scalable data platforms on Azure and AWS
Proficient in implementing Medallion Architecture for data governance
Hands-on experience with event-driven serverless data pipelines

Work Experience

HCL Technologies

Lead Consultant (4 yrs 6 mos)

Brillio

Data Specialist (1 yr 1 mo)

Future Group India

Senior Software Engineer (1 yr 4 mos)

Infosys

Technology Analyst (2 yrs 1 mo)

Monster India

Software Engineer (11 mos)

Paytm

Associate (1 yr 11 mos)

Education

Bachelor's degree at Truba Institute Of Engineering And Information Technology

senior secondary at Gyan Bharti Residential Complex Bodhgaya.

secondary education at National Open School

Vishal Kumar

Associate Consultant

Bengaluru, Karnataka, India11 yrs 10 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Expert in building scalable data platforms on Azure and AWS
Proficient in implementing Medallion Architecture for data governance
Hands-on experience with event-driven serverless data pipelines

Stackforce AI infers this person is a Data Engineer specializing in Fintech and cloud-native data solutions.

Contact

Skills

Core Skills

Azure Data EngineeringData Governance

Other Skills

Azure SynapseAzure Data FactoryAzure DatabricksAzure Data LakeAzure DevOps ServicesAzure SQLPySparkAzure Synapse AnalyticsData ModelingData QualityCollaborationAWS GlueAWS LambdaAWS KinesisApache Spark

About

Experience

11 yrs 10 mos

Total Experience

1 yr 11 mos

Average Tenure

4 yrs 6 mos

Current Experience

Hcl technologies

Lead Consultant

Nov 2021 – Present · 4 yrs 6 mos · Bengaluru, Karnataka, India

Projects: Engineered and maintained scalable data ingestion pipelines using Azure
Data Factory and Databricks to process multi-terabyte datasets from core banking
systems and APIs.
Designed and implemented a modern data warehouse in Azure Synapse Analytics,
optimizing data models to improve query performance by 40% for financial reporting.
Collaborated with data scientists and fraud detection teams to provide cleansed and
validated data, enabling the development of predictive models that reduced financial
losses.
Implemented Delta Lake on Azure Data Lake Storage Gen2 to ensure data reliability
and enforce schema on write, guaranteeing data quality for downstream consumption.
Developed custom PySpark scripts on Databricks to perform complex data
transformations and aggregations, supporting key business intelligence and compliance
reporting.

Azure SynapseAzure Data FactoryAzure DatabricksAzure Data LakeAzure DevOps ServicesAzure SQL+2

Brillio

Data Specialist

Oct 2020 – Nov 2021 · 1 yr 1 mo · Bengaluru, Karnataka, India

-Building the CDC pipeline using Amazon Appflow, handling CRUD operation, schema evolution.

Future group india

Senior Software Engineer

Jun 2019 – Oct 2020 · 1 yr 4 mos · Bangalore

Build Change Data Capture Pipeline using Apache Spark.
Building data warehouse for the data feed in Hive.
Building Apache Nifi Pipelines to solve day to day business use cases.
Build Point of Sale pipelines using multiple data sources using Apache Spark, Nifi, Kafka.

Infosys

Technology Analyst

May 2017 – Jun 2019 · 2 yrs 1 mo · Pune, Maharashtra

Live Streaming using Kinesis for customer consent and automated.
Building entire warehouse for the data feed in Sagemaker
Developing AWS Glue Job for the entire ETL process and triggering.
Using AWS Lambda Function based on events.
Deployments using Serverless Framework.
Live Streaming of GCIP (Live Registered Customers) data using Apache Spark and Kafka (Spark Streaming and Kafka Integration)
Complete Oracle Data Warehouse movement on Apache Hive, applying all optimization on the hive table.
Creating Apache Spark Jobs for daily run and scheduling into Oozie jobs.
Analysing the user behaviour analysis using Apache Spark SQL Data Frames (EDA).

Monster india

Software Engineer

Jun 2016 – May 2017 · 11 mos · Noida Area, India

Worked as Big Data Developer And Machine Learning Engineer.
Working over Job Recommendation Systems based on user‘s skill set and history pattern.
Working on Job Search Predictive Analytics, Machine learning Algorithms.
Used Machine Learning Algorithms (Association Rule Mining, Regression, Scoring Models, KNN, K - Mean, Clustering, Naive Bayes, Decision Tree, PCA, Sampling)
Coding Language: Python (Numpy, Pandas, NLTK, Scipy, Scikit-Learn, Pyspark
Key Skills:-Hadoop,Apache Spark, Scala, Pig, Hive, Sqoop, Kafka, Oozie, Yarn, Linux, Shell Scripting, SQL, Advanced Excel, Machine Learning Algorithms(Linear Regression,Logistic Regression, Apriori Algorithm,Naive Bayes Algorithm, Random Forest Algorithm)

Paytm

Associate

Jun 2014 – May 2016 · 1 yr 11 mos · Noida Area, India

Offline Retail Wallet Analysis-Processing the entire Transactional Data and drawing the Insights, Analysing the active merchants, analyzing the GMV(MTD)(LMTD) basis.
Reporting the Offline Wallet Growth(Retail, Auto/Taxi, Hotels)Number and Amount of Transactions from Various Geographical Area
Capable in processing large set of structured, semi-structured and unstructured large data sets and supporting systems application architecture.
Provisioning, installing, configuring, monitoring, and maintaining HDFS, Yarn, HBase Flume, Sqoop, Pig, Hive Write efficient MapReduce jobs, Pig Scripts, Hive scripts,R Programming. Installation of various Hadoop ecosystems and its daemons.