Harmanjeet Singh

Engineering Manager

Delhi, India9 yrs 10 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Expert in building data pipelines for big data projects.
Proven track record in AWS migration and optimization.
Strong leadership in data engineering and machine learning initiatives.

Stackforce AI infers this person is a Data Engineering expert with a focus on Big Data and Machine Learning in Fintech.

Contact

Skills

Core Skills

Data EngineeringAmazon Web Services (aws)Go (programming Language)Apache KafkaData VisualizationHadoop

Other Skills

Agile MethodologiesAirflowAndroidApache FlinkApache SparkApache Spark StreamingBig DataCC++CSSCassandraContinuous Integration and Continuous Delivery (CI/CD)Data ModelingDocker ProductsExtract, Transform, Load (ETL)

About

Data engineer/plumber, work around building pipelines for big data and machine learning projects. e^(days-today)

Experience

9 yrs 10 mos

Total Experience

2 yrs 5 mos

Average Tenure

4 yrs 7 mos

Current Experience

Paytm

3 roles

Engineering Manager

Promoted

Apr 2025 – Present · 1 yr 2 mos · Noida, Uttar Pradesh, India

Lead strategy & execution for high-scale merchant risk monitoring and fraud detection.
Built an anomaly detection framework utilizing statistical count analysis and data
discrepancy validation.
Architected MCP Server for dataset discovery and code generation.
Engineered Spark MCP Server for log analysis, scaling & cost optimization.
Fostered adoption of modern data engineering paradigms and AI-powered methods.

Apache SparkAmazon Web Services (AWS)Data Engineering

Senior Technical Lead

Apr 2023 – Mar 2025 · 1 yr 11 mos · Noida, Uttar Pradesh, India

Designed and developed a rule engine in GoLang with monitoring/logging.
Built real-time ingestion framework in Flink with customizable sources & sinks.
Drove Spark optimization, reducing cluster runtime and improving SLA adherence.
Led migration of complex workloads to AWS, ensuring resilience & cost efficiency.
Mentored engineers, improving workflows via AI-assisted testing and reviews.

AirflowJava Web ServicesData EngineeringGo (Programming Language)

Technical Lead

Sep 2021 – Mar 2023 · 1 yr 6 mos · Noida, Uttar Pradesh, India

Built pipelines for Transactional and Merchant Settlement Reports.
Built ODMR (On Demand Merchant Request) streaming framework to provide historical settlement reports to merchants in near real-time.
Migrated Paytm projects from Alibaba Cloud & On-premise cluster to AWS.
Worked on cost and performance optimization of spark applications across projects.

AirflowJava Web ServicesData EngineeringAmazon Web Services (AWS)

Rakuten

Software Engineer 2

Dec 2019 – Sep 2021 · 1 yr 9 mos · Greater Bengaluru Area

Built spark streaming pipeline to ingest Kafka events from Viki (OTT platform) and provide real-time analytics.
Built Sleeper Cell Detection by analyzing time series data of cell towers to ensure zero downtime. Performed spark & hive optimization to bring job completion time to 15 minutes from an hour. And helped improve model accuracy from 85% to 95%.
Developed MNO Data Insight pipeline to enrich data for customer segmentation and targeted ad campaigning. Data involve transactional data from the Rakuten e-commerce platform.
Built dashboard for data utilization of hadoop resources by analyzing ranger logs for auditing.
Setup data quality control pipeline to ensure data correctness and freshness for various business units. Pipeline is built to handle TBs of data and run every hour.

Java Web ServicesDocker ProductsData EngineeringApache Kafka

Css corp

Senior Data Engineer

Oct 2018 – Dec 2019 · 1 yr 2 mos · Bengaluru, Karnataka, India

Discovering data acquisition opportunities and building web crawling processes to scrap data from social media sites and various forums.
Worked on chatbot POC for Hindustan Unilever in Google DialogFlow, REST services and NodeJS.
Built tableau dashboard for data visualization and analytics.

Java Web ServicesObject-Oriented Programming (OOP)Data EngineeringData Visualization

Times internet

2 roles

SDE 2

Promoted

Jun 2017 – Oct 2018 · 1 yr 4 mos · Noida, Uttar Pradesh, India

Built Recommendation Engine for The Economics Times to provide personalized content to its user. Ensemble learning technique were used for better predictions.
Engaged in managing and configuring Hadoop clusters, contributing to the design and development of Data Transformation Framework components for ETL processes, and optimizing Spark job performance while crafting efficient queries for large datasets.

Java Web ServicesObject-Oriented Programming (OOP)Data EngineeringHadoop

SDE 1

Jun 2016 – Jun 2017 · 1 yr · Noida, Uttar Pradesh, India

Engaged in profiling user data for The Economics Times site. Built system to retrieve user activity and monitor engagement on the website, and contributed to creating web services for contests and surveys utilizing restful services.

Java Web ServicesObject-Oriented Programming (OOP)Data Engineering