Hirday Kapoor

Data Engineer

Delhi, India1 yr 10 mos experience

AI EnabledAI ML Practitioner

Key Highlights

Delivered ₹10+ Cr annual cost savings through in-house platforms.
Architected high-frequency data ingestion systems for 2,000+ turbines.
Optimized workflows, improving performance by 40%.

Stackforce AI infers this person is a Data Engineer specializing in scalable data platforms within the Renewable Energy sector.

Contact

Skills

Core Skills

Real-time Data Systems & StreamingDistributed Data ProcessingData EngineeringData Modeling & Etl DesignMachine LearningData Science

Other Skills

Apache SparkMongoDBAzureDatabricksReal-time data pipelinesAPIsAnalytics systemsData ingestionProcessing systemsPySparkETLPython (Programming Language)System DeploymentOperational PlanningMLOps

About

I design and build scalable, real-time data platforms that handle high-volume data at production scale. My work focuses on engineering systems that ingest, process, and serve high-frequency data, enabling analytics, monitoring, and intelligent decision-making. I’ve built in-house platforms replacing vendor systems, delivering ₹10+ Cr annual cost savings, and enabling real-time visibility across large-scale industrial operations. ⚙️ What I do: Build real-time and batch data pipelines (PySpark, Databricks) Design high-throughput ingestion systems for time-series data Develop low-latency APIs and data services Architect scalable data platforms (Delta Lake, MongoDB) Optimize systems for performance, reliability, and cost 📊 Core Expertise: Distributed Data Processing (Spark, PySpark) Real-Time Data Systems & Streaming Data Modeling & ETL Design Cloud Platforms (Azure, AWS) System Design & Performance Optimization 🧠 How I think: I approach problems as a systems engineer, focusing on: Scalability Reliability Efficiency 📈 Certifications: Databricks Certified Data Engineer Associate Databricks Certified Spark Developer Databricks Certified ML Associate 🎯 What I’m looking for: Building large-scale data platforms and infrastructure, where engineering excellence drives real-world impact.

Experience

1 yr 10 mos

Total Experience

1 yr 10 mos

Average Tenure

1 yr 10 mos

Current Experience

Renew

3 roles

Data Engineer

Promoted

Jul 2024 – Present · 1 yr 10 mos · Gurugram · On-site

I build scalable, real-time data platforms for renewable energy operations across wind and solar assets, replacing legacy vendor systems and enabling in-house intelligence at scale.
⚡ Key Contributions:
Architected high-frequency data ingestion systems (OPC UA/DA, ODBC) processing 10GB/day from 2,000+ wind turbines at 1s–10min intervals, enabling sub-minute analytics
🚀 Built Cognyt Platforms (Core Achievement):
Led backend engineering for Cognyt Wind platform:
Designed real-time data pipelines, APIs, and analytics systems
Replaced third-party vendor solution, delivering ₹5–6 Cr annual cost savings
Enabled live turbine monitoring, alarms, and performance analytics at scale
Built Cognyt Solar platform:
Developed ingestion and processing systems for solar assets (ICR, MCR, WMS, MICRO)
Implemented parallel, low-latency ingestion architecture (<10s intervals)
Replaced external vendor systems, saving ₹4–5 Cr annually
⚙️ Data Platforms & APIs:
Developed CognytAPI (Azure Function Apps):
Secure token-based APIs for real-time & historical data access
Integrated MongoDB + dynamic query system for turbine/tag-level data retrieval
Enabled seamless cross-team and external system integrations
Designed high-performance time-series data architecture in MongoDB:
Optimized for high write throughput + low-latency querying
Structured for analytics, monitoring, and ML pipelines
🏗️ Systems & Optimization:
Built parallel OPC UA ingestion framework handling multiple assets and categories with high efficiency
Optimized Databricks + Azure workflows, improving performance by 40%
Automated external data ingestion pipelines, reducing vendor dependency and cutting costs by 70%
🔒 Reliability & Business Continuity:
Developed SAP offline application, ensuring 99.9% operational continuity during network failures
Built robust data pipelines ensuring high availability and fault tolerance

Apache SparkMongoDBAzureDatabricksData EngineeringReal-Time Data Systems & Streaming+1

Data Scientist

Jan 2024 – Jul 2024 · 6 mos · Gurugram · On-site

Built wind-speed imputation models using cross-turbine correlation and interpolation techniques, improving data completeness to ~95% for downstream analytics
Developed and optimized ETL pipelines (PySpark, pandas) integrating telemetry, weather, and market data, reducing data preparation time by 50%
Designed predictive models and simulations for renewable energy use cases (wind, solar, green hydrogen), enabling better forecasting of energy production and consumption
Automated web scraping pipelines to ingest external energy and market data, eliminating third-party dependencies and saving 10+ hours/week of manual effort
Implemented data-driven use cases supporting resource optimization and operational decision-making across renewable energy systems

Apache SparkPySparkETLMachine LearningData ScienceData Modeling & ETL Design

Data Scientist

Jun 2023 – Aug 2023 · 2 mos · Gurugram · On-site

Built predictive models for solar and wind energy forecasting, leveraging historical and environmental data
Conducted data analysis and simulation experiments to improve accuracy of energy production estimates
Integrated model outputs into data workflows and analytics systems, enabling better operational visibility
Worked with cross-functional teams to translate analytical insights into practical system improvements

Machine LearningPython (Programming Language)Data Science