Nishant Kumar — Data Engineer

I am a 𝐃𝐚𝐭𝐚 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫 specializing in building large-scale, real-time data pipelines and scalable data platforms that power business-critical insights. Currently at IBM, I design and optimize high-throughput systems leveraging AWS, PySpark, Databricks, SQL, Airflow, and Kafka. With hands-on experience in data migration, pipeline automation, and cloud-native architectures, I’ve: - Automated ingestion of 25+ on-prem datasets (TB scale) into AWS S3 with Glue, Step Functions & Lambda. - Designed a configuration-driven data aggregation framework eliminating manual intervention. - Optimized tokenization workflows for 500M+ PII records, improving query performance. 𝐊𝐞𝐲 𝐒𝐭𝐫𝐞𝐧𝐠𝐭𝐡𝐬: ▪️ Data Engineering at Scale – End-to-end pipeline design, migration, and automation ▪️ Cloud Platforms – AWS (Glue, S3, Lambda, Redshift, EMR), Databricks, GCP (BigQuery, Pub/Sub) ▪️ Big Data Frameworks – PySpark, Hadoop, Kafka ▪️ Modern Data Stack – Snowflake, dbt, Airflow, Delta Lake, Iceberg ▪️ Data Modeling & Warehousing – CDC, SCD, dimensional modeling Beyond engineering, I’ve built a 85K+ LinkedIn community by sharing insights on Data Engineering, and have mentored 400+ professionals globally. This blend of deep technical expertise + knowledge sharing makes me passionate about solving data challenges that impact millions. Let’s connect if you’re looking for someone to design scalable, production-grade data systems or to exchange ideas on modern data engineering. I enjoy helping the community 🌱, sharing knowledge 💡, and making an impact 🔨. Let’s connect to explore collaboration or insights 💡For insights, idea exchange, or opportunities, feel free to contact me at: nishantnayan2002@gmail.com 🤝 For collaboration, you can reach me at: nishant.datacollab@gmail.com

Stackforce AI infers this person is a Data Engineer specializing in Fintech and SaaS with expertise in cloud-native data solutions.

Location: Bengaluru, Karnataka, India

Experience: 5 yrs 6 mos

Skills

Data Engineering
Cloud Platforms
Data Security
Data Migration
Etl Processes

Career Highlights

Expert in designing scalable data pipelines.
Automated ingestion of large datasets into AWS.
Mentored over 400 professionals globally.

Work Experience

IBM

Data Engineer (1 yr 9 mos)

Cognizant

Programmer Analyst(AIA-ModernDataEngineering) (2 yrs 3 mos)

Programmer Analyst Intern (4 mos)

Self-employed

Website Developer (1 yr 6 mos)

Education

Bachelor of Technology - B-Tech at Vel Tech Rangarajan Dr. Sagunthala R & D Institute of Science & Technology, Chennai

Nishant Kumar

Data Engineer

Bengaluru, Karnataka, India5 yrs 6 mos experience

Key Highlights

Expert in designing scalable data pipelines.
Automated ingestion of large datasets into AWS.
Mentored over 400 professionals globally.

Stackforce AI infers this person is a Data Engineer specializing in Fintech and SaaS with expertise in cloud-native data solutions.

Contact

nishantnayan2002@gmail.com LinkedIn

Skills

Core Skills

Data EngineeringCloud PlatformsData SecurityData MigrationEtl Processes

Other Skills

SQLPythonAWSGlueStep FunctionsLambdaAthenaAWS Step FunctionsAWS LambdaAWS GluePySparkAWS DMSAWS RDS-PostgreSQLWix Website BuilderWeb developer

About

Experience

5 yrs 6 mos

Total Experience

1 yr 10 mos

Average Tenure

1 yr 9 mos

Current Experience

Ibm

Data Engineer

Aug 2024 – Present · 1 yr 9 mos · Bengaluru, Karnataka, India · Hybrid

As a Data Engineer at IBM, I focus on designing and managing large-scale, cloud-native data pipelines in the finance domain, with a strong emphasis on security, compliance, and performance optimization.
𝐊𝐞𝐲 𝐂𝐨𝐧𝐭𝐫𝐢𝐛𝐮𝐭𝐢𝐨𝐧𝐬:
Automated Data Ingestion: Built a daily pipeline ingesting 25+ on-prem P2P files into Amazon S3, orchestrated via AWS Step Functions, Lambda, and Glue, enabling partition refreshes and incremental querying in Athena.
Aggregation Framework : Designed a configuration-driven data aggregation framework where dependencies and SQL transformations were defined in JSON and executed with Step Functions + PySpark Glue + Lambda, eliminating manual intervention in daily aggregations.
Data Security & Tokenization: Designed and benchmarked tokenization workflows for sensitive PII (e.g., Account Numbers, SSNs), leveraging AWS Lambda + Athena to process 500M+ records and optimize query performance while ensuring compliance.
Database Migration: Migrated non-PII financial datasets from Oracle DB to Amazon S3 via AWS DMS, ensuring schema compatibility, integrity checks, and reliable large-scale data transfer.
Performance Optimization: Supported analytics teams by optimizing SQL queries and managing database configurations, improving efficiency and data integrity.
Cross-Account Setup: Assisted in cross-account DMS setups, enabling seamless and secure database migrations across environments.

SQLPythonAWSGlueStep FunctionsLambda+3

Cognizant

2 roles

Programmer Analyst(AIA-ModernDataEngineering)

May 2022 – Aug 2024 · 2 yrs 3 mos

As a Data Engineer, I independently developed and implemented a comprehensive data acquisition and processing system. This included extracting data from 10+ APIs, creating 40+ flat files for analytics, and streamlining data loading and merging into target tables with efficient stored procedures, improving data processing workflows and accessibility.
Led end-to-end ETL processes, successfully migrating complex data from on-prem DB2 and mainframe systems to AWS RDS-PostgreSQL, improving data retrieval speed by 30%. I also redesigned system architecture for better scalability and developed automated workflows using AWS Lambda and Step Functions. Additionally, I implemented email notifications via SNS to keep track of job statuses, making the whole process more efficient.
Orchestrated complex data transformations by managing Glue jobs with PySpark and parsing Cobol files, reducing processing time by 20%. I also integrated Essbase Cube data extraction into the migration framework and implemented automated Change Data Capture processes.
I created detailed architectural diagrams for ETL workflows and mainframe logic, which improved communication across teams. I also conducted thorough unit testing on PySpark code, ensuring high reliability and delivering solutions that exceeded client expectations.

AWS RDS-PostgreSQLAWS LambdaStep FunctionsPySparkSQLData Engineering+1

Programmer Analyst Intern

Jan 2022 – May 2022 · 4 mos

During my internship, I gained hands-on experience in AWS technology, SQL, Python, and basic big data concepts. I collaborated with a team of professionals to develop and implement data solutions, leveraging the power of AWS services such as EC2, S3, Lambda, and Glue. I gained proficiency in writing complex SQL queries and utilized Python for data processing and analysis.

AWSSQLPythonData Engineering

Self-employed

Website Developer

Jun 2020 – Dec 2021 · 1 yr 6 mos

During the lockdown period while pursuing my B-Tech degree, I worked as a freelance website developer. I successfully created numerous websites for clients and relatives, including both paid and free projects, using popular website builder tools such as WordPress and Wix. That opportunity made me to learn, grow, and contribute to the online presence of various businesses and individuals.

Wix Website BuilderWeb developer