Kartavya Singh

Software Engineer

Noida, Uttar Pradesh, India3 yrs 6 mos experience

AI EnabledAI ML Practitioner

Key Highlights

Architected scalable ETL frameworks for complex data processing.
Led automation of data pipelines, enhancing compliance and efficiency.
Developed real-time dashboards driving data-driven decision-making.

Stackforce AI infers this person is a Data Engineering expert in Fintech with a focus on scalable ETL solutions.

Contact

Skills

Core Skills

Data EngineeringEtlData MigrationData VisualizationCloud Computing

Other Skills

AWS GlueAWS LambdaActimizeAgile MethodologiesAlgorithmsAmazon Web Services (AWS)Analytical SkillsApacheApache KafkaApache SparkArtificial Intelligence (AI)Azure Data FactoryAzure Data LakeAzure DatabricksAzure File Share

About

As a Data Engineer with over 3 years of experience, I specialize in building scalable data pipelines, optimizing data processing workflows, and ensuring seamless data integration across various platforms. My expertise includes working with cloud technologies like Azure and AWS, and I have hands-on experience with SQL, PySpark, and ETL tools like Azure Data Factory and Apache Airflow. I am passionate about solving complex data challenges, from data ingestion and transformation to developing optimized solutions for reporting and analytics. I’ve worked with cross-functional teams to design data architecture that drives actionable insights and enhances business decision-making. My goal is to continue advancing my skills in cloud-based data engineering while contributing to innovative and impactful projects. Always eager to collaborate and solve problems, I thrive in fast-paced environments that encourage continuous learning and growth.

Experience

3 yrs 6 mos

Total Experience

2 yrs 8 mos

Average Tenure

10 mos

Current Experience

Agreeya solutions

Senior Data Engineer

Aug 2025 – Present · 10 mos · Noida, Uttar Pradesh, India · Hybrid

Architected a multi-module ETL Framework with Landing, Delta, Transform, Validation, and Logging layers using Python, Pandas, SQLAlchemy, and YAML-driven configuration.
Built a dynamic Landing engine supporting file-based and DB-based ingestion, with configurable delimiters, pattern-based file detection, timestamp-based versioning, and archival/error-folder routing.
Implemented Delta computation logic (new/modified/deleted records) using record-level hashing, key matching, and merge-status tracking.
Developed a robust Transformation Engine using over 20 reusable transformation types (mapping, lookup, conditional, datetime, binning, splitting, masking, merging, rollover, key generation, cleanup, etc.) driven completely by config—no code changes needed for new sources.
Created SCD-style History snapshotting to preserve historical states for audit and compliance.
Designed the Monitoring & Logging module that writes granular LandingLogs, DeltaLogs, ValidationLogs, and TransformationLogs into a centralized monitoring DB for auditability.
Integrated with Control-M, building EXE-packaged Python jobs and enabling scheduler-driven automation for nightly AML data pipelines.
Implemented Source Availability logic using SQL-backed status tracking (VALIDATION → EXPECTED → AVAILABLE), enabling file-arrival-based triggering of ETL jobs.
Built error-resilient pipelines that automatically detect malformed files, incorrect delimiters, missing columns, or row-level inconsistencies and route files to Error/Archive.
Improved pipeline performance & reliability by optimizing transformations, reducing IO overhead, and revising schema-alignment and bulk-insert patterns.
Collaborated with compliance, InfoSec, and platform teams to deploy ETL EXEs on secure Windows servers with proper file-share permissions and Active Directory integration.
Technologies:
Python, Pandas, SQLAlchemy, YAML, Azure File Share, Control-M, SSMS, Git.

PythonPandasSQLAlchemyYAMLAzure File ShareControl-M+4

Western alliance bank

Data Pipeline Engineer | ETL Architecture

Aug 2025 – Present · 10 mos · Phoenix, Arizona, United States · Remote

At Western Alliance Bank, I engineered the entire ETL platform powering AML/OFAC data processing for multiple enterprise systems. I led the buildout of a config-driven, fully automated Python ETL framework that handles landing, delta processing, transformations, history snapshotting, and multi-layer audit tracking.
I developed reusable engines for file ingestion, schema validation, transformation orchestration, compliance rule mapping, and batch processing, all controlled through YAML and dynamically executed across sources like Workday, Transtar, and Corporate Trust.
I also built the pipeline monitoring ecosystem — system logs, micro/macro audit layers, source readiness checks, and Control-M orchestration using PyInstaller-packaged executables. This included deep integrations with Azure File Shares, SQL Server, and enterprise scheduling to deliver a production-ready data engineering platform.
My work enabled the bank to move from manual data handling to a scalable, modular, fully automated ETL architecture with clear lineage, auditability, and error transparency.
Core Highlights:
Designed multi-layer ETL engine (Landing → Delta → Transform → History)
Built transformation framework with 20+ reusable operators
Automated schema alignment, SCD handling, history snapshots
Implemented enterprise audit logging & monitoring
Integrated pipelines with Control-M, Azure File Share, SQL Server
Delivered a production-ready ETL platform for AML/OFAC compliance

Data EngineeringETL

Menoob

Data Engineer

Jan 2025 – Jun 2025 · 5 mos · Noida, Uttar Pradesh, India · On-site

Integrated Firebase Analytics and Google Tag Manager to enhance tracking of user sessions and purchase funnels.
Designed database for company and orchestrated Data Migration pipeline from admin dashboard.
Developed a real-time dashboard for product performance monitoring, leading to data-driven decisions.
Improved checkout funnel efficiency, resulting in a 22% increase in average session time.

Data EngineeringData Migration

Cognizant

2 roles

Programming Analyst

May 2022 – Jan 2025 · 2 yrs 8 mos

As a Programmer Analyst at Cognizant, I specialized in Data Engineering and Cloud Computing, leading initiatives to optimize data workflows and enhance system performance. I built and managed robust ETL pipelines, extracting, transforming, and loading data from diverse sources into PostgreSQL and MySQL databases. My efforts in query optimization and automated data preprocessing resulted in a 67% reduction in manual work and a significant boost in pipeline reliability, reducing incidents by 40%.
Key Achievements:
Designing and implementing a scalable ETL pipeline that processed over 5TB of data daily with 99.98% uptime. I optimized the pipeline to handle data from multiple sources, ensuring seamless integration and efficient processing.
Developed a data validation framework using Python and Airflow, automating anomaly detection and schema validation.
Gained hands-on experience with Microsoft Azure and AWS, leveraging tools like Azure Blob Storage, Azure Data Factory, and AWS Redshift to enhance data pipelines.
Built and optimized machine learning models using scikit-learn, TensorFlow, and Keras, driving improved predictions and insights.
My role involved close collaboration with data analysts and data scientists, ensuring the delivery of clean, structured datasets that empowered actionable business insights and strategic decision-making.

Data EngineeringCloud Computing