Chaitanya Tyagi

Data Engineer

Jaipur, Rajasthan, India2 yrs 8 mos experience

Most Likely To Switch

Key Highlights

Expert in GCP and BigQuery for data migration.
Designed automated frameworks for regulatory reporting.
Passionate about building scalable cloud-native data pipelines.

Stackforce AI infers this person is a Data Engineer specializing in cloud data solutions and automation in SaaS environments.

Contact

Skills

Core Skills

Google Cloud Platform (gcp)BigqueryPyspark

Other Skills

API TestingAlgorithmsAmazon Web Services (AWS)Apache AirflowApache KafkaBig DataBootstrapC (Programming Language)CSSData AnnotationData StructuresDatabase Management System (DBMS)DataprocDjangoExpress.js

About

A Data Engineer at Accenture, I contribute to large-scale migration programs, including transitioning multiple global markets from AMEX’s on-prem Cornerstone platform to the GCP-based LUMI ecosystem. My work focuses on modernizing regulatory reporting and bureau submission workflows, including migrating complex Hive-based ETL pipelines to BigQuery and resolving transformation challenges to achieve output parity. With a BTech from Malaviya National Institute of Technology Jaipur, I have honed my expertise in GCP, BigQuery, and PySpark. I’ve also designed a PySpark-based validation framework to enable automated reconciliation for regulatory reports, ensuring accuracy and efficiency across GCP projects. Passionate about building reliable, scalable, and cloud-native data pipelines, I continuously explore innovative approaches to optimize workflows for production-scale systems.

Experience

2 yrs 8 mos

Total Experience

1 yr 4 mos

Average Tenure

1 yr 11 mos

Current Experience

Accenture

Data Engineer

Jul 2024 – Present · 1 yr 11 mos · Indore, Madhya Pradesh, India · On-site

Contributed as a Data Engineer in a large-scale migration program, moving 10+ global markets from AMEX’s on-prem Cornerstone platform to the LUMI (GCP) ecosystem, modernizing regulatory reporting and bureau submission workflows.
1. Migrated complex Hive-based ETL pipelines to BigQuery, resolving syntax gaps, logic mismatches, and transformation differences to ensure complete output parity between Cornerstone and LUMI.
2. Designed a PySpark-based validation framework (Airflow DAG) to compare regulatory reports generated in LUMI vs. Cornerstone across separate GCP projects, enabling 100% automated reconciliation and removing manual comparison.
3. Handeled ingestion logic to seamlessly move generated reports from use-case datasets into the LUMI Unified Warehouse, ensuring schema harmonization and consistency with legacy standards.
4. Architected the Maestro Orchestration DAG—a Pub/Sub–driven automation engine that:
Receives ingestion completion notifications,
Dynamically identifies the target market,
Triggers downstream ETL pipelines automatically,
Eliminates manual scheduling or human intervention.
5. Built downstream “External” DAGs executing shell scripts and Python utilities to validate edge cases, extract data from warehouse tables, generate CSV bureau files, and deliver files securely via SFTP.
6. Developed a reusable DaaS PySpark utility to generate bureau-ready files, and clean temporary artifacts—fully automating multi-market file creation.