Chandni Gupta

Data Engineer

Lithuania6 yrs 10 mos experience

Key Highlights

Expert in building scalable data pipelines.
Proficient in cloud technologies with AWS and GCP certifications.
Strong background in data quality and validation frameworks.

Stackforce AI infers this person is a Data Engineering expert with a strong focus on cloud-based solutions and data quality frameworks.

Contact

Skills

Core Skills

Data EngineeringData QualityCloud ComputingData ArchitectureData Processing

Other Skills

AWSAWS GlueAgile DevelopmentAirflowAmazon AthenaAmazon Web Services (AWS)Apache SparkAutomation ScriptAzureAzure Data FactoryAzure Data LakeAzure Data StudioAzure DatabricksAzure SQLBig Data

About

Certified Data Engineer with expertise in Python, SQL, Spark, and data orchestration using Apache Airflow. Skilled in building scalable data pipelines and optimizing data workflows for enhanced efficiency and performance. Proficient in cloud technologies, holding certifications in AWS and GCP, and experienced in leveraging these platforms to architect robust and cost-effective data solutions. Extensive hands-on experience with distributed computing frameworks such as Hadoop and Hive, enabling the processing and analysis of large-scale datasets. Additionally, proficient in utilizing Tableau for data visualization, providing actionable insights to drive informed decision-making. Passionate about leveraging the power of cutting-edge technologies to transform raw data into valuable business insights. Committed to continuously learning and staying updated with emerging trends and advancements in the field of data engineering.

Experience

6 yrs 10 mos

Total Experience

1 yr 8 mos

Average Tenure

2 yrs

Current Experience

Accenture baltics

Data Architecture Senior Analyst

Jun 2024 – Present · 2 yrs · Vilnius, Vilniaus, Lithuania · Hybrid

Built a pip-installable data quality validation tool for semi-structured and structured data using JSON Schema.
Enabled schema flattening, hydration, and automated validation using Snowflake, Python, and SQL.
Integrated Pytest for automated testing and Jinja for dynamic schema generation.
Developed a proof of concept (PoC) for data migration from Posit server to Azure, evaluating pipelines and tools for future transition.

PythonSQLSnowflakeData EngineeringData Quality

Globant

Senior Data Architect

Sep 2022 – Jun 2024 · 1 yr 9 mos · Pune, Maharashtra, India

1. Built a scalable data quality framework for batch and streaming ingestion using Talend, Oracle, Salesforce, and DMS, with AWS services (Lambda, Glue, SNS, SQS, EventBridge) for automated validation and cleaning.
2. Enabled secure, real-time analytics by implementing tokenization and Snowpipe-based ingestion from S3 to Snowflake.
3. Configured and managed Amazon EMR clusters, streamlining resource utilization to significantly reduce migration time.
4. Collaborated across diverse teams, orchestrating performance optimization strategies and crafting scalable data migration processes using Apache Spark, Hadoop, and EMR.
5. Leveraged SQL queries and Python scripting for ETL tasks, ensuring data integrity and accuracy throughout the migration lifecycle.
6. I thrive on crafting innovative solutions and collaborating with cross-functional teams to drive seamless data migration, ensuring optimal performance and accuracy.

TalendOracleSalesforceAWSPythonSQL+2

Themathcompany

2 roles

Data Engineer - Associate

Promoted

Jul 2021 – Sep 2022 · 1 yr 2 mos

1. Designed scalable data processing pipelines with BigQuery, Dataproc, and Airflow
2. Leveraged SQL and Python for data manipulation and informed decision-making
3. Developed Apache Spark workflows for large dataset optimization
4. Ensured data reliability through SQL and Python-based checks
5. Collaborated with cross-functional teams and managed data pipelines with Airflow for reliable execution

BigQueryDataprocAirflowSQLPythonData Engineering+1

Data Engineer - Analyst

Jul 2021 – Jul 2022 · 1 yr

Accenture

Data Engineer

Aug 2019 – Jul 2021 · 1 yr 11 mos · Maharashtra, India · On-site

1. Designing and implementing an Automation Script that streamlined the validation process for data loading into databases, enhancing efficiency and generating insightful reports.
2. Proactively monitoring and analyzing data flow from source to lake, ensuring the quality and integrity of data at every stage.
3. Managing incidents and service requests effectively, demonstrating adeptness in issue resolution and ensuring seamless operations.
4. Played a pivotal role in resolving and optimizing AWS Glue jobs and step functions, streamlining data processing and ensuring the smooth execution of workflows.

Automation ScriptAWS GlueData EngineeringData Quality