Apr 2023 – Oct 2024 · 1 yr 6 mos · Toronto, Ontario, Canada · Remote

• Created comprehensive data pipelines on Google Cloud Platform (GCP) using services like BigQuery, Google cloud storage, dataflow, Pub/Sub and Big table to ensure the dependable ingestion and transformation of data.
• Implemented Pub/Sub as a trigger mechanism for initiating Dataflow jobs.
• Introduced a dynamic approach for generating dataflow flex template jobs, resulting in an 80% improvement in both cost and performance when transferring data between BigQuery tables which were later triggered by airflow dags.
• Deployed containerized application to Google Cloud run.
• Developed Jenkins pipelines, using Groovy, to facilitate the data ingestion from DB2 and Oracle databases to BigQuery.
• Implemented CI/CD pipelines using GitHub Actions workflows to deploying code seamlessly from the Development environment to User Acceptance Testing (UAT) and subsequently to Production.
• Diagnosed and resolved critical production issues in a high-availability environment, reducing downtime by 30% through root cause analysis and proactive system monitoring.
• Actively involved in all phases of the Agile lifecycle, including sprint planning, scrum calls/stand-ups, story grooming, and sprint retrospectives.

Sep 2022 – Dec 2022 · 3 mos · Toronto, Ontario, Canada · Hybrid

• Development and execution of SQL scripts for data validation across multiple environments.
• Project task tracking was done through Azure DevOps.
• Analyzed and developed various test data scenarios so that the process would run smoothly in production.
• Development and documentation of project artifacts.
• Debugging failures in production and providing resolutions.
• Assisted in deployment activities.

Feb 2019 – May 2021 · 2 yrs 3 mos

• End-to-end implementation right from requirement gathering, analysis, estimation, validation, and data quality assurance.
• Worked on Migration of client’s Teradata Datawarehouse to google cloud platform using Big Query scripts and stored procedures.
• Rectifying and fixing issues encountered while matching production data on GCP and Teradata.
• Implemented data pipelines using google cloud dataflow and scheduling through Apache airflow.
• Worked on loading data into big query tables through files present in Google Cloud Storage.
• Involved in code reviews for efficient and quality deliverables.

Jun 2016 – Jan 2019 · 2 yrs 7 mos

• Development of scripts to migrate and load historical data from the Teradata Relational database to the Google cloud platform using Sqoop and Hive.
• Development of SQL scripts for extraction, transformation, and loading of data using Big Query.
• Developed batch data processing through Spark.
• Aware of monitoring jobs on Control-M and taking required actions as per requirements.
• Rectifying and fixing issues encountered while matching production data on GCP and Teradata.
• Developed Ingestion pipelines (batch and incremental) using Apache Sqoop.
• Developed Hive scripts according to user requirements for analysis purposes.
• Importing and exporting data from RDBMS to HDFS and vice versa using Sqoop.
• Worked with structured as well as unstructured data.
• Good understanding of partitioning and bucketing in the hive.
• Solved performance issues by optimizing queries.
• Analysis of Data with help of ETL code.
• Collaborated with the onshore team to establish business needs.