May 2019 – Apr 2022 · 2 yrs 11 mos · Hyderabad, Telangana, India

• Played the role of Technical lead and senior developer in the design and implementation of MDM Pipeline to read complex JSON data coming from MDM sources, flatten the data, maintain active snapshots, history, incremental, SCD and Person Xwalks implementations.
• Closely worked with Business Analyst to understand the requirement and converting it into a technical design approach.
• Implemented the frameworks for Data Quality checks, Audit and Lineage report generation, Specific table level data load, Specific data load based on Ingestion/Source.
• Created the Pipeline using the Step function to create the EMR cluster, orchestrate the Lambda/Python/Pyspark scripts, which processes huge amount of data.
• Developed Lambda function to read the source table, record counts and create the Athena views and load the data dynamically into tables based on the record count.
• Developed the Pyspark script to read the nested data from S3/Athena, unnest and generate the processed file for each of the 11 tables.
• Developed the Python script to read the latest processed files and load the data into Redshift stage tables and load the data into the mart table after applying the SCD logic.
• Worked on the design and implementation of Redshift SCD queries.
• Developed the Redshift query to load the Person Xwalks in an SCD Type-2 manner.
• Collaborated and supported Quality Assurance (QA) team in building functional scenarios and validating results.

Jul 2018 – May 2019 · 10 mos · Hyderabad Area, India

• Converting the legacy system to Relational Database system.
• Performing the impact analysis of the new requirement/change requests on the existing DataStage jobs/SQL database objects.
• Creating/Updating the ETL jobs as per the new requirements.
• Preparing and executing the Unit Testcases for the new changes.

Mar 2018 – Jun 2018 · 3 mos · Hyderabad Area, India

• Developing Stored Procedures,
• Creating the reports in SSRS,
• Writing the data validation queries,
• Preparing the unit test cases,
• Resolving the defects raised by the testing team,

Mar 2017 – Feb 2018 · 11 mos · Hyderabad Area, India

• Designing and developing Source to target documents for database tables and Job flows.
• Developing tables in SQL Database.
• Developing Job flows in SAP BODS.
• Unit testing the jobs.
• Writing the validation and count SQL queries for the Job and the SAP BO reports.
• Resolving the defects raised by testing team

Dec 2015 – Mar 2017 · 1 yr 3 mos · Hyderabad Area, India

• Designing/modifying the jobs based on the user requirements/functional changes.
• Improving the performance of existing DataStage jobs in production environment.
• Performing Impact analysis for functional changes/business rules to ensure consistent and error free delivery.

Jun 2014 – Nov 2015 · 1 yr 5 mos · Hyderabad Area, India

• Providing end to end support to end users by monitoring continuous flow of data across the systems globally.
• Performing initial incident investigation and diagnosis for all production issues and keeping a track of time taken by jobs for completion and coordinating with Development Team to improve its performance.
• Conducting break-fix activities for the jobs in hanged/aborted state.
• Scheduling the DataStage jobs in Crontab and Migrating the DataStage jobs.
• Working on ad-hoc requests from various users to load/provide data.

Dec 2013 – May 2014 · 5 mos · Hyderabad Area, India

• Trained on Informatica, Business Objects, Tableau