Aug 2019 – May 2023 · 3 yrs 9 mos · Pune Area, India · Hybrid

• Working with NLP team as Senior Data Enngineer on projects related to Data enrichment include ETL work which includes the analysis of trades related activtities received from the clients.
• It involves sourcing of data from various platforms like from HBase,Kafka ,extracting the data using pyspark jobs ,making transformations as needed load to various hive table.
• Hive table content are used for reporting purpose which includes client-trader activities,Mainly the reporting is done on tableau platform.
• Tools used:Autosys(Job Scheduling ),Oracle,Pyspark,SparkSQL,Python, Shell Script

May 2018 – Aug 2019 · 1 yr 3 mos · Pune Area, India

• I worked as Data Engineer & Analyst in BI Team.
• The Project handles various partner’s data in collaboration with tableau Reports which was our end output for the client.
• Worked on GCP with Bigquery,Buckets ApacheBeam using python & Core Java language.
• As per my Role, my focus is on the data, how better analytics can be performed on the data So that it can useful for our customer.
• Created various python & java jobs in Apache beam framework for data ingestion and visualization.
• We had multiple jobs running for various data sources, every job runs on a daily basis to fetch required data and used in tableau reports.
• I do have experience in Unix automation, shell scripting.
• Technologies : BigQuery, Tableau, Python, Shell.

Jan 2017 – Apr 2018 · 1 yr 3 mos

• ● Developed Hive queries to parse the raw data, populate staging tables in parquet format and store the refined data in partitioned tables in the EDW.
• ● Provided desired transformed data to the analytics team for comparison between EDW and historical data.
• ● Created Oozie workflows to automate data loading from/into the Hadoop Distributed File System.
• ● Created Hive tables to migrate the entire history and archive from the legacy Oracle data warehouse.
• ● Collaborated with infrastructure, network and BI team to ensure work quality and availability.
• ● Tested and validated the raw data as well as the end results in order to ensure the quality of work.

Dec 2014 – Dec 2016 · 2 yrs

• Anthem Health Care company had planned to analysis and move historical data from TERADATA to BIGDATA Environment for different transformations through various zones and aim is to fit all the data in the processed data back to Teradata.
• Part of Data Preparation, Enrichment and Analysis Team.
• Preparation of ddl and hql scripts for different zones.
• TDCH is used for importing and exporting data from Teradata to Hadoop and vice-versa.
• Technologies used Sqoop,Oozie,Hive,Pyspark,SparkSQL for implementation work.
• Designed and developed Hive queries (HiveQL) as per the business requirement.