Jun 2023 – Nov 2025 · 2 yrs 5 mos · Beaverton, Oregon, United States · Remote

• Being a part of Consumer and Marketplace organization in North America region, I have helped team in building efficient reports using BI tool and supported the team in providing the data consistently from Snowflake.
• Worked as an interim team lead to drive the business and help peers to fasten the process to meet project deadlines.
• Worked on Flash Data IQ tool which helps business stakeholders to subscribe the subscription and leveraged Meta llama Maverick LLMs and NLP to generate Deeper Insights for each subscription. Have used technologies like Databricks, Maverick Llama 4, Prompt Engineering, Unity Catalog, Ec2 etc.,
• Developed and deployed a Unit Test cases, spark-expectations with data quality checks like calculating moving average for past 7 days for critical data pipelines land it helps to identify issues before business finds out.
• Led a team of 6 data engineers and support engineers in collaborative projects, providing mentorship and technical guidance.
• Optimized existing daily consumer sales pipelines from 2 hrs to 1hr by adding AQE techniques and Spark optimization techniques and refresh Power BI dashboard before SLA.
• Utilized Python, SQL, PySpark, Brickflow, Github Co-pilot, Snowflake, Power BI, AWS S3, EMR, Ec2, Airflow and Docker for ETL development and deployed in Github and Databricks cloud using CI/CD Jenkins.
• Power BI Log Analysis: Used BI log data and created a Power BI dashboard in Databricks using Datalake monitor and presented to our business to understand the scope of different KPI’s used in our project.
• Databricks Migration: Helped our team to migrate all the ETL pipelines in Databricks-sole and able to migrate all our pipelines within deadline and integrate Spark Expectations and meeting conding standards of Maturity Level 2.
• Cost Enhancement: Productionize the two cost metrics like Wholesale Equivalent Second Cost and Standard Cost in Daily Consumer Sales pipeline with 100% coverage.

May 2022 – May 2023 · 1 yr · Irving, Texas, United States · On-site

• Supported providing data solutions for MARS (Michaels Analytics and Reporting System) and AI & Data Platforms teams.
• Worked with multiple teams like RMS, OMS and RESA as source teams and processed the data into multiple Birst reports by providing the data from Hive final curated tables.
• Experienced in processing different delimited format files, parquet files and processed into MARS using Impala and Pyspark.
• Build ETL pipelines using Hql’s, Hive, Hadoop and Spark run all the scripts through shell scripts in cloudera.
• Conducted data cleaning, preprocessing, exploratory data analysis on large datasets, ensuring data integrity & data quality.
• Experienced in running the shell script jobs and spark jobs in Talend7.1
• Utilized Python, SQL, PySpark, Cloudera, Talend8, Google Cloud Platform, Looker/LookML, Big query, Dataproc, Cloud Storage, Power BI, Airflow, Kubernetes and Docker for ETL development and deployed in Bitbucket and GCP cloud using CI/CD Jenkins.
• ● GCP Migration: Worked on migrating all the data from Hadoop on-premise to Google cloud provider Cloud Storage.
• ● Build complex reports for users in Market Place and Maker Place using LookML and worked on building complex queries using Big Query.
• Deploy pipelines in Airflow and used GCP services like Dataflow and Dataproc as spinning the clusters.
• MARS: Experienced in running the shell script jobs and spark jobs in Talend7.1
• Worked on building the jobs using Data Stage and ingesting the data to MS SQL server.
• Good hands-on Experience in automating the shell scripts and Talend job steps in Control M.
• Worked on migrating the jobs from Talend7.1 to Talend8.

Jan 2020 – Apr 2022 · 2 yrs 3 mos · Beaverton, Oregon, United States · Remote

• Data Engineer in Inventory Planning team which is part of Demand Supply & Planning Management organization, I have helped the team in providing data solutions for reporting and alanytics.
• Experience in developing and building scripts in Python and shell scripting languages.
• Good hands-on experience on launching the AWS EMR and EC2 instances and the process of AWS IAM properties.
• Experienced in selecting the right nodes, instance types according to the account limit to increase the performance of the ETL.
• Experience in pySpark with Hadoop platform for processing billions of records which uses in-memory data processing.
• Involved in designing Hive schemas, using performance tuning techniques like partitioning, bucketing.
• Used Airflow scheduler end to end data processing pipelines and scheduling the workflows.
• Utilized Python, SQL, PySpark, Snowflake, Tableau, AWS s3, EMR, Ec2, Cloudtrail, Airflow for ETL development and deployed in Github and AWS cloud using CI/CD Jenkins.
• Airflow Migration: Worked on migrating all airflow dags close to 50 dags from Airflow one to Airflow MAP from IP team.
• CHAI product: Buit ETLs from raw layer to aggregated layer for Channel Allocated Inventory sitting across Global in DSM for different channels like DC(Wholesale), NFS, NVS, Factory, Intransit, Purchase Order, Nike.com, 3pp (3rd party products) at the grain of channel code and product code (style + color).
• ACAI product: Built ETL’s for DC and PO aggregated Inventory at the grain of gtin (style-color-size) code and channel. It will help the business to look for top trending products in different geos and take better decisions based on demand.
• Data Quality Framework: Built common frameworks for Data Quality Framework to improve no loss of data while performing ETL’s across different platforms and canary checks in ETL’s.
• Platform Upgrade Testing: Developed new features like Platform Upgrade Testing using Cloudtrail data and Data Snowflake Telemetry as a team.

Jan 2019 – Dec 2019 · 11 mos · Warrensburg, Missouri, United States · On-site

• Responsible for doing requirement analysis, prepare data model.
• Code implementing for importing the data using Sqoop from RDBMS and Spark Scala for loading the files to Hadoop Hive tables and Spark data frames for transforming the data based on business requirements. Shell scripting for automating the Sqoop and Spark jobs. Code quality with SonarQube
• Responsible for devops activities such as code maintenance in GIT, Jenkins.
• Utilized Python, SQL, PySpark, Tableau, AWS s3, EMR, Ec2 for ETL development and deployed in Github and AWS cloud using CI/CD Jenkins.

Jun 2016 – Dec 2018 · 2 yrs 6 mos · Hyderabad Area, India · On-site

• Responsible in developing this project from scratch using Python, Spark using python (Pyspark) in Agile model.
• Experience in hive partitioning, bucketing and perform joins on hive tables.
• Worked in Spark to read the data from Hive and write it to Cassandra.
• Used Autosys job scheduler end to end data processing pipelines and scheduling the workflows.
• Developed complex logics and generic code for different validations using Spark.
• Worked on Spark SQL for table level validations and fetching meta data from static tables.
• Created job execution engine using Hive-HBase, which tracks job status for each process level, file level and at validations level.
• Trained and lead team of associates and colleagues in understanding the functionality of framework and uplifted them technically in Hadoop, Spark and Bigdata technologies.
• Used bit bucket as common repositories for code sharing.
• Maintained timely delivery in every sprint.