Sep 2023 – Feb 2025 · 1 yr 5 mos · Warsaw, Mazowieckie, Poland · On-site

• Implemented and handling the end-to-end application and resolving real time issues in Batch & Real time Streaming processing of Data.
• Expertise Coding in Scala & PySpark with emphasis on tuning/Optimization in Performance.
• Worked on Databricks using PySpark for handling the complex data frame to Structured Data.
• Worked on ETL development using SAS/PySpark and Databricks.
• Involved in Architectural design and Big Data Pipeline Designing.
• Scheduling the Jobs using SAS Flow manager and worked on Oozie Scheduler and Airflow.
• Worked on SAS DI Studio to Extract and Transform the data.
• Involved in gathering requirements, impact analysis, designing and development.
• Updating the Code in the GitHub in Prod & Dev for Synchronous the Code.
• Collaborating with Business Analysts to understand the functional requirements.
• Experience on the root cause analysis’ mindset to problem and issue resolution.
• Experience on break down complex problems and find solutions by using logical and analytical thinking.
• Self-motivated to explore new technologies.

Mar 2022 – Sep 2023 · 1 yr 6 mos · Bengaluru, Karnataka, India

• Responsibilities:
• 1 Implemented and handled the end-to-end application and resolved real-time issues in Batch &
• Real-time Streaming processing of Data.
• 2 Ingesting the data from on-premises to AWS Cloud by using Data Migration Service..
• 3 Expertise in Coding in Scala & PySpark with emphasis on tuning/Optimization in Performance.
• 4 Set up IAM policies and roles for users and services.
• 5 Provided technical leadership and delivered innovative ideas for data modernization on AWS.
• 6 Implement data pipeline using s3, Athena, Lambda, and Quicksight for data analysis.
• 7 Created lambda functions to trigger glue job and crawler.
• 8 Experienced with workflow orchestrator tools with Oozie and Airflow.
• 9 Worked on Databricks using PySpark for handling the complex data frame to Structured Data.
• 10 Used Bit bucket & Jenkins for Deployment

Mar 2021 – Mar 2022 · 1 yr · Bengaluru, Karnataka, India

• In the Cargill, from the CMT commercial the Sales transaction of JSON data we are getting and creating the insights from Impala by the Business Logic and sending those insights to the Power BI to create Dashboards.
• Skills Involved: Spark, Scala, Hive, GitHub, Tableau dashboard, Unix, and OOzie workflow.
• Programming/Scripting Languages used: Scala & Shell Scripting.
• Responsibilities :
• Implementing the business logic using Spark and providing the business teams with insights through Impala tables.
• Development of Data Lake components in Spark and storing the tables data in Impala & Hive.
• Processing JSON data to create structured Impala tables.
• Writing the DDL, DML, VIEWS, ALTER queries in the Hive/Impala according to the requirement and displaying the Results.
• Involved in gathering requirements, impact analysis, designing, and development.
• Involved in writing Hive scripts and Spark SQL scripts for data processing as per business requirements.
• Collaborating with Business Analysts to understand the functional requirements.

Oct 2017 – Mar 2021 · 3 yrs 5 mos · Hyderabad Area, India

• Data Engineer Analytics Team
• In British telecom, I am working at Hadoop as a Service Platform (HAAS) in the Broadband team. BT is providing Big data Solutions for diagnosing and resolving broadband service issues proactively even before customer realizes the Problem. There are millions of hubs (home and business) that are posting data through the IoT gateway to us. We will collect those huge data and send it to the Kafka Cluster. The Spark Streaming will process the data from Kafka and enriched those data and store it in the HDFS through Flume. We also send that information to the Dashboard which we will graphically represent the data in the Grafana Dashboard.
• We will generate the Hubs data and its count information and analysis & insights those information and send it to the client. In the Cloudera Manager, we will see the health checks of the cluster and monitor the remaining information in the Cloudera Manager server.
• Skills Involved: Hive, Spark, HDFS, Sqoop, Oozie, Kafka, Map Reduce, Flume & Impala.
• Programming/Scripting Languages used: Scala & Shell Scripting.
• Responsibilities and Contributions :
• ● Responsible for end-to-end deployment of multiple modules/datasets
• ● Involved in multiple data migration projects (RDBMS to Hadoop)
• ● Responsible for data processing, data analysis, data validation, and impact analysis
• ● Real-time data processing using Spark Streaming
• ● Involved in code reviews and test case reviews

Jan 2017 – Jun 2017 · 5 mos · Delhi, India

• Embedded Quiz Monitoring System With Team Performance And Evaluation.
• This project is useful for a 4-team quiz contest, although it can be modified for more number of teams. This system is sensitive. The circuit can detect and record the first hit contestant among all the contestants that may appear to be simultaneous.This project not only monitors the fastest finger, but also evaluates the performance of the contestants by saving marks of all participants in microcontroller registers as per Quiz Masters guidance.