Jul 2023 – Nov 2024 · 1 yr 4 mos · Bengaluru, Karnataka, India

• 📌 Leading all data-engineering initiatives with Unacademy.
• 📌 Cost Optimization across all Data engineering initiatives.
• 📌 Designed and maintaining scalable data pipelines, and ensuring data infrastructure aligns with business needs.
• 📌 Driving innovation by implementing new technologies, optimizing data processing systems, and collaborating with cross-functional teams to support data-driven decision-making.

Jun 2022 – Jun 2023 · 1 yr · Bengaluru, Karnataka, India

• 📌 Developed high-quality modern pipelining, warehousing, and reporting solutions and deliver them to customers across teams, making data more useful to the enterprise
• 📌 Designing data solutions to help customers, and so the organization, grow fast
• 📌 Developed a transactional Data Lake utilising Hudi and Spark supporting wide range of sources/formats.
• ➡️ ACID Compliance using Apache Hudi
• ➡️ Utilize Any Data Source - Support different data sources like RDBMS, Rest APIs, Web Sockets,
• Streaming Data, Flat Files, Clicks Streams
• ➡️ Cost-Efficient Solution
• ➡️ Universal Platform for all Data
• ➡️ Built adaptable pipelines for data lake using Spark and Airflow, facilitating schema evolution and data validations.
• ➡️ Integrated Hive query engine for BI reporting on data lake and transitioned analytical SQL workloads from Redshift using DSL.
• 📌 Constructed ETL/reverse ETL pipelines transferring terabytes of CRM data between RDBMS while preserving entity relationships.

Feb 2021 – Jul 2022 · 1 yr 5 mos · Bengaluru, Karnataka, India

• As a data engineer for Walmart's SEO product, we were putting efforts to drive organic traffic and enhance the company's online visibility on Google and other search engines. Through innovative data engineering approaches, We've created a powerful Keyword Datalake that offers a comprehensive 360-degree view of all keywords with retail intent being searched by users. With this valuable tool at our disposal, we're able to make strategic decisions and optimize our SEO strategy, ultimately improving Walmart's search engine rankings and driving more organic traffic to the site.
• Tech Stack: Java, Python, Scala, Apache Spark, Hadoop, Google Cloud Plateform (GCP), graphQL, Hive, Cassandra, Airflow and others.

Feb 2018 – Jan 2021 · 2 yrs 11 mos · Pune Area, India

• Current Project - Responsibilities include Data Ingestion, Data Transformation and Data Analysis using various Hadoop Techniques.
• Important insights below.
• Deal with about a billion events per day with overall data size of around 4 TB in the Data Lake
• Building automated ingestion pipeline to handle the snapshots of the SOR tables.
• Building common libraries that can be reused across various project for client.
• Implementation of the ETL pipelines that spawn across different networks.
• Delivery of the project in phases.
• Working with the client’s technical team to understand the integration points.
• Production Support
• Previous Project - Worked on project based on Internet of Things(IoT) solution using Big Data technologies.
• Important insights below.
• Developed reliable historical data(Sensor/ Calibrators data ) load pipeline from sql-server to apache Kudu using Sqoop, Apache Hive, Apache Impala.
• Prediction of abnormalities on real time sensors data coming through Eclipse Kura using OSGi bundle embedding PMMl file having Machine Learning model for prediction.
• Worked on Eclipse Kapua which IoT Cloud Platform.
• Developed the shell script to monitor the various process.
• Storing the same real time data directly into Cloud(Eclipse Kapua) and then sending that data into Apache Kudu for business analysis using Kafka and Spark Streaming.

Mar 2017 – Feb 2018 · 11 mos

• Roles and Responsibilities: Developing and helping team members to complete POC’s for client. Involved in POC’s like Sensex Log Data Processing (Excel File Processing in Map Reduce), Health Care Management System.
• Big Data Stack: MapReduce, Pig, Hive, MySQL, Sqoop, HBase.
• Key Points:
• Analyse or transform stored data by writing MapReduce or Pig jobs based on business requirements.
• Participated in multiple big data POC to evaluate different architectures, tools and vendor products.
• Experimented to dump the data from MySQL to HDFS using Sqoop.
• Involved in developing the Pig scripts for processing customer data.
• Involved in creating Hive tables, and loading and analyzing data using hive queries.
• Handled importing of customer data from various data sources.
• Load and transform large sets of structured, semi structured.
• Implemented Partitioning, Dynamic Partitions, Buckets in HIVE.
• Worked with Sqoop for importing metadata from relational databases.

Mar 2016 – Feb 2018 · 1 yr 11 mos

• Currently working for Barclays's Project of Online Payment Services.
• Project mainly based on development and enhancement of online payments, funds transfer and bill payment .
• Key Points:
• Reintegration of code from live branch to existing branch using GIT.
• Using JIRA to track task completion.
• Using Jenkins to deploy project releases on local testing environments (SIT)
• Using Confluence to create a repository of information gained while working on project.
• Using hp quality centre to keep track of bugs/defects of each release.
• Unit testing each scenario.
• Supporting releases every month for new changes and fixes.
• Monitoring the new changes using Splunk in production.
• Fixing problem records to solve live incidents in payments.
• Using FATWIRE/ZBTT for content management.

Jan 2016 – Mar 2016 · 2 mos

• Started carrier with TCS at Garima Park,Ghandhinagar(Gujrat).
• Completed Initial Learning Programming(ILP) in JAVA Stream. Training was mainly focused on Java, Servlets, Collections, JDBC, ORACLE Database, HTML, CSS,JSP, JQuery, AJAX and Bootstrap.
• At the end of training I completed one online trading project using above technologies.

Feb 2014 – Jun 2014 · 4 mos · Greater Jaipur Area

• I completed my internship at Bodacious IT Hub Pvt. Ltd. in Core Java Technology.