Jul 2020 – Feb 2021 · 7 mos

• led a team of five in migrating the data pipeline from Hortonworks infrastructure to managed AWS Glue. This project was crucial in enhancing the efficiency and reliability of our data management. I also managed daily requirements for different analytical and reporting purposes, which allowed me to demonstrate my leadership skills and ability to manage complex migration projects.
• Tools/Technologies - Spark, AWS, Hive, HDP(Ambari), Java, Scala, Spark MlLib, Redshift, Kafka, S3, EMR, Azkaban and Jenkins

Aug 2018 – Jul 2020 · 1 yr 11 mos

• At Wynk, I developed and managed the Data Flow Pipeline for Wynk Music and Airtel TV, which was designed to perform Big Data analytics. This project involved handling enormous volumes of data daily, reaching up to 3+TB/day, with a Data Warehouse exceeding 1+PB in size. Additionally, the system was built to accommodate a billion music streams every month. A significant part of my role was creating various revenue reports for our Content Partners, which was a critical part of our business model.
• I also designed and implemented the Overture, a Machine Learning Pipeline, from scratch. This project ranged from the proof of concept to the final product. My responsibilities included developing a spark-based ML pipeline, model development, training, and writing a scalable module. The module was designed to understand more than 60 million active users' intent by crafting features from their implicit behaviour on the app, showcasing my skills in Machine Learning and User Behavior Analysis.
• Tools/Technologies - Spark, AWS, Hive, HDP(Ambari), Java, Scala, Spark MlLib, Redshift, Kafka, S3, EMR, Azkaban and Jenkins

Sep 2016 – Jul 2018 · 1 yr 10 mos · Gurgaon, India

• At my previous position, I developed and managed a Data Flow Pipeline that handled significant volumes of data, approximately 1TB/day, and a Data Warehouse (DWH) that had capacity exceeding 1PB. This project involved dealing with extensive amounts of information and required exceptional skills in data management and analysis.
• Another key project I worked on was the development of an Invoice Generation and Reconciliation system. This system was designed to automatically generate invoices for interconnect partners, allowing for the recovery of costs due to billing discrepancies and duration anomalies. I delivered this scalable application from proof of concept to final product, demonstrating my ability to manage a project from its inception to completion.
• Additionally, I worked on the Margin Simulation and Analysis application and the Generic SQL Framework. The Margin Simulation and Analysis application provided operators with the ability to perform wholesale-retail profitability analysis and optimisation. On the other hand, the Generic SQL Framework was designed to process both old and new data streams and automate various batch jobs. The objective of this framework was to facilitate the processing of new client data. Both of these projects were delivered from scratch, from proof of concept to final product, showcasing my ability to handle diverse projects successfully.
• Tools/Technologies - Spark, HBase, Hive, CDH, Java, Scala, Hadoop, Kafka, HDFS, Presto, Oozie and MySQL

Aug 2015 – Aug 2016 · 1 yr · Gurgaon, India

• At HT, I had the opportunity to work on a key project, the Micro-site Generator. This project involved creating a multi-page site to showcase various aspects of the company such as products, work culture, job openings, and contacts. The site was customizable and could be used for branding and recruitment campaigns. I was part of a two-member team responsible for delivering this application from scratch. My primary responsibilities included schema design, model development, writing APIs, and ensuring backward compatibility with the existing system.
• Another significant project I worked on was the Shine Recruiter, a platform where recruiters could post jobs online. The unique feature of this platform was its 2-way matching engine and access to the fastest growing candidate database. My role in this project was to develop restful APIs, integrate new features like OTP, manage job dashboards, admin for managing sales and packages, and bug fixing. Both these projects have equipped me with valuable skills and experience in application development and project management.
• Tools/Technologies: Django, MySQL, Angular JS, Mongo Engine, Django Rest Framework

Jan 2014 – Jul 2015 · 1 yr 6 mos · Gurgaon, India

• Involved in the end to end development of Merchant Dashboard, for managing new or already registered merchants. Using registration module, upload product module and billing module merchants are able to provide product feed and bid for CPC (cost per click) respectively.
• Crawling reviews from different websites, we use these reviews to calculate wize-rating of product and also generate the points which is best suited feature for a particular product. Python, Numpy, Sklearn, Pandas, UrlLib, NLP.
• Internship 5 month (Jan2014-May2014)
• Developed a tool WizeBottomline, used by internal member of company for writing expert reviews for product, using live site snapshot of product and user reviews provided by the tool.Also developed Review crawler for 3rd Party websites, in turn used to calculate wize-rating of product and pinpoint the best features of the product(Python, sklearn, Numpy, UrlLib, Machine Learning).