Feb 2021 – Nov 2022 · 1 yr 9 mos

• Projects:
• 1. Flipkart Data Platform
• One of the largest BigData infra in India
• Managing Petabytes of data across hot, warm, cold storage
• Crunching Terabytes of data hourly over 100K+ cores
• Optimizing critical analytical workloads in Spark, HiveQL, Tez, MR, etc
• Optimizing Hadoop cluster - Qos, Yarn queues, Linux cgroups, etc
• System stability and management at scale
• GCP Analytics stack - BQ / GCS / DataProc / Spark Serverless

Jul 2016 – Mar 2022 · 5 yrs 8 mos

• Projects:
• 1. Catalog Ingestion System
• 2. Flipkart Data Platform
• One of the largest BigData infra in India
• Managing Petabytes of data across hot, warm, cold storage
• Crunching Terabytes of data hourly over 100K+ cores
• Optimizing critical analytical workloads in HiveQL, Spark, Tez, MR, etc
• Optimizing Hadoop cluster - Qos, Yarn queues, Linux cgroups, etc
• Guard railing against rogue jobs
• Tech: HDFS, Yarn, Hive, MR, Tez, Spark, ORC

Sep 2015 – Jul 2016 · 10 mos

• Projects:
• 1. Competitive Intelligence System - Built prioritized crawler system to schedule scraping price/listing data from competitors and signal them to the pricing system. Supported integration with external anonymous proxies such as HMA, Luminati. Built custom generic IP-recycle/IP-churn mechanism over public cloud vendors such as AWS, Azure. Built support for adhoc and short-lived crawls using scrapy
• 2. Catalog Ingestion System - Onboarding product catalog, Fuzzy De-duplication, Catalog indexing, Intelligent Quality check of content - Million products on-boarded daily
• Tech: Nginx, Crawler, Anonymity network, Python, Scrapy, Redis, HBase, S3, Zookeeper, Global throttling, RabbitMQ, Kafka, HBase, Network Anonymity, Apache Camel, Mysql
• Opensource: Priority Kafka Client (https://github.com/flipkart-incubator/priority-kafka-client)

Apr 2014 – Sep 2015 · 1 yr 5 mos

• Projects:
• Dynamic Pricing System - Built system to enable dynamic pricing of millions of Flipkart products based on various signals including competition, stock flux rate, trend, etc.
• Category/Business ops would provide their formula, model or other means as input to the system to carry out price fluctuations.
• Tech: Dropwiz, Mysql, RabbitMQ, Segmentation, Redis

Aug 2013 – Apr 2014 · 8 mos · Bengaluru Area, India

• Projects:
• VMware VMDK Cinder Driver for Openstack - Enabling Openstack user to be able to create mountable volumes or block devices based on VMDK files over vSphere datastores such as VMFS backed or vSAN.
• Tech: Python, Nova, Cinder, Mysql, RabbitMq, vSphere SDK, Openstack
• Opensource: https://blueprints.launchpad.net/cinder/+spec/vmware-vmdk-cinder-driver

Oct 2011 – Aug 2013 · 1 yr 10 mos · Bengaluru Area, India

• Projects:
• Policy based Storage placement in vCloud - Scalable intelligent storage platform, to make decisions about virtual machine placement in the VMware's infrastructure for end user based on policy needs. This platform automates compliance and remediation of the policy.
• Tech: Java, VMODL (internal), Multi-threading, vSphere/vCloud SDK, Software defined storage

Jul 2009 – Oct 2011 · 2 yrs 3 mos · Bengaluru Area, India

• Projects:
• Chargeback for vCloud - Chargeback is a billing/metering software for various compute, network, storage metrics and Qos in the VMware's cloud infrastructure. Using this, service providers can start billing their tenants and end users on reservation or pay per usage basis.
• Tech: MSSQL, vCloudService, vCenterServer, WSDL (SOAP), Java, Spring framework, Hibernate over SQL, Distributed systems

May 2008 – Jul 2008 · 2 mos · Bangalore

• Projects:
• Bunter - SQL query engine over BigTable, SSTable
• Supported SQL constructs: Where, GroupBy, Aggregate functions, 1-Level Join, Projections
• Parse SQL query and auto-generate Chained MapReduce programs
• This was a useful tool for contractors at Google who often had to perform analytics.
• Tech: Map-Reduce, Bigtable, SQL