Jan 2018 – Jul 2018 · 6 mos

• Model-Management Service: A thrift RPC service that performs ML predictions (based on the clipper paper)
• Developed a Thrift RPC service on Java that fetches predictions from a currently deployed model on MLeap or TensorFLow, abstracting out the deployment to clients.
• Enabled automatic discovery and registry of the service on production cluster using Airbnb’s SmartStack.
• Storm on Azure: Deployment code to brings up an azure cluster and deploys a multi node Apache Storm Environment
• Designed and developed a one-click tool to brings up an Azure cluster and deploys a multi node Apache Storm Environment bringing down the deployment time of a storm cluster from 12 hours to 15 minutes.

Jan 2017 – Dec 2017 · 11 mos

• Egret: Real-time user data processing platform on top of Apache Storm and Apache Kafka.
• Created docker deployment and a test framework to test the platform and the clients onboarding.
• Databus Platform: Data transportation platform to move terabytes from servers to hdfs and analytic stores
• Evaluated Apache Flume as a scalable solution for streaming data from event servers to Kafka and created a complete test regression suite on Java and testNG to cut down test time to an 1-2 hours from a day.
• Benchmarked the platform (Flume, Kafka, HDFS) and removed the bottlenecks to improve the throughput and reduce latencies.

Jul 2015 – Dec 2016 · 1 yr 5 mos

• Fast ETL Pipelines: Advertisement data processing pipelines on top of Apache Hadoop
• Enhanced ETL pipelines (LocalDC), processing raw ad request response, render, click, download and billing events based on MapReduce. We were processing 10 billion request per day.
• Developed a test framework on Java and PostGresSQL and a continuous integration setup on Jenkins for LocalDC Pipelines which lead to faster deployment cycles.

Jan 2015 – Jun 2015 · 5 mos

Jun 2014 – Jun 2014 · 0 mo

• WPFApplication Development