May 2017 – Dec 2021 · 4 yrs 7 mos

• Technical lead for the data infrastructure area. We are responsible for providing Dropbox with data warehousing, distributed compute, streaming, workflow orchestration systems, queuing, data visualization, and BI tools. We have a multi-region presence and are responsible for 40s of PB data, 10s of million events/s.

Apr 2016 – May 2017 · 1 yr 1 mo · San Francisco Bay Area

• Worked in Map Services to build the next generation platform for training models for predicting traffic conditions. This reduced the operational load by ~6x. These models powered a multitude of Uber services such as upfront pricing, in-app driver routing, UberPool matching, among other things. We used AWS, EMR, S3, Kafka, Cascading, MapReduce, Thrift, Hive, Presto, and other big data technologies.
• Also, very briefly contributed to improving reliability of the main backend proxy layer that sat in front of all requests. Wrote a high performance general-purpose circuit breaker Go library as part of this work.

Aug 2014 – May 2016 · 1 yr 9 mos · San Francisco Bay Area

• One of the early founding members of the data team. Led the development
• of high-volume (multi-million events/s) data pipelines. Built a general-purpose very high performance high reliability data ingestion framework that was used for everything from collecting events from the edge network (100s of PoPs) to building customer-facing analytics. Led the end-to-end development of a system to would protect our customers from layer 7 distributed denial of service (DDoS) attacks. Also led the development of a large scale distributed immutable key-value system that was optimized for cost compared to off-the-shelf offerings. Technologies used include Go, Kafka, HDFS, PostgreSQL, Redis, SSTables, ElasticSearch, among other things.
• A very small piece of this work is open-sourced at https://github.com/ashishgandhi/buffer.