Jun 2024 – Present · 1 yr 10 mos

• I lead teams consisting of ~30 Data Scientists, Analytics Engineers, and ML Engineers covering Monetization, Pricing, Commerce Life Cycle, Payments, Partnerships, and Forecasting at Netflix.

Jan 2023 – Jun 2024 · 1 yr 5 mos

• I lead a team of Algorithms Product Managers to support Growth and Commerce at Netflix. Our work includes using ML to optimize and improve consumer experiences and business outcomes in Messaging, Identity, Monetization, and Payments.

Dec 2021 – Jan 2023 · 1 yr 1 mo

• I lead cross-functional teams to apply machine learning in ways that make Netflix better for our consumers. My domain includes:
• Our personalized messaging program, including email, push-notifications and in-app notification center.
• All applications of ML in our Growth Organization, including customer acquisition, programmatic advertising, payments processing, help-center, identity, and security

Mar 2019 – Dec 2021 · 2 yrs 9 mos

• I am a product manager on the Algorithms team, largely responsible for personalization in the Netflix service. I focus on machine learning algorithms that recommend titles to our users through email and push notification. I have broad ownership of algos that purchase programmatic ads for Netflix by deciding what titles to promote in which countries at what spend levels. I also broadly support our Growth teams, developing algorithms that assist in customer service, payments processing, and identifying fraud. I work with cross-functional teams of data scientists, engineers, and business stakeholders to identify the biggest opportunities for machine learning to improve the business, develop roadmaps to prioritize investments in the space, and design A/B tests to measure the impact of our work.
• The common theme of these projects is an obsession with incrementality; we build algorithms that measure the causal impact of the interventions we serve and then optimize the experience. Nothing is more fun for me than spending time with stunning colleagues to identify a new lever we can pull, figuring out how best to measure it, designing a system to optimize it, and seeing how it worked in an experiment.

May 2017 – Feb 2019 · 1 yr 9 mos

• We are obsessed with the economic causal impact of our marketing campaigns; we design experiments measure their marginal cost against their marginal value and build algorithms that respond to incrementality. I specialize in setting up unique experiments when simple A/B designs will not provide the necessary learnings, then applying econometrics to Netflix’s rich datasets to obtain actionable insights.
• My contributions in this space include:
• Working with stakeholders and leaders to build and prioritize U.S. marketing testing agenda to inform key strategic and tactical decisions
• Analyzing complex experiments using econometric methods
• Authoring analysis memos providing actionable business insights to top decision makers
• Building software to automate experimental analysis pipeline and facilitate panel data causal inference from billions of unique data points
• Presenting experimental results to wide array business stakeholders
• Programming and facilitating key scientific meetings to push methodological alignment amongst scientists and business partners.
• Building partnerships across product, marketing ,and content organizations.
• Recruiting and interviewing potential team members
• Mentoring team members
• In my role at Netflix, I have broad leeway to prioritize my own workflow, I establish relationships with key business stakeholders to identify opportunities where I can add immense business value, and have established a track record of delivering on those opportunities.

May 2021 – Present · 4 yrs 11 mos · San Francisco Bay Area

• I am taking everything I have learned in industry about running experiments and teaching it to future economists and data scientists

Jan 2017 – Apr 2017 · 3 mos · San Francisco Bay Area

• As a Manager in the labs, I remained an individual contributor on identity fraud use-cases. I designed a fraud model using a simple bayesian filter that would update the probability that any cardholder was a fraudster, based on a new stream of information. I also designed the evaluation framework for this use case, which would measure the point-in-time accuracy of the model for any cardholder, and measure the total losses saved with fraud prevention. In my capacity as a manager, I also served as the execution lead for a team of data-scientist, identifying and assigning tasks within a sprint, and tracking the progress of our teams work to meet business objectives.
• I am passionate about understanding the expertise that partners and stakeholders within the business have, and using science to scale and inform their decisions. I collaborated with Capital One Designers to develop a set of tools, that combined data expertise and Design Thinking style ethnographic research to uncover the latent models that experts have and encode them into features in a model. We made use of this toolkit to identify and test several hypotheses about features in a predict fraud model. I also co-designed a curriculum: Design Thinking For Data Scientists, and trained upwards of 40 data scientists with the skills to see business partners as yet another source of data.

Jan 2015 – Jan 2017 · 2 yrs · San Francisco Bay Area

• As Data Scientist in the labs, I worked on a number of innovative machine learning use-cases with Capital One’s rich data sets. These projects included transaction fraud algorithm to identify compromised-merchants (patent-pending). This work included feature engineering and testing a variety of possible learning algorithms including naive-bayes, kernal density, random forrests and GBMs. I also focussed on use-cases to help our customer’s financial health, including algorithms to predict and warn customers about overdraft. Some of my work was focussed on new tools to assist analysts with data accessibility, building a simple front end query tool for a NoSQL database.
• In this capacity programmed primarily in python, and used tools such luigi, spark, and h2o. During this time, I also led collaborations across the labs, with engineers and designers, and managed a summer intern.

Sep 2014 – Nov 2014 · 2 mos · Palo Alto, CA

• As fellow, I developed pricemyticket.me, a tool to help people sell baseball tickets on the secondary market. This app scrapes ticket sales data and stores the observations in a mySQL database. The supply and demand of tickets are then analyzed in python, using random forests regression to predict the selling price for any ticket in the stadium and random forrest classification to predict the probability of selling a ticket at any listing price. This app also includes D3 to visualize the selling price of tickets.

Apr 2013 – Aug 2014 · 1 yr 4 mos · Santa Monica

• I provided decision-support services to client organization as they faced critical business decisions using Robust Decision techniques. In this capacity, integrated demand forecasts, cost estimates, and a queuing model into suite of modeling tools to support an analysis of alternative business investments. I managed datasets of model results, completed sensitivity analysis, and used statistical techniques to define planning scenarios. In doing so, I delivered recommendations of robust strategies to executives.

Sep 2009 – Dec 2014 · 5 yrs 3 mos · Santa Monica, CA

• My research provides decision-support to policy stakeholders as they identify strategies that are robust to deeply uncertain future conditions.
• This research entails consulting with client organizations to understand a particular policy context, coupled with a toolkit of economics, data-mining and statistics, and data visualization. In this capacity, I have served the lead modeler on multiple projects to generate simulation models of policy systems. I manage databases with millions of simulation inputs and results, generating code to summarize observations as policy-relevant metrics. Next, I use statistical and data-mining techniques (regression analysis, clustering algorithms, and Bayesian classifiers) to draw policy insight. I produce interactive Tableau dashboards to allow stakeholders to consider the trade-offs between strategies and explore across planning scenarios.
• As an application of this research, I have served as a consultant for Los Angeles Department of Health to help them allocate resources amongst multiple interventions to prevent the incidence of HIV. I have also consulted for water agencies across the Western U.S., the World Bank, and the U.S. Bureau of Reclamation to incorporate decision-analytic tools into their planning processes to address climate change.

Aug 2007 – Jun 2009 · 1 yr 10 mos · Menlo Park, CA

• I served as consultant in litigation and business strategy with applications in health insurance, pharmaceutical pricing, Medicare claims data, antitrust, and intellectual property. I used SAS and Excel to manage data while executing statistical and economic analysis with large data sets. I assisted in all phases of economic research, including data collection and management, statistical analysis, generating graphics and report writing, while working successfully in small teams.