Feb 2025 – Present · 1 yr 2 mos

• Make Unity Catalog feature rich and simple to use.

Aug 2023 – Feb 2025 · 1 yr 6 mos

• Unity Catalog (https://www.unitycatalog.io/).
• Hive Metastore federation (Dec 2024 launch!) https://www.databricks.com/blog/announcing-public-preview-hive-metastore-and-aws-glue-federation-unity-catalog
• Lakehouse Federation Blogs:
• https://www.databricks.com/blog/announcing-general-availability-lakehouse-federation
• https://www.databricks.com/blog/introducing-lakehouse-federation-capabilities-unity-catalog
• Data + AI summit demo: https://www.databricks.com/dataaisummit/session/lakehouse-federation-discover-query-and-govern-any-data-unity-catalog
• Lakeguard: Making shared access to Spark environment efficient, secure, and feature rich: https://www.databricks.com/blog/whats-new-shared-clusters-unity-catalog

Jun 2022 – Jul 2023 · 1 yr 1 mo

• Working on Data Governance - we strive to bring permission control, table/view/ML dataset management, data discovery, lineage and sharing into the unstructured data lake world (S3/Azure Blob/Google Cloud Storage) and beyond (Data Warehouses and Relational Databases). Checkout Matei's announcement on Unity Catalog and Delta Sharing: https://www.youtube.com/watch?v=aRMfxPZxnfc.
• For the last 6 months, I have been working on this amazing new feature called Lakehouse Federation (checkout our blog post): https://www.databricks.com/blog/introducing-lakehouse-federation-capabilities-unity-catalog
• Before that, I had been working on managed/external file system support for Unity Catalog GA! Simon has a good video summarizing what I worked on: https://www.youtube.com/watch?v=aAswROA1bM8

Mar 2020 – May 2022 · 2 yrs 2 mos

• Databricks Compute Fabrics: We work tirelessly so that more than 10 million of Virtual Machines are happily crunching exabytes of data every day. TL;DR: we host spark, and make sure it works out of the box :). Checkout Matei's keynote on our large scale cloud service:
• https://www.slideshare.net/matei/scaling-databricks-to-run-data-and-ml-workloads-on-millions-of-vms?next_slideshow=239227656
• Over the two years I've mainly worked on three projects:
• 1. Secure Cluster Connectivity https://docs.databricks.com/security/secure-cluster-connectivity.html that allows customers to trust connecting their cloud compute to Databricks servers so we can process data in their account. This is the project that taught me how to build a reliable and secure network service.
• 2. Cluster scoped library management https://docs.databricks.com/libraries/cluster-libraries.html. This is the first service I've owned end to end - we built a new microservice from scratch and make sure it's robust.
• 3. Cluster management https://docs.databricks.com/clusters/index.html
• I've mainly worked on instance acquirement and container environment setup, but also many other components in the general cluster creation & acquirement workflow. This experience taught me enormous system/OS knowledge and how to build robust cloud compute services.
• Other projects I've served:
• Enhanced Security Monitoring that provides realtime audit trail of activities on your Spark VMs to detect potential security anomalies.
• https://docs.databricks.com/administration-guide/cloud-configurations/aws/enhanced-security-mode.html
• Databricks Fedramp Moderate Certification: https://marketplace.fedramp.gov/#!/product/databricks?sort=productName&productNameSearch=databricks
• Design of the dataplane -> controlplane mTLS framework that allows API interfaces private to Databricks compute to become possible.

May 2019 – Aug 2019 · 3 mos · San Francisco, California, United States

• Working on the Growth/Billing and Usage Infra Team as an SDE intern, building Spark pipelines and platform services for Databricks' Billing and Usage Infrastructure, which enables customers with better visibility & analysis towards their Databricks subscription.