Nov 2023 – Aug 2024 · 9 mos

Mar 2023 – Oct 2023 · 7 mos

• Architecture for replatforming project:
• Design and document components and their interactions.
• Design the consolidation of all search platforms to opensearch serverless in AWS.
• Define search APIs and proof of concept for Opensearch queries.
• Design integration of the new platform with a data quality tool. Define data quality rules.
• Work with the data architect to define object models, interaction and storage in dynamo db.
• Use SST to create infrastructure as code for above components.
• Design move from home grown etl to DBT
• Skills: Data Build Tool (dbt) · AWS Lambda · SST · Opensearch Serverless · Opensearch · TypeScript · Amazon Dynamodb · Amazon Web Services (AWS)

Sep 2017 – Mar 2023 · 5 yrs 6 mos

• Designed and implemented:
• Event schemas to track searches, impressions, reservations and more across web and mobile applications.
• API for receiving the above events including event validation and production to kafka topics
• Apache Airflow DAGs for loading and processing events.
• Data quality investigations, monitoring and alerting.
• Apache Superset deployment and dashboards
• PySpark
• Real time event consumption from kafka topics using pyspark structured streaming, storage in s3 as parquet, and snowflake table structures for raw events and aggregations.
• Designed and wrote spark jobs to join and aggregate large data sets.
• EMR and data center (kubernetes) implementations
• Troubleshooting and optimization for memory and speed.
• Skills: Snowflake · Kubernetes · Python (Programming Language) · Apache Spark · Apache Kafka · Apache Superset · Apache Airflow · Java · Amazon Web Services (AWS)

Sep 2016 – Sep 2017 · 1 yr

• Update data ETL processes and data structures in transition from AWS to Google Cloud Platform (GCP).
• All data being ingested using Apache Nifi, and imported into Google Cloud Storage (GCS) and Big Query.
• Data ETL layer is a combination of DataFlow java processes and Spark on Dataproc. Tableau is used for data visualization run on Big Query data. Kubernetes in conjunction with Docker is used for deployment of components.
• Languages used: Python, Java

Apr 2016 – Sep 2016 · 5 mos

• Client - Experian
• Architect/Design for moving data from legacy systems to Hadoop.
• Technologies: CDH 5.7, Spark, Scala, Avro, Amazon EC2.

Mar 2014 – Apr 2016 · 2 yrs 1 mo · Santa Monica

• Tech lead for a team of engineers
• Architecture and implementation for using Elastic search as the end location of our map reduce jobs to improve front end search performance and troubleshooting.
• Architecture and implementation of TC's vehicle inventory processing system to use hadoop. Improved processing time from 12-20 hours to 1 hour per run.
• Architecture and proof of concept for prospect search to be powered by an elastic search cluster.
• Architecture and implementation of a system to deduce build data
• Performed code reviews via git pull requests
• Currently evaluating Amazon Elastic Map Reduce (EMR) for our workflows
• Technologies:
• Hortonworks Hadoop Distribution
• Elastic Search
• Avro
• Oozie
• Java Map/Reduce
• Amazon EMR

May 2013 – Mar 2014 · 10 mos

• Redesigned the traffic processing system to use hadoop and associated technologies.
• Designed/Implemented various projects for the retail vertical. Designed move from a legacy in house search system to Elastic Search.
• Technologies:
• Java Map/Reduce
• Elastic Search
• Solr
• Hadoop
• Avro
• Oozie
• CDH 4.4
• Hive

Mar 2012 – May 2013 · 1 yr 2 mos · Pasadena, CA

• Responsible for managing 500+ node hadoop cluster. Projects/Tasks include:
• Capacity and hardware planning
• M/R Job performance profiling and rewriting
• Sprint and release planning
• Choosing a new hadoop distribution; bake off with MapR and Cloudera
• Code reviews
• Hadoop operational support
• Wrote some Rest services to support grid operations and data flow
• Technologies and Methodologies used:
• Hadoop
• Map/Reduce
• Hive
• CDH
• MapR
• Java
• Agile
• Jira Greenhopper
• git

Mar 2009 – Mar 2012 · 3 yrs

• Designed the event schema that all applications used to report searches, clicks and impressions.
• Re-architected the traffic data processing to use hadoop map/reduce jobs as part of processing pipelines managed by Oozie.
• Designed and Implemented partitioned hive tables for analysis by various other teams.
• Capacity/Resource analysis and planning
• Technologies:
• Java
• Cloudera Distribution of Hadoop (CDH)
• Map/Reduce
• Hive
• Oozie
• Sqoop
• git

Aug 2007 – Mar 2009 · 1 yr 7 mos

• Various coding projects for Apex project
• Managed Paid Inclusion team
• Architect for ysm desktop application
• Technologies
• Java
• Hibernate

Jan 2007 – Aug 2007 · 7 mos

• Designed and implemented rest api for consumer application

Mar 2005 – Jan 2007 · 1 yr 10 mos

• Aided in redesign and reimplementation of back end matching services.

Jan 2000 – Jan 2005 · 5 yrs

• Various programming projects for the search team
• Managed the Direct Traffic Center (DTC) development team
• Completely rewrote DTC as a side project to improve maintainability, scalabillity

Jan 1999 – Mar 2000 · 1 yr 2 mos

• Worked on various software development projects regarding the ZeroPort. This was the desktop toolbar that had to be active on the screen in order to use the NetZero internet services.

Jan 1996 – Jan 1999 · 3 yrs

• Software engineer for command center development group. Projects included:
• SharedNet
• DII COE (common operating environment for the department of defense)
• Top Secret clearance

Jan 1995 – Jan 1996 · 1 yr

• Math department unix system administrator.