Feb 2022 – Present · 4 yrs 3 mos · London

• Open sourcing the rapid application development patterns and tools developed at Hidden Road.
• Developing an open-source framework for quickly build & testing an ecosystem of Kafka Streams based micro-services.
• Responsible for full product roadmap, features, development, testing, documentation, website, networking, presenting… everything!
• Keeping the mind sharp while taking some time out with the family and going mountain biking.

Dec 2020 – Jan 2022 · 1 yr 1 mo · London, England, United Kingdom · Remote

• Startup building a new fintech solution, utilising a cloud based, Kafka Streams, micro-service architecture.
• Initial work focused on improving dev process effectiveness and efficiency. The lessons learned, and solutions found, are being open-sourced.

Feb 2018 – Nov 2020 · 2 yrs 9 mos · London, United Kingdom · Remote

• Sequoia's fastest-growing startup (IPO June 2021). Main company behind the hugely successful open source Apache Kafka.
• I had the pleasure of working as part of a team of experienced, talented and super smart engineers.
• Working mainly on ksqlDB and Kafka Streams, with some work in core Kafka and other areas.
• We took ksql from a prototype through to a on-prem production ready product, then launched as a service available on Confluent cloud.
• I played a pivotal role, along with another great engineer by the name of Matthias Sax, to define extensions to the SQL standard to allow the language to express streaming queries, i.e. queries of data-in-motion, not the standard data-in-rest of traditional databases.
• I was the principle engineer working on the core of ksqlDB, involved in many projects as well as I was also being involved in many design discussions in the wider organisation. I played a key role in choosing the direction of the product, working closely with the product team.
• The client facing features I was most proud of were:
• 1. Adding "pull" query support. This enabled users to use standard SQL queries to access real-time updating tables, distributed across multiple nodes.
• 2. Supporting multiple key columns. The prototype only supported a single STRING key column. Adding support for multiple columns, of varying types, involved a lot of careful refactoring over many releases.
• To enable this work, on a product that was frequently released, I added extensive test cases, written in YAML and SQL, to cover most areas of the SQL language ksqlDB supported, and built a test harness capable of feeding in test cases through Kafka and into ksqlDB, validating expected output.

Jul 2015 – Jan 2018 · 2 yrs 6 mos · London, United Kingdom · On-site

• Leading a small, highly skilled team of engineers in the iTunes big-data engineering team.
• Responsible for designing large Kafka clusters to handle Apple-scale loads. Building data ingestion pipelines, cluster monitoring tooling & UIs, client libraries to enable message tracing and lineage tracking. Training SAs supporting Kafka. Evangelising and engaging with other teams. Key consultant for the team building an Apple-wide Kafka-as-a-Service offering.

May 2013 – Jul 2015 · 2 yrs 2 mos · London, United Kingdom · On-site

• Part of a team building an real time, in memory, trade store, using Oracle Coherence.

Sep 2009 – May 2013 · 3 yrs 8 mos · London, United Kingdom

• Brought in at the start of a new architecture function to replace the ailing 3rd-party software platform with a scalable & flexible platform, built on modern technology. The existing system supports multiple geographic regions, providing sports betting and games.
• Design of Oracle Coherence data grids, underlying infrastructure and monitoring, across production & NPEs. Inc. technical leadership for a newly created DevOps team to define processes, proactive monitoring of the grid & its applications, and capacity management.
• Design of data distribution/publishing system responsible for getting sports book data to front end presentation tier, direct to customers’ browsers and B2B partners. Involving an interim solution where existing C++ applications were integrated with Oracle Coherence over Extend to overcome the immediate bottleneck of SAN IOPS, and a strategic solution that utilised Java on-grid processing & merging of data, Push Technology’s Diffusion product to provide a pub/sub interface to the volatile data, & CDN for static content, This supports a wide variety of protocols and devices. Improving customer experience, cutting OpEx and providing a foundation for future market-leading functionality.
• Championed the importance of monitoring and introduced LiquidLab’s Logscape for software platform and distributed business process monitoring, and embedded the product into the development process.
• Design of new in-house trading system, including an algo-trading platform that enabled the trading analysts to quickly take new models and products to market, utilising c# diffusion client, Java Coherence applications, Esper CEP, Java Diffusion logic & event driven beans running in JBoss AS 7 and HornetQ.
• Design of Coherence grid based system for caching translations to provide websites localisation.
• Migration of IIS session caching from ScaleOut to Oracle Coherence*Web for a farm over 100 web servers.

May 2008 – Aug 2009 · 1 yr 3 mos · London, United Kingdom

• Hired to stabilize and enhance the Betfair poker gaming platform, composes of a C++ ActiveX client, C++ games server and auxiliary Java processes.
• Improved development process through the introduction of continuous integration: automated builds, deployments and testing
• Lead for a project to build a test harness for the poker API, using Agile methodologies
• Work closely with product owners to priorities bugs and feature requests to maximize business benefit of work and ensure it met requirements
• Used a logical and questioning approach to lead a team through the investigation of un-reproducible production issues
• Worked with IS Service Improvements team to formulate and implement a strategy to improve release quality and component stability.

May 2006 – May 2008 · 2 yrs · Sydney, Australia

• Ran a team of eight developers responsible for building the tool set used to build the PS3 game ‘LA Noire’.
• Worked within the SMT to identify causes of high staff turnover and implement changes to reduce it by over 50%.
• Instigated cultural changes and put in place processes to improve the quality of code being produced.
• Staff recruitment and mentoring.
• Design and implementation of a new ‘property system’ and data driven editor, which decoupled the game from the toolset, simplified the system from users and reduced resource usage.
• Redesigned & refactored a bespoke distributed dependency-driven art-assert build system used to compile the game, bringing a full-map build down from 8 days to less than 3

Nov 2000 – Feb 2006 · 5 yrs 3 mos · London, United Kingdom

• As part of the application architecture team the work spanned all areas of the departments application portfolio, which included portfolio management, risk calculations and real-time distribution of financial data, all written in C++.
• Recognized specialist in many areas including multi-threading, sockets, lock-free, templates, etc.
• Architectural strategy and new technologies.
• Leading small teams of developers, mentoring and recruitment.
• Championed the introduction of new code review policies, use of standard design patterns and department coding standards.
• Rebuilt client confidence in the real-time rates distribution product through enormous improvements to performance, maintainability and stability of the product. One client reporting 10 thousand ticks per second and nothing reporting the service running for 3 months without incident.

Nov 1998 – Nov 2000 · 2 yrs · Telford, United Kingdom

• Graduate training scheme working for EDS at the Inland Revenue in Telford, within a team responsible for a suite of applications used to move documents around the company in a guaranteed, audited and controlled manner.
• Designed and development of a replacement MS Windows version of the teams tool, capable of communicated with other agents on other servers.
• Design and development of a file transfer tool capable of working over a unidirectional fiber network link, used to transfer electronic tax returns into the secure IR network.