S

Shivji kumar Jha

Software Engineer

Bengaluru, Karnataka, India13 yrs 6 mos experience
Most Likely To SwitchHighly Stable

Key Highlights

  • Expert in designing and scaling distributed data platforms.
  • Active contributor to multiple open-source projects.
  • Experienced speaker at industry conferences.
Stackforce AI infers this person is a SaaS and Open Source expert with a strong focus on data platforms and distributed systems.

Contact

Skills

Core Skills

Data Platform ArchitectureDistributed SystemsTeam LeadershipOpen Source EngagementStream Platform DevelopmentSaas ManagementDevops ManagementStream ManagementIntegration DevelopmentData GovernanceBackend DevelopmentPlatform DevelopmentMonitoringDatabase Management

Other Skills

API MonitoringAWSAccount ManagementAlgorithm DesignAlgorithmsApache AtlasApache FlinkApache PulsarBazaarCC++CSSCascading Style Sheets (CSS)Communication Platform DesignCore Java

About

Shiv is a Staff Engineer / Senior Manager at Nutanix and works on all things data platforms. Shiv is responsible for Apache Pulsar, NATS, Druid and Debezium and works on availability, scalability, observability, use cases, architecture, wrapper libraries, maintaining internal source code fork, contributing upstream etc. The data platforms are self hosted in AWS cloud and offered as a service to n nutanix products. Shiv is also a regular speaker at open source conferences having talked on 16 topics including MySQL replication internals, Apache Pulsar, Apache Bookkeeper, Apache Zookeeper and Event Sourcing. Below is a list of Shiv’s blog posts that have been published by different companies based on his work in open source codebases: https://streamnative.io/blog/streaming-war-and-how-apache-pulsar-is-acing-the-battle https://www.datastax.com/blog/why-nutanix-beam-selected-apache-pulsar-over-apache-kafka https://dev.mysql.com/blog-archive/mysql-5-7-6-introducing-multi-source-replication/ https://dev.mysql.com/blog-archive/mysql-5-7-4-change-master-without-stopping-slave-altogether/ https://dev.mysql.com/blog-archive/mysql-5-7-6-it-is-easier-to-switch-master-now/ https://www.shivjijha.in/2013/09/mysql-57-monitoring-replication-with.html Shiv's interests include software architecture, web backend design, micro-service architecture and platform, distributed systems, data/stream stores and open source software in general.

Experience

Nutanix

3 roles

Staff Engineer

Promoted

Aug 2022Present · 3 yrs 7 mos

  • As the founding engineer of the data platform team, I grew it from the ground up into a strong group of 10+, shaping its vision, mentoring engineers, and driving hiring. I specialize in designing and scaling distributed data platforms—providing internal platform-as-a-service offerings for messaging, streaming, analytics, and time-series workloads used across “n” Nutanix products.
  • 1. Architecture & Enablement → Partnering with product teams and staff engineers to define data platform best practices, ensuring the right architecture choices for distributed systems.
  • 2. Systems at Scale → Designing, debugging, and benchmarking critical workloads (e.g., running ClickHouse on low-memory Kubernetes, integrating Apache Iceberg with ClickHouse, and optimizing analytics workloads in PostgreSQL).
  • 3. Open Source Engagement → Maintaining internal forks of Apache Pulsar, NATS, Apache Druid, and ClickHouse, contributing fixes and features upstream, and helping bridge cutting-edge open-source stacks into Nutanix products.
  • 4. Community & Evangelism → Active in the open-source and deep-tech ecosystem: I’ve delivered 30+ talks at Apache, CNCF, and Postgres conferences; organized 10+ meetups at Nutanix bringing together 1000+ engineers in Bangalore; and regularly write/mentor to grow the ecosystem.
  • 5. Partnerships → Led strategic collaborations with ecosystem partners to bring enterprise-grade support for critical data platforms into Nutanix.
  • 6. Leadership & Mentorship → Building, mentoring, and scaling a high-performing team while driving cross-functional initiatives across engineering, product, and open-source communities.
Apache PulsarNATSDruidDebeziumAWSDistributed Systems+2

Senior Member Of Technical Staff (SDE 5)

Promoted

Aug 2020Aug 2022 · 2 yrs

  • 1. Leading stream platform team backing multiple SAAS products at nutanix. The platform is built atop apache pulsar, apache flink and presto query engine.
  • 2. Lead an existing SaaS platform apps team (including user management, account management, billing and subscription, authorization service, external integrations with customer communication and ticketing platforms etc). Tasks involved stabilizing, reducing cutomer issues, steading ship and turning it to align with the new product roadmap.
  • 3. Lead the first milestone release of clutch platform based on the open source lyft/clutch for a single pane devops experience for microservice life cycle management, configuration as a service, secret store etc).
  • 4. Apache Pulsar committer, SME and Developer Advocate at Nutanix.
  • 5. Evangelist for Apache Pulsar - Multiple guest blogs / did talks (https://www.youtube.com/watch?v=Bx4csRi1b8Y&list=PLA7KYGkuAD071myyg4X5ShsDHsOaIpHOq) to give back to Apache Pulsar community.
  • 6. Hiring, tech lead and mentor for two tech teams.
Apache PulsarApache FlinkPrestoSaaSUser ManagementAccount Management+2

Member Of Technical Staff 4

Jan 2019Aug 2020 · 1 yr 7 mos

  • 1. Responsible for all things stream at Nutanix Beam. Includes maintaining a fork of Apache pulsar, contributing everything upstream to community, manage infra (4 clusters of 30 nodes), usage architecture, libraries, wrappers etc.
  • 2. Tech lead for all external integrations (email, rest webhook, splunk, serviceNow etc) to connect to customers and intimate them of beam (cost governance, security compliance) findings on their preferred choice of notification / ticketing platform.
  • 3. Designing audit trail platform via event sourcing pattern.
Apache PulsarEvent SourcingIntegrationInfrastructure ManagementStream ManagementIntegration Development

Swiggy

2 roles

Data Platform Developer 2

Promoted

Jul 2018Jan 2019 · 6 mos

  • 1. Implemented Schema registry platform on top of Apache Atlas project. Provides data governance via visibility with data pipelines as first class citizen, fulltext search on all kafka schema and data / change ownership.
  • 2. Designed CDC stream to provide changes on transactional DBs as kafka messages. Explored denezium and uber hudi.
Apache AtlasData GovernanceKafka

Software Developer 2

Aug 2017Jun 2018 · 10 mos

  • Lead backend development (mentoring 2 engineers) for swiggy app’s help feature. Implemented in java a helpcenter chat bot based on decision tree.
JavaBackend Development

Moveinsync

3 roles

Senior Software Developer

Promoted

Jul 2016Jul 2017 · 1 yr

  • Designed and implemented a communication platform from scratch (sms + mobile notifications + email, extensible) including service provider failover, channel level failover and configurable rules.
Communication Platform DesignPlatform Development

Software Developer

Promoted

Nov 2015Jul 2016 · 8 mos

Software Application Engineer

May 2015Nov 2015 · 6 mos

  • 1. Prototype an infrastructure to Instrument, Monitor and alert on API calls. Tools used: Dropwizard java library, Graphite DB, Grafaba for dashboards.Syren for alerts
  • 2. Prototype ELK based log archival and cleanup framework to support micro-service based distributed architecture. Impact: no server crash because of disk space on server instances, no manual errors.
  • 3. Research, test / verify and fill the scripting gaps to move all natively run postgres instance usage to amazon RDS instances.

Oracle mysql

2 roles

Software Developer 2

Aug 2013Apr 2015 · 1 yr 8 mos

  • Developed following features for Oracle's MySQL replication team: (details on blog : http://shiv4289.blogspot.in/search/label/work)
  • 1. performance_schema tables to show replication health and performance.
  • 2. Relaxed the condition to stop slave before change master, stop only what needs to be configured.
  • 3. Removed the manual wait function before switching masters making and failover in MySQL easier.
  • 4. Re-factored the code for changing masters making it easier to extend.
  • 5. Removed shortcomings of lag monitoring to make delay monitoring in replication easier.
  • Also, proposed a generic transaction header format for MySQL binary logging infrastructure, contributed to multi-source replication in MySQL with code and code reviews and lead the MySQL Replication engineering team for 5 months leading to two development milestone releases.
API MonitoringLog ArchivalMonitoring

Software Developer 1

Jul 2012Aug 2013 · 1 yr 1 mo

MySQLReplicationDatabase Management

Indian institute of technology, bombay

Software Development Intern

May 2010Jul 2010 · 2 mos · Bombay

  • We were a group of 5 people .We desgned a software that provides the facility to create slides and save them at the desired location on the hard disk. The software can open files saved with ppt,pdf,jpg,jpeg,txt and all other normal extensions and these can be viewed as slide show. The slides can also be reorganized and saved as another project on the hard disk.
  • Guide: Mr. Deepak B. Phatak, Dept of CSE, IIT Bombay.
  • Technologies Used: Javafx,Java,CSS

Education

Birla Institute of Technology, Mesra

Bachelor of Engineering (BE) — Information Technology

Jan 2008Jan 2012

Kendriya Vidyalaya

Jan 1999Jan 2007

Stackforce found 100+ more professionals with Data Platform Architecture & Distributed Systems

Explore similar profiles based on matching skills and experience