Navinder Pal Singh Brar

Engineering Manager

Bengaluru, Karnataka, India10 yrs 11 mos experience
AI ML PractitionerAI Enabled

Key Highlights

  • Led the development of a high-scale Data Platform at Walmart.
  • Contributed to Apache Kafka and received Confluent Kafka Catalyst award.
  • Public speaker at multiple international conferences.
Stackforce AI infers this person is a Data Engineering and Infrastructure expert with significant contributions to high-scale systems.

Contact

LinkedIn

Skills

Core Skills

Infrastructure ManagementData ProcessingData EngineeringOpen Source Contribution

Other Skills

AI Control SystemsAJAXAirflowAlgorithmsApache AmbryApache HudiApache KafkaBig DataBlob StorageCC++CSSCore JavaData MiningData Platform

About

Navinder is an Engineering Leader with expertise in building Data and AI platforms at scale. He has built teams and platforms from scratch in Fortune 500 companies and high-growth scaleups. Recently at Phaidra, Navinder joined as one of the early engineers in the seed stage and quickly transitioned to lead the Data Platform vertical. Before this, he had been one of the founding members of Walmart's Customer Data Platform, to store and serve 1P, 3P customer data from various sources, and maintain a single view of the customer across all these sources. The platform processes around 40 billion events a day and serves the processed knowledge at <10 ms(95%) latencies. This knowledge is used by various teams across Walmart to build Machine Learning Models such as Fraud detection, Recommendation Engine, or served directly to the Walmart website. Working on the system, Navinder and his team received 4 patents in the US. - System and Method for Bootstrapping Replicas from Active Partitions (US 16/256,856) - Methods and Apparatus for Data Repartitioning (US 16/548,628) - Event-Driven Identity Graph Conflation(US 16/777,692) - Asynchronous Multi-Tenant Model Inferences on Streaming Databases(US - 17/508857) Navinder is an active contributor to Apache Kafka and Hudi, and has organized Kafka/Streaming meetups in Bangalore. He has proposed and implemented various important KIPs(KIP-535, KIP-562) and, reported and resolved various Kafka bugs. His active engagement with the Apache Kafka community has earned him an award as a Confluent Kafka Catalyst. He is an experienced public speaker with experience of speaking in many renowned International Conferences. - O'Reilly Strata Data Conference, New York(Sept-2019) - Big Data & AI World, London(Mar-2020) - O'Reilly Strata Data & AI Conference, San Jose(Mar-2020) - International Developer Summit, Bangalore(Apr-2020) - Kafka Summit, London(Apr-2020) - Data Summit, Boston(June-2020) - Kafka Summit, Austin(Aug-2020)

Experience

Linkedin

2 roles

Software Engineering Manager

Jul 2025 – Present · 8 mos · Bengaluru, Karnataka, India

  • As an Engineering Manager in the Core Infrastructure organization at LinkedIn, I lead multiple Tier-0 infrastructure teams responsible for foundational systems in the Metadata and Streams (PubSub) space. My teams build and manage critical components like ZooKeeper and Kafka, while also driving the next generation of infrastructure by building their replacements.
  • Given Kafka's scaling limitations at LinkedIn's scale, we are transitioning to a new platform called NorthGuard: https://www.linkedin.com/blog/engineering/infrastructure/introducing-northguard-and-xinfra, and I lead multiple teams contributing to this initiative. Detailed teams:
  • Zookeeper: Overseeing efforts to modernize and optimize metadata management systems at scale. This serves as a Metadata store for Apache Kafka, Pinot, Espresso, D2 etc within LinkedIn.
  • XInfra Clients: Leading the development of a unified pub/sub client that abstracts underlying systems like Kafka and NorthGuard emitting >32 Trillion events/day, offering guarantees like ordering for different use cases. This involves ownership of multi-language Thin Clients(Rust, Python, Java, Go) for easier use.
  • SDC (Shared Dictionary Compression): Driving the design of a compression system for real-time data pipelines, enabling efficient compression of small records and addressing challenges like batching restrictions and encryption.
KafkaZooKeeperPubSubMetadata ManagementReal-time Data ProcessingInfrastructure Management+1

Software Engineering Manager

Aug 2024 – Jul 2025 · 11 mos · Bengaluru, Karnataka, India

  • Leading the Ambry team in Core Infra org, to scale the blob store at LinkedIn.
  • Apache Ambry(https://github.com/linkedin/ambry) stores more than 1000 PBs of photos, videos, resumes and DB checkpoint states within LI. This is a critical Tier0 service, used by various Application and Infra teams.
Blob StorageApache AmbryInfrastructure Management

Phaidra

Head of Data Platform

Jun 2022 – Jul 2024 · 2 yrs 1 mo · Remote

  • I lead the Data Platform vertical at Phaidra(https://www.phaidra.ai). Phaidra creates AI-powered control systems for the industrial sector(data centers, manufacturing plants, etc) enabling industrial facilities to automatically learn and improve over time.
  • As core responsibilities, I lead the technical direction and architecture of the platform along with supporting the team in growth opportunities.
Data PlatformAI Control SystemsData Engineering

Walmart global tech

3 roles

Staff Engineer

Promoted

May 2021 – Jul 2022 · 1 yr 2 mos

  • Working as a Technical Architect with the Customer Backbone team, which tries to bridge the gap between the Walmart stores and online customers(be it from walmart.com or jet.com or any other acquisitions) and provide an omnichannel experience to all the users. We have built a data platform capable of processing >50M events/minute, and serving LEAF(Linkages, Events, Attributes, Facts) data to users and models at sub-ms latencies.
Data PlatformEvent ProcessingData Engineering

Senior Software Engineer

Aug 2019 – May 2021 · 1 yr 9 mos

Engineer III - Development

Sep 2017 – Aug 2019 · 1 yr 11 mos

O'reilly strata data conference

Speaker

Sep 2019 – Sep 2019 · 0 mo · Manhattan, New York

  • Talk titled: Building a multitenant data processing and model inferencing platform with Kafka Streams

The apache software foundation

Contributor

Sep 2018 – Aug 2022 · 3 yrs 11 mos · Bengaluru, Karnataka, India

  • Contributor to Apache Kafka and Hudi
Apache KafkaApache HudiOpen Source Contribution

Keastone

Product Developer

Apr 2017 – Sep 2017 · 5 mos · Bengaluru, Karnataka, India

  • Worked as a Backend Developer to design and develop product in an early stage start-up, focused to solve problem of information fragmentation for individual users. Also, share responsibility for decision making on the technical aspects and features of the product.

Amadeus labs

2 roles

Software Developer

Jul 2015 – Mar 2017 · 1 yr 8 mos

  • Worked as a Full Stack Developer in a cross functional team to design and develop proactive and reactive monitoring framework for Lufthansa, Swiss, Austrian and Brussels airline’s applications. I work with various teams and clients, understanding their requirements for new applications, working on a plan and developing end to end applications. An important part of my job is to automate processes and provide solutions that can be used by various teams and division, to minimize manual tasks and increase productivity of the whole team.

Software Developer Intern

Jan 2015 – Jun 2015 · 5 mos

Education

Thapar Institute of Engineering & Technology

Bachelor of Engineering (BE) — Computer Science

Stackforce found 100+ more professionals with Infrastructure Management & Data Processing

Explore similar profiles based on matching skills and experience

Arul Murugan

Arul Murugan

Research Fellow

at University of California, Berkeley

Berkeley, United States5 yrs 7 mos exp
AnalyticsBackend DevelopmentMachine LearningProduct Development
Anoop Jawahar

Anoop Jawahar

CTO

at Sundial

Bengaluru, India15 yrs 8 mos exp
Big Data AnalyticsDistributed StorageMulti-agent Systems
Om Shakya

Om Shakya

Databricks/SQL Engineer

at PyPro Tech

Delhi, India15 yrs 6 mos exp
C#DatabricksETLPythonSQL
NR

Nihal Raza

Unisys

at 2 yrs 10 mos

Gurugram, India0 mo exp
Machine Learning
PS

Pete Soderling

Founder & General Partner

at Zero Prime Ventures

San Francisco, United States30 yrs exp
Application DevelopmentClient EngagementCommunity BuildingCommunity EngagementContent Development
Amit Kumar

Amit Kumar

Director of Engineering

at Narvar

Bengaluru, India23 yrs 7 mos exp
Data ArchitectureInfrastructure OptimizationInterpersonal SkillsLow-Level DesignPlatform Architecture
HM

Harish M

Data Engineer

at Accion Labs

Hyderabad, India5 yrs 5 mos exp
Rahul Kumar

Rahul Kumar

Senior Software Engineering Manager

at Krutrim

Bengaluru, India6 yrs 6 mos exp
RoutingAI-driven innovationSystem architectureAI frameworksCost optimization