Deepak Goyal

Senior Software Engineer

Bengaluru, Karnataka, India9 yrs 11 mos experience
Highly Stable

Key Highlights

  • Innovated serverless orchestration for Kafka clusters.
  • Architected a real-time Customer Data Platform for 600 million users.
  • Developed algorithms enhancing data processing efficiency.
Stackforce AI infers this person is a SaaS expert specializing in real-time data processing and distributed systems.

Contact

Skills

Core Skills

KafkaConfluent CloudReal-time Data ProcessingDistributed SystemsDistributed ProcessingKafka StreamsKnowledge GraphPerformance Optimization

Other Skills

Serverless OrchestrationCustomer Data PlatformIdentity Graph ConflationDatabase PlatformData Publishing PipelineMonitoring ToolsDebuggingReportingAlgorithmsProgrammingC++Android DevelopmentData StructuresSoftware DevelopmentSoftware Engineering

About

At Confluent, my focus is on innovating serverless orchestration for self-balancing and self-scaling kafka clusters, leveraging my proficiency in data engineering and distributed computing to craft robust solutions. My recent tenure at Walmart Labs involved architecting a real-time Customer Data Platform, contributing to the seamless integration of information for over 600 million customers. The platform's success hinged on high availability, low-latency operations, and unwavering data durability. Leading with a forward-thinking approach, I played a pivotal role in developing the Identity Graph Conflation algorithm, enhancing the platform's efficiency by eradicating tail latencies. This foundational work, coupled with my algorithmic acumen and command of Confluent Cloud, positions me at the forefront of addressing complex challenges in real-time data processing and orchestration. Patents - System and Method for Bootstrapping Replicas from Active Partitions (US 16/256,856) - Methods and Apparatus for Data Repartitioning (US 16/548,628) - Event-Driven Identity Graph Conflation (US 16/777,692) Speaker at the following conferences: - Kafka Summit, London 2019 - Kafka Summit, New York 2019 - Rootconf, Bengaluru 2019

Experience

9 yrs 11 mos
Total Experience
3 yrs 11 mos
Average Tenure
2 yrs 1 mo
Current Experience

Confluent

Senior Software Engineer II

Apr 2024Present · 2 yrs 1 mo

  • Kafka - Kora Serverless Orchestration
Confluent CloudKafkaServerless Orchestration

Walmart labs

2 roles

Senior Software Engineer

Promoted

Aug 2019May 2023 · 3 yrs 9 mos

  • Worked on a novel Customer Data Platform for Walmart Labs to build, model and serve the 360° view of both online and store customers in real-time.
  • A real-time event-driven distributed QTS (Queue-Task-State) catering to the needs of 600M customers of Walmart , it’s subsidiaries ( Sams , Asda , etc.) and internal Walmart client teams.
  • Capable of processing multi-million customer events ensuring low read-write-api latencies, high availability and data durability.
  • Developed Identity Graph Conflation, an asynchronous distributed algorithm to co-locate all customer data thereby removing any tail latencies in QTS.
  • As one of the founding members of the team, leading the key architectural fundamentals.
Real-time Data ProcessingCustomer Data PlatformIdentity Graph ConflationDistributed Systems

Software Engineer 2

Oct 2017Aug 2019 · 1 yr 10 mos

  • Working on a realtime Distributed Processing and Database platform based on Kafka streams with key focus being on data colocation with processing.
Kafka StreamsDistributed ProcessingDatabase Platform

Microsoft india

2 roles

Software Engineer

Jun 2015Sep 2017 · 2 yrs 3 mos · India

  • Worked on Bing’s Knowledge Graph (Satori- Data publishing pipeline)
  • Designed a robust feedback mechanism enabling Knowledge Graph to keep good quality data, which improved runtime performance by 2% and reduced storage by 5% on the scale of TBs.
  • Sole owner of the pipeline which feeds the data to “People also search for” carousel and in the entity pane for 9 major markets.
  • Optimized runtime performance by 80% of a generic ranking algorithm (an adaptation of bag-of-words model).
  • Mentor at Microsoft AI Hackathon 2017 at Anna University, Chennai.
Knowledge GraphData Publishing PipelinePerformance Optimization

Software Engineering Intern

May 2014Jun 2014 · 1 mo · Hyderabad

  • Worked with Bing at Microsoft IDC to build a monitoring, debugging and reporting tool which helps enhancing the Entity Building process of Bing.
Monitoring ToolsDebuggingReporting

Education

International Institute of Information Technology Hyderabad (IIITH)

Bachelor's Degree — Computer Science Engineering

Jan 2011Jan 2015

Government Model Sr Sec School, Sector-33, Chandigarh

High School — CBSE

Jan 2009Jan 2011

St. Xaviers Chandigarh

Secondary School — ICSE

Jan 2004Jan 2009

Stackforce found 100+ more professionals with Kafka & Confluent Cloud

Explore similar profiles based on matching skills and experience