Vigya Sharma

Product Manager

San Francisco, California, United States12 yrs 8 mos experience

AI EnabledAI ML Practitioner

Key Highlights

Expert in distributed systems and machine learning.
Led the development of Amazon Elasticsearch Service.
Active contributor to Apache Lucene project.

Stackforce AI infers this person is a SaaS expert with a focus on distributed systems and machine learning.

Contact

vigyasharma@outlook.com LinkedIn

Skills

Core Skills

Distributed SystemsMachine LearningInformation RetrievalFault ToleranceCluster ManagementJava ConcurrencyBig Data

Other Skills

AlgorithmsAmazon Web Services (AWS)CC++CUDAData AnalysisData StructuresElasticsearchHBaseJavaLinuxMPIMicrosoft SQL ServerNumPyOperating Systems

About

I'm a Principal Engineer at Amazon Search, working at the intersection of distributed systems, information retrieval, and AI-powered search. My focus is on making search more scalable, intelligent, and responsive to how people naturally express what they're looking for.I lead initiatives that apply modern ML techniques to search quality, expand product discovery for customers early in their shopping journey, and optimize our core search infrastructure for scale and efficiency. My work spans semantic matching and ranking systems, multi-modal content understanding, and distributed systems architecture — all focused on making search more intelligent and responsive to natural language queries.I am also a Committer and PMC member on the Apache Lucene project. You can find a living summary of my work at https://vigyasharma.github.io/about/

Experience

12 yrs 8 mos

Total Experience

2 yrs 8 mos

Average Tenure

3 yrs 9 mos

Current Experience

Amazon

4 roles

Principal Engineer

Promoted

Sep 2024 – Present · 1 yr 7 mos

Search at Amazon

Information RetrievalDistributed SystemsMachine Learning

Sr. SDE at Amazon Search

Apr 2021 – Oct 2024 · 3 yrs 6 mos

Product Search at Amazon.

Information RetrievalDistributed SystemsMachine Learning

Senior Software Engineer

Jul 2018 – Apr 2021 · 2 yrs 9 mos

Working on resiliency, fault tolerance, cluster management, and distributed systems for Amazon Elasticsearch Service.
Contributions (as technical lead) -
"Autotune for Amazon ES" - a self adapting feedback loop mechanism for intelligently optimizing Elasticsearch clusters.
Hyper-scale shard allocation and fault tolerance in Ultrawarm enabled Amazon ES clusters.
A strongly consistent framework for in place configuration updates in a distributed system.
Self Healing framework to auto-heal clusters.
Other projects I've helped build:
Split brain avoidance mechanisms
Internal monitoring systems
Internal architecture of the service across control and data plane.
Different parts of a domain's lifecycle supporting scaling updates and configurational changes.
I'm routinely involved in operational deep dives and mentoring other engineers.

ResiliencyFault ToleranceCluster ManagementDistributed Systems

Founding Engineer, Amazon Elasticsearch Service

Oct 2014 – Apr 2021 · 6 yrs 6 mos

Part of the core team of engineers that created and launched Amazon Elasticsearch Service.
Technical Lead for:
Cluster management and shard allocation in Amazon Elasticsearch
Hyperscale shard allocation in Ultrawarm
AutoTune for Amazon Elasticsearch

Cluster ManagementShard AllocationDistributed Systems

The apache software foundation

Lucene Committer

Jul 2022 – Present · 3 yrs 9 mos · Palo Alto, California, United States

Committer on the Apache Lucene open source search engine. My current focus has been indexing and merging in Lucene, and I'm actively exploring other areas. I help with reviewing and committing changes, making Lucene enhancements, and maintaining the code base in general.
I implemented a concurrent, non-blocking, transactional version of the addIndexes() API, that leverages concurrent background merges to enable users to combine indexes with a low add-to-search latency. The change is foundational in unlocking the ability to decouple indexing and search shards in applications that use segment replication.
I'll speak more about this change and decoupling indexing from search at my talk in the upcoming ApacheCon NA, '22, on October 6th at New Orleans, Louisiana - https://www.apachecon.com/acna2022/schedule.html

Java ConcurrencyInformation Retrieval

Open distro for elasticsearch

Contributor

Jun 2020 – Apr 2021 · 10 mos · Palo Alto, California, United States

Contributor and Committer for the Performance Analyzer RCA framework - the engine that powers AutoTune for Amazon Elasticsearch.
https://github.com/opendistro-for-elasticsearch/performance-analyzer-rca

Performance AnalysisElasticsearch

Elasticsearch

Contributor

May 2019 – Dec 2020 · 1 yr 7 mos · Palo Alto, California, United States

Contributed fixes and improvements to open source Elasticsearch.
https://github.com/elastic/elasticsearch/pull/42066
https://github.com/elastic/elasticsearch/pull/42658

Adobe systems india

Member of Technical Staff

Aug 2013 – Oct 2014 · 1 yr 2 mos · Noida, Uttar Pradesh, India

Predictive Analytics on user data
Machine learning problems to predict customer churn.
Feature mining to extract high impact parameters.
Deep Learning in Neural Networks.
Big data processing and warehousing in Hadoop and Hive.
Designed efficient ETL processes on Hive for data warehousing.
Optimized Hive tables and data warehouse structure for fast data retrieval.
Worked on building an analytics data warehouse ground up in SAP HANA. Initial modules live in production.

Predictive AnalyticsMachine LearningBig Data