Aug 2024 – Present · 1 yr 7 mos

• 1. Working for metrics platform team responsible for the overall ingestion and query part.
• 2. Implemented a shadow mode for the new architecture (data stored in S3) to ensure the correctness and overall performance with our old tech stack ie Cassandra and Elasticsearch by running the queries in background triggered by the customer.
• 3. Did robust unit and integration testing as well.

Mar 2020 – Jul 2024 · 4 yrs 4 mos

• 1. Working for search query team responsible for query part of the log search platform ingesting 500 Tb of data daily.
• 2. Build generalized indexes using Arrow files to improve search performance on indexes stored in parquet format by 30%.
• 3. Managed the migration of the query pipeline from EC2 to Kubernetes for Engine-2 ie Lucene to Parquet Format of the log search platform. This transition transformed the architecture from cached to completely cacheless, facilitating seamless integration with CI/CD pipelines and significantly reducing maintenance overhead, improved the search performance by 65%, failed queries lowered by 50% and cost reduction by $3M annually while maintaining scalability of the system without any downtime.
• 4. Onboarded 3 people in the Logs query team.
• 5. Enhanced customer query execution isolation within the Sparrow scheduling algorithm, resulting in a 15-25% reduction in queue time and also the impact of one customer on another due to high query load decreased by 2x.
• 6. Revamped customer query routing rules, transitioning from JSON format to key-value structure. This shift in infrastructure, achieved by a simple property adjustment, significantly streamlined onboarding and reduced query routing time from approximately 45 minutes to just 10 minutes. Also helped in the Engine-2 platform.
• 7. Improved the overall query team reliability/KTLO by reducing the operational overhead/on-call alerts from 30/day to less than 5/day.
• 8. Played a key role in testing the Sumo Logic data collection integration on the Kubernetes platform across diverse cloud providers including AWS, GCP, Azure. This effort, which included creating a Python pytest framework, lead to a impressive 80% reduction in verification time for new cloud provider versions.

Aug 2018 – Mar 2020 · 1 yr 7 mos

• 1. Established the QueryLoadGenerator tool, which enables running multiple search queries concurrently with varying load assisting developers in shipping their features 35% faster.
• 2. Automating tricky workflows such as Multi-factor Authentication using AWS Lambda, DynamoDB, and API Gateway.
• 3. Helped in setting up/mentoring and onboarding a whole new Parser developement team (Team of 4 people)

Feb 2018 – Jul 2018 · 5 mos

• Demonstrated 70% improvement in release time by automating workflows using APIs.