Shivani .

Operations Associate

Gurugram, Haryana, India11 yrs 10 mos experience

Most Likely To SwitchHighly Stable

Key Highlights

Led the most AI-enabled testing team in the organization.
Achieved 90% call success rate in voice translation.
Reduced post-release defects by 35% through quality metrics.

Stackforce AI infers this person is a QA Manager with expertise in AI-enabled testing and healthcare applications.

Contact

Skills

Core Skills

Ai-enabled TestingTest Automation Framework DevelopmentRag TestingQa LeadershipQuality Assurance TestingTest Management

Other Skills

PythonPlaywrightSIPpClaude AIGitHub ActionsJirapytestASR/TTS quality validationSIP protocol testingBLEU score analysisAutomationLog analysisSeleniumClaude DesktopTestRail

About

Results-driven QA Manager with 11+ years of expertise in software testing, specializing in Gen AI/LLM validation, real-time voice translation systems, and end-to-end test automation. I lead an 8-member QA team that has become the most AI-enabled testing team in our organization. My focus areas include: → Gen AI/LLM Testing: RAG validation, hallucination detection, prompt engineering, bias & toxicity testing, model grounding → Voice Translation Systems: ASR/TTS quality validation, SIP protocol testing, BLEU score analysis across multiple language pairs → Test Automation: Building custom frameworks with Playwright, Python, and Selenium that have reduced testing cycles by 30-40% → AI-Powered Workflows: Pioneering MCP server integrations with Claude Desktop for automated test generation and gap analysis Key achievements: - Achieved 90% call success rate and 8-second latency reduction for enterprise voice translation platform - Scaled RAG-based Knowledge Assistant across 5+ client implementations with zero critical post-launch issues - Reduced post-release defects by 35% through systematic quality metrics and root cause analysis I'm passionate about leveraging AI tools to transform QA processes — not just testing AI, but using AI to test smarter. Currently exploring opportunities where I can lead quality initiatives and drive innovation in AI-enabled testing environments.

Experience

11 yrs 10 mos

Total Experience

5 yrs 11 mos

Average Tenure

6 yrs 3 mos

Current Experience

Ttec

3 roles

Manager

Promoted

Oct 2025 – Present · 7 mos · Remote

Leading an 8-member QA team, transforming it into the most AI-enabled testing team in the organization through strategic adoption of Gen AI tools and automated workflows.
Key Responsibilities & Achievements:
Spearhead testing for Real-Time Voice Translation Platform (SIP-based): validated ASR/TTS quality and translation accuracy across 5 language pairs, achieving 90% call success rate and 8-second latency reduction
Built custom Python framework for voice testing: automated mulaw WAV generation, end-to-end speech-to-translation pipeline with BLEU score analysis, and Gen AI-powered log analysis generating HTML reports
Developed custom QA dashboard integrating GitHub & Jira APIs for real-time PR tracking, story management, and consolidated status reporting
Created automated log analysis system processing thousands of raw log lines into executive-ready HTML reports highlighting process improvements and model comparisons for ASR, Translation, and TTS engines
Execute SIPp load testing scripts for 10-concurrent, 8-hour call windows; conduct prompt injection, bias/toxicity testing, and model version A/B testing
Technologies: Python, Playwright, SIPp, Claude AI, MCP Servers, GitHub Actions, Jira, pytest

AI-enabled TestingTest ManagementPythonPlaywrightSIPpClaude AI+4

Principal QA Engineer

Aug 2023 – Oct 2025 · 2 yrs 2 mos · Remote

Led QA strategy for enterprise AI applications including RAG-based knowledge systems and conversational AI platforms.
Key Responsibilities & Achievements:
Spearheaded QA for enterprise RAG-based Knowledge Assistant: validated retrieval accuracy, hallucination detection through negative testing, context window handling, embedding quality, and source citation accuracy across multi-tenant system with 100+ test scenarios per tenant
Pioneered MCP server POCs with Claude Desktop for end-to-end test generation, API collection creation from Jira stories, and automated gap analysis between UI and acceptance criteria
Led testing for Internal Chatbot system: validated response accuracy for agent/executive queries, ensured HR system integration, and improved employee self-service capabilities
Designed comprehensive test strategies for NLP models and conversational AI systems, improving response accuracy by 15% and reducing customer escalations
Created standardized prompt templates and QA process documentation for team-wide adoption
Technologies: Python, Selenium, Playwright, Claude Desktop, MCP Servers, Jira, TestRail, REST APIs

RAG testingQA leadershipPythonSeleniumPlaywrightClaude Desktop+3

Senior Quality Assurance Analyst

Feb 2020 – Aug 2023 · 3 yrs 6 mos · Remote

Delivered comprehensive testing for enterprise healthcare applications, establishing automation frameworks and quality standards.
Key Responsibilities & Achievements:
Led end-to-end testing for Kaiser Permanente (Healthcare domain), establishing quality metrics that reduced post-release defects by 35%
Designed and implemented Selenium + Java automation framework, reducing manual testing efforts by 40% and increasing test coverage by 25%
Executed functional, regression, and performance testing for mission-critical healthcare software, maintaining 99%+ uptime standards
Conducted root cause analysis using Jira, collaborating with development teams to resolve critical issues within SLA requirements
Mentored 2 junior testers on automation best practices and test design principles
Technologies: Selenium, Java, TestNG, Jira, HP-QC, SQL, REST APIs

SeleniumJavaTestNGJiraHP-QCSQL+3

Infosys

2 roles

Test Engineer

Promoted

Jul 2016 – Feb 2020 · 3 yrs 7 mos

Executed testing for premier banking clients across on-site and offshore delivery models, progressing from Test Analyst to Test Engineer.
Key Responsibilities & Achievements:
Delivered functional, regression, and automation testing for major banking clients including UBS, Bank of America, and RBS
Developed RPA solutions for process automation; built and maintained test automation suites for critical banking applications
Executed testing for web and mobile applications serving 100K+ users across multiple product releases
Contributed to sprint planning and test execution in Agile environments, ensuring test coverage aligned with business requirements
Developed comprehensive test cases and validation protocols, maintaining quality benchmarks across releases
Technologies: Selenium, Java, SQL, HP-QC, ALM, Agile/Scrum