Jan 2022 – Present · 4 yrs 2 mos · On-site

• Led evaluation pipelines for Amazon models (Bedrock, Strands, MCP, Agentic AI), benchmarking against Claude, Sonnet, and Nova using curated ground truth datasets
• Managed similarity detection and task-based quality scoring (TQA) for annotation responses
• Conducted structured IAA implementation to standardize consistency across vendor-annotated datasets
• Built rule-driven AWS API call trajectories aligned with documentation for high-quality SFT data
• Validated CLI command structure and accuracy for grounding GenAI responses
• Created dynamic evaluation logic enabling context-aware API usage assessment
• Owned $500K–$1M vendor programs scaling with data volume, delivering 38K+ annotations
• Designed Configuration Builder – a rule-based QA system powering annotation evaluation and error detection
• Reduced rework by 85% via real-time QA workflows and automated question dependencies
• Built multi-layer (bronze-silver-gold) ETL pipelines for Q Developer adoption tracking
• Delivered cohort-based insights and product telemetry dashboards, improving engagement analysis by 40%
• Streamlined QuickSight reporting, cutting $100+ in monthly costs and eliminating 2–3 hours of manual work weekly
• Worked on Amazon SageMaker Ground Truth platform for creating rule-based human-in-the-loop evaluation tasks
• Designed configurable workflows with conditional question logic to assess annotation quality and model response correctness
• Used Ground Truth to launch scalable evaluation templates for model quality scoring and accuracy assessment using dynamic evaluation rules
• Enabled automatic scoring of annotations based on evaluation templates, reducing evaluator subjectivity and increasing reliability of training data

Jun 2019 – Dec 2022 · 3 yrs 6 mos · On-site

Jun 2014 – Jun 2018 · 4 yrs · Bengaluru Area, India

• Experience in automation of Web based E-commerce sites.
• 4+years of experience in Software Test Automation & Manual Testing.
• Experienced in STLC Concepts like Functional, Non-functional, Regression, Integration and Automation Testing across Web applications.
• Experience of Selenium IDE and selenium web drivers.
• Excellent experience of element locator X-path and CSS locators.
• Experience in working with software development technologies like Core Java and OOPS concepts.
• Experience in database related languages like MySQL and ETL query execution.
• Experience in web development technologies like HTML/CSS and JavaScript.
• Proven skills in supporting, maintaining, Leading and training teams.
• Extensive experience in gathering and analyzing business requirements, processes and developing specification documents.
• Proven ability in Windows/UNIX Commands, Scripts and testing on Windows/UNIX platforms.
• Proven Experience in Agile methodology and sprints.
• Proven ability in working individually or as a part of team, Problem solving, Process improvement activities, Analytical skills, Multitasking and function efficiently with very aggressive schedules/timelines.
• Working with development team and other sources to understand and identify all requirements information that is relevant to the project.