Aleksandr Meshkov

CEO

Budva, Montenegro13 yrs 9 mos experience

Most Likely To SwitchAI Enabled

Key Highlights

Over 12 years of experience in QA and AI evaluation.
Developed comprehensive AI evaluation framework supporting 20+ metrics.
Led QA practices in diverse industries including Healthcare and Finance.

Stackforce AI infers this person is a seasoned AI Evaluation Architect with extensive experience in Quality Assurance across multiple industries.

Contact

Skills

Core Skills

Ai EvaluationQuality AssuranceTest AutomationAgile MethodologiesPerformance TestingProject ManagementProcess ImprovementTesting StrategyTesting Execution

Other Skills

Security TestingArtificial Intelligence (AI)PythonSeleniumJMeterAgile LeadershipGitCitrusSerenity BDDJenkinsGrafanaTeam ManagementTMMiSoftware TestingSQL

About

With over 12 years of experience in quality assurance and AI, I lead QA and AI evaluation practice at First Line Software. My role focuses on defining AI evaluation strategies, building evaluation tools for AI solutions, and implementing robust guardrails to enhance AI safety. I worked with platforms like Azure AI Studio, Google AI Studio, DeepEval, Opik Comet, and Promptfoo to ensure comprehensive evaluation and reliability of cutting-edge AI technologies. Also, I'm the author of Eval-ai-library, a comprehensive AI evaluation framework that supports 20+ metrics and uses Temperature-Controlled Verdict Aggregation via Generalized Power Mean, which I also developed. My mission is to advance AI evaluation practices by integrating innovative approaches and evaluation methodologies for generative AI systems. At First Line Software, I focus on fostering collaboration and developing scalable testing frameworks to deliver high-quality, industry-leading solutions that empower our clients and align with the evolving AI technology landscape.

Experience

13 yrs 9 mos

Total Experience

3 yrs 5 mos

Average Tenure

3 yrs 7 mos

Current Experience

First line software

Head of QA and AI Evaluation

Oct 2022 – Present · 3 yrs 7 mos · Budva, Montenegro

Managing teams of manual software testing, automation testing, performance testing, and security testing in different projects for clients of the company.
Established a methodology for evaluating the AI and Gen-AI Solutions
Create EvalTool for evaluating Gen-AI-based solutions, RAG, AI Agents, and Fine-tuning models
Developed an own approach to measure the quality of Gen AI solutions based on a combination of AI-judge and mathematical methods of evaluation
Developing GenAI evaluation and Guardrails management system
Developed approaches of AI-assisted testing and implemented them inside the company's projects
Working with Azure AI Studio, Google AI Studio, and DeepEval
Defining and implementing QA Strategy (Risk-Based testing, Model-Based testing, Rqmt-based testing)
Engage as a QA Lead to build automation testing and performance testing from scratch (several projects) (Automation - Selenium Java, JS, WebDriverIO, Cucumber/SpecFlow(Gherkin), Pytest, Python, Performance - Jmeter, Gatling, InfluxDB, Grafana, Jenkins).
Managed QA and AI projects in Healthcare, Delivery, Warehouse, and Finance industries

Security TestingArtificial Intelligence (AI)AI EvaluationQuality Assurance

Home credit russia

Director QA Department

Jul 2017 – Oct 2022 · 5 yrs 3 mos · Moscow, Russia

Managed QA department (Manual integration testing, Automation testing and Performance testing and Release team), > 120 employees
Implemented Testing in Scaled Agile Framework in the bank
Organized and facilitated Release activities (release every 2 weeks)
Had a direct impact on IT Technical Transformation to integrate DevOps and QA practices
Implemented Behavior-driven testing for Agile teams
My achievements:
1. Created automation testing infrastructure by Citrus framework and Serenity BDD Automation Framework which reduced integration regression testing from 1 month to one day. Costs decreased from 1500 MD to 100 MD.
2. Created performance integration testing process by Jmeter, GIT, Jenkins, Grafana, Graylog, Kibana, which reduced TTM performance tests from 1 week (including more than 20+ system performance tests) to 5.
3. Involvement of each manual tester in the tasks of test automation, ultimately resulted in a significant 50% reduction in the cost of automation developers to support the regression model.

Agile LeadershipGitQuality AssuranceAgile Methodologies

Performance lab

2 roles

Technical QA Director

Dec 2015 – Jun 2017 · 1 yr 6 mos · Moscow, Russia

Managed team > 300 employees (manual, automation, performance testing)
Launched > 30 projects (finance, retails, telecommunication, IT companies, and others)
Developed and planned the assessment model for employees based on ISTQB
Developed technical strategy
Startup new direction - Development projects
Represented company on QA Meet-up, Conferences

Agile LeadershipTeam ManagementQuality AssuranceProject Management

Head of QA Consulting

Mar 2014 – Nov 2015 · 1 yr 8 mos · Moscow, Russia

Adopted and implemented TPI Next QA model for Russian companies
Executed audits testing and QA, also Release processes (5 projects)
Took part in developing Russia Quality Report (Research of QA in Russia)
Worked in bank, retail, and telecom projects (over 15) as Test Lead and Test Manager
Created the Metric System based ITIL for measuring the testing process
Created the Quality Gates methodology for Russian companies

Agile MethodologiesTMMiQuality AssuranceProcess Improvement