Skip to content
View criguex's full-sized avatar

Block or report criguex

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
criguex/README.md

Leer en español

Cristian Guerra — Senior SDET · WAR Paradigm

LinkedIn Location Open to remote contract roles — message me on LinkedIn

In simple words

I make sure software does what it promises. I build automated checks that use an app the way a person would and raise a flag the moment something breaks, and I wire them into the pipeline so they run on every change. Lately I also check AI: I test what it builds and how it is graded, and I break things on purpose to prove the checks really notice.

Start here

Project Focus
scientific-qa-evidence Formal-proof reproduction and scientific mutation testing
predictor-debugger Physics checks for neural fluid predictions
civica-playwright-challenge UI automation, accessibility and mutation testing
restful-booker-quality-suite API testing and release quality gates
agent-eval-jev Evaluation of AI agents that use tools
qa-automation-course Interactive QA course in Spanish

Three cases

Three projects, one per kind of work. Each one says the problem, what I built, how to check it yourself and what was measured. The numbers come from lab runs, not from production systems.

1 · Test automation for a release: restful-booker-quality-suite

  • Problem: decide whether an API can ship, and keep known bugs visible instead of hiding them.
  • What I built: risk-based Playwright API suite with Zod contracts and latency budgets, run in four CI gates against a pinned Docker copy of the app.
  • Check it: docker compose up --build -d && npm ci && npm run quality
  • My part: a demo run as a release. I set the risk model (P0: unauthorized changes, booking-lifecycle integrity, authentication) and what blocks a release; I kept the 8 defects visible and wrote down every exclusion. AI assistants wrote the Playwright/Zod code and the report under my direction.
  • Measured (lab): 43 checks pass in 2.0 s; 8 confirmed defects kept as expected failures, so a fix turns the suite red until it is acknowledged.

2 · Evaluating an AI agent: agent-eval-jev

  • Problem: know when a tool-using agent invents numbers or skips a step, without trusting one LLM judge.
  • What I built: code checks first, a calibrated Jev judge second, an LLM judge only for comparison, and a CI gate with a human-review band.
  • Check it: python -m agenteval evaluate && python -m agenteval gate
  • My part: I decided what to test, the criteria weights and the threshold rules; an AI assistant drafted the 60 reference labels against my rubric and I reviewed them, with no case-by-case log of that review; AI assistants built and ran it under my direction.
  • Measured (lab): the evaluator agrees with the reference labels (AI-drafted, reviewed by me without a case-by-case log, no independent annotator) on 93.1% of 29 held-out cases (kappa 0.86); the same gate passes agent v2 and blocks v1.

3 · Scientific QA: predictor-debugger

  • Problem: check whether a neural network that predicts a 3D fluid respects physics, without moving the goalposts after seeing the data.
  • What I built: a physics debugger with hypotheses and thresholds committed before each round, run on several machines.
  • Check it: uv run --locked verify.py --saved mac (minutes, no data download)
  • My part: I orchestrated the rounds: three machines and operating systems, pinned versions, publish failures as failures, a reproducible v1.0. The AI proposed hypotheses and thresholds, which I approved before each round, and wrote and ran the code.
  • Measured: 49 pre-registered hypotheses over four rounds, 48 pass and one (H14) failed and is published as such; the same fingerprints in six environments.

Technical leadership

I lead as an individual contributor: I set the risks, the gates and the rules, and AI agents do much of the execution under them. Three cases, anonymized:

  • Stopping production for a false green. In an automated content pipeline I own, an audit showed the image quality gate scored 90 and "passed" while rejecting 10 of 10 shots. I stopped generation, decided the 12 open gates of the fix plan and only resumed when the gate measured again; fix and deploy took about 5 hours. Rule: a gate that says OK without evidence is worse than no gate.
  • Release criteria for 10 small client apps. I set the definition (P0 is whatever blocks today or creates legal or real-money risk now; an unrequested feature is never P0; a false claim in the docs is P0) and the rule that nothing ships without smoke and E2E green and a reviewer who is not the fixer. One review round, run by AI agents under that protocol, found 17 P0s and closed 16, with about 125 new tests.
  • Correcting AI with evidence. When an AI assistant recomputed my public experiment and called it an independent review, I made it relabel the run as AI-operated under my direction, and published it as a separate release (repro-ia-20261026). Who ran something changes what you can claim.

About

Senior SDET with 7+ years in test automation, mostly in fintech and banking. I design automation frameworks from zero, test APIs and web apps end to end, and put quality gates into CI.

Lately most of my work is testing what AI builds: making verdicts measurable when outputs are not deterministic, and using AI assistants to design and explore tests faster while a person still signs off.

Featured work

Ordered from most to least important.

Flagship · verifying science and AI results

Project What it shows Stack
scientific-qa-evidence Independent, reproducible QA audit of OpenAI's Lean 4 proofs for Navier–Stokes and Euler: rebuilt from source, rechecked with two independent proof checkers, and mutation-tested on purpose. Read the site Lean 4 · Python · CI
predictor-debugger A pre-registered physics debugger for a neural network that predicts a 3D fluid: thresholds committed before the data, verdicts reproduced on three machines, and failed hypotheses published too Python · numpy · uv · CI

Teaching

Project What it shows Stack
qa-automation-course Free interactive QA Automation course (content in Spanish): 35 modules in 4 levels, from test oracles and Playwright to AI evaluation, mutation testing and scientific QA, each with a self-graded quiz, a verifiable challenge and a rubric. Take the course HTML · JavaScript · GitHub Pages

Evaluating AI

Project What it shows Stack
qa-eval-lab Playwright UI tests and LLM evaluation (DeepEval + Promptfoo) wired into one CI eval-gate Playwright · TypeScript · DeepEval · Promptfoo · CI
agent-eval-jev Evaluating a real tool-using AI agent with code checks, a calibrated Jev judge and an LLM judge, wired into a CI gate Python · Jev · LLM judge · CI
jev-ci-triage Sorts every failing CI test into regression, flaky, environment, test-data or unknown: rules first, Jev only for the ambiguous rest; never re-runs or hides a failure TypeScript · Playwright/JUnit · Jev
playwright-jev Semantic assertions for Playwright Test (toMean, toChoose, toBeScored) powered by Jev, recorded once and replayed in CI Playwright · TypeScript · Jev
jev-test-impact Runs only the tests a change touched: import graph and ownership rules first, Jev for the ambiguous rest, the full suite on any doubt TypeScript · Vitest · Playwright · Jev
laya-server A free local judge (the Laya model, on CPU) behind the Jev contract, with a cascade to the cloud for doubtful cases Node.js · Laya · Windows

Testing the tester · mutation

Project What it shows Stack
war-paradigm WAR Paradigm: test the tester. Break reality on purpose (a physical law in a simulation, a step of a flow, an AI answer) and check whether the checks notice. Runnable experiments behind my posts Playwright · TypeScript · CI
civica-playwright-challenge Production-shaped framework: Page Object Model, semantic locators, deterministic mock layer, accessibility checks, self-learning mutant scenarios Playwright · TypeScript · GitHub Actions

Requirements QA

Project What it shows Stack
spec-review-case-study How I review a spec before anyone builds: 19 findings (numbers that do not add up, costs stated as facts, rules nobody could test), a reusable checklist and a script that recomputes the numbers Markdown · Python

Test automation frameworks

Project What it shows Stack
restful-booker-quality-suite Release-style API testing for Restful Booker: lifecycle, auth, Zod contracts and latency budgets in four CI gates, with eight confirmed defects kept visible Playwright · TypeScript · Zod · Docker
playwright-saucedemo-e2e Cross-browser E2E suite with POM and CI on every push Playwright · TypeScript
karate-api-testing Data-driven REST API testing with happy and failure paths Karate DSL · Java · Maven
serenity-screenplay-salesforce Screenplay pattern for business web flows with living documentation Serenity BDD · Java · Cucumber
selenium-cucumber-pom Classic layered web framework: drivers, hooks, page objects, test data Selenium · Java 17 · Cucumber

How I work

  • Golden flows first, then mutate them. One change per mutant; if the suite stays green, that's a finding.
  • API first, UI for evidence. Fast, stable checks at the service layer; the browser proves the user journey.
  • Deterministic verdicts for non-deterministic systems. Pass/fail where the spec allows it, scores against acceptance criteria where it doesn't, and the same input run 3 times.
  • Hard rules live in code, not in prompts. Hooks, CI gates and reviews that can't be forgotten.

Toolbox

Web & mobile: Playwright · Selenium · Cypress · Serenity BDD (Screenplay) · Appium API & performance: Karate DSL · REST Assured · Postman · k6 · JMeter Languages: TypeScript · JavaScript · Java · SQL CI & infra: GitHub Actions · GitLab CI · Jenkins · Docker Practices: BDD (Cucumber/Gherkin) · contract testing · test management with Jira/Xray · AI-assisted testing (Claude Code, Copilot, MCP)

Contact

Reach me on LinkedIn. Open to remote contract roles in QA automation and SDET.

Pinned Loading

  1. scientific-qa-evidence scientific-qa-evidence Public

    Independent formal-proof reproduction and scientific mutation testing, with traceable data and animated explainers. By Cristian Guerra, QA Automation, Medellín, Colombia.

    HTML 6

  2. predictor-debugger predictor-debugger Public

    Pre-registered physics debugger for a learned 3D Navier–Stokes predictor: evidence, logs, three machines and a blind round 5

    Python

  3. civica-playwright-challenge civica-playwright-challenge Public

    IMDb test automation framework — Playwright + TypeScript, Page Object Model, semantic locators (no XPath/CSS), deterministic mock fallback. Technical challenge.

    TypeScript

  4. restful-booker-quality-suite restful-booker-quality-suite Public

    Risk-based Playwright API test suite and QA strategy for Restful Booker

    TypeScript

  5. agent-eval-jev agent-eval-jev Public

    Evaluating a real tool-using AI agent with code checks, a calibrated Jev judge and an LLM judge, wired into CI

    Python 1

  6. qa-automation-course qa-automation-course Public

    Interactive QA Automation course (Spanish): 35 modules from oracles to AI evaluation, mutation testing and scientific QA, with self-graded quizzes and rubrics

    JavaScript