AI skills on top of a Java · Cucumber · Serenity · Selenium framework. Each nightly report is checked, every failure is classified and sent to the team that has to fix it, test cases are written from specs, and the test data they need is created automatically. It all runs through GitLab CI and Jenkins.
About 25 product modules across several production regions run every night. Someone had to read every failure by hand and decide: is this our script, our data, a flaky run, or a real bug?
From a merge request or the nightly schedule to a fixed script or a confirmed defect.
flowchart LR
A[GitLab MR / nightly] -->|REST trigger| B[Jenkins pipeline]
B --> C[Serenity reports]
C --> D[Slack summary]
D -->|trigger| E[AI report verification]
E --> F{Classify}
F -->|Script fix| G[Slack: automation-quickfix]
F -->|Test data| G
F -->|Intermittent| G
F -->|Defect| H[Slack: qa-manual + draft Jira]
F -->|Low confidence| I[Human review]
G --> J[fix-and-rerun → MR]
J --> A
E -.learns.-> K[(Knowledge base)]
K -.-> E
Each one is an AI skill: a written procedure with rules and helper scripts. The agent follows it the same way every time.
Once the nightly summary is in Slack, a post-build stage starts the agent in headless mode for that build.
sequenceDiagram
participant J as Jenkins
participant S as Slack
participant A as AI agent
participant Q as automation-quickfix
participant M as qa-manual
J->>S: Summary block
J->>A: post-build trigger (build #)
A->>J: Fetch reports
A->>A: Group + classify
A->>J: Rerun suspected flaky
J-->>A: Rerun result
A->>Q: Script / data / intermittent
A->>M: Defects + repro + Jira draft
A->>S: Thread summary
Turns a build number into two reports, one for each audience. It's the engine behind component 1, and you can also run it yourself with /regression-triage <build>.
flowchart TD
A[Build #] --> B[Fetch report]
B --> C[Group by signature]
C --> D[Read knowledge base]
D --> E[Trace code path]
E --> F[Check recent commits]
F --> G[Classify + confidence]
G --> H[Automation report]
G --> I[Manual QA report]
H --> J[fix-and-rerun]
Takes a Confluence/PRD page, plus manual cases if you have them, and produces a traceable case set. Approved cases are then automated.
flowchart TD
A[Spec + cases] --> B[Requirement model]
B --> C[Positive]
B --> D[Edge]
B --> E[Negative]
B --> F[State]
B --> G[Non-functional]
C & D & E & F & G --> H[Traceable xlsx]
H --> I{Automate?}
I -->|yes| J[Scenarios reuse steps]
J --> K[Test data skill]
Triggered by test case generation (a new prerequisite), by triage (a test-data verdict), or by an engineer.
flowchart LR G[Generation] --> T[Test data skill] R[Triage] --> T T --> S[Data suite
idempotent] T --> Q{Store} Q --> DB[(MySQL)] Q --> SF[(SFTP)] Q --> RP[(Repo)]
A test stage maps changed files to product pods and calls Jenkins over REST, so only those pods' suites run.
buildWithParameters call, then polls the queue and the build
sequenceDiagram
participant G as GitLab CI
participant J as Jenkins
participant S as Slack
G->>G: changed files → pods
G->>J: POST buildWithParameters
J-->>G: queue URL
G->>J: poll build
J->>S: report + AI triage
J-->>G: SUCCESS / FAILURE
Rules are applied in order and the first strong match wins. Every verdict cites evidence: the error line, expected vs actual, and a screenshot.
Locator or timeout failure while the screenshot shows the feature working, or the wrong dropdown, or a recent commit to that page.
A missing prerequisite, a name collision or a stale template. Sent to the test data skill.
Passed on the automatic rerun. A flake counter turns it into a script fix once it crosses a threshold.
Login or session errors, network errors, or many unrelated modules failing with the same signature.
Correct input but a business mismatch, an error toast or a 4xx/5xx, and it fails again on rerun.
Low confidence or conflicting evidence. The report says what evidence would settle it.
At runtime the framework looks in the DB first, then the repo, then SFTP. The skill always says which store it used and why.
Structured JSON that varies by scenario or environment, changes often, or is large. Keyed by scenario name.
journey templates payloadsFiles every Jenkins agent reads, binaries and bulk uploads, and data that pipelines refresh.
testData.xlsx upload CSVsSmall, static fixtures that get reviewed along with the code that uses them.
CSV headers XML fixturesWhat keeps an AI-driven pipeline trustworthy.
Nothing is called a defect without expected vs actual, a screenshot or an API response.
40 failures usually come from 5–8 root causes. Analyse those.
A failure is only labelled intermittent after the automatic rerun passes.
Jira tickets, pushes and remote data writes need a person to approve them.
Confirmed patterns and corrections go into a knowledge base that's read first on the next run.
Credentials are referenced by key name and read through permission prompts.