Open Source  · MIT License · v1.2.2

Black-Box Testing
without the boilerplate

Write expressive Gherkin tests against REST APIs, databases, and any backend — no glue code, no framework lock-in, no matter what your application is built with.

REST APIs · SQL Databases · Gherkin DSL · 6 official plugins · VS Code extension · ✨ AI-powered
Get Started ↓ View on GitHub
azertio — exec
❯ azertio exec --suite "REST API"
Building test plan…
Installing plugins: gherkin, rest
────────────────────────────────────
✔ GET /posts returns 200 42ms
✔ GET /users has 10 results 38ms
✔ POST /posts creates a resource 61ms
✗ DELETE /posts/1 returns 204 29ms
────────────────────────────────────
3 passed 1 failed in 2.1s
Quick Start

Up and running in minutes

No framework to integrate, no agent to configure. Download, point at your app, run.

1

Download & install

Download the distribution ZIP, unzip it anywhere and add the bin/ directory to your PATH.

Download v1.2.2
shell 
$ unzip azertio-cli-1.2.2-dist.zip
$ export PATH="$PATH:$(pwd)/azertio-cli-1.2.2/bin"
$ azertio version
azertio 1.2.2
2

Configure

Create an azertio.yaml in your project root. Declare the plugins you need — Azertio fetches them at startup.

azertio.yaml 
plugins:
  - gherkin
  - rest

configuration:
  rest:
    baseURL: 'https://api.example.com'
3

Write a test

Create a .feature file. Plain language, no glue code — each step is resolved by the active plugins.

tests/my-api.feature 
Feature: User API

  Scenario: List users returns 200
    When I make a GET request to "users"
    Then the HTTP status code is equal to 200
    And the response body contains:
      """json
      [{"id": 1}]
      """
4

Run

Execute from the terminal. Azertio resolves plugins, builds the test plan and runs it.

Full documentation →
Azertio execution example
5

View the report

Add the htmlreport plugin and Azertio writes a self-contained HTML file after each run — donut charts, collapsible result tree, and step-level error details, no server needed.

Azertio HTML execution report showing test results with donut charts
Powered by AI

From idea to test in seconds

The VS Code extension integrates with GitHub Copilot, Claude, GPT-4o, and any other language model available in VS Code to accelerate feature authoring — from inline completions to full file generation.

✨

Inline step completions

As you type a step line in a .feature file, the extension suggests the next step drawn from the real index of available steps in your project — not generic templates.

🤖

Generate feature from description

Describe what you want to test in plain language. Azertio generates a complete, ready-to-run Gherkin feature file — including correct structural keywords for your chosen locale.

📋

Generate feature from OpenAPI spec

Point Azertio at a Swagger / OpenAPI URL and it reads the endpoints and generates a full feature file covering the API surface automatically.

Works with any language model registered in VS Code.   See the VS Code extension →

Why Azertio

Everything you need to test, nothing you don't

A lightweight runtime with a powerful extension model. Bring your own plugins, connect any backend, run anywhere.

🧩

Plugin architecture

Every capability — REST, database, Gherkin parsing — is a first-class plugin resolved from Maven at runtime. No classpath wrestling.

📝

Human-readable DSL

Tests are plain feature files written in a domain-specific language. Non-technical stakeholders can read and review them directly.

🔌

Any JDBC database

Connect to MySQL, PostgreSQL, SQLite, H2, DuckDB or any JDBC driver — just add it as a runtime dependency in azertio.yaml.

🌐

REST API testing

First-class steps for HTTP requests, response assertion, JSON path extraction, and header validation. No glue code required.

⏱

Benchmarks built-in

Any step can be turned into a load benchmark with configurable concurrency and iteration count. Statistics are attached to the execution report.

🖥

VS Code integration

Browse plans, inspect execution trees, view attachments and re-run past executions — all from the editor, via a JSON-RPC language server.

sample.feature 
# language: en
Feature: JSONPlaceholder Posts API

  @ID-1 @GET
  Scenario: List all posts
    When I make a GET request to "posts"
    Then the HTTP status code is equal to 200
    And the response body contains:
      """json
      [{"userId": 1, "id": 1}]
      """

  @ID-10 @POST @GET
  Scenario: Create a post and get it after
    When I make a POST request to "posts" with body:
      """json
      {
        "title": "My new post",
        "body": "Content of my new post.",
        "userId": 1
      }
      """
    Then the HTTP status code is equal to 201
    And I store the value of field 'id' from the response body into variable id
    When I make a GET request to 'posts/${id}'
    Then the HTTP status code is 200

  @ID-11 @GET
  Scenario: Measure response time
    Given benchmark mode is enabled with 200 executions and 10 threads
    When I make a GET request to "posts/1"
    Then the benchmark P95 response time (ms) is greater than 2
The DSL

Tests as close to English as it gets

Write scenarios in plain language. Azertio matches each line to a step implementation at runtime — zero boilerplate, zero glue code.

  • 🏷

    Tag-based filtering

    Run any subset of tests with boolean tag expressions: GET and not slow

  • 🔄

    Shared Background steps

    Set up context once per feature with Background — no test setup duplication.

  • 💾

    Variables between steps

    Store values from one step and inject them into the next using ${'{'}name{'}'} interpolation.

  • 🌍

    Multilingual

    Steps can be written in English, Spanish, or a compact DSL alias — all from the same plugin.

Plugins

Install only what you need

Declare in azertio.yaml → resolved from Maven at runtime

gherkin Gherkin Parser

Loads .feature files and assembles a structured test plan. Supports Gherkin syntax and the compact DSL dialect.

Feature files Scenarios Background Tags
plugins: - gherkin
rest REST API

Steps for HTTP requests (GET, POST, PUT, DELETE), response assertions, JSON path extraction and header checks. Zero boilerplate.

HTTP verbs JSON path Status codes Headers
plugins: - rest configuration: rest: baseURL: 'https://api.example.com'
db Database

Execute SQL queries, assert row counts and table contents against any JDBC datasource. Bring your own driver as a runtime dependency.

SQL queries Count assertions Any JDBC Multiple DSes
plugins: - "db with com.mysql:mysql-connector-j"
htmlreport HTML Report

Generates a self-contained HTML execution report — donut charts, collapsible result tree, step-level error details, and a configurable accent color. Single file, no external dependencies.

Self-contained Donut charts Result tree
plugins: - htmlreport configuration: htmlreport: outputDir: reports/
pdfreport PDF Report

Generates a paginated PDF execution report with cover page, statistics and full result tree. Ready to share or archive after each run.

Paginated PDF Cover page Statistics
plugins: - pdfreport configuration: pdfreport: outputDir: reports/
messaging Messaging (JMS)

Publish and consume JMS messages in Gherkin. Subscribe before an action, then assert the resulting message. Works with ActiveMQ, Artemis, RabbitMQ, IBM MQ and any JMS-compliant broker — just add the provider JAR.

JMS ActiveMQ RabbitMQ Event-driven
plugins: - messaging configuration: messaging: systems: main: connectionFactoryClass: org.apache.activemq.ActiveMQConnectionFactory brokerUrl: tcp://localhost:61616
custom Build your own

Implement StepProvider and publish to Maven. Azertio resolves and loads it at startup — no forks, no rebuilds.

Java SPI Maven artifact Hot-install
plugins: - "com.example:my-plugin:1.0"
Azertio VS Code extension showing execution results
Azertio VS Code extension showing step documentation
VS Code Extension  ·  v1.1.0
IDE Integration

Run and inspect tests without leaving the editor

The Azertio extension connects to a local JSON-RPC server and gives you a full testing workflow inside VS Code — no terminal juggling required.

Install in VS Code View on Marketplace
📋

Execution history

Browse every past run with its date, duration and overall result. Each execution is stored locally and always available to review.

🌲

Full result tree

Drill down from test plan → suite → scenario → step. Each node shows its individual pass/fail status and execution time.

📎

Attachments inline

Steps that produce data — CSV query results, JSON responses — are saved as attachments and opened directly from the result tree.

🔁

One-click re-run

Re-execute any past run against its original test plan and profile with a single button — no need to reconfigure anything.

✨

AI step completions

Inline suggestions as you type in .feature files, powered by GitHub Copilot or any VS Code language model. Suggestions are based on the steps actually available in your project.

🤖

AI feature generation

Generate a complete Gherkin feature file from a plain-text description, or from an OpenAPI/Swagger spec. Supports multiple languages with correct structural keywords.

Comparisons

How does Azertio stack up?

Honest, detailed comparisons against the most popular Java API testing tools.

🥋

Azertio vs Karate

Both use .feature files and support REST + DB testing, but differ in architecture, philosophy, and extensibility.

Read comparison →
🥒

Azertio vs Cucumber + RestAssured

The most common Java BDD stack. Powerful but ceremonious — see where each approach shines and where the boilerplate adds up.

Read comparison →
📮

Azertio vs Postman / Newman

The most widely used API tool. Great for exploration, but version-controlled automated testing is a different game.

Read comparison →
Contributing

This is a work in progress

Azertio is actively evolving. Core features work, but APIs may change and there is plenty of room to grow. If you try it out, your feedback genuinely matters.

🐛

Found a bug?

Open an issue describing what you did, what you expected, and what actually happened.

💡

Missing a feature?

Open an issue explaining your use case — even a rough idea is worth sharing.

🤝

Want to contribute?

Open an issue first to align on the approach, then submit a pull request. All participation is welcome.

Open an issue on GitHub