Performance Testing

PreviewOps can automatically run a k6 load test against your preview environment after every deploy, giving you latency percentiles, throughput, and error rates directly in the PR. You can also run tests on demand with a comment command.

Quick start

Zero config — just comment

/perf-previewops

PreviewOps detects the endpoints you changed in this PR and runs a 30-second load test with 10 VUs. Results appear as a PR comment with p50/p90/p95/p99 latency, RPS, and error rate.

Enable auto-testing on every deploy

Add to your .previewops.yaml:

performance:
  enabled: true

The test starts automatically after each successful deploy — no comment needed.

Plan limits

Performance testing requires Premium BYOC, Pro, or higher. It runs k6 load tests using operator Cloud Build resources, which is why it is not available on the Free plan.

Self-hosted? If you are running your own previewops instance (not the managed SaaS), all plans including free are allowed — you are providing the Cloud Build resources yourself.

¹ Premium BYOC AI features require the Custom LLM add-on. Activate it in the dashboard and store your LLM_PROVIDER, LLM_API_KEY, and LLM_MODEL credentials. Without it the test runs but AI scenario generation and post-run insights are skipped. See add-ons.md.

With the Advanced Perf add-on, all paid plans have a 60-minute max duration.

When your config requests more VUs or time than your plan allows, the test runs with the plan cap applied and a notice is added to the PR comment. No silent clamping.

Example cap notice (appears in the PR comment before results):

⚠️ Plan limits applied Your config requested 500 VUs · 2min but the Premium BYOC plan allows max of 10 VUs · 30min. The test is running with these limits. Upgrade to a paid plan → to unlock 100 VUs · 1hr + AI analysis.

Commands

Run a test

/perf-previewops

Runs with defaults (10 VUs, 30s, endpoints auto-detected from the PR diff).

Override parameters

/perf-previewops --vus=50 --duration=2m --target-tps=100

Save a baseline (Pro / Enterprise / Premium BYOC)

/perf-previewops --set-baseline

Future runs on this repo will show a "vs. Baseline" table with p95/p99 delta and error-rate change. Only one baseline is stored per repo — --set-baseline replaces any previous one.

Configuration

Add a performance block to your .previewops.yaml:

performance:
  enabled: false          # Auto-run after every deploy (default: false)
  vus: 10                 # Virtual users (default: 10, max per plan)
  duration: 30s           # Test duration — s, m, or h (default: 30s)
  rampUp: 5s              # How long to ramp up to full VU count (default: 5s)
  targetTps: 100          # Optional max requests/second cap
  apiBasePath: /api/v1    # Prepended to all detected/configured paths

  slo:                    # Service Level Objectives (violations are warnings, not failures)
    p90: 400              # ms — p90 latency threshold (optional)
    p95: 500              # ms — p95 latency threshold
    p99: 1000             # ms — p99 latency threshold
    errorRate: 0.05       # fraction — max error rate (default: 0.05 = 5%)

  scenarios:              # Explicit endpoints to test (overrides auto-detection)
    - path: /users
      method: GET
    - path: /orders
      method: POST
      body: '{"item": "test"}'
      headers:
        Content-Type: application/json
      weight: 2           # Relative frequency (default: 1)

  scriptPath: k6/load-test.js   # Use a custom k6 script from your repo

Using a custom k6 script

If you specify scriptPath, PreviewOps reads that file from your repo at the PR head and injects TARGET_URL as an environment variable so your script can use const BASE_URL = __ENV.TARGET_URL.

All other configuration options (VUs, duration, SLO) are still applied from .previewops.yaml and command flags.

How endpoints are detected

When no explicit scenarios are configured in .previewops.yaml, PreviewOps automatically detects which endpoints changed in the PR:

1. File scan (all plans) — Inspects the diff for route definitions matching common patterns:

   • Express / Fastify / Koa — app.get('/path', ...), router.post('/path', ...)
   • Next.js App Router — app/**/route.ts file paths mapped to API routes
   • FastAPI / Flask — @app.get('/path') decorator patterns
   • Generic — // GET /api/path comment conventions

2. AI-generated scenarios (Pro / Enterprise / Custom — or Premium BYOC with the Custom LLM add-on) — When a LLM is configured, the PR diff is sent to the model which generates more accurate, context-aware scenarios with appropriate weights. Falls back to file scan if the LLM is unavailable.

If no endpoints are detected, PreviewOps tests the root path / (or apiBasePath + / if configured).

Understanding the results

After the test completes, PreviewOps posts (or updates) the PR comment with:

✅ Performance test complete for PR #42

### 📊 Results

| Metric | Value |
|---|---|
| p50 latency | 48ms |
| p90 latency | 120ms |
| p95 latency | 180ms |
| p99 latency | 350ms |
| Avg latency | 90ms |
| Throughput | 15.3 req/s |
| Error rate | 0.20% |
| Total requests | 918 |

**SLO:** ✅ SLO PASS

### 📈 vs. Baseline

| Metric | Delta |
|---|---|
| p95 | 🔴 +45ms (+33.3%) |
| p99 | 🟡 +15ms (+4.5%) |
| Error rate | +0.01pp |

### 🤖 AI Analysis — gpt-4o

The p95 regression appears tied to the new middleware added in `src/middleware/auth.ts`…

SLO violations

SLO violations are informational only — they never block a deploy or fail the PR. They are surfaced as warnings so your team can decide whether to merge.

Baseline comparison

The 🔴 🟡 🟢 traffic-light icons on the baseline delta indicate:

• 🟢 Improvement or within 0%
• 🟡 Regression up to 10%
• 🔴 Regression greater than 10%

AI analysis (Pro / Enterprise / Custom — or Premium BYOC with the Custom LLM add-on)

Pro / Enterprise / Custom use the shared operator LLM key automatically — no setup required.

Premium BYOC requires the Custom LLM add-on: activate it in the dashboard and store your LLM_PROVIDER, LLM_API_KEY, and (optionally) LLM_MODEL credentials. Self-hosted operators configure the same vars directly in their environment.

• Summary — 2–4 sentence overview of performance health
• Bottlenecks — Per-endpoint issues with actionable suggestions
• Root cause — Single-sentence diagnosis, when identifiable
• Recommendation — Overall action item for the team

The model is told about SLO violations, baseline regressions, and the PR diff so its analysis is grounded in your actual code changes.

FAQ

Does a perf test failure block the deploy? No. Performance tests run asynchronously after the deploy succeeds. SLO violations are warnings only.

Can I run the test before deploying? Not currently — the test requires a live preview URL to test against.

What k6 version is used? PreviewOps uses the latest grafana/k6 Docker image via Cloud Build.

How long does the test run in CI/billing terms? Cloud Build charges for the total build duration including ramp-up and wind-down. A 30s test typically completes in under 2 minutes of Cloud Build time.

Can I have different configs per branch or environment? Not per-branch. The .previewops.yaml config applies to all PRs. Use command flags (--vus, --duration) to override per-run.