Back to Docs
Advanced

CI/CD Integration

Run evaluations in GitHub Actions

Last updated: August 19, 2025
Category: advanced

CI/CD Integration

Run Vecta evaluations in your CI pipeline. Catch regressions before they reach production.

Coming Soon

We're building comprehensive CI/CD integrations for Github Actions.

What You'll Be Able To Do

Automated Testing

  • Run evaluations on every PR
  • Block merges that degrade performance
  • Track metrics over time

Regression Detection

  • Catch retrieval quality drops
  • Detect accuracy degradation
  • Monitor latency changes

Performance Tracking

  • Historical metrics in dashboard
  • Trend analysis
  • Team alerts

Get Early Access

Want to try CI/CD integration now?

Book a demo to get early access.

Manual CI Integration Today

You can set up basic CI today:

# In your test suite
def test_rag_performance():
    from vecta import VectaAPIClient
    
    client = VectaAPIClient(api_key="your-key")
    
    results = client.evaluate_retrieval(
        benchmark_id="your-benchmark-id",
        retrieval_function=my_retriever
    )
    
    # Assert minimum quality
    assert results.chunk_level.f1_score >= 0.70, \
        f"F1 score {results.chunk_level.f1_score:.3f} below threshold"

Next Steps

Need Help?

Can't find what you're looking for? Our team is here to help.