Integrating BotGauge with GitHub Actions

GitHub Actions has become the default CI surface for a large share of product teams, and for good reason: it's close to the code, the YAML configuration lives in the repository, and the marketplace ecosystem means most integration boilerplate already exists as an action. This guide walks through setting up BotGauge in a GitHub Actions workflow — from the basic configuration through parallel execution and Slack notifications — using patterns that hold up in real pipelines.

Prerequisites Before You Touch the YAML

Before writing a single line of workflow configuration, make sure three things are in place. First, your BotGauge project must exist and have at least one test suite defined. If you haven't connected your repository yet, do that in the BotGauge dashboard under Projects → Connect Repo. Second, you'll need a BotGauge API key — generate one under Settings → API Keys. Third, store that key as a GitHub Actions secret (BOTGAUGE_API_KEY) in your repository's Settings → Secrets and Variables → Actions.

These steps sound obvious but are the source of about half the "my workflow isn't working" questions in any CI integration. Get them right before writing YAML.

The Baseline Workflow

The minimal working configuration triggers BotGauge tests on every pull request against your main branch:

name: BotGauge Tests

on:
  pull_request:
    branches: [ main, master ]

jobs:
  e2e-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Run BotGauge test suite
        uses: botgauge/run-tests-action@v1
        with:
          api-key: ${{ secrets.BOTGAUGE_API_KEY }}
          project-id: ${{ vars.BOTGAUGE_PROJECT_ID }}
          suite: regression
          fail-on-flaky: false

A few notes on these parameters. suite: regression targets your regression suite specifically — you don't want to run your full exploratory suite on every PR, only on merge or nightly. fail-on-flaky: false prevents the workflow from failing on tests BotGauge has already identified as historically flaky; those show as warnings rather than failures. You'll revisit that flag once your flakiness baseline is cleaned up.

Adding Parallel Execution

The default configuration runs tests serially. For most regression suites beyond 50 tests, serial execution will push your PR feedback time above 15 minutes — which defeats the point of running on every pull request. Parallel execution distributes test cases across multiple runners:

jobs:
  e2e-tests:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - name: Run BotGauge test shard
        uses: botgauge/run-tests-action@v1
        with:
          api-key: ${{ secrets.BOTGAUGE_API_KEY }}
          project-id: ${{ vars.BOTGAUGE_PROJECT_ID }}
          suite: regression
          shard-index: ${{ matrix.shard }}
          shard-total: 4
          fail-on-flaky: false

With four shards, a 200-test regression suite that takes 20 minutes serially typically finishes in 6–8 minutes in parallel — accounting for runner spin-up overhead. Increase the shard count for larger suites; four is a reasonable default for Growth plan users (the plan supports up to 20 concurrent runners).

We're not saying parallel execution has no cost. Each parallel job consumes GitHub Actions minutes, and on the free tier, those are finite. For teams on GitHub's Team or Enterprise plans, the minutes budget is generous enough that 4-shard parallelism is a non-issue; for teams on the free tier, run full parallel only on the main branch and use a single shard for feature-branch PRs.

Slack Notifications

Test results are only useful if the right person sees them promptly. Adding a Slack notification step is straightforward with the slackapi/slack-github-action action:

      - name: Notify Slack on failure
        if: failure()
        uses: slackapi/[email protected]
        with:
          payload: |
            {
              "text": "BotGauge tests failed on *${{ github.ref_name }}*",
              "attachments": [{
                "color": "danger",
                "fields": [{
                  "title": "PR",
                  "value": "${{ github.event.pull_request.html_url }}",
                  "short": false
                }]
              }]
            }
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
          SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK

The if: failure() condition means the notification only fires when tests actually fail — you don't want a green-build notification on every PR. Store your Slack webhook URL as a secret (SLACK_WEBHOOK_URL) using the same approach as the API key.

Environment-Specific Testing

Most teams have at least two environments: a staging environment deployed from feature branches, and a production environment that only gets main. You want BotGauge to test against the right URL for each context. Use GitHub environments and environment-scoped variables:

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  e2e-tests:
    runs-on: ubuntu-latest
    environment: ${{ github.ref == 'refs/heads/main' && 'production' || 'staging' }}
    steps:
      - name: Run BotGauge tests
        uses: botgauge/run-tests-action@v1
        with:
          api-key: ${{ secrets.BOTGAUGE_API_KEY }}
          project-id: ${{ vars.BOTGAUGE_PROJECT_ID }}
          base-url: ${{ vars.APP_BASE_URL }}
          suite: regression

With this configuration, APP_BASE_URL is scoped to each GitHub environment — set it to your staging URL in the staging environment and your production URL in the production environment. BotGauge uses the base-url parameter to resolve all relative paths in the test suite, so a single suite definition covers both environments without duplication.

Handling the Results Dashboard Link

One workflow improvement that saves significant time in large teams: surface the BotGauge results link directly in the pull request. When a test run fails, engineers will navigate to BotGauge to see the failing screenshots and traces. You can put that link one click away by using a GitHub step summary:

      - name: Post results summary
        if: always()
        run: |
          echo "## BotGauge Test Results" >> $GITHUB_STEP_SUMMARY
          echo "View detailed results: ${{ steps.run-tests.outputs.results-url }}" >> $GITHUB_STEP_SUMMARY
          echo "Pass rate: ${{ steps.run-tests.outputs.pass-rate }}%" >> $GITHUB_STEP_SUMMARY

The results-url and pass-rate outputs are provided by the BotGauge action. This puts a direct link to the results dashboard in the GitHub Actions summary tab of every workflow run, reducing the round-trip of copying a run ID and navigating to the BotGauge UI manually.

Common Issues and What They Mean

Two failure patterns come up consistently when teams first set up this integration. The first is authentication failures on the test runner: if your staging environment requires HTTP basic auth or IP allowlisting, the BotGauge runner's IP ranges need to be added to your allowlist. Check the BotGauge docs → CI/CD → IP Ranges section for the current egress IPs. The second pattern is tests that pass locally (in the BotGauge web UI) but fail in CI: this usually means the staging deploy hasn't fully finished by the time the BotGauge action starts. Add a readiness check step before the test step to poll your app's health endpoint and confirm the deployment is live before running.

Once the integration is stable, the signal quality from your PR checks improves substantially. Engineers get actionable feedback — specific failing flows with screenshots and element traces, not just "build failed" — within minutes of pushing code. That feedback loop is what makes autonomous testing worth the setup time.