Skip to main content

Running in CI

Run the small owned-browser suite on pull requests. It requires no external browser server: Hosanna global setup starts the debugger, hst run, Vite, and Chromium and waits for actual port/readiness signals.

The hosanna-ui-samples package scripts are the source example for the commands below. The sample intentionally does not add a repository workflow; the existing external or application-owned CI invokes the same deterministic scripts developers use locally.

GitHub Actions example​

name: Regression tests

on:
  pull_request:
  push:
    branches: [main]

jobs:
  regression-web:
    runs-on: ubuntu-latest
    timeout-minutes: 20
    permissions:
      contents: read
    env:
      HSC_LICENSE_KEY: ${{ secrets.HSC_LICENSE_KEY }}

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version-file: .nvmrc
          cache: npm

      - name: Install dependencies
        run: npm ci --no-audit --no-fund

      - name: Restore build-config secret overlays
        run: npx hst build-config:restore-secrets --quiet
        env:
          BUILD_CONFIG_SECRETS_DEV_BASE64: ${{ secrets.BUILD_CONFIG_SECRETS_DEV_BASE64 }}
          BUILD_CONFIG_SECRETS_PROD_BASE64: ${{ secrets.BUILD_CONFIG_SECRETS_PROD_BASE64 }}

      - name: Restore the pinned Hosanna SDK
        run: |
          if [ -z "${HSC_LICENSE_KEY}" ]; then
            echo "::error::HSC_LICENSE_KEY is required for the compiler and licensed SDK"
            exit 1
          fi
          npx hst sdk:install
          npx hst config source

      - name: Install Chromium and system dependencies
        run: npm run test:ui:install -- --with-deps

      - name: Run browser regression suite
        run: npm run test:ui:ci

      - name: Upload regression report and diagnostics
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: regression-web-${{ github.run_id }}
          path: test-results/
          if-no-files-found: warn
          retention-days: 14

Before copying this into another application, replace npm run test:ui:ci with its checked-in CI regression script and restore only the SDK and build-config overlays that application actually needs. An application-owned workflow may combine this job with lint, build, and unit checks. Its checked-in scripts and lockfile remain authoritative.

Runtime and dependency setup​

  • Read the Node version from .nvmrc or the repository's supported equivalent.
  • Use npm ci, not a floating install.
  • Cache npm's download cache through setup-node; do not cache generated build config containing secrets.
  • Install the Chromium revision matched by the locked Playwright dependency.
  • Let global setup start and stop the application. Do not add an arbitrary sleep or a second manually started Vite server.

Parallelism, retries, and timeouts​

A Hosanna Vitest project owns one application runtime and one fixed port set. Keep fileParallelism: false, maxWorkers: 1, and non-concurrent sequencing inside that project.

If CI sharding is required, give every shard a distinct management/app/Vite/browser port set and artifact root. The shared launcher does not allocate independent shard ports automatically.

Vitest owns test retry policy. Prefer no local retries and at most a small CI-only retry count after tests are deterministic; a retry must not replace state reset or readiness waits. Keep launch timeout (HS_TEST_TIMEOUT_MS/config.timeoutMs), Vitest hookTimeout, and testTimeout aligned with the slowest supported target.

The sample config uses one retry when CI is set and no retries locally. npm run test:ui:ci first runs the dedicated integration TypeScript check, then the desktop Browser suite.

Artifacts and failure status​

Upload the whole artifact directory with if: always() so reports survive both passing and failing test steps. A failed Vitest assertion still returns a non-zero command status; artifact upload must not mask it.

The artifact can include:

  • report.html and report.json;
  • named proof screenshots and state;
  • automatic failure screenshot, hierarchy, focus, debugger context, logs, and commands;
  • actual and diff PNGs from visual comparison;
  • full-run WebM, MP4, or MOV when enabled for a supported target.

After downloading the artifact, open <testRunId>/report.html, not the redirect if your artifact viewer rewrites relative paths.

Environment variables and secrets​

The deterministic hosanna-ui-samples browser suite should not require an external service or user credential. If an application does:

  • map the value with envSecret(...);
  • expose it only to the test step;
  • never echo it or put it in build-config source;
  • avoid video for a flow that renders the secret;
  • do not run secret-dependent jobs for untrusted fork pull requests.

Roku developer passwords and compiler/source access keys belong in CI secrets. Physical-device jobs also require attached, reachable hardware and are usually scheduled or manually dispatched rather than required on every pull request. Tizen jobs additionally require the external Samsung SDK/signing profile and an already trusted TV; webOS jobs require the external LG CLI and an already paired Developer Mode alias. The deterministic entry points are npm run test:ui:tizen:device and npm run test:ui:webos:device. Retain each run's report and external capture with the declared TV model as application release evidence.

Require the fast browser regression job for changes that affect runtime navigation, input, rendering, or sample rigs. Report browser-expression and physical-device matrices separately so each result identifies the environment it exercised.

For investigation steps, see Reports and Failure Investigation.

Reference commands: the application-owned test:ui:ci, test:ui:tizen:device, and test:ui:webos:device package scripts.

Talk to us