Skip to content

chore(ci): batch unit tests, add timeouts, optimize Dockerfile#1135

Merged
pyramation merged 1 commit into
mainfrom
chore/optimize-ci-workflows
May 11, 2026
Merged

chore(ci): batch unit tests, add timeouts, optimize Dockerfile#1135
pyramation merged 1 commit into
mainfrom
chore/optimize-ci-workflows

Conversation

@pyramation
Copy link
Copy Markdown
Contributor

Summary

Three optimizations to reduce CI costs and prevent waste:

1. Batch unit tests (22 jobs → 4 jobs)

Unit tests previously ran as 22 individual 2vcpu jobs, each with ~1 min of download/install overhead for tests that run in seconds. Due to per-minute billing rounding, this was billed as 44 2vcpu-minutes for ~5 min of actual test work.

Now grouped into 4 batched jobs:

Batch Packages
uploads mime-bytes, uuid-hash, uuid-stream, etag-hash, etag-stream, stream-to-etag, content-type-stream, upload-names
packages-core url-domains, query-builder, csrf, oauth, 12factor-env, orm
packages-services postmaster, smtppostmaster, csv-to-pg, cli, pgsql-client, pg-ast
graphql query, codegen

Each batch runs tests sequentially using ::group:: markers for clear log separation. Estimated savings: ~73% of unit test minutes (44 → ~12 2vcpu-minutes).

2. Add timeout-minutes to all jobs

Prevents runaway jobs from burning money (default GitHub timeout is 6 hours):

  • build: 10 min (usually ~2 min)
  • unit-tests: 10 min (usually ~2 min)
  • pg-tests: 15 min (usually ~5-8 min)
  • integration-tests: 15 min (usually ~5-8 min)
  • examples-types: 15 min

3. Dockerfile: pnpm fetch for layer caching

Adds a pnpm fetch layer that pre-populates the pnpm store using only pnpm-lock.yaml. This layer is cached and only invalidated when dependencies change — not when source code changes. Subsequent pnpm install --frozen-lockfile --offline links from the local store without network access.

Review & Testing Checklist for Human

  • Verify all 4 unit test batches pass — confirm no test relies on being in its own isolated runner (e.g., global state pollution between tests in different packages within the same batch)
  • Verify Dockerfile builds correctly — the pnpm fetch + --offline pattern should work with pnpm 10, but confirm the Docker image builds and the CLI shims work
  • Spot-check timeout values — if any job routinely takes longer than the set timeout, it will start failing. The values are set at ~3× typical runtime but edge cases may exist

Notes

  • pg-tests and integration-tests are left as individual matrix jobs since each has different env vars and test durations. Batching these would require more careful grouping.
  • The ::group:: / ::endgroup:: markers in the batch test step provide collapsible sections in the GitHub Actions log so you can still see which specific test in a batch failed.

Link to Devin session: https://app.devin.ai/sessions/f42f42528fb0465684159227efbef017
Requested by: @pyramation

- Batch 22 unit test jobs into 4 grouped jobs (uploads, packages-core,
  packages-services, graphql) — saves ~73% of unit test minutes from
  per-minute billing rounding.
- Add timeout-minutes to all jobs: build (10m), unit-tests (10m),
  pg-tests (15m), integration-tests (15m), examples-types (15m).
  Prevents runaway jobs from burning money (default is 6h).
- Optimize Dockerfile layer caching: use pnpm fetch to pre-populate
  the pnpm store before copying source code. The store layer only
  invalidates when pnpm-lock.yaml changes, not on every code change.
@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@pyramation pyramation merged commit c3abd8d into main May 11, 2026
36 checks passed
@pyramation pyramation deleted the chore/optimize-ci-workflows branch May 11, 2026 23:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant