How much faster is macOS CI on Manzanita? Five open-source projects, measured
Marketing pages say "faster." This is the measurement. We took five public iOS and macOS repositories, forked them, changed a single runs-on line, and ran their own GitHub Actions workflows on both Manzanita and GitHub-hosted macOS runners. Every number below links to the public run it came from, so you can check it yourself.
How we measured
- Same workflow, one line changed. We only swapped the runner label (for example
macos-26tomanzanita-standard). We never edited application code to make a job green. - Warm vs warm. Where a project uses a cache, the fair comparison is a repeat run where both sides restored the cache. We report those, and note cold numbers separately.
- Execution and queue are separate. The tables are step and job execution time. During the pilot a Manzanita job can also wait behind your own purchased concurrency; that queue time is called out, not hidden.
- Honest scope. Some projects pin an exact Xcode we do not publish, or couple non-Apple toolchains into their macOS jobs. We say so in each case rather than dropping the project.
The short version
| Project | Workload | GitHub-hosted | Manzanita | Result |
|---|---|---|---|---|
| argmax-oss-swift | iOS test job (whole) | 25m54s | 8m16s | 3.13× faster |
| TablePro | iOS test workflow (wall) | 10m23s | 3m27s | 3.0× faster |
| SwiftLM | Build & unit test | 20m31s | 8m22s | 2.45× faster |
| speech-swift | SwiftPM + Metal (cold) | 13m44s | 9m04s | 1.5× faster |
| skip | Apple build step | 5m46s | 1m13s | 4.74× faster |
The pattern: simulator-heavy iOS test jobs and clean compiles see the biggest wins, because a dedicated M4 Pro with 6 vCPU beats a shared 3–4 vCPU VM and the simulator is already warm. Now the detail, including where the picture is mixed.
argmax-oss-swift — 3.13× on iOS, verified
Argmax's open-source Swift SDK runs a macOS and an iOS lane. On the repeat (warm-cache) run the iOS job went from 25m54s to 8m16s; the simulator boot alone dropped from 2m45s to 17s and the test step from 20m06s to 6m28s. The macOS lane was a more modest 9m02s to 6m27s (1.40×). Whole run: 25m58s to 8m48s.
Scope note: the repo requests exact Xcode 26.4.1, which we do not publish; timing runs used our Xcode 26.6 image, so this is a like-for-like performance result, not an exact-version compatibility claim.
Verify: Manzanita run · GitHub-hosted baseline
TablePro — 3.81× on the iOS test step
TablePro is a native macOS/iOS database client. Its iOS tests ran 9m50s to 2m35s (3.81×), the iOS unit-test step 8m13s to 1m39s, and the whole iOS workflow 10m23s to 3m27s (3.0×).
Honest limitation: the native macOS matrix could not finish on our image at the time because the project's build script calls rustup, which GitHub's macos-26 image ships and ours did not yet. That is an image gap we are closing, not a compute result, so we report only the iOS numbers here.
Verify: Manzanita iOS run
SwiftLM — 2.45× on build, and where we were slower
SwiftLM is an MLX (Apple-Silicon ML) project that targets Xcode 16, so we ran it on the manzanita-xcode16 channel. All eight retained jobs passed twice. The build-and-unit-test job went 20m31s to 8m22s (2.45×), and the larger inference jobs were 1.3× to 1.7× faster.
But three short jobs were slower on Manzanita — the SSD/draft memory guard (1m44s to 3m20s), vision integration, and fixtures — largely because this model-heavy repo churns through many large GitHub Actions cache keys and some missed on the repeat. We are showing this because a benchmark that only reports its wins is not a benchmark. On the standard Xcode 26 channel the unit tests were 2.70× faster but MLX Metal kernels failed to load, so the right migration for this project is the Xcode 16 channel, not the default.
Verify: Manzanita run · GitHub-hosted baseline
skip — a 4.74× build step, and an honest incomplete
Skip builds Swift apps for both Apple and Android. Its Apple build step ran 5m46s to 1m13s (4.74×), and Xcode selection dropped from 3m33s to 8s. We do not claim a full-job speedup: the project's test target invokes Gradle and needs the Android SDK, which we deliberately do not put on an Apple runner, so the full job stops at that cross-platform step. Useful as a build-step data point; not a complete end-to-end benchmark.
Verify: Manzanita run · GitHub-hosted baseline
speech-swift — 1.5× cold, against a cached baseline
soniqo's speech-swift is a SwiftPM + Metal workload. Even a cold Manzanita run (nothing cached) finished in 9m04s against a 13m44s GitHub-hosted run that hit its cache; the compile step alone was about 3.4× faster. With the cache warm the gap widens further.
Verify: Manzanita cold run · GitHub-hosted baseline
What this does and does not prove
It proves that on real, unmodified Apple workflows a dedicated M4 Pro runner is materially faster than a shared GitHub-hosted macOS VM — most on simulator and test-heavy jobs, less on already-short steps. It does not prove every job gets faster: short jobs dominated by cache I/O can be flat or slower, and a project that pins an Xcode we do not carry, or needs a non-Apple toolchain, will not be a clean win. Those are in the tables above on purpose.
The other half of the story is cost. Manzanita is a flat monthly price per runner instead of per-minute; the comparison page works through when that is cheaper.
runs-on to manzanita-standard and keep everything else. Start free · Read the docs