D2 / D3 — the three tiers, and a dist runtime with nothing outside macOS in it
Status as of a908f81 on vphone-intg-update. This is the handover for the
self-containment work: what the rule actually is, what was removed and where
each thing went, what is verified and how, and what is left.
1. The rule was never one rule
check_aux.sh used to hold a single list of "programs we still reach for",
applied to the whole repository. That made xcrun in a build script and
xcrun in something the .app ships count the same, so the list shrinking
said nothing about whether the product was any closer to standing on its own.
Three environments run code here, and they have different rules:
Every script declares its tier on line 2:
Scripts/dist_manifest.sh reads those declarations and prints the dist
payload. Scripts/build.sh and make bundle both stage from it, so the
bundler is an allowlist. It used to be a list of exclusions, which is how
the .app came to carry build.sh, check_aux.sh and setup_tools.sh — the
last of which runs brew install. An undeclared script now ships nowhere and
gate 0 says so.
make bundlestagesContents/Resourcesnow. It did not, andbuild.shdid, somake check-auxwas inspecting a five-binary bundle while users got twenty scripts and a Homebrew-linkedtrustcache. The gate was green because it was looking at the wrong artifact. If you add a bundling step, add it to both or the gate goes blind again.
2. What the dist tier reached for, and where each one went
DIST_REMAINING in check_aux.sh — the list a release requires to be empty —
is empty. Keep it that way; adding a line is a decision to ship something
that does not work on a clean Mac.
The compile/install split
The guest bundle includes vphoned, vpregister, libvcamcaptured.dylib,
libcamfix.dylib, launchdhook-vphone.dylib, and the currently inert
SystemHook-vphone.dylib. ElleKit's TweakLoader.dylib is installed later by
Irisin into the chosen bootstrap. The earlier guest binaries used to be
compiled at CFW-install time — three by the installers, and vphoned twice
more (again by FirmwarePatcher.buildVphoned). That made a full Xcode install
a prerequisite for putting firmware on a VM.
Compiling moved to build time. Signing did not, and cannot: it uses the
target VM's own cfw_input/signcert.p12, which does not exist until a VM does.
cfw install still runs in dist, as it must — it is something vphone-cli
does on the user's machine.
scripts/guest_binaries.mk keeps the installers' own argument order,
framework list included. That looks arbitrary and is load-bearing:
-framework order decides LC_LOAD_DYLIB order, which decides the indirect
symbol table's numbering. With it, vpregister and libvcamcaptured.dylib
come out byte-identical to what the installers used to produce.
The vphone launchd hook is compiled as C and links only libSystem; it is
loaded through a short /vh alias during boot. The process hook currently has
no constructor, spawn interception, or ElleKit chain load.
3. Reading files: map, don't slurp
This bit cost a machine, so it is worth stating plainly.
Data(contentsOf:) with no options reads the whole file into resident memory.
Fine for a plist; ruinous for what this project opens:
DSCLocalSymbolTableparseddyld_shared_cache_arm64e.symbols, 1.17 GB on iOS 27, reading its string table and nlist array whole.DSCChunkSet.findStringVMAsread each executable mapping whole — ~130 MB each, two dozen of them — so one full-cache string search was 3.3 GB ofMalloc Large.ManifestHashPatcherhashes every component the build identity names, and that list includesOS: a ten gigabyte filesystem image.
All three map now. Measured on DSCFlatAddressingTests, footprint went
3419 MB → 13 MB. RSS is still ~1.1 GB and that is the point: it is clean
file-backed pages the kernel reclaims for free, not dirty heap it has to swap.
Two details that are easy to get wrong:
-
Do not slice a mapping and hand it to
loadLE(_:at:). That helper passes its offset toData.copyBytes(to:from:), whose range is in the collection's index space — correct only wherestartIndexis zero, which a slice of a mapping never is.DSCLocalSymbolTableholds one mapping and offsets into it for this reason. -
Never map a buffer you will mutate and write back over its own file.
Data.write(to:)replaces the file the buffer is mapped from; the mapping is invalidated underneath the write that is reading through it, and the next page fault is a SIGBUS — a killed process, no failed assertion, nothing in the log. Those reads say so in the spelling:Data(contentsOfFileToRewrite:), defined once inSources/FirmwarePatcher/Binary/InPlaceRewrite.swift.VPhoneSignerkeeps its mapping and is safe because it renames over the file rather than truncating it; rename does not invalidate a mapping.
make check-aux enforces the rule for Sources/. Tests/ is deliberately
outside it: a test opens a committed fixture of a few megabytes, so mapping
buys nothing, and those fixtures are exactly what the patch tests mutate and
write back over. Holding the tests to the production rule would trade a memory
problem they do not have for a SIGBUS they would.
4. The gates
make check-aux. None of them proves self-containment alone.
Gate 4 is still missing: a machine with no Homebrew at all, running a
matrix of real work. Gate 3 clears the environment but /opt/homebrew is still
on disk, so it cannot see a hardcoded absolute path. Do not read a green run
here as "it will work elsewhere".
5. Verified
Against the real tools, not by inspection:
fw urls --device iPhone17,3— identical 31 URLs toipsw download ipsw --urls.fw aea-key— byte-identical toipsw fw aea --key, and it decrypted both 24A435 volumes (the 8.8 GB rootfs and the 2.3 GB SystemOS cryptex) with/usr/bin/aea.fw im4p-create— byte-identical containers forisys/0,trst/1,msys/0;im4p-extractround-trips to the original payload.fw seal-tool --version 26.1— a working arm64eapfs_sealvolumeout of an 18 GB remote macOS IPSW, in 7.4 s, from a relocated bundle underenv -i PATH=/usr/bin:/bin.fw_prepare.sh --list— the full firmware matrix, same conditions.vphone-cli sign— same CodeDirectory, same CDHash asldidon the realvphoned. (The CMS blob differs; it always has. The cdhash is what AMFI checks.)- All gates green,
DIST_REMAININGempty. - 641 tests in 113 suites, zero issues, exit 0.
Not verified
A CFW install has not been run end to end since this change. It needs root
and a powered-off VM. The pieces it touches were each checked in isolation, but
the flow — cfw install --variant jb staging the five prebuilt guest binaries,
signing them with vphone-cli sign, unpacking the bootstrap with
vphone-archive — has not been driven once. That is the first thing to do.
6. Test fixtures — where they came from
ipsws/ is gitignored; the fixtures are large and must be rebuilt locally.
ipsws/ref_extract/macho_pristine/andref_extract/dsc_pristine/came out ofiPhone17,3_27.0_24A435_Restore.ipsw: the six Mach-Os from the rootfs (043-69462-653.dmg.aea), the 6.7 GB shared cache from the SystemOS cryptex (043-70113-702.dmg.aea). Both decrypted withvphone-cli fw aea-keyplus/usr/bin/aea. 24A435 is the right build and that is checkable:watchdogdhashes to0309b868…andmobileactivationdto89233513…, which are the digests frozen inDSCHVVMMPatcherTests. If those do not match, the fixture is from the wrong firmware and nothing below it is a comparison.ipsws/patch_refactor_input/raw_payloads/was extracted from a prepared 26.1/23B85 restore tree withvphone-cli fw im4p-extract.ipsws/patch_refactor_input/reference_patches/cannot be regenerated. It is the output of the deleted Python patchers.PatchComparisonTestsskips when it is absent, with that written into the file. What still guards those patchers is the frozen-digest work in the same directory, which carries its reference values in the source rather than in a file nobody has.
7. Left open
cfw-kitis build tier, declared in writing incfw-kit/run.shwith the reason: it still reaches forldid/gtar/zstd/ipswthe wayscripts/cfw_install*.shdid before they were cleaned out, so it does not ship and a dist user cannot run it. Whether it gets the same treatment or stays a development tool is a P3 question, left open on purpose. Do not add it todist_manifest.shuntil it is cleaned.- D4 — the host-side shell. The dist tier is still ten
.shfiles. Turning them into Swift is the next line of work and is deliberately not started here; the tiers and the gates are what make it possible to do it one script at a time without losing track of what ships. - Gate 4, above.
git-lfsis the one Homebrew formula left insetup_machine.sh, and it is not a tool this project runs — it is whatgit cloneneeds forscripts/resources.