Drive Consolidation & Backup Plan

Grant Full Disk Access before anything else

System Settings → Privacy & Security → Full Disk Access → enable Terminal → quit and reopen Terminal. The quit-and-reopen is mandatory; the entitlement is read at process launch. Without it tmutil returns empty backup lists and find silently skips protected directories — which makes every count in Phase 4 wrong in a way that is hard to diagnose.

Baseline with nothing plugged in

mkdir -p /Users/SeanVargas/AI_OS/comms/outputs/2026-09-11-drive-consolidation/inventory
cd /Users/SeanVargas/AI_OS/comms/outputs/2026-09-11-drive-consolidation/inventory

diskutil list > 00-baseline-disklist.txt
cat 00-baseline-disklist.txt

/dev/disk0 is M2's internal SSD (500.3 GB, GUID partition scheme); disk1 and disk3 are its APFS containers. Anything appearing later marked (external, physical) is one of your drives.

One drive at a time. Always.

diskutil list external physical

Read the IDENTIFIER column carefully. disk4 is a whole drive; disk4s2 is a volume on it. Erase commands aimed at the wrong one destroy the wrong thing. Plugging in two drives at once is the single most common way people erase the drive they meant to keep.

Capture everything about that drive

DRIVE=disk4   # <-- substitute

diskutil info -all                  > 01-${DRIVE}-info-all.txt
diskutil info /dev/${DRIVE}  | tee    02-${DRIVE}-whole.txt
diskutil list /dev/${DRIVE}  | tee    03-${DRIVE}-layout.txt
system_profiler SPUSBDataType       > 04-usb.txt
system_profiler SPThunderboltDataType > 05-thunderbolt.txt

SPUSBDataType gives the enclosure's real product name and the negotiated link speed (Up to 10 Gb/s vs Up to 480 Mb/s). That negotiated speed is how you tell fast from slow as currently cabled — a 10 Gb/s NVMe drive on a USB-2 cable reports 480 Mb/s and behaves like a slow drive. Try a different cable and port before concluding a drive is slow.

Telling the filesystems apart

# APFS — the container pool is the truth
diskutil info /dev/disk4s2 | grep -E "Container Total Space|Container Free Space"
diskutil apfs list | sed -n '/disk4/,/^$/p'

# HFS+ / exFAT — the volume numbers are the truth
diskutil info /dev/disk4s2 | grep -E "Volume Total Space|Volume Free Space|Volume Used Space"

# Cross-check anything mounted
df -h | grep -v "^map"

Mount read-only where you can

For any drive that is old, suspect, or precious, mount it read-only so macOS cannot write journal replays, Spotlight indexes, or .DS_Store files onto it.

diskutil unmount /dev/disk4s2
diskutil mount readOnly nobrowse -mountPoint /Volumes/RO_disk4s2 /dev/disk4s2

# confirm it actually took
diskutil info /dev/disk4s2 | grep -E "Volume Read-Only|Media Read-Only"
# want: Volume Read-Only: Yes

A read-only mount is a software guard, not a hardware one. It stops accidents, not a determined sudo.

Stop Spotlight from grinding the old drives

sudo mdutil -i off /Volumes/<DRIVE>
# -X removes the existing index store, which WRITES to the drive.
# Skip -X entirely on any read-only or suspect drive.
sudo mdutil -X /Volumes/<DRIVE>

Indexing a large old drive is hours of sustained random reads on hardware you are trying to be gentle with.

Check Mini, and record M2's current TM registration

tailscale status | grep -i mini
ping -c 3 100.72.22.100
ssh -o ConnectTimeout=8 seanvargas@100.72.22.100 'hostname; df -h /; tmutil destinationinfo'

# M2's own registered destinations, and their UUIDs (needed in Phase 6)
tmutil destinationinfo
tmutil destinationinfo -X

If Mini does not answer, write "Mini: unreachable as of 2026-09-11, state unknown" into the inventory and proceed. Mini is not on M2's critical path, but you cannot claim a fleet-wide backup posture without it.

Label the drives physically

Tape and a marker. Every later phase refers to drives by number. Confusing drive 1 with drive 2 during Phase 6 is how people erase a backup they meant to keep.

SMART status — and why "Not Supported" is fine

diskutil info /dev/disk4 | grep "SMART Status"

Optional deeper look, if you want it: brew install smartmontools then smartctl -d sat -a /dev/disk4 — the -d sat flag often gets through a bridge that diskutil cannot. If smartctl also fails, that confirms the enclosure blocks SMART. Move on.

Filesystem verification — verify, do not repair

diskutil verifyVolume /dev/disk4s2
diskutil verifyDisk   /dev/disk4      # APFS container level

Behavioral health — more useful than SMART on USB

log stream --predicate 'eventMessage CONTAINS "I/O error" OR eventMessage CONTAINS "disk4"' --style compact

Red flags: repeated I/O error or media-error messages · clicking or a spin-up/spin-down cycle every few seconds · transfer rate collapsing from tens of MB/s to single-digit KB/s and staying there · the volume unmounting itself mid-copy.

dd if=/dev/rdisk4 of=/dev/null bs=1m count=2000 status=progress

Reference throughput: healthy spinning USB drive 60–140 MB/s · SATA SSD over USB 3 350–450 MB/s · NVMe over 10 Gb/s USB 700–1000 MB/s. Anything sustained under ~20 MB/s on a modern drive means a USB-2 link or a sick drive.

Staging tree on the 8TB

STAGE="/Volumes/<8TB>/_STAGING"      # <-- substitute
mkdir -p "$STAGE"/{tm-extract,loose,photoslibs,manifests,quarantine}

tm-extract/

one folder per old machine pulled from a TM backup

loose/

one folder per source drive — keeps provenance

photoslibs/

whole .photoslibrary bundles, copied intact

manifests/

hash manifests and counts — this is your Phase 4 evidence

quarantine/

anything unclassified. Keep it; sort later.

Keeping source-drive provenance in the folder names is what makes dedup and verification tractable. Do not merge everything flat yet.

Finder route — try this first, lowest risk

1. Connect the drive; let it mount normally.
2. Hold Option and click the Time Machine icon in the menu bar.
3. Choose "Browse Other Backup Disks…"
4. Pick drive → machine → date.
5. Drag what you want out to $STAGE/tm-extract/<machine>/.

No Time Machine menu-bar icon? System Settings → Control Center → Time Machine → Show in Menu Bar, or open /System/Applications/Time Machine.app directly.

When this route works, use it. Slower to click through, but it restores metadata, permissions, and resource forks correctly and it cannot corrupt the source.

tmutil route — for bulk extraction

DRV="/Volumes/<DRIVE>"

tmutil listbackups -d "$DRV"
tmutil listbackups -d "$DRV" -t      # terse timestamps
tmutil latestbackup  -d "$DRV"

BK="/Volumes/<DRIVE>/Backups.backupdb/<Machine>/2021-06-14-093211"
ls -la "$BK"
ls -la "$BK"/*/Users/

# preferred: preserves ACLs, xattrs, Finder metadata
tmutil restore -v "$BK/Macintosh HD - Data/Users/<olduser>/Pictures" \
                  "$STAGE/tm-extract/<machine>/Pictures"

# fallback when tmutil restore refuses (common across machines/users)
ditto "$BK/Macintosh HD - Data/Users/<olduser>/Pictures" \
      "$STAGE/tm-extract/<machine>/Pictures"

Mounting a .sparsebundle by hand

hdiutil attach -readonly "/Volumes/<DRIVE>/<name>.sparsebundle"
# mounts under /Volumes/<BundleVolumeName>, usually "Time Machine Backups"

ls -la "/Volumes/Time Machine Backups/Backups.backupdb/"
tmutil listbackups -d "/Volumes/Time Machine Backups"

hdiutil detach "/Volumes/Time Machine Backups"

Use -readonly every time — it means a mistake inside the mounted backup cannot damage the backup. If detach says busy, close Finder windows and any Terminal whose working directory is inside it; -force is a last resort.

If the bundle refuses to attach, copy the whole .sparsebundle directory to the 8TB with ditto first, then run any repair attempt against the copy. Never against the original.

Encrypted backups — confirm the password early

diskutil info /dev/disk4s2 | grep -E "FileVault|Encrypted|Unlocked"
hdiutil attach -readonly "/Volumes/<DRIVE>/<name>.sparsebundle"   # will prompt
security find-generic-password -s "<sparsebundle name>" 2>&1 | head -5

Also check Keychain Access.app → search the drive or backup name → look for a Time Machine or disk-image password entry.

Photos Libraries — the rule that saves your photos

What you lose by copying files out of the bundle:

• You get derivative JPEGs instead of originals — lower quality, stripped of edit history.
• Albums, favorites, faces, keywords, captions, memories, shared-album membership all live in the database. All lost.
• Live Photos split into an orphan still and an orphan video.
• Edits are database records, so an "edited" photo copies out as the unedited original — or as a flattened derivative. You will not know which.
• Filenames under originals/ are UUID-derived. The result is unusable as an archive.

Method 1 — copy the whole bundle (archival)

ditto "/Volumes/<DRIVE>/Pictures/Photos Library.photoslibrary" \
      "$STAGE/photoslibs/<drive-N>-Photos Library.photoslibrary"

ditto treats the bundle as the opaque unit it is. The library stays openable in Photos.app — that is what makes it a real archive rather than a pile of JPEGs.

Method 2 — export originals (for merging and dedup)

1. Hold Option while launching Photos.app → Choose Library → pick the copy on the 8TB, not the original.
2. Cmd-A to select all.
3. File → Export → Export Unmodified Originals for N Photos…
4. Enable Export IPTC as XMP so keywords and captions survive as sidecars.
5. Export into $STAGE/loose/<drive-N>-photos-export/.

Use Export Unmodified Originals, not plain Export. Plain Export writes rendered JPEGs with edits baked in and metadata partially stripped.

Deduplication — by content hash, never by filename

cd "$STAGE"
find "$STAGE/tm-extract" "$STAGE/loose" -type f -size +1k \
  -not -name ".DS_Store" -not -path "*/.Spotlight-V100/*" -print0 \
  | xargs -0 shasum -a 256 > "$STAGE/manifests/stage-$(date +%Y%m%d).sha256"

wc -l "$STAGE/manifests/stage-$(date +%Y%m%d).sha256"

M="$STAGE/manifests/stage-$(date +%Y%m%d).sha256"

awk '{print $1}' "$M" | sort | uniq -d > "$STAGE/manifests/dupe-hashes.txt"
wc -l "$STAGE/manifests/dupe-hashes.txt"

# logical bytes across all copies — stat, NOT du
grep -F -f "$STAGE/manifests/dupe-hashes.txt" "$M" \
  | sed 's/^[a-f0-9]*  //' \
  | while IFS= read -r f; do stat -f '%z' "$f"; done \
  | awk '{s+=$1} END {printf "%.2f GiB across all copies\n", s/1024/1024/1024}'

sort "$M" > "$STAGE/manifests/stage-sorted.sha256"
awk 'NR==FNR{d[$1];next} ($1 in d)' \
    "$STAGE/manifests/dupe-hashes.txt" "$STAGE/manifests/stage-sorted.sha256" \
    > "$STAGE/manifests/dupe-groups.txt"
head -60 "$STAGE/manifests/dupe-groups.txt"

mkdir -p "$STAGE/_dupes-pending-delete"
mv "/path/to/redundant/copy.jpg" "$STAGE/_dupes-pending-delete/"

Each group of lines sharing a hash is a set of byte-identical files. Keep the copy with the most informative path — usually tm-extract/<machine>/…/Pictures/…, not quarantine/. Move, do not rm. Delete the holding folder only after Phase 4 passes and you have lived with the result for a week.

Immediately: copy the irreplaceable subset (photos + documents after dedup) to the 250GB, verify with Phase 4, and store it somewhere other than the desk the 8TB sits on. That is your "1 offsite" and it costs nothing.

Resolve the drive count. If Reading B is true, move M2's Time Machine to a spare 2TB and the single point of failure disappears entirely.

Cloud for the irreplaceable subset. The archive core is the part that cannot be rebuilt. Object storage for the photos+documents core is the only true offsite that survives a house fire. This is the one place where spending money is the right answer, and it is a small amount for the risk actually retired.

If Reading A holds and you can spend: a second 8TB as an archive mirror is the clean fix. Rotate it offsite.

Fleet-internal partial mitigation: Nano has 163 GiB free; Mini is unknown. Neither can hold the full archive. Do not pretend fleet copies substitute for a real second device — but a copy of the documents portion on Nano costs nothing and beats not having it.

Emergency — give M2 a working backup. Do this FIRST.

Precondition: 8TB exists, passed Phase 1, and Phase 2 says it is empty or holds only already-classified material.

Why first: every hour spent on consolidation before this is an hour of uninsured exposure on 337 GiB. If M2's SSD dies tonight, all of it goes.

# 1. confirm the identifier — only this drive connected
diskutil list external physical

# 2. create a dedicated TM volume inside the existing APFS container
diskutil apfs addVolume /dev/disk<N> APFS "M2-TimeMachine"

# 3. register it, APPENDING (-a) rather than replacing
sudo tmutil setdestination -a /Volumes/M2-TimeMachine
tmutil destinationinfo

# 4. first backup, blocking so you can watch it succeed
sudo tmutil startbackup --block --destination <destination-id>

# 5. prove it
tmutil latestbackup

If the 8TB already holds data you have not extracted, skip to Step 2 and come back. Do not erase it. If the 8TB is not APFS and must be converted, that is destructive — read the gate below.

Inventory and health, every drive  (Phases 0 + 1)

Precondition: none — read-only, always safe. Output: complete drive table, count resolved, trust levels assigned, Mini's status recorded. Resting state: you know what you have; nothing changed.

Classify  (Phase 2)

Order: process the most suspect drive first. A dying drive's data is the most time-sensitive thing you own; healthy drives will wait, failing ones will not. Resting state: you know what is on each drive; nothing changed.

Extract, drive by drive  (Phase 3)

Precondition: Steps 1–3 done; $STAGE has ≥ 2× the source drive's used space free. Per drive: mount read-only → extract → hash manifest → erase nothing.

Rollback: the source is untouched behind a read-only mount. Delete the staging folder and start over — zero risk to the source. Resting state: that drive's data exists in two places.

Verify each extraction  (Phase 4)

Output: GATE-drive-<N>.md reading PASS on every line. Rollback: re-extract; source is still intact. Resting state: extraction is proven — and this is the last point at which everything is still recoverable from the original drives. Past this line, erasures begin.

Dedup and consolidate  (Phase 3.6)

Action: build the single Archive tree; move duplicates to _dupes-pending-delete/ and do not empty it yet. Rollback: move files back out — fully reversible.

Second copy of the irreplaceable subset → 250GB

Action: copy photos + documents core to the 250GB, verify with Phase 4, take it to a different physical location. Why here: this is the step that makes the erasures in Step 8 safe. Do not reorder it after them.

Repurpose the old drives  FIRST DESTRUCTIVE STEP

1. Lowest-value drive first (the 120GB). If something about the process is wrong, you find out on the drive you care about least.
2. Then the 250GB — but only if Step 7 did not claim it. If it is now the irreplaceable mirror, it does not get erased at all.
3. Then the Crucial X8 → scratch. Unregister it from Time Machine before erasing, so macOS stops reporting a destination that no longer exists:

tmutil destinationinfo -X                  # find the destination ID for "Backup"
sudo tmutil removedestination <destination-id>
tmutil destinationinfo                     # confirm it is gone

A stale registration is exactly what makes M2 look backed up today while having no backup. Clear it.

Set up scratch and offload M2

Action: create the scratch layout, move ~/Downloads and AI_OS working sets, add TM exclusions. Rollback: move the data back — reversible. Target: M2 below 70% used (81% today).

Empty the holding folders

Precondition: everything above done, plus a full week of normal use with nothing found missing. Action: delete _dupes-pending-delete/; collapse _STAGING into the final Archive layout. Rollback: none after deletion — which is why it is last, and why it waits.