Taking Mac App Screenshots In A VM
Jake MarshIn this post
Taking screenshots of a Mac app means opening its windows, clicking through its menus and flipping the system between light and dark mode. Do that on the Mac you’re working on and windows keep popping up in front of whatever you’re doing. Multimeter’s older capture script ran on my own Mac, and it had to back up my real preferences file and Light/Dark setting and put them back when it finished.
Now the screenshots for Postage and Multimeter get taken in a headless macOS VM. The app gets built on my Mac, copied into the VM, launched and clicked over ssh, and the PNG comes back over scp. Nothing opens on my screen. Here’s how it’s set up, including the permission step that took the longest to figure out.
A Clean VM and a Throwaway Clone#
The VM runs under Tart, which wraps Apple’s Virtualization framework. A macOS VM takes about 35 GB, so everything lives on an external drive. Tart reads TART_HOME to decide where VMs go:
export TART_HOME=/Volumes/xwing/tart/home
tart listTrimmed to the columns that matter:
Source Name Disk Size State
local review-clean 50 GB 35 GB stopped
local review-vm 50 GB 35 GB stopped
OCI ghcr.io/cirruslabs/macos-golden-gate-vanilla:latest 50 GB 35 GB stoppedreview-vm came from Cirrus Labs’ vanilla macOS 27 image. I set it up once (more on that below) and cloned it to review-clean, which never gets booted. Every session starts with a fresh clone of it:
tart clone review-clean review-run
nohup tart run review-run --no-graphics > run.log 2>&1 &
IP=$(tart ip review-run)tart clone uses APFS clones, so the copy is instant and only the blocks the session changes take new space. --no-graphics means no window on the host at all. When the session is over, tart stop and tart delete throw the clone away, and the next one starts from exactly the same state: same wallpaper, same settings, no leftover app data.
All of this is wrapped in a small vm.sh with up, ssh, put, get, shot and down. Since up deletes and recreates review-run, only one script can own the VM at a time.
Getting the Display to 2880x1800#
I wanted every capture to be a 1440x900 point Retina screen, 2880x1800 pixels. Tart has a setting for it:
tart set review-vm --display 1440x900ptThe guest still booted at 1024x768 points. So there’s a tiny Swift tool in the guest that sets the mode from inside macOS. Here’s the part that sets it (the full version also lists the modes when you run it with no arguments):
import CoreGraphics
import Foundation
let display = CGMainDisplayID()
let opts = [kCGDisplayShowDuplicateLowResolutionModes: kCFBooleanTrue] as CFDictionary
let modes = (CGDisplayCopyAllDisplayModes(display, opts) as? [CGDisplayMode]) ?? []
let w = Int(CommandLine.arguments[1])!, h = Int(CommandLine.arguments[2])!
let candidates = modes.filter { $0.width == w && $0.height == h }.sorted { $0.pixelWidth > $1.pixelWidth }
guard let mode = candidates.first else { print("no \(w)x\(h) mode"); exit(1) }
var config: CGDisplayConfigRef?
CGBeginDisplayConfiguration(&config)
CGConfigureDisplayWithDisplayMode(config, display, mode, nil)
CGCompleteDisplayConfiguration(config, .permanently)kCGDisplayShowDuplicateLowResolutionModes is what makes the HiDPI modes show up in the list. Sorting by pixel width picks the 2x version of 1440x900 over the 1x one. vm.sh up runs displaymode 1440 900 after every boot, because a fresh boot can come up in a different mode.
Permissions for Everything Started Over ssh#
Taking a screenshot, clicking and telling System Events to switch to dark mode all need privacy permissions, and over ssh the process macOS asks about is /usr/libexec/sshd-keygen-wrapper. Every command you run over ssh counts as that one process.
You can’t script those grants. The system TCC database is protected by System Integrity Protection, so it’s read-only even as root. I made them by hand once in review-vm, in System Settings over Screen Sharing:
- Device Control and Data Access (macOS 27’s name for the Accessibility pane): on for
sshd-keygen-wrapper. - Screen & System Audio Recording: added with the + button, then Allow on the one-time prompt about bypassing the private window picker.
- Automation, for System Events: Allow.
The Allow buttons ignored synthetic clicks, from cliclick and from VNC clicks alike. Pressing Tab and then Space over VNC worked.
While I was in there I copied cliclick into /usr/local/bin, turned off Spotlight indexing (mdutil -a -i off) and automatic update checks, hid the desktop widgets, turned off Dock recents and dismissed the “What’s new in macOS 27” banner. Then I cloned the VM to review-clean. Every clone since has all of it.
Getting the App In#
Build on the host, then copy it over. Multimeter’s script zips the app with ditto, copies the zip over, unpacks it in the guest with ditto again and clears its extended attributes:
ditto -c -k --keepParent build/Multimeter.app /tmp/mm.zip
scp -i "$KEY" /tmp/mm.zip admin@$IP:/tmp/mm.zip
ssh -i "$KEY" admin@$IP 'mkdir -p /tmp/mm && ditto -x -k /tmp/mm.zip /tmp/mm && xattr -cr /tmp/mm/Multimeter.app'Launch it with open. A binary you run straight from ssh isn’t connected to the window server, so it never gets a window. open starts the app in the logged-in GUI session:
ssh -i "$KEY" admin@$IP "open -n /tmp/mm/Multimeter.app --args -ApplePersistenceIgnoreState YES"Everything after --args goes to the app, which makes launch arguments the easiest way to stage a scene. Postage has a demo mode with sample mail that’s turned on by a launch argument, and its sidebar, avatars and list density can all be set the same way. Multimeter’s script copies a preferences file in before launch and sets the defaults key that hides its welcome banner, so it opens with exactly the meters the screenshot needs.
Driving It#
Clicks go through cliclick over ssh. Its coordinates are points, 0 to 1440 across, and screenshots are in pixels, so halve whatever you measure on a capture:
ssh -i "$KEY" admin@$IP '/usr/local/bin/cliclick -w 150 c:852,12'cliclick can’t scroll, so there’s a second small Swift tool that moves the pointer with CGWarpMouseCursorPosition and posts line-based CGEvent scroll-wheel events. Those behave like a mouse wheel, without elastic overscroll or momentum. To check Postage’s pull to refresh I needed trackpad-style events that carry a gesture phase, so that got its own tool too.
Light and dark mode go through System Events, which the Automation grant above allows:
ssh -i "$KEY" admin@$IP "osascript -e 'tell application \"System Events\" to tell appearance preferences to set dark mode to true'"Set it before you launch the app so it starts in the right appearance.
Capturing at the Right Size#
The full-screen capture runs inside the guest:
ssh -i "$KEY" admin@$IP 'caffeinate -u -t 2; screencapture -x -D 1 /tmp/shot.png'
scp -i "$KEY" admin@$IP:/tmp/shot.png shot.pngcaffeinate -u declares user activity, which wakes the display first. -x turns off the shutter sound and -D 1 picks the main display. You get 2880x1800 every time.
For a single window, screencapture -l <window id> -o captures just that window without its shadow. Postage’s demo mode writes its window number to a small ready file once the window is up, so the script waits for that file, reads the id and captures it. A 1280x800 point window comes back 2560 pixels wide. If the window capture fails, the script asks CoreGraphics for the window’s bounds with CGWindowListCopyWindowInfo, takes a full-screen shot and crops it, doubling every number because the bounds are in points.
Two checks run after each capture. The script compares every variant’s size with the first one, because the website crossfades between them and a size change would make it jump. And it measures each image’s mean brightness with ImageMagick to make sure dark mode applied when it was supposed to and not when it wasn’t.
Video works the same way. screencapture -v -k records the screen and draws a ring at each click. The files have a variable frame rate, since frames are only written when the screen changes, so convert them to a constant rate before editing.
Things That Bit Us#
- Tart’s
--vnc-experimentalcrashes the VM. It uses the Virtualization framework’s own VNC server, and the whole VM went down withEXC_BREAKPOINTwhen a client reconnected (Tart 2.41.1). For the rare time I need to see the screen live, I connect to the guest’s own Screen Sharing instead.screencaptureover ssh is the main path, and it kept working at times when Screen Sharing sent no frames. - A fresh clone is busy for a while. For 20 minutes or more after boot, background services like
dasdand the XProtect remediators kept the CPU pegged.sudo killall dasdcalmed it down. Wait it out before recording anything with timing in it. - ssh can time out before the command runs. When the host is busy, the guest’s sshd can time out during the banner exchange, and ssh exits with 255 before your command ever starts.
vm.shretries only exit code 255, up to eight times, so a real failure from the command still fails. The video scripts keep one connection open withControlMasterandControlPersistinstead of handshaking for every command. - The VM has no battery or fans. Anything that reads hardware sensors shows nothing. Multimeter’s fan and power meters show a dash in the VM, and the battery shots use the app’s own Simulate Battery option.
screencapture -vcan stop early. Some takes ended after 11 to 43 seconds of a longer recording with nothing in the logs. Keep takes short and check every clip’s duration.
The payoff is that screenshots are a script now. The Postage layout shots for its website are one command that builds the app, captures four variants in the VM and writes the web copies, and I can run it while I keep working on something else.