Apple Releases Depth Pro 2 for Metric Depth in Photos and Video
Apple released Depth Pro 2, 24M to 872M, for metric depth from photos, video or a live camera, with a 23.5M focal-length estimator.
Research-only weights.
Apple has posted Depth Pro 2 weights and code on Hugging Face and GitHub: four model sizes, from 24 million to 872 million parameters, and one network that estimates metric depth in meters for every pixel from a single photo, a video or a live camera feed. The license allows only non-commercial scientific research and academic development. The first related post on X appeared at 08:08 Beijing time on November 20. Apple's official accounts did not publish a matching announcement in the same window.
This is not a consumer API, and it is not a commercial SDK. The four depth models are S (24M, 97 MB), B (97M, 386 MB), L (334M, 1.3 GB) and the default H+ (872M, 3.5 GB). Larger sizes produce sharper, steadier depth and run more slowly. All four share a 23.5 million-parameter, 94 MB focal-length estimator, which predicts focal length when it is missing and is skipped when you already have one. The code runs on NVIDIA GPUs (CUDA), Apple silicon GPUs (MPS) or the CPU. The repository also ships Core ML exports for Apple hardware, plus a live camera demo that needs Apple silicon, macOS Sequoia 15 or later, and Xcode 16 or its command-line tools.
Apple's chart puts the new models on the low-jitter side
The model card's comparison plot measures sharpness with Boundary F1 and temporal consistency with jitter, averaged over the Sintel and Spring datasets. The caption says the top right is best, and it groups earlier methods as single-frame or multi-frame. All four Depth Pro 2 sizes sit on the lower-jitter side of the chart, with L and H+ highest in boundary F1. The original Depth Pro lands much farther toward high jitter. Those points come from Apple's own figure, not from an independent rerun.
The speed and memory numbers are also from the size-specific model cards, under a tightly specified setup: an NVIDIA B200, bfloat16, PyTorch eager mode with no compilation, a 32-frame clip processed in one call at native resolution. Throughput is average frames per second over the clip; memory is peak GPU memory reserved by PyTorch; the focal-length estimator is included. At VGA (640×480), S reaches 347.0 fps with a 1.9 GiB peak, and H+ reaches 95.9 fps with 8.3 GiB. At 720p the figures are 143.9 fps / 3.8 GiB and 20.2 fps / 16.3 GiB. At 1080p they are 44.8 fps / 8.0 GiB and 5.3 fps / 29.5 GiB. At 4K, S still does 3.6 fps but needs 31.9 GiB; H+ falls to 0.4 fps with a 114.6 GiB peak. B and L sit in between: at 720p, 82.5 fps / 6.6 GiB and 33.7 fps / 10.6 GiB.
The official demo notes add a second table, measured as median milliseconds per frame on a MacBook Pro with an M4 Max and 64 GB of memory, on the GPU, which is the app's default. The released packages use a temporal window of four frames. S stays at 30 fps or more up to 896×512; B is real-time only at 640×360. H+ takes 1,233 milliseconds per frame at 1280×720. The notes say preprocessing and rendering are excluded, and the machine was not otherwise idle.
The depth is metric along the camera's viewing axis, and scale follows focal length
The model card describes a multi-view transformer that processes one or more frames jointly, with a DPT decoder predicting an inverse depth map for each frame. After the first third of its blocks, the transformer alternates attention within each frame and attention across frames. The decoder assembles tokens from several blocks into feature maps at 1/32 to 1/4 of the input resolution. For video files, the default is 16-frame clips that overlap by four frames, to keep depth stable where clips meet. Live mode predicts each frame as it arrives, using that frame and the 15 before it. The repository warns that memory grows with frame size and window length, not with the length of the video. If a clip does not fit, shorten the window first, then switch to a smaller model, and resize the input only as a last resort: a much smaller input loses detail and can shift the scale of the depth.
The depth value is the distance along the camera's viewing direction, not the straight-line distance from the camera. Sky and other infinitely far regions are written as infinite depth; smaller models sometimes emit a finite depth of several kilometers, so the range should be clipped before a point cloud or a color preview. Scale is tied to focal length. The official notes say a focal length that is 10% too long places everything 10% too far. Photos often carry a 35mm-equivalent focal length in EXIF. A video gets one focal length for every frame, the median of the estimates over its first window. A live stream cannot see future frames, so it uses the estimate so far until that first window is full, then holds the value.
The license draws a hard line: the Apple Machine Learning Research Model license, for non-commercial scientific research and academic development only. What you get is a runnable research stack—a command line, a Python API and a Core ML demo—not a commercial grant to ship the weights in a product. If you need metric depth, check that your use fits the license, then pick a size between S's speed and the sharpness, and memory cost, of H+.