Shevaan Jayasinghe
← More work

SpatialLM · Apple GPU engineering

The model had learned.
The hardware was the hurdle.

SpatialLM can turn a scan into a room layout: walls, openings and furniture. But one operation tied it to NVIDIA hardware. I rebuilt that part so the existing model could run on a Mac.

Problem
One sparse-convolution operation tied SpatialLM to NVIDIA GPUs.
Role
I rebuilt that operation in PyTorch for Apple’s GPU and kept the authors’ trained weights.
Result
The model runs on a Mac laptop without retraining, checked against the reference on ten scenes.
The work underneath the model

Same understanding.
Different hardware.

TRAINED MODELOriginal weights, unchangedBEFORENVIDIA-only sparse operationREBUILT FOR APPLE’S GPUSparse convolution in PyTorchAPPLE GPURuns locally on a MacOUTPUTChecked against reference scenesTRAINED MODELOriginal weights, unchangedBEFORENVIDIA-only sparse operationREBUILT FOR APPLE’S GPUSparse convolution in PyTorchAPPLE GPURuns locally on a MacOUTPUTChecked against reference scenes
Architecture illustration · Original trained weights, a rebuilt operation, local execution.
01

The intervention

Change the part that is in the way.

SpatialLM reads a scan as sparse 3D data, and the operation that processed it depended on a library built for NVIDIA GPUs. I rebuilt that sparse-convolution operation in PyTorch for Apple’s GPU. The authors’ trained weights stayed as they were, so the model did not need retraining.

02

The check

Then look at what comes out.

I checked its predicted layouts against the reference for ten scenes; the table below sets the results beside the authors’ published baseline. The model now runs on a Mac laptop.

For a closer look at the engineering
Category results

F1 at 0.25 IoU: the port covers 10 scenes; the authors’ published baseline covers 107 scenes. Different scene sets mean this comparison doesn’t establish an accuracy advantage for either GPU implementation.

Port (10 scenes) beside the published baseline (107 scenes)
CategoryChartPub.Port
bed · 8
0.9521.000
nightstand · 7
0.6700.810
wall · 10
0.6820.765
window · 10
0.5140.640
door · 10
0.4740.571
curtain · 10
0.3700.550
wardrobe · 5
0.3960.333
sofa small sample · 2
0.6911.000
chair small sample · 3
0.3230.556
chandelier small sample · 1
0.3680.500
cabinet small sample · 3
0.1120.222

Pub. is the authors’ published result; Port is the Apple GPU result. The count beside each category is the number of port scenes containing it. Categories marked “small sample” appear in three or fewer scenes.

Live products

Open what is already shipped.

Open the live demo

Chappie is on the App Store. The AI Bubble Index is live on this site. This page is a record of earlier work.