Rembert Daems, PhD, presented our work on image fusion at the AI4M3D workshop of the European Conference on Computer Vision (ECCV) in Malmö.
The problem
A knee is imaged in two ways. A CT scan shows the bones in 3D. A plain X-ray is a flat 2D picture: quick, low-dose and routine, taken with the leg in whatever position it is in at the time. Fusing the two means working out exactly where each bone in the X-ray sits relative to the CT, from one image that has no depth. Get that right and a routine X-ray becomes a 3D measurement of the joint, without the extra dose of a repeat CT.
The paper
The work is published as Learning Dense 2D-3D Correspondence for X-ray-to-CT Registration of Knee Bones, by Rembert Daems, Jonas Grammens, Caro Roten, Andrew Meyer, Thomas Luyckx and Matthias Verstraete. It learns to match each pixel of a single X-ray to the patient's CT, and the position of the bone then follows in one step, with no starting guess and no iterative refinement. One model, trained on 758 patients, works on patients it has never seen.
Why present it at a computer vision conference
Fusing that imaging is a computer vision problem before it is a clinical one, and ECCV is full of people who will find the weak points. Putting the method in front of them is how it gets tested outside our own team, which matters when the output ends up in a surgical plan.
Knee ID is under development. It is not CE marked, has not been cleared by the FDA, and is not available for sale.