Tadeo Jones in Augmented Reality: Rig, Mocap and Retargeting

I was asked to bring Tadeo Jones into a news broadcast in augmented reality format, with a two-week deadline and no starting point at all: no rig, no capture, nothing built yet. Here's how the piece came together, from the bone structure to the final result on screen.

The rig, from scratch

There was no existing Tadeo Jones rig for Unreal, so I built the whole thing myself: body and face, on top of Unreal Engine 5.7's mannequin skeleton, using Control Rig. Starting from the mannequin meant I could rely on the engine's built-in animation and retargeting system instead of having to build that infrastructure from scratch too.

The facial rig was also built entirely from scratch, not just the blendshapes: the whole facial control system, starting from nothing, calibrated for the specific expressions this piece needed.

Retargeting and cleaning up the mocap

The motion capture came in on a Mocaptury skeleton, so the first step was retargeting it to the Tadeo Jones rig. From there I had to fix quite a bit: adjust the timing on several actions, improve poses, rework some actions entirely, exaggerate some movements and tone down others, add details the capture simply didn't have.

All of that work went through a layered control rig, with additive layers on top of the base animation. That way I can change posture, weight or timing without having to re-animate everything from scratch every time something isn't working.

From Unreal Engine to live AR

With the animation ready, I exported the result into Unreal Engine 5.3 to integrate it into the broadcast's augmented reality format, with the same attention to camera tracking and studio fit that any AR piece for broadcast demands.

The piece aired on Informativos Telecinco, with Carlos Franganillo interviewing Tadeo Jones. Mediaset España shared the result on their LinkedIn.

Two weeks, from zero to a real piece

None of this started from a half-built foundation. It started from zero: a character that didn't exist in Unreal Engine until that point, with two weeks to make it exist and work live.

That's what interests me most about this kind of work: building a complete rig, solving the retargeting of someone else's motion capture, and getting it all ready for broadcast, on a timeline that in most other contexts would barely cover testing.