Mapping The Gray Man

During the pre-production data collection phase of The Gray Man, the 2022 Netflix hit starring Ryan Gosling and Chris Evans, VFX studio Scanline reached out to Mosaic for assistance in data capture of the streets of Prague. Mosaic is a leader in 360° hardware and geospatial solutions and their abilities to capture and create large scale 3D models and reconstructions with their line of 360° mobile mapping cameras caught the eye of Bryan Grill, a visual effects supervisor with Scanline.

Grill is a well-known and respected VFX supervisor with over 25 years in the industry, and has been nominated for three Academy Awards and one BAFTA award. He is best known for his recent work in such movies as Free Guy with Ryan Reynolds, as well as several movies from the Marvel cinematic universe.

Having seen some of the work achieved by the Mosaic team with their state-of-the-art 360° mobile mapping cameras, Grill reached out to Mosaic CEO Jeffrey Martin to inquire about the possibility of using this cutting-edge technology to assist in capturing street-level data for The Gray Man.

“I had seen some of Jeffrey’s work online and immediately saw the potential for what he and his company is doing with what we are dealing with in the VFX world – mainly the difficulties of capturing large environments and large data sets, easily, quickly and at a reduced cost,” says Grill.

The initial goal

Grill explains that he was looking for a better and more efficient means of capturing the required data to create a quick representation of the geometry and have photography that lines up with that ‘geo’

MEGASCANS – LARGE SCALE SCANS THAT CAN BE USED ACROSS A VARIETY OF PLATFORMS – ARE INCREASINGLY IN DEMAND

Previously Grill and Scanline (and presumably other VFX studios) have used a workflow which was labor and time intensive. When covering such large areas their usual technique of using LiDAR plus photography to create geometry and hand match the pictures and geometry added considerable time, labor and therefore costs.

With Mosaic’s mobile mapping camera rig, the Mosaic team was able to get thorough coverage of large areas much quicker than previously done. It also captured in a relatively short amount of time, limiting lighting changes in the scene from the sun.

Additionally, following filming, Scanline usually gets the plate shots from the client and creates a 3D track from these. Plate shots (or ‘plates) are images of the background, without any actors or other subjects in view. These are then typically used for the purpose of creating a composite (or plate composite) in post-production.

Again, this process is very labour/time intensive, and with the timeline and budget, Scanline needed a faster, easier and more cost-effective method.

But due to the 360-degree environment, they were able to line it up to where they wanted it on the first frame and set the keyframe, let the camera go, and adjust the position (tilt, pan) and set another keyframe.

This process along with adding 2D tracking allowed Scanline to create over 400 temp comps which helped the filmmakers design their final edit.

The future

“I do think this will be very useful to many studios very soon, and it will be the next big tech for VFX and gaming… It opens a lot of possibilities for innovative solutions in VFX (and similar industries) in the very near future,” says Grill.

With the latest trends of creating Digital Twins, Smart Cities, Reality Capture, Unreal Engine, the Metaverse and the like, Megascans – large scale scans that can be used across a variety of platforms – are increasingly in demand. In order to feed these platforms, a new system of collecting data and processing it is necessary.

Mosaic seems to be on the front line of this new trend and is setting themselves up to lead the way with the cameras, computer vision expertise and team of engineers and software developers.

Martin says: “With the power of RealityCapture and UE5, and the highest caliber data collected by our mobile mapping cameras, it’s possible to offer the world’s fastest, easiest, and best large-scale 3d asset creation system, including TV/film production, VFX, video games, surveying, BIM, asset inspection, telecommunications, and infrastructure inspection.“

Mosaic continues to see an elevated interest from the VFX and gaming industry regarding the use of high-resolution imagery captured with a 360º mobile mapping camera, plus their unique post-processing capabilities for creating photorealistic 3D models without the use of LiDAR.

After a recent visit to a virtual studio EVOKE in Prague, Martin remarked, “I have seen the future, and we’re a part of it!” His words refer to the fact that in the future, virtual production will likely be conducted without having to travel to locations. Rather, artists and studios will rely heavily on digital twins created and displayed on large LED screens in a virtual studio.

The quality and quantity of visual data necessary for such uses could potentially be very costly and labor-intensive to create and process. But Martin is confident that the time and costs involved will significantly decrease.

I DO THINK THIS WILL BE VERY USEFUL TO MANY STUDIOS VERY SOON

New projects

The Mosaic team is currently working with another VFX studio in the U. to capture data for an upcoming film. Much like Scanline, this New York-based VFX studio reached out to Mosaic with help in collecting the data along several miles of highways and streets in Boston needed to later create realistic models of the scenes. This data will serve as reference material for creating background plates in post-production. The studio sees this as a more time and cost-efficient manner of data collection than previous tools and methods.

The Mosaic team has also been in talks with several e-sport companies with help in the data and processing of 3D models to be ingested into game engines, similar to Unreal Engine 5 or Unity. Driving the streets of a city at normal driving speeds to recreate an entire racing simulation track is enough to generate the necessary environments and give game developers more variety in their game tracks and enrich the environments without having to source more 3D artists