Apr. 07, 2018
Art by Anthony Salvi
When done right, it is the most accurate way to digitally reproduce a physical material available today. For a long time, however, the only way to access it was with a high-end scanner, which many in the 3D community do not own.
Allegorithmic couldn’t buy everyone a scanner, but we could open up the technology to more people. And the exciting thing is the research paid off. In February 2017, we launched Substance Designer 6, which included a complete and easy scan processing workflow that finally allows anyone with a smartphone to scan physical materials.
In this tutorial, I’ll describe each step of the scanning process, from the photoshoot to the final Substance material, incorporating insights we’ve learned over the last few years.
I hope you enjoy this tutorial and have fun with your first smartphone-powered material scan!
To understand the image capture process, we have to imagine our material on a much larger scale. Think of your material at the scale of a mountain. At this size, the sun could represent our light. If the sun turns around the mountain, we can see the shape of the shadows cast in black. These shapes represent the indirect information about the relief of the mountain itself. If we combine enough information, at least four images (using eight for a better result), the algorithm can calculate the relief. So to achieve a proper scan, we just need to turn the light eight times at the same distance around our material.
It’s important to find the right balance between quality and cost. To keep our experiment cost-effective, let’s use some cardboard, a stack of tracing paper sheets and a LED light for our lighting setup. My preferred LED light is made by Manfrotto. It produces a well-balanced daylight. For the image capture, it’s important to attain the best process possible, so for our tutorial, we used an iPhone 7 and the Adobe Lightroom Mobile app. This app allows you to capture in the RAW (uncompressed) format on iOS and Android. Other apps like ProCam or Camerafv5 will work as well. Finally, we need a color chart from X-Rite. Of course, you can use a large range of smartphones for this tutorial – only the RAW capture capability is required.
Our ScanBox, meaning the interior area where the scan will occur, is designed to capture a large number of details from a material that measures up to 10cm x10 cm with opacity. We’ll also need a box to go around the interior, which in our case will be a 27.0cm x 22.0cm x 15.5cm cardboard box. It’s a good idea to cover the interior faces in white.
To retain the ability to capture the opacity of a material, we will cut a square hole (10cm x 10cm) at the center of the box and place a stack of 6 sheets of tracing paper on top of it to diffuse the light through the material. On top of the paper, we’ll add a Scanning Chart to help us during the capture and after for the post-process. This chart is black to reduce the lighting bounces from our LED light onto our material sample. We’ll also add some shapes (square, triangle, moon, star, etc.) at the eight angles, which are useful during post-processing. These shapes are used by the photomerge process in Adobe Photoshop to produce a fast and accurate merging of the scan components.
You can download our template right here.
The second accessory is a LightBox. For the scanning process, it’s useful to produce a soft diffuse light with a little gradient. The LightBox is designed specifically for our ScanBox. The concept is simple, and you can build your own softbox based on this one. The first goal is to produce a light at 45 degrees with the material. The second is to produce the biggest light size possible.
The key to obtaining the right amount of soft light is to find a balance between the size of your subject, the light and the distance between the two. All faces inside the LightBox should be white, while the outside faces should be black.
To build this DIY SoftBox, use four white foam boards (50cm x 50cm x 3mm), three black foam boards (50cm x 50cm x 5mm), two black paper sheets (42cm x 59.4cm) and a roll of Scotch tape. Cut the ScanBox on six foam board elements, plus one paper tracer sheet. On the back, cut a door to put the LED light inside. The diffuse part is created by a tracing paper sheet.
To keep this project cost-effective, use a simple cardboard tube with a foam core plate to create a stand for your smartphone. To maximize the final frame, calibrate the stand size to the size of the box. Next, add a black paper sheet to remove all potential color bounces coming from the cylinder. Finally, attach your smartphone with four pieces of regular Scotch tape.
Here is our final setup with our smartphone, stand, ScanBox and LightBox. Note: This setup is a recipe – you can create your own and enhance it however you want. You can switch the smartphone to a DSLR, or switch the LED light by using a wireless flash. It’s up to you and your budget.
In this tutorial, I’m going to focus on the capture process for a complex sportswear fabric from Tex-Ray, a famous material manufacturer. To do this, we need to make sure that the only light on our material is coming from the LightBox. If the shadow cast is wrong, you’ll have to deal with an erroneous computation during post-processing in Substance Designer. So, just close the windows or curtains to darken the room and eliminate as much ambient light as possible. Then turn on your LED light on and place it inside the LightBox. Next, place the fabric on the ScanBox. Be sure the material is not covering the little shapes in the Scanning Chart (square, triangle, moon, star…). The fabric needs to fit perfectly inside the 10cm x 10cm square.
Next, clean your sample, as dust or hair will often fall onto the material. The most important thing is to not move the material sample during the shoot. If the material is moved, the photo-merge cannot be done correctly. Once everything is set, it’s time to capture the material. Set the LED lighting power to maximum. Make sure you have enough battery power (smartphone and LED light) and that the LED light doesn’t become too hot.
During the shoot, it’s important to keep the same framing as much as possible. However, don’t panic if your pictures are not perfectly aligned. It’s more important to keep all of the shapes (square, triangle, moon, star etc.) present on the Scanning Chart visible in your images.
With Adobe Lightroom Mobile, you have two options. You can use the PRO mode and manually set up your shot or use the HDR mode. Both deliver good results. In the PRO mode, turn the flash off, activate the DNG format, set the ISO as low as possible (between 25 to 100) and set the White Balance to Daylight.
To avoid motion blur in your image, keep a minimum speed of around 1/40 sec and adjust your ISO to get a good exposure. It can be helpful to add the grid and the level on screen for framing the shot. You can also add a timer for 5 seconds to act as a remote trigger for capturing the image. This will provide a more stable result as manually touching the shutter button on the camera can inadvertently add a shake, which may produce a blurrier image.
If you have an Adobe Creative Cloud account you can import your images into your Adobe Lightroom Desktop library using your Wi-Fi or 4G connection.
During the capture, it’s always a good idea to neutralize the color shift coming from the lighting. All lights naturally have a color tint, with some tints stronger than others. For example, a candle is red, a tungsten bulb is orange, the sky is blue. To neutralize this color and keep color consistency, a ColorChecker is required. Basically, it’s a reference, with a grayscale printed and calibrated. In post-production, we’ll adjust the captured gray color with Lightroom’s White Balance Tool. In this example we have only one light source, so one picture with the gray card is required. You can find a gray card easily in a photo supply store, or on the web. X-Rite, DataColor, QP Card or WhiBal all produce some great White Balance targets.
The next step is to take eight pictures, with the LightBox around the ScanBox (counterclockwise). We start with the SoftBox in the number 1 side of the Scanning Chart. The smartphone is in front of the number 7.
We then take a picture and check the exposure and sharpness. If everything is OK, we move the LightBox to the number 2 of the Scanning Chart. We repeat this process up to the fifth picture. Now we invert our position with the smartphone in front of the number 3.
Now that we’ve completed this process, this step is finished.
For opacity, open the curtains or turn the room lights on, and set up the LED light inside the ScanBox. With the LED light at a lower power, you’ll have enough backlight to capture the opacity. For a better light diffusion, you can also add white paper on the inside, with the light centered in the middle of the ScanBox. A foam core with a hole works well.
Now we have our eight angles, opacity and ColorChecker images.
Next, we import our images into the desktop version of Lightroom. Once loaded, select the ColorChecker image for the LED light and use the White Balance tool. Just click on the middle gray example.
Next, set the Whites, Blacks and Clarity parameters at 0. Then copy and paste the parameter value to the eight angle images.
For the opacity image, use the gradient tool in Lightroom to reduce the vignette effect. For the last step, export all of the images as TIFF 16-bit using the Adobe 1998 ICC profile.
Using the photomerge feature in Adobe Photoshop, it’s possible to merge and align the scan images. Just load your nine pictures in photomerge and uncheck the “Blend Images Together” option.
After that, add a solid black layer at the bottom of the layer stack and crop your image on a square, resizing it to 4096 pixels. Finally, export your layer stack in a new individual TIFF 16-bit file with the “Remove Layers” option. If you set up a stronger stand with additional accessories, this photomerge is not necessary, just crop your images in Lightroom in 4K.
It’s time to convert our images into a PBR material. Open Substance Designer (a 30-day trial of the latest version is available here) and start by creating a new Substance file using the Physically Based (Metallic/Roughness) template set to 4096 x 4096. Right-click on the Substance name, then select Link menu\Bitmap to link the nine pictures to our project.
After a drag-and-drop from our resources images in the graph node window, you can add our first scanning node: the Multi Crop node. To add it, just hit the spacebar in the graph view and type “Multi Crop”. Selecting a specific image area to target is very useful, especially when tiling is difficult. With the procedural approach, you can easily test different crops for a better tiling. In our case, just target the center of the fabric and make sure you have enough space to cover at least a complete pattern. Set the Input Count to 8 and the Input Size to 4096 on X and Y. To view and edit the crop, double-click on the last node output named Area on a flyover mouse.
In the same way, add the Multi-Angle to Normal node. Set the Normal Format to DirectX (if needed), Sample Amount to 8, Intensity to 1, First Sample Light Angle to 0.5/180 and the Next Sample Light Angle to Counterclockwise. Then connect the first eight outputs from the Multi Crop to the Multi-Angle to Normal node.
To remove the slightly normal variation, add a Color Equalizer node after your Multi-Angle to Normal node. The soft lighting with this node produces a soft normal as well. To push it, add a Normal Combine node with the two inputs coming from the Color Equalizer output. Then set the node up with the Technique Channel Mixer (High Quality). Notice that we have a soft light on our material – in case of stronger lighting, it’s recommended to add a first Color Equalizer node on each image between the Multi Crop node and the Multi-Angle to Normal node.
Once the Normal is set, we can work on the color map. For the Base Color, we have a dedicated node named Multi-Angle to Albedo. Add it in the graph from the Library or with the spacebar shortcut.
For this specific fabric, there are only two samples – the angle number 1 and 5. If you want to use fewer than eight samples, just always select the opposite angles. For example the 1 and 5, or the 3 and 7.
Make sure to add a Color Equalizer on the Normal to fix the little color variations. Finally, add a Level and a Hue Saturation Lightness node to slightly adjust the color if necessary.
We have a specific image for this opacity map. After a drag-and-drop from our resources, copy-paste the Multi Crop node and reduce the Input Count to 1. In doing this, you’ll sync with the other maps.
After this node, add a Grayscale Conversion node, then a Histogram Scan node to push the contrast and produce a clean mask.
Next, put a Blur HQ Grayscale node to add a little softness on the mask. To control the amount of opacity, use this mask in a Blend node with two Uniform Color nodes as the inputs. All of these nodes are quickly available through the spacebar shortcut.
The Smart Auto Tile is a new node that can be found via Library\Material Filters\Scan Processing. You can plug the Color map, Normal map, and Opacity map into the three available inputs. One trick is to use the Height input for the Opacity map.
As with the Multi Crop node, the last output is the pattern viewer/editor. This node is very powerful: you can adjust the X and Y transform, and you have an edge detection with a blur. For better control when you edit your tiling, press spacebar in the 2D view to see the tile repetition.
One last map that’s missing is the Height or Displacement map. By adding a Normal To Height HQ node, we can convert our Normal map to a Height map. Connect the Normal output from the Smart Auto Tile node to this Normal To Height HQ node. Since we set up the Multi-Angle To Normal on DirectX format, let’s do the same for the Normal to Height HQ node. You can play with the Relief Balance and Height Intensity to find the right balance. Then turn the Quality to High and it’s done.
In the Disney PBR shader, we’ll set the parameters for our materials, choosing which will be dielectric and which will be metallic. With this information, the shader can apply a different behavior. For example, a metallic material will color its specular response with the Base Color and no diffuse response. Conversely, a dielectric specular color will stay white and the Base Color will be used in the diffuse response. Our material is a fabric without any metallic parts. So, in this case, the Metallic map is simply a Uniform Color node turned to Grayscale mode and set to 0.
The fabric is composed of the same base material. This textile has no aging either. In this case, we can suppose the Roughness is uniform. As with the metallic, you will also use a uniform color set to Grayscale, but here, the value depends on what you observe when you manipulate the fabric under the light. I set the value at 70.
Like before, press spacebar in the graph view and write “Output”. Then set one with the Opacity Usage. The second will be used for the Specular Level. With the Metallic/Roughness definition, when the Metallic is set to 0, the material is understood to be a dielectric and the reflectance value at the Fresnel zero angle or f0 is set to 4% reflective. This works for most common dielectric materials, but some dielectrics can have a different Index of Refraction or IOR. The Specular Level can be used to override the default 4% value used in the metallic/roughness definition. For this material, a Specular Level at 80 was perfect. To update the 3D rendering with the two new outputs, right click on the graph view, then click on the View outputs in the 3D view option.
Our material is big, encapsulating nine 4K, 16-bit images. This is a huge amount of data if you want to export your Substance material, use it in another 3D package, or even share it with your team members as a downloadable file.
One solution is simply to export the Base Color, Normal and Opacity maps. It can be done with a right-click on the graph name\Exports outputs as bitmaps in the Explorer window.
Then we create a new Substance material with only these three maps as inputs to keep the file size well balanced. We can keep the original Substance material as a working space, and then use the second for sharing and runtime usage. This new Substance .sbsar file will be four times smaller than the original.
A scanned material is nice to use, but sometimes you want to go further. It might be nice to change the color, or add a pattern on top of the material, for instance. This is where hybrid materials come in. In the next steps, we’ll learn how to customize our scanned material using the hybrid technology within Substance Designer.
One of the best nodes for the hybridization process is the Color Match node. You can find this node in the Library, under Filters\Adjustments. This node allows us to replace a source color with a target color. With a lot of controls for the color variation and the mask creation, it’s a very efficient node.
As you can see in this example, you can replace the yellow with a nice blue color. This operation is clean without loss of any fine details. Because the color details are a key element of an accurate material, it’s important to keep them in the Base Color texture. If needed, we can do the same on the green color for the second fabric component.
With the help of the Color Match node, the fabric now has two colors. Let’s combine them with a procedural pattern.
In Library\Generators\Patterns you have a lot of basic components. In this example, we used the Polygon 1 node. Just drag and drop the node from the Library to the graph view to start. This node is interesting due to its special Explode parameter. You can play around for hours creating hundreds of variations with an artistic look. For this tutorial, we have set up this node to create a double triangle pattern around a circle.
A good material is often a material that’s close to reality. In our case, that means close to the industrial process. We can imagine our pattern like a printed color on top of a fabric.
To create that effect, use the Normal map to create some variations on the pattern. Under the Library\Filters\Effects we find the Vector Warp node for Grayscale input. Then plug in the node, the pattern, and the Normal map. Set the intensity and put the Vector Format in DirectX.
Next, add a Blur HQ Grayscale for a soft touch.
A procedural approach fits well with the blending concept. Indeed any pattern changes made during iterations will be reflected in the final mask. This can be a huge timesaver in your work.
To simulate a painted pattern on top of our fabric, we have to use our pattern as a mask. With a bunch of Blend nodes, we can manage the Base Color, Roughness and Specular Level mix. The paint in this example shows a Roughness at 35 and a Specular Level at 128. The color comes from our Color Match node, allowing us to keep all the subtle details we need.
Substance is incredible for controlling each aspect of a material. One of these aspects is the Normal Intensity. We have the ability to fine-tune our Normal and Height intensity and adjust their strength by adding a simple parameter.
To do this, we use a Normal Blend node and a Normal Color node. The Normal Blend node is set with an Opacity at 1 and Use Mask at “True”. The Normal Color node is set at 0. To control the blending, a Uniform Color node set in Grayscale is required.
Be careful – all the Normal nodes in our graph need to be set on a 16-bit color depth. Now, the Uniform Color node can be set from 0 to 255. These values correspond to a range from No-Normal to Normal at Maximum.
To finish our work and create a hybrid Substance material, we have to expose some of the parameters. In doing this, you keep the ability to modify the material without any access to the node graph. You can expose the Target Color in the Color Match node. To expose a node’s parameter, click on the sine graph icon at the right of the parameter name and choose Expose. Next, the parameter will be available directly on the Substance Graph Input Parameters. In this example, the color will be tweakable outside Substance Designer and leave you free to choose other color variations.
Repeat this process for every parameter, including the Roughness, the Base Color Blending, etc. Finally, by right-clicking on the Substance material in the Explorer we can publish our Substance graph as an .sbsar. This file format will embed the maps and the node graph in a compact file. This .sbsar will also be readable by Substance Painter or any compatible 3D package like Unity, Unreal Engine 4, 3ds Max, Cinema 4D, Houdini and more.
It’s now time to use your material in a project! Here is an example of the material applied to a running shoe. Rendered in Substance Designer with Iray, this .sbsar file can be loaded by 3ds Max, Maya, Modo, Cinema 4D, Adobe Fuse, iClone, Houdini, Unreal Engine 4, Unity, and Lumberyard.
If your application is not in this list, you can always export your texture set (BaseColor, Normal, Height, Roughness, Metallic, Opacity, Specular Level, etc) from any Substance tool and use it in your shader. Other materials in this image come from Substance 3D Assets.
Your pattern needs to fit at least in an 8cm x 8cm square to accommodate cropping and tiling. If the fabric is available without any printed pattern, it’s possible to scan it and add the pattern afterward in Substance Designer. Also, avoid all fully black or metallic materials.
Anthony Salvi works at Allegorithmic as a Creative Technologist. He is always looking for new technologies, new devices, and new workflows for 3D artists, trying to find new and disruptive ways to create. He has previously worked for IKEA’s VR Experience and other VR projects.
This article was initially published in 3D World Magazine. It is an updated version of last year’s “Your Smartphone is a Material Scanner”, featuring a new lightbox and scanning of a sports shoe fabric.