Unity Photogrammetry De-lighting Workflow

Back to overview

Update: Thanks to Cyril from Unity for pointing out errors in this tutorial, I have updated the text where possible but I recommend you read the tips he suggest in his comment below.

This tutorial will take you through the stages to set up a workflow for de-lighting a 3D scanned mesh using xNormal and Unity 3D‘s de-lighting tool and then uploading the de-lit scan directly to Sketchfab from Unity using the Sketchfab Unity Exporter. Here are some example outputs:

https://sketchfab.com/calvertcreates/collections/unity-delighting-tool

Jump to section:

Creating the Required Maps for Delighting with xNormal

The unity delighting tool requires certain texture maps to function, specifically:

  • Base Colour Map
  • Ambient Occlusion Map
  • Object Space Normal Map
  • Bent Normal Map

Most photogrammetry software (PhotoScan, RealityCapture, ReCap Photo, 3DF Zephyr etc.) will allow you to export a base colour map and possibly other maps in this list too. For the purposes of this tutorial we are going to assume that you only have a base colour map and use xNormal to create the other maps.

This workflow will also assume that you have one hi-res mesh and one low-poly, UV unwrapped mesh of the same models to work with – we’re essentially trying to go from a hi-poly scan export to something more game / VR / AR ready. Note that xNormal only accepts meshes that are 100% quads or triangles.

  1. Download, install and open xNormal.
  2. Select the High Definition Meshes panel, drag’n’drop your hi-res mesh onto the window to load it in.
  3. Right-click on table cell under Base Texture to Bake, select Base Texture to Bake from the menu that appears. Navigate to the base colour texture for your hi-res mesh and load it in.
  4. Select the Low Definition Meshes panel, drag n drop your low-poly mesh onto window to load it in.
  5. Select the Baking Options panel.

    • Under Output File, set you the folder you wish to save your texture maps to and add a suitable file name. xNormal will add the relevant suffix (i.e. _normals) to each exported map. Choose the image type you cant to save as too (.tga, .png, .jpg etc.).
    • Choose your desired texture size, we’ve selected 4k (4096 x 4096) here.
    • We’ll leave the rest of the setting at the top as default.
    • In the Maps to Render section check the tick boxes for Normal Map, Bake base texture, Ambient Occlusion and Bent normal map.
    • Update: We need to make one tweak to xNormal’s default normal map calculator. Click the green square with three dots in it next to ‘Normal map’. In the dialogue that opens up, un-check ‘Tangent space’ and then close.

  6. Click the big round Generate Maps button to begin processing your maps, you should see something like this as it process:
  7. Once xNormal is finished processing (this can take several minutes), navigate to the folder that you saved your maps to. We need to add some special suffixes at the end of our texture map filenames to get them to work (as per page 8 of Unity’s own documentation):
    • mymodelname_baseTexBaked.jpg  >  mymodelname_baseTexBaked_DLBC.jpg
    • mymodelname_occlusion.jpg  >  mymodelname_occlusion_DLAO.jpg
    • mymodelname_normals.jpg  >  mymodelname_normals_DLN.jpg
    • mymodelname_bent_normals.jpg  >  mymodelname_baseTexBaked_DLBN.jpg
  8. For now, put all the generated and renamed texture maps and the low-poly mesh in a folder of their own:

Unity’s Delighting Tool

Now that we’ve done the necessary prep for our textures we can move on to the main event – de-lighting the colour texture in Unity.

  1. Download and install Unity.
  2.  Download and unzip the DeLightingTool package from GitHub.
    Update: You can now get the Delighting Tool directly from the Asset Store here.
  3. Start up Unity, select Open from the startup dialogue and point it to the DelightingTool-master project folder that you just unzipped (the one with the Assets, Documentation, Library etc. folders inside).
  4. Don’t worry if you see a warning about the project being created in a previous version of Unity, just click continue.
  5. You should see the layout below. If not, select Layout > Default in the drop down in the top right.
  6. Next, drag the folder containing your low poly mesh and rename textures into the Asset folder:
  7. From the Window menu in the top, select Delighting Tool
  8. The Delighting window might open fullscreen, move it to the side a bit and then drag your imported folder onto the panel that ‘Drop a folder containing the baked textures here’. If you’ve done everything correctly the tool will compute a delighted colour texture using the default settings.
  9. Adjust the Delighting Parameters and hit Compute to see if you can get a better result. Once you are happy click Export to save your freshly delit texture!
  10. Close the Delighting Tool window.

Assigning your delit Material to your mesh

Ok, now let’s see what our efforts look like inside the unity engine.

  1. In the Project window, navigate your imported folder. Drag and drop your mesh onto the Scene window, zoom out to view it if you need to. It should be rendered in a plain white material:
  2. Back in the imported folder in the Project window, left click your imported normal texture (not the bent normal) to open it’s settings in the Inspector window.
  3. In the Inspector window, select the Texture Type drop down, choose normal then hit the Apply button a bit lower down.
  4. Back in the Project window, right click somewhere in your imported folder and choose Create > Material

    The material will show up as a sphere icon in the folder. Left click to select it and open it’s properties in the Inspector window.
  5. Now we plug in our texture maps to the material by dragging them onto the corresponding material slot
    • Base Colour Map > Albedo slot
    • Normal Map > Normal slot
    • Ambient Occlusion Map > Occlusion slot
      You can tweak each slots settings if you like.
  6. Next, drag your newly set up material onto your mesh in the Scene window. If all’s gone well you’ll have a nicely textured mesh ready to go in your game scene or we can export to Sketchfab using the handy Sketchfab Exporter. Read on to find out how…

Exporting from Unity using the Sketchfab Exporter

Nice work! Now we have a low poly scan complete with a delit texture and it’s time to show the world your work!

  1. Download the latest release of the Unity2Sketchfab exporter from GitHub. You want to grab the .unitypackage file.
  2. With your delighting project open in unity, open the Unity2Sketchfab .unitypackage and click Import in the Import Unity Package dialogue.
  3. Left click your mesh in the Scene window to select it (it’ll get an orange outline)
  4. In the Tools menu at the top, select Publish to Sketchfab
  5. In the pop up, sign in to Sketchfab with your email and password or, if you don’t have an account, click the Create Account link, set up an account then sign in.
  6. Once signed in, give your model a title, description and relevant tags. Your scan probably isn’t animated so un-check Export Animation and we want to adjust the look of our model before publishing so un-check Publish Immediately too:

    Once you’re done, hit Upload (x) objects where (x) is the number of objects you have selected. You’ll see a progress bar as it uploads:
  7. Once your model has uploaded, your default browser will open on the Sketchfab model page. From here you can jump into the 3D Settings to change the material, lighting, post processing filter, VR settings and  more:

https://sketchfab.com/models/8f9b8f1e21d8427783c829af0f10ae51

About the author

Thomas Flynn

Community & Cultural Heritage Lead at Sketchfab.

Expert in 3D scanning, photogrammetry, online publishing & dissemination.



No Comments

    Related articles