{"id":142059979173,"date":"2017-08-17T11:44:19","date_gmt":"2017-08-17T09:44:19","guid":{"rendered":"https:\/\/blog.sketchfab.com\/?p=142059979173"},"modified":"2021-04-16T17:24:50","modified_gmt":"2021-04-16T15:24:50","slug":"de-lighting-3d-scans-in-unity-by-pete-mcnally","status":"publish","type":"post","link":"https:\/\/sketchfab.com\/blogs\/community\/de-lighting-3d-scans-in-unity-by-pete-mcnally\/","title":{"rendered":"De-Lighting 3D Scans in Unity with Pete McNally"},"content":{"rendered":"<p><em>Sketchfab user <a href=\"https:\/\/sketchfab.com\/petemc\">Pete McNally<\/a> give us the run down on how he uses Unity&#8217;s new de-lighting utility to make his scans look great in any lighting conditions.<\/em><\/p>\n<p>How\u2019s it going Sketchfab Community?! I\u2019m Pete Mc Nally, a Senior 3D Artist based in Dublin, Ireland with a growing interest in 3D scanning, specifically photogrammetry.<\/p>\n<p>Photogrammetry is such a useful tool for the 3D artist\u2019s box, even without cumbersome or expensive equipment. As long as you have a half decent phone with you there is potential to capture a model.<\/p>\n<p>Ireland generally has favourable conditions for outdoor photogrammetry: the weather is generally overcast, or at least that\u2019s how it seems! This makes it relatively easy to correct albedo textures derived from photogrammetry for more general use in various PBR lighting setups, like this one:<\/p>\n<div class=\"sketchfab-embed-wrapper\">    <iframe title=\"Beach rock\" class=\"\" width=\"690\" height=\"388\" src=\"https:\/\/sketchfab.com\/models\/84dc87fa1ebf4950b592d41874cac05b\/embed\" frameborder=\"0\" allow=\"autoplay; fullscreen; xr-spatial-tracking\" allowfullscreen=\"\" mozallowfullscreen=\"true\" webkitallowfullscreen=\"true\" xr-spatial-tracking=\"true\" execution-while-out-of-viewport=\"true\" execution-while-not-rendered=\"true\" web-share=\"true\"><\/iframe><\/div>\n<p>Unfortunately the day I went to capture the subject of this article &#8211; a sea stack near Ireland\u2019s most Northerly point, Malin &#8211; there were blue skies and blinding sunshine. 15 hours of sunshine that day in fact! Despite there being significant contrast between the lit and shadowed sides of the subject, I wasn\u2019t put off, I figured this might be a good chance to try out some of the de-lighting techniques I\u2019d been reading about.<\/p>\n<p>Unity had just released an <a href=\"https:\/\/labs.unity.com\/article\/experimental-feature-de-lighting-tool\" target=\"_blank\" rel=\"nofollow\">experimental de-lighting tool by Cyril Jover<\/a> to help with such situations, so I captured about 120 photographs of the stack to process in <a href=\"https:\/\/www.capturingreality.com\/\" target=\"_blank\" rel=\"nofollow\">RealityCapture<\/a>. Here is one of the images in the sequence, you can see some artifacts like lens flare and significant darkening in the shadowed side.<\/p>\n<p><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979177 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAIAAAAAAAP\/\/\/yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\" data-src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image5-2.jpg\" alt=\"\" width=\"1920\" height=\"1280\" \/><noscript><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979177 lazyload\" src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image5-2.jpg\" alt=\"\" width=\"1920\" height=\"1280\" srcset=\"https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image5-2.jpg 1920w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image5-2-300x200.jpg 300w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image5-2-768x512.jpg 768w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image5-2-1030x687.jpg 1030w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image5-2-750x500.jpg 750w\" sizes=\"(max-width: 1920px) 100vw, 1920px\" \/><\/noscript><\/p>\n<p>Here is the resulting textured model from Reality Capture, decimated to about 16 million triangles with a 16k texture:<\/p>\n<p><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979176 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAIAAAAAAAP\/\/\/yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\" data-src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image8-1.jpg\" alt=\"\" width=\"1999\" height=\"1117\" \/><noscript><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979176 lazyload\" src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image8-1.jpg\" alt=\"\" width=\"1999\" height=\"1117\" srcset=\"https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image8-1.jpg 1999w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image8-1-300x168.jpg 300w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image8-1-768x429.jpg 768w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image8-1-1030x576.jpg 1030w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image8-1-895x500.jpg 895w\" sizes=\"(max-width: 1999px) 100vw, 1999px\" \/><\/noscript><\/p>\n<p>The goal was to create a game-ready asset from this source, removing as much lighting information as possible from the albedo without resorting to time consuming hand painting. With that in mind, I reduced polys in <a href=\"http:\/\/www.meshlab.net\/\" target=\"_blank\" rel=\"nofollow\">Meshlab<\/a> to 10k and unwrapped in <a href=\"https:\/\/www.autodesk.co.uk\/products\/3ds-max\/overview\" target=\"_blank\" rel=\"nofollow\">3DSMax<\/a>, then downloaded the Unity De-Lighting tools from here: <a href=\"https:\/\/github.com\/Unity-Technologies\/DeLightingTool\" target=\"_blank\" rel=\"nofollow\">https:\/\/github.com\/Unity-Technologies\/DeLightingTool<\/a><\/p>\n<p>Traditionally I have used 3dSMax for texture baking, but the Unity de-lighting workflow requires a bent normals texture, which meant looking elsewhere. I tried <a href=\"https:\/\/www.knaldtech.com\/\" target=\"_blank\" rel=\"nofollow\">Knald<\/a> and it was pretty much a one stop shop for everything I needed, and very fast. Unity requires the original lit diffuse texture (ideally with an alpha channel around the UV islands), object space normals, bent normals and ambient occlusion for it\u2019s de-lighting tool &#8211; Knald baked each of these in one pass at 4k.<\/p>\n<p><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979178 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAIAAAAAAAP\/\/\/yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\" data-src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image3-1.jpg\" alt=\"\" width=\"1112\" height=\"1080\" \/><noscript><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979178 lazyload\" src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image3-1.jpg\" alt=\"\" width=\"1112\" height=\"1080\" srcset=\"https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image3-1.jpg 1112w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image3-1-300x291.jpg 300w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image3-1-768x746.jpg 768w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image3-1-1030x1000.jpg 1030w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image3-1-515x500.jpg 515w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image3-1-45x45.jpg 45w\" sizes=\"(max-width: 1112px) 100vw, 1112px\" \/><\/noscript><\/p>\n<p>Once you have the baked textures, it\u2019s a simple matter to conform them for the tool by using the recommended naming convention and loading them into the Unity package:<\/p>\n<blockquote><p>*_DLBC.tga (Base Map, RGBA, Original color texture to unlit)<br \/>\n*_DLAO.tga (Greyscale, Ambient Occlusion Map)<br \/>\n*_DLN.tga (RGB, Object-Space Normal Map)<br \/>\n*_DLBN.tga (RGB, Object-Space Bent Normal Map)<\/p><\/blockquote>\n<p style=\"padding-left: 30px\">(The above notes are taken from the tool\u2019s <a href=\"https:\/\/github.com\/Unity-Technologies\/DeLightingTool\/blob\/master\/Documentation\/De-LightingTool.pdf\" target=\"_blank\" rel=\"nofollow\">excellent documentation on GitHub.<\/a>)<\/p>\n<p>The process works very quickly, in a matter of seconds it had removed a lot of the baked light, you can clearly see the effect over the top right island, which was mostly in shadow. See before and after below:<\/p>\n<p><img decoding=\"async\" class=\"alignnone wp-image-142059979179 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAIAAAAAAAP\/\/\/yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\" data-src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image6.gif\" alt=\"\" width=\"688\" height=\"688\" \/><noscript><img decoding=\"async\" class=\"alignnone wp-image-142059979179 lazyload\" src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image6.gif\" alt=\"\" width=\"688\" height=\"688\" \/><\/noscript><\/p>\n<p>If there are areas where artifacts occur, it\u2019s easy to paint up a mask for those parts and compute again. Unity also accepts a position map for more accurate de-lighting (a forthcoming version of Knald will also allow position map baking).<\/p>\n<p>I\u2019m impressed that it worked so well out of the box. I did paint a mask for some of the areas where there were obvious repeating detail issues, but this was super quick to do in Photoshop. It was then loaded into the \u201cmask\u201d area of the tool and re-computed, which gave better results.<\/p>\n<p><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979180 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAIAAAAAAAP\/\/\/yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\" data-src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image2-1.jpg\" alt=\"\" width=\"1999\" height=\"1094\" \/><noscript><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979180 lazyload\" src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image2-1.jpg\" alt=\"\" width=\"1999\" height=\"1094\" srcset=\"https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image2-1.jpg 1999w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image2-1-300x164.jpg 300w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image2-1-768x420.jpg 768w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image2-1-1030x564.jpg 1030w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image2-1-914x500.jpg 914w\" sizes=\"(max-width: 1999px) 100vw, 1999px\" \/><\/noscript><\/p>\n<p>Later, I fixed up areas with missed ray hits in <a href=\"https:\/\/www.allegorithmic.com\/products\/substance-painter\" target=\"_blank\" rel=\"nofollow\">Substance Painter<\/a> (visible as red blotches on the texture above) by clone stamping, then adjusted the overall albedo for PBR. A roughness map was created by manipulating the corrected albedo in Photoshop and hand painting for a wetter look on the lower stack, which was being hit with sea spray constantly.<\/p>\n<p>I selected some of the darker areas of the albedo on the sea facing side and used those as a basis for working out where the wave spray was hitting, then masked using AO and curvature from Knald. Next, I used the \u201cLeaks\u201d particle brush in Substance Painter, which filled crevices and flat areas and created believable rivulets of runoff water.<\/p>\n<p><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979181 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAIAAAAAAAP\/\/\/yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\" data-src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image7-1.jpg\" alt=\"\" width=\"1999\" height=\"1094\" \/><noscript><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979181 lazyload\" src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image7-1.jpg\" alt=\"\" width=\"1999\" height=\"1094\" srcset=\"https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image7-1.jpg 1999w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image7-1-300x164.jpg 300w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image7-1-768x420.jpg 768w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image7-1-1030x564.jpg 1030w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image7-1-914x500.jpg 914w\" sizes=\"(max-width: 1999px) 100vw, 1999px\" \/><\/noscript><\/p>\n<p>Lastly, it remained to upload it to Sketchfab and see how it behaved under different real-time lighting conditions, here are some images:<\/p>\n<p><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979182 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAIAAAAAAAP\/\/\/yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\" data-src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image4.jpg\" alt=\"\" width=\"1999\" height=\"1125\" \/><noscript><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979182 lazyload\" src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image4.jpg\" alt=\"\" width=\"1999\" height=\"1125\" srcset=\"https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image4.jpg 1999w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image4-300x169.jpg 300w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image4-768x432.jpg 768w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image4-1030x580.jpg 1030w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image4-888x500.jpg 888w\" sizes=\"(max-width: 1999px) 100vw, 1999px\" \/><\/noscript><\/p>\n<p><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979175 lazyload\" src=\"data:image\/gif;base64,R0lGODlhAQABAIAAAAAAAP\/\/\/yH5BAEAAAAALAAAAAABAAEAAAIBRAA7\" data-src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image1-1.jpg\" alt=\"\" width=\"1999\" height=\"1125\" \/><noscript><img decoding=\"async\" class=\"alignnone size-full wp-image-142059979175 lazyload\" src=\"https:\/\/blog.sketchfab.com\/wp-content\/uploads\/2017\/08\/image1-1.jpg\" alt=\"\" width=\"1999\" height=\"1125\" srcset=\"https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image1-1.jpg 1999w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image1-1-300x169.jpg 300w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image1-1-768x432.jpg 768w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image1-1-1030x580.jpg 1030w, https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/image1-1-888x500.jpg 888w\" sizes=\"(max-width: 1999px) 100vw, 1999px\" \/><\/noscript><\/p>\n<p>Finally, here is the scene on Sketchfab:<\/p>\n<div class=\"sketchfab-embed-wrapper\">    <iframe title=\"Sea stack 3D Game ready asset\" class=\"\" width=\"690\" height=\"388\" src=\"https:\/\/sketchfab.com\/models\/c9ddda63c5a641b2bb860224a541d4c0\/embed\" frameborder=\"0\" allow=\"autoplay; fullscreen; xr-spatial-tracking\" allowfullscreen=\"\" mozallowfullscreen=\"true\" webkitallowfullscreen=\"true\" xr-spatial-tracking=\"true\" execution-while-out-of-viewport=\"true\" execution-while-not-rendered=\"true\" web-share=\"true\"><\/iframe><\/div>\n<p>The location of the subject is here:<br \/>\n<div class=\"googlemaps\">\n\t\t\t\t<iframe width=\"600\" height=\"450\" frameborder=\"0\" scrolling=\"no\" marginheight=\"0\" marginwidth=\"0\"  src=\"https:\/\/www.google.com\/maps\/embed?pb=!1m0!4v1502822116253!6m8!1m7!1sThk_cKGdx0ef_XffysaIQg!2m2!1d55.3682866526309!2d-7.317361793577135!3f70.72!4f-3.230000000000004!5f2.9334094813758367\"><\/iframe>\n\t\t\t<\/div><\/p>\n<p>I\u2019ll be incorporating the Unity De-lighter into my future 3D scanning workflow, it\u2019s fast and automatic for the most part, and at the very least will give you a worthwhile head start on creating believable albedo textures from photogrammetry, at best you won\u2019t have to re-touch it at all.<\/p>\n<p>For more examples of 3D scanned game art assets, and other 3D art experiments, swing by my blog at <a href=\"http:\/\/www.petemcnally.com\" target=\"_blank\" rel=\"nofollow\">petemcnally.com<\/a> or follow me on Twitter <a href=\"https:\/\/twitter.com\/petesonearth\" target=\"_blank\" rel=\"nofollow\">@petesonearth<\/a>. Thanks for reading :)<\/p>\n<p><em>Thanks Pete!\u00a0Be sure to check out more of Pete&#8217;s work on his <a href=\"https:\/\/sketchfab.com\/petemc\">Sketchfab Profile<\/a>. If you have a go at using the delighting tool yourself, why not share the results in the comments?<\/em><\/p>\n<p><em>If you&#8217;re nterested in this topic, be sure to check out <a href=\"https:\/\/unity3d.com\/solutions\/photogrammetry\" target=\"_blank\" rel=\"nofollow\">Unity&#8217;s Photogrammetry guide<\/a> as well.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Sketchfab user Pete McNally give us the run down on how he uses Unity&#8217;s new de-lighting utility to make his scans look great in any lighting conditions.<\/p>\n","protected":false},"author":32,"featured_media":142059979183,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[582],"tags":[1059,835,49,984,64,991,194],"class_list":["post-142059979173","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tutorial","tag-advanced-photogrammetry","tag-de-lighting","tag-photogrammetry","tag-tutorial-photogrammetry","tag-texturing","tag-tutorial-3d-scanning","tag-unity"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/sketchfab.com\/blogs\/community\/wp-content\/uploads\/2017\/08\/header02.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/sketchfab.com\/blogs\/community\/wp-json\/wp\/v2\/posts\/142059979173","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sketchfab.com\/blogs\/community\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sketchfab.com\/blogs\/community\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sketchfab.com\/blogs\/community\/wp-json\/wp\/v2\/users\/32"}],"replies":[{"embeddable":true,"href":"https:\/\/sketchfab.com\/blogs\/community\/wp-json\/wp\/v2\/comments?post=142059979173"}],"version-history":[{"count":10,"href":"https:\/\/sketchfab.com\/blogs\/community\/wp-json\/wp\/v2\/posts\/142059979173\/revisions"}],"predecessor-version":[{"id":142059990665,"href":"https:\/\/sketchfab.com\/blogs\/community\/wp-json\/wp\/v2\/posts\/142059979173\/revisions\/142059990665"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sketchfab.com\/blogs\/community\/wp-json\/wp\/v2\/media\/142059979183"}],"wp:attachment":[{"href":"https:\/\/sketchfab.com\/blogs\/community\/wp-json\/wp\/v2\/media?parent=142059979173"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sketchfab.com\/blogs\/community\/wp-json\/wp\/v2\/categories?post=142059979173"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sketchfab.com\/blogs\/community\/wp-json\/wp\/v2\/tags?post=142059979173"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}