Tutorial: Processing Point Cloud Data with Unity

Back to overview

Sketchfab Community Member Leon Denise created his own solution to make point clouds, the output of many 3D scanning apps, more visually attractive. While we have in the meantime released a similar feature for point clouds, his approach to using and transforming such data is very interesting and can be applied in many different and new ways. In addition his script has some nice tricks that we don’t, so if you’re into point clouds be sure to read this!

Hi! My name is Leon Denise, I’m an independent artist programmer. I draw comics, make games, code shaders, hug trees and talk to cows. You can find my work on GitHub, Giphy and of course here on Sketchfab!

In this tutorial, I’m going to explain how I did the Simon Splash:

I used Agisoft PhotoScan to generate the point cloud and Unity3D to create the mesh.

The Unity3D project can be downloaded at github.com/leon196/PointCloudExporter.

About the project

The project is a point cloud tool used for generative art. It is a real-time software that generates triangle particles from a point cloud and displace them in a vector field with specific rules for velocity behavior. The tool also exports the generated particles into a standard 3d mesh and bakes the vertex colors into a map with generated texture coordinates.

The motivation

The motivation came from the fact that point cloud viewers are not very advanced, since point clouds are not the real matter of 3d worlds such as video games. When you zoom in, points stay as dots, it creates to me a frustration that needs to be satisfied. [you can set the point size now – ed]

So I’ve coded a program that generate triangles from the dots, taking position, normal and color. I’m sharing the source code of the program, it’s a simplified version (no GPU particles) of my point cloud engine. For the sake of comprehension and because the advanced version is part of a commercial project.

Export a point cloud

After you have generated a point cloud in your favourite software, export it as PLY format:
binary encoding with position, normal and color.

Examples with Agisoft Photoscan and MeshLab:

My PLY importer C# script is pretty naive and there is chances that it won’t work with a PLY exported from a different software. If you improve the importer script, and like to share, do not hesitate to send a pull request on the git repository.

Start the Unity3D project

Download, install Unity3D and grab the project here. If you don’t know how to use git, find the download zip button:

You will find the scripts and shaders used to generate triangle mesh from a point cloud. The main script is PointCloudGenerator.


If you don’t know about Unity3D, the official documentation is great and there are plenty tutorial resources all over the web. To use the tool, open the scene “PointCloudExporter”. Start the play button, select the PointCloud game object in the hierarchy window, so you can adjust parameters in the inspector window.

Set the load parameters

PointCloudGenerator will load a file in the StreamingAssets folder and automatically adds the .ply extension. That’s why File Name is just “Simon2”. You can adjust the Maximum Vertices count to get more details.

Set the renderer parameters

  • Size is the radius of the triangle.
  • Sprite allows you to choose a texture for the triangle.


About the sprite

The texture used for the sprite must have clamp wrap mode. Disabling mipmapping will give better results. Since we use triangles instead of quads for performance, the UV mapping is sized to cover the triangle.

On the left, sprite with clamp wrap mode. On the right, sprite with repeat wrap mode.

About the shader

You can check the Shader code used to generate triangles. (warning: geometry shader stuff)

If you inspect the “Generate” function in the PointCloudGenerator script, there is a part where it creates mesh with point topology. So the script create a mesh with no triangles, only points. Then a geometry shader is generating triangle from point.

If you change the shader for “Unlit/Color” for example, you will get this render: points.

With the PointCloud shader, you will get this render: triangles with color and direction.

Here is a scary screenshot with hieroglyph characters known as shader language code. That part shows a function that create a triangle from a point: a geometry shader.


The challenge is to display triangles that will face a certain direction. Manipulating vertex in a shader is about parallelism and knowing if a point in space belong to a triangle requires storing data into vertices. I’m using geometry shader mainly because you can not setup custom attributes for vertices in Unity3D. (There are tricks, but let’s keep it simple for this tutorial)

Displace parameters

  • Should is the ratio that selects which particles should be displaced.
  • Time is the duration of the displacement.
  • Speed is the velocity unit.
  • Noise Scale is the level of details of the Perlin noise.
  • Noise Speed is the coefficient of the noise offset.
  • Target Speed is the coefficient of the target offset.
  • Noisy is like salt, it adds a grain to coefficients.


The displacement behaviour can be found in the “Displace” function. This simple version is about two vectors: a smooth random direction and moving away from a point in space. This is where you can adjust and customise the displacement behaviour.


Baking parameters

  • Details is a sort of level of details for color.
  • Circle Radius is the size of the color on the map.
  • Shader Baked is the shader to use when triangle mesh is generated.

A baked map with a ‘details’ value of 4:


A baked map with a ‘details’ value of 16:


Because you can’t always have custom shader, you can’t set the triangle color from the vertices color attribute. That’s why this tool have a baking map colors, so the mesh can be imported on Sketchfab or any 3d viewer. If you inspect the baked mesh, you will find the generated texture coordinates.


Action buttons

  • Generate is triggered when pressing start button, and can be used to reload point cloud.
  • Reset will move the vertices to their original positions.
  • Displace will start a displacement sequence, with the chosen parameters.
  • Export will create a mesh ready to be exported. (it does not create a file on the disk)
  • Save Baked Colours will open a save dialog window, asking where you write the texture file.


To export the model, you have to press the export button. It will create a triangle topology mesh that will replace the point topology mesh. The easiest way to publish this to Sketchfab is to use the Unity Exporter. You can also use the script ExportOBJ from the UnifyCommunity, that is already in the project. It works like this: select any game object, click through File / Export / Wavefront OBJ.


If everything worked out, you should be able to import it in you favourite 3D software, and then share it on Sketchfab!


Voilà !

I hope it will work at the first try for your point cloud data. It’s an open source work in progress project, so there is unstable code, optimizable algorithm and performance issues. Also after an export, you will have to generate again a point topology mesh, to be able to displace again. There is a lot of things that could be improved.

Thanks for reading through this whole article. You can find more open source resources and personal projects on my little compilation repository.  Also I’m spamming gifs on Twitter @leondenise.

About the author

Bart Veldhuizen

Head of Community at Sketchfab. 3D Scanning enthusiast and Blenderhead. Running BlenderNation in my spare time.


  • Hey Bart, do we need to do all the steps inside Unity3D to get this “Point Cloud” format to work in Sketchfab? I will try to make a smoother pipeline utilizing Blender and Add-On such as Sverchok or Animation Nodes. If I know what’s exactly this Point Cloud format that is good and will work with Sketchfab. I am guessing there is some kind of shader needed.

  • Bart Veldhuizen says:

    No you don’t; this is just an illustration of how to process point clouds with your own code. You can upload point cloud data directly as .PLY files.

  • Marian says:

    I got a better result changing the normal to this in the pointcloud shader (geom):
    pIn.normal = UNITY_MATRIX_IT_MV[2].xyz;
    This makes the normals always point to the camera so they look the same at every viewing angle.

  • Giorgos Neokleous says:

    Hello Bart. I can’t load my own point cloud for some reason. It’s a ply file. I also extracted it from MeshLab as you shown above. The code doesn’t give any errors but the point cloud is not loaded. any thoughts?

  • Hi Giorgos – where are you getting your data from? PhotoScan or a surface mesh you’ve loaded into MeshLab?

  • Giorgos Neokleous says:

    My point clouds are downloaded from this database here http://rgbd-dataset.cs.washington.edu/dataset/rgbd-scenes-v2/
    Does the code support them?? Thanks for replying!

  • Arnaud Briche says:

    Hey Bart,
    First, thank you for offering this free tutorial online, It’s really cool of you !
    I’ve tried to use your project to add a new point cloud to your scene, but I currently have no point displayed when I click on generate …
    I’ve used MeshLab to export several .txt files with coordinates and RGB colors to .ply file in binary, as you can precise above. I then add the file to the Streaming Assets and change the name in the Point Cloud (without the .ply extension of course).

    The only think that is not good (I think) is that when I export the txt file to ply in ASCII, we can see that all the normal generated are at 0 (x, y and z for all points). Is there a chance the problem could come from this ?

    Thank you really much !

  • Ryan Aisthorpe says:

    I found this really interesting as I am currently using faro scene software 3D scanning food factories all over the uk. I am subcontracting for one of the leading Engineering Companies in the UK. In my spare time I use unity to make simple games and currently learning to code for VR using the HTC Vive. I have some great ideas for the industry using this tech but I am still in my infancy of C#. Any insights in getting point cloud data into a usable format for VR would be really appreciated. I currently convert point cloud data into simple mesh scenes using LOD to increase performance. Using the native points and converting on the fly would be a great advantage if it is possible.

    Anyway I have blabbed enough, thank you for your contribution to this upcoming technology, hope to hear from you.

    Kind regards,

    Ryan Aisthorpe
    CAD Eng & Des Ltd. (Director)

  • Val says:

    Hi Bart!
    I really like the project you’ve created! One question, how do you display more than one point cloud? Just duplicating the folder in the hierarchy window (and changing the file name) is OK, but not enough if you later want to create the animation (the data cannot be included in it)… Do you have any suggestions? Thank you! 🙂

  • Bart Veldhuizen says:

    I’ve asked Leon to answer this.

  • Leon Denise says:

    Hi everybody! I’m Leon, the author of the project and the tutorial.
    (Bart is the author of the blog’s post)

    I’ve heard your questions! But I’m afraid I won’t update this project. It has too many flaws, should be optimized, upgraded and documented. May be I’ll do it and then try to sell it on the Unity3D Asset Store.
    It was an experiment, a strange idea from my shader laboratory.

    If you need a point cloud importer/renderer, Keijiro Takahashi made this for Unity3D: https://github.com/keijiro/Pcx

    Have fun with your point clouds!

  • Jorge Rodiles says:

    I am doing some research for capturing animated point clouds, like volumetric Videos can you make it to load a PLY sequences?

  • misi says:

    Hello, I am a newbie, I want to know if I can use the mouse to intercept a point cloud like a screenshot for analysis in unity.

Leave a Reply

Your email address will not be published. Required fields are marked *

Related articles