Exploring Medical Visualization in 3D

About Me

Hi, I am Doctor Jana, a medical doctor based in India with a passion for 3D technology. I have been learning, creating and exploring my imagination in the 3D industry for the last 10 years. My journey started in the first year of medical school, as I envisioned the artistic genius of Dr. Frank H. Netter’s illustration of human anatomy being transformed into a vivid 21st century masterpiece.

As I pursued my dream of being a doctor, the years of being a movie and technology buff helped me translate the fascinating wonders of medicine into the graphic reality of 3D. The world of computer-generated imagery has grown enormously over the years, challenging the boundaries of creative art. I have been fortunate to be a part of this wonderful journey, which inspires me professionally and personally at every step.

The opportunity to combine my dreams, passion, hobbies and restless imagination in creating 3D models of medical relevance is a satisfying and splendid personal achievement. Currently, I work as a primary care physician, part time teaching medicine faculty, and a freelance 3D medical artist. My work is used for educational and scholarly purposes, and as a firm believer in open source projects, I also contribute regularly to Wikimedia commons. I thoroughly enjoy the social aspect of patient and student interaction and wish to share and expand the excitement of medical art across the globe.

Background in 3D

I never had formal training in 3D, and like many self-educated artists, a large part of my knowledge comes from YouTube tutorials. I started with SketchUp almost 10 years ago, and eventually continued my work with Cinema 4D and ZBrush. Most of my projects used Cinema 4D and After Effects until 2019, when I started experimenting with Blender. I had tried Blender before, but the UI was difficult for me initially and I was reluctant to leave the coziness of my default Cinema 4D layout. But the closed nature and heavy licensing costs of Cinema 4D made me shift to Blender with the 2.8 builds, and there has been no looking back since then. I believe that the moment a 3D artist starts wondering about making a shift to Blender, it is already time to take that leap. Blender is free, stable, easy to use, has a ton of features and even more resources. The new 2.8 UI redesign is wonderful. There are a ton of new features coming up in Blender, including the new sculpt branch and npr rendering. The community is pretty sweet and the funding is improving with many studios shifting to Blender. There are a ton of add-ons to improve your workflow, heck, there is no other CAD software packing as many features as Blender. Modeling, UV editing, texturing, shading, physics simulations, sculpting, rigging, animating, node based compositing, video editing, rendering, scripting, grease pencil, retopology, mesh baking, volumetric lighting, script support, etc. If there ever was a digital version of the Swiss army knife, it would be Blender.

3D for Science

Though I fiddled with character animation, video editing, coding, and the gaming industry for awhile, my background in medicine made it an obvious choice when choosing my niche. I believed that conventional medical literature had to step up to the magnificent platform of computer-based imagery, with user-friendly 3D models serving as great learning tools. The ability to reproduce complex intricacies of human anatomy and physiology in the form of user-friendly online applications was the resounding inspiration behind my first project called “Interactive Neuroanatomy.”

The application was not as successful as I had wished, and I had to discontinue the project due to lack of funding and time. However, the failure taught me a lot of things such as starting with the basics and creating more refined products. That is when I made my first successful project, the animated realistic heart.

I created it as a personal project, and it was received very well by the Sketchfab community making it a staff pick. I used the same model to create the CG heart render which was nominated as the featured image on Wikipedia for the year 2017 and won the 3rd prize.

As a student of medicine, I understand the challenges faced by students and young learners in grasping the complexities of human anatomy. In creating an easy-to-use interface, open to all the curious minds, I wish to present this astounding science in an exciting manner. It shall benefit people across the world, and contributions from other artists can help us make a new age comprehensive illustration of medicine. My professors and the doctors before them had to rely totally on 2D images and pictures to see and understand the way the body looks. Cadavers aren’t always easy to get and it was really difficult to understand the anatomy of the body if you are not very imaginative.

With the advent of these technologies, it is much easier now to see how a specific structure looks in the real world. These technologies become even better, especially when we are dealing with complex anatomical structures. For example, it is one thing to learn the course of a named artery through text, it is different when you look at a 2D image of it, and it is even better when you find it in the cadaver. The advent of digital atlases made it easier to understand and remember several anatomical structures. Recent advances in radiology made it possible to extract 3D data from MRI or CAT scans and show the pathology to the patient and explain what the problem is. A case came up in orthopedics where they had to mend a defect in bone. All they did was to get the 3D data, made a mesh that filled the defect in 3D, then 3D printed the mesh with implantable material, and used it to cover the defect in the real patient. Overall these technologies are really cool; if nothing else, you can always 3D print your patient’s organ after a scan and give it to them as a souvenir.

Current work

I am currently working on creating a lot of content on human anatomy, especially human osteology, and releasing it for free under a creative commons license.

I am working on tutorials as well as various illustrations of the human body and will be releasing them soon.

I have also been looking into 3D reconstruction from real patient data, like this model:

I kept myself busy in the last couple of months looking into the gazillion features that Blender has to offer. As of now, I am practicing retopology on models like these to strengthen the basics of topology and hope to use them for creating better 3D models with optimal vertex counts and neat topology flow.

My workflow

When creating the 3D model of the heart, I first started with a high resolution sculpt and later retopologized and baked the diffuse and normal maps.

Once I had the low poly model, I started making 25 blendshapes based on my understanding of the cardiac cycle and then animated the blendshapes as posemorphs in Cinema 4D. I then exported the fbx file and uploaded it to Sketchfab. Here this is the first version:

As we see in the model, the timing of the cardiac cycle is correct, but the movement doesn’t look very natural. Hence, I consulted a few peers and took their reviews on what they felt about it. Based on their input, I released a second version which looked much more natural.

However, this was still based on my understanding of the cardiac cycle and in no way a completely accurate representation of what happens in the heart. This led me to discover that we can extract the 3D data from real patient CT scans, and I obtained a set of dicom images, helping me create this model using the Sketchfab timeline option for animation.

Clearly the polygon count is a huge issue and I had to decimate the model a bit for it to run smoothly on low end devices, but this gives a general idea on how the heart pumps. In the next version, I shall try to superimpose the handmade low poly model over the extracted CT data and create a more accurate model of the animated heart.

My Advice for Anyone Wishing to Enter the 3D Industry

The 3D industry is probably one of the most exciting fields out there, with a steep learning curve. The 5 hours you spend on retopologizing a model to make it look better, the countless hours you spend UV unwrapping a model, the sleepless nights you spend trying to catch a bug in your process, are efforts and sacrifices which are greatly rewarding. It’s not always about the money, it’s not always about the fame, it is about the creative journey a 3D artist embarks on. A bad review or an unsuccessful project should be an encouraging force in this relentless path of learning and improvement, so we rise to create a masterpiece from our hard work, love, and
imagination.

Predicting the Future

I am pretty sure that the way 3d is advancing will revolutionize the field of learning. Mobile platforms, virtual reality, and augmented reality have already shown that mundane topics can be made easy to understand by making things interactive. A 3D view makes you understand things that are almost impossible to grasp in a 2D image. With better physics engines and simulations, we can understand several interactions that occur in the physical world.

Innovative platforms like Sketchfab’s massive can help us skip the barriers of poly count and use level of detail to show an entire universe on a low spec mobile phone. 3D printing prosthetics have already been a norm in a lot of surgeries. The most important aspect of 3D right now is for its educational value. 3D Modules for explaining complex anatomical structures are such great tools to teach students about anatomy. Medical visualization is going to grow in the future to simplify complex concepts. Just like any other thing in science, the possibilities in 3D are infinite and the only barrier is your imagination.

If you have any cool projects to discuss or for business proposals, feel free to email me.

Sincerely,

Doctor Jana

Website / Instagram