About Us
Mexico is one of the most biodiversity-rich (so-called megadiverse) countries of the world, hosting more than 10% of Earth’s species. To explore and document, but also to protect and conserve this high species richness, the Mexican National Commission for the Knowledge and Use of Biodiversity (CONABIO) has been working on the development, maintenance and update of the National Biodiversity Information System (SNIB) for more than 25 years now.
How to convert the data of the SNIB into information that is accessible and relevant to society and stakeholders has always been a main concern of CONABIO. Since biodiversity information is essentially spatial, geovisualization is a method of choice. Among the different tools of geovisualization we have adopted so far, VR is only the latest. The VR coral reef presented subsequently is part of the “Mexico VR” project, where we try to represent different Mexican ecosystems in a photorealistic manner. This shall facilitate a better understanding of the ecosystemic importance, but also of the beauty of these environments – even to people who have never physically been there.
Why 3D and VR?
We experience the world around us in various dimensions. The three dimensions of space (3D) can be extended in different manners, for example by adding changes in time (4D). In order to represent and communicate environmental issues in such a multi-dimensional space, 3D VR environments are fine instruments: They not only allow us to explore and visualize geospatial problems distant in time and/or space, but also encourage engagement and involvement with the problems visualized. Hence, while 2D-visualizations like common web-maps are just mere representations, immersive VR systems can help us to form spatial presence, this is: a feeling of being there.
Workflow
We try to represent ecosystems not just photo-realistically, but also on a georeferenced data basis. In the present case of a coral reef in the Mexican Caribbean (near the city of Cancun), high resolution data of the underwater terrain (bathymetry) was derived from WorldView-2 satellite imagery. Moreover, information on animals and plants living on the seafloor (so-called benthic habitats) was generated both from satellite images and on-site data (including video and photo material), and aggregated into a seven-part classification of reef communities (see fig. below). These main reef communities were translated into lists of (the most representative) species in accordance with the SNIB.
Each species is then modelled (using Blender) upon guide images/photos, which are taken from the side-, back-/front- and top-/bottom views and allow us to fit basic geometry objects to the identifying features of the species to be represented. Moreover, these images serve as reference to define realistic textures. Skeletal animations are applied to simulate movements typical for each species.
To merge geospatial baseline data and 3D species models into a format suitable for an immersive visualization on a head-mounted display (HMD), Unreal Engine was used to compose an interactive VR application. The features of this game engine allow us to define a realistic environment setting, e.g. regarding water color, visibility range, wave movements and underwater light refraction.
Results
Game engines allow packaging of the final VR application for all of the (still few) HMD systems currently available on the market. In our case, the Oculus Rift is the target hardware the VR app is optimized for. However, in order to provide maximum immersion and spatial presence, both an HMD and powerful graphics hardware are required.
Since this combination is not yet accessible for most users, we are also publishing parts of the project results in a less- or non-immersive manner in the form of VR-enabled applications, where a head mounted smartphone provides the display and processor of the VR system (e.g. Google’s Cardboard). We take advantage of this alternative by means of a smartphone app but also through Sketchfab, where the user can interact with (currently 50) 3D-species models both in a desktop- and cardboard mode. Audio files are also included in Sketchfab and the smartphone app. Moreover, all models can be downloaded for free in genuine format from Sketchfab. Finally, for those users with no type of HMDs at hand, a non-immersive desktop VR application of the coral reef was built. Both the high-, low- and non-immersive results are available on the project website.
Scenes from the prototype of the VR coral reef app
Challenges
Large-scale ecosystem modeling is, especially for natural (compared to built) environments, still a time-consuming process. Species modeling and animation within the workflow described were therefore conducted manually from scratch. However, several trends can be observed that will act to significantly speed up 3D-modeling.
Even with low resources, techniques of close-range photogrammetry (e.g. Structure-from-Motion: SfM) allow capture of both the geometry and texture of static objects in high resolution. Although we achieved unsatisfactory results under the present underwater scenario, for terrestrial ecosystems SfM already provides practicable solutions to the automatic generation of high-quality 3D content, especially where zoological or type specimens are available.
Another issue that is up-and-coming is the realistic modelling of interactions between elements of flora and fauna (e.g. food chains or predator-prey relationships) with the aid of artificial intelligence (AI). As part of the present project, we are working on AI-based algorithms to represent the flocking behavior of reef fishes.
The Future
We are currently completing the VR replica of the coral reef described here. With the implementation of the aforementioned AI behaviour we hope to include more than 10,000 animated fish models, which shall, together with the static members of reef flora and fauna, provide a faithful replica of a Caribbean coral reef under present environmental conditions. In a next step, we want to take advantage of VR as computer-generated experience allowing us to simulate past (e.g. a pristine coral reef) or future scenarios (e.g. coral bleaching) of the site we have modelled.
Such a combination of present, past and (possible) future environmental conditions visualized in an immersive VR system can be also of interest to science centres and museums. Hence, we are contacting science museums in Mexico and abroad to discuss an installation of the application for a wider audience and within a more specific context of knowledge communication.
Additionally we have started with the modelling of a second ecosystem, which will be a mangrove forest near the coral reef we have visualized. The first species for this ecosystem are already available:
We encourage readers to check our recent publication in the International Journal of Digital Earth, discussing the project in more detail: https://www.tandfonline.com/doi/full/10.1080/17538947.2018.1501106
About the authors
The project “Mexico VR” is being developed by CONABIO. Main 3D artists are Genghis de la Borbolla (with a biology degree) and Oscar Paz (with a cinematography background). Biologist Verónica Aguilar approves the realism of 3D-models and animations, and physicist Gustavo Magallanes trains our VR agents on spontaneous and intelligent-like behaviour. Project leaders are Rainer Ressl and Florian Hruby.
Project Website / Institutional Website / Facebook / Twitter / YouTube / Vimeo / Instagram
About the author
CONABIO
National Commission for the Knowledge and Use of Biodiversity
