Snapdragon and Qualcomm branded products are products of
Qualcomm Technologies, Inc. and/or its subsidiaries.
In my recent blog, Increasing Realism for an Elite Gaming Experience, we introduced physics based rendering (PBR) as a key feature of Snapdragon® Elite Gaming aimed at increasing realism in mobile games.
PBR is an approach to rendering that treats light as it behaves in the real world, taking into account the conservation of energy (i.e., reflected light doesn't exceed the source light), while adhering to physically-correct distribution patterns and physically-accurate material definitions. The goal is to increase the photorealism of computer graphics by attempting to mimic the real-world effects of lighting on different materials. In general, this provides the following advantages:
- Materials look correct regardless of lighting conditions
- Artists can spend less time creating multiple maps to define lighting behaviors, and instead focus on assigning material types that become baked into a texture layer in the art asset pipeline
- Different types of surfaces such as metal, leather, plastic, glass, etc., should look more photorealistic and more natural, because of how the light reflects and the resulting accuracy of the shadows
In practice, this can mean the following for game development teams:
- Programmers will write shaders that implement various PBR calculations to run in real time, and update their rendering pipelines accordingly
- Artists will need to update and/or create art content (e.g., texture maps) that can be used by PBR calculations in shaders
- Programmers will need a platform that features a powerful rendering processor to do PBR calculations
Traditionally, PBR hasn't been as prevalent on mobile since it adds more rendering computations at runtime to calculate lighting effects. However, the Snapdragon® 855 Mobile Platform gives developers the potential to overcome this, namely through its new Qualcomm® Adreno™ 640 GPU which provides up to 20% improved performance over its predecessor: the Qualcomm® Adreno™ Adreno 630 GPU, and upcoming support for Vulkan 1.1. With that in mind, let's take a closer look at the physics behind PBR.
A Physics-based Approach
PBR subscribes to the idea that all materials reflect light to some degree. The following are some of the more common PBR techniques used to model physical properties and behaviors of materials and lights, and the interaction between the two:
Reflection and diffusion: Reflection describes how light reflects off a surface in relation to the surface normal. This will cause smoother surfaces to look more mirror-like and rough surfaces to look duller. Diffusion describes the degree to which light is absorbed by the material, and the amount, direction (aka scattering), and color of unabsorbed diffuse light that is able to reflect back off the material.
One approach for modelling this is to define an albedo color that a shader can use to calculate the diffuse color. This defines the degree to which various colors will reflect back out of the surface. Note that when performing these calculations, developers will also need to factor in micro surfaces (described below).
Translucency and transparency: This describes how much light enters and travels through a material, versus the amount which is reflected back. Translucency and transparency can affect diffusion, and may require special shaders to handle highly-transparent materials such as glass, which may result in little or no scattering of light. For example, the following image shows how light varies based on the translucency and transparency of surfaces and materials:
Implementing this behavior can be done through a multitude of techniques such as using a transparency texture that defines alpha channels, or casting a ray through various points on a surface with artist-defined opacity, to determine transparency.
Micro surfaces: All surfaces and their materials have tiny imperfections that affect the reflection of light (see image below). Modelling such imperfections through additional vertices is impractical, so one approach is to allow artists to define a general smoothness for a given texture or object, and then use a shader to calculate its effect on light. Alternatively, developers could set up their graphics pipelines to take in a smoothness texture that defines a more granular level of smoothness across smaller regions of surfaces.
Energy conservation: This involves calculating how much light leaves the surface of an object, while enforcing the rule that the amount of light reflected can never exceed that which fell upon the material. One approach is to simply calculate a reduced light intensity (e.g., through an artist-driven parameter) before calculating diffusion.
Metals: Metals tend to be much more reflective and shinier than other materials, and their color may affect the color of light that gets reflected. In addition, diffusion with metals may not be present due to energy absorption or may be affected by the cleanliness of a surface. If there are a lot of metallic materials in your game, it may be more efficient to set up a metallic parameter that artists can use to define such materials, as opposed to tweaking other properties like reflectivity.
Ambient occlusion: This is the darkening effect (i.e., shadows) that occurs in parts of a scene, which is caused by objects that are blocking the direct path from a point in the scene to the light source.
One approach for calculating this is to perform a hemispheric sampling algorithm in a shader, that casts rays in different directions from each point on a given surface. The algorithm then determines how many of the rays are blocked by other surfaces, to derive the overall darkness of the point.
Fresnel: Describes how the angle at which light hits a surface affects the reflectivity. This generally means that the light hitting the edges of an object is more likely to reflect than the light that hits the surface straight-on from the perspective of the current viewport. This can be seen in the following image:
One approach for replicating this behavior is to handle the Fresnel calculation mostly in shader code, and then provide art asset pipeline parameters through which the artist specifies material properties like reflectivity and glossiness. The code can then calculate the Fresnel effect by taking this artist-defined base reflectivity into account.
Get Real
The items above are just a few of the many PBR behaviors and properties of lighting that can help to increase realism. Also keep in mind that even with today's high-resolution mobile screens, the amount of complexity you need to implement will depend on the level of detail available to the player(s) on any given frame. Therefore, you should constantly assess the level of detail that the player sees to find a balance between running complex lighting shader algorithms versus rendering at an acceptable level of realism. Also keep in mind that this physics-based approach doesn't mean that these rules are written in stone. As a game developer/artist, you are free to bend the rules to achieve a certain look, meet performance requirements, and reduce pipeline complexity to provide your players with the best experience for your game concept.
Developers can get started with rendering on Snapdragon using the Adreno GPU SDK, which is available for Vulkan 1.0, and the Snapdragon 855 HDK.
