codingxr.com Open in urlscan Pro
172.104.143.130  Public Scan

Form analysis
1 forms found in the DOM

GET /search/

<form action="/search/" method="GET">
  <input type="text" name="query" class="query">
  <input type="submit" value="Search" class="search-button" title="Search">
</form>

Text Content

Coding XR Show/Hide Mobile Menu
 * Articles
 * About


SHADERS IN REALITYKIT

Jan 14, 2022


In RealityKit 2 Apple now allows you to create vertex and fragment shaders for
materials. You specify the shaders using CustomMaterial, similar to how you add
shaders to ShaderMaterial in Three.js. But one major difference with
RealityKit's approach is that you modify properties of the existing material
types rather than having to start from scratch. You can create a custom material
from an existing base material:

let mtlLibrary = MTLCreateSystemDefaultDevice()!.makeDefaultLibrary()!

let surfaceShader = CustomMaterial.SurfaceShader(named: "mySurfaceShader", in: mtlLibrary)
let geometryModifer = CustomMaterial.GeometryModifier(named: "myGeometryModifer", in: mtlLibrary)

var baseMaterial = PhysicallyBasedMaterial()
baseMaterial.baseColor = PhysicallyBasedMaterial.BaseColor(tint: UIColor.blue)

var customMaterial = try! CustomMaterial(from: baseMaterial, surfaceShader: surfaceShader, geometryModifier: geometryModifer)

The above code doesn't actually create a blue custom material - you would still
have to set the base color in the shader but you can access the properties of
the base material in the shader. Because of this it's probably clearer to use
the constructor where you specify the lighting model:

var customMaterial = try! CustomMaterial(surfaceShader: surfaceShader, geometryModifier: geometryModifer, lightingModel: .lit)

The lighting model options are: .lit: PBR (physically based rendering) without
clearcoat, .clearcoat: PBR with clearcoat, .unlit: renders without shading or
lighting, similar to UnlitMaterial.

RealityKit's name for fragment shader is surface shader, and for vertex shader
is geometry modifier. RealityKit is quite restrictive in what parameters you can
pass from Swift to the shaders. You can only pass a vector with 4 components and
a texture. The vector can be treated as 4 floats if you want. If that's not
enough you could set the standard material properties and access them inside the
shader. To pass a float you would do this:

customMaterial.custom.value[0] = 0.1

To pass a 4 component vector (SIMD4<Float>) you would do this:

customMaterial.custom.value = [0, 0, 0, 0]

To pass a texture, you would first create a texture set in the asset catalogue
and then do this:

if let resource = try? TextureResource.load(named: "MyTexture") {
  let texture = CustomMaterial.Texture(resource)
  customMaterial.custom.texture = .init(texture)
}

Next you need to create the shaders. You do this by creating a Metal file (File
> New > Metal File). Metal Shader Language (MSL) is written in C++ 14. The
surface shader and geometry modifier can be in the same file but the functions
need to have a visible attribute. Attributes in C++ are written like this:
[[visible]]. The name of the function needs to match what you specified above
when creating CustomMaterial.SurfaceShader() and
CustomMaterial.GeometryModifier() and each function needs to be passed a
particular parameter as you can see below. The details of these parameters are
specified here.

#include <metal_stdlib>
#include <RealityKit/RealityKit.h>

using namespace metal;

[[visible]]
void mySurfaceShader(realitykit::surface_parameters params) {
  // Get the tint of the base material
  half3 color = (half3)params.material_constants().base_color_tint();
  // Set the base color to the tint
  params.surface().set_base_color(color)
  // Set other material parameters
  params.surface().set_metallic(1);
  params.surface().set_roughness(0.5);
}

[[visible]]
void myGeometryModifer(realitykit::geometry_parameters params) {
  // Move each vertex out by 0.1
  params.geometry().set_model_position_offset(params.geometry().normal() * 0.1)
}

You don't have the type half in GLSL - it's basically a half-precision float.
Instead of vec2, vec3, vec4 that you have in GLSL, you have float2, float3,
float4 or half2, half3, half4. MSL isn't fussy about always adding decimal
points to float literals.

To access a custom parameter in a shader you would do this:

float customParameter = params.uniforms().custom_parameter()[0];

// or

float4 customParameter = params.uniforms().custom_parameter();

To access a custom texture you have to define a sampler and then use it to grab
a pixel from it:

constexpr sampler textureSampler(coord::normalized, address::repeat, filter::linear, mip_filter::linear);

[[visible]]
void mySurfaceShader(realitykit::surface_parameters params) {
  float2 uv = params.geometry().uv0();
  half3 color = params.textures().custom().sample(textureSampler, uv).rgb;
  params.surface().set_base_color(color);
}

You can access time (the number of seconds since RealityKit began rendering the
current scene) inside a shader with params.uniforms().time() so you don't have
to pass it in from Swift.


RELATED LINKS

 * Modifying RealityKit Rendering Using Custom Materials
 * RealityKit Custom Shader API
 * Metal Shading Language Specification


TAGS

 * RealityKit
 * Shaders

 * Contact