GlTF PBR

Opensimulator: Documentation: Asset types: GlTF PBR

Both Second Life and Open Simulator are adding a new class of assets for physically-based rendering. For an overview, see PBR Materials in the Second Lif Wiki.[1] This Wiki entry covers how the data about physically based materials is conveyed from server to viewer.

Object updates

Object updates can have "extra parameters", for objects such as lights and flexis. For glTF materials, two new types of "extra parameter" have been added.

Render Material

Extra parameter code: 0x80.

Content: an array of (face index, UUID).

An example is

• LLRenderMaterialParamsItem { idx: 0, id: a99f26b5-1cc1-40bc-a666-a6825592b188 },
• LLRenderMaterialParamsItem { idx: 1, id: a99f26b5-1cc1-40bc-a666-a6825592b188 },
• LLRenderMaterialParamsItem { idx: 2, id: a99f26b5-1cc1-40bc-a666-a6825592b188 },
• LLRenderMaterialParamsItem { idx: 3, id: a99f26b5-1cc1-40bc-a666-a6825592b188 },
• LLRenderMaterialParamsItem { idx: 4, id: a99f26b5-1cc1-40bc-a666-a6825592b188 },
• LLRenderMaterialParamsItem { idx: 5, id: a99f26b5-1cc1-40bc-a666-a6825592b188 }

This indicates that material UUID a99f26b5-1cc1-40bc-a666-a6825592b188 is to be applied to faces 0..5, overriding any classic material information.

All zero UUIDs have been seen in Second Life messages but not in Open Simulator messages. This may be a bug.

The asset for this UUID is fetched using the "ViewerAsset" capability obtained from the seed capability. This will be a URL with no trailing slash. An example from an OSGrid test region:

ViewerAsset": "http://plaza16.osgrid.org:9300/147ae4d1-b064-442c-80b3-f76b59e3fc71

The UUID points to a remote asset stored in binary LLSD format, with a proper binary LLSD header. This contains information for finding the relevant glTF assets. The format of the URL to fetch is

   http://VIEWERASSETVALUE?material_id=UUID

which, in this case, would be

   http://plaza16.osgrid.org:9300/75357890-ffde-4d96-8473-a663238dcd21?material_id=a99f26b5-1cc1-40bc-a666-a6825592b188

Note that, for Open Simulator, these URLs are only valid for the duration of the login. For Second LIfe, they are persistent.

Reading this URL yields a set of key-value pairs in binary LLSD format, with keys such as "baseColor" and "pbrMetallicRoughness". The values are more UUIDs, requiring another stage of asset fetching.

Example

This was translated from binary LLSD to XML LLSD for readability. SL viewer code indicates that both XML and binary forms of LLSD should be accepted.

   <?xml version="1.0" encoding="UTF-8"?>
   <llsd>
   <map>
       <key>data</key>
   <string>{"asset":{"version":"2.0"},"images":[{"uri":"d1f91bb7-f7d6-d26f-e0d7-568f0ff74279"},{"uri":"d1f91bb7-f7d6-d26f-e0d7-568f0ff74279"},{"uri":"8a45c99a-cf84-77c5-9d9d-5c9875213fe9"}],"materials":[{"normalTexture":{"index":2},"pbrMetallicRoughness":{"baseColorTexture":{"index":0},"metallicRoughnessTexture":{"index":1}}}],"textures":[{"source":0},{"source":1},{"source":2}]}
   </string>
       <key>type</key>
       <string>GLTF 2.0</string>
       <key>version</key>
       <string>1.0</string>
   </map>
   </llsd>

The LLSD contains a string which is glTF data in JSON format.

   {"asset":
       {"version":"2.0"},"images":[
           {"uri":"d1f91bb7-f7d6-d26f-e0d7-568f0ff74279"},
           {"uri":"d1f91bb7-f7d6-d26f-e0d7-568f0ff74279"},
           {"uri":"8a45c99a-cf84-77c5-9d9d-5c9875213fe9"}],
   "materials":[
       {
           "normalTexture":{"index":2},
           "pbrMetallicRoughness":{"baseColorTexture":{"index":0},metallicRoughnessTexture":{"index":1}}}],
   "textures":[{"source":0},{"source":1},{"source":2}]
   }

The format of this data corresponds roughly to the official glTF format specification for materials.. The formal schema for that JSON is in section A22 - JSON Schema for Material.

Note that this is not the same as the glTF uploaded by users. That has filenames in the "uri" field. Somewhere during the upload process, URIs are generated for those files.

glTF overrides

Changes underway

Linden Lab is, as of late August 2023, revising how texture scale. rotation, offset, and UUID information is sent from simulator to viewer. The initial design, below, generates so much data per change that rapid changes overload the simulator to viewer link. There's also the problem that the UDP information that an object has a PBR material and the paramteters necessary to display it come in via separate channels and not necessarily in order. This leads to some potential race conditions. Information on the new data formats is not yet available.

Original implementation, under review

Standard glTF material format does not contain texture scale, rotation, or offset information. Those are properties of a face of an in-world object, not the material. That information is sent to the viewer as "glTF overrides".

These are sent to the viewer via the Event Queue system.[2]. Along with other events, they show up as LLSD items, with a key "GLTFMaterialOverride". The content is encapsulated in glTF "notation" format, which in turn encapsulates glTF in JSON format.

A raw sample follows. (This will be replaced with a cleaner format).

<llsd><map><key>events</key><array><map><key>body</key><map><key>AgentData</key><array><map><key>AgentID</key><uuid>b5fde908-eebf-4505-b686-ca74c3f91979</uuid><key>SessionID</key><uuid /><key>TransactionID</key><uuid /></map></array><key>MethodData</key><array><map><key>Invoice</key><uuid /><key>Method</key><string>GLTFMaterialOverride</string></map></array><key>ParamList</key>
<array><map><key>Parameter</key><string>
<? llsd/notation ?>
{'gltf_json':['{"asset":{"version":"2.0"},"images":[{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"}],
"materials":[{"emissiveTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":3},"normalTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":1},"occlusionTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":4},"pbrMetallicRoughness":{"baseColorTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":0},"metallicRoughnessTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":2}}}],"textures":[{"source":0},{"source":1},{"source":2},{"source":3},{"source":4}]}\n','{"asset":{"version":"2.0"},"images":[{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"}],"materials":[{"emissiveTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":3},"normalTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":1},"occlusionTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":4},"pbrMetallicRoughness":{"baseColorTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":0},"metallicRoughnessTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":2}}}],"textures":[{"source":0},{"source":1},{"source":2},{"source":3},{"source":4}]}\n','{"asset":{"version":"2.0"},"images":[{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"}],"materials":[{"emissiveTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":3},"normalTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":1},"occlusionTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":4},"pbrMetallicRoughness":{"baseColorTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":0},"metallicRoughnessTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":2}}}],"textures":[{"source":0},{"source":1},{"source":2},{"source":3},{"source":4}]}\n','{"asset":{"version":"2.0"},"images":[{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000000"}],"materials":[{"emissiveTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":3},"normalTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":1},"occlusionTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":4},"pbrMetallicRoughness":{"baseColorTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":0},"metallicRoughnessTexture":{"extensions":{"KHR_texture_transform":{"offset":[0.0,0.0],"rotation":0.0,"scale":[3.0,2.0]}},"index":2}}}],"textures":[{"source":0},{"source":1},{"source":2},{"source":3},{"source":4}]}\n','{"asset":{"version":"2.0"},"images":[{"uri":"00000000-0000-0000-0000-000000000000"},{"uri":"00000000-0000-0000-0000-000000000

Reflection Probe

Extra parameter code: 0x90.

Content: One entry of (ambience, clip distance, flags)

More information is needed.