ALab gaussian splat v1

.gitattributes

@@ -0,0 +1,4 @@
+ALab/extras/alab.ply filter=lfs diff=lfs merge=lfs -text
+ALab/extras/alab.usdc filter=lfs diff=lfs merge=lfs -text
+*.ply filter=lfs diff=lfs merge=lfs -text
+*.usdc filter=lfs diff=lfs merge=lfs -text

ALab/extras/alab_splat.usda

@@ -0,0 +1,71 @@
+#usda 1.0
+(
+    metersPerUnit = 0.01
+    defaultPrim = "root"
+)
+
+
+def Xform "root"
+{
+
+
+    def Xform "alab_splat_hrc" (
+        variants = {
+            string train_img_colorspace = "sdr-8bit-rec709"
+        }
+        prepend variantSets = "train_img_colorspace"
+    )
+    {
+        matrix4d xformOp:transform:transform1 = ( (156.62323975946586, -14.504396813283966, 14.925925108068377, 0), (13.835221110785671, -12.091359437091958, -156.92796335828334, 0), (15.548228968745558, 156.86752850748252, -10.715926273058074, 0), (68.63612747192383, 139.93829154968262, 112.14171981811523, 1) )
+        uniform token[] xformOpOrder = ["xformOp:transform:transform1"]
+        variantSet "train_img_colorspace" = {
+            "sdr-8bit-rec709" (
+            ) {
+                def "alab_splat" (
+                    variants = {
+                        string file_format = "usd"
+                    }
+                    prepend variantSets = "file_format"
+                )
+                {
+                    variantSet "file_format" = {
+                        "usd" (
+                            payload = @./alab_sdr_splat.usdc@
+                        ) {
+
+                        }
+                        "ply" (
+                            payload = @./alab_sdr_splat.ply@
+                        ) {
+
+                        }
+                    }
+                }                           
+            }
+            "hdr-8bit-rec2100" (
+            ) {
+                def "alab_splat" (
+                    variants = {
+                        string file_format = "usd"
+                    }
+                    prepend variantSets = "file_format"
+                )
+                {
+                    variantSet "file_format" = {
+                        "usd" (
+                            payload = @./alab_hdr_splat.usdc@
+                        ) {
+
+                        }
+                        "ply" (
+                            payload = @./alab_hdr_splat.ply@
+                        ) {
+
+                        }
+                    }
+                }
+            }
+        }
+    }
+
+}

ALab/extras/alab_splat_with_camera.usda

@@ -0,0 +1,18 @@
+#usda 1.0
+(
+    endTimeCode = 570
+    framesPerSecond = 24
+    metersPerUnit = 0.01
+    startTimeCode = 1
+    timeCodesPerSecond = 24
+    upAxis = "Y"
+    subLayers = [@./alab_splat.usda@]
+)
+
+def "camera" (
+    prepend references = @./camera.usda@</cameras>
+)
+{
+}
+
+

README.md

@@ -127,6 +127,19 @@ From the **Camera** > **Select Camera** menu, choose the required shot camera.
 
 ![select_cameras](docs/public/multiple_camera_prims.png)
 
+## Splat Pack (WIP)
+* This contains the ALab as a single gaussian splat in USD, using the new [Lightfields USD schema](https://openusd.org/release/user_guides/schemas/usdVol/ParticleField.html). We want to get an asset out to help with the testing and future development of particle fields in USD, starting with guassian splats!
+
+Files currently in this pack:
+- The original .ply file
+- A .usdc file of the ALab Splat
+- A .usda file of transformations to make the splat match up with ALab (this is temporary)
+- The USD camera for the rendered images used to generate the splat
+
+In order to successfully view the usdc file you will need to have buil the USD dev branch or wait for the 26.3 release, this includes the schema and renderer used here.
+
+More details on how to test with this at the moment are [here in the docs](docs/src/pages/gaussianSplat.md)
+
 # Contact & Feedback
 
 We are proud to have been accepted into the [Academy Software Foundation Digital Production Example Library](https://dpel.aswf.io). As such we have moved further conversations to the [#assets-alab](https://academysoftwarefdn.slack.com/channels/C03S4QB8N04) slack channel.

docs/src/pages/gaussianSplat.md

@@ -0,0 +1,94 @@
+
+# ALab Splat Pack Documentation
+
+This is the initial version of the ALab as gaussian splats, to help support development and adoption of gaussian splats in USD.
+
+The particleField UsdVol schema has now been merged into the dev branch, and will be available with USD 26.3
+
+We wanted to make this available for early testers and hopefully as a tool to help refine the splat schema work.
+If you're interested in getting involved with the development of the schema, you should join the [AOUSD Emerging Geometry Interest Group](https://aousd.org/community/interest-groups/#:~:text=VIEW%20CHARTER-,USD%20Emerging%20Geometry,-The%20USD%20Emerging)
+
+#### Note: you will need to install the adobe usd file format plugins to be able to load .ply files
+https://github.com/adobe/USD-Fileformat-plugins
+
+## Steps to view the ALab usdc splat in usdview
+
+1. Build the USD from the dev branch or USD 26.3
+
+		https://github.com/PixarAnimationStudios/OpenUSD/tree/dev
+
+2. Ensure the PXR_PLUGINPATH_NAME is extended to include 
+
+		share/usd/examples/plugin/hdParticleField/resources/plugInfo.json
+
+3. Clone the DPEL ALab repo - OR - if you're just interested in the splat, you just need the files in the extras directory
+
+   Note that the splat files use git LFS - https://github.com/git-lfs/git-lfs
+
+		/ALab/tree/main/ALab/extras
+
+5. The alab_sdr_splat.usdc and alab_hdr_splat.usdc files will have contents *like* this:
+
+		def ParticleField_3DGaussianSplat "alab"
+		{
+		    float3[] extent = [(-4.2353315, -4.928282, -4.960755), (4.912536, 1.9337461, 1.628153)]
+		    float[] opacities = << float[1502196] >>
+		    quatf[] orientations = << GfQuatf[1502196] >>
+		    point3f[] positions = << GfVec3f[1502196] >>
+		    float3[] radiance:sphericalHarmonicsCoefficients = << GfVec3f[24035136] >> (
+		        elementSize = 16
+		        interpolation = "vertex"
+		    )
+		    uniform int radiance:sphericalHarmonicsDegree = 3
+		    float3[] scales = << GfVec3f[1502196] >>
+		}
+
+6. Launch USDView and set the renderer to hdParticleField
+
+    usdview ALab/ALab/extras/gaussian_splats_with_camera.usda
+
+Now you should be able to open it in usdview like this:
+
+![gif](../../public/ALab_splat_in_usdview.gif)
+
+## Here's the SDR Splat in some other DCC
+
+### Nuke 17 beta
+![view](../../public/nuke_17_sdr.jpg)
+
+### Houdini 21 beta
+![view](../../public/hou_21_sdr.jpg)
+
+## How was the first version of the splat created?
+
+1. Used Houdini to do a quick ray cast setup test to try and define enough cameras to capture the majority of the scene (ending up with 570~ frames)
+2. Rendered 1k*1k squares of the ALab using Glimpse (NAS's proprietary renderer) with those cameras- these files were acescg exrs
+3. Converted these into .png (SDR and HDR variants)
+4. Fed these into COLMAP to quickly get an input data format that would work with existing gsplat implementations
+5. Used nerfstudio/gsplat implementation of 3dgs to splat with pretty standard settings for 30000 steps
+6. The result of that training produces the original .ply file
+
+## Colour Workflows!
+Eventually we would like to train the ALab splat on acescg 16 half float or 32 bit exrs.
+
+For the moment we have included SDR and HDR 8 bit png variants of the splat
+
+- SDR: sRGB - ACES 2.0 - SDR 100 nits (Rec709) (8 bit png)
+- HDR: Rec.2100-PQ - ACES-2.0 HDR 1000 nits (P3 D65 Limited) (8 bit png)
+
+## Future plans
+
+1. Train the splat using 16 bit half floating point acescg exrs
+	- Need a gaussian rasterizer to output 16 bit half float for the training
+	- colorspaceAPI wasn't part of the schema yet when we published this, but it's been suggested
+	- Clear documentation on the motivation and healthy acescg workflows
+
+2. The lightrig used for this splat isn't the exact same lighting that is publicly available, so we're considering the best way forward.
+3. We will be re-rendering and re-splatting at higher resolution once we decide on the lightrig to use
+4. COLMAP was just a shortcut at the time, but we will be taking that out of the workflow eventually, which means you wouldn't need the transform adjustments in the usda
+5. Ongoing work to decide the best production workflow colorspace to train splats in.
+6. Splatting with different shape kernels, e.g. 2dgs, and triangles
+7. Splatting with different falloffs besides gaussian
+8. Animated Splats of Remi and the Stoat
+9. Improve camera coverage for the hard to reach spots in the scene
+