Definitions, derivations, notes, and long tables start folded.
How Splashery is built
Everything else in this manual is about what a Splash is. This chapter is about the program that holds the Splashes: where its files are, why it is made of plain files with no build step, how its own renderer draws a frame, and every outside library that ships with the site. The file names are real. Open the source beside this page and you can follow along.
Plain files, and nothing else
Splashery is static files and ES modules. There is no bundler (nothing turns many
files into one), no server program, no database, no content delivery network, and no key to
a paid service. A web page asks for index.html; that page asks for
src/app.js; and the browser follows each import line to the next
file, one request each. The site is served as it sits in the repository, and to run it on
your own computer you start a file server in the folder, such as the source's own
node tools/serve.mjs 4401 /splashery-experimental=. (Chapter 2's
Run it yourself). package.json has no build step:
npm ci installs only the tools that check and format the code (Playwright and
Prettier).
Why this way? No bundler, no server
Our choice It is one of the project's rules, kept on purpose.
Four things follow from it. A Splash can be read: the file you open in the
browser's tools is the file in the repository, not a squeezed copy. The site
costs nothing to run beyond free file hosting, and has no application server of its
own that could fail. (The files still have to be served by some host, or by a file server
on your own computer, and the page loads them over the network, so it can still be
unreachable when the host or the network is down.) Your scenes and files
stay on your device, because there is no server to send them to (the Terms of use
in the About tab say so). And a page that embeds a Splash (embed/index.html,
the <splashery-toy> element in src/element.js) needs only
a few static files.
If it were chosen differently. A bundler would make the first load smaller and faster (one file instead of hundreds, with unused code removed). A server would allow short links, saved accounts, and heavy work done off the phone. Each costs something the four points above give for free. The cost of the choice today is that a rarely used part must be loaded by hand, only when its Splash opens (see the libraries below).
Where it lives: index.html, package.json,
tools/serve.mjs (the file server, which the checks use too), and the
project's rules.
Where the files are
| Folder or file | What is in it |
|---|---|
index.html, styles.css |
The app: one page that holds the whole interface as plain HTML, its styles, and a small import map. |
src/ |
The program. The core is a few files: boot.js (reads the Splash's id
from the link and starts the two workers), app.js (the app shell: it
builds the stage, picks the detail tier, reads the link, builds the shelf and the
panels, and shows the first Splash), stage.js (the one place that owns
the canvas, the renderer, the camera, and the workers), and
toys.js (the shelf, with each Splash's opening view).
|
src/gl/ |
The renderer: renderer.js (the draw), shaders.js (the
shader code), and depth.js (the depth pass that moving splats need for
sorting). "Splashery's own renderer", below, follows a frame through them.
|
src/workers/ |
Work kept off the page's thread: data.js reads and builds captures, and
sort.js sorts the splats by depth.
|
src/engine/, src/lib/ |
What the stage and the app share: the orbit camera (camera.js), the
detail tiers (detail.js), the readers for .sog files and
for files you open yourself (sog.js, own-splats.js), and
the code that builds a kit or rigged Splash for the stage.
|
src/ui/ |
The interface: the shelf, Effects, the Splash sheet, Record, Hands-on, focus mode,
the stage's buttons, and each Splash's splat counts (toy-counts.js).
|
src/lab/, src/studio/, src/arcade-host/
|
Systems loaded only when a Splash that needs them opens: the lab Splashes' live screens, picture surfaces, and chunks; the photo and video tools, the 360° photo, the QR tools, and the sharp relief; and the host that runs the Arcade's games over the stage. |
src/splashery/ |
Splashery's own code that does not touch the renderer: the kit with
generators.js (Chapter 2), the effects' shader code
(effects.js), the scene codec and state (codec.js,
state.js; see "Share links and QR codes"), the sound engine, and the
catalog (toys.js). Its folders follow.
|
src/splashery/packs/ |
The Splashes' recipes, one file for each family of Splashes (about a hundred files):
balls.js, food.js, splat-equation.js
(the language of Chapter 3), and so on. A recipe is the JavaScript of Chapter 2;
src/splashery/toys.js lists each Splash with the pack it comes from and
whether it is behind the labs switch.
|
src/splashery/arcade/, fluids/, physics/,
qr/, live/, and other folders
|
Bigger systems that more than one Splash shares, each in its own folder: the games, the liquids and flames, the hands-on physics, QR codes, and the microphone and camera. Most are loaded only when a Splash that needs them opens. |
lab/ |
The lab hub: the viewing tools (views/; see
Using Splashery), the measuring panel
(measure/), and the Sound Lab (sound/).
|
assets/toys/ |
Each Splash's own files: its thumbnail, for some captures the splat files, and the
sounds and data some Splashes read. Most captures are served from outside the
repository, from a second assets site and from Cloudflare R2. Every asset's license
is in CREDITS.md.
|
vendor/, src/vendor/ |
Other people's libraries, copied in as they were published. LICENSES.md
records each one, and any change made to it. The next two sections list them all.
|
tools/ |
Node scripts that run on a build machine and are never loaded by the site: they
check the site, measure it, report on one Splash (toy-doctor.mjs is a
report card), sweep the splat counts, prepare assets, build the site's pages
(tools/site-build.mjs), and print this manual's PDF
(tools/manual-pdf.mjs).
|
tools/check.mjs, tools/parity.mjs |
The checks, which open the real site in a real browser. check.mjs
drives Chromium through Playwright in six parts (the captures, kit Splashes, the
whole shelf, the old links, the core, and the Studio and labs).
parity.mjs opens Splashes and records whether each is on the shelf,
loads, draws, takes a tap, and logs no errors.
|
docs/ |
The project's notes: how it works (HOW-IT-WORKS.md), every change and
why (EXPERIMENTAL.md), the engine notes (engine/), one
file for each class of problem met (cases/), and the license sheets
(licenses/).
|
embed/, showcase/, site/,
manual/, pdf-lab/
|
The other pages of the site: the embed player, the Showcase, the guides and Learn
hub (site/), this manual, and the PDF lab.
|
Workers, and the problems already met
Heavy work runs off the page's thread, so the page keeps drawing while it goes. The
data worker reads and builds captures, and the sort worker puts the splats in
order. Other workers do one job each: Picture QR's weave, the QR Code Splash's splats, 3D
World's scenes, the depth model, Shape Morph's forms, Walnut CT's lattice, and the piano
listener. docs/cases/ holds one file for each class of problem met so far, with
the symptom, how to detect it, the root cause, the fix, and the Splashes it hit.
Splashery's own renderer
Splashery draws its splats with a renderer of its own, written for WebGL2 and kept in
src/gl/. No outside graphics engine is in the site: the library list below has
none. Chapter 1 follows one splat from 3D to the screen; this section follows a whole frame.
The stage's loop runs once for every animation frame:
- The camera. It eases the zoom, coasts a flick, and turns the turntable after a pause.
- The sorted order. If the view has turned by more than about half a degree, it asks the sort worker for a new order. A frame draws with the last order it has, and a new one is applied when it arrives.
- The effects. It works out the frame's effects: Poke, Wind, a kit Splash's own motion, and the rest.
- The draw. Only if something changed since the last drawn frame (the view, the effects' clock, the canvas size, or the order). The renderer draws one instanced quad for each splat, back to front, with premultiplied blending.
- The overlay. It draws what goes on top, such as the labels of the layer pull-apart, and then tells the listeners.
| The part | What it does |
|---|---|
The renderer (SplatRenderer in src/gl/renderer.js) |
Binds the sorted order, then runs one instanced draw for each run of splats: the Splash, and any extra groups. Blending is premultiplied, back to front. |
The shaders (src/gl/shaders.js) |
Carry each splat's 3D covariance through the view and the perspective to a 2D one,
which sets the quad's axes (Chapter 1). A splat that is too faint, too small, behind
the camera, or off screen is culled in the vertex shader. The effects' shader code
(src/splashery/effects.js) is spliced in as modifier functions, so the
same code moves a splat in the draw and in the depth pass.
|
The sort worker (src/workers/sort.js) |
A 16-bit counting sort of each splat's depth along the camera's forward axis,
farthest first. While an effect moves splats, the graphics chip works out the moved
depths with the effects' own shader code (src/gl/depth.js) and the
worker sorts those, so moving splats still blend in the right order. A kit Splash
whose depth passes find its splats where they rest is sorted from those resting
places on the worker when the view turns, with no depth pass and no read back (the
still sort). A probe pass every 250 milliseconds checks that they still rest, and an
action, a control, an effect, or Hands-on ends it at once. A Splash whose splats
move with the camera, such as 3D Photo, fails the check and keeps its passes.
|
The data worker (src/workers/data.js) |
Downloads a capture and decodes it, centers it and scales it to a radius of 1, builds the count the Splats slider asks for, and packs it for the graphics chip. |
Extra splat groups (stage.addSplats) |
Splats drawn beside the Splash with shader code of their own: a rig's add-ons (the Alarm Clock's hands), the Arcade's layer, picture sheets, Powers of Ten's chunks, the fluids, and Hands-on's water surface. The stage holds at most 31 groups at once. |
The orbit camera (src/lib/camera.js) |
The camera, and the touch gestures that move it. |
The numbers that keep a small splat visible are in docs/engine/BLUR.md. The
renderer adds 0.075 px² to the diagonal of each splat's 2D covariance, keeps the minor
axis's variance at least 0.025 px², and skips a splat whose major variance is under 0.125
px². The quad reaches 2√2 standard deviations, cut where the splat's alpha falls to 1/255. A
splat half a pixel across spreads 2π(0.25 + 0.075) = 2.04 px² of alpha (2.01 measured), so
it does not vanish. When Save image draws a picture 2 or 4 times as wide, these numbers grow
with it, so the picture has the same splats on more pixels.
So the renderer is ours, and so are the Splashes. Splashery is primarily splat based,
not entirely: the Sharp relief of 3D Photo, 3D Video, and Splat Mirror is a textured mesh of
triangles (src/studio/relief.js), Night Sky draws its sky, ground, stars, Moon,
Sun, and planets as textured meshes (only its constellation lines, its tap ring, and its
compass letters are splats), and the hands-on physics and the games' rules are ordinary
JavaScript with no drawing in them at all. (See "What else Splashery does now".)
Where it lives: src/gl/renderer.js (the draw),
src/gl/shaders.js (the shader code), src/workers/sort.js and
src/gl/depth.js (the sort), src/stage.js (the frame loop),
src/splashery/effects.js (the effects), and
docs/engine/ (BLUR.md, SORT.md,
SPLATS.md).
Every library that ships, and when it loads
These are all the outside libraries and models in the vendor/ folders
(vendor/, src/vendor/, and src/splashery/vendor/),
the only files in the site that we did not write. LICENSES.md has the full
notice and source for each. The rule is that a library is vendored, loaded only when the
Splash that needs it opens, and listed there before it is used. Nothing is fetched from a
content delivery network or an outside server, and no key is needed.
Library (folder in vendor/) |
Its job | License | When it loads |
|---|---|---|---|
gifenc 1.0.3 (gifenc) |
Writes an animated GIF when you export one. | MIT |
With the export code, which loads the first time something needs it: a Record take,
a saved GIF, or a Splash's own save (src/splashery/exports.js imports
it through the import map in index.html); it is small.
|
PDF.js 6.3.289 (pdfjs) |
Reads a PDF so its pages can become splats. | Apache-2.0 (its image decoders: BSD and MIT) | Only when you open a PDF (src/splashery/media.js). |
omggif 1.0.10 (omggif) |
Reads a GIF's frames in browsers that cannot decode them (Safari). | MIT | Only when you open a GIF in such a browser. |
ONNX Runtime Web 1.30.0 (onnxruntime-web) |
Runs a neural network on your device, in WebAssembly or, where the browser offers
WebGPU, on the graphics chip (a WebGPU build, ort.webgpu.min.mjs at 66
KB with a 26.8 MB runtime, sits beside the WebAssembly one).
|
MIT | Only when you open a photo or start a camera for depth, when a piano hears a recording, or when you open a recording in Note Rider. |
Depth Anything V2 Small (depth-anything-v2-small): int8, 27.3 MB, and
half precision (fp16), 49.6 MB
|
The model that guesses how far away each part of a photo is. The half-precision build runs on the graphics chip where the browser has WebGPU with shader-f16; the int8 build, on WebAssembly, is the fallback. | Apache-2.0 | Only when a Splash needs depth (3D Photo and its relatives). |
Basic Pitch model (basic-pitch, about 230 KB) |
The model that finds the notes in a recording. | Apache-2.0 | Only when you open a recording as your own song in Note Rider, or on a piano's song panel with "Any instrument" chosen. |
Piano note model (piano-transcription, about 41 MB) |
The model that finds the notes in a recording of a piano. | CC BY 4.0 | Only when a piano's song panel opens a recording with "Piano" chosen, on your device, in a worker. |
zxing-wasm 3.1.4, with zxing-cpp (zxing) |
A second, independent reader that checks a QR code. | MIT (the wrapper) and Apache-2.0 (zxing-cpp) | Only when Picture QR's Clear picture checks a code with a second reader. |
Splat.js 0.1.0 (splatjs) |
Trains a splat scene from the frames of a video. | MIT | Only when you open a video in 3D World (labs). |
Project Nayuki's QR Code generator (qrcodegen) |
Writes the QR code's modules. | MIT | Only when something needs a code drawn: the QR Code Splash, the Share tab's scene code, or Save PDF. |
jsQR 1.4.0 (jsqr) |
Reads a QR code back, to check that it scans. | Apache-2.0 | Only when a code is read back and the browser's own barcode reader is not used or cannot read it. The QR Code Splash tries the browser's reader first and falls back to jsQR; Picture QR, and the check that Save PDF runs on its code, ask for jsQR directly. The Share tab's scene code is not read back. |
ZXing for JavaScript 0.21.3 (zxing-js) |
Writes and reads Data Matrix and Aztec codes. | Apache-2.0 | Only when the Other Barcodes Splash opens (labs). |
pdf-lib 1.17.1 (pdf-lib) |
Writes a PDF when you choose Save PDF. | MIT | Only when you save a PDF from the Share tab. |
laz-perf 0.0.7 (laz-perf) |
Reads a .laz point cloud. |
Apache-2.0 | Only when you open a .laz file in Point Clouds (labs). |
dicom-parser 1.8.21 (dicom-parser) |
Reads a DICOM medical scan. | MIT | Only when you open a DICOM file in the Volume Viewer (labs). |
libwebp, from splat-transform 3.6.1 (webp) |
Reads and writes the lossless pictures inside a .sog file. |
MIT and BSD-3-Clause |
When the data worker starts, since it reads every .sog capture; the
Splat Toolkit (labs) uses it too, to open or save a .sog.
|
Fredoka, bold, Latin subset (fredoka, about 15 KB) |
The rounded letters of "Splat Name". | SIL Open Font License 1.1 | Only when "Splat Name" opens (labs). |
The Earthquakes Splash and the Molecule Viewer's fetch by Protein Data Bank code do read
from outside the site while you use it, but they fetch data (a feed, a structure), not code,
only when you ask, and a dated snapshot of the earthquake feed ships for when it cannot be
reached. The Stork Migration Splash is not one of them: it reads a data file that ships with
the site. The outside download of that data happens earlier, when a tool in
tools/ builds the file on a build machine, not in your browser.
Sounds, maps, and captures are not libraries, so they are not in the table. Each carries its
credit where you can see it: in the header's credit line, in the About tab, and in
CREDITS.md. Everything Ryan Kamp made carries the name Ryan Kamp, with the
license it already had. The site's AI-made pictures are AI pictures by Ryan Kamp, and each
names the model that made it, beside the picture and in the site's credits; the model used
so far is ChatGPT Images 2.5. The credits list them under "AI pictures by Ryan Kamp" in
CREDITS.md. Nine Splash sounds play recordings from Kenney's Impact Sounds
(CC0). Night Sky draws from three files of its own: the Milky Way map from ESA/Gaia (CC
BY-SA 3.0 IGO), 20,720 fainter stars from the HYG database (CC BY-SA 4.0), and the Moon map
from NASA's LROC (public domain). Video QR's seven sample clips each carry their own credit:
Big Buck Bunny and Sintel (Blender Foundation, CC BY 3.0), Saturn V launch and Artemis II
lifts off (NASA, public domain in the United States), A Trip to the Moon (Georges Méliès,
public domain), Kīlauea lava fountain (USGS, public domain), and Rozi the cheetah (CC0 1.0).
Each captured Splash has its own credit.
What is used to build the site, but is not in it
package.json lists two pinned devDependencies: Playwright, which
drives Chromium for the checks, and Prettier, which formats the pages. PlayCanvas's
splat-transform tool, installed apart, packs edited captures into SOG files. None of them
is served; LICENSES.md, under "Build tools (not shipped)", lists them with
their licenses.
How a Splash gets on the shelf
Putting this together: a Splash is a recipe in a file in src/splashery/packs/,
one entry in src/splashery/toys.js (its id, label, category, pack, and whether
it is labs), its sound in src/splashery/toy-sounds.js, its help text in
src/splashery/toy-help.js, its splat counts in
src/ui/toy-counts.js, its opening view in src/toys.js, its files
in assets/toys/<id>/, and a row in
docs/licenses/toys.csv (written by tools/license-audit.mjs), which
lists each Splash's licenses, and in docs/licenses/redactions.csv, which
records the marks hidden on captures. A Splash opens when its pack is in
SYNCED_PACKS (src/splashery/synced-packs.js), and
tools/name-check.mjs checks that its name follows the rules for names (see
Using Splashery). Nothing is compiled or
registered anywhere else. The next time the page loads, the Splash is on the shelf, and the
checks find it. docs/TOY-CHECKLIST.md lists what to check before a change.
Where it lives: README.md, LICENSES.md, CREDITS.md,
docs/engine/KIT.md (how a recipe is built), and
docs/TOY-CHECKLIST.md.
One list of changes: the ?old= switches
docs/EXPERIMENTAL.md records each change made to the Playground it grew from,
with why it was made, the ?old= switch that brings the old behavior back, and
how it was measured. Every change keeps the old behavior one switch away: add
?old=blur to the address, or several at once, as in
?old=blur,zoom. To find why something behaves as it does, look up its switch in
that file.
The measuring panel
For people who make Splashes, ?measure=1 opens a measuring panel. It shows
frame times, the page's work for each frame, the graphics chip's time where the browser
offers it, long tasks, sort orders, one-frame flashes, canvas resizes, a zoom phase for
Splashes that zoom, the first ten seconds of a Splash, and the page's loading. Copy puts all
of it on the clipboard as JSON. Interact taps, drags, and slides a Splash's controls
by itself, and Watch records 10 seconds of a person's own play. A frame that takes
16.7 ms is 60 frames a second, so a frame time under that keeps up with a 60 Hz screen.
docs/cases/ lists the problems already met and the detector for each, and
tools/toy-doctor.mjs prints a report card for one Splash.