What object is usually made of wood, is a canvas for an artistic presentation, is tricky to fit through a doorway, and can make for a memorable day with a child?

In "Philosophical Investigations"[1], Wittgenstein put forward a "Tool" Theory of Language. In that theory, to understand a word, one must understand its use, or how it exists within an activity[2]. When a carpenter reaches for his hammer, he may be ready to drive a nail into a joist. When an auctioneer says, "Sold!", he brings down his hammer. The objects of course resemble each other, but the work that the linguistic term "hammer" is doing in each of these situations is determined by the task or ritual it participates in. In one case, it is a tool for transferring an impulse into a fastener, in other it is a social convention. These are different games, to use Wittgenstein's term, and these games cascade meaning to its participants.

Our discussion of world documents and OpenUSD, continuing from The Paper Golem, applies this tool theory to three-dimensional assets.

If you haven't guessed it, the riddle in the first paragraph applies four games to a table[3], whose representation in a scene is fairly straightforward. Using OpenUSD, a table would normally have a path to distinguish it from all other objects (including other tables), a transform to locate it in space relative to its parent, a bounding box for culling, tessellations for geometry, materials for appearance, and other attached attributes to support any given kind of experience.

The first game is an authoring game. It might be a workshop, where the carpenter assembles the table from its constituent parts, or the creation of that form in a CAD system[4]. In such games, history, topology, boundary math, and judgement produce the control points, topology sets, and material selections that physically make up the table.

The second game is an integration game. It might be for a wedding reception, a mundane surface for the decorator to place an elaborate centerpiece. The table might be covered in a long cloth, obscuring the materials and shape that were so important when authoring. Here, what matters is that the table exists in a given room and provides a surface upon which the centerpiece is placed. The cascading rules of OpenUSD[5] determine the position, orientation, and occlusion of the table.

The third game is a simulation game[6]. It might be an attempt to move the table around a corner or through a doorway. Collision, navigation, and physics determine the outcome. The physics may be defined using a simple collider capsule, a convex hull, a discretization of colliders, or even higher fidelity methods capable of modeling damage to the doorway or table, depending on the goal of the simulation. I use a coined term artiphysics: the physical rules associated with a given definition of a world. In the world we live in or a perfect simulation, the artiphysics is the physics we are familiar with. In all other worlds, an artiphysics affects entities through interactions that define its next state relative to its current state and other entities, rather than its presentation or composition with the rest of the scene. Importantly, OpenUSD describes the properties of the relevant physics, while some external >solver actually performs the necessary calculatations.

The last game is play. As the name implies, this is something of a special category[7]. The child might treat the table as a fort, or a bed, or a sled, or a secret lair. Here, the rules that define the experience with the world are themselves determined as part of the act of play. This is more of a meta-game than a game, and in my definition of world documents - a spatial representation that is experienced through rules - these sorts of games are foundational. Games are conrete instantiations of play[8]. It is therefore of no great surprise that given that learning occurs both in humans and artificial intelligence systems through play, the world document defined by games is the substrate for entertainment, robotics, and physical intelligence.

OpenUSD's most fundamental insight, whether or not it was intentional, is that spatial representations can experience or be experienced by any arbitrary game. The Prim, the base unit of information in an OpenUSD document, doesn't have one meaning, it is that which can be played with. This is more general even than HTML, where an element, its basic unit of information, is that which can be rendered. In HTML, there is one game in town, and the name of the game is presentation. In OpenUSD, the game is whatever the player(s) want it to be. I think this is where the comparison with HTML does not do OpenUSD justice.

Walt Whitman wrote, "Do I contradict myself? Very well then I contradict myself, (I am large, I contain multitudes.)" He understood that within an individual lives contradicting drives, desires, identity, and belief. In world documents, OpenUSD imposes these contradictions on each and every Prim. There is no "true" Prim, no platonic form, no logically coherent and internally consistent understanding of it. Instead, the prim experiences these contradictions as expressions of a coherent game. What makes each world, or stage, coherent is that it follows from one game (even fractally so, a game can contain a game can contain a game), and that the pieces of any possible game can be expressed as prims on a stage.

Though it is a substrate for these and all possible games, OpenUSD remains entirely declarative. That means that the gameplay is left unspecified. Vendor-specific Application Environments (for example Unreal, Omniverse, and Maya, Reality Composer) offer those mechanics for these world documents, but they each have a different opinion on how the gameplay, or behavior, is realized. [The Paper Golem] discussed how JavaScript took on the behavior role for paper documents[9] in an imperative fashion, ushering in a sort of dark ages for HTML, where it could no longer represent the paper document without the intervention of an animating force.

For OpenUSD, Application Environments host disparate runtimes that perform operations on Prims. Declarative patterns are already re-emerging as higher level User Experience choices within each environment, but these patterns are not affecting the standard directly, much like React.

There are countless situations, or games, world documents could be useful for, and in each of these situations, the gameplay must be implemented. Until these declarative behavior patterns become something more open, more standard, and more understood, these implementations will have to be done with great difficulty[10].

How these declarative patterns could influence the evolution of OpenUSD in subtle ways, but it will be a long time before then. In the meantime, animators, roboticists, and engineers must find productive uses for these world documents.

In the next installment, I will introduce my taxonomy of artiphysics, their design, implementation, and assumptions, their history and their natural participation in world documents.