Panoramical developers Fernando Ramallo and David Kanaga take a deep dive into a simple question: How do you design and tune player input for a game that's also a music visualizer?
October 16, 2015
Author: by Fernando Ramallo, David Kanaga
Game Design Deep Dive is an ongoing Gamasutra series with the goal of shedding light on specific design features or mechanics within a video game, in order to show how seemingly simple, fundamental design decisions aren't really that simple at all.
Check out earlier installments on the heredity system of Hero Generations, traffic systems of Cities: Skylines, and the plant-growing mechanics of Grow Home.
Also, dig into our ever-growing Deep Dive archive for developer-minded features on everything from Amnesia's sanity meter to Alien: Isolation's save system.
Fernando is a designer who has worked on puzzle games like Adult Swim’s Cardboard Box Assembler as well as experimental software such as the drawing ecosystem Lienzo Fertil (“Fertile Canvas”).
David is a composer and designer best known for the interactive music in Proteus and Dyad.
Panoramical is a game in which players float through a variety of semi-animate landscapes, influencing the synchronized looks, musics, and feels of the worlds using an omnipresent ‘mixer’ of analog controls. Each of these landscapes, considered formally, is an 18-dimensional sculptural-orchestral manifold.
Two other kinds of manifold
In concrete terms, this means there are 18 ‘degrees of freedom’, the expressions of which the player is able to control as a conductor in a quasi-totalizing fantasy of cyborg unity.
Let’s consider briefly the size of the actual control possibility space before we move on to consider the less quantifiable interplay of the virtual and affective dimensions.
Panoramical feels smooth -- this is because each of the control dimensions creates the impression of apparently continuous -- or analog -- movement. If this continuity were really present, the space we are dealing with would be infinite, as in between any two real numbers there is already infinite ‘space’.
In fact, as with ‘analog sticks’, we are rather dealing with a fine-grained discrete space. Each dimension is split into 128 discrete chunks, as per the MIDI control protocol. Taking the number of units comprising each dimension (128), and hyper-hyper-etc-cubing it to the power of the space (18), the number of possible global control states is a dizzying 128 to the 18th power, or approx. 8.51 x 10 to the 37th power.
This is one description of the ‘formal possibility space’ of this game. As a description of the control space, it is accurate, but as a description of the game as a whole it is both too big and too little.
Let’s imagine ‘switching minds’ with the game. Now, the game is a world-sensing body, a classical empiricist, with 18 sensory organs. Its external world is YOU. This is the meaning of all the ‘disco-god’ theology, and the implicit atheology of videogames in general, that players become “the external world”, and as it were, they begin to peel the self-world relationship inside-out.
Panoramical has 128 to the 18th power possible bodily-input states. But the number of its possible outputs is greater than this, hinting at an ‘inner world’ of unspoken a priori concepts and emotions independent of the influence of the external world. To calculate these emotions, we need to consider every possible state that the software can express, including but not limited to the 18 control dimensions.
To begin: the game requires 1 GB of hard-drive space. This is of course 1 thousand million bytes, each byte of which is 8 bits, each bit of which exists in two possible states. A bit is analogous to a slider in Panoramical, and its 2 possible states analogous to the 128 possible states of MIDI-protocol control.
Knowing only the size of the file, we can say that the maximum number of states in Panoramical’s TotalSystem possibility space is 2 to the 8,000,000,000th power. When I plug this into calculators online to try to reduce to scientific notation, the answer provided is “infinity”, which of course isn’t true because you can’t get to infinity just by going far enough, but it gets the point across that this value is VERY BIG.
In any case, somewhere between 128 to the 18th power and 2 to the 8,000,000,000th power there exists the actual size of the game’s quantifiable possibility space. And those parts of which are greater than 128 to the 18th power correspond to the mental a prioris of the game space as a rationalist rather than an empiricist.
The 18 control dimensions are variously augumented and diminished by a variety of affective dimensions, both virtual (in the same sense that a mirror holds a ‘virtual world’) and actual. Actually, there is a time dimension that the player does not have control over. This is manifest in musical and visual loops. Virtually, the content which populates these loops exist, for a player, in dialogue with their own imaginative ecosystems. Altering the dimensionality of several elements suggests imaginative possible forms that do not exist, just like a an incomplete melody suggests cadential notes which may not be the ones actually used.
Qualitative virtual dimensions are implied by the ‘content’ which occupies the more formal quantitative control space 128 18 . Insofar as audio-visual content lives in forms which are invariant over a large slice of one dimensional axis, the affective magnitude of that dimension seems to shrink accordingly. And conversely, insofar as content affects players beyond the feeling of having changed a number, the dimensionality seems to grow accordingly. Furthermore, the presence of automatic ‘content’ in the forms of timeloops and objects and such means that the illusion of ‘total control’ must be replaced by one of cyborg dialogue with halfliving/halfdead (halfvariable/halfconstant) communities of nonhumans, operating in clock time.
All of the ‘content’ in Panoramical exists, as all artwork does, as an actual ‘point’ embedded in a larger implied virtual space. Radiating outward in 18 dimensions from this actual ‘point’, there are virtual circles which feel invariant to the player, and thus reduce the affective dimensionality of the game.
There is a famous scene from Edwin A. Abbot’s Flatland in which a sphere from the third dimension travels through the plane of flatland, and the flatlanders perceive the form as a point which appears, grows into a line, and then shrinks back down to a point (they see this 1dimensional projection rather than a circle because seeing the circle as circle rather than line requires movement above or below in the third dimension).
Imagine: 2 flatlanders are inspired to create two different video game simulations of this occurrence. One of them models an imaginary space in 3-dimensions with a knob controlling the ‘vertical’ position of the sphere, allowing it to sink all the way through the plane of flatland, appearing in the vision of the FLers as point then line then point. The other simply models a 1-dimensional line, also controlled by a knob, the length of which grows and shrinks. No one can tell their two games apart, despite fairly significant core formal differences.
The scale of an object has at least a twofold dimensional implication. The first corresponds to the Flatland parable, where apparent growth represents a different “slice” of a manydimensional being’s intersection into our 3-dimensional world.
The second corresponds to the everyday aesthetic experience of the fractal dimension, as described is Mandelbrot’s essay “Scalebound or Scaling Shapes: A Useful Distinction in the Visual Arts and in the Natural Sciences”. When we look to a hill in the distance, populated by tiny houses and trees, and imagine ourselves on it, this affects us with the everyday experience of fractal dimensionality and its intimate relationship to the sense of possibility; the fractal dimension describes relations between parts and wholes, and it is in the space between these that we are able to breathe and speculate and organize.
We made the original prototype by running Ableton and Unity alongside one another, each controlled independently by the same MIDI controller, sending instructions to both programs, which were blind to one another. If you’d like to imitate David’s early process, you can:
1. Use a MIDI controller
2. Turn off Audio from Panoramical
3. Map all 18 MIDI dimensions to live parameters in Ableton (or other music software)
In order to distribute the game, however, all audio needed to be handled by Unity. Fernando made generic sliders to control visual parameters and we used the exact same tools to change the sound parameters, such that the idea of audiovisual isomorphism was given a mechanical as well as a poetic meaning. The same ‘faucets’ spew sound and light.
In the desktop version we needed a way to let the player control the 18 dimensions. At first, we laid out XY axes across the bottom row of 9 keys, from “z” to “.” and it felt awful but we kept it like this for over a year. Eventually, we changed the controls to a 3x3 grid of keys QWE/ASD/ZXCl, and they felt great. You can reassign keys yourself if you want a firsthand experience of why the first try was no good.
Some have compared Panoramical to a Winamp visualizer. Such visualizers ‘listen’ to existing recordings, and on this account only have 2 dimensions of actual signal to work off of, amplitude through time, where even frequency is a derivative dimension. But the signal of a fixed recording is wholly Dead, time serving to reconstruct a spatial representation of life. Whereas Panoramical’s sliders are both dead and alive, with a dead looptime dimension intermixing with a living freecontrol dimension.
“What happened to Katzie? Where did he go?”
Soensa said, “Where do you come from?”
“From my mother’s belly.”
“Where does your mother come from?” Gita was silent.
Soensa said, “Everything in the world comes from the same one thing. It is like in a cookie factory. Many different kinds of cookies are made — lions, tigers, elephants, houses, people. They all have different shapes and different names, but they are all made from the same dough and they all taste the same. So all the different things that you see — a cat, a person, a tree, the sun, this floor — all these things are really the same.”
(from Dropping Ashes on the Buddha: The Teachings of Zen Master Seung Sahn)
You May Also Like
.png?width=300&auto=webp&quality=80&disable=upscale)
