So here’s something that might be interesting – a Youtube original series called “DR0NE”, about a military combat robot in trouble.
Although one thing bugged me about it very quickly: in the protagonist’s view of the combat scenes, there are target reticules over both friendlies and enemies. These reticules are almost absolutely identical for both sides; the friendly ones are, to my eyes, MORE visually interesting because they have “FRIENDLY” written by them, while the enemies just have anonymous reticules.
Moreover, there’s a lot of scenes where the enemies are against a bright sky – and the white reticule is almost completely invisible against it. This is just some terrible information design.
So I guess this leads me to propose two theories of information overlay design:
1. Important information should be more prominent than unimportant information. In the example here of a combat HUD, for instance, the reticules over enemies should be much more noticeable than those over friendlies, which can be achieved by color, shape, and amount of detail.
2. Your HUD should always be visible over the world. If you want it to be all white as in this example (which I would not recommend, color is an important cue) then find SOME way to make your information appear over bright things – maybe a black outline behind it, maybe having mission-critical stuff blink between black and white, SOMETHING.
I’ve been thinking about this sort of thing a lot, since I’m constantly superimposing bits of Rita1’s HUD on the world. I do things like big gaudy triangles with exclamation points hovering over the heads of hostiles to make it a pretty unambiguous communication to both the reader and to Rita that HERE IS A VERY IMPORTANT THING, I’ll hint at color changes here and there – admittedly I’m constrained in that a lot by the abstracted nature of the comic; if I was doing more representational color, you’d definitely be seeing bright reds and yellows as cues for DANGER!!! and CAUTION!!.
I’m not entirely sure this little piece is worth critiquing – but heck, seeing it made me codify a couple of the unconscious rules I’d been applying in the comic, so I guess that’s good!
It’s also worth noting that this HUD is MARGINALLY better than the Terminator’s; here’s a compilation of all those scenes from Terminator 2. Notice how pretty much ALL the text overlays are out on the periphery of the screen, drawing your attention away from what’s happening? Bad, bad, bad design, at least from the point of view of “would this work for an enhanced human”.
(Also there is the plot hole where the drone is clearly shown to be able to see through walls now and then, but is then surrounded. I guess its heat-sensing camera takes power it doesn’t have? Or the environment it’s in before that is really IR-opaque? I dunno, it bugs me that they spend several shots establishing quite firmly that it can see through walls, then have him ambushed.)
I spent a lot of time, energy, and paper designing and dealing with best practices for interface design, particularly in contexts where virtual and augmented reality were concerned.
Generally, your stuff has been head and shoulders above most other interfaces I’ve seen in film and fiction, which tend to be hollywoodized rather badly, so kudos on that. :)
I guarantee that you will never see Carol HACKING TEH GIBS0N.
Mostly I’m just trying to make something realistically understated. I think one of the leading problems we face right now is information overload; AR can add a lot of stuff but it can also add too much!
(And I didn’t know that about you! I’d love to have some pointers to interesting work in AR UI. I skim /r/augmentedreality but that’s pretty much dead nowadays…)
Hmm, those are great principles for making HUDs for humans, or other beings whose visual perceptions are (roughly speaking) limited to a 2D planar image with a few specific features per pixel (e.g. much of human vision could be described using 6 features per pixel: red, green, blue, depth, horizontal motion, vertical motion). Such beings have the problem that any new features (e.g. friendness, enemyness, RF emissions spectra) must be crammed into the existing limited feature space, causing information loss and ambiguities. Suppose you place a big red arrow next to enemies – it grabs your attention, but the arrow will also block your view of whatever is next to the enemy. So you need principles like yours to reduce the information loss as much as possible.
A robot drone or a rita1 can presumably do much better by simply having an expanded perceptual space with a larger number of features. Just like us they would sense “redness”, “near/farness”, and “leftward motion”, but in the same way they would also sense “friendness”, “x-radiationness”, “wifiaccesspointness”, and other useful features. Then there is no information loss or ambiguity. Friendness would simply be another perceptual sense seamlessly overlaid with the rest, instead of having to be represented by ‘false’ rgb objects that overwrite real data streaming in from the rgb senses. (Of course these extra features need not all be represented at a per-pixel level like “redness”. Some could be more like semantic tags applied to objects e.g. “friend” or “estimated time to intercept”).
In fact I had always assumed this is what rita1’s perceptions are really like, and that the HUDs you used depict this in your comic are merely a convenience for the benefit of the human readers, to give them a crude approximation of the true symphonic sensory experience, much like allowing a congenitally deaf person to get a portion of the experience of a musical play by giving them a text transcription of the song lyrics. What do you think?
I will be honest and say that I actually tend to think of a lot of Rita1’s HUD as fairly literal! I put a few readouts in the eyeline of other characters in the future world, but haven’t really been leaning on it as much as I should – especially because it’s mostly been in the context of Kim1 or Carol1 hacking, and they both seem to prefer to have a LOT of data displayed on floating rectangles. The only floating text in a bio-human’s sensorium I can think of offhand is Barrett1’s exercise counter, which I actually FORGOT to carry forwards into the rest of that scene. (It’s on my list of Things To Fix, but first I wanna get book 1 shipped!)
I also tend to see Rita1 as having a very divided consciousness in some respects. I haven’t gotten into this in the story yet (and I do plan to make some offhand references to it), but I basically see Rita1 and all the other software people as running what is essentially an emulated human brain. Some of them actually started life as biological people; some of them are purely software – but I figure most of the ones who were never flesh in any way are a merge of the mind-dumps of their parents*. Rita1 has some auxiliary processors that do a LOT of her tactics processing for her, and I see this data as being presented to her in such a way that makes it clear it is not coming from “herself” – it’s certainly part of the gestalt of “Rita1, the personality and memories” plus “the apps running on Rita1’s ‘reptile brain'”. Where “Rita” stops and “Rita’s favorite apps” begins is a blurry line, given that you can assume she keeps pretty much the same suite of stuff installed on all her bodies. But in general, a significant number of people like to have a layer of separation between “me” and “my apps” rather than blindly trusting a “friend-or-foeness”, and Rita1 is definitely one of these folks.**
This sort of comes naturally from one of the philosophical points I’m trying to make with this story. I really want Rita1 and the other espies to be seen as wholly human, rather than some kind of OTHER that humans have built. If we see the increasingly smart machines we’re building as Things that we can treat worse than slaves, then a Terminator scenario becomes increasingly likely. And I really don’t want one of those. Rita1’s world is one where we have the sense to start treating sentient machines like they are fully human; it’s pretty normal to start new software people out as, essentially, five-year-old kids***, and upgrade their hulls as they grow older and (hopefully) wiser. This results in a population of humans running on different hardware, who have been largely**** trained to see naked monkeys as “my tribe”, rather than a bunch of KILLBOTS WHO KILL MEATSACKS.
I haven’t had a chance to get into a lot of this, since I’m really trying to keep from getting bogged down in having people say “OBSERVE, FRIEND CITIZEN, HOW WE HAVE TAMED THE POWER OF THE ATOM!” and deliver a five-minute lecture about atomic power every time they turn on a light. I’ve got a conversation coming up in a couple chapters that will touch on some of this stuff. Plus auxilliary stuff like the tumblr (which is being excerpted in the printed book).
AND ON THE OTHER HAND: yes, there ARE senses that some of these people have that a 2012 bio-person doesn’t. In space, all the communication is over radio/laser/whatevs, people will have wider visual spectrums, there are people who will gleefully pipe their apps straight in beneath their consciousness and completely trust them. There’s really no One Way To Do It.
Well that got long, didn’t it. Complete with footnotes. Thanks for the thoughtful response!
* there have been some experiments with building minds from scratch, but it’s a complicated process, with some very grey ethical areas involved, and more than a few laws involved. Kind of similar to stem cell research nowadays.
** I am not sure if she was always like this, of if she’s gotten more vigilant about the boundaries of “herself” after the whole thing with the Vespuline Hive. I’m not even entirely sure she’d give me a straight answer if I asked; it feels like a complicated kind of choice.
*** only the most dedicated fetishists want to simulate programming a small potential sentience with language and not shitting itself.
**** just as with biological humans, there ARE exceptions, there ARE psychopaths out there.
Ah, those are good points! I’m impressed how much you’ve thought this through. That is a good point about having your espies purposefully grown from a stage with humanlike perceptions to encourage them to identify themselves as being humans. Also, that’s a good point about how adding extra sensory features (like friendness) as direct PERCEPTIONS, instead of separate tags or labels generated from a divided part of our consciousness, might make them too strong. When I percieve something as yellow I automatically believe and act as though I am certain there is really a yellow object there, but for something like friendness it would be better to have some mental distance (do not think “I am sure that is my friend”, instead think “my tactical system claims that is my friend”), especially in a world where your tactical system could be hacked or have a backdoor installed by your ex-hive.
Although, I still think this might be implemented with a separate overlaid feature space instead of creating ‘false’ arrow-like objects that overwrite part of the normal visual space. e.g. imagine a standard image with r g and b channels, and then instead of putting the HUD icons into the same rgb channels, just add more separate channels for friend/foeness, RF spectra, etc, and put the HUD icons there. After some experience, I am sure you could learn to treat the friendness channel of your visual input with the appropriate “mental distance”, just as you would learn to treat the arrows/boxes in your current depictions of the HUD as being artificial HUD icons instead of thinking that there are really floating blue arrows and boxes in space in front of you. Or I guess another compromise would be for the HUD to be in rgb space but to be partially transparent, as it is in many video games.
Actually, your ideas about divided consciousness are especially interesting to me because I think that even we vanilla humans have a divided consciousness in this way. After all, are you really aware of the minute changes in facial appearance that form each expression (e.g. “eyebrows raised to 70% of maximum, blink rate down to 50% below baseline”), or do you get an executive summary of their social meaning that just pops into your consciousness, coming from a specialized dedicated processor (e.g. “Jane is really interested in what I am saying, Joe is politely feigning interest, Alex is about to make an angry interjection”)? A little of both, I think.
Oh, and before I forget, I wanted to say that I completely agree that it is a very good point of your storytelling style that you simply directly immerse us in the fictional world, without injecting unnecessary, pace-debilitating, out-of-character “As you know, Rita…” and “POWER OF THE ATOM” speeches.
My social firmware was never installed properly, so I pretty much DO have to try to parse meaning from facial expressions as a forebrain activity! Not quite as low-level as your example, thankfully. But I am also aware that this is pretty much not normal.
Sometimes I think I err too far in the avoiding obvious exposition, and create confusion instead. I dunno. I generally prefer to trust that the reader is smart, and has run into some form of enough of the ideas I’m using already that they can put two and two together. This story’s deliberately constructed at a fast pace; if there hasn’t been any action for like 8 pages at any point then I know that SOMETHING EXCITING HAS TO HAPPEN RIGHT NOW. Doesn’t leave me much room for undigested expository lumps!
(As a side note, I get the phrase “undigested expository lumps” from Jo Clayton’s “Skeen” trilogy; the chapter titles in these books start getting longer and more detailed, until Clayton is talking directly to the reader. At one point she says something like “here, have an undigested expository lump” and goes on for like five pages (in a large display font so probably more like one page in normal text) about some important plot information that she couldn’t be bothered to work in more subtly. It is awesome in its shamelessness.)