Thinking outside the GUI

Filed under: thinking, alan-jacobs, structure, user-interface, social-media

Before I begin, despite how it seems, this is not a technical post.

GUI is short for Graphical User Interface - something that, I think, is a big part of the figurative 'box' in the wisdom of 'thinking outside the box'. Recently, I've been exploring data visualization - design of visuals for encoding and propagating information - leading me to take a closer look at the structure behind webpages and applications that we use regularly. More specifically, I'm pondering how the structural properties of Social Media networks like Twitter and Instagram or content distribution apps like Spotify effect my cognition (process of acquiring knowledge and understanding through thought, experience, and the senses).

User-Interface (UI) Structures Influence How We Think

It's analogous to Linguistic Relativity (Sapir–Whorf hypothesis) where the structure of language influence (or in a stronger version, completely determine ) our thoughts and decisions. I often hear people complaining about platforms like Twitter because of the behaviour of other users (free agents). Things like 'Everyone on Twitter jumps to conclusion!', 'there is so much rudeness and trolling that it's making me reactive', or 'all the ego on twitter is making me egotistical' - all of these complaints and others like them share the fact that the problems of Twitter arise from how others use it and, implicitly, that if everyone used it like us, it would be okay. But what about, aside from the manner of usage, the structure of the platform itself?

The obvious structural elements that jump out as influencing our thought are likes or retweets but our mainstream concern with them are actually more around their social properties. As an example, 'a high # of retweets make us think this idea is good' may seem like a structural property influencing our thought but that's actually the validation from people (a social property) influencing our thought. The structural property I'm referring to is more so the very idea of allowing such a mass validation to take place by the retweet button - might it effect our cognition that the platform allows us to validate anything by a click of a button? Or how are our tweets affected by the absence of a dislike button? In the field of media theory, some would argue analogous to a very strong version of linguistic relativity: that the medium (i.e. structure) is the message.

That is, the technology delivering the message — the printing press, television, radio, the Internet — inherently changes how we communicate, and, in doing that, alters the message, even tweaks the circuits in our brains, and thus transforms society. The message in the bottle means nothing compared to the bottle as message delivery service.

Structural Properties of UI

By structural properties I mean the arrangements and relations that facilitate the complex usage of the platforms. Things like ordering your timeline by time (it seems obvious.. but why?), allowing for threads to chain tweets, the separation of mentions into a separate tab, or having an option for public/private profiles. These are structural properties imbued into the very fabric of Twitter that we often judge based on their value to our convenience, our usability, or our accessibility, but not really in terms of their influence on our cognition - generally, we don't really consider how a most-recent ordering of tweets effects how we think about information or knowledge.. it just is. Yes, the structure alone, aside from any user's specific usage of it, can conflict with healthy thinking like developing a better 'temporal bandwidth' as Alan Jacob discusses.

An example that I'm particularly bothered about is Instagram's new feature of showing 'You're all caught up!' once you've scrolled down to the last picture you saw. One can value the convenience it provides and say, 'ah that's nice, it stops me from scrolling through things I already know'. Isn't that a little paradoxical and weird? If I 'already know it', why do I need to be reminded that 'you know it'? Maybe it says a little about the culture of ritualistic scrolling in the age of FOMO. But just by that small change in the user interface, Instagram has ascribed value to information, differentiating the 'don't bother with this' from the rest. Does this, in turn, effect how we evaluate information? Do I also begin to see knowledge like a news feed where my mind says 'stop, you're all caught up! Don't bother with the rest'? Or at the very least, is my thinking on Instagram constrained because of this? This is what I mean when I say we are stuck thinking inside the GUI. In the digital age, it is particularly difficult to think outside the constrains of the UI - we are, in a way, condemned to organize knowledge in the same methods that a particular online platform organizes information.

Becoming Informed of Structural Influences

The point is not to find some conspiracy brainwashing network - the application designers themselves may perhaps only be in pursuit of the oh so noble goals of usability, accessibility, convenience, etc. (Though some workers of big tech companies might have something to say about that). My point is not to play a moralistic role and damn certain structures over others (maybe this is an ethical step later). I simply think there is value in being aware of the effects of UI structures. From there, we can go on to critique and then change (hopefully for the better).

I don’t see the influence of structure as necessarily bad or good, (its influence is, in fact, inevitable) but I think we are less tuned to seeing how it plays out in online interfaces. As I said earlier, the UI designs for most of us are designs catering to our needs that we judge as good or bad - we think of ourselves as too "intelligent" to have our cognition influenced by the size of some Helvetica font... But sit down and really think about it, I think you'd be surprised.

I think it is up to you to decide if the way your thought is facilitated under a structure is fine with you - but I think it’s important to be informed. I am not throwing agency out the window. Yes, Twitter, Instagram and Facebook have structures, some of which cause you problems. However, you also have control to change how you deal with them. For that agency to be used, one needs to be aware of the structure. David Foster Wallace has a nice story to depict this:

There are two young fish swimming along who happen to meet an older fish. The older fish nods at them and says:

‘Morning boys, how’s the water?’

The two young fish swim on for a bit and then eventually one of them looks over at the other and asks:

‘What the hell is water?’

— David Foster Wallace.

Until you know the water you're swimming in, you can’t really decide how you want to interact with it. Our human agency is powerful and very much present, but only once we rise above the ‘spell’ of the structure can we truly begin to practice meaningful free choice. As in the case of Instagram's 'you're all caught up' feature, of course I can easily scroll past it and engage with 'the past'.. but why would I? Maybe now that I've thought about its implication, I will ignore it just to rebel against the system, maybe I'll use it for its convenience and go on with my life. Either way, I enjoy a self-awareness in my behaviour, it gives back some of the control addictive technologies have taken from us.

All in all, for me, this is a call to pay attention. Be aware of design decisions and consider how they will affect what and how you think. If there is something you find that you feel is disastrous to your curiosity, engagement, mental health, etc. be bold enough to find methods to subvert those effects. Austin Kleon recommends disabling your notifications - this is a fight against structure. But this is more so the easy fight - to change what the structure itself allows us to change about itself is in itself a structural property (that was a fun bit to write). The tougher parts are the things we can’t change, the views we can’t see, the perspectives inaccessible to us. These could be certain pieces of data or even more crudely, certain UI configurations. Sometimes we have vertical lists implying hierarchies rather than grids, sometimes we interact with horizontal swipe options but we can only see one item at a time... how do I think under the spell of a specific configuration?... and more importantly, is that how I want to be thinking? (For example, I don't know much about Tinder but I don't think it would be a conspiracy to question the cognitive effects of swiping away a person's photo based on their looks.)

Recently I've been thinking of creating experimental graph-based UIs that give us a more birds eye view of data rather than the narrow boxes we are forced to interact with. May be posting some new projects soon.

Some concrete examples of structural properties I have thought about:

  • Songs displayed in Spotify are 'listed', showing no relation among each other - causes a lack of cognitive spatial mapping of songs

  • Instagram's You're all caught up! message - promotion of FOMO-ish thinking?

  • Endless Pagination on blogs with posts ordered only by time - recent more worthy?

  • Blogs offer no control over customizable previews (my blog allows toggling of images and text)

  • Blog posts show no relation between posts within a tag/category - does it make me (cognitively) organize content by single descriptors?

  • Marketing practices avoid 'dead end' webpages (i.e. every page influences a user to continue the journey with some link that goes to another), why? Is there no conclusion? Something should tell me 'you've seen enough, now make a damn decision already.'

  • Twitter separates tabs for tweets and mentions/replies - do I differentiate the worth of those?

Previous PostLenin in Seattle, Stolen like an Artist
Next PostThe Umbrella Academy as Slavoj Zizek's Violence