Year: 2024

• Unknown Song By…

  

  A week or two ago I went to help my Mum downsize before she moves house. As with any move, there was a lot of accumulated ‘stuff’ to go through; of course, this isn’t just manual labour of sorting and moving and removing, but also all the associated historical, emotional, material, psychological labour to go along with it. Plenty of old heirlooms and photos and treasures, but also a ton of junk.

  While the trip out there was partly to help out, it was also to claim anything I wanted, lest it accidentally end up passed off or chucked away. I ended up ‘inheriting’ a few bits and bobs, not least of which an old PC, which may necessitate a follow-up to my tinkering earlier this year.

  Among the treasures I claimed was an innocuous-looking black and red USB stick. On opening up the drive, I was presented with a bunch of folders, clearly some kind of music collection.

  

  While some — ‘Come Back Again’ and ‘Time Life Presents…’ — were obviously albums, others were filled with hundreds of files. Some sort of library/catalogue, perhaps. Most intriguing, though, not to mention intimidating, was that many of these files had no discernible name or metadata. Like zero. Blank. You’ve got a number for a title, duration, mono/stereo, and a sample rate. Most are MP3s, there are a handful of WAVs.

  

  Cross-checking dates and listening to a few of the mystery files, Mum and I figured out that this USB belonged to a late family friend. This friend worked for much of his life in radio; this USB was the ‘core’ of his library, presumably that he would take from station to station as he moved about the country.

  Like most media, music happens primarily online now, on platforms. For folx of my generation and older, it doesn’t seem that long ago that music was all physical, on cassettes, vinyl, CDs. But then, seemingly all of a sudden, music happened on the computer. We ripped all our CDs to burn our own, or to put them on an MP3 player or iPod, or to build up our libraries. We downloaded songs off LimeWire or KaZaA, then later torrented albums or even entire discographies.

  With physical media, the packaging is the metadata. Titles, track listings, personnel/crew, descriptions and durations adorn jewel cases, DVD covers, liner notes, and so on. Being thrust online as we were, we relied partly on the goodwill and labour of others — be they record labels or generous enthusiasts — to have entered metadata for CDs. On the not infrequent occasion where we encountered a CD without this info, we had to enter it ourselves.

  

  This process ensured that you could look at the little screen on your MP3 player or iPod and see what the song was. If you were particularly fussy about such things (definitely not me) you would download album art to include, too; if you couldn’t find the album art, it’d be a picture of the artist, or of something else that represented the music to you.

  This labour set up a relationship between the music listener and their library; between the user and the file. The ways that software like iTunes or Winamp or Media Player would catalogue or sort your files (or not), and how your music would be presented in the interface; these things changed your relationship to your music.

  Despite the incredible privilege and access that apps like Spotify, Apple Music, Tidal, and the like, offer, we have these things at the expense of this user-file-library relationship. I’m not placing a judgement on this, necessarily, just noting how things have changed. Users and listeners will always find meaningful ways to engage with their media: the proliferation of hyper-specific playlists for each different mood or time of day or activity is an example of this. But what do we lose when we no longer control the metadata?

  On that USB I found, there are over 3500 music files. From a quick glance, I’d say about 75% have some kind of metadata attached, even if it’s just the artist and song title in the filename. Many of the rest, we know for certain, were directly digitised from vinyl, compact cassette, or spooled tape (for a reel-to-reel player). There is no automatic database search for these files. Dipping in and out, it will likely take me months to listen to the songs, note down enough lyrics for a search, then try to pin down which artist/version/album/recording I’m hearing. Many of these probably won’t exist on apps like Spotify, or even in dingy corners of YouTube.

  A detective mystery, for sure, but also a journey through music and media history: and one I’m very much looking forward to.

• Generatainment 101

  

  In putting together a few bits and bobs for academic work on generative AI and creativity, I’m poking around in all sorts of strange places, where all manner of undead monsters lurk.

  The notion of AI-generated entertainment is not a new one, but the first recent start-up I found in the space was Hypercinema. The copy on the website is typically vague, but I think the company is attempting to build apps for sites like stores, museums and theme parks that add visitors into virtual experiences or branded narratives.

  After noodling about on Hypercinema’s LinkedIn and X pages, it wasn’t long before I then found Fable Studios and their Showrunner project; from there it was but a hop, skip and a jump to Showrunner’s parent concept, The Simulation.

  

  The Simulation is a project being developed by Fable Studios, a group of techies and storytellers who are interested in a seamless blend of their respective knowledges. To quote their recent announcement: “We believe the future is a mix of game & movie. Simulations powering 1000s of Truman Shows populated by interactive AI characters.” I realise this is still all guff. From what I can tell, The Simulation is a sandbox virtual world populated by a huge variety of AI characters. The idea is that you can guide the AI characters, influencing their lives and decisions; you can then also zoom into a particular character or setting, then ask The Simulation to generate an ‘entertainment’ for you of a particular length, e.g. a 20-minute episode.

  In 2023, Fable Studios released a research paper on their initial work on ‘showrunner agents in multi-agent simulations’. To date, one of the largest issues with AI-generated narratives is that character and plot logics nearly always fall apart; the machine learning systems cannot keep track over prolonged story arcs. In conventional TV/film production, this sort of thing is the role of the director, often in conjunction with the continuity team and first assistant director. But genAI systems are by and large predictive content machines; they’ll examine the context of a given moment and then build the next moment from there, then repeat, then repeat. This process isn’t driven by ‘continuity’ in a traditional cinematic or even narrative sense, but by the cold logic of computation:

  “[A] computer running a program, if left powered up, can sit in a loop and run forever, never losing energy or enthusiasm. It’s a metamechanical machine that never experiences surface friction and is never subject to the forces of gravity like a real mechanical machine – so it runs in complete perfection.”

  John Maeda, How to Speak Machine, p3

  The ML system will repeat the same process over and over again, but note that it does not reframe its entire context from moment to moment, in the way that humans might. The ML system starts again with the next moment, then starts again. This is why generating video with ML tools is so difficult (at least, it still is at the time of writing).

  What if, though, you make a video game, with a set of characters with their own motivations and relationships, and you just let life continue, let characters grow, as per a set of rules? Many sandbox or simulation games can be described in this way. There are also some open-world role-playing games that play out against what feels like a simulated, continous world that exists with or without the player character. The player character, in this latter example, becomes the focaliser, the lens through which action is framed, or from which the narrative emerges. And in the case of simulators or city-builders, it’s the experience of planning out your little world, the embedding of your gameplay choices into the lives of virtual people (as either biography or extended history), that embodies the experience. What The Simulation proposes is similar to both these experiences, but at scale.

  

  Sim Francisco is the first megacity that The Simulation has built, and they’re presently working on Neo-Tokyo. These virtual cities are the storyworlds within which you can, supposedly, find your stories. AI creators can jump into these cities, find characters to influence, and then prompt another AI system to capture the ensuing narrative. Again, this is all wild speculation, and the specific mechanics, beyond a couple of vague in-experience clips, are a mystery.

  As is my wont, I’m ever reminded of precedents, not least of which were the types of games discussed above: SimCity, The Sims, The Movies, even back to the old classic Microsoft 3D Movie Maker, but also Skyrim, Grand Theft Auto, Cyberpunk 2077. All of these offer some kind of open-world sandbox element that allows the player to craft their own experience. Elements of these examples seem like they might almost be directly ported to The Simulation: influencing AI characters as in The Sims, or directing them specifically as in 3D Movie Maker? Maybe it’ll be a little less direct, where you simply arrange certain elements and watch the result, like in The Movies. But rather than just the resulting ‘entertainments’, will The Simulation allow users to embody player characters? That way they might then be able to interact with AI characters in single-player, or both AIs and other users in a kind of MMO experience (Fable considers The Simulation to be a kind of Westworld). If this kind of gameplay is combined with graphics like those we’re seeing out of the latest Unreal Engine, this could be Something Else.

  But then, isn’t this just another CyberTown? Another Second Life? Surely the same problems that plagued (sometimes continue to plague) those projects will recur here. And didn’t we just leave some of this nonsense behind us with web3? Even in the last few months, desperate experiments around extended realities have fallen flat; wholesale virtual worlds might not be the goût du moment, er, maintenant. But then, if the generative entertainment feature works well, and the audience becomes invested in their favourite little sim-characters, maybe it’ll kick off.

  It’s hard to know anything for sure without actually seeing the mechanics of it all. That said, the alpha of Showrunner is presently taking applications, so maybe a glimpse under the hood is more possible than it seems.

  Based on this snippet from a Claude-generated sitcom script, however, even knowing how it works never guarantees quality.

  

  ---

  Post-script: How the above was made

  With a nod to looking under the hood, and also documenting my genAI adventures as part of the initial research I mentioned, here’s how I reached the above script snippet from the never-to-be-produced Two Girls, A Guy, and a WeWork.

  Initial prompt to Claude:

  I have an idea for a sitcom starring three characters: two girls and a guy. One girl works a high-flying corporate job, the other girl has gone back to school to re-train for a new career after being fired. The guy runs a co-working space where the two girls often meet up: most of the sitcom's scenes take place here. What might some possible conflicts be for these characters? How might I develop these into episode plotlines?

  Of the resulting extended output, I selected this option to develop further:

  Conflict 6: An investor wants to partner with the guy and turn his co-working space into a chain, forcing him to choose between profits and the community vibe his friends love. The girls remind him what really matters.

  I liked the idea of a WeWork-esque storyline, and seeing how that might play out in this format and setting. I asked Claude for a plot outline for an episode, which was fine? I guess? Then asked it to generate a draft script for the scene between the workspace owner (one of our main characters) and the potential investor.

  To be fair to the machine, the quality isn’t awful, particularly by sitcom standards. And once I started thinking about sitcom regulars who might play certain characters, the dialogue seemed to make a little more sense, even if said actors would be near-impossible at best, and necromantic at worst.

• Elusive images

  

  Up until this year, AI-generated video was something of a white whale for tech developers. Early experiments resulted in janky-looking acid dream GIFs; vaguely recognisable frames and figures, but nothing in terms of consistent, logical motion. Then things started to get a little, or rather a lot, better. Through constant experimentation and development, the nerds (and I use this term in a nice way) managed to get the machines (and I use this term in a knowingly reductive way) to produce little videos that could have been clips from a film or a human-made animation. To reduce thousands of hours of math and programming into a pithy quotable, the key was this: they encoded time.

  RunwayML and Leonardo.Ai are probably the current forerunners in the space, allowing text-to-image-to-(short)video as a seamless user-driven process. RunwayML also offers text-to-audio generation, which you can then use to generate an animated avatar speaking those words; this avatar can be yourself, another real human, a generated image, or something else entirely. There’s also Pika, Genmo and many others offering variations on this theme.

  Earlier this year, OpenAI announced Sora, their video generation tool. One assumes this will be built into ChatGPT, the chatbot which is serving as the interface for other OpenAI products like DALL-E and custom GPTs. The published results of Sora are pretty staggering, though it’s an open secret that these samples were chosen from many not-so-great results. Critics have also noted that even the supposed exemplars have their flaws. Similar things were said about image generators only a few years ago, though, so one assumes that the current state of things is the worst it will ever be.

  Creators are now experimenting with AI films. The aforementioned RunwayML is currently running their second AI Film Festival in New York. Many AI films are little better than abstract pieces that lack the dynamism and consideration to be called even avant-garde. However, there are a handful that manage to transcend their technical origins. But how this is not true of all media, all art, manages to elude critics and commentators, and worst of all, my fellow scholars.

  It is currently possible, of course, to use AI tools to generate most components, and even to compile found footage into a complete video. But this is an unreliable method that offers little of the creative control that filmmakers might wish for. Creators employ an infinite variety of different tools, workflows, and methods. The simplest might prompt ChatGPT with an idea, ask for a fleshed-out treatment, and then use other tools to generate or source audiovisual material that the user then edits in software like Resolve, Final Cut or Premiere. Others build on this post-production workflow by generating music with Suno or Udio; or they might compose music themselves and have it played by an AI band or orchestra.

  As with everything, though, the tools don’t matter. If the finished product doesn’t have a coherent narrative, theme, or idea, it remains a muddle of modes and outputs that offers nothing to the viewer. ChatGPT may generate some poetic ideas on a theme for you, but you still have to do the cognitive work of fleshing that out, sourcing your media, arranging that media (or guiding a tool to do it for you). Depending on what you cede to the machine, you may or may not be happy with the result — cue more refining, revisiting, more processing, more thinking.

  AI can probably replace us humans for low-stakes media-making, sure. Copywriting, social media ads and posts, the nebulous corporate guff that comprises most of the dead internet. For AI video, the missing component of the formula was time. But for AI film, time-based AI media of any meaning or consequence, encoding time was just the beginning.

  AI media won’t last as a genre or format. Call that wild speculation if you like, but I’m pretty confident in stating it. AI media isn’t a fad, though, I think, in the same ways that blockchain and NFTs were. AI media is showing itself to be a capable content creator and creative collaborator; events like the AI Film Festival are how these tools test and prove themselves in this regard. To choose a handy analogue, the original ‘film’ — celluloid exposed to light to capture an image — still exists. But that format is distinct from film as a form. It’s distinct from film as a cultural idea. From film as a meme or filter. Film, somehow, remains a complex cultural assemblage of technical, social, material and cultural phenomena. Following that historical logic, I don’t think AI media will last in its current technical or cultural form. That’s not to say we shouldn’t be on it right now: quite the opposite, in fact. But to do that, don’t look to the past, or to textbooks, or even to people like me, to be honest. Look to the true creators: the tinkerers, the experimenters, what Apple might once have called the crazy ones.

  Creators and artists have always pushed the boundaries, have always guessed at what matters and what doesn’t, have always shared those guesses with the rest of us. Invariably, those guesses miss some of the mark, but taken collectively they give a good sense of a probable direction. That instinct to take wild stabs is something that LLMs, even a General Artificial Intelligence, will never be truly capable of. Similarly, the complexity of something like, for instance, a novel, or a feature film, eludes these technologies. The ways the tools become embedded, the ways the tools are treated or rejected, the ways they become social or cultural; that’s not for AI tools to do. That’s on us. Anyway, right now AI media is obsessed with its own nature and role in the world; it’s little better than a sequel to 2001: A Space Odyssey or Her. But like those films and countless other media objects, it does itself show us some of the ways we might either lean in to the change, or purposefully resist it. Any thoughts here on your own uses are very welcome!

  The creative and scientific methods blend in a fascinating way with AI media. Developers build tools that do a handful of things; users then learn to daisy-chain those tools together in personal workflows that suit their ideas and processes. To be truly innovative, creators will develop bold and strong original ideas (themes, stories, experiences), and then leverage their workflows to produce those ideas. It’s not just AI media. It’s AI media folded into everything else we already do, use, produce. That’s where the rubber meets the road, so to speak; where a tool or technique becomes the culture. That’s how it worked with printing and publishing, cinema and TV, computers, the internet, and that’s how it will work with AI. That’s where we’re headed. It’s not the singularity. It’s not the end of the world. it’s far more boring and fascinating than either of those could ever hope to be.

• The roaring 2020s and their spectacular picture palaces

  

  I took myself off to the movies lastnight. First time since 1917. The Sam Mendes film I mean, uh, obviously.

  Having gone on my little Godzilla binge earlier in the year, I thought it fitting that I take myself out to the latest instalment. The film itself was fine. Good loud dumb fun. Exactly the same formula as the others. A great soundtrack. Rebecca Hall being her wonderful earnest self. Dan Stevens being… whatever he is now (though he’ll always be Matthew to me). Content to one side, though, it was just great to be in the cinema again. For someone who allegedly studies the stuff from time to time, I don’t watch as much as I’d like; and I certainly don’t go to the cinema often at all. Lastnight showed me I probably should change that.

  I’ve often ruminated, in text and in brain, about the changing media landscape. I’m far from the only one, and recently Paris Marx put up a post about his quest to find Dune: Part One on home media. This story resonated with me. I have a sizeable physical media collection; it’s a dear asset and hobby, and one I am constantly surprised is not even close to mainstream nowadays.

  The production of physical has shifted considerably as demand has waned in the streaming era. DVDs are still, somehow, fairly popular; mostly due to an ageing and/or non-discerning audience (though that last bastion of DVDs, the supermarket DVD section, seems to have died off, finally). Blurays maintain a fair market share, but still require specialist hardware and are region-locked. Despite 4K Blurays being region-free and, with even a semi-decent TV, utterly gorgeous, they hold next to nothing of the market, being really only targeted at huge AV nerds like me.

  During COVID, the streaming platforms cemented their place in the homes and lives of everyone. I am certainly no exception to this. It was insanely convenient to have pretty much the world’s media output at the touch of a button. It was a good time: subscription prices were still relatively low, and the catalogues were decent enough to be worth having more than one or two services on the Apple TV at any given time.

  Netflix, Stan (an Aussie streaming service), and Prime Video were staples. They were also producing their own content, so in a way, they were modelling themselves on the megalithic studios of yore — as producers, distributors, marketers, even as archivists of popular culture.

  Things change, of course. They always do.

  Post-COVID, catalogues were culled. Most streaming services were operating, if not at a loss, then at least just breaking even with the equation of producing original content and/or buying distribution rights to older properties, or just other stuff in general.

  Then the original producers (in some cases the original studios) figured out they could just do it themselves. Disney+, Paramount+, Sony Core (aka Columbia); their own catalogues, their own archives, their own films straight from the cinema deal to the home media deal with no pesky negotiation.

  Prices for all streaming services have steadily risen over the last few years. For your average household, hell, even your above-average household, having all subscriptions active at one time simply isn’t feasible. It’s usually a question of who’s got what content at what time; or employ our house’s strategy and binge one or two platforms in one- or two-monthly bursts.

  Finding something specific in a given streaming catalogue is not a given. So you either pay Apple or Google or whoever to rent for a day or two or a week or whatever; or you pay them to ‘lease’ a copy of the film for you to view on-demand (they call this ‘buying’ the film). If giving money to the megacorps isn’t what you had in mind, maybe your brain would turn to the possibility of buying a physical copy of said media item for yourself.

  So you load up a web browser and punch in your best local media retailer. In my case, it’s a loud yellow behemoth called JB Hi-Fi; for more obscure titles or editions, it’d be something like Play DVD. These places are thin on the ground and, increasingly, even thin in the cloud. But JB’s physical media collection is dwindling, and has been for years. Their DVD/Bluray shelves used to occupy half of their huge stores; now they have maybe half a dozen tucked down the back, with the old real estate now occupied by more drones, more influencer starter kits, more appliances or pop culture paraphernalia.

  It struck me lastnight, as I headed into the cinema, that perhaps the film experience could see a bit of a bump if streaming services continue to fracture, and if physical media stock continues to disappear. If it’s a specific film that you want to see, and you know it’s on at the cinema, it’s probably more efficient overall to go and see it then and there. There are no guarantees any given film will be put up on a given streaming platform, nor that it will even get a physical media release any more. And if it does appear in either form, what quality will it be in? Would the experience be somehow diminished?

  There’s also something to be said for the sheer ubiquity and disposability of media in our current moment, particularly within the home, or home-based clouds. If I spot something on Netflix, I’ll add it to my List. I may watch it, but 7 times out of 10, I’ll forget it existed; once Netflix changes their catalogue, that item just floats away. I’m not notified; I’m not warned; unless it’s something on my watchlist on Letterboxd, or in a note or something, it just vanishes into the ether. Similarly with home media; if there’s a sale on at JB for Blurays, I might pick up a couple. They’ll then go on the shelf with the many, many others, and it might take years until I eventually get to it.

  There’s an intentionality to seeing a film at the cinema. In general, people are there to be absorbed in a singular experience. To not be distracted. To actually switch off from the outside world. I don’t claim any media superiority; I am a big old tech bro through and through, but there is something to the, ahem, CINEMATIC EXPERIENCE that really does retain the slightest touch of magic.

  So yes, perhaps we will see a little hike in moviegoing, if the platform economy continues to expand, explode, consume. Either that, or torrents will once again be IN for 2025 and beyond. Switch on that VPN and place your bets.

• Inertia

  

  Since the interminable Melbourne lockdowns and their horrific effect on the population of the city, my place of work has implemented ‘slow-down’ periods. These are usually timed around the usual holiday periods, e.g. Christmas, Easter, but there’s usually also a slowdown scheduled around mid-semester and mid-year breaks. The idea isn’t exactly to stop work (in this economy? ahahahaha no, peasant.) but rather to skip or postpone any non-essential meetings and spend time on focused work. Most often for teacher-researchers like myself, this constitutes catching up on marking assignments or prepping for the coming weeks of classes, though sometimes we can scrape up some time to think about long-gestating research projects or creative work. That’s the theory, anyway.

  I will say it’s nice to pause meetings for a week or two. The nature of academic work is (and should be) collaborative, dependent on bouncing ideas off others, working together to solve gnarly pedagogical issues, pooling resources to compile rich and nuanced ciritical work. But if you’re balancing teaching or coordination along with administrative or managerial duties, plus postgraduate supervisions and research stuff, it can be a lot of being on, a lot of just… people work. I’ll throw in a quick disclaimer here that I’m very lucky to have a bunch of lovely colleagues, and the vast majority of my students have been almost saccharinely delightful to work with. It can still be a lot, though, if you’re pretty woeful at scheduling around your energy levels, as I often am. Hashtag high achiever, hashtag people pleaser, hashtag burnout, hashtag hashtag etc etc etc.

  Academics are notorious for keeping weird hours, or for working too much, or for not having any boundaries around work and life. And I say this as someone who has embodied that stereotype with aplomb for years (even pre-academia, to be honest). I’ve had many conversations with colleagues where we bemoan working late into the evening, or over the weekend, or around other commitments. I’ve often been hard-pressed to find anyone who has any hard boundaries around work and not-work.

  Taking extended leave last year was the first time I’ve ever properly stopped working. No sneaky finishing of research projects, no brainstorming the next media class, no cheeky research reading, no emails. It showed me many things, but primarily how insidious work can be for someone with my disposition and approach to life in general. It is also insidious when you are passionate, and when you care. I care deeply about media education and research, and have become familiar with its rhythms and contours, its stresses and its delights, its (many) foibles and much deeper issues. I care about students and ensuring they feel not just ‘delivered to’ or ‘spoken at’, but rather that they’re exposed to new ways of thinking; inspired to learn well beyond graduation, indeed, to never stop learning; enabled and empowered to tell their stories, and whatever stories they want to tell. I care about producing research, e.g. journal articles, video essays, presentations and events, that is not tired, stale, staid, boring, dense, conventional, but rather is experimental, vibrant, connected, open-ended, and appeals broadly across multiple disciplines and outside the academy.

  I’m not alone here. As mentioned above, I have colleagues who almost universally feel exactly the same way. And I’ve built a local and international research network who share these passions and questions and concerns. A global support group. I’m very lucky and privileged in this way.

  But yeah: all this shit is fucking exhausting. The environment, the sector, the period, certainly doesn’t help. The current model of academia, university management, tertiary education, the industry/academy nexus, capitalism (in summary: neoliberalism), all of it is quite happy to capitalise on passion, on modern productivity dicta around never-being-done, irons-in-the-fire, publish or perish, manage it all or die, no life for you, hang the consequences and anyone you’re dealing with who isn’t work (e.g. partners, kids, friends, families). To anyone who says academics have a cushy job and get paid too much: kindly take yourself into the sea, thanks. That may have been true in the past, but we’re living on the other side of whatever spectrum you’re looking at.

  Suffice to say, slowdowns are nice. Taking proper breaks and/or having an executive echelon that genuinely supports and structures wellbeing and balance would be ideal, but beggars can’t be choosers.