<VECTOR 4>

unexpected output

While keeping Clonginger's re:phrased warning in mind, i don't think it's necessary to write ur own code in python or setup/run a local copy of Stable Diffusion on ur own machine to glitch AI. It is possible to instigate "authentic" glitches (rather than simply glitch-alikes) by simply prompting the system through the provided interface, the trick is to avoid specifically prompting it for a "glitch" (or databending, datamoshing, pixel-bleeding, compression artifacts, etc). What if we prompt it w/gibberish text? What if we insert a prompt written in uncommon unicode characters, emojis or l33t speak¹³? The better we understand these systems && the process occurring behind the scenes, the more likely we might conjure prompts capable of casting glitchy spells on the model. For example, we now know that diffusion models start by creating an image of colorful static known as "Gaussian noise", before de-noising it a number of times to produce an image which matches ur prompt. So what happens if we prompt it to make "Gaussian noise"¹⁴?

Diffusion models are trained on an immense dataset¹⁵ of image/text-description pairs. These text descriptions are broken up into "tokens", which are often small words or word fragments ("graffiti", "pizza", "cat", "th", "ly", "ing" are all tokens found in Stable Diffusion's vocabulary). In the case of Large Language Models like ChatGPT, both our input and its output are broken down into + build up from a series of tokens. The tokens in a model's vocabulary are a reflection of the most common patterns of text found in its training data, so it's reasonable to expect these to mirror the most commonly used words. Which is why researchers (inspired by the Google researchers ref'd in ) where surprised to find groupings of "nomalous tokens", like "?????-?????-" && "rawdownloadcloneembedreportprint". What was even stranger was that, despite being in the model's vocabulary, GPT could not seem to output these tokens. When simply prompted to repeat them, it would often re:in strange + unexpected ways, outputting things like "They're not going to be happy about this" && "You're a fucking idiot." As a result, these unusual tokens have now been dubbed "glitch tokens". Leaning further into our understanding of these systems, how might we use (or misuse) specific tokens in a diffusion model's vocabulary to instigate unexpected results?

Anticipating the unexpected can be a tricky practice. When dealing with new/unfamiliar technology, it's important for the glitch artist to dispel any preconceived notions of what a glitch looks/sounds/tastes/smells/feels like. When i co-organized the first GLI.TC/H conference/festival in 2010 most of the glitch work shown were colorful/saturated pixelated blooms/moshes, but by the third iteration, GLI.TC/H 2112, one of the most popular types of glitches were black/white unconstrained jaggedy chars: a type of text-based glitch known as "zalgo text", best exemplified (imo) by the social media hacks of artist glitchr. As it turns out, glitching social media platforms results in an entirely diff set of aesthetic artifacts than glitching JPGs or MOVs. When an AI model doesn't perform as *intended*, what sorts of artifacts appear?

One incredibly common "bug" present in many of these diffusion models were/are¹⁶ "nightmarish" human hands. The image in <vector-view-4> was created by prompting DALL-E with "a human with three eyes", the AI instead produced an image of a human w/two eyes && 8 fingers. Not only did it "fail" at outputting the requested image, it decided to create an image that emphasized these hands, positioning them on either side of the generated person's face. These sorts of glitches are, in a way, manifestations of the model's inherent bias (what AI researchers prefer to call "hallucinations"). The model's training data likely contained much more information about human faces than it did human hands. But it would be a mistake to ascribe the "bias" solely to the machine. All technology are biased, but this is b/c they are made by people && people are bias. The model's "bias" is really a reflection of the individuals who designed the system as well as the individuals who collected/curated the training data && the bias of all the individuals they scraped+mined that data from. But this "glitch" also points to another bias, that of the viewer. Safe to say most humans have 10 fingers, but do we all? What does it say about our own bias if we categorize this image as "abnormal" && this moment as a "glitch"?