dream together
dream together is a casual art project where invited contributors leave a thought every hour. At the top of the hour, a machine dreams them into a single image over the course of a 30 minute long dream.
The next dream cycle begins at
How it works: Before a dream cycle begins, contributors can submit their dream thoughts. Once the dream cycle begins, a computer runs a process that generates an image.
This process begins with a local LLM reading contributors' thoughts, which are used to compose a prompt for a neural net generating an image, a refinement prompt for a repainting neural net, music guidance and some metadata. An image diffuser neural net produces an initial image from random noise, guided by the dream. Then a second neural net is fed that image and upscales it to a larger resolution. Next, a third neural net produces the final image, seeded by the first image. This third step is mainly concerned with replacing details and refining texture. Finally, post-processing steps like a film-grain GPU shader and halation effects is run to produce the final image. The image is then uploaded to the website along with metadata produced by the LLM.
Guided by the LLM's decisions about music, a custom sound-generation process is started once an image has been dreamed up. It begins with the generation of several pieces of music, each based on a different random starting point. Then, a mixture of neural nets assign score to each track on aesthetics and prompt adherence. The winning score may be given a pre-renderer background soundscape like a recording of a campfire, wind in a forest or ocean waves, decided by the LLM. The score is finally processed through a physical model of an analog tape machine to give it realistic micro texture.
This pipeline is something that I keep experimenting with so it might not work exactly this way by the time you read this.
Images are licensed CC BY-NC 4.0: share and adapt them for non-commercial purposes with credit to dream together (dream.rsms.me). Contributors may use them for non-commercial purposes without any conditions. For commercial use, contact Rasmus.