Programming for Musicians and Digital Artists: Creating music with ChucK (2015)

Chapter 0. Introduction: ChucK programming for artists

This chapter covers

· Why do artists need to program?

· What is ChucK?

· Why program in ChucK?

· Why do we and others use ChucK?

· Some of our own computer-mediated artworks

For many years, the words musician and artist have been changing meaning, rapidly, almost daily, largely due to the introduction of computer technology. Artists perform live with computer technology all the time. The ones who interact directly with computers as part of their performances might call themselves DJs, laptop artists, controllerists, live coders, and a host of other names. Many of these musicians don’t program or write software, but an increasing number want more direct control over their process and the results. Learning to program is one way to get that extra level of control.

Other artists want to make new instruments or controllers or to configure existing controllers such as drum pads, DJ control decks, and the like to use in new ways for their live performances. Still others want to produce songs and albums (.wav and/or .mp3 files) as the final result but would like more control in the process than off-the-shelf music software provides. Some others like (or want) to program as the basis of their creative process and workflow.

Nonmusical artists also use computers in their art making, such as graphic designers, animators, film editors, set designers, sculptors, and others who use computer graphics and design software. Many of these software tool users also want more control over their creations than their commercial software packages provide.

Increasingly, many multimedia artists create installation art, public art, sound sculptures, or soundscapes. These new works involve using combinations of sound, sensors, graphics, video, and displays to create interactive environments. These pieces can make the audience part of the performance or experience. In some cases, members of the public witnessing these pieces don’t know exactly what the artists/performers are doing or controlling, but the discovery, learning, and interacting are much of the point and of the reward. Although this book doesn’t specifically teach you to make all kinds of artworks, many forms of new media involve computerized sound and music, and learning specific tools to create and manipulate sound in the computer is precisely what this book is about.

Many people call these new types of art and performance opportunities computer-mediated art systems. In all cases, the musician or artist either needed to know how to program or collaborated with someone who could. You’ll learn to program through the chapters, examples, and exercises in this book.

This increasingly popular movement of computers as a new medium for creativity motivated our team to write this book to make it easier and more fun for everyone to learn to write their own programs. Specifically, we want to teach you how to program through making code that translates into music and sound, and we accomplish this via a programming language called ChucK, which is specially designed for sound and music.

We begin with a discussion on why we think musicians and artists need to learn how to program. We explain why we believe ChucK is a great first language to learn. We conclude by describing how programming has allowed the authors and others to create new works using ChucK and its predecessor languages.

0.1. Why do musicians and artists need to program?

As we mentioned earlier, many artists are happy with over-the-counter software systems and controllers for real-time performance work. And there are many who only want to use computers to produce static final products in the form of .wav/.mp3 files, CDs or collections of songs, sound tracks for videos, and more. A large number of those artists are happy to learn and use the packages and tools from commercial or free sources.

But there are many, and we’re betting you’re one, who want more. Maybe you’re coming to this book with a big idea (or many big ideas) and want the tools to help you realize it/them. Maybe you’re looking to shift directions in your art making. Or perhaps you already know how to program in a language such as Java, but you find it doesn’t do what you want.

Others think that learning programming will help them get a job. Although we can’t promise immediate outplacement after you finish this book, we can say that we have nice jobs, and our students have nice jobs (those who want them ), due in no small part to their ability to program and solve problems with computers.

We, in our individual projects, art works, and teaching, have used ChucK and other computer music/art languages for years. Some of us have programmed, and still do, in multiple languages. Whether or not you’re already a programmer, you’ll think differently after working through the examples and exercises presented here. Few painters ever suffered because they knew more about the chemistry and physics of their paints, canvases, brushes, and solvents. Even self-taught artists have developed and used natural knowledge of the processes underlying their art making. Knowing programming is similar for the digital artist. Any artist who knows one or more computer languages, even one who doesn’t write computer code daily, still has a better sense of what’s going on when they drag down a menu, select an item, and watch the progress bar move.

We’re certain that by the end of this book you’ll be able to do lots of what you want and likely much more than you might have thought possible. The power available in ChucK will suggest new things to do, and you’ll know how to do them—or how to figure out how. And, even if you use commercial software a lot, we’re going to show you new ways to control and interact with it using ChucK. It’s like being able to put a new engine in your car.

0.2. What is ChucK? How is it different?

If you’re used to working with popular sound tools on computers, you might be used to connecting sound synthesis and processing boxes, managing tracks and instruments, cut/copy/paste sound editing, moving virtual dials and sliders on digital audio workstation (DAW) programs, and so on. In this book, you’ll be learning an actual programming language, ChucK, which will let you do essentially anything, but you’ll need to type in some lines of computer code (text) in order to accomplish your goals. The first few examples might make it seem like it’s harder than the tools you’re used to for making sound and music, but soon you’ll be doing things that you’d have never thought of or known possible. In this sense you can consider ChucK a power tool. There’s a little to learn up front, but soon your results will make you wonder how you lived without it.

ChucK is a programming language designed specifically for real-time sound synthesis and music creation. Real time means that ChucK synthesizes the sound as you’re hearing it (rather than playing back a sound file), often in response to signals and gestures from the outside world. Gestures to control sound might include your typing on the keyboard, moving the computer mouse, manipulating a joystick or other game controller, or playing the keys on a musical keyboard connected to your computer.

ChucK is also good for controlling and/or interacting with almost any type of real-time computer media and art, such as graphics, robots, or whatever can communicate with your computer.

ChucK was designed specifically to allow and encourage on-the-fly programming, which means you can add, remove, modify, edit, and layer segments of code at any and all times, hearing the results instantly, without interrupting other sounds being synthesized and heard. This is one of the primary ways ChucK differs from all other languages, which makes it extremely fun to learn and use, because you can try things and immediately hear the results. Most other languages require you to compile, run, and debug code in a way that doesn’t let you hear immediately what you’re doing. Most computer languages, such as C, C++, or Java, weren’t designed specifically from the ground up for sound, music, and other real-time tasks. ChucK makes immediate, real-time sound a priority.

If you know other computer languages such as Java or C++, or even other music/ sound languages and systems such as Csound, SuperCollider, JSyn, Max/MSP, or PD (Pure Data), you’ll soon see that ChucK is really different. It’s more expressive and powerful at manipulating time and sound than the graphical interfaces of Max/MSP and PD, giving you greater under-the-hood access than these other languages and systems. Compared to other text-based music/sound languages such as SuperCollider or Csound, ChucK is generally more succinct, requiring much less code (lines of typed text) than these other languages in order to accomplish any particular task.

If you don’t know any computer languages, when you’ve finished this book, it will be easier for you to learn Java, C, C++, and any other language you desire to learn. ChucK is different from other languages for sure, but it shares many things that will be similar and recognizable to programmers of nearly any language.

Another great feature of ChucK is that it’s open source (not secret or protected by licenses, passwords, keys, and so on), and it’s freely available on all major computer platforms, including Mac OS X, Windows, and Linux. Open source means that the community of ChucK users can have direct input into the process of making ChucK better in the future. It also means that ChucK doesn’t cost anything to get and use.

0.3. Why program in ChucK?

A tool, which is one way to look at a programming language, can’t help but shape its user’s mindset, and it naturally suggests ways of achieving various tasks. And like any tool, a programming language should change the way you think and go about doing things. ChucK definitely presents a different way to program sound and music. Although there are tradeoffs that make certain things straightforward and other things more difficult, our sincere hope is that for the programmer, the language design choices help more than they hinder a particular task.

ChucK was created by Ge Wang, formerly coauthor Perry Cook’s graduate student at Princeton University (and now an assistant professor at Stanford University and coauthor of this book), to provide a different way to think about programming sound using a rapid prototyping (trying out lots of ideas quickly) mindset. More than a decade later, ChucK research and development have only intensified. Ge’s detailed chronicle of the history, motivation, and people behind ChucK can be found in this book’s preface. As for more detail on why you might learn and use ChucK, here are a few more reasons:

· It’s all about time. Time is at the core of how ChucK works and how you work with ChucK to make sound. As a programmer, you specify how to move through time and assert control at specific points, and sound essentially just happens—conveniently, for precisely the amount of time you’ve moved through. Why such emphasis on time? Sound is a time-based phenomenon; without the passage of time, there would be no sound. By controlling how and when you do things through time, you have a different and powerful way to work with sound at every level—everymolecule of it.

· It’s text, plain and simple. Although programming with text may initially seem more abstract or complex than, say, with graphical representations, it’s arguably much easier once you start adding a lot of expressive nuance and logic into your code (which you’ll invariably need to do). Little is hidden or inferred. The important parts are in plain sight; for example, how time flows in a program. At the same time, many mundane aspects are taken care of under the hood: scheduling, real-time sound input/output, bookkeeping for all the sound generators, and so on. Readability is a central design goal of the language, and that makes it a good learning tool as well.

· It’s fun and immediate. ChucK was designed to be a fun language and environment to work in, to experiment in. You can synthesize sounds, make fantastical automations, map physical gestures (for example, with controllers) to sound, network computers together, and even use signal analysis to (computationally) make sense of sound.

0.4. ChucK-powered and pre-ChucK computer-mediated art

The goal of this section is to present examples of computer-mediated artworks that we’ve created using computer programming—creating musical instruments, controllers, ensembles, and systems. One time-honored way of making a new instrument is to look at existing instruments and find ways to augment or improve them. Many of our experiences lie in this area. From the Cook/Morrill MIDI Trumpet in 1988 (this didn’t use ChucK, because it didn’t yet exist, but it used the predecessor of ChucK, STK, which is now included in ChucK) and Kapur’s ESitar of 2004, through the Laptop Orchestras of 2005 to present, to mobile musical instruments and beyond, we, along with our crazy friends, have created a variety of computer-mediated systems based on and inspired by traditional musical instruments and groups. In some cases we put sensors on the instrument, leaving the inherent sound-producing capabilities in place. In other cases, we gutted the acoustical parts of the instrument and filled it with processors, sensors, and speakers, leaving only the shell and form to suggest the interaction (experience). These include the DigitalDoo, SqueezeVox (accordions, Perry Cook and Colby Leider), the ETabla and EDholak (Ajay Kapur), BoSSA (the Bowed Sensor, Speaker Array, Dan Trueman and Perry Cook), and the SBass (Sensor Bass, Curtis Bahn). Figure 0.1 shows some of these computer-augmented instruments. In all of these cases, we needed to write custom code to have a microchip or a cell phone transform gesture into musical experience.

Figure 0.1. Top row: Cook-Morrill Trumpet, DigitalDoo (Cook), SqueezeVox Maggie (Cook), SBass (Bahn); bottom row: BoSSA (Trueman), ESitar (Kapur), Smule’s Ocarina (Wang)

The proliferation of powerful mobile devices such as the iPhone and iPad gave rise to a re-envisioning for the mobile era of traditional musical instruments, such as the Ocarina for iPhone (Ge Wang/Smule), as well as entirely new mobile musical interactions. With millions of worldwide users, Ocarina and a number of other mobile apps have used ChucK for synthesis.

Another way to make entirely new musical performance systems is to put sensors on dancers or other artists who do not traditionally perform music. Figure 0.2 shows PikaPika, an Anime-inspired dancer character created by Tomie Hahn. PikaPika is fully loaded with sensors, wireless transceivers, amplifiers, and speakers. Pika’s SSPeaPer (Sensor-Speaker Performer) dancer hardware was created and programmed by Curtis Bahn. Figure 0.2 also shows Raakhi Kapur, an Indian dancer, wearing sensors on her wrists that control musical robots in a KarmetiK Machine Orchestra production featuring musical robots and human musicians (more on that soon). A lot of computer programming was needed to successfully map these dancers’ gestures to sonic responses. Pika uses STK (as included in Max/MSP), and Raakhi’s dance system uses ChucK for synthesis and robot control.

Figure 0.2. Cyber-Anime PikaPika (dancer Tomie Hahn). Raakhi Kapur (right) controls musical robots.

Another means to make amusing art and music works is to put sensors on other objects that are not traditionally associated with musical performance. Figure 0.3 shows musical kitchenware; Perry Cook’s jazz JavaMug and the Fillup Glass atop the musical table of P-Ray’s Café. Also shown is an early prototype of a computer-mediated musical TapShoe. Turning these everyday objects into musical instruments obviously requires programming and, once again, mapping, to make them sonically interesting. These devices all put out signals that the computer must use to make sound or control other objects/processes, and ChucK is the easiest place to do that type of mapping. In one of our favorite ChucK programs, Perry turned the JavaMug (computerized coffee mug shown in figure 0.3) into a digital trumpet.

Figure 0.3. P-Ray’s Café musical table with JavaMug and Fillup Glass; (right) musical TapShoe

The possibilities of entire ensembles of laptops and/or other large-scale computer-mediated musical groups brought about the Princeton Laptop Orchestra (PLOrk), followed by the Stanford Laptop Orchestra (SLOrk), and the human + robot–populated KarmetiK Machine Orchestra. All of these ensembles rely heavily on augmenting human performers with sensors, networking, and ChucK. Further, each orchestra instrument features player-local sound via individual hemispherical speakers. This means that there’s no routing of laptops to a mixer and house sound, but rather each laptop connects to an individual multichannel (six speakers arranged evenly on the face of a hemisphere) speaker located next to each player. Thus the sound of the ensemble is the acoustic mixture, in the room or concert hall, of the individual sounds from each player, just as it would be with a traditional orchestra. Figure 0.4 shows the original (freshman class of) PLOrk, the first mobile phone orchestra at Stanford, SLOrk, and the KarmetiK Machine Orchestra.

Figure 0.4. Top row: Princeton Laptop Orchestra and KarmetiK Machine Orchestra mobile music performance; bottom row: Stanford Mobile Phone Orchestra (MoPhO), Stanford Laptop Orchestra. At right: coauthor Spencer Salazar, a MoPhO-ist sporting wearable speakers.

These examples only scratch the surface of possible artistic things when you know how to hack. Here we use hack in the best sense of the word, combining the inventor’s dream with the ability to modify, build, experiment, and, possibly most important, program.

0.5. Summary

In this chapter, we hope we’ve convinced you that learning to program is important for digital artists and that ChucK is a really good way to learn programming. We spent quite a bit of time talking about what ChucK is, a little on how it was designed, and why we and many others think it’s the best programming language for doing digital art. For now, the things to remember are these:

· Artists, especially digital artists, can benefit from knowing how to program. You can make your own tools and wholly new creations, and programming gives you ideas you wouldn’t have had without being skilled at a computer language.

· ChucK, as a language, is different from other languages and music programs. It was designed from the ground up to support powerful control over time and sound, and input devices from the outside world (joysticks, trackpads, and the like), and networking.

· For those who already know a programming language, we think you’ll find that ChucK allows you to do many things that you might find hard or impossible in other languages, and do them very quickly.

We have showed you some of the types of projects we’ve created using computers and programming, but those only scratch the surface of what can be done with a powerful tool like ChucK.

We hope you’re convinced, inspired, and ready to begin to learn to program in ChucK!