The Robotic Artist: Aaron in Living Color
Harold Cohen at The Computer Museum
April 1- May 29, 1995
CONVERSATION: HAROLD COHEN & BECKY COHEN
La Jolla, California, March, 1995
BC. AARON has been making drawings autonomously for more than two decades, and now you are celebrating its new ability to color its drawings with dyes and special brushes. How did you get it to paint?
HC. Putting dye on paper is easy: you just build a machine! This one consists of a small robot arm carried around over a large flat table on what we call an “xy device.” The arm has a “hand” that’s able to pick up the cups and brushes that are located at the edges of the table, manipulate the taps on bottles of dyes, and so on.
Of course I’m joking about it being easy to build a painting machine. But the truth is that it was a relatively straightforward task compared with writing the code that would give AARON the ability to think about color. That has been my major pre-occupation the past two or three years, and there would have been no point in building a machine if I hadn’t been able to do it.
BC. Of course, what people can see in the museum is the machine painting. What they can’t see is how AARON is thinking about color. Can you describe how the coloring program works? Why was it a difficult problem?
HC. For many reasons. Consider, to begin with, that human beings can see the results of putting two colors next to each other, and can proceed on the basis of this feedback. The program is able to keep a very complete record of what it’s doing, but it can’t see in the same sense that you or I can see. I had to come up with rules about color juxtaposition that would serve in place of the visual feedback that humans use. As a painter, with a lifetime of experience of color, I must obviously have known what some of those rules were, yet I found it frustratingly difficult to say what they were.
BC. Presumably you needed some sort of feedback in trying to develop the rules for coloring, but you were working on that problem before you had the painting machine.
HC. I did most of the color development in a screen-based version, using a Silicon Graphics workstation. At the same time I knew there would be a big problem in transferring these results to the painting machine. I wasn’t even sure it could be done.
BC. Why?
HC. Well, a color workstation does “additive” mixing. There are only three primaries red, green and blue. All colors are made by mixing them. You are mixing light, and, consequently, the more you add the brighter the result. For example, on the screen you get yellow by adding red and green together. People generally find that hard to believe, because it doesn’t seem to correspond to experience. Actually it does; but most color mixing in the physical world is “subtractive” mixing. When you mix red and green paint together each color filters out some part of the visible spectrum. The result is still in the yellow part of the spectrum, but so much light has been filtered out that we would describe it as dirty brown rather than yellow.
I did eventually come up with a screen-based solution that functioned rather well. ln fact, I was able to use AARON’s screen-based coloring designs for the last group of paintings I made, in some cases almost without modification. I photographed the images off the screen and used the slides rather the way a painter might use a color sketch. I’ve never used sketches before; it was a rather interesting change for me.
BC. Were you then able to map the rules you had built for the screen-based coloring program onto the coloring program for the painting machine?
HC. Well, actually not. I spent some time trying to translate the red-green-blue mixtures that AARON specified into combinations of the dyes I was using, but it never worked to my satisfaction. lt turned out that I could only translate about half of AARON’s colors; rather obvious, actually, since dyes can’t possibly be as bright as the colors you see on the screen. Finally I abandoned that approach and started to build up a new version based directly upon the dyes.
BC. Shouldn’t that have been easier for you? After all, you’ve spent your life working with physical color, not with colored light.
HC. Perhaps. But the fifteen different colors of the dyes I use aren’t evenly distributed along the spectrum, and they can act in very peculiar ways. ln fact, any physical material acts in peculiar ways – not at all well-behaved and predictable the way light is. These dyes have been formulated for bulk-dying fabrics, not for painting. On the paper, for instance, you can start with what you think is a very vivid forest green in the pot and finish up with a rather dull green surrounded by outlines of yellow on the paper. My current task is to continue to build an adequate knowledge base about how all these materials behave and at the same time to modify the materials themselves so that they behave a little better.
I’d much preferred to have used oil paint, which I’ve always found to be the most versatile and the most beautiful of media. lt wouldn’t have been at all practical for the painting machine, unfortunately. Oil paint is a more or less transparent material, and you have to control the thickness of the paint film rather precisely to get the most from it. My machine is much too crude a device to do that; in fact, I’m not sure that any current robot could exercise that level of control.
BC. What kind of dyes have you chosen for AARON to use? And why dyes? Do they suffer from impermanence?
HC. Oh no, not at all. That was true in the nineteenth century, with some of the earliest industrial dyes, but no longer. I have a shirt that’s been in the Californian sun for almost two decades and in and out of the washing machine I don’t know how many times; it still has most of its original color.
I’ve been using these Procion fabric dyes for several years for working on paper; they’re very beautiful in color and they all rate six or seven on a permanence scale from one to seven. I thought I knew them very well. I didn’t know them quite as well as I thought I did.
BC. Well, you probably haven’t used them to this sort of stretch before.
HC. Right.
BC. After listening to this technical description about what you considered in creating your rule base for coloring, I’m also wondering what other kinds of knowledge have found their way into the part of AARON that colors: your own background, for instance. You are English. You established yourself as an artist, a painter, and you are obviously very much aware of European art. Does some part of this experience affect what sense of meaningfulness you code into the coloring program?
HC. Since I bring nearly fifty years of experience as a painter to the problem of coloring, I suppose one has to make that assumption, but I’m sure one couldn’t find evidence for it one way or the other just by looking at the code.
BC. Let’s talk a bit more about what I guess you would call the AARON system since it now includes the painting machine. ln your installation there are two different computers, one for generating the drawings, complete with coloring, the other for running the painting machine. What programming languages do you use?
HC. AARON is written in LISP and runs on a Silicon Graphics computer, while the painting machine is controlled by a PC – a generic 486 – and the program is written in C++.
When AARON generates a painting, it stores it in a file as a set of instructions. Most of these instructions will control the movement of the brush on the paper, both in making the initial drawing and in filling in the color. Some of them specify the mixing of dyes for individual areas of the painting, and some of them specify the size of brush to be used. The file is read over a network connection by the 486 which then interprets those instructions and scales the dimensions of the Silicon Graphics screen to whatever size drawing is being made. lt also scales the volume of the dye to be mixed for any color and the size of the brush, and then it generates the lowest-level commands that drive the painting machine.
To do everything it is supposed to do, the 486 program has to control the movement of the arm across the table, the horizontal rotation of the shoulder, the vertical rotation of the elbow, two rotations of the wrist, the opening and closing of the hand, and the reach – how far the hand can extend from the elbow. The program also has to know where the cups and brushes are kept, where the tap handles are and how much to move them up and down, and so on.
BC. So, the order of events is: AARON first generates the drawing, then the coloring for the drawing, and finally sends orders to the 486. AARON never thinks about coloring before drawing, does it?
HC. No, the drawing is done first and then AARON decides about color. But the coloring part doesn’t only involve the color choice. lt must also map out the path the brush must take filling in the various shapes in the drawing.
BC. Yes, I could see the brush following the internal contours of shapes as it was coloring; but it seems that AARON must also have a sense of portraiture: that it has some idea of what sorts of color might be good for face and hands, what colors might be good for clothing, or plants.
HC. Oh sure. AARON has a very clear idea of what it is doing.
BC. How does AARON assign color?
HC. ln AARON’s understanding of the drawings, different elements are characterised by their different attributes. lt knows, for example, that a face has two eyes, and it will never draw a face with three. To the degree that color is also an attribute of a face, there are a limited number of colors it can use. lt would never decide to paint a face green because it doesn’t believe that faces can be green. However, there is no such limitation on the assignment of colors to things like sweaters or backgrounds. Color assignment here reflects the program’s concern for the color “signature” of the whole painting. If AARON decides to do a red sweater, for example, it will probably not decide to do a red background.
BC. Ah! So, this is an ad hoc style of reasoning similar to your earliest AARON drawing program, the purpose of which was to simulate human free-hand drawing. You never meant to mimic existing drawings but to model how drawings are produced by human beings. You seem to be treating the generation of coloring in the same way . . . which leads me to ask whether you consider speculation through reasoning to be at the heart of what you do as an artist?
HC. I suppose I do. But then, you’ve put your finger on the central problem in providing AARON with knowledge about the use of color. lt is not entirely clear how human artists handle color internally. lt is certainly not clear that we reason about it.
BC. But you can reason about drawing. Why the one and not the other?
HC. Well, not everything that goes on in the head is reasoning, or even thinking, if what one means by thinking is the symbolic representation that has to precede any utterance we make. When I started work on the painting version of AARON I was struck by the fact that we have a very poor vocabulary for talking about color relationships, and that almost all of what’s been written as color theory has been either theory about color perception or theory about color measurement. There is almost nothing about color use.
I’ve never tried to simulate my own work, but whenever I find myself faced with a problem about how the program should proceed, I’ve asked myself how I would proceed. I was deeply frustrated to find that I couldn’t describe what was happening in my own head when I was manipulating color as a painter.
BC. Nevertheless you did find a way into the problem.
HC. There is one small piece of knowledge, which I’ve been handing out to painting students as long as I’ve been teaching. Heaven knows why it took me so long to recognize it as a key to my own concerns. lt is that the most important single thing about color is not spectral hue; it’s how light or dark it is. Actually that’s not too strange; the eye functions mostly as a brightness perceptor. Color vision seems to be more of a luxury than a survival mechanism in human beings.
BC. You’re talking about how light or dark something is in relation to its neighbor. Within your system of painting, shapes sharing a common border can be differentiated by both hue and brightness. The difference that one can see between one shape and another can actually indicate spatiality.
HC. Certainly.
BC. You avoid regular perspective yet you have ways of indicating space. Your pictures tend to be sort of two-and-a-half dimensional: not 2d, not 3d, but somewhere in between – sort of like Pompeiian frescoes.
HC. But all representation is two-and-a-half dimensional, isn’t it? The viewer is always confronted with a flat surface that evokes something in the physical – 3-dimensional world.
BC. But still, there are clear differences between Pompeiian frescoes and the work of Masaccio, say, or a Rembrandt.
HC. Or between African Bushman art, or Persian miniatures, or Australian Aboriginal art, and European art. lt seems to me that the last five hundred years of western culture have been quite aberrant with respect to world history. At no other time in human history will you find our own characteristic obsession with appearances, nor its consequence, which led to the underlying technology both for photography and for computer graphics the reflection of light off the surfaces of things in the world. That’s a mystery to me. Do we really believe that we can find out truth by the way things look?
BC. The mathematical and visual consideration of three dimensional space that Ucello, Piero della Francesca and others gave us through the invention of perspective seems to have been fairly useful in the practice of architecture or war or tracking movement through space.
HC. No doubt. But let’s not overlook the underlying philosophical position of the Renaissance that gives rise to perspective, which proposes that man is central in the universe and that the eye of man is the point through which to regard everything. Do we hold to that position at the end of the twentieth century?
Regarding spatial representation from that time forward, there are a number of different ways of writing history. One possible way is to say that as soon as it became possible to actualize perspective – considering photography as a mode of representation without human invervention – artists lost interest in it as a way of making images.
You can also write history to say that artists never were that interested in perspective. Most of the major painters played games around the edges of perspective. lt has been a device for lending plausibility to an image, but even Uccello seems to have been much more interested in the geometry of the flat surface than in anything evoked in the physical world. Perspective has always been a kind of academic backbone to painting. Aside from its inventors, I don’t think it has ever been considered the central issue by painters.
BC. So, in the spatiality you’ve invented for AARON, in the coloring part of the program, it comes down to an instruction in the program saying you can have this hue or that hue, but really the most important thing is the shift in brightness between adjacent hues. This is one means of defining depth, but what about the drawing program? Do you have any rules for perspective in AARON?
HC. Oh sure. To the degree that perspective has to do with the sizes of objects relative to their distance from the viewer, AARON deploys a more-or-less conventional perspective.
BC. That’s true. You have large figures in the foreground, but you don’t have a horizontal line indicating deep space.
HC. No, I don’t have deep space, not in recent work, any way. But, you know, this linear perspective we talk about is only a part of what the Renaissance invented. The more important part was the seperation of drawing and coloring; the idea that you should build a representation by starting with a monochrome underpainting dealing with the amount of light reflected off surfaces, and only then concern yourself with local color applied over the top of the underpainting. The point is that AARON doesn’t know anything about surfaces. European I may be, but I throw in my lot- and AARON’s – with much older and much less pedestrian modes of representation.
BC. You seem to have created a sort of magical space where AARON’s “organisms”, figures, and plants have a special interrelationship with each other. Even in the room-like environments, it is as if the figures have a truly imaginative relationship with each other.
HC. I must obviously hesitate on the word “imaginative” because that implies capabilities to the program that I know perfectly well that the program doesn’t have. AARON’s domain of _expertise is the building of representations, not knowledge of the outside world. Hmm … Well, it has some knowledge of the outside world.
BC. Like what?
HC. For example, it knows how people are put together. lt knows how they are capable of moving. lt knows how plants grow. lt knows that rooms have walls at the back. lt knows all of those things, though that isn’t to say that it knows them in the same way that you or I know them. I suspect that whatever success the program has had has rested upon devising a representational mode perfectly fitted to the structure of its knowledge.
BC. Is drawing less interesting to you now than it was before you became involved with color?
HC. Not really. I think I would say that I want to define a mode of representation in which color is a central element, not a decorative afterthought.
BC. What you are talking about is something that certainly interested the impressionists, but your own desire seems for color to have meaning superior to that of drawing.
HC. No, just how to make color the central organizing principle.
BC. If I am remembering correctly, one of your experiments as a painter (before you left England and met your first computer) was to eliminate drawing altogether.
HC. That’s true. During the ’60’s, I became so involved with the issue of color that I found myself literally pushing drawing off the edge of the canvas.
BC. What made you think that you had to reintroduce drawing?
HC. The preoccupation with color is something that goes back a long way for me, but it is also something that had to be put on the back burner for a very long time once I started working with computers. Bear in mind that there would have been no way for me to work on color with the computing resources we had twenty-five years ago. I became almost exclusively preoccupied with drawing, and tried to answer a very fundamental question for myself: what are the minimum conditions under which a set of marks on a flat surface functions as an image.
BC. lt seems to me that the very question implies a certain discontent about having reduced your painting to a minimal colored surface.
HC. That’s the way it seems to me too. ln fact, I would go further; I was becoming increasingly disturbed and antipathetic towards the whole modernist movement in painting, in art. We had turned painting into a very specialized game that only a very few people could understand and respond to. I’ve always felt that the health of any art depends upon its relationship to the culture that is serves, and I wasn’t happy with where I stood. I don’t want this to sound more conscious an aim than it was at the time – 1968 – but I suppose that in turning away from color to spend several years investigating drawing, I was beginning to look for a way of getting back to a kind of imagery that would be available to more people.
BC. Listening to what you’ve just told me, it seems that you reinvented drawing as a means of reinventing color.
HC. Well, it turned out that way. Over time I began to think that there was something slightly unsatisfactory about having AARON do all these drawings that I was then required to color.
BC. I recall you saying, many years ago, that you reserve the coloring for yourself. On the other hand, you have always had this deeper drive to make your speculative machine thoroughly autonomous.
HC. “Autonomous” is a relativistic term. Even though AARON now has a good grip on the problem of coloring, there are still many other things the program has to take responsiblility for that it doesn’t quite yet..
BC. You continue to teach AARON new things, but it seems to me that from the beginning of your dialogue with your creation, you have always wanted its work to qualify according to your own high standards of interest, use, and beauty.
HC. Of course; why would I demand less of it? One of the bargains I made with myself from the very earliest days was that I would never accept the position of having to apologize because this was done by a computer. I have always insisted that the work the program did would have to stand on equal terms with art made by hand.
BC. You mean, art made by the artist?
HC. Yes. But not necessarily this artist. As I said earlier, it has never been my intention to simulate my own work.
BC. Still you want what you’ve modelled in AARON and AARON’s drawings to be truly within the domain of the endeavor of art. Presumably that is why you’ve spent so much time running the other way from so-called “computer art”.
HC. Yes. But I should say that my goals have changed subtly over the years. For a very long time I thought AARON’s work should be indistinguishable from the work made by human artists. That isn’t quite the case any more. I want the work to look as if it has been made by an intelligence, but it doesn’t have to be a human intelligence. I am much happier now when I see the program produce an image that looks as if it had been made by somebody who is seeing the world for the first time: seeing the world from a different point of view than someone who grew up human.
BC. So, you want a whole new language?
HC. No, not a whole new language. If it was a whole new language nobody could understand it at all. I would like AARON’s work to look to the human viewer the way, for example, African sculpture must have looked to western artists at the end of the 19th century. They knew it was made by people somewhere, but these people were from a different, even alien, culture. Their art had very little to do with European traditons of art-making, yet for Picasso, looking at African sculpture provided a whole new way of thinking about painting. I would like AARON’s work to have that kind of alien-ness. I don’t want it to look as if yet another human artist had made an image of yet another human being.
BC. You give AARON a rather innocent quality, placing it just at the boundary of discovery all the time. I am wondering if you are ever surprised by any of the actions AARON takes. Does it ever seem to know something that you don’t know?
HC. Of course, I know exactly what AARON knows, but I can still be surprised. When you work on a program as I’ve worked on AARON, you make the program the heir to some subset of your own knowledge. When it plays that knowledge back to you, you can find yourself saying, “Hey, where did that come from? I didn’t realize that that is what I believe.” ln that sense the whole endeavor is quite a shocking and remarkable experience. Of course I’m not thinking of knowledge as a mere collection of facts.
I suppose I am in the position of being entirely responsible for someone else’s education. I go on feeding facts and numbers and positions and opinions and beliefs into this someone, and it turns out finally that I hadn’t really put in what I thought I had at all. ·
BC. What you said about knowledge makes sense to me because knowledge isn’t static. lt has direction. We can’t form a question unless it is from a certain knowledge base and a certain desire.
HC. Knowledge base and belief system and desire, as you say. This is the problem that faces every parent: how to create an autonomous individual that you can still communicate with.
BC. Ha! Right, you can’t create a pure entity. You are always teaching it something that you don’t realize you are teaching it. Which reminds me … you invented a special word for this in relation to programming AARON: “entitality,” isn’t it? By this you mean that the machine has the equivalent of personality. What you’re really doing is looking down on what you’ve created and realizing that there is some signature to it that wasn’t part of your original intention but that is certainly very specific to the system that you are responsible for.
HC. Yes.
BC. This all has to do with the question of what knowledge is, and whether you know what you know. ln fact, one is likely not to until some piece of knowledge is called forth by some specific need. Building a model as a way of exploring one’s knowledge produces results that …
HC. You don’t quite recognize.
BC. I don’t know how this would trouble scientists, but how does it sit with an artist?
HC. lt depends. I am always delighted when it does things that I don’t expect except for those times when I am not delighted with what it does. I never intervene on the level of a single drawing; in fact, I’ve never provided myself with any way of intervening. Yet if it makes drawings that indicate a direction I don’t want to go in, I will change the program. I’ll make whatever modifications are necessary to take it in what I thnk is a reasonable direction.
BC. Yet AARON has taught you something.
HC. AARON is teaching me things all the way down the line. From the beginning it has always been very much a two-way interaction. I have learned things about what I want from AARON that I could never have learned without AARON.
BC. So, this decades-long conversation with AARON has enabled you to build on your understanding of your own knowledge. AARON is probably the oldest, continuously developed program in computing history at this point. lt has also allowed you to create a new medium for yourself as an artist, even to redefine what we mean by art.
HC. Interestingly enough, I think the very age of the program contributes a great deal to the quality of what it does. What ever else happens after twenty years of continuous development, AARON has a kind of complexity about it that you don’t get when you sit down and knock off a program in three months or three years.
BC. Who are the people in the pictures that AARON draws right now?
HC. Oh, well! One of them turned out to be someone I taught as a graduate student years ago. One of them is a graceful black woman whom I have never met but who I think would be wonderful to talk to. The amazing thing to me is the frequency with which the drawings turn out to look like people I know. Sometimes I will be having coffee with one of my assistants in the university coffee shop and she will look across the room and say, “That woman there looks just like one of your drawings.”
Of course, there are no human models, and the program is not attempting to portray any given individual. Only once did I ever get it to portray a particular person with reasonable sucess, and I found the enterprise rather uncomfortable, boring. lt was like manipulating a rather complicated police identikit. I didn’t enjoy it at all. ln fact, I rather thought that the people AARON portrays, having no one in mind at all, were more interesting looking people. As drawings, that is; no reflection on the model.
BC. lt sounds like, even now, you are still dealing with the idea of the evocative computer that you began in the early ’70’s after we went out to look at petroglyphs. They conveyed meaningfulness even though their original meanings are lost to us.
HC. Yes. I have never subscribed to what I once called the telecommunication model of art: the artist has something in mind which is encoded in a message and sent across the art medium, or the internet, or whatever, and is then recieved and decoded, with the result that the audience understands just what the artist had in mind.
I don’t think the artist ever transmits what he has in mind and I don’t think the audience ever knows what the artist has in mind. The artist is concerned with the design of meaning generators, not meaning communicators. The power of the program still is that it is capable of generating some personality on a piece of paper; it will initiate some response on the part of the viewer in terms of what the viewer knows about human personality and human experience.
BC. Whatever the human viewer might speculate looking at your drawings, certainly, the speculation has to lead back to you.
HC. Not really to the degree you might think. If I said earlier that it was never my intention to have AARON simulate what I do, it is because I have always felt that what I do on a cognitive level is just about exactly what everybody else does. My goal always was to try to uncover the general cognitive practice, not my own particular one.
BC. What traditional artistic goals have you been escaping for the last quarter century by casting your lot with artificial intelligence? What artistic future are you indicating with your work?
HC. I am not sure I am escaping any goals, or even trying to. Oh, of course it isn’t exactly traditional to have a machine generate one’s artworks. But – in the twentieth century, certainly- art making is a highly self-reflective activity, and what is central is the degree to which the making of art contributes to an ongoing dialogue about the nature of art. ln that sense I think my work is absolutely orthodox.
As for the future, I don’t know. Public attitudes towards computers are by no means neutral. ln a market-driven society the manufacturer shoots for the biggest possible, not for the most sophisticated, market. The result is that the vast majority of users today identify the computer as a box on which to run ready made packages. That is not what I do. There is no package for what I do, and there couldn’t possibly be. lt seems to me that using one would be absolutely antithetical to the artists’ position. lt would imply that someone else is in a position to tell me what I am supposed to be doing.
BC. Or what you need to do.
HC. More to the point. I am in the fortunate position of having been in this game from the time when there weren’t any packages to be bought. There was no choice in the matter; if you wanted a program, you wrote one. That’s not the case now, and it would take an extraordinary act of insight on the part of a young artist to conclude that to have real power you must do your own programming.
BC. You sound as if you wouldn’t use a package under any circumstances.
HC. Oh no, that’s not the case. I use a computer-aided design package for designing machinery. I use a word processor that I didn’t write myself. I wouldn’t dream of writing my own communication package. I am still using my own accounting package, but that is because I can’t be bothered to learn a new one. It’s always cheaper in the long run to buy something than to make it yourself; but you can only do that safely when there is perfect accord about what the package is supposed to do.
Perhaps someone will come along who will see the readily available, off-the-shelf capabilities – Photoshop, whatever – as being just exactly what he or she wants and will produce major art by this means. It’s just not my way of doing things. I inevitably get nervous about the notion that somebody could make art without a profound grasp of the underlying disciplines involved. That doesn’t mean that it won’t happen, obviously; merely that it doesn’t look as if the future is going in my direction.
BC. You mean in the direction of free-thinking autodidacticism, I think. Well, what about the near future? What are you doing tomorrow?
HC. I have to make covers over the motors on the painting machine so that children won’t get their fingers caught.