Features 12/18/01

A journey through the mind of Barry Kort, prime mover of MicroMuse

By Leon D'souza

Barry Kort enjoys brain-teasing puzzles such as the Tower of Hanoi, on the desk in front of him. / Photo by Leon D'souza

Barry Kort is inventive, ingenious, and inspired. He represents a unique genre of the super-intellectual. Not quite the socially deprived deskbound mastermind you might expect a famous techie to be, Kort is a social technocrat. He enjoys company, and loves to get on his soapbox about issues from Artificial Intelligence to the ineffectiveness of rule-based societies and institutions.

Kort is a founding director of MicroMuse; the first Multi-User Simulation Environment (MUSE) site dedicated entirely to educational purposes. He received a bachelor of science degree in electrical engineering with high distinction from the University of Nebraska, and joined what was then AT&T Bell Laboratories in 1968 as a member of the technical staff in the Network Planning Division. Under Bell's Graduate Study Program, he earned his master's and Ph.D. in electrical engineering at Stanford University.

In 1984, when Judge Harold Greene presided over the break-up of AT&T, and made antitrust law a topic of mainstream conversation in the United States, Kort received Bell's Distinguished Technical Staff Award for sustained contributions to Network Planning. After the break-up, he went on to join the Network Technology Group at the MITRE Corporation as a Lead Engineer working on communications infrastructure for the NASA Space Station. He is a consulting scientist at Bolt, Beranek, and Newman (BBN) in Cambridge, Massachusetts, where he is involved in developing systems and concepts for network-mediated education and informal science education, including the use of computer animations and virtual communities. Kort is also a research affiliate in the Affective Computing Research Group at the prestigious Massachusetts Institute of Technology (MIT).

I had the pleasure of visiting Barry Kort at his condominium in Bedford, Mass., last week while on a short trip to Boston. We sat down to chat amid a hodgepodge of machines and other fascinating puzzles and gadgetry. In an extensive interview spanning several hours, Kort left me dazzled with his engrossing ideas on emotions and learning, intelligent tutoring systems, affective computing, artificial intelligence, MicroMuse, and model-based reasoning.


D'souza: Your research explores the relationship between emotions and learning. You have been working in this area for 16 years. Can you elaborate on your research? What got you started in this area? Moreover, what do you hope to achieve with your experimentation?

Kort: This is 1985. I was in New Jersey, working at Bell Labs, and I had met this woman, a couple of years younger than me, who was a first-grade teacher, and I had something of a relationship with her at the time. She had decided to advance her career by getting her master's in education. Her difficulty was that one of the courses she had to take was "Statistics for Education." She had tremendous math anxiety, and was very concerned about this. It also turned out, as I learned later, that she suffered from dyslexia, which is a genetically inherited condition that affects certain areas of the brain, which as it turns out are relatively important for the kinds of information processing one uses in mathematics. This is why she had all this anxiety about math.

So I offered to coach her through this problem. She got a little way into the course, and then she got to Chapter 4. This chapter was about hypothesis testing. She was absolutely stuck; she didn't know where to begin. I said, well, let me see what your homework assignment is. The first problem she was working on was a word problem. The scenario was that there is an elementary school, and the principal of the school is wandering around observing the children, and he observes that certain children in the classrooms are very eager to recite. They raise their hands, and they are very verbal. He notices that these are the same children who are getting the high grades. Those that were reticent and not volunteering to speak up were getting the low grades. So the principal forms a hypothesis. The hypothesis that he forms is that being verbal -- speaking up -- causes high grades. And the question is, analyze the principal's hypothesis. Is there anything wrong with his reasoning?

My usual method of mentoring or coaching is not just to get the answer because that kind of spoils the exercise. I try to get them to construct the reasoning by using the Socratic method -- just ask them questions until you get to the point where they can construct the reasoning systematically. In the Socratic method, you generally start with a question that might be a bigger leap than they're ready to make, and then if that doesn't work, you ask ever more simpler atomic questions until you get down to the absolutely atomic level. In this case, I wasn't eliciting any constructive responses, and I kept going into ever more elementary and simpler questions, till I got down to the most atomic question, where you basically give away the answer in the question. So I said to her, would you say that being verbal causes high grades, or maybe the other way around, earning high grades causes one to be verbal, or maybe it's just a coincidence, or maybe something else like studying the night before and knowing the answer causes both. Which way do you think the causality links point?

At that point, she had an enormous emotional release -- an epiphany. She started screaming, yelling and furiously writing these things down in her notebook. This seemed to her to be a very important idea. I was astonished at this enormous emotional response connected with the observation that she was now learning something that had totally escaped her for her entire adult life, the notion that I take for granted in science. You observe two phenomena that seem to be connected or correlated, and ask yourself which way does the arrow of causality point? Does A cause B or does B cause A, or are they both caused by a precursor? When you form a hypothesis, you have to consider all possible combinations and then test it. Therefore, in this case, the principal had formed an unlikely causal connection, and that is where she was stuck. What I observed in this episode were two clearly connected events. One was an enormously potent episode of learning where she learned an idea, which is so profound and so fundamental, and I saw this enormous emotional release that was connected to it.

Now, certainly all my life I've been aware of people learning, and I've been aware of emotions, but most of the time, the amount that you learn in any one learning is pretty small "grain size" and the amount of emotion that's connected to it is also pretty small. So I had never made the connection between emotions and learning until I saw this case where both were looming enormously large. That got me thinking. What is the connection between emotions and learning? And that's when I had my epiphany, which was that they are connected in a deep, profound, and even mathematical sense, and I spent the next several years working out the structure of this relationship. The observation that emotions and learning are connected is not so profound as determining the structure or nature of the connection. That was the episode that turned me on to the notion that learning and emotions were not independent features of being human.

D'souza: You had your epiphany. Nirvana was attained. How did you get started?

Kort: Well, most of the time, before that time, when people had emotional outbursts, it was somewhat baffling. I had no idea where this was coming from, what was the issue? or how do you respond to it? So emotions were sort of like relatively meaningless behavior patterns that I could not diagnose. It turned out that emotions were coming from a gap or an error in cumulative knowledge. I thought that maybe I could begin to make sense of these puzzling and enigmatic emotions. I hypothesized that emotional behavior was a clue to some gap or error in knowledge.

With that assumption, I thought that maybe I could figure out what it was that this person needed to learn or wanted to learn, that they couldn't recite. I understood then that if I wanted to be a better educator, I needed to pay attention to emotions and interpret them in light of the notion that they are connected to learning. So I began testing this new idea by changing the way I did education so that I was more attentive to the subtle emotions that you don't normally pay attention to very carefully, and explicitly involving the meta cognitive process, which is the discussion of the learning process, so that when people look confused or bewildered or frustrated, we mention that. We find out why the confusion persists, so that both the learner and I would sort of keep a kind of a running track of our cognitive state. What do we know correctly? What don't we know at all? In addition, about what do we have a misconception? What emotional states are we in as a function of that? If you know something, you have confidence. If you do not know it, you have anxiety. If you have an idea, but it is incorrect, you have confusion. If you have an idea about something, and it is an inadequate idea, you use it and it does not work so you get frustrated.

So you have these connections between cognitive states of partial knowledge and the associated emotive states that arise, and paying attention to their relationship helps you diagnose where the sticking points are, which then makes the educator a better mentor. The rest of my work after that was to try to develop the art of intervention in somebody else's educational development in a way that paid explicit attention to emotions. Most gifted educators are very intuitive. They are aware of emotions, they are reasoning about them intuitively, but it is not part of the conversation. It is not out front. Therefore, I wanted to make it a theory that was out front and participatory with the learner.

D'souza: Your involvement with MIT, how did that come about? Moreover, how is your research contributing toward the development of an intelligent tutoring system?

Kort: The way that happened is one of my professional colleagues, again an elementary school teacher, and this time it's a gentleman from Western Massachusetts who was an early adopter of educational technology (computers in education) using the Internet -- a few years ago decided to advance his career, in this case by getting a doctorate degree in education at the University of Massachusetts, Amherst. After he completed his education, he decided that he wanted to get more involved in research, and particularly wanted to learn how to write a grant. So he came around and chatted about it, and he said I'm going to write a grant proposal to the National Science Foundation, my first one, and I need some ideas.

So I went down on kind of a laundry list of possibilities I had been working on, including the emotions and learning stuff, which is really the most over-the-horizon stuff of all. Usually, the first time you write a grant proposal, it is just for practice because you never get it. So he said let's write the one on emotions so that I will not feel bad that we didn't get it. He did all the legwork of writing the grant. He sort of pulled the theory out of me over a period of several interviews, and wrote the whole thing up. When you write a grant proposal, you have to have an institution in which you house the grant. He already had a kind of an affiliation with MIT in one of the departments there. In this case, because it had to do with emotions and learning, that particular group was not really the right one.

So he went to a friend of mine, a professor at MIT, Roz Picard, who created the field of Affective Computing, and he proposed to write this plan with her as the faculty principal investigator, and then she pulled in a second faculty member to flesh out the plan. So he wrote this grant proposal and submitted it to the NSF. Now, he sent it in to this one division, and it went directly to the NSF. They said that the plan was very interesting, but not quite within their charter. So they sent it to this other new directorate that was just getting started called Research on Learning and Education (ROLE), and unexpectedly, they funded us for two years. So now we have this project at MIT Media Lab called the Learning Companion or Affective Learning Companion that's within the Affective Computing Research Group, and the Narrative and Gesture Group, which is a sister group. We have two faculty members and three graduate students on the project, besides my colleague and me. We are exactly a year into the grant, and there is a very good chance that the NSF will be satisfied with our work and will give us at least a third year of grant money.

D'souza: What is Affective Computing?

Kort: The first thing that your readers should know is the URL of the Affective Computing website. It is www.media.mit.edu/affect That's the front page and there is a lot of stuff behind it. Affective Computing was founded in the mid-90s by Professor Roz Picard, a longtime friend and colleague. Her idea was to bring into the realm of computing the ability to recognize process and respond to emotions. Computers are autistic. They are oblivious. So if users are sitting at a machine, and they're upset, or frustrated, or angry, or bored, the computer is essentially unaware and unresponsive, unlike a person who has some kind of empathy tuned in. The idea is not so much to give computers emotions in their own right, but to give computers the ability to process emotional information, make sense of it, and do something -- make an adaptation -- so that for example, if the computer is presenting information that's too complex, or coming too fast, or sort of off-track, it could recognize that and adjust the pace, the complexity, or the direction of the material so it's more appropriately matched to the individual. The idea is to get the user to make the maximum productive gain, and derive the greatest emotional satisfaction, a sense of progress.

D'souza: Have you begun to test your model? Has it been applied in any Boston area institutions?

Kort: Well, there are two ways to test a theory. The goal of the project is to build a computer tutor that has this ability, but we're not there yet. We do not have any educational software that has affective computing capability. So what we're doing now is simply testing the model and the methods with humans acting as what we call "wizards" We've done many experiments, both informal and formal, over the past 14 years at the Boston Museum of Science, where I volunteer every Saturday. I practice this art. People come in; I have a bunch of puzzles, and as they work the puzzles, I determine whether it is appropriate to intervene to keep them productively engaged and emotionally satisfied with their progress. So partly, the art of developing this process is a human thing, and we have done carefully documented experiments in the classroom where the affective part of it was simulated by a human playing the role of the emotionally intelligent being. However, the computer was presenting the subject. So the wizard behind the curtain was providing the part of it we have not yet programmed into the computer, and that's sort of developed what will end up being the content of the code.

D'souza: You were founding director of MicroMuse , the first Multi-User Science Education Network. MuseNet won the 1996 National Information Infrastructure award for pioneering innovations in children's education via the Internet. What really is MicroMuse? How did it come about?

Kort: MicroMuse started around 1990 as a variety of what's called a Multi-User Dimension or MUD. It is essentially an evolved version of an adventure game. The early adventure games were text adventure games that were single-user. The author created the world. The user would sort of enter this world and become the central character in this adventure. You would have to solve the puzzles, and reach the goal. They were a lot of fun, but they were single-user, and the author of the program authored the world. The MUD has the same kind of structure, except that there is more than one person in there, in real time, and the world is editable and buildable by the participants. It may not even be authored at all by any previous author. So the participants' form a community, and they build a world where they can communicate with each other. It's a virtual world that's created out of text within a program that's called a server, which runs on a central computer. People connect to it, log in as a character, and then enter the game.

The early MUDs that came out as a result of a research project at Carnegie Mellon University in the late '80s ended up being used for social and recreational purposes. Young college students, mostly undergraduates, tended to frequent these. I had discovered this MUD phenomenon in the late-80s, and thought it was a very interesting technology. It had very good potential for education, and I had enjoyed the original text adventure games of the early-80s that were single-user worlds. I thought, well let's try this out for education. I had met on one of these MUDs, a senior in computer science at California State University - Fresno, who I became friends with, and I had adopted a character on these MUDs that was just like my science museum role. I had this collection of puzzles and gadgets that were all programmed into the MUD language, but they were all replicas of the actual things I had in real life, and they behaved the same way.

He found that charming and interesting, and he invited me to join him in building a new MUD that didn't have so much of dungeons and dragons or cliquishness. He wanted to build a system that was a little more open and friendly. So we created one of the first educational MUDs called MicroMush, and it ran on a little Sun 360 desktop computer in the computer science lab at Cal State -Fresno. MicroMush was later transferred onto a nice computer at MIT's Artificial Intelligence lab, and we renamed it MicroMuse. The name MUSH was a derivative of MUD. MUD was an acronym. There was a different dialect called MUSH, which didn't mean anything, and we made yet another dialect of this language which we decided to call MUSE, which stands for Multi-User Simulation Environment, because it's a world in which you build models. Everything in it is a model of something either real or imaginary. MicroMuse really got off the ground when we ported it to a much bigger, more powerful machine. The project grew and we attracted some very talented people. It has now run for about 11 years. Over the years, some 5000 people have participated in MicroMuse.

D'souza: Model-based reasoning is one of your mantras. MicroMuse, in a certain way, involves the creation of models for various scenarios. Tell me about model-based reasoning. Why is it important? How can we teach model-based reasoning to coming generations?

Kort: Model-based reasoning is what you do in science, and not just in science, but science creates theories and models which purport to be representations of phenomena that we observe, and you can compute or run the model, or analyze the model and gain insight. For example, you can make predictions from the model and see whether your predictions are borne out.

So model-based reasoning is the mainstay of science, and also of engineering, and engineering systems theory. Few people in the lay public are engaged in model-based reasoning, or are even aware of it. Most people are rule-based. The world sort of imposes these rules on you, and you simply follow the rules and you'll be safe. However, the problem is that rule-based systems are not very powerful. You cannot do a lot of stuff with rule-based methods of reasoning. So I went on a jag about introducing model-based reasoning in science education. How do you get the kids to reason like a scientist? How do you get them to understand what is a model? What is the difference between the model and the thing that it replicates? How do you build a model that is an accurate representation of something real? Moreover, how do you use models scientifically to gain insight and understanding and make predictions in a practical way? So what MicroMuse did is it opened up an opportunity for ordinary people who did not have access to big brand computers or huge mathematical systems to do very simple modeling activities.

The simplest model is an object that has the name that is the same as the thing it represents, but it actually has no content. Therefore, if you have an object called "a red balloon," then that is exactly what it is -- a red balloon. It has no behavior. But then you can begin to add to this red balloon object, behavior. So if you pick it up, you drop it and it flies away, you poke it and it breaks. You begin to add features to it so that it is not just a picture or a representation of a red balloon. It actually begins to behave like a real balloon. From here, you can move on to models that are more complex. Anything you can imagine.

As you model more elaborate and complex things, the content of the model begins to flesh out lots and lots of elements, descriptions, code, behavior, and you can interact with the object on the computer and it behaves as you would expect it to behave. So the children rather enjoy this opportunity to have access to a modeling environment where you could, without spending years studying computer science, in a matter of maybe a few weeks, learn enough of the language to build simple but fun models. Then you had other people in there who could interact with you and the stuff that you had built. Almost like an instant show-and-tell. You had this pride of authorship, and this issue of creativity and cleverness, and making the model behave in sophisticated ways, which is a programming challenge. So you had all of this opportunity to engage in model construction where the models could be quite sophisticated. Unlike building, say, a model out of plastic or Styrofoam, this model could have behavior. This sort of expanded the opportunity for people to discover, participate in, engage in, and enjoy, the model construction process. And I thought that this was a good thing to introduce children to, and it sort of got them into this paradigm that science educators are trying to get children into, which is understanding what a model is, what a theory is, and how a model could be made to resemble and behave like the thing which it is a model of.

D'souza: What age groups has MicroMuse been able to attract?

Kort: Initially, it was governed by those who had access to the Internet, because around 1990, almost nobody had access to the Internet unless they were connected with academia. Then, eventually, undergraduate students were given Internet access. So we had many undergraduate college students. Gradually, high schools started getting Internet access. Actually, first from Canada. The Canadians had lot more connectivity before the Americans, so we got many Canadians. In addition, we were getting kids logging on from the various free nets. There were some in Ohio that were started up, and we were getting an early wave of adolescents from those services connected to the MUDs, as we were one of the more fun things young users could get on to.

As middle schools and elementary schools got access, and as people got home access, the access population grew and we worked our way down to essentially ever younger target population, until we got to sort of about the youngest age that could successfully handle the reading and writing load that a text-based world requires. That is normally about fifth grade. So we found that from about fifth grade and up, the kids could handle it, and the optimum age range was from about fifth grade to about tenth grade, because these are the ages when you're not old enough to drive a car, so after school you're stuck at home.

Now, after kids are old enough to drive a car, which is around 16, other things begin to draw their attention away from things like MicroMuse. So, over about the age of 16, only the ones that really, really loved this stuff would stay on, and younger than that, this was the most fun thing to do after school. We say we targeted K-12, but younger than about eight or nine is difficult, except for extremely gifted kids. We got many gifted kids, many home-schoolers, and children of academic or research professionals.

D'souza: Tell me about the MuseNet K-12 project. As I understand it, this project involved salvaging surplus and donated computers for schools. Could you elaborate?

Kort: Some schoolteachers discovered MicroMuse. We had a fair amount of publicity, and a few schoolteachers learned about us and decided to check us out to see if we were an interesting resource. Several schoolteachers started projects to use MicroMuse itself, or a sister system like MicroMuse. One school in Massachusetts went down to grades three through six. We started the project and actually gave them some workstations so that they had enough equipment. There was another project in Phoenix, also an elementary school project, which targeted inner-city schools. Inner-city Phoenix has a lot of black, Hispanic, and Native American students, many of whom are from very poor socioeconomic classes, very disadvantaged. Many from broken homes and drug-infested communities. It is pretty hard to get these kids to learn anything. These schools started a sister program like MicroMuse with our help, and the results were phenomenal. Kids would come in early, stay late, and they loved to use the system. They were learning literacy skills. Both the Massachusetts school, and the one in Phoenix, had this phenomenal result. Teachers were astounded. There were other K-12 projects, besides these two which I was most closely connected to. The projects reached their peak around the mid-90s.

D'souza: Artificial Intelligence. You've studied this closely. When you refer to emotionally intelligent systems, this is essentially what you're alluding to. What is the future of AI as you see it? Anything like what we see in the movies?

Kort: Eventually, yes. The movie, 2001: A Space Odyssey was probably the most famous one that introduced the notion of advanced intelligence. The story was written about 20-30 years ago. So it is set in the future. Now the "future" in which the movie is set is actually in the late '90s. HAL was supposedly born in 1997. On the occasion, there was an editor who said, let's see how much of Arthur C. Clarke's predictions have come true. So he contacted a bunch of people in AI and said, from your perspective how prescient was Arthur C. Clarke? What of HAL has, in fact, come of age? What hasn't come of age? What have we exceeded? His book was called HAL's Legacy. My friend, Roz Picard, was asked to write one of the chapters. Her writing was called "Does HAL Cry Digital Tears? Computers and Emotions." So there is a lot of interest in how prescient were science fiction writers like Ray Bradbury, Arthur C. Clarke, and Isaac Asimov, who envisioned intelligent robots.

If you look at the history of the development of machine intelligence, the first computers were wizards at mathematics and logic. They had math-logical intelligence better than most humans from the get-go. The next big push was in language skills. With this came word processing in about the 1970s. The next 20 years were spent in giving computers what amounts to about a sixth grade level of mechanical skill in language. They were not going to write the great American novel, but they could do the mechanics. Grammar and syntax. So now we had two intelligences, we had math and logical intelligence, and language intelligence. About the same time, there were people working on robots, and robots have body-kinesthetic. They can stand up without falling down. They can maneuver. That was the third intelligence. People like Ray Kurzweil gave machines musical intelligence, the ability to do music processing like word processing, and even to compose music. The machines, it turns out, were pretty decent composers, because music composition is kind of a mathematical thing. There was enough theory there that people like Kurzweil could create machines that could compose Jazz, Latin Jazz, classical, or some other genre. And that was like elevator music. It was decent. It was not Mozart, but it was probably better than a lot of stuff that comes out.

We now had four intelligences -- math-logical, verbal, body-kinesthetic, and musical. However, the really hard intelligence to get was emotional intelligence, and that's partly because computers couldn't get the data, and partly because there wasn't any theory. It you study the theory of multiple intelligences, the one that really counts the most is emotional intelligence. It's hard to define because unlike math and verbal where there is lots and lots of curriculum to teach people, there really wasn't any express curriculum for teaching people emotional intelligence. You had to kind of pick it up as you went along. There are some intelligences that you can learn, and others that must be taught. Emotional intelligence is in-between. Affective computing is actually pushing the frontiers of emotional intelligence. We now understand emotions well enough to be able to explain to a computer how to reason about emotions.

Ray Kurzweil and others look ahead and envision what we call sentient machines. A sentient being is on a journey through some space it resides in, and as it explores its space, it encounters stuff, which it then has to make a map of. This business of exploring a territory that you live in, building an internal mental map or model, and then using the map or model to navigate to achieve goals within your world, is the fundamental of sentience. We actually have on MicroMuse, a sentient robot, and as he walks, he maps the MUSE. So if you ask him how do you get from here to there, he will tell you the shortest path, that is if he knows the shortest path. He is essentially an elementary mapping robot, which is sort of the beginning of sentience. What's interesting is for example, if you change the layout, if a path is suddenly blocked, he'll go there and get blocked, and he behaves just like a human would be expected to behave when knowledge unexpectedly proves to be obsolete or incorrect. Once computers become autodidactic learning systems, the breakdowns and the failures, the diagnostic messages that we're used to, will manifest themselves in a much richer way, and I claim that in humans, the breakdowns in learning manifest themselves as emotions, and they're going to appear analogous in computers.

As soon as computers evolve to become learning systems in their own right, they're going to display affect, they're going to display curiosity, and confusion, and puzzlement, and frustration and insight, and satisfaction, and confidence.


Archived Months:

September 1998
October 1998

January 1999
February 1999
March 1999
April 1999
September 1999
October 1999
November 1999
December 1999

January 2000
February 2000
March 2000
April 2000
May 2000
June 2000
July 2000
August 2000
September 2000
October 2000
November 2000
December 2000

January 2001
February 2001
March 2001
April 2001
May 2001
June 2001
July 2001
August 2001
September 2001
October 2001
November 2001
December 2001