CSCE 496/896

Topic Summary Assignment 1: Intelligent Agents

Questions and Answers

September 4, 2002

Q1: .. [It] is in regard to the problem of resolution. I am concerned with the statement that “the same action performed twice in apparently identical circumstances might appear to have entirely different effects and in particular, it may fail to have the desired effect” [2]. Namely, is the problem in the environment is apparently in the same state, the agent fails to perform its action exactly the same each time or both?

A1: In regard to the problem of resolution, two problems in the environment may appear the same, and the agent may perform its action exactly the same, and the outcomes still are different. This is due to the fact that the designer has failed to factor in other issues, to incorporate some details, for example, that may affect the outcome of an action. On the other hand, it is also possible for the agent to perform the same action and does not get the same outcome. This is especially true in a dynamic multiagent system.

Q2: Why can a dynamic environment be deterministic?

A2: The common definition of a dynamic environment is that the environment changes, and it is possible that the environment changes while an agent is making a decision. The common definition of a deterministic environment is that the states of the environment can be determined or known by an agent. So, of course, a dynamic environment can be deterministic. You can let the environment change as frequently as it wants, but as long as the agent knows what the changes are, the environment is deterministic.

Q3: Why is the balance between “reactive” and “proactive” crucial to the agent design?

A3: Reactive refers to the ability of an agent to handle events/problems in a timely fashion. Proactive refers to the ability of an agent to have goal-directed behaviors. A purely reactive agent may lose sight of the overall goal. A purely proactive agent may end up constantly thinking about what to do and not doing anything to solve a pressing problem. So, it is important to see what the problem is and strike a balance between the two.

Q4: Which architecture to use when we encounter a real-world problem? How can we balance efficiently between mobility and intelligence of agents?

A4: Each architecture has its advantages and disadvantages. When you encounter a problem, define its characteristics. Then, you should be able to pick the most suitable architecture. For the second part, usually, the more intelligent an agent is, the less mobile it becomes. So, to find the balance, look at the problem. How much mobility is needed? How much intelligence is needed? Look at the bandwidth, look at the communication cost, etc. See also A14.

Q5: In class, we were told that “Agents must prepare for the possibility of failure. An agent will have a set of actions available to it. Effective capability is its ability to modify its environment. Each action has a pre-condition.” If all of this [is] pre-wired, then the agent is obviously not intelligent. Going by the definition that an intelligent agent is an agent that learns, does this mean that learning is the ability to modify/create pre-conditions, the ability to modify/create actions, or both?

A5: You have touched upon a very important, and highly debated issue in AI. Some may argue that if the agent, even pre-wired, is intelligent if it is able to prepare for the possibility of failure, has a set of actions to change its environment, etc. A famous Turing test discussed in AI is that if a human cannot tell apart during his/her interaction with a human and another system, then that system is considered to be intelligent. And, a while back, somebody wrote a very simplistic system that was able to converse with a user. And humans were fooled into believing that they were actually conversing with a human behind the computer. Is that system intelligent? When the researchers revealed what was behind the conversation ability of the system, it was simply a set of if-then rules. So, what is intelligence? It is a very important question with many different answers!

Learning in an agent can be in terms of many aspects: learning to find out who the neighbors are, learning to find out who can do what better, learning to choose a particular action over another, learning to solve a particular problem with some particular solution, etc. And yes, it can also include learning the ability to modify/create pre-conditions and actions. See also A11.

Q6: From the book: “It should be easy to see that for every purely reactive agent, there is an equivalent standard agent; the reverse, however, is not generally the case.” Does it mean generally a standard agent has more complex behavior (from sequences of environment states to actions) than a purely reactive agent, so the function of some standard agents cannot be implemented by purely reactive agents?

A6: Yes. Exactly. For example, an agent that uses a production rule-based system to derive (a complex behavior) what to do given a situation cannot by implemented as a purely reactive agent.

Q7: For logical agents, have they discovered a wait to get around the [NP-completeness] problem? … If I remember correctly from the end of my CSE310 class Boolean Logic is NP-complete. So it would be impossible for the logical agent to solve the problem in exponential time using Boolean Logic. For agents do we need to consider cost?

A7: The easy question first: yes, we do need to consider cost for agents when we consider the application and implementation issues. In most real-world applications, agents must be able to compute fast and require small computational workload. Your website “personal agents”, your mail-filtering agents, your search engine agents, and so on, have to work fast real-time. Some use optimized algorithms, some cut corners. This is what motivates the “good enough, soon enough” paradigm in AI. On top of that, agents are distributed. An operation that requires exchange of thousands of kilobytes of data between two agents is too costly. So, we have mobile agents or mobile code. Instead of fetching the data from a remote site to a local site for processing, we send the mobile agent to the remote site to take care of the business. As for the NP-completeness of logic, actually, it depends on what types of logic you talk about. In a rule-based production system, there are heuristics that allow one to choose the most suitable rule to fire, and so on. That cuts down the time spent on searching. Search is one the fundamental issue, if not the one, in Artificial Intelligence. And logical deduction is one type of search. And this issue is still alive and well in the community. But these days, for most applications, people get around it by using optimization algorithms, approximation algorithms, anytime algorithms, just-in-time algorithms, and so on.

Q8: Is there anyway to predict how cautious or bold a BDI agent should be before designing it?

A8: Yes, to a certain extent. A bold agent is more reactive, in a way, or once a decision is made, it carries it out. A cautious agent is more deliberative, and constantly stop to reconsider a decision. There are several parameters one can look for to see how bold or cautious he/she can design the agent to be. First, how dynamic is the environment? If it is rapidly changing, are the changes significant enough to make a difference in the decision making process of an agent? If yes, then your agent may need to be more deliberative, more cautious. Second, if on the other hand, the environment is highly “malleable”, that is: an agent can repeatedly perform actions in it, and will be able to fix whatever errors it has done before, then you can afford to have a bold agent: Just do it, since if the action no longer fits the problem, we can do it again to fix it. That way, we can guarantee to react to a problem right away. Third, how deterministic is the environment? How many agents are there that can affect the environment on their own? If the environment is highly non-deterministic, then you may want to be cautious, to think twice before acting. But if it is overly non-deterministic, then you may just want to be bold and stop considering these other factors. See A12.

Q9: Essentially an agent is still a piece of software that tries to mimic human behavior and decision-making patterns. Human beings can make stupid, irrational decisions. Will the agent always make the right decision? If that is the case, how can we either as user or designer, guarantee that?

A9: Actually, when we design an agent, it is not necessary for it to mimic human behavior and decision-making patterns. This is also a popular issue in AI: descriptive vs. prescriptive. Descriptive refers to, in this case, building an agent that describes what a normal human will do, and that includes making unwise decisions. Prescriptive refers to, in this case, building an agent that does what a wise human should do. In some economic simulation and modeling, researchers build descriptive agents (to better mimic the buying and selling behaviors of the consumers, suppliers, etc.) and also prescriptive agents (to show that if the actual population follows the behavior of these “wise” agents, what benefits follow, etc.). No, an agent will not always make the right decision. And more often than not, in a real-world application, agents will only make the best decision they can, and in some instances, those best decisions are the wrong decisions. Remember, when the world is dynamic and uncertain, it is not easy to make the best decision. And agents can’t. To guarantee that, a designer has to be sure that the environment is deterministic and when an agent makes a decision, it has complete information it needs to make that decision. See also A15.

Q10: Essentially an agent is still a piece of software that tries to mimic human behavior and decision-making patterns. Human beings can make stupid, irrational decisions. Will the agent always make the right decision? If that is the case, how can we either as user or designer, guarantee that?

A10: Actually, when we design an agent, it is not necessary for it to mimic human behavior and decision-making patterns. This is also a popular issue in AI: descriptive vs. prescriptive. Descriptive refers to, in this case, building an agent that describes what a normal human will do, and that includes making unwise decisions. Prescriptive refers to, in this case, building an agent that does what a wise human should do. In some economic simulation and modeling, researchers build descriptive agents (to better mimic the buying and selling behaviors of the consumers, suppliers, etc.) and also prescriptive agents (to show that if the actual population follows the behavior of these “wise” agents, what benefits follow, etc.). No, an agent will not always make the right decision. And more often than not, in a real-world application, agents will only make the best decision they can, and in some instances, those best decisions are the wrong decisions. Remember, when the world is dynamic and uncertain, it is not easy to make the best decision. And agents can’t. To guarantee that, a designer has to be sure that the environment is deterministic and when an agent makes a decision, it has complete information it needs to make that decision

Q11: Intelligent agents are supposed to learn from their experiences, how is this implemented?

A11: Well, actually, according to my definition, yes, intelligent agents are supposed to learn from their experiences. But remember that this is not a universally-accepted requirement. Many times, learning from experiences is implemented using reinforcement learning. For example, neural networks is a reinforcement learning mechanism. In general, an agent can keep a profile of the solutions that it has used to solve problems. Suppose for each problem, there is a set of solutions. In the beginning, the agent simply picks a random solution out of the set for the problem. And then, it also keeps track of the success/failure of the solution. After a while, it knows that solution #4, for example, yields the highest success rate for the problem, so, it may tend to pick that solution more often. And in this case, the amount of data is stored is the same throughout. We simply update the success rate fro each solution. So, the agent’s experience is contained in the solutions’ success rate counter. See also A5 for other examples.

Q12: In a situation where the environment changes so drastically that the moment the action is performed it is not the right decision any more. Will the agent be able to adjust its decisions real quickly?

This depends. It may not be able to adjust its decisions in time for the problem. It may simply ignore the changes. From the design standpoint, you have to test the system in order to obtain the correct timing. When do you say, “Okay, I think I still have time to reconsider,” or “Well, forget it. I think I will just go ahead with my current decision. If it no longer applies, so be it.” See A8.

Q13: Which [agent architecture} models are active in current research?

A13: These days, the reactive and the layered architectures are quite active in current research due to the shift of multiagent systems to the real-world applications and problems. Most models these days are hybrid, by the way, a little bit of both reactive and deliberative. And many models have done some sort of the layered architecture. But the one proposed by Peter Stone and Manuela Veloso is the most well-formulated one.

Q14: I did not totally understand what the issue with mobile agents versus intelligent agents was about. It seems to me that whether or not a mobile agent is needed is dependent on the application. Is this just an issue of communication?

A14: A mobile agent needs to be lightweight. The more objects it is compiled with, the more knowledge bases it needs, etc., the more costly it is for it to travel, to be mobile. But when you build a lightweight agent, then it limits the features of intelligence you can have on the agent. And thus the tradeoff. See also A4.

Q15: Are reactive agents really intelligent? And can they be useful in dynamic environments? If they simply react with programmed response can they be useful in any but the most predictable of environments?

A15: There is not a single answer to this question. According to my definition, I require agents to learn to be intelligent. So, if a reactive agent does not learn, then it is not intelligent. And by most definitions of a reactive agent, it is not intelligent. Most researchers agree that reactive agents simply react, and do not have a goal-directed behavior. And such, it cannot be intelligent. But keep in mind that a swarm of reactive agents can exhibit intelligent behavior. Actually, most reactive agents are used in dynamic environments where “too much deliberation” cannot be afforded. For the last part of your question, yes and no. Yes, if the problem you want to solve does not expect “emergent behavior”. No, if the problem you want to solve is dynamic and unpredictable at a local level, but does expect some sort of “emergent behavior”. Some reactive agents are simple: given situation A, perform action T. They do not care about other environmental parameters. But that does not mean the environments are predictable. They simply do not care! For example, economic modeling of consumers and producers have simple agents, but adding in different “trigger” behaviors, the outcome maybe quite unpredictable. See also A9.

Q16: I am still a little unclear of the differences in layered architectures. Is this just a ways of implementing the decision making process or is it a separate process?

A16: The four architectures discussed in class can be divided into 3 basic approaches: reactive, deliberative, and hybrid. And the layered architecture is a hybrid. It says that we will do both. When there is an emergency or a task that needs my immediate attention, I will switch on my reactive mode to handle it. If the solution deviates from my overall, higher-level goals, so be it. If my deliberative module has some other things in mind, I ignore them. But when there is not an emergency or a task that needs my immediate attention, then maybe I should take a look at how well I have been doing to accomplish my higher-level goals. This is the elegance of the layered architecture. This makes sense in real-life. Suppose you are studying for your qualifying exams. You have everything planned out under the overall goal of passing your qualifying exams. But between now and the exam day, you may have other things to take care of—such as homework assignments, going to the doctor, going to a concert—that deviate from your overall goal. So, the layered architecture offers this flexibility that we do not see in the reactive or deliberative architectures.