CSCE 496/896

Topic Summary Assignment 3:

Distributed Problem Solving and Planning

Questions and Answers

September 26, 2002

First of all, I would like comment on a general theme that I have observed from your questions. A majority of you have talked about “if a multiagent system is so complex, and is not guaranteed to work, why don’t we just program the agents to make sure it works?” Do not forget about why we need to use a multiagent system in the first place. People working in the multiagent systems area have different objectives. Researchers study it to understand communication and coordination. Programmers use it to better solve a problem, or even to just solve a problem. In most cases, a multiagent system is chosen because it is necessary. So, we must understand why it is necessary. We thus have to address the issues such as uncertainty, dynamic environment, and so on as we have discussed in Chapters 1 and 2. And within that kind of environments, we cannot guarantee that programs will work, let alone work efficiently. So, it is often not fair to ask the above question.

A programming solution using a multiagent system cannot be taken lightly. It often is the only solution, and if so, we can either reap the resultant simplicity benefits in the agent design or bear the complexity consequences imposed by the coordination protocol.

Second of all, distributed problem solving is a much older subject than multiagent systems. Thus, distributed problem solving does not necessarily include agents. In your response to the stupid question, quite a few of you refer to the dryer and washer as agents. But are these machines agents? Remember what we say about agents. Is a dryer autonomous? Is a washer autonomous?

And finally, some of you are still unable to grasp the elegance of multiagent systems as a programming paradigm. It is a paradigm in which each agent has partial control in the environment and each agent can make its decisions to accomplish tasks. This is the power of a multiagent system. If we do not make use of this, then we do not need to build a multiagent system.

Q1: Is the agents’ ability to work with others largely a matter of programming?

A1: This is a very good philosophical question. Different AI researchers will give you different answers since most have different views about what an agent should be. Remember that an agent lives in an environment. Sometimes it is impossible to predict how other agents behave or how the environment behaves. Thus, an agent that is able to adapt and learn on its own may be able to work better with others. But if the environment is well predictable, then an agent that is not able to adapt or learn can do just as well as a learning agent. So, yes, in some sense, the agents’ ability to work with others is related to how you program them, but it is also related to what kind of an environment they will be working in.

Q2: Wouldn’t you just make sure to use agents that would cooperate?

A2: Not really. That would defeat the purpose of having autonomous agents. We want the agents to make the decisions to whether to cooperate. That is why agents are needed. If we could make agents cooperate always, that means we would have known about the environment and the problem domain very precisely during the design phase. Sometime, we do not have that kind of luxury. As a result, we can design the system such that the agents collectively exhibit a coherent behavior in which each agent makes decisions to take care of its well being and then the system’s. That way, we simply the problem into local subproblems.

Q3: Are the agents that are used not always under the control of the system planner? Is this all a trade off in creating an efficient system?

A3: This question is related to Q2. No, the agents are not always under the control of the system planner. Remember that a multiagent system may consist of several agents built by different companies/programmers. Each agent may be under control of one company, but other companies have not direct control over that agent. On the other hand, since agents live in an environment that they do not have complete control over, a programmer/company does not have total control over its agent. Why? Because the agent will be affected by other agents in the environment; the agent does not live in a closed world! We could treat this as a trade off in creating an efficient system. Sometimes that is good. In traditional programming paradigms, we always have to think too much and try to anticipate all problems and solve them, laying out plans for every user. In the multiagent system paradigm, we look to solving problems for one user (agent) and let the agent get help from other users or solve their problems on their own. In that way, a designer may not have to plan too much. And a solution may be reached earlier as well. (We will see this evidence in Chapter 4 when we talk about bi-directional searches.)

Q4: … [an agent] has to dump the whole plan and start with a new plan. This is a waste of resources of its own and other agents … this particular agent might become infamous for bad planning. Is this affordable?

A4: There are two issues here. The first one is whether dumping the whole plan and starting a new one is affordable. First, this is sometimes necessary. Due to the dynamic and uncertain characteristics of the environment, a plan may not work out. Is this affordable? If the failure of an old plan does not cause a catastrophic failure in the system, then the agents should try to come up with a new plan. But, agents have limited resources. Starting a new plan may be too costly. But if it is necessary, then the agents have no choice but to start a new plan. So, the problem comes down to this: should we design agents so that the plans always work? We cannot. So, the second problem is: how can we make re-planning more affordable? (1) agents can make better plans in terms of flexibility, (2) agents can learn, (3) agents can cooperate more closely, and so on.

Now the second issue is about the reputation of an agent. Recently, this has become an important research issue. How do we measure the reputation of an agent? Should an agent with bad planning skills be labeled infamous? Is this affordable? Yes, this could actually be beneficial! How? If the agents, after several failures, learn that agent A1 is particular bad at planning a solution for problem type P, then next time around, they will not ask A1 to plan for a problem of type P. Thus, through learning, the system becomes better at planning. Researchers have intentionally made use of the “reputation” of agents to better perform task allocation (an agent that does task T well should be given the task more often) and result sharing (an agent that always gives accurate measurements should be given more weight for its results).

See also Q14.

Q5: How to exactly understand “When the space of possible interesting results is large compared to the actual results that are generated, then communicating results make sense. But when the space of results formed is large and only few are really needed by others, then sending requests (or more generally, goals) to other s makes more sense.” In the fourth paragraph of page 137?

A5: First, we have to understand that we want agents to communicate efficiently and effectively. That means a message should be useful. Now, when the space of possible interesting results is large compared to the actual results that are generated, that means many results are interesting. So, agents should always send their results to other agents. In this case, it is efficient since it is very likely that the message will be useful. The phrase “communicating results” means that the agents send their results to other agents without being asked. On the other hand, if the space of possible interesting results is small compared to the space of actual results, then most results are useless for other agents. In that case, agents should not send their results to other agents without being asked. Instead, agents should request for particular results if they want them. That way, we prevent the agents from exchanging useless messages. See also Q12.

Q6: Distributed planning can be thought of simply as a specialization of distributed problem solving in many respects. What are the particular features of planning problems?

A6: First, a plan is a solution that satisfies constraints, requirements, and goals. Now, what are the particular features of a planning problem? A plan has a schedule. It also has a partial order of tasks. A plan may require synchronization. A plan also does not always work. A plan may have alternatives.

Q7: … task decomposition is difficult and that it increases the overhead for both agent computation and communication. Are there any studies or metrics available that address these issues? Is there a method to determine the best balance for computation time … and communication time?

A7: Actually, task decomposition may or may not increase the overhead for agent computation.

Yes, there are studies/metrics that address these issues. Usually, communication is measured in transmission cost (delay, message sending, etc.) and message processing (composition, parsing, understanding, etc.) For computation, we look at storage space, memory space, CPU requirement, number of milliseconds required, number of CPU cycles required, number of disk accesses, etc. In most studies, comparative analyses are performed. For example, we see whether a particular design improves the computation performance or requires less communication. Also, some studies assume communication to be free and unbound, and thus concentrate only on reducing computation load. More importantly, there is not a single method to determine the best balance for computation time and communication time. The rule of thumb is to have a good enough balance that works for the agents. It depends on how costly computation time is and how costly communication time is. Whichever is costlier, you try to minimize that one more.

Q8: a multiagent system … task decomposition can be a timesaver … a distributed design paradigm … besides these two, is there another reason to design the system in this manner?

A8: Yes, there are many other reasons: flexibility, robustness, scalability, responsiveness, timeliness, confidence, efficiency, etc. These are issues we have talked about in Chapter 2.

Q9: In distributed problem solving, multiple agents can work in parallel in solving one problem. Is that in any way related to the grid computing?

A9: Remotely, yes, the two are related. But they are two very different topics. Grid computing is an infrastructure, taking advantage of parallel processing capabilities of computers. Distributed problem solving is how you could decompose a task into subtasks and solve the overall problem effectively and efficiently among distributed processes. Grid computing can only do as well as parallelism allows. Distributed problem solving takes it a level higher—it is a paradigm that deals the fundamental issues of a problem.

Q10: Can runtime plan coordination be used for purely reactive agents?

A10: Yes. A purely reactive agent can observe and react in a situation-action manner. It can keep doing the same action as long as the situation persists. Thus, the coordination here is based on the belief that the situation will be solved sooner or later.

Q11: For what types of agents has interleaved planning execution been successfully applied?

A11: Rational, deliberative agents. Agents that know how to minimize changes in their re-planning so as to not affect other un-related agents.

Q12: Is distraction among agents always bad, what about optimization problems? Does distraction imply that agents will converge on the same solution?

Q12: According to the book, distraction is defined as the phenomenon of changing the course of an agent’s search due to received messages. There is negative distraction, but positive distraction also can occur. To address the questions, first let us review what we think of agents. In general, we want each agent to be autonomous, to be able to handle its part of the environment, to be able to good at solving its particular set of problems. This is not always the case. But in general, it is a good metric to see how worthwhile a multiagent truly is. So, now, if there are too many messages exchanged among the agents, halting agents’ current searches, modifying agents’ solution paths, this may encourage collusion. It may not necessary converge on the same solution. Collusion may be a dead end as well as a solution. Too much distraction from an agent who has incorrect data may influence other agents into believing in a wrong approach, resulting in a failure. But there is positive distraction. It is possible that an agent, working on its own, is heading into a wrong solution path (e.g., a local minima or maxima in a hill-climbing algorithm). In this case, if the agent receives a message that steers it from that path just in time, or provides a backtracking alternative from the wrong path, then the message is useful. But, to constitute a distraction, there has to be many messages or results shared. I can see one example: if an agent is trying to control a fast-moving flying object, it needs constant messages from other agents controlling other objects in the airspace.

As we have discussed in the class: too much communication consumes bandwidth and processing resources, too little communication may result in decision making based on stale/out-of-date information. It is a delicate balance to achieve an optimal usage of communication. Usually, researchers look for a range of acceptable communication: no more than the limits, and just enough to ensure effective and efficient coordination.

See also Q5.

Q13: How does the distributed hierarchical planning work?

Q13: I have specifically talked about this in class. Still, one student had this question in his Topic Summary. This is an important and elegant concept. So, I will re-iterate what I have said in class here. First, what is hierarchical planning? It is done through abstraction. For example, if I know you can give me output O1 given input I2, then I do not really need to know how you achieve O1 given I2, as long as we both assume that what you do will not affect me. This is abstraction. This allows two agents to come to an acceptable plan quickly, without having to examine every step of an action—in this case, the steps that generate O1 out of I2. The use of abstraction is evident in object-oriented programming (OOP), and one could infer its benefits from the context of OOP.

Now, what is distributed hierarchical planning work? If agent 1 needs to perform 4 tasks generate O1 and agent 2 needs to perform 5 tasks to generate O2, now, agent 1 and agent 2 needs to make sure that what they do do not conflict with each other’s tasks. One way to do this is: agent 1 tells agent 2 about the 4 tasks that it will do, and agent 2 tells agent 2 about the 5 tasks that it will do. They will try to look at possible resource contention, blocking, and so on. This is indeed cumbersome.

Now, if agent 1 abstracts the tasks into a blackbox and agent 2 does the same, what will happen? If the abstraction is well-done, then agent 1 can simply let agent 2 know about its input and output states, and vice versa. And that is it. The interaction is much more simpler. The two agents can come to a deal much more quickly.

Of course, the more abstract it gets, the higher the chance is for the agents to agree and only to find out the plan does have conflicts later.

This is how the distributed hierarchical planning works.

Q14: How to reduce [re-planning]?

Q14: There are several ways. First, make a good first plan. Second, make a first plan that is very flexible. Third, make a plan that is adaptive.

Now, when we do have to re-plan, we can have the following strategies to reduce the process of re-planning: (1) learn from previous mistakes, (2) obtain updated information from other agents, (3) have contingency plans readily available for quick fixes, and so on. If we go to the extreme, we can actually ignore failures and keep trying the same plan until the problem goes away.

See also Q4.

Q15: [Can we say that the best way to implement the distributed constrained heuristic search is to use a blackboard, or to have one agent do the planning, because otherwise it would have too much communication overhead?]

Q15: Remember this: first of all, figure out whether you need to use distributed constrained heuristic search. If you need to, then there must be a reason why. And given that reason, you know that there is no way you can have a single agent doing the planning alone! This is absolutely critical. You must understand that multiagent systems is a necessary paradigm for some problems. Given that, you have no choice but to have distributed solutions. If the distributed search is required, then it is required. Thus, “to have one agent do the planning” is not an option. Do not lose sight of this.

Now, however, we can use a blackboard. But that does not necessarily cut down the communication overhead. Remember, in a blackboard design, every agent posts every thing on the blackboard regardless of whether the message is useful or not. Try to recall why blackboard is useful. And if a blackboard is suitable for a particular problem, then it may be worthwhile to implement the distributed constrained heuristic search using a blackboard.