CSCE 496/896

Topic Summary Assignment 5:

Distributed Rational Decision Making

Questions and Answers

October 17, 2002

First, I would like point out some issues not addressed in your response to the Stupid Question.

What if the agents know the voting rules will change during a voting process? What does that do to the agents’ reasoning process? What if the agents know the voting rules may change during a voting process? What does that do to the agents’ reasoning process with that kind of uncertainty?

From the viewpoint of a multiagent system (MAS) designer, what does that mean? Could you design a coherent and competent system? Can the system’s performance or outcome be predicted? Do you have to design new “adaptive” features into your agents to make them adapt to the rule changes? How much more effort would that incur on your design process?

Q1: I don’t really understand why an agent would need to be insincere.

A1: This is a philosophical issue. The key here hinges upon how we prevent an agent from being insincere. If we can, that implies that we (the designers) can guarantee the best payoff to that agent (because we want to eliminate the motivations for that agent to be insincere). As a result, we will end up with a more coherent and successful MAS. So, we do not really want agents to be insincere. But by assuming them would be, and by trying to eliminate that behavior, we will have a better design.

Of course, in multiagent simulation and modeling of economy and games, we do want to have insincere agents, so we can test our design or multiagent system under different, observed human behaviors.

Q2: If cheating within a coalition makes it non-stable, does simply cheating in singular dealings make a system non-stable?

A2: This depends. If your system benefits from insincere agents, then cheating in singular dealings may be desirable. But in most literature and MAS research, researchers working in real world applications (e.g., trying to come up with a MAS that works for some task allocation an resource allocation problems) do not want any cheating in any sort of dealings. That would be quite counter-productive.

So far, I haven’t encountered a real-world MAS that utilizes cheating to achieve system coherence and competence. So, in general, the answer to the above question is yes.

Q3: Auction being a very fast process and decisions need to be made dynamically, how much is it feasible to use agents to do these transactions while they also have to do computations, communications and also observations. Can we have different agents doing different tasks and acting as a group?

A3: I am not sure that I understand this question fully. I will try to respond here. There are two things here. First, auction is a logically fast process. For example, bidding goes much faster than having an agent asking door-to-door for buying prices. In bidding, agents who are interested bid, agents who are not interested do not bid. So, from the standpoint of an auctioneer, it does not have to communicate with every agent. Second, as a designer, you have the power to slow things down. Decisions can still be made dynamically.

In short, auction is fast because it cuts down communication and subsequent computations and observations. Try look at this from a designer’s point of view, not from an auctioneer’s point of view, or a bidder’s point of view.

Q4: For the social welfare part, how can we judge the welfare of each agent, especially for the situation that the task can’t be done by only one agent?

A4: This is exactly we can judge them. First, a task must be decomposable. So an agent will do subtask 1, another agent will do subtask 2, and so on. The welfare of an agent is determined by its individual rationality. If it thinks it is rational to do so, then it will do. What is individual rationality? If it gains more by cooperating than not, then it is rational for it to cooperate.

Q5: In a Nash equilibrium, an agent chooses a strategy which is the best response to other agents’ strategies. [What would cause a subgroup of the agents to deviate in a coordinated manner?]

A5: Suppose in a Nash equilibrium, there are three agents: A1, A2, and A3. Suppose each agent has two options: sell (S) and buy (B).

And here are the utilities:

Options			Utilities
A1	A2	A3	A1	A2	A3
S	S	S	0	4	10
S	S	B	1	1	1
S	B	S	1	1	1
S	B	B	1	1	1
B	S	S	1	1	1
B	S	B	1	1	1
B	B	S	3	4	10
B	B	B	1	1	1

So, when A1 sells, A2 sells, and A3 sells, A1 will get nothing, A2 will get a utility of 4, and A3 will get a utility of 10. However, when A1 and A2 buy, and A3 sells, A1 will get a utility of 3. This is actually a Nash equilibrium. However, it does not prevent A2 and A3 to deviate if they are self-interested. For example, if A2 and A3 do not want let A1 gain, they can form their own subgroup, since they still gain the same utilities. This is one example where a subgroup of the agents would and could deviate in a coordinated manner.

Q6: If we design a system with the … distributed rational decision making protocols, how do we choose which one to implement? [If the problem] situation changes, what should we do?

A6: First, how do we design a multiagent system (MAS)? From what we have learned in Chapters 1, 2, and 3, we first want to define the problem. We then decide the issues: communication costs, computation costs, distributed resources, task allocation, resource allocation, etc. After that, we pick the best protocol for our needs. For example, if communication is very costly, then bargaining is not a good idea. If there cannot be a centralized module, then auction may not be a good idea. If it is for resource allocation, then contract nets may not be a good idea. And so on. Apply what you have learned in the previous chapters in Chapter 5. As I have said in class, Chapter 5 teaches us the many wonderful ideas about how we should design our agents under complete information and rationality. However, these ideas are not applicable directly for our agents because a MAS is dynamic and uncertain. Thus, we have to make use of the strategies and evaluation criteria that we have learned in previous chapters to truly apply these ideas in our MAS design. This is a very important matter. This is one of the most important things that I want you to learn in this class: how to apply MAS technologies?

Q7: How does a multiagent system determine if it is in a Nash equilibrium, or finds a solution that is Pareto Efficient? Enumerating all the possible states is costly in a sizable system.

A7: No, you do not let the multiagent system determine. It is next to impossible at run-time. Remember this. Why do we design a multiagent system with distributed rational decision making agents? The goal is to find local strategies that are easy to compute for each agent such that the system will exhibit an emergent or coherent behavior when every agent carries out its own rational reasoning accordingly. Do you see the elegance of the multiagent system paradigm? We want to guarantee Nash equilibria or pareto-efficiency not by doing it from a global point of view or from a centralized decision maker, but by specifying what each agent should do locally. So the implementation of the system is simple, and the computation is minimal. So, the system does not have to enumerate all the possible states.

This is probably the most important idea about this class. I have discussed this when we covered Chapters 3 and 5. Do not forget this.

See Q9.

Q8: How easily can the protocols mentioned in the chapter be applied to problem in general?

A8: can be quite easy to apply the protocols to problem in general. For example, suppose you want to build a system for a cyber traveling agency. Software agents are located all over the world. Each agent plans trips and finds the best traveling plans and so on. Now, we can have an auctioneer. When this auctioneer encounters a task in the environment (“find the best traveling plan for location X”), then it can announce this, and the software agents will bid for it truthfully according to their utilities. And the auctioneer will obtain the best deal. It is also pareto-efficient. So that is one way. The protocols can be easily applied to many resource and task allocation problems.

Q9: Does finding an optimal solution require extensive expertise and planning on the part of the programmer?

A9: No. The very idea of distributed rational decision making is that one can specify a set of simple, local strategies for an agent and if every agent does its job rationally, then an optimal solution is guaranteed. This is the elegance of distributed rational decision making, and very useful to design multiagent systems. See Q7.

Of course, in the case where the problem cannot be modeled under distributed rational decision making, which is very common, what should we do? In that case, we no longer can talk about finding an optimal solution. In these cases, we talk about sub-optimality, good-enough soon-enough, etc. As I have discussed in class, in situations where agents’ knowledge of the world is uncertain, noisy, and dynamic, optimal solutions cannot be guaranteed. So, now we do need domain expertise (e.g., heuristics) and a programmer’s problem solving skills to try to achieve that. This is also where Artificial Intelligence comes in. For example, we can build an agent that learns. If it does not find the optimal solution the first time, maybe it will after learning how to perform a task 100 times.

Do you see the trade-offs? There is a debate here. As discussed in class, distributed rational decision making theorems are elegant, but they are not generally applicable to complex problems. Optimal solutions thus cannot be expected. So, usually, since optimality cannot be achieved, you make do. You can still derive sub-optimality based on what you have. But the solution’s quality will not be completely guaranteed.

Q10: Can you give an example of a dominant strategy that is stable?

A10: See Q5. If we use the social welfare as the utility, then the combination where A1 and A2 sell and A3 buys is the dominant strategy, and it is stable.

Q11: Is it possible for one negotiation protocol to meet all evaluation criteria?

A11: It is possible, depending on the domain, the problem, and the application. Actually, it is more often than one might tend to expect. If an agent is individual rational, and it can find a solution with another agent that is acceptable to both, then the solution can be designed to be pareto-efficient, and if that is the case, then the solution is stable. If both gain from this solution, then social welfare is achieved. So, now, what we have to do is the computational, communication and distribution efficiencies. But in complex problems, where information is noisy, dynamic, uncertain, and so on, it gets more difficult for a negotiation protocol to meet all evaluation criteria. And the first one to go is usually individual rationality. As the result, some research has touched upon bounded rationality. The second thing to go is pareto efficiency. But usually, stability and social welfare can be maintained by forcing an agent to commit to do its best. So, even if stability cannot be maintained, it is still for the good of the entire system.

Q12: Can agents always be motivated to tell the truth?

A12: No. There are two views here. First, is it necessary to want to motivate agents to tell the truth? Sometimes it is not. Second, if agents can be motivated to tell the truth, do they have the necessary resources (or information) to tell the truth? If the designer has the control over all agents, and can make sure that all resources and information needed for an agent to tell the truth are available whenever the agent needs to tell the truth, then yes, agents can be motivated to tell the truth.

Q13: In the pool example, who would be taxed by the Clarke tax algorithm assuming the pool was built?

A13: According to the Clarke tax algorithm, anyone whose vote would have made a difference had his/her vote was withdrawn from the tally would be taxed.

Q14: Is bidder collusion possible when all agents do not belong to one coalition?

A14: Yes. Agents do not even have to communicate directly to come up with collusions. But once they collude, they implicitly form, albeit temporarily, a coalition. So, it depends on how you define a coalition formation process. If you include the sizing up each other, the modeling and profiling, and the works leading up to a collusion into the coalition formation process, then the answer to the above question may be no.

Q15: Do all markets eventually reach equilibrium?

A15: In the real world, no. That is why you sometimes have busted economy, inflation that goes through roof, such as what happened in Argentina and several Asian countries in recent years.

Q16: Would you ever design an agent to deliberately send out false bids to get info about another agent’s supply/demand, e.g., spy agent?

A16: If you were building an agent or a group of agents to compete with other agents, yes, you would design such an agent. But if you were building a multiagent system to accomplish a task, then you would not design spy agents.

Q17: Is there any relation to using strategic bargaining theory to find a one-step, unique subgame equilibrium for the combinatorial explosion of items in coalition formation during contingency contracting?

A17: Yes. During contingency contracting, the agents know more than they knew before, and thus, they could move towards an equilibrium. And the strategic bargaining theory is one of the mechanisms that the agents could use.

Q18: Can OCSM-contracts be parallelized to make them more practical for MAS?

A18: Yes, this is practically a search in the solution space. So, it can be parallelized.

Q19: How would a system punish a lying agent to reduce the chance of the agent lying again?

A19: There are several issues here. First, what is the role of the system? Is it the multiagent system, or the policing system, or the monitoring system, or the environment where the lies occurred? If it is the multiagent system, then usually we do not want to have lying agents in the first place. If it is the policing system, then the system can penalize the lying agent with a substantial fine, or announce the lying agent’s name to all other agents and let other agents make their decisions. If it is a monitoring system, then the system can improve its safe-guarding mechanisms to reduce the chance of the agent lying, by subpoenaing various information sources of the lying agent (basically checking every future claim of the lying agent against these sources). If the system is the environment where the lies occurred, then the system cannot punish the lying agent directly in a convenient manner. But, it can still react or produce items/events in the environment that makes things uncomfortable for the lying agent.

However, another fundamental issue is how would a system detect a lie and then identify the lying agent? Think about it.

Q20: “In correlated value auctions the rules are often varied to make the auctioneer increase the price at a constant rate or at a rate he thinks appropriate.” Why at such a rate? Is it related with the auction outcome?

A20: First, what is a correlated value auction? It combines both the private value of an agent’s and other agents’ values of the item. That means what? Suppose you are at an auction. Your private value of the item (on the auction block) is $1000. However, when the bidding starts, other bidders give bids of $1,500, $2,000, $3,000, etc. Now, what does that affect you? Do you think that may be the item is actually worth more? Why? Since you buy the item, you can re-sell it at a price much higher than $1000!

Now, what does the auctioneer think? The auctioneer thinks, “Hmm … what is going on here? Everybody loves this item. I should increase the price at a constant rate or at a rate that is going to give me the maximum payoff!” So, this is how the auction outcome affects the mentality of the auctioneer and the bidders, especially regarding the correlated value of the item.