CSCE 475/875

Handout 20: Cooperation Day Analysis

December 3, 2009

1. Tables of Results

Day 1

Team	Items Retrieved	Critical Item	Sequence	Utility
Aquawit	5, 3, 2, 1, 6, 7, 8, 9, 10 (9)	4 (+$3900)	5 (1)	$100
Triagents	2, 4, 6, 1, 3, 7, 9 (7)	8 (+$200)	2, 4, 6 (3)	$300
Victry	1, 2, 3, 4, 5, 6, 8, 9, 10 (9)	7 (+$3000)	1, 2, 3, 4, 5, 6 (6)	$1000
Laika	10, 9, 7, 5, 4, 3, 2 (7)	8 (+$300)	10, 9 (2)	$200
Matchmakers	1, 3, 7, 9, 2, 4, 6, 10 (8)	5 (+$1800)	1, 3 (2)	$200
Uno MAS	1, 2, 3, 4, 5, 10, 9, 8, 6 (9)	7 (+$2000)	1, 2, 3, 4, 5, 10, 9, 8 (8)	$2000

Table 1: Items retrieved, critical item needed to make a longer sequence, the actual sequence formed, and the actual utility obtained.

From the above table, we see that some critical items could have brought large utility gains to some teams. However, these critical items were not obtained. Based on Day 1, UnoMAS gained the highest utility with $2000 from their sequencing task, with Victry gained the second highest utility with $1000 from their sequencing task. And others gained between $100 and $300 in utility from their sequencing task.

Provider	Consumer	Price	Information
Aquawit	Triagents	$100	Item 4
Triagents	Matchmakers	$200	Item 3
Triagents	UnoMAS	$100	Item 6
Laika	Matchmakers	$300	Item 4
Matchmakers	UnoMAS	$150	Item 2
Matchmakers	UnoMAS	$150	Item 7*
Matchmakers	Aquawit	$100	Item 1
	TOTAL	$850	6 items
	Average	$141.67
* This item is problematic: Matchmakers provided poor location information. Excluded from calculations below.

Table 2: Information providers, consumers, prices, and information for the transactions. See Table 4 below as well.

Team	#Items Sold	#Items Bought	Total
Aquawit	1	1	2
Triagents	2	1	3
Victry	0	0	0
Laika	1	0	1
Matchmakers	2	2	4
Uno MAS	0	2	2
Total	6	6

Table 3: Number of items sold and bought. See Table 5 below as well.

From Table 3, we see that there were teams that were active in buying and selling information (e.g., Matchmakers), teams that were not active at all (e.g., Victry), teams that were focused on buying information (e.g., Uno MAS), and teams that were focused on selling (e.g., Laika).

Day 2

Team	Items Retrieved	Critical Item	Sequence	Utility
Aquawit	5, 4, 3, 2, 1, 6, 7, 8, 9, 10 (10)	None	Entire (10)	$4000
Triagents	2, 4, 6, 8, 10, 1, 3, 7 (8)	5 (+$500)	10, 9 (2)	$1500
Victry	1, 2, 3, 4, 6, 7, 8, 9, 10	5 (+$3500)	1, 2, 3, 4 (4)	$500
Laika	10, 9, 8, 7, 6, 5, 4, 3, 2, 1 (10)	None	Entire (10)	$4000
Matchmakers	1, 3, 5, 7, 9, 2, 4, 6, 8, 10	None	Entire (10)	$4000
Uno MAS	1, 2, 3, 5, 10, 9, 8, 7, 6 (9)	4 (+$3700)	1, 2, 3 (3)	$300

Table 1: Items retrieved, critical item needed to make a longer sequence, the actual sequence formed, and the actual utility obtained.

From the above table, we see that some critical items could have brought large utility gains to some teams. However, these critical items were not obtained. Based on Day 2, Aquaqit, Laika, and Matchmakers gained the largest utility with $4000 from their sequencing task. Triagents gained the fourth place in terms of sequencing task utility with $1500. Victry and Uno MAS rounded out the final two places.

Provider	Consumer	Price	Information
Victry	Aquawit	$300	Item 5
Triagents	Victry	$200	Item 2
Matchmakers	Victry	$150	Item 5
Laika	Matchmakers	$250	Item 3
Laika	Matchmakers	$250	Item 9
Laika	Uno MAS	$150	Item 1
Laika	Triagent	$300	Item 6
Uno MAS	Laika	$150	Item 7
	TOTAL	$1750	8 items
	Average	$218.75

Table 4: Information providers, consumers, prices, and information for the transactions. See Table 2 above as well.

Team	#Items Sold	#Items Bought	Total
Aquawit	0	1	1
Triagents	1	1	2
Victry	1	2	3
Laika	4	1	5
Matchmakers	1	2	3
Uno MAS	1	1	2
Total	8	8

Table 5: Numbers of items sold and bought. See Table 3 above as well.

From the above table, most teams were more active than on Day 1. Laika focused on selling information after they obtained all their items. Aquawit was so focused on finding their own items, they did not sell any information.

		Aquawit	Triagents	Victry	Laika	Matchmakers	Uno MAS	Average
Day1	Start	$1000	$1000	$1000	$1000	$1000	$1000	$1000
	Sale	$100	$300	$0	$300	$250	$0	$158
	Purchase	-$100	-$100	-$0	-$0	-$500	-$250	-$158
	Utility	$100	$300	$1000	$200	$200	$2000	$633
	SubTotal	$1100	$1500	$2000	$1500	$950	$2750	$1633
Day 2	Start	$1000	$1000	$1000	$1000	$1000	$1000
	Sale	$0	$200	$300	$950	$150	$150	$292
	Purchase	-$300	-$300	-$350	-$150	-$500	-$150	-$292
	Utility	$4000	$1500	$500	$4000	$4000	$300	$2383
	SubTotal	$4700	$2400	$1550	$5800	$4650	$1300	$3400
TOTAL		$5800	$3900	$3550	$7300	$5600	$4050

Table 6: Subtotals and totals in terms of utility ($) for each team for the game day.

From Table 6, we see that Laika had the most utility ($7300). Thus, Laika is the winner of Game Day 3. They distanced themselves from the other teams on Day 2 by net-gaining $800 in their transactions. The second place goes to Aquawit ($5800), edging out Matchmakers ($5600). Uno MAS finished 4^th with $4050. Triagents (with $3900) and Victry (with $3550) were 5^th and 6^th, respectively.

2. General Observations

Here are some general observations:

1. Similar Strategies: Most teams used the same underlying strategies: utility-maximizing. One team (Matchmakers) were more tactical (reactive) than strategic. One team (Uno MAS) was overly strategic and not opportunistic enough.

2. Day 1 vs. Day 2:

· More transactions took place on Day 2 than on Day 1 (8 items vs. 6 items). This could be due to the shorter search-and-retrieve time allotted for Day 2.

· The average price for each item sold or bought on Day 2 was significantly higher than that on Day 1 ($218.75 vs. $141.67).

· More teams sold at least an item on Day 2 than on Day 1 (5 vs. 4).

· More teams bought at least an item on Day 2 than on Day 1 (6 vs. 4).

· No team solved the search-and-retrieve task on Day 1. Three teams solved the task on Day 2.

· Overall, Day 2 was much less hectic as Day 1. On Day 1, the teams were less willing to purchase information as each thought they would be able to find all the items they needed. On Day 2, the teams were more willing to purchase information and also willing to purchase information at a much higher price.

· Further, on Day 2, there were more “cooperation”—as reported at the end of the game day—as the teams realized that they could gain significantly much more utility from finding their sequence of items than selling information to others. That actually caused the multiagent system to cooperate. And that was the objective of this design!! The Game Day was designed to motivate the agents to cooperate. On Day 1, the agents did not cooperate as much since each believed that it could solve its tasks within the time constraint and resource constraint. I was actually puzzled that there was so much more postings for selling information than postings for buying information. I intentionally designed the system such that obtaining long sequences of items was very profitable; however, this was not fully exploited on Day 1. On Day 2, the agents, having learned from Day 1, were “motivated” to cooperate out of necessity, and that was based on the utility gains and the constraints.

· Further, on Day 2, less inter-thread communication was incurred but more blackboard communication (in terms of postings) was incurred. This is a very good transfer of computational resources. On Day 1, each agent spent too much time communicate between threads, relaying and recording information that might or might not be useful, and that actually caused loss of information and ineffective blackboard postings. On Day 2, more teams had more time posting and monitoring the blackboard.

3. Team-Specific Observations

· Aquawit: This team did a decent job of tracking and recording their activities. On Day 1, they did not complete recording one transaction. Their pre-game and mid-game strategies were concise. However, their Day 1 strategies did not consider basing their price offers on other offers. Their strategies were mainly more game-playing and utility-maximizing than individual rational as they would hold out on buying information even though they could gain from it. Their mid-game strategies were more targeted and focused compared to the pre-game ones. They observed that holding negotiations later would be probably better than holding them early. Holding off until later also allowed them to find their own items instead of paying for them. However, they searched the basement on both days even though I had specifically announced that the items were only going to be on 1^st, 2^nd, and 3^rd floors. On Day 1, they were unlucky in terms of not finding the second item in their sequence. On Day 2, they were focused on getting their own items and neglecting the potential gain of selling information about other team’s items. In short, this team was quite individualistic on Day 2.

· Triagents: This team did a fairly good job of tracking and recording their activities. Their pre-game and mid-game strategies were good. They planned to observe others’ offers to adjust their offers. They also split the game day into two halves for different behaviors—in the second half, when time is pressing, they planned to be more conceding. They realized that they focused too much on finding their own items on Day 1 and did not put up enough offers. They also pointed out that within such a dynamic environment, they did not have enough time to think or change their plans. This is a key observation as reactive agents usually lose out on strategic agents when strategies could be useful in such a dynamic—fast-paced—environment. They realized that they did not do enough negotiations on Day 2 as they could have probably obtained all of their items if they had. Their strategies were just a bit more utility-maximizing than individual rational. If they had been more individual rational driven, they probably would have started negotiations earlier and more often.

· Victry: This team did not do a good job tracking—they even had wrong information on their contract in terms of which team was the consumer/provider. They did not compute the finally utility for each day. This team had brief pre-game and mid-game strategies, mainly utility-maximizing and individual rational. Their lessons learned include “do not negotiate with the team coming from the opposite direction as it is possible that one could find the items on its own.” This team’s strategies were not as focused in relative to some other teams’.

· Laika: This team did a rather good job of tracking and recording their activities. Their pre-game strategies were quite good, but did not consider other teams’ prices when setting their own prices. Their mid-game strategies were okay—flexible and contingent upon items’ “find-rate”. However, they also included old strategies from pre-game that did not make sense. Overall, this team was game-playing, utility-maximizing more than individual rational. They were able to obtain all their items on Day 2 and subsequently focused on selling information. And they were able to sell $950 worth of information.

· Matchmakers: This team did a poor job of tracking and recording their activities. They recorded transactions that did not take place. Further, on Day 1, they sold incorrect information to UnoMAS. Their pregame strategies did not give any impression of whether they would be utility maximizing, game-playing, or individual rational. They were more tactical than strategic. Their pricing scheme did not consider other teams’ prices. For their mid-game strategies, they decided to start their asking price at a higher value. Also, similar several other teams, they also tried to observe where other teams had been traversing and follow them. They also decided to observe other teams’ negotiations and followed the team going to their location if a transaction was made. In other words, this team strategy was more reactive and tactical than strategic.

· Uno MAS: This team did a poor job of tracking and recording their activities. Their pre-game and mid-game strategies were utility-maximizing and game-playing. Their pre-game strategies were very comprehensive and did extensive modeling of other teams. Impressive. However, on Day 1, they purchased inaccurate information from Matchmakers on one particular item, causing them to spend significant time searching for that item to no avail. Still, they were the big winner on Day 1. However, for Day 2, even though they decided to be more reactive—based on a good-enough, soon-enough strategy, their strategies were not as opportunistic as others. They did not react fast enough in changing their tactics on Day 2. They provided a very insightful lesson learned. That is, one had to balance negotiations and explorations. They observed that most teams spent proportionally more time in negotiations as time progressed. They also noted that they had to balance note-taking and finding their own items. These insights point to an aspect that is very true in most agent reasoning in such an environment: how to tradeoff between two issues.

4. Lessons Learned

· On Day 1, several teams made poor decisions: emphasizing selling information too much as opposed to focusing on purchasing information. From the viewpoint of utility-maximizing, that means they failed. However, looking at the pre-game strategies, all teams planned to utility-maximizing. So, what went wrong?

o Most teams did not consider the resource constraints. When an agent posts on the blackboard, it is obligated to entertain responses to its postings. That is a resource-draining activity.

o Most teams also did not consider the time constraints. They thought that they would be able to find (if not all) most of the items on their own.

This shows that it is important to consider both resource and time constraints when designing a MAS, especially in such a dynamic, uncertain, time-constrained environment.

· Inter-thread communication plays a role. Too much information relayed between threads cause the threads to slow down—not able to conduct other tasks.

· Accurate sensing is important. It reduces uncertainty about the environment, and in turn allowing more confident decision making. When designing a MAS, one must consider how the agents sense their environments and how certain the sensing results can be. This allows the MAS designer to decide how to shape the agents’ reasoning process accordingly. Some teams did not search carefully resulting in repeated visits to the same locations.

· How to design a cooperative MAS? Is the MAS in Game Day 3 a cooperative MAS? Fundamentally, it is a competitive MAS since each agent competes with the other when making their local decisions. However, the emergent, coherent behavior (on Day 2) was actually cooperative. Without cooperation, most teams would not be able to find all items for the time allocated. Because of information exchange, all teams solved their search-and-retrieval task. The “cooperative spirit” was motivated through utility: each self-interested agent was willing to “cooperate” as long as it gained from the “cooperation”. This is quite commonly done in MASs. By designing agents this way, one still gives them their autonomy, but “implicitly” leads the system to achieve coherence.

· When and how to post an information offer or an information need is important. On both days, I see weaknesses in the postings. On Day 1, there were too few “NEED” postings. And thus, most of the pricings on the “OFFER” postings were “unguided” (no “ceiling” on how much one was willing to pay for a piece of information). As a result, it was a bit of guesswork. What are “ineffective messages”? These are postings with pricings outside of a common zone between a seller and a buyer. Key is to motivate the agents to post what they need and what they can offer in a timely fashion.

5. Game Days League

Teams	Auction Day	Contract Day	Cooperation Day	Total
Laika	1	2	1	4
Matchmakers	2	4	3	9
Uno MAS	5	1	4	10
Victry	2	3	6	11
AquaWit	4	6	2	12
Triagents	6	5	5	16

The winner of the league is Laika with 4 points. Matchmakers place 2^nd with 9 points, while Uno MAS is 3^{rd with 10 points}. Victry and AquaWit scores 11 and 12 respectively for 4^th and 5^th. Triagents finish sixth with 16 points.

Figure 1. Ranking of each team for the three game days. Laika was consistently near the top. Aquawit made a good comeback. Victry and Uno MAS dropped their performances.