CSCE475/875 Multiagent Systems

Handout 16: Game Day 3Auction Day Analysis

November 11, 2011

Auction Rounds

Table 1 shows some of the basic information for each of the 15 rounds of auction. SIB won three times. Reagent, Split Second, and Triplet Threat won twice. And the rest (6 teams) won once each.

Rd.	Auction	Winner (#)	Winning Bid	Payment
1	English	Power Agents (1)	$300	$300
2	Japanese	Split Second (1)	$310	$310
3	Dutch	ULM (1)	$310	$310
4	Sealed, First Price	Wolfpack (1)	$300	$300
5	Vickrey	Free Agents (1)	$305	$301
6	English	Reagent (1)	$300	$300
7	Japanese	SIB (1)	$240	$240
8	Dutch	DJ Carpet (1)	$310	$310
9	Sealed, First Price	SIB (2)	$301	$301
10	Vickrey	Triple Threat (1)	$309	$300
11	English	SIB (3)	$299	$299
12	Japanese	Split Second (2)	$301	$301
13	Dutch	Reagent (2)	$310	$310
14	Sealed, First Price	JRL (1)	$405	$405
15	Vickrey	Triple Threat (2)	$301	$250
			TOTAL	$4,537

The auctioneer’s market value for each item is the same, $50. As shown later in Table 3, there were 81 “ins” and with a fee $2 for each participation, the auctioneer received $162 from the participation fees. Together, the auctioneer made a gain of $4,537 + $162 – 15x$50 = $3,949.

Team Statistics

Table 2 shows the private valuations of each team for 15 rounds of auctioned items. All teams followed the rules correctly. Most teams (6) assigned $300 to the remaining 5 items outside the 10 required values. I observe no trends in the mean values from round to round. I also observe no patterns per type of auction protocols. It seems that how the valuations were assigned is random.

Now, I look for “opportunities” for teams to make large gains. That is, if there was only one team whose valuation for an item was maximum, then that would be an opportunity for that team to make a gain, especially if the closest valuation was much smaller. For Round 6 (English), Free Agents valued the item at $300. The next closest valuation was $230 by JRL and Reagent. As shown in Table 3, Free Agents participated in this round but did not win the round. Why? From Table 1, we see that Reagent won this round, bidding $300 for an item that they valued only as much as $230. Then there were also weaker opportunities where there were only two teams having the same maximum values, as in Rounds 4, 5, 7, 11, 14, and 15. I designated these opportunities as “weaker” as they depended on whether one of the two teams participated in the round. If one of them did not and the other did, then it would become a real opportunity. From Table 3, we see that in all those rounds, both the maximum valuation teams participated. But in Round 15, one of the maximum valuation team (i.e., Triple Threat, see Table 1) netted a big gain. Why? SIB, one of the maximum valuation team, under-valued and put in a bid of $150, much lower than their valuation of the item ($300). (Incidentally, Wolfpack over-valued and put in a bid of $250, $40 higher than their valuation of the item ($210).) Had SIB put in a more rational bid (e.g., $298), they would have prevented any other teams from making such a big gain in Round 15. In a way, Round 15 cost the auctioneer money too—it could have received around $300 instead of $250 for the T-rex!

Further, Table 2 also showed that JRL’s valuations were strategically different from the rest. They did not value a single item at $300. Their total of valuations was $2,300, the lowest. Essentially, they set up their valuations for them to be not competitive. Likewise, teams such as Wolfpack, Reagent, and SIB also put themselves at a slight disadvantage with lower valuations. Thus, it was no wonder that some of these teams finished at the bottom half (see Table 4).

Rd.	Wolf-pack	DJ Carpet	ULM	Free Agents	Power Agent	JRL	Re-agent	SIB	Split Second	Triple Threat	Mean
1	$210	$300	$210	$70	$300	$110	$210	$30	$110	$300	$1,850
2	$190	$300	$150	$150	$70	$150	$190	$190	$300	$300	$1,990
3	$170	$300	$300	$130	$300	$130	$300	$230	$170	$50	$2,080
4*	$300	$50	$300	$190	$210	$210	$110	$170	$130	$210	$1,880
5*	$230	$70	$50	$300	$190	$150	$90	$110	$190	$300	$1,680
6**	$150	$90	$190	$300	$170	$230	$230	$150	$210	$130	$1,850
7*	$130	$110	$130	$230	$150	$160	$130	$210	$230	$110	$1,590
8	$110	$300	$300	$110	$130	$190	$300	$130	$300	$70	$1,940
9	$300	$130	$300	$300	$110	$180	$70	$300	$150	$190	$2,030
10	$230	$150	$70	$300	$90	$170	$300	$300	$300	$300	$2,210
11*	$90	$170	$170	$170	$230	$90	$170	$300	$300	$150	$1,840
12	$70	$190	$110	$300	$300	$70	$150	$90	$300	$170	$1,750
13	$50	$300	$300	$90	$300	$150	$300	$70	$90	$90	$1,740
14*	$300	$230	$230	$210	$300	$260	$50	$50	$70	$230	$1,930
15*	$210	$210	$90	$50	$50	$50	$30	$300	$50	$300	$1,340
TOT	$2,740	$2,900	$2,900	$2,900	$2,900	$2,300	$2,630	$2,630	$2,900	$2,900	$27,700

Table 2.Private valuations of each team for 15 rounds of auctioned items. ** indicates a round where there was only one maximum valuation. * indicates a round here there were only two maximum valuations.

Table 3 showed the auction rounds that each team participated. ULM participated in ALL rounds (15) while Power Agent participated in only 3 rounds. Power Agent won the first round, and strategized to stay out of most rounds to save the $2 fees. Further, the table shows that not all teams participated in the rounds where they knew they at least shared the maximum valuation of the auctioned item. There were also teams that participated in rounds that they knew they were unlikely to win the auction because of their low valuation of the item being auctioned. That means, these teams were willing to lose $2 for each such round for nothing.

Team

TOT

Wolfpack

√

DJ Carpet

√

ULM

√

Free Agents

√

Power Agent

√

JRL

√

Reagent

√

SIB

√

Split Second

√

Triple Threat

√

TOT

Table 3.The number of teams “in” (√) for each round of auctions. Bold √ indicates a team’s participation when the team knew going in that it had the maximum valuation (i.e., $300), while a shaded entry indicates a team’s knowledge that it had the maximum valuation (i.e., $300).

Table 4 shows the final utility tally for each team. Per this table, Triple Threat won the Auction Day with $1,656. Indeed, Triple Threat was the only team that made a net positive gain (i.e., $1,656 - $1,622 = $34). JRL fell below $1,500. ULM would have finished higher if they had been more selective in terms of participation in which rounds. (Remember that they participated in all rounds indiscriminately and that cost them a total of $30.) Power Agent, on the other hand, participated in very few rounds and was able to finish high in the ranking. But in general, because of their “risk-averse” behaviors, they were unlikely to make high reward gains.

Rd.	Original Money	Utility Gained from Items Won (#)	Payment	Participation Fee	TOTAL
Triple Threat	$1,622	$600 (2)	$550.00	16	$1,656.00
Power Agent		$300 (1)	$300.00	6	$1,616.00
DJ Carpet		$300 (1)	$310.00	10	$1,602.00
Wolfpack		$300 (1)	$300.00	20	$1,602.00
Free Agents		$300 (1)	$301.00	20	$1,601.00
Split Second		$600 (2)	$611.00	16	$1,595.00
ULM		$300 (1)	$310.00	30	$1,582.00
SIB		$810 (3)	$840.00	14	$1,578.00
Reagent		$530 (2)	$610.00	12	$1,530.00
JRL		$260 (1)	$405.00	18	$1,459.00

Table 4.Total utility for each team. Total Utility = Original Money + Utility Gained from Items Won – Payment – Participation Fee.

Individual Team Analysis

Table5 documents my comments on each team’s worksheet and reports.

Team Name	Pre-Game	Tracking	Mid-Game/Post-Game
Free Agents	Quite well though out; Putting lower valuations in the first few rounds; putting low values in Dutch rounds; distributing $300-bids between auction types; Assigning $300 to the 5 remaining rounds; Did not discuss however when to opt in.	OK.	Quite insightful post game analyses; some mid-game observations. Pointed out that some good decisions during the rounds –giving up on an item to not end up losing money, that teams who appeared to do well are teams who valued an item high in the rounds where no other team valued an item high (not always true though). My observation: the team acted rationally and consistently overall.
Split Second	Thoughtful pregame; considering not only how to bid, but also how to bid in response to models of others; also provided time-based strategies, round-per-round, and type of auction protocols.	Didn’t calculate the final total utility.	Mid-game observations were good. Pointed out that JRL should have waited until Vickrey auction to use such a big bid, in order to avoid a large loss. Very true. However, if other teams also knew that JRL still didn’t win an item after Round 14, and they knew that JRL would have to make sure that they won Round 15, then some teams might be willing to on purpose increase the second price by bidding really high, as long as the bid was lower than JRL would bid. And JRL would still end up with either a big penalty (-$1400+) or a big payment. * Did not tally the total final utility. Not good. Noticed the significance of the $2 entry fee; Good lessons learned, insightful; provided reason for while Vickrey auction did not yield truth-telling behavior; pointed out the smart strategy by Triple Threat in Round 15
Power Agent	Quite comprehensive; strategized on how to come up with an ideal valuation set for the rounds, analyses quite in-depth; but assumptions that other teams would act rationally at all times, and also that other teams would not also practice the same tactics were not that valid.	OK.	Mid-game and post-game observations and analyses were quite insightful. This team perhaps was the most rational team in terms of deciding whether to bid. However, they didn’t win the Game Day. One factor was the chance factor. But also since the team was very conservative, and “nothing ventured, nothing gained”, it was unlikely for them to make a move or to “hurt” other teams. Triple Threat took a chance and scored big in Round 15.
DJ Carpet	Did a nice background analysis that determines that “buyers have enough cash!!! So it is almost impossible that items are sold in low price due to no money in teams … so we can expect every item would be sold at nearly or more than $300”—indeed a very accurate prediction!	OK.	Mid-game observations okay. Post-game lessons learned made two good suggestions. (See lessons learned below.)
ULM	Pre-game strategy took a “somewhat contrarian approach”. For example, believing that “other teams will think these [Vickrey] auctions are the best opportunity to gain utility … don’t gain much utility due to the bids being roughly the same (and high) … we are using our lowest values …”	OK.	Mid-game observations okay. Post-game lessons learned had some good insights. (See lessons learned below.)
Reagent	Pre-game strategies didn’t explain why the team didn’t assign $300 to the remaining 5 items; not quite well though out	Didn’t track the winners of rounds except their own.	Mid-game and post-game observations were minimal.
Triple Threat	Pre-game strategies were quite strategic: “Dutch is too risky to gamble upon … we have placed our lowest valuations on them. … using the Vickrey as our safety, with the maximum allowed valuations on them”; switching risk attitudes based on whether an item has been won; but they didn’t talk about driving the price up for other teams	OK.	Mid-game observations were quite well done. Showed that the team constantly observed and adapted their strategies. A very “flexible” agent. Post-game lessons learned were quite insightful. (See lessons learned below.)
Wolfpack	OK. They didn’t assign $300 to the remaining 5 items. Their rationale did not match their valuations.	OK.	Mid-game observations good. Post-game lessons learned was not definitive and insightful enough, lack of derived information from their modeling of the rounds.
SIB	Pre-game too vague; didn’t offer any discussions on how they came up with their valuations	OK.	Mid-game observations ok. Post-game lessons learned were minimal. This team won three items and lost quite a bit of utility as a result due to overbidding; no explanations were provided
JRL	Lack of consistency in the pre-game strategy, and also did not provide reasons for the low valuations.	Final total utility miscalculated by $2.	Mid-game observations were okay. Post-game observations were minimal.

Table 5. My comments and observations of team strategies, worksheets, and reports.

Lessons Learned

Here are some overall lessons learned.

1. Teams should rationally assign $300 for the five remaining items. There were four teams that did not do so. Immediately, that put them at a disadvantage. SIB, Reagent, JRL, and Wolfpack were the four teams, and they finished #7, #8, #9, and #10, respectively.

2. Rationally, a team, without knowing or speculating on what other teams might do, should value the Vickrey round items at $300. Why? This would allow a team a chance to be aggressive and put a higher bid with the hope that it would win the item at a second price that is lower than $300.

3. Participating in every round indiscriminately was unwise as the loss of $2 as fee for each round lowered the utility of a team. Team ULM was the only team that participated in all 15 rounds while the most that other teams participated was 10 rounds. All-pay auctions are all-pay for a reason: to make sure that the only teams that have a chance to win an auction are the ones participating. Thus, if a team values an item lowly, then that means the team will have close to 0 chance in winning the item. Then the team should not participate.

4. On the other hand, being conservative and only participating a few rounds might not afford enough opportunities for a team to get high rewards. Power Agent was such a team, playing very rationally. But they lost out to Triple Threat that was willing to take risk.

5. Did the teams with more “opportunities” fare better than teams with fewer “opportunities”?

Table 6 below shows something interesting. I computed the differences between the valuation of each team and the valuation of every other team for each round. I then counted the times a difference is higher, the same, and lower. And then I averaged these numbers across all rounds. Take Wolfpack for example. It has an average #higher evaluation per round of 3.667, an average #same valuation per round of 1.133, and average #lower valuation per round of 4.2. That means, more likely than not, if all agents participated in a round, Wolfpack’s valuation would put the team at a disadvantage. Further in the table, I also included the average difference in valuations per round. JRL, for example, had an average difference in valuations of -$284 per round. As six teams had the same $2,900 total valuations (see Table 2), they all shared the same average difference of $76. Which of these factors would a better predictor as to how the teams would fare? I computed the correlations of each of the four factors with the Final Total Utility: 0.80, 0.26, -0.61, and 0.89. What do these mean? It means that the Average Difference in Valuations per round and Final Total Utility are highly positively correlated (i.e., 0.89). It also means that the Average #Higher Valuation per round and Final Total Utility are also quite highly positively correlated (i.e., 0.80). It implies that the actual difference (in $) is a more accurate predictor. Further, it implies that how a team dominated others is a better predictor than how a team is dominated by others: 0.80 vs. -0.61.

But because the correlation values are not 1.0 or -1.0, that means it was still possible for teams to do better even though their starting point (valuations) was not as good. This is due to two reasons: (1) the actions of other teams, and (2) the actions of the team itself, such as the number of rounds they participated, and the rational or irrational behaviors. Look at Wolfpack, their starting position was #7, but finished tied at #2.

Team	Average #Higher Valuation per round	Average #Same Valuation per round	Average #Lower Valuation per round	Average Difference in Valuations per round	Final Total Utility
Triple Threat	3.933	1.133	3.933	76	1656
Power Agent	3.867	1.2	3.4	76	1616
Wolfpack	3.667	1.133	4.2	-20	1602
DJ Carpet	3.4	1.333	3.933	76	1602
Free Agents	3.533	1.533	3.467	76	1601
Split Second	4.067	1.4	3.133	76	1595
ULM	3.4	1.6	3.467	76	1582
SIB	3.4	0.933	4.267	-86	1578
Reagent	2.933	1.6	4.067	-86	1530
JRL	3	0.8	4.8	-284	1459

Table 6. Comparing each team’s valuations to other teams’ for each round. A large average #higher valuation per round means a team’s valuation for that round dominates quite a few other teams’ and so on.

6. From team DJ Carpet:

We have 5 opportunities to set the IPV with price from $0 to $300. …I would suggest you do a little change. For example, teams still have 5 opportunities to set the IPV, but sum IPV of these 5 items must be less than $2,000.

Setting a higher starting price for Dutch auction.

I noticed the second point during the Auction Day as well. But after having made the poor choice in the first Dutch, in order to be fair to other teams in the second and third Dutch, I stuck with $310 as the starting price for Dutch. I will use these in my future Auction Day.

7. From team Triple Threat:

English bidding was the most active. It was very fun to watch. It was nice competitively because you knew who you were bidding against. The biggest advantage seemed to be to jump to your evaluation first. But most auctions had two or more bidders with an evaluation $300. In these, no one went above $300.

Japanese seems like a game of chicken and who flinches first. This was the only auction that needed arbitration. It was relatively easy to drive up the price to above your private value.

Sealed first bid was not ideal for the situation because there was always a $300 bid so you had to lose utility in order to win. There was no way to “test the water” or anything like that. The only advantage is the minimization of communication needed. We tried to use this for an opportunity to get better utility, but failed.

Dutch should have started higher. The auctioneer could have made (much) more money if they had started higher. Our decision to stay out of these worked out well for us.

8. From team ULM:

Our environment was not terribly competitive since teams only had to win one item and then “play it safe” if they desired.

Because of this, the variation in final utility across teams will be very low.

It appears that overall, teams went on to meet the constraint of winning one item and gain some utility. No team realized the potential cash power that they had at their disposal. For example, in our case, if we would have realized this, we would have played the first 2 rounds in the same way just to make a picture of the winning prices. Then, on the next 5 to 6 rounds, we could have overbid around 320-330. This way, the rest of 7 teams (except us and the teams winning the first 2 rounds) would have been more desperate to win in the last half and overbid more, thus ending up with less money.

9. Due to the large potential penalty of not winning an item, the teams practiced different types of risk attitudes. When a team was still in need of an item, it was likely to be risk averse – afraid of not winning and thus offering a higher bid; and when a team had won an item, it was likely to be risk seeking – not offering a higher bid.

10. Having said the above, there were several teams, markedly Triple Threat, who was persistent and determined to make sure that other teams lost utility if they won an item. But Triple Threat did this cautiously, respective to their valuations. On the other hand, ULM, who also tried to bid to force the price up, did it a bit too indiscriminately. Thus, it is important to balance one’s approach especially when tradeoffs of opposite factors are present in the environment.

11. From viewpoint of delivering the auctions, Dutch, and the sealed-bids were fast. English was next. Japanese last. In terms of gain, Dutch and the sealed-bids were better for the auctioneer in our environment due to the additional requirements—causing the private values to be not entirely independent—and some less-than-rational bids. English and Japanese both forced teams to reveal their values and thus teams were able to model the others.

12. Software automation made this Game Day smoother than Game Day 2. For future semesters, we will have software for all Game Days.

Game Day League

Here is the final League Standings. Free Agents won the League with a total of 7 points. DJ Carpet finished as a strong second team. Power Agent and Triple Threat both finished at the third position with 11 points, with Split Second closely behind with 13 points. ULM placed sixth. Wolfpack, Reagent, SIB, and JRL finished as the last four teams. Triple Threat improved their position steadily from 7 to 3 to 1.

Team Name	Learning Day	Voting Day	Auction Day	League Standings
Free Agents	1	1	5	7
DJ Carpet	3	2	3	9
Power Agent	2	7	2	11
Triple Threat	7	3	1	11
Split Second	3	4	6	13
ULM	5	5	7	17
Wolfpack	7	10	3	20
Reagent	6	9	9	24
SIB	9	8	8	25
JRL	10	6	10	26

Free Agents will sign the Ball of Fame later this semester in class for winning the Game Days League.