The Problem With User Rankings

Not too long ago, on the Conquer Club website, they implemented a new system for player ratings.  Previously it had been based on an eBay style positive/neutral/negative scale with a comment.  You simply voted weather your experience playing them was positive, negative, or neither and then said something like “played all his turns in a timely manner” or “excellent team mate, worked well together” or “stupid dumbass only attacked his own team”.  Anyway, they switch over to a 4 stat 5 star rating system.  If you don’t care to follow the link, the short version is that after you play someone you can leave them a rating which is a rating from 0 (no rating given) to 5 (excellent) stars in 4 categories: Fair Play, Attendance, Attitude, Teamwork.  And the rating isn’t revealed until the game goes into the archive (no more ratings can be made on that game).

Going back to the eBay style ratings system, I hated it on eBay, largely because it was heavily retaliatory.  I once bought my wife a cello on eBay.  I didn’t buy the best model out there, I just wanted to get her a practice cello so she could start playing again (seriously, I’m not going to spend five grand on anything for anyone if there is a chance its just going to sit in the corner collecting dust).  The one I bought arrived and the bridge was not set up, which I expected.  What I did not expect was to wind up driving around to about two dozen music stores in an attempt to get someone to set up the bridge and have none of them willing to work on it because it was an “off brand” they didn’t sell or support.  I ended up setting it up myself, and did a very poor job of it, and after I gave it to my wife we finally found a music shop where someone relented and set up the bridge and tuned it.  So, I went to eBay and left a review of my purchase experience.  I was limited by the number of characters, but I said, “sale and shipment fine, but cello was ‘off brand’ and most music stores would not set up bridge or tune it.” and I gave him a neutral review.  To which, he replied by giving me a negative review that said “clueless user, DO NOT SELL TO HIM AND DO NOT BUY FROM HIM, liar and cheat!”  Now, because this was the only review I had, I ended up over the next 6 months unable to buy anything.  All my bids were refused.  The entire time, I was talking to the cello guy trying to get him to change his review.  I had been totally honest and had not given him a negative review, I just felt that anyone who purchased his items might want to know about difficulty getting service on those items.  Finally he relented and changed the review to neutral saying, “inexperienced eBay user, expects to get more than he pays for”.  After having a few more bids of my dropped, I just stopped using eBay.

As this relates to Conquer Club… well, I played a game with some people and I rated most of them 3 out 5 in most categories, except attendance in which case missing zero turns got people a 5, the occasional missed turn got a 4 or 3, and every player who dead beat (missed three turns in a row and was kicked from the game) got a 1.  Once the games got locked down and ratings were revealed, I got a flood of private messages from people complaining about getting a 3.  But looking at the rating scale:

0 means No Rating.  Or in other words, I don’t feel like I can judge you on this.

1 means Bad.  A rating of 1 means you sucked at whatever it was.

2 means Below Average.  This means you performed worse than I would expect.

3 means Average.  Meaning you performed as expected.

4 means Above Average.  You exceeded my expectations.

5 means Excellent.  You are awesome.

In most cases, people are Average.  To get Above Average in Attitude, for example, you just need to be gracious when you lose… or win.  To get Excellent, you need to also chat and be a good natured guy.  If you are silent the entire game, I cannot give you anything other than Average because there is nothing to base it on.  The ratings are from Bad to Excellent, not from I Want To Kill Them to Didn’t Piss Me Off.  There is a phrase where they call something “going above and beyond” and this is what they mean, you have to go Above and Beyond the Average to get better than an Average rating.

The problem is, people don’t see it that way.  3 out of 5 is a 50%, its halfway, and 50% is failing.  Most user based rating systems end up being all but useless for the same reason.  People being rated expect to be given the highest score possible unless there is a problem, and even then most of them want to still get the highest score possible after explaining or fixing the problem through other channels.  People rating others get pulled into the retaliation loop where they are giving higher ratings than they should because they don’t want to be rated poorly themselves.  And then most importantly, the site/game/whatever that is using the rating system, if they allow comments, usually have the length limited to the point where meaningful comments are not allowed.

I don’t know if there is any solution… you want to have the ratings to assist you in avoiding problem users, but the ratings can be, and are being, gamed so you don’t even know if a problem user is really a problem user.

Movie Reviewing Goodness

You might notice that there are more movie reviews lately (and more coming) and less game design talk. There are a few reasons for that, and I thought I’d take a moment to address them…

One. Its that time of year. Christmas, for me, is always a big movie watching season. Blame Hollywood. They stack the deck, both for revenue and for Oscars, putting lots of good stuff on the screen all at once. I hate them for it. But, I still go, because as much as I hate the price of going (more on that another day), the big screen experience is still something I really enjoy. A good crowd (as opposed to a message texting, cell phone yapping, movie explaining, annoying crowd) does enhance a film. To me anyway. So yeah, I’ll be going to the movies once or twice a week for the next few weeks.

Two. Its that time of year. This sounds like a repeat, but I need to stress that Hollywood wants your money. All the good DVDs all come out at Christmas too, just in time for gift giving. So many will be bought, or gotten as gifts, and with TV shows cycling down for the winter break (even moreso with the writer’s strike this year) I’ve got time.

Three. I’m not gaming a whole lot. I’m playing Rock Band, Dead Rising, the occasional game from XBox Live (I’m addicted to Puzzle Quest)… I play Urban Dead on the web, Conquer Club, I occasionally drop in to Guild Wars (got it for $9.99 on Black Friday), and spend a little time in a few betas that I am in, some I can’t talk about and some I can. I’ve just dropped out of all the MMO gaming I used to do. It got too expensive. Although, I am considering the Sony Station Pass thingy since I’d get access to a whole bunch of games, some of which might actually be worth playing.

Four. Since I’m not gaming a whole lot, my thoughts haven’t been focused around game design. Yes, I’m still working on my own wonderful game, and its actually beginning to see a foundation, but its a long way from daylight yet and I don’t want to talk about it anymore until I have at least something I can say is complete and functions.

Five… hmm… can’t think of one. Cool.

So, four reasons. Anyway, until things change, you’ll probably be seeing alot of movie and/or book reviews. I do want to get back to more varied content, and I will. Just bear with me.

Urban Dead Greasemonkey

I have been messing around in the world of Greasemonkey lately. If you don’t know what Greasemonkey is, it is an add-on for the FireFox web browser that allows javascripts to run after a page loads. That may not sound special, but you can do some very interesting stuff with it. For example, the main reason I’ve been dabbling is that I use a Greasemonkey script that someone else wrote for Conquer Club that does some map analysis that the creator of the website doesn’t do, like keep track of card set redemption values in escalating games, hover over attack paths on the maps, and more. Nothing game breaking, nothing you couldn’t do by hand yourself, but very nice in that you don’t have to do it by hand. Well, recently an upgrade to the Conquer Club website broke the Greasemonkey script, so I’ve been looking in to fixing it.

But this isn’t about Conquer Club, as I haven’t finished that script yet. This is about Urban Dead.

One of the fastest ways to get experience points in Urban Dead is to use first aid kits to heal people. And the best way to do that is either to start as a doctor, or make sure the first skill you buy is Diagnosis (maybe second, Freerunning is very important). Diagnosis allows you to see the health of each player in the same block as you. Without it, you just have to randomly try to heal people, wasting action points as the game tells you that they are full of health (you don’t lose the first aid kit though, which is nice). Once you have Diagnosis, the next stumbling block is simply seeing who needs healing. In some places with five or ten people around, its easy, but if you go into a mall where you are likely to find in excess of one hundred people per block, it becomes a giant pain in the ass.

To that end, I have begun working on the ProbablyNot’s Urban Dead Goggles script. If you are an Urban Dead player and have Greasemonkey, click this link to install it. Right now, all the script does is change the text color of the hit point count for anyone with 50 or 60 hit points, which likely means they are full of health (people at 50 might have the Body Building skill and be able to go up to 60, but that takes effort or wasting action points to find out), it make the hit points appear black. If the person is not full (if they have anything but 50 or 60) it will leave it the normal white color. So with the script running, all you need to do look for the people with white hit points and heal them.

Very basic, but also, in my opinion, quite helpful. If I think of more things to add, I will.

Playing Well With Others

Many moons ago, some friends and I all agreed to pool our money and buy network cards for our PCs. Coaxial token ring cards to do an ad-hoc IPX network. All for the purpose of playing DOOM. We would actually load up our computers in the trunks of our cars and drive over to one person’s house, throw together our network, and spend all night playing, laughing and eating pizza.

It was during these gaming sessions that I was introduced to a little game called Warcraft: Orcs & Humans. We all bought copies and our game nights shifted from DOOM to splitting time between DOOM and Warcraft. It was great, and like the FPS game, the two factions of Warcraft were pretty much equal in design, all that mattered was random luck and a little player skill. When to hold back, when to charge. Warcraft II continued that trend, mostly, and we played the heck out of it.

Then came Starcraft. But it was different than the other games in that the sides were not equal. Supposedly there was balance, but since the units were different, they developed different strategies, and often times defending against an enemy’s attack strategy meant playing to your weaknesses while they played to their strengths. In online play, you either agreed that no one played the Zerg, or everyone played the Zerg because they were strong when the other races were weak. The Zergling rush was very effective and while at the beginning it was considered a cheap tactic, it eventually became a part of standard play with people devising rush strategies for the other two sides. And rush tactics began to invade Warcraft II and Total Annihilation, would become a part of Warcraft III and every RTS game around.

I hated rushing. I liked to settle in and build out units so that we could have later, larger tactical wars that ranged all over. Its not that I couldn’t rush, I could, but I just didn’t like it. It ended the game too quick and utilized so little of the game. Among friends we would set timers and say “No fighting for the first ten minutes.” or something like that, but online, it was nothing but rushing.

Real Time Strategy games were not the only place I ran into this sort of thing. Over in the First Person Shooter world, circle strafing had taken over. Running a circle around a person at high speed (sometimes while jumping, moving in impossible ways), all the while with them unable to reliably target you. Thankfully this was less prevalent in Team Fortress, which I has started playing exclusively. Death Match and regular CTF were closed to me, like with rushing in the RTS games, I could circle strafe, I just thought it was stupid.

When EverQuest came out, I looked forward to the idea of settling back and playing a table top RPG in a 3D world. I’d actually enjoyed UO, even PvP in UO, and this looked to be like a step up. But EQ attracted the FPS crowd, and flaws in (or a simple lack of) game design allowed circle strafe jumping to take over the PvP side of the game. Again, I could do it, I just thought it was lame. Honestly, have you ever tried holding a sword, running around a target in a circle, jumping repeatedly AND successfully hitting that target? Its nearly impossible to hit, and even if you hit that is going to be very little power in that swing. Even now in World of Warcraft, the Battlegrounds are ruled in melee range by circle strafe jumping morons.

One of my more recent addictions (which I recently realized I’ve been playing for over a year) is Conquer Club. Its basically RISK online, but with a couple dozen or so maps, some with very different rules. Playing random pickup games, I run about a 33% win ratio. I’m happy with that. However, I’m a little envious of some of the people there who have regular teammates and work together to develop strategies. So, last week I hooked up with a guy I’d recently won a game with and we decided to play 6 games together as a team. We created the games on Thursday, and in less than 2 days, we’ve already lost 2 of them. In both games we lost to the same strategy. I spent some time looking around and have found, like rushing and circle strafing, this tactic has become prevalent in the team game world on Conquer Club. I don’t like it. I don’t want to play games this way. It makes the games short, and usually ignores any rules of the map… to me, its cheap. Sure, I could do it too, the strategy isn’t that hard, but I don’t want to. Luckily, the strategy only works in team games, so all I need to do is avoid games against prearranged teams.

Overall, all these strategies boil down to one thing that I don’t like… the best defense for them is to use them yourself. Using any of these tactics dumbs down the game and removes nuance. Both teams are now fighting exactly the same way and the only factors remaining are “luck of the dice” and who can execute the strategy faster. I can respect when someone executes a thoughtful attack and forces me to change my plans to react, but I just find it distasteful when the strategy they use is a trump card: I either lose, or I play the game exactly like them.

Can this be fixed? Should this be fixed?

I don’t know… but I do know that slowly it is driving me away from real time multi player games and more toward single player and turn based games with clearly defined rules of game play. You can find me on Xbox Live playing Catan, Carcassonne, Uno and others.

To Crush Your Enemies

Mongol General: We have won again. That is good! But what is best in life?
Mongolian trainee: The open steppe, fleet horse, falcon on your wrist, wind in your hair!
Mongol General: Wrong! Conan, what is best in life?
Conan: To crush your enemies, see them driven before you, and to hear the lamentation of the women!
Mongol General: That is good.

-exchange from Conan the Barbarian (1982)

Charlie, Keely and I used to get together at their house and play Risk. Except by luck of the dice, Keely always won. But over time Charlie and I got better at learning when to strike and where to fortify. Those days of gaming were good days. Risk was a good filler between days of AD&D and Top Secret.

Yesterday, a few gents from the ofasoft boards introduced me to the Conquer Club. Basically, its online Risk. You can play tradition board with sequential turns, you can play teams, you can freestyle the turns, or even play on one of numerous variant boards like Lord of the Rings, Discworld, USA Apocalypse, Russia, Canada, and more. If you join for free you can play in up to 4 games at once. If you buy a membership for $20 you can play unlimited games.

It has been a long time since I played Risk, so my skills are rusty. But its fun just the same. So if you like Risk but can’t always get people to play with you, sign up and maybe we will meet on the fields of battle.