Ok, so our current Puzzle/Replay rating system may need some work. Originally a 5-point rating system, we felt that the rating system was too broad and inflexible.
For instance, people would often rate a puzzle a 4 or 5, sometimes a 3, but never anything else because everyone’s generally a nice person (with exceptions, of course). Additionally, when it came time for people to rate things, many people found it difficult to express their feeling into a simple, generous 3, 4, or 5.
So what did we do?
We made everything out of 10 
The system we currently use, although simple and provides users with more rating options, still has the same problems as the old system. Instead of just three usual choices in the old system, most people gave puzzles/replays a rating of 7, 8, 9, or 10 (with exceptions of course.) More importantly, everyone’s got a different interpretation of how much they enjoy a level/replay. Whether it may be the effort someone put into the puzzle, it’s difficulty, looks, or just encouragement, everyone’s got a different interpretation of what that number out of 10 represents. Unfortunately, all those different interpretations do not translate perfectly into a single number out of 10, or better yet, a 7, 8, 9, or 10.
Consequently, with all these different interpretations and expectations going around, it’s inevitable there will be/are disagreements and arguments going around as a result of our current rating system. While other sites have a similar rating system out of 5 or 10 designed to make rating easy (YouTube comes to mind), I have a feeling that we may need to revamp our own rating system. That, or Eets Puzzles or Replays are just that special 
With that all said and done, here are some ideas I thought of that we can do with our rating system, along with some ideas proposed by some of our community members in an earlier forum post:
-
Do nothing: Some people may argue that the current rating system is fine and doesn’t need fixing at all.
Pros: The original reasoning behind our current rating system is that it’s designed to be quick and easy to rate puzzles/replays. Someone could play a puzzle/replay and right after, pick a rating and be done with it.
Cons: As stated above, having only a single number to represent a player’s opinion of a puzzle/replay can be vague and perhaps uninformative, given widely differing criteria and interpretations amongst reviewers. -
Anonymous voting: Allow non-registered people to rate puzzles/replays.
Pros: Potentially allows more people to rate a puzzle/replay, leading to a more comprehensive overall review. Also makes rating things much more easier and quicker.
Cons: Allows for someone to spam ratings in order to skew puzzle/replay ratings. -
Re-word existing rating categories: What’s the difference between ‘great’ and ‘awesome’, ‘poor’ and ‘bad’? Perhaps the existing rating categories are too vague in their descriptions and need to be reworded, or adjusted to give a broader range of ratings (e.g. As Lillarion suggested, a 5 would be considered ‘good’)
Pros: Can broaden up the rating system so that players have more options to rate puzzles/replays.
Cons: It still leaves ratings open to individual interpretation, as stated above. -
Adam’s Criteria: Stated by Community member Adam, these are a set of guidelines he uses to rate puzzles. Perhaps one idea would be to create and include a set of general guidelines that people can read in order to help them properly rate a puzzle/replay.
Pros: Makes the everyone’s interpretations of rating a puzzle/replay using the existing rating system a bit more uniform and clear.
Cons: It’s always tough to make people read a set of guidelines and follow through on them
-
Multiple categories: One idea to remove the problems with the existing rating system is to introduce multiple categories to rate, similar to how most game review sites rate their games usually based on graphics, sound, fun, etc. For Eets puzzles/replays, the categories could be something like Fun, Creativeness/Originality, Looks, Difficulty, and Bonus points for those special levels/replays.
Pros: Clears up the rating ambiguity present in the current system somewhat.
Cons: Makes rating levels longer, since there are more categories to rate.
These ideas aren’t by any means a final list. Some of these options could make things better, but since this is your community, what do you think should be done? What combinations of options do you like/dislike? Do you have a good puzzle rating system to suggest? Hope to hear those ideas, comments, and suggestions!
I’ll respond to these in order:
Do nothing: Not really seeming like an option at this point, but hey, you put it out there for everybody like you said. There may indeed be those who like it.
Anonymous Voting: Don’t think so. Other people wouldn’t know what they’re talking about. Plus I think that fact would make those of us who are more seasoned be insulted more often.
Re-word existing categories: I think they are clear and even fun the way that they are.
Adam’s Criteria: Haha! I’m glad you like it, if it helps everyone, then it is good. Anyway, I don’t like the idea of asking people to read them (flattered as I am). They may even get the impression that we don’t think they know how to think. Of course, that’s worst case scenario.
Mutiple Categories: Really like this idea. I don’t think it should be to many like a big multiple choice test or something. I think that this is what those of us who have been debating have been looking for. Maybe only 3 or 4 more options would be enough. And if you use my “criteria” as a spring board, again, I’m flattered, please feel free to do so.
Now, of course, I had a lot to say about this in Website Level Up “Sorta”, and at this point I would take back most of those that apply here. I think the difficulty rating system is great, but only for Puzzles and Clickies. And I still like my idea about the “Top Rated Puzzles”. Just saying that so you don’t think I take it back
Okay, there’s my take on it
Anyone else?
Comment by Adam — February 6, 2007 @ 1:00 pm
I would also like multiple categories, but it might be a bad idea if fewer people will vote. I wouldn’t mind anonymous voting either. But I definitly would like to have a broader range of options. Like:
1 Bleh
2 Bad
3 Mediocre
4 Okey
5 Good
6 Very Good
7 Great
8 Awesome
9 Super Awesome
10 Eets Fantastic
Comment by David — February 6, 2007 @ 2:42 pm
Hmmm, hey David, that looks pretty good!
Comment by Adam — February 7, 2007 @ 9:55 am
The interesting thing here is that, users are very friendly on this board, which is why only 7 and up are used. After all, who wants to recieve a rating of 5 / 10, which usually means “crap”?
Another way we could approach this is by a voting system. Everyone gets 3 votes per level, so if you don’t vote, the level disappears into obscurity, like in Digg. If you do vote, it means you like it — either 1, 2, or 3. You can also comment without voting to suggest improvements.
Levels are then ranked by the number of votes they got. This would allow, for example, my much loved Air Jumping to float to the top. We can also put in heuristics so newer levels float more than older levels, if we so chose.
Thoughts?
Comment by BigFoot — February 7, 2007 @ 7:11 pm
Sorry, BigFoot, I don’t understand the “voting” part. But the “floating” I think is definately worth consideration.
Comment by Adam — February 7, 2007 @ 8:27 pm
I think Bigfoot’s is suggestion is sort of like Digg, where people vote (digg) stories that they like, where the popular ones get shown at the top.
In our case, Bigfoot’s idea would allow for a rating system where you can vote up to three times on a puzzle/replay, while keeping anonymous. That way no one really gets offended
That’s a pretty good suggestion, as is everyone else’s so far!
Comment by CheesyRamen — February 8, 2007 @ 1:35 pm
Ah, ic now. Ya, I’d have no objection to that.
Comment by Adam — February 8, 2007 @ 7:35 pm
I don’t really like your sugestion, BigFoot. If anything I would like to see the reverse in order to make it more probable that people vote for the puzzles that have really few votes (I hope this makes sense)
And it’s still quite hard to even get to the puzzles in the middle, since you can only swap one page at a time when you choose what puzzles to download. I would like to be able to swap maybe four pages at a time, and I would also like to be able to go to a certain page by writing the number of that page. I think that there might be a lot of puzzles in the middle that have a really hard time of getting attention.
Comment by David — February 13, 2007 @ 6:08 am
My only idea would to have a rating out of 3 which would make 1 bad 2 ok and 3 good. This combined with catagories to rate would be perfect and would make people be more blunt.
Comment by Matthew — February 13, 2007 @ 9:03 am
Well… Not really Matthew, since I would just rate 95% of the puzzles and replays as “good”, since they are fun to play. I wouldn’t rate any puzzle as “bad”, since people tend to not post bad puzzles.
Comment by David — February 13, 2007 @ 9:07 am
I’d tend to go with David on this one. The more I think about it, the more I see that greater numbers of options is necessary. We tend to be making a big deal out of the fact that virtually everything gets a 7 and up. But looking at pretty much everything else out there that gets rated online, you see the same thing. The only things with “bad” ratings are those that it seemed the makers had no idea about anyway. So, it is then that the higher ratings of only a range of the last 3 to 5 options is the “standard”. Lower than that signifies something truely bad or ignorant. For this reason, I think David’s idea for re-wording the 10 options looks fantastic. Because it can widen the “standard” options and give us more satisfaction that we fully expressed our views.
Continuing to think about BigFoot’s ideas, I still think “floating” puzzles and replays is worth considering. But, being anonymous does not sound so great. This community insofar has promoted a very good sense of unity among members. It is indeed a risk that we may find disagreements at times, but this is not the rules’ fault, it is the members themselves who must work it out. Obviously this can turn ugly at times, but in the name of uniting the community, I think it is more than worth the risk. Truely, if new folks come into a community that really is a community, and receives them warmly (as I think we have) they will probably be more inclined to get honestly involved than if it is but a machanicle posting place with no people in plain sight. It makes it lonely, don’t you think?
Now, in order to make it easier for people to rate in general, perhaps the OPTION to be anonymous is worth consideration. For those who do not wish to really get “involved” yet, it may be a great way to start. And if they can get a good start in the community, perhaps it will inspire them to take more action.
Comment by Adam — February 13, 2007 @ 10:04 pm