Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Aquillion]

They can just leave them unranked.  Y'all don't have to reinvent the wheel here.

No, they can't.  I can't come in and meaningfully say "I love this one suggestion" or "I hate this one suggestion" (with no comment on any other suggestions) using ranking.  It also makes it a massive pain to rank a new suggestion, because you'd have to go back and redo all your ratings to insert the new thing.

My feeling is that the vast majority of users are going to care (negatively or positively) about only a small number of suggestions.  This makes ranking useless -- you can't rank one suggestion.

Ranking is a system devised for coming up with one clear winner among a reasonable pool of candidates that everyone knows about from the beginning.  It's useless for our purposes.

Vote up, vote down, vote neutral is best for our purposes, definitely.  It's much more suitable to having a huge number of options, and it's much easier to rate new things as they come in, individually, without going through your whole list of rankings.  We don't need a perfect system to show ideal preferences; we just need one that shows which things are generally-liked by a good number of users.

[Footkerchief]

They can just leave them unranked.  Y'all don't have to reinvent the wheel here.

No, they can't.  I can't come in and meaningfully say "I love this one suggestion" or "I hate this one suggestion" (with no comment on any other suggestions) using ranking.

My feeling is that the vast majority of users are going to care (negatively or positively) about only a small number of suggestions.  This makes ranking useless -- you can't rank one suggestion.

You leave an item unranked if you don't care about it.  If you hate it, you rank it, but lower than the ones you like.  The counting algorithm ensures that whenever an item you like and an item you hate are in direct contention, your vote will always go to the one you like.

It's not the best system for expressing extreme negativity, it's true, but we don't have to limit ourselves to one system.

[Aquillion]

You leave an item unranked if you don't care about it.  If you hate it, you rank it, but lower than the ones you like.  The counting algorithm ensures that whenever an item you like and an item you hate are in direct contention, your vote will always go to the one you like.

Yes, but what if I only have an opinion about one or two items?  What if only a small number of people have any feeling about certain items?

My feeling is that that is how many people will be.

What if new things are added later?  How can I express my opinion on them easily, without having to go back and change my prior votes?  Rereading the entire page and reconsidering all my rankings to fit in one new idea doesn't make much sense.

[Shades]

Yes, but what if I only have an opinion about one or two items?  What if only a small number of people have any feeling about certain items?

My feeling is that that is how many people will be.

What if new things are added later?  How can I express my opinion on them easily, without having to go back and change my prior votes?  Rereading the entire page and reconsidering all my rankings to fit in one new idea doesn't make much sense.

Why is that a problem? you don't need to re-rate anything, only rate the new item if it interests you.

Are you thinking we mean ranked in order? so 1 at rank 1, 1 and rank 2 and so on? because we are talking about individual ratings now.

[Aquillion]

Are you thinking we mean ranked in order? so 1 at rank 1, 1 and rank 2 and so on? because we are talking about individual ratings now.

Wait, what?  I thought Footkerchief was talking about Instant-Runoff Voting.  IRV requires that you put the things you're voting for in a specific order (so, yes, 1 at rank 1, 1 at rank 2, and so on.)

IRV is only intended for elections where you need someone in first place, and where all the candidates are known at the time of the voting.  It isn't really very useful for our purposes.

Likewise, you said:

Then just allow everyone to rate any idea they like from zero to ten. Or better yet to be able to rank them in order.

That way you avoid the all my fav ideas are 10 all the ones I don't like are 0 issue.

What did you mean by that?  "Rank them in order" doesn't seem to be a feasible option for a system where people may only want to express a preference on one thing, or where new items may be added as time passes.

[zagibu]

Considering the latest posts and giving the matter some more thought, I've come to the conclusion that the best method for us is the fractional method, where you can rate any suggestion however you want, and the effective vote going to it is based on the attributed rating over total ratings. So, you can either support many suggestions a little, or few suggestions a lot. Of course, negative values are also possible.
The only thing bad about this system is the fact that it discourages people to vote for many things, because individual support of a suggestion decreases as the number of supported suggestions increases. But I think this can be countered by making the fractions decrease non-linearily. Lets discuss using the following example:

User A assigns 100 points to suggestion 3. This adds 1.0 to this suggestion (100/100).

User B assigns 1 point to suggestion  1 and 3 points to suggestion 2. In the linear system, this would result in 0.25 being added to suggestion 1 and 0.75  to suggestion 2.
In the system I propose, however, these fractions would be scaled by a small amount over 1, so that in total, User B's vote would result in more points allocated totally. The scaling factor should increase with number of suggestions supported, maybe something like 1 + (rated suggestions / total suggestions), or scaled versions of that. Or maybe someone has a better idea for a scaling factor...I'm actually not very good at designing such formulas.

The second-best method I currently see gives everyone 100 points to assign, and a max of 10 points positive or negative per suggestion. Or variants of that.

[Silverionmox]

I don't see it as a bad thing, if it were the case that people concentrated their votes. If they push the top suggestion with all their weight, it will be implemented all the faster. At that point people will start to look for another one, and if they limit themselves to the first thirty or so that would still reward people for having given small bits to suggestions.

[Aquillion]

My problem with this is that it encourages people to focus their votes on big, sweeping suggestions and to ignore smaller ones.  People care most about the big stuff -- who is going to want to weaken their vote for a big change to the game, just to vote for a small, clever improvement?  There are many small things in the current list that have barely any votes, while most of the top stuff is big flashy overhauls to core components.

But the small clever improvements that can be implemented easily are the most useful suggestions, in general.  So my feeling is that it's better to chose a voting method that doesn't force people to choose between suggestions; it's better to let them indicate every one they like.

[Shades]

Likewise, you said:

Then just allow everyone to rate any idea they like from zero to ten. Or better yet to be able to rank them in order.

To which you pointed out the floor in ranked voting and I agreed your idea worked with the one exception

doesn't let you distinguish between 'this is a good idea' and 'this is awesome do it now' kinda views.

and we moved on, at least I thought we did

So my feeling is that it's better to chose a voting method that doesn't force people to choose between suggestions; it's better to let them indicate every one they like.

I agree, but It would be nice to see some kind of focus on things people really liked over things they just thought was a good idea.

Likewise there is the possibility of a badly worded title idea being voted down without people bothering to read the rest of the idea. And we should probably avoid that.

Edit: fixed quote tree.. I should check before posting

[Aquillion]

I agree, but It would be nice to see some kind of focus on things people really liked over things they just thought was a good idea.

Likewise there is the possibility of a badly worded title idea being voted down without people bothering to read the rest of the idea. And we should probably avoid that.

Eh.  The things people (as a whole) really like will drift to the top because they will get a lot of support.

Now, needless to say, people voting up a suggestion on one glance is every bit as bad as people voting it down (since you're basically pushing other suggestions down and, in a system with limited votes, ignoring other suggestions.)

But ultimately, I feel the system has to let people vote at a glance.  I mean, look at the sheer length of the ideas list.  It is essential, for the better or worse, that people have the ability to vote things up or down based on a quick look, because it's simply not possible to go in-depth on every single one.  I mean, sure, if everyone took forever to read a detailed essay on every one, that'd be great, but it's much more important to ensure that every single suggestion gets a fair shake at feedback.

Right now there's a huge number of things sitting at the bottom with single-digit votes (or even no votes at all.)  Most of those aren't bad suggestions, it's just that since people only get a few votes, they never reach that point.

So aside from giving people the ability to vote at a glance, and to vote both up and down (essential to let them give "quick input" and to get a more valid result for controversial suggestions), it is important that people not be limited in the total number of things they can give input for.

Small narrow suggestions are never going to get any support or interest in the face of huge sweeping requests; we can see that, generally, in the current list.

[Osmosis Jones]

Might I make one suggestion? Randomize the order the list is presented in, regardless of votes. This will ensure you don't get the items in the top half getting more votes due to location.

[zagibu]

I thought I'd separate the results from the rating process, so that you have to click a separate link to see the totals. And in the rating process, the list of unrated suggestions is presented in randomized order, except the latest X suggestions are presented on top.

[BlckKnght]

I posted a while back that pairwise comparisons between suggestions might be a good way to build a ranking that treats both new and old entries fairly.  I finally found the sites where that kind system is used to build several rankings (using different metrics): http://BestThing.info, http://TheFairest.info, and http://TheFunniest.info (some contributions to those sites may be NSFW).

Those sites were created by Randall Munroe, the author of the comic xkcd, and he explains the ranking algorithm (in general terms) here: http://bestthing.info/algorithms.html.

I think our suggestion list could be sorted with that algorithm, asking the voters:

"Which of these two suggestions would you like Toady One to work on first? Please consider how long each one might take to implement while making your decision."

[Gazz]

Right now there's a huge number of things sitting at the bottom with single-digit votes (or even no votes at all.)  Most of those aren't bad suggestions, it's just that since people only get a few votes, they never reach that point.

I think that's due to a lack of consolidation. 200+ items and 3 votes.

Take the issue "jobs" for instance.

There are at least 7 or 8 entries asking for various aspects of "jobs" to be improved. Sometimes they are even doubling up, such as "Auto-Mining" also mentioning issues that are included in "Automated zones of treecutting and plant gathering".
So which one would you pick with only 3 votes?

There is also an entry for "improved Hauling" and "Improve Hauling to Workshops".
One is #1, the other #225. Why exactly is one of them so unimportant?
Both are effectively about... "Hauling".
Same problem.

And then there's the human herd instinct.
Why vote for somethingthat currently has 2 points to it when I could vote for something borderline related that has 200? My vote would be on a winner item. It may not be what I want but the vote wouldn't be "wasted" on a loser.
That's not what voting is for...
In order for people to vote for the things they really want, the votes must have weight. All votes.

So... what if all these entries would have a tag added so they could be evaluated by the script?

It would list how many "jobs" improvements had been asked for and how many "crafting" improvements.

That would put some sort of value into all the low-order suggestions because even if that particularly tiny aspect of "jobs" was not in the top ten, it
would increase the significance of the general area "jobs".
Every vote would actually be useful instead of merely vanishing into the eternal oblivion of #50+

[teloft]

I came across this tool for ranking Suggestions and more:

http://github.com/rbjarnason/open-active-democracy

I hope this will be useful.

Example:

Spoiler (click to show/hide)

 You can see it in action at http://skuggathing.is/portal for part of the the Icelandic debate on national crises.
(N.B. it is in Icelandic language)