Difference between revisions of "History:Editor Rating"

From MusicBrainz Wiki
Jump to navigationJump to search
((Imported from MoinMoin))
(No difference)

Revision as of 16:50, 2 May 2008

  • An alternative suggestion is presented in EditRating

The Problem

The only thing required to create a new editor is a unique username and password. A brand new editor can immediately edit the Database, sometimes with immediate effect (e.g. case-change autoedits). Other edits might be "bad" having a negative impact on the quality of the data and should be voted upon. With the existing system it's very hard to stop this because even if you do something to their account to stop them (e.g. lock them out, or downgrade their privileges), they can just create a new editor account and set about damaging the data again.

The Solution

A possible solution is to create a system whereby newly-created editor accounts are as unprivileged as possible, so they can do no damage -- increased privileges should be earned.

What Privilege Levels Might There Be?

  • Can autoedit case-change edits (currently, this is the default).
  • Can autoedit "add TRM" edits (currently, this is the default).
  • Can autoedit various other edits (the current AutoEditor privilege).
  • Can delete other user's edits.
  • Can immediately cast a deciding vote (e.g. like the accept/reject buttons currently only enabled on staging servers)

How Might These Privileges Be Earned?

Option 1: Direct Editor Rating

One possibility is some sort of "KarmaSystem". From time to time, an editor may choose to record their opinion of another editor - whether they are entering good edits, or bad ones; whether they are voting well, or poorly. Thus the community forms a collective opinion about the worth of each editor. Maybe as the worth passes some preset thresholds, the increased privileges are enabled.

The problem with that is that unprivileged editors could vote on other unprivileged editors, thus falsely earning privilege from nowhere (so to speak). So maybe only the opinions of more-privileged editors should be taken into account. Effectively then, you are "invited" into a more privileged group by the people already there.

In order to make this work with the minimum of fuss, the process of turning one's good reputation into better privileges should be as automatic as possible - in particular, the way that we currently handle making new AutoEditors is still an lengthy process, even since the AutoEditorElection system was created. Ideally the whole process should "just happen", assuming that editor-to-editor ratings are happening continually and in sufficient volume.

Option 2: Using Votes on Edits

Actually, there is an 'auto-vote' system already in place. The votes on moderations. A system might be set up to work based on percentage of edits that got voter approval, from those editors already at a certain level.

For example:

  • New editors can only add info, they can't change it, or vote on it.
  • Low level can vote, but not change info.
  • Mid level can vote and change (with vote approval)
  • High level can vote and change (certain things getting auto status)
  • God level can auto change anything :)

Now, suppose I (new editor) submit lots of correct releases get approval votes for a certain percentage (we all make mistake now and then). After submitting a set number (say 50, just for discussion) with a 90% (again, just for discussion) rating, I get advanced to low level. Now I have voting power. I keep contributing, correctly, and use my power to vote correctly (i.e., higher levels don't always vote no to things I've voted yes for - perhaps low level votes need a supporting higher level vote to count)

Soon I get to mid level, and my changes get good votes... etc., etc.

This would not only tend to keep those who submit info 'honest', it keeps bad editors from getting to a position where they can degrade the data and rewards those who actually contribute! Using a system like this, it would be possible to really work at it, and get bad editors in a position to be bad... but it would take enough work that who would bother?

Outstanding Questions and Discussion

  • How might the editor-to-editor ratings work? i.e. how can we encourage people to vote on other editors, thus contributing to the community opinion? If insufficient ratings are recorded, then no privileges can ever be earnt.
  • How, if at all, might privileges be automatically revoked - say, if someone's ratings drop below some threshold?

Brian McGroarty points out the following interesting links:

Just a thought, it seems likely to me that this type of system would work for MB, since edits are pretty much right or wrong. Contrast with many discussion sites where you have left/right bias of editors and people down-voting when they disagree (as opposed to down-voting irrelevant or inarticulate comments, which is what "should" happen).

What I do not like with the idea of EarningPrivileges is that it suspects that every editor could be bad. IMO we should expect editors to be good, correct them if they do bad edits and only rate them down if we see that they continue doing bad edits anyway. This would mean that editors do not start powerless.

But then this shows two conflicting tendencies:

  1. Give users the power to edit and use MusicBrainz effectively data now (this would include the ability to change something and use it in the tagger immediately).
    • Because this makes MusicBrainz useful it will attract more users who will edit and vote more and enhance MusicBrainz.
    • But as this gives newbies releatively high rating it will not help against bad editors as these can always log in with a new username.
  1. Let editors earn their privileges as described above.
    • This will prevent bad editors
    • But this will make MusicBrainz less usable for new editors, thus turning them away from MusicBrainz unless they are willing to invest quite a lot of work into it.

There is a conflict between these two options and in the community there are people, who prefer the first one, and others, who prefer the second one. The proposed SurvivalOfTheFittest moderation system might make a clear descision towards option Nr. 1 because there everybody who uses a value automatically votes for it. Here a KarmaSystem (where the weight of the vote gets bigger with more karma) might still be helpful. But a system, where you have to earn the rights to change the data with immediate effect would be counter-productive. --DonRedman

Some interesting ideas here, but I'm not sure any would actually lead to a "better" database. The article starts by listing the "The Problem" as "The only thing required to create a new editor is a unique username and password." That's not the problem, in my opinion. That's a good thing. The problem appears to be that new editors may submit "bad" data. An unweighted voting system seems to fix the mistakes iteratively, which I prefer to arbitrary judgments, as I regard consensus decisions to be more accurate and equable. I'm not convinced that a newbie can really do much "damage" to the database and I think the current level of priviledge is about right. Perhaps there have been cases in the past where edit spam from users with multiple accounts has been discovered, but I've not personally witnessed it. I'm a firm believer in one person, one vote. Having different hierarchical priviledge levels can lead to jealousy, bitterness and is counter to my democratic ideals. Newbie editors might come along and think "I have no chance of my proposed edits getting approved, because I'm just the little guy" so they won't bother joining. You have to give some powers to newbies, so that they feel part of the community, rather than excluded from what can appear to be a clique of established editors. --artysmokes

  • As stated earlier in numerous occasions, this is not a political system: this is a database, with data review processes. The problem is indeed that nothing prevents a motivated bad editor to create a new account (once his previous account has been deleted or restrained) by just using a new username. This is the problem this article tries to address. Currently, the system already has many different privileges levels and variants on EditTypes. I absolutely fail to see how tweaking these privileges as to address the aforementioned problem would lead to users "not feeling part of the community" - furthermore, I fail to see how "feeling part of the community" relates at all to these problems: some people who are not editing at all, or editing very few, are indeed prominent members of this community, while others, very active editors never take part in any community activities. -- dmppanda 16:02, 25 May 2007 (UTC) The reason I want weighted voting is so that edits can be accepted more quickly and still receive ample verification. By giving "trusted users" more weight to their vote, they can help new editors' edits pass more quickly (by potentially not requiring them to wait the entire 2 week open edit period). -AaronCooper, 2007-08-20

Authors: (Who is the original/first author?), Jinxie, DonRedman