[Under consideration] Developers should get karma based on the relevance of their software
 

A proposal just filed to the Maemo Brainstorm:

Developers should get karma based on the relevance of their software

Currently the developers receive karma basically based on how many Garage projects are maintaining or involved with. This is actually a not very relevant statistic: one developer might have only one project making happy to 100.000 users (see Mplayer) while other might have opened 14 garage projects for 14 command-line direct ports of Debian that actually nobody uses or could care less about.

This is not easy to address as there are many possibilities to be unfair e.g. several developers in a project with different degree of involvement, apps downloaded by thousands that are "easy" ports of projects developed hardly by someone else. I wouldn't make big fuzz of this karma for developers, but it would be good to take it into account somehow.

Currently it's easier to get plenty of karma blogging about apps, talking about apps and commenting bugs in apps... but at the end are the dvelopers who carry with a lot of the hard work. Let's praise them!

Please rate the solutions proposed in the Maemo Brainstorm, add your own solutions and comment (in the Brainstorm if possible).

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Ooo, I love contextual rating solutions! Very nice, Quim!

Wouldn't Downloads and comments on apps be good metrics to start with?

EDIT: I'll take that question to the Brainstorm.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

The trouble with download metrics is that you have no feedback how many people UN-installed the app :) Imagine you have an app that has 1.000 downloads and a 4.9 score, and one with 10.000 downloads and a 3.9 score... How would you compare the karma of the two ? Are the extra downloads results of quality and recommendations, or just flashy screenshots and catchy project descriptions ?

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

I'll write here, better no pollution in the Brainstorm page.

@Quim: I don't see an edit button anywhere and I only have Create solution and idea in the drop-down menu. How do I edit my solution?

@Texrat: They're counted, but very slow :)

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

I have solutions to that... but should we move the discussion to the Brainstorm, or would people prefer answering here as well?

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Hmmm, so once I ahve enough karma I can vote and run for council, what other reason should I care about Karma?

Why do I care if my Karma is 350 or 3500, or 35000???

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

For editing, there should be a "notepad with pencil" icon next to your solution. I think.

If you absolutely cannot edit your solution, post what you want it to say here. I can update it. (I think.)

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

I will write a Karmasutra app to boost my dev karma :-)

In earnest: it will be extremely difficult to find a solution for that. The maemo dev community is still very small (just subtract all the somehow-Nokia-related devs in garage/downloads and see what remains!) which makes me think that it is currently too early to introduce more granularity / more complex rules into the karma formula.
In fact, each app available in downloads should just get huge karma just for trying and getting there! (and additional points should be given for apps ending up in the maemo extras repository).

Also, the state of many apps in garage is quite "open"...

In addition, how should we count the extremely high differences in application complexity? Examples: Canola vs. some (of my) Desktop Search Plugins vs. (future Maemo 5) WRT widgets.

I'd say: let it rest for another 6 months.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

No notepad or pencil (or both) :)

I just want to remove the excessive "Solution #4" in the title.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

I confirm Kamen's experience-- I never see that notepad either.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Yep, I had that problem too. Either we're all dead blind, not seeing the forest from the trees or it's really missing. Considering this is a recurring thing, I filed a bug, hopefully it'll make this clearer...

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Look, this is another one of those fuzzy things that tends to get pulled into polar corners. It shouldn't be.

In principle Quim's point is highly valid, and I doubt there would be much disagreement overall. The doubt and discomfort lie in the details.

The best mindset when approaching a fuzzy measuring challenge is to accept at the start that it will be imperfect, and you are looking for the least imperfect solution(s) without expending unreasonable effort.

To that extent we start by recognizing the obvious metrics, determining a useful algorithm and mitigating errors and unknowns.

One unknown is uninstalls. Ok, fair enough. But surely this unknown is mitigated by the fact that ratings tend to be applied after a download. And utlimately almost every app will be uninstalled, updated, reconfigured, etc so this is an acceptable, common risk not isolated to just a few apps but rather spread out over many.

Downloads are a very useful metric. So are ratings. But the point was made that a highly-rated app may have fewer downloads than one rated a bit lower. Fine-- then we weight them accordingly and/or introduce factors into the algorithm that accomodate this. Nothing new here; measurement systems encounter this all the time.

Statistics will tell you that reasonable perfection is attained at about 95% of your goal-- any further and you enter the realm of diminishing returns.

Anyway I think this is achievable... but maybe it does require more thought experiments...

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

I added a solution (typo in title and formatting didn't work (do I need to put <p></p> or <br/> tags?), can't find option to edit, please fix):

Combination of ratings and distribution among developers but per version

This proposed solution is combining solutions #2 and #3 but factoring in versioning.

The aim is for better apps, so if the developers themselves tell end-users to rate their apps as well as provide feedback, ratings and comments becomes then help improve the apps.

When rating apps, it should be by version (major release only). A version 1.0 of an app can get 3 stars but on the next major version release, say 2.0, it can get 5 stars.

As for distribution of karma, it should be by version (major release only) too. Version 1.0 karma can have 3 developers sharing, but for version 2.0, there can be 5 developers sharing.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

I'm a good example of karma-looser :D
I slightly disagree with "version" metrics Reggie proposed above, cause, for example, in our OMWeather project versions are 0.xx all along just because they started like this from the beginning and they are basically for developer and in less part for user (just to check if you have the newest version).
Maybe we can count update rate?

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

What does karma "do" for the developer or application?
Does it make Nokia support and maintain the application or will Nokia disregard these applications just like the ones they delivered with the device?

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

I agree -- there should be a tag on the version if it's a major release, minor, bug fix, etc.. That way you can tag v0.0105 and v0.0209 as major releases if you want to.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Well, basicaly, it's like "speciallist rating". You can ask a person with high karma something and you'll get your answer/help for sure. It's a community thing, not Nokia or IBM :)

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Reggie, that's a great start to providing useful context. Post-beta and mature releases should, IMO, enjoy additional karma... but are we dependent on an honor system here, or is versioning strictly enforced?

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

It would be nice to avoid provocation... I don't see it as productive.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Is that;

a) "If you don't have anything nice to say then don't say anything"

or

b) "It's a community as long as you agree and like it - if you don't shut up".

Just curious.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Thanks, I appreciate the clarification.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Thanks. See my reply to wazd above: http://talk.maemo.org/showpost.php?p...0&postcount=16

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

c) be a reasonable human being.

Thanks

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

I'll take that as 'a'. Thanks.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

It's simply that your opinion of how well Nokia maintains their apps just gets in the way of the topic at hand and doesn't contribute anything useful towards it.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Interpret as you wish, but I deliberately discarded A and wrote my C for a specific reason. Different meaning.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Well, versioning can be disregarded - every project has maturity in its settings.

P.S. This is also a quote to wazd's post.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

It just that it made my stomach turn a bit when I see a post from qgil (who I BELIEVE is a Nokia employee) starting a post about community developed applications and highlighting fluff like " but at the end are the dvelopers who carry with a lot of the hard work. Let's praise them!" - when at the same time they are unable to even keep their own developed applications updated - and/or come out with new ones.

Now...If Nokia would actually pay these developers (based on some Karma system) then I think that would be a fantastic idea and I'd cheer along.

So to me, it was apt.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

It's helpful to separate roles. Assuming Quim as a Nokia employee is directly responsible for unfinished or unsupported official apps is disingenuous and counterproductive.

can we move on?

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Well, I WAS going to send him a private message asking who/where I can send inquiries to about support for Nokia developed applications...but it turns out that I got this message:

qgil has chosen not to receive private messages or may not be allowed to receive private messages. Therefore you may not send your message to him/her.

A somewhat interesting choice by someone who in his profile says:

Hi! I work in the Maemo SW team advocating for open source. In practice this means I'm responsible of promoting peace, love and intelligence at maemo.org + all our relationships with free software projects and the community in general.

I guess that doesn't extend to private messages.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

It s mean that he prefer also open sourced messages than private one.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Who's going to start the "community members should get karma based on the relevance of their posts" thread? ;)

bummer, in all seriousness I wouldn't be shocked if Quim turned off his private messages so that he wouldn't have to deal with a flood of "can you send me a developer device?!" private messages. Poor guy's probably got an inbox overflowing with requests already.

And now back to the brainstorm at hand.

I do like the idea of mixing 2 and 3 as mentioned earlier. Hmm.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

I'm going to have to defend bummers right to ask; why does Quim or Nokia even care what a members karma is? Are there prizes to be won?

I appreciate the push to Maemo5 but at some point someone is going to realize that quite a few of the members/developers/enthusiasts on this board are here for what the N8**'s aspired to be. And that BTW, did not include a dang cell phone.

Now Nokia is asking these same members to pay 60%* more for a device that has features they either don't want or can't use. For many, that price is to high to pay. Particularly when they are already holding a perfectly good device in their hand (an existing tablet) that has yet to reach its full potential.

At the same time these same members are seeing threads telling them that they can "help" Maemo5 by contacting developers and telling them to pour all future efforts into new versions of their apps that won't run on the older devices.

I can see where some might feel left out or left behind. Can anyone else?

*N800 original MSRP = $399


@bummer... Unfortunately that's the way the world goes 'round. However, I believe that we will see a new wave of enthusiasm and member support for the older devices as they are passed down or sold. Take it easy on Quim, he has been good for this community so far and I believe he will continue to be a valuable resource to users of the older devices, as his time permits.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

No one is denying his right to ask. This thread just isn't exactly the best time for asking and it's for that reason that I won't go into why a good amount of what you said afterwards is wrong.

If you guys want to discuss it, open a new thread.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Not me :) ...

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Karma is a reputation system that was discussed, implemented and fine tuned within the Maemo community. Like in any reputation system, if you don't like it you can just ignore it, push for improving it or remove it. But this is not the point of this thread.

Karma is one of the elements taken into account in the device programs targetting Maemo contrigbutors. These days we are working on the N900 device program and it's quite evident that developers are ranked with all kinds of distortions and the system seems to praise more the ones talking than the ones coding. Some coders talk a lot, some others don't.

It's too late for any fix now, but it's good to brainstorm for the future. I think we should discuss these ideas thinking on the Maemo 5 timeline, and the implementation should help evaluating the reputation of developers in the near future.

My proposals are not discriminating downloads or evaluations of apps per targeted devices so I don't see the point of discussing something like the N900 price point and the telephony features. There are threads for that already open. Actually the karma system in itself tends to praise the developers that have been here for a long time over the newcomers attracted by the N900, that will start from scratch (but could catch up fast if their apps were great and some of the ideas in this proposal were implemented).

I disabled the private messages in ITT few days after registering, since I don't want to get questions that can be made in public. People that really want to contact me privately can find out pretty easily how to do it. But this is beyond the point of this proposal.

Any other rants you have with Nokia are beyond the point of this proposal. Please get used to see Nokia employees discussing openly about things and keep the discussions on topic no matter what. Creating a new thread linking it from here takes only a little extra effort. Really appreciated.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

What about people who provide large amounts of software that is not in the garage. Is there some way I can provide my download numbers from my websites (penguinbait.com / tablethacker.com) and have them count towards karma?

Or is not being in the garage BAD karma :confused:

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

As long as your software goes through Extras and shows up in maemo.org/downloads, the rest is quite irrelevant.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Do we really get karma for "Garage projects"? The http://wiki.maemo.org/Karma page says so, but then does not explain how's it calculated. And today's the first time I've read about it.

If it's true and Garage is a karma source, then I'd vote for it to be removed too. It would only promote the creation of "empty" garage projects; there are quite a few already.

Re: Brainstorm: Developers should get karma based on the relevance of their software
 

Thanks for reminding me. I've got one myself and I can't seem to figure out how to get rid of it. Help?