Many tested apps in Extras-testing are not promoted timely
 

In Extras testing there are at the moment 222 apps & themes and 44 of them are in status "Promotion unlocked, waiting for maintainter to promote". These 20% is a pretty high number.

How can this be reduced?

Brainstorm at Many tested apps in Extras-testing are not promoted timely

Re: Many tested apps in Extras-testing are not promoted timely
 

In my case it's because of two things:
- I got server error when tried to promote severall my themes right after receiving promotion unlocked -mails

- I can't remember which of my themes are unlocked since there are so many of them. A single page which would show status of all my maintained packages would be more than helpful. Now it's slow to search for all packages and then go and check if the testing version is promotable or not.

Re: Many tested apps in Extras-testing are not promoted timely
 

I’ll try and see with our webmaster if we can resend the promotion emails.

I think there is actually a page for that, but I don’t know the URL by heart. The easiest way, barring that, is just to go to http://maemo.org/packages/repository...xtras-testing/ and see if your packages have *green* karma numbers (which means unlocked).

Re: Many tested apps in Extras-testing are not promoted timely
 

Thanks for your thoughts.
The server issue is really an subotpimal. What do you think of solution 1?
If you had an auto-promote option enabled your packages would get promotion when the server recovered.
How about this list?
If the Karma number is green, the package is unlocked and with 5 pages the list is not to long.
And this list is sorted alphabetically and your packages have all the same prefix.

Nevertheless, maybe you can add more solutions to the brainstorm issue?

Re: Many tested apps in Extras-testing are not promoted timely
 

Thanks for the urls. I cleared the list on my part:
- promoted packages that were PR1.2 compatible
- thumbed down the ones that were not

Re: Many tested apps in Extras-testing are not promoted timely
 

I created a solution suggesting to add an option for a maintainer "will not promote".
What do you think?

Re: Many tested apps in Extras-testing are not promoted timely
 

Is there no possibilty for a package maintainer to demote a package if he does not wish to promote it?

Re: Many tested apps in Extras-testing are not promoted timely
 

What would the rationale be for that ? Testing is a temporary repo before getting to Extras. If you don't want it in extras, you should keep it in extras-devel. If you mean 'will not promote' as in want to remove the package from testing, they just need to thumb their package down.

Re: Many tested apps in Extras-testing are not promoted timely
 

One teensy request - please upload screenshots of the themes on the respective pages (as it is now, they all have the '?' instead of screenshots).

Re: Many tested apps in Extras-testing are not promoted timely
 

Okay, I did not know. The intention was, that a developer might just want to collect some feedback for a alpha/beta version, but does not want to have this version in Extras.

Re: Many tested apps in Extras-testing are not promoted timely
 

There are other problems too: See the packege "maegirls". The version in testing has a bug fix and portrait mode support but it cannot be promoted yet. This prevents us from uploading versions with minor changes since they take forever to be uploaded. For example, the last version of maegirls has a typo in its description but I'm not going to fix this and loose the 6 votes it collected in 2 months.

Once again, IMHO, the 10 votes requirement is not a good thing. It makes the process a popularity contest instead of a QA process. It also somehow discourages authors and testers. Again, I've not seen any argument in favour of 10 instead (e.g) of 5 or 20.

Also, testing serves as an intermediate repository for packages that work correctly but have some glitches. Perhaps if there was another repository, where those packages could be promoted from testing with fewer votes, the queue would be reduced. Something like extras-not-so-testing.

My 0.02 euros.

Re: Many tested apps in Extras-testing are not promoted timely
 

The main problem isn't getting an other repository ... i think ... the problem is how the QA is done ...

Re: Many tested apps in Extras-testing are not promoted timely
 

While I agree 100% with that (I also posted this as a solution to the brainstorm), another repository may be needed. But that's perhaps another thing to discuss.

Re: Many tested apps in Extras-testing are not promoted timely
 

Yes, gonna do that but only when I'm on PC. It's paint to collect screenies with the device and upload 'em with it also.

And on my part the thumbing down was because those themes were in testing before PR1.2 came out and became promotable after (PR1.2 has new vkeyboard and old themes aren't compatible with it).

Edit: and probably I'm not gonna update those themes since I already have received complains about me flooding the repos with so many themes and making certain categories @ ham look ugly (why on earth we don't have a category called "themes"????).

This is why I started developing Plastic Theme Pack so I could deliver different schemes of one base theme in one package.

Re: Many tested apps in Extras-testing are not promoted timely
 

Hum ... maybe ...

But to be honest, i didn't upload anything anymore to all maemo extras repositories ... and i use my own repository. And i ven't follow the actual problems of the M repositories.

Re: Many tested apps in Extras-testing are not promoted timely
 

i completely agree with d ivil. a themes repo would make it much much better.

Re: Many tested apps in Extras-testing are not promoted timely
 

Not repo. Just category @ maemo-extras.

Re: Many tested apps in Extras-testing are not promoted timely
 

I find it odd that we don’t have a category for it. I don’t have high hopes, but I’ll file an enhancement request...

Re: Many tested apps in Extras-testing are not promoted timely
 

That’s why we introduced super-testers. You can pass with less votes (not even 5, but 3), but then we need to be sure they come from people who have a proven track record. Also, you need to lobby for it a bit. I don’t know if it’s a coincidence, but as soon as you mentioned it, a testing-squad member jumped on it and voila, your package is in Extras.

Re: Many tested apps in Extras-testing are not promoted timely
 

In my opinion the hurdle is to high to vote for a package (find it and you need a differend account) and the users are to lazy!!!

I collected 4 votes (all positive) within 2 months for my package. And the new version at extra devel was so stable and has so many new features and fixes... I promoted just before the coding competition the extra devel version to testing to avoid confusion. And so I have to get again thru the QA prozess.

I promoted the application at mwkn to vote for the new version in extra-testing. And I got a lot of feedback during the coding competition. People asking for new features, many compliments and over 20 votes at the competition. So, someone must have installed my application and has it in use.
I wrote it in several places here at talk.maemo.org and asked to vote for my package in extra-testing and in the end I got not a single vote for my package. And I'm to disappointed about this to ask every user per PM to vote for my package.

In the end it's fine for me. I can use my application myself. And when I look at blogs, news pages and here at maemo. Everywhere I can read: "get it from extra-devel".
So, why should someone vote when anyway all users are getting their applications from extra-devel?

In my opinion there should be some kind of blocking in the application manager. A annoying hurdle if the user wants to install something from there. They should want it themselves in extras instead of extra-devel!

EDIT: And I guess with xfade I had the last time a super-tester involved. If he is not ranked as super-tester, who then?

Re: Many tested apps in Extras-testing are not promoted timely
 

That was me, and semi my bad that it didnt unlock for a promote a few days ago - I downloaded the package and gave it a look a couple of weeks ago, but evidently forgot to actually thumbs-up it afterwards, v13 mentioning it above jogged my memory to do it.

The list of supertesters is at:
https://garage.maemo.org/projects/qatesters

While it's hard to do anything about lazy users who are actually using your application and getting on well with it beyond just keeping prodding them and hoping for the best, it is worth bearing in mind that from a testing squad/supertester point of view you're going to have a harder job getting test votes with an application like yours - As the application isnt standalone and basically requires a dreambox set-top box for the application to do anything, and thus be able to test it properly.
Sadly I don't own one, and I imagine from the lack of votes none of the other testers do either.

Re: Many tested apps in Extras-testing are not promoted timely
 

We're working on making the process a lot easier by integrating it with on-device applications, so no desktop activities, login-logout, package hunting, etc, are needed. As for laziness - you have my special permission to rub that in every time somebody starts ranting that there are not enough apps in Extras or that they are half-baked.

While I agree it's a shame, part of the problem is that it seems to me the package is quite specific - I didn't test it because I was/am under the impression you need to have a specific set-top box, and thus I was unable to test the basic functionality (one of the prime requirements of testing). Add to this the problem that the package has not previously reached Extras (and has a lot less people aware of it than if it did). Thus, let's not mix these two things - the difficulty to find testers of a highly targeted application not currently available in Extras, and the QA process in general. As a guy with a moustache said once - we have to find a solution that is as simple as possible, but no simpler than that :)

Super-testers have a (tester) tag in the description - you can see that X-Fade has such a tag, too - but you need 3 guys with that tag if you want to pass on super-tester votes (even super-testers can make mistakes :) ).

Re: Many tested apps in Extras-testing are not promoted timely
 

To have this resolved, please vote here (yes, more votes can help)

https://bugs.maemo.org/show_bug.cgi?id=11045

Also, D-livil, send this link to those complaining so they can resolve their problem, too. Plus, I will personally slap everyone around with a trout on the next Summit who complains there are too many packages in Extras (I mean... seriously ?).

Re: Many tested apps in Extras-testing are not promoted timely
 

Thanks a lot for the reply nidO and attila77. Good to hear that there are improvements comming. :)

Testing the functionality is a good point. But in the end we have the QA process to get no harmful or dangerous, unoptied or applications with missing bucktracker links to the extra repros. In my opinion this are all testable things with my application. So don't be afraid. :)

They are very important and they should be tested faithful. I don't want a app that fills up my root-FS or mess my contacts with the calendar. Because of this I only ask users whats about voting, but I don't want to prodding them all the time. I'm afraid that they could only vote for the sake of peace and quiet. (no one want's this situation and I saw such things at the games section - the ecstatic userbase seems to vote at some games totally blindly only to get faster a shiny new game at extras)

Functionality test should also be done. But for my application in the end I'm sure no one are able to make a 100% functionality test. It supports different kind of receivers (I updated my signature to let you all know) and I'm sure no one owns all of them. I own myself only a single receiver. The rest I integrated with help from people who has asked for support. They have done the testing for me with their devices. Sadly, only several from those have also voted at extra-testing for a earlier version. From one I got a email, asking how it works cos his talk account is not working at the package site. After I had explained it he hasen't created one. A other guy complained about a dependency problem, but it was only a misunderstanding about the version number for Qt and PyQt. After several hours of reading and checking (I'm a beginner hence I have a lot to learn) and solving this confusion I finally asked him: How about voting now? I never heard anything from him. :(

Sometimes the functional test is in its entirety not possible. Sometimes it could be enought to test the starting of a app, is the setting screen working, are the data properly stored, are there no privacy problems and the QA system has to trust that the rest of the app works. In my case it is only a small part untestable. If you don't have such a special kind of receiver at the other end of your network connection you got simply a red dot at the network info symbol instead of a green one. Thats all.

The ovi store makes no kind of functionality testing: Cube Touch! And I'm sure the other, dangerous, criteria are also handled in a lax way.
In some cases it should be enough to test all criteria that could damage a other device, mess up with data or send private content to the CIA or Joseph Stalin.

But I will stop now. Its getting a little bit offtopic. In my opinion the not working promoting from testing to extras is a minor problem.

What is the reason for Khertan to not longer upload his great applications to extras? (I used his pygtkeditor a lot during my development)

EDIT:
Conclusion: I like more the Idea to let a professional, skilled tester look for problems with my application without the possibility of a 100% functionality test instead of forcing 9 clueless community members to vote for my package and in the end to promote it, with my final (own) 10th vote, to extras.

Re: Many tested apps in Extras-testing are not promoted timely
 

ian_r suggested a theme category at the tail end of the "package categories" discussion, but no one responded. His comment may have come too late.

Or perhaps there were just so few themes pre-Fremantle that no one saw the need.

Re: Many tested apps in Extras-testing are not promoted timely
 

Do you know why for example this package does not get removed?
It is thumbed down by the maintainer.
http://maemo.org/packages/package_in...flame/2.0.0.1/
Maybe the 3 super tester votes block it? Is it a bug?

Re: Many tested apps in Extras-testing are not promoted timely
 

Apparently a bug - I have contacted X-Fade and now the package(s) have been removed.

Re: Many tested apps in Extras-testing are not promoted timely
 

Brilliant.
To give a bit statistics, at the moment there are 18/193 (9%) in Extras-testing are in status "Promotion unlocked, waiting for maintainter to promote". This is quite a good progress from yesterday (44/122).
But maybe even 1 app not promoted or demoted is too much?

BTW: The Brainstorm has 9 positive votes. One more is needed to start voting on the solutions.

Re: Many tested apps in Extras-testing are not promoted timely
 

I've not followwed this thread but I just fell victim to this problem for the first time.

I didn't want to promote immediately after getting the email because last time someone complained in the window between promoting and taking the time to add a screenshot to the downloads page about there not being a screenshot. I meant to promote within 12 hours but somehow lost track.

To help with adding a screenshot or anything else for the downloads page is why I'm voting for a super tester to promote the package.

EDIT: Small correction: I got the promotion email at below 10 thumbs up and it error'ed when promoting. Unsure which part was the bug (since there had been talk at one point of super-testers bypassing the 10 vote rule) but never got around to filing it. I then didn't know when I hit 10 votes and could promote.

Re: Many tested apps in Extras-testing are not promoted timely
 

Now you can vote for the different solutions.
I added a fourth solution: To send a weekly email reminder to the maintainer, that the package is ready for promotion.
Maybe it is just that simple?

Re: Many tested apps in Extras-testing are not promoted timely
 

There are two cases where a maintainer cannot promote a package because it depends on a not yet promoted user package.
eSpeak GUI Client 0.1-5 has 19 Karma but it depends on espeak-extra-data.
FCamera 0.1.3-1 has 12 Karma but it depends on fcam-drivers.
The second might show a bug, since fcam-drivers has 3 super tester votes but is not unlocked for promotion.

How can this be more clear for testers?

Re: Many tested apps in Extras-testing are not promoted timely
 

This isn't a bug - Current policy is that the three supertester votes will only unlock a package for promotion after 20 days, rather than the normal 10. The idea is that the three supertester votes unlocking a package are a last resort to prevent a not-very-popular package being locked in testing forever, it's not aimed as a replacement to getting 10 votes.

Generally speaking, dependencies should be non-user packages so that theyre promoted automatically when the user package is promoted.

In cases where the developer for one reason or another has a dependency that's also a user package so needs separate voting however, I see two potential solutions:
On the ad-hoc side, maintainers may wish to simply add into their descriptions or version notes somewhere that the package depends on another user package, so that once it's been tested, the tester can then also appropriately vote for the dependency, which they've also implicitly tested.
Alternatively, a change to the package interface could be in order so that any vote for a package also chalks up a vote automatically for any dependency that's also a user package - In this case, 10 votes for the original application will also register as 10 votes for the dependency. That's probably not a trivial change to the package interface, though.

Re: Many tested apps in Extras-testing are not promoted timely
 

Hm, quite torn among the suggested solutions... I'd prefer some combo, like 10 days after unlock supertesters can promote. I don't think it's feasible to get people to re-vote (I guess if you said it's good in the first place then it's a yes). OTOH I'm not a great fan of auto-promotion, maybe the author wants to synchronize a release with another package, a blog post, talk thread, etc.

Re: Many tested apps in Extras-testing are not promoted timely
 

The great thing is that you took the effort to explain this. It is exactly what is needed, we are human, we can be reasoned with, the goal of testing is not a race who can fail more apps. As said, I just skimmed over and skipped the package (as explained, lack of hardware), but did not give it as much thought as now when we have this issue on the table. I can now say OK, I understand what's going on and take a look and thumb it exactly because of this special consideration - that's why we have multiple votes required - sometimes the QA rules don't give a clear yes-no outcome in all contexts and that's why in those cases we need a majority vote. Reading back, I realize I'm quite confusing, I hope people at least get the gist of what I'm saying :)

I hope Ovi folks don't take this the wrong way, but we have standards :)

Lack of patience ? :) It's best for him to explain, but the point is that some people might disagree with the QA process - and that's OK as in we're not the thought police. One of the reasons we got a carte blanche for Extras, including being enabled on end-user devices by default on Maemo 5 (as opposed to Maemo 4.x) is exactly the effort we put in to make sure people don't get super-raw software that is plain broken. If you don't agree with the Extras terms, and do not wish to cooperate in working on it, improving it - you certainly are free to operate your repository (but then you don't get the Extras promotion and maemo.org infrastructure support).

Actually this idea has surfaced several tiems, and I even lobbied for a QA tester position in the maemo.org team, but our budget has been cut and we're struggling as is to rise above the level of pure maintenance mode. :(

Re: Many tested apps in Extras-testing are not promoted timely
 

Yes, I guess I understand what you have tried to say. And it's exactly my opinion.
My rant was not against the system. It was against the lazy users and how the system is sometimes handled. Especially the functionality testing has sometimes a to high weight. In my opinion you can do a full feature/bug check only with simple software. But, we don't own a Device from Apple. :)
And how long will you run the functionality test at a software like KOffice?

The functionality check happens mostly at extra-devel. If a developer has enought good feedback and he has considered to promote the package to testing the fundamental functionality should already work.

Personally I miss this standards at ovi. But perhaps it's only my own opinion. ;)

Ouh, I'm sorry. I'm not a native speaker. I didn't mean it like that.

Let my correct my sentence:

Conclusion: I like more the Idea to let a professional, skilled tester look for problems with my application without the possibility of a 100% functionality test instead of forcing 9 clueless community members to vote for my package and in the end to promote it, with my final (own) 10th vote, to extras.

I wouldn't ask for a paid tester. Sorry. :)

What I tried to say is: In my opinion it is dangerous to ask all the time the users of your own application to vote for your package. They are attached to the developer because they want updates and more features. "Could you do me a favor? - Yes, of course. I like your work." And in my opinion the risk is very high to get this way votes from total noobs and in the end a low quality package promoted to extras.

What about to create a rule that a package in testing needs at least at minimum a single positive vote from a senior tester additionally to the other votes from the normal users?

Re: Many tested apps in Extras-testing are not promoted timely
 

The auto-promotion should be a check box the author can activate if he wants to. But I think the reminder email solution would be something that is easy to implement and would catch a good amount of the forgotten packages.

Re: Many tested apps in Extras-testing are not promoted timely
 

As I learned he rule is 3 positive super tester votes and at least 20 days in Extras-testing promotion is unlocked.
Did you inform anyone about this error?

Re: Many tested apps in Extras-testing are not promoted timely
 

While somehow off-topic, here's a thought to help the extras-testing procedure:

Whenever a new version of a package is uploaded, send an e-mail to users that voted a previous version and ask them to test and vote again:

Here is what I think:

• I vote for a program (yes or no)
• A new version comes out
• I receive an e-mail that notifies me of the new version and asks me to evaluate it and vote it.
• (perhaps) after a couple of days, I receive another mail asking me to vote for this package (in case I forgot - a couple of days should be enough for testing it)

The method could use two user options:

• An option per user, per package, where each user can ask not to be notified about new package versions.
• An option per user, indicating that he is willing to test new apps.

The first option can be a new per-package in extras-testing option "Notify me for new releases" which would be auto-enabled whenever I vote for it.

The second option can be omitted and be considered as implicit since (in theory) whoever votes for an application in extras-testing is testing it.

This will also help in cases where someone thumbs-down an application and removes it. Currently he will not find out of a newer release.

From the user's POV, this method would be like a list of all packages that she evaluated.
From the developer's POV, this will be like sending e-mail to all previous voters.