maemo.org - Talk

maemo.org - Talk (https://talk.maemo.org/index.php)
-   Development (https://talk.maemo.org/forumdisplay.php?f=13)
-   -   [extras-testing QA] giving thumbs up in testing without following QA (https://talk.maemo.org/showthread.php?t=60739)

geneven 2010-08-22 12:40

Re: [extras-testing QA] giving thumbs up in testing without following QA
 
I don't think changing the names will help anything.

I personally would like to know whether the developer thinks a given program is ready for popular consumption or not. An actual comment by a developer might even be handy. Like "this might have a few bugs but is probably not dangerous".

I think that the hysterical warnings quoted above are less than helpful. They have the effect of warning fearful users not to try out programs that are perfectly safe in general.

Jaffa 2010-08-23 13:16

Re: [extras-testing QA] giving thumbs up in testing without following QA
 
Quote:

Originally Posted by fms (Post 794775)
To me, the number of downloads is a sufficient measure of repo popularity.

Not really, since everything has to go through Extras-Devel and Extras-Testing - but not everything from Extras-Devel and Extras-Testing makes it to Extras. If I push 3 in-development versions to -devel, and it is installed by 10 people but only the last one works, that's 20 downloads of broken software and 10 downloads of working software.

When that version gets through to Extras and is used by 10 people, it gets ten downloads of working software.

As Maemo pushes updates for your whatever repo you have enabled, you could potentially see more "users of -devel" in this (not uncommon) scenario, than actually there are.

I think there are, however, useful stats we can use to gauge the success of the QA criteria and the -testing process. We could also measure these to see the effect of various changes and strive for the numbers to be as good as possible:
  1. A comparison between the number of Section: user/* packages in Extras-Devel, Extras-Testing and Extras.
  2. The average number of downloads per day for the current version of each user-facing package in each repo.
  3. The average time it takes for a version of software which gets through to Extras to get through.
  4. The number of "user/" packages older than 10 days in Extras-Testing.
  5. The number of separate testers.
  6. The average number of packages rated by each tester each month.

Saturn 2010-09-25 14:01

Re: [extras-testing QA] giving thumbs up in testing without following QA
 
The only problem I see is this (from Super-testers):
Quote:

Super-testers

Super-testers are members of the testing-squad with a proven track record (currently testers with more than 100 package testing karma). Their votes (can) count a bit differently than those of regular testers.
That is, testing karma of 100 is not any prof at all if the tester knows anything about what he's doing!

Cheers

attila77 2010-09-27 07:46

Re: [extras-testing QA] giving thumbs up in testing without following QA
 
Quote:

Originally Posted by Saturn (Post 826010)
The only problem I see is this (from Super-testers):


That is, testing karma of 100 is not any prof at all if the tester knows anything about what he's doing!

Cheers

Agreed. If you take a look at my proposal you'll see that 100 karma is just a *prerequisite*, not a number that will automatically grant you that status (yes, the wiki excerpt misses that part of the proposal). In addition to the karma you need to be nominated/supported by a Council member or an existing tester based on your past activities - and those can be seen on the comments list. I hope this addresses your concerns.

gryedouge 2010-09-28 08:44

Re: [extras-testing QA] giving thumbs up in testing without following QA
 
Quick question; with reference to the following

'If you don't feel ready to join the testing Squad or don't have enough skills, you can still help by testing the applications from a user point of view (functionalities, description, etc...). For more information read article Help testing software.'

I am wanting to test/assist along these lines (and then maybe, when i have enough experience apply as a tester) and have done so for a few apps. Do i need to do so as per the QA checklist or, as I have been doing so far, from a general end-user perspective of basic looks, functionality, integration, navigation, completeness of app, etc?

I really would like to help but I dont want to damn any apps to the netherworlds as a result of not following the required QA process.

Any advice or suggestions would be welcome.

attila77 2010-09-29 10:00

Re: [extras-testing QA] giving thumbs up in testing without following QA
 
Quote:

Originally Posted by gryedouge (Post 828165)
'If you don't feel ready to join the testing Squad or don't have enough skills, you can still help by testing the applications from a user point of view (functionalities, description, etc...). For more information read article Help testing software.'

I am wanting to test/assist along these lines (and then maybe, when i have enough experience apply as a tester) and have done so for a few apps. Do i need to do so as per the QA checklist or, as I have been doing so far, from a general end-user perspective of basic looks, functionality, integration, navigation, completeness of app, etc?

You can always leave feedback for the packages (even without voting), and it is most welcome from the aspect of both testers and package maintainers. Generally you should *not* be thumbing up packages just because they are able to start and look cool. OTOH if you DO run into problems that cause breakage, feel free to thumb down (with adequate explanation).

gryedouge 2010-09-29 12:06

Re: [extras-testing QA] giving thumbs up in testing without following QA
 
Thanks for the feedback!

Berserk 2010-10-16 22:46

Re: [extras-testing QA] giving thumbs up in testing without following QA
 
Quote:

Originally Posted by Khertan (Post 793058)
if you have a "niche" apps with less than 10 users your apps will never reach extras ...

Sorry for the thread bump, but I think I made one of those niche apps. The problem I run into is that votes are at 0 for every new version.

So let's say my app's got 7 votes after a long while, and I (or one of the testers) notice a bug that really needs to be fixed. I upload a new version, promote it to testing, and all those precious votes are gone :(
I don't want to harrass the testers to look into it again, of course..

I do understand that a version 0.3 can be safe, and that a version 0.3.1 might murder kittens, even though it looks like a bugfix to 0.3.

But this way, niche apps will indeed never reach Extras, not when updates are released on a regular basis (while updating and fixing should be encouraged).

Any thoughts on this?


All times are GMT. The time now is 02:43.

vBulletin® Version 3.8.8