Re: Community Testing Squad (Sprint task:10.01-02)
 

Yes, the package interface will show the changelog available in the package.

Yes, we need to ponder about the complexity of the apps.

Re: Community Testing Squad (Sprint task:10.01-02)
 

As I suggested before, app complexity should drive several things, especially a quarantine period if we continue to have one. Lines of code (LOC) is certainly a vague indicator but useful at a high level. Perhaps type/number of libraries used could be too. We don't have to get too deep into details either-- high level approach should be good enough.

Re: Community Testing Squad (Sprint task:10.01-02)
 

I wonder about some sort of place where people could signify what they are currently testing? To prevent a situation where 30 people test one app and none another. Or am I being overly complex here?

Re: Community Testing Squad (Sprint task:10.01-02)
 

An app testing status page is a must IMO.

Re: Community Testing Squad (Sprint task:10.01-02)
 

I agree that a complexity category would be good. Using lines of codes would be the easiest way to do it. For example, there could be 3 categories:

LOC < X = Simple
LOC > X and < Y = Normal
LOC > Y = Complex

Re: Community Testing Squad (Sprint task:10.01-02)
 

Yes, this is definitely necessary. I would even go so far that the Admins should post a list of the apps that should be tested now. Critical updates would get priority and for the rest there could be a simple FIFO (first in, first out) procedure.

Re: Community Testing Squad (Sprint task:10.01-02)
 

First of all, I'd like to emphasize how important this task is and how much I appreciate Valerio's (et al) efforts. This is just not to misunderstand what I'm saying below, you have my full support and I'll try to be a good member of the bugsquad when the time comes. Let me elaborate:

If you want to test just one app, you should be able to do that, without any hassle, and be a tester. If tester here means something else (as in a tester who is a more devoted tester than an ordinary tester), it should be made clear. We don't want to scare people away with special procedures or by making them think they should do something special to be a 'tester' (i.e. not to loose people to nomenclature, like seen from RevdKathy's post).

The point is that IMHO it's the admin who *decides* what's a blocker based on the testing guidelines (and the actions above are just consequences of that decision). One of the major reasons to have an admin is exactly to filter the feedback so packages would not get invalid thumbs-downs because of misunderstandings.

[ plug ]Not really related, but I use AppWatch with great success for this. [ /plug ]

And this is my main beef. The testing squad should certainly help stuck apps, but should also stay clear of replacing users. If the person in charge of an app does not understand what an app is supposed to do (or interested in it), it's not realistic to expect a good QA process. Organized testings also have the problem that the different testing squad members may be in different time zones, different schedules, etc (for example most of my free time is weekends - that's more than 5 days, and I will hardly be able to do anything during workdays so anything that should be done by Friday is an automatic fail for me).

Stuff that is missed out from my QAmaster proposal altogether (just mentioning it so we are in the clear if this is intentional): the cooperation of tester admins with actual developers. Testing, if issues found, should ALWAYS result in a bug report(s) and, preferably a pointer for the developer how to resolve it. Remember, the goal is not to clean out extras-testing (that's just a consequence), but to help streamline promotion to Extras with a special focus on quality.

Another thing which is probably very hard to implement, so I'll just list it as an opinion. Per user thumbs down IMHO is a bad metric and is shows very little about what has (not) been checked and how much overlap there is among the reports. Not only should we have categories, but THAT is our score/karma. I should not care if there are a 100 thumbs-ups and just one thumb down, if that thumb down is really a thumb down (blocker). So the 'karma' of the app is, in fact, not the number of people who checked it, but the number of categories it has been checked on. This works the other way round, too - it's pointless to have 10 thumbs downs because of a common issue. It should be one thumb down (in the summary), which multiple users can confirm, with a bug report, end of story.

Now, if there were folks who read through this, happy to hear comments. No pressure, no matter what solution we end up with, it can't really be worse than the current situation :)

Re: Community Testing Squad (Sprint task:10.01-02)
 

Yup, we need to clarify that, as I said anyone can do testing, like now, however the testers referred above will belong to a special garage group.

I don't see any problem here, also as I said, the plan is have the applications tested by the end of the week doesn't matter when you do the testing. Of course sometimes we can arrange small meetings, but only in special cases.

Yes of course, thumbs down must be commented by the testers, ideally with a bug report, that's one of the proposed improvements.
I'm not sure if I understood the QAMaster role, do you think someone would have time to a role like these ? Or are you talking about a paid position ?

Re: Community Testing Squad (Sprint task:10.01-02)
 

Ideally, a paid position, if nothing else, because of the seriousness of the task - that way things would not get delayed because someone had no time due to job/school/etc, and people stuck in testing would know they have a person to turn to who is supposed to help them (the idea of testing is not to have a hundred people individually go and hunt around about their showstoppers - if we know what the problem is, we should have a fix in place in hours if we can, not weeks !). But, even if there are no funds or possibilities for such an option, we could think about things like rotating QAmastership (say, monthly or biweekly, from the pool of testing admins/moderators) to avoid burning out and making some breathing space for volunteers. Effectively, of course, what I called QAmaster, you could say is a part of testing squad administratorship, my point being that such activity (helping developers through testing) should be considered part of the QA procedure itself (the paid position helping to secure dedicated time for this). Seriously, if one person can just double the throughput of testing by helping the developers not just to identify problems but solve their issues, it would mean a lot more applications in Extras, motivated developers, no 'calculative promotions', summa summarum, an investment IMHO well worth the money.

Re: Community Testing Squad (Sprint task:10.01-02)
 

Humm, I can see some pros but a lot of cons as well. I'm against a paid position as least for now, since I strongly believe that the community can take care of testing and this money can be spent in other things that we can't do. We already proved that with a broken system and disorganized testing :) .
A paid person will scare away a lot of the current testers.

As for rotating the QAmaster among the testing team, well the proposal includes something similar, but of course that person doesn't have total control, when we have a group of capable persons why put all the responsibility in just one person ?