Supertesters - Make, accept, nominations
 

We are in need of supertesters to promote packages on Extras. http://wiki.maemo.org/Testing_Squad

This is a crucial function which is currently suffering because the list of supertesters and promotion rules have not been updated in a long time. http://lists.maemo.org/pipermail/mae...ry/028825.html


https://garage.maemo.org/projects/qatesters/

Of the current list, I propose to keep Andre Klapper and Harald Schmitt because they continue to be involved. If anyone has been involved and I missed it, please speak up. I'm sorry and you will be kept as well.

For new supertesters, we should have people who have a demonstrated interest in maemo and ability to test maemo packages. It is vital that supertesters are trusted members of the community. A single supertester will be able to promote someone else's (not their own) package. Please feel free to suggest different or more objective criteria for supertesters.

So far the following have been nominated:

Pali
freemangordon
nicolai
merlin19xx
MAG

Re: Supertesters - Make, accept, nominations
 

From my side, I completely trust the nominees.

Thank you for pushing this initiative.

Re: Supertesters - Make, accept, nominations
 

The problems I see with the nominees is that they are:

1. developers, so instead of coding they have to do QA
2. part of CSSU team and KP (AIUI the most important comminity projects by now), again, instead of coding/maintaining CSSU and KP they have to do QA.

It will be much better if we have other QA people nominated too. And there are good candidates for QA team, look at the adeclock thread or replacement media player thread (for example).

Re: Supertesters - Make, accept, nominations
 

Agreed!
Just what I was thinking when I saw the nomitated users

Re: Supertesters - Make, accept, nominations
 

Wee question:

Does a criteria / documented process exist for the testing?

I'd hate to think it was a free-for-all chuck-it-on ala batterypatch, or conversely a box-ticking exercise ala Ovi.

Re: Supertesters - Make, accept, nominations
 

I think these testers shouldn't be developers but normal users which have knowledge of the workings of maemo/linux in general and are able to give good and comprehensive feedback. I am looking at Estel or Mentalist Traceur. Ofcourse only if they want to

For the rest of my opinion I refer back to Freemangordon's post (#3)

Re: Supertesters - Make, accept, nominations
 

I can nominee myself, although I have to warn that I don't have enough time to test everything that pops out in the repositories.

Now I try to test as much as possible and report from time to time (you can check my votes) but, as said I can't promise to dedicate too much time to the task.

Re: Supertesters - Make, accept, nominations
 

I think developers as supertesters is OK, as they will be sensitive to what to look for.

Also, I think volunteers should be asked for first, then voted on, to save any embarassment to someone who may be "drafted" but cannot for time, burnout, or some other reason become a supertester.

gregoranderson: there is a Quality Control procedure somewhere in the wikiswamp, but some of its criteria are assessments of issues not immediately understood or handily measured by casual users - optification, battery consumption, etc.

Re: Supertesters - Make, accept, nominations
 

Is it too much hassle to have a community vote?
For example, nominations open until 23:59 xTZ* on 7th Mar 2012, voting 00:00 xTZ on 8th March until 23:59 xTZ on 14th March 2012?

The notion of super-testers is a very good one. However, I have to I agree with freemangordon's comment re: administrating testing vs responding to tests (i.e. fixing bugs & writing/editing code). (post #3)

While super-testers should be code-aware, perhaps they should not be responsible for contributing code to the piece of software they are testing (and trying to promote through the QA system). The reasons for this are
(a) time is finite so it's very much a case of, either administer the bugs or write code; a programmer will, quite rightly, believe he or she is of more value to the cause by writing code - especially if the bugs pertain to software he/she has very little to do with.
(b) managing bugs is not about programming itself, it's about describing problems and finding out whether/how they can be resolved.

Another reason for Not having key developers administering bugs is that a significant bug for many end-users is the absence of documentation. Preparing documentation is time-consuming so arguably someone whose strengths are writing C/C++ etc. are probably best directed to creating software, not paperwork! On the other hand, decent explanation of how to use software is truly vital. Indeed, I would consider it part of the UI, especially for terminal programmes where there is no graphical UI; "<command> --help" isn't always that enlightening!

Once a tester has investigated a programme, he or she should be able to verbalise how to get the most out it, as well as converting (sometimes terse) developer comments into cohernt prose. The most technologically averse user can be surprisingly helpful in doing this.

There is obviously a problem when developers no longer attend (for want of a better word) the community. In this case, talented coders may be needed to coerse the problematic parts of programmes that are languishing in -devel or -testing to get them functional enough to do the intended task (even if with no-frills). All the same, is it not a waste of time for active developers to be doing testing when they could be coding instead?

The criteria for being an admin is more about competence in communication than anything else:
- Can he/she extract info from the developer about how-to?
- Can he/she explain clearly what isn't working in a way that can be addressed be a developer (i.e. write a proper bug-report and separate one feature from another)?
- Does he/she want to bring as much of the software in the repositories up to release standard?
>> the only proviso is that the tester is aware of what the programme "should" do. This is more of a pre-requisite for a testing than a testing-admin role, I think?

Regarding how to test and what to test, this is an area that needs to be negotiated by community developers (as a group) and those willing to test. There needs to be a very simple checklist for software to pass, which any user can test against; where a programme fails a check point, the tester simply needs to say doesn't pass [test] and describe why. Where bugs have already been filed on a programme, the testing admins role would be to gather reports into a todo list for development. Much quicker than have a/the developer go through the bugs because the collater wouldn't be trying to solve things while organising them!

There is this programme - perhaps it could be the first item to get sorted and pushed through the system? KISStester Package | KISStester TMO. Its official bug-tracker is empty but the TMO thread is full of references to problems. KISStester would provide a good basis for promotion. Some variation is required for different types f software e.g. GUI/terminal/library/etc.

Aside: there are some programmes in extras that are barely passable in terms of doing what they're supposed to. Perhaps all software in the repos needs to be 'bugtested' and there also needs to be a means of downgrading software from extras to -testing?

The priority/order for pushing through programmes will always be contentious. IMHO, it would be best to get all latent** -devel programmes to -testing. Once that state has been reached, process latent -testing programmes on a popularity basis (i.e. num total downloads).

Concluding suggestion: appeal for testers and testing-administrators: users and fringe-members might like to do/know more about maemo and their devices. So long as how-to information exists, the main pre-requisite is moviation and willingness.

In this way, there could be a collection of small teams, each with a testing admin, and a couple of testers. These teams would each have a section of the repositories to bug test and promote. For those programmes that failed the test a bug summary could be posted.

To actually fix bugs, very clever people (developers) are required, who are prepared to edit others' code. For this purpose, might a number of community developers might be happy to act as a pool? The idea being that testing-admins could present a todo list for one of these forgotten programmes to the pool, and the next available programmer can convert a well described bug to either fixed or wontfix. I know bug-fixing can be dull but if you're a programmer it's better than writing instructions, right?

*xTZ: any specified time zone - whatever is specified but the same for all.
**latent - not moving through the repos - e.g. un touched for 4+ months.

Re: Supertesters - Make, accept, nominations
 

This is true to a point. The matter of battery consumption is very difficult to define. Can scripts be developed to do the non-trivial/non-obvious things? I mentioned KISStester above, perhaps appending scripts to this? Manually testing of some things may require a better knowledge of the inner workings of the software than the initial developer, which isn't going to grow the pool of testers.

To my mind, there's also difference between a tester and a testing admin. As I've tried to describe above, the testing-admin needs to be able to bring outstanding bugs together and ensure testing/fixing has taken place. Whilst one person can perform testing and admin roles, I'm not convinced the person signing off a peice software needs to be acutely aware of its inner workings so long as it does what it's supposed to and interaction is either intuitive (think grandparents) or described?

I also believe that a testing admin who is not too aware of programming is likely to have a lower threashold of acceptable hence when to promote, resulting in a more working piece of software (obviously the low threashold is a double-edged sword!).

// Edit:

meant say - I'like to volunteer to contribute to this revamped testing effort that aims to get all & sundry from -devel.to full extras. My skills are native english(en-gb) speaker. I can read C/C++ (& edit badly), as well as other languages with C-style structure and syntacx. Been Maemo-ised nearly a year, well and truly! Also, I'd like a way into developing, eventually so by testing is good.

Re: Supertesters - Make, accept, nominations
 

Another thing we need is administrators that could give maintenance over packages to those that request it.

Re: Supertesters - Make, accept, nominations
 

Agreed. I believe three former council members have these admin rights but they are no longer using them. I've asked that they appoint someone else. I would prefer that it be someone other than myself, and would gladly accept volunteers/nominees.

http://lists.maemo.org/pipermail/mae...ry/028841.html

Re: Supertesters - Make, accept, nominations
 

Thank you for volunteering demolition. Would you be willing to be a testing admin? I like your ideas and think you would be great at it!

Re: Supertesters - Make, accept, nominations
 

Would it be possible for these supertesters to do some basic packaging tasks to clean some unmaintained packages?

I'm thinking, for example, in changing categories, adding better descriptions, adding icons, etc. For all these tasks (packagers tasks) there is no need to know the underlying code of the applications and a bit of polishing is needed in our badly managed repos.

Re: Supertesters - Make, accept, nominations
 

Ok.
Actually, should we enable regular testers to do those tasks as well? What could it hurt?

Re: Supertesters - Make, accept, nominations
 

Glad you like the suggestions. Count me in, so long as I pass community-approved selection criteria!

While there is some urgency to get the ball rolling, taking a few weeks to iron out the fine print is worthwhile, I think, anyway.
Today is Mon 20th Feb. How about aiming for Mon 19th Mar, as a date to have the hows and wherefores complete? With a view to verifying*/selecting testing-admins' credentials by 9th April (Easter Monday) so work can start from 10th April 2012.
How does that sound?

*verifiying:
- as much as is necessary to get access, so these people can do the actual promotion.
- possibly a job for the likes of former/current members of The Council?

Matters to resolve...

• The basic remit for prospective testing-admins (TAs) and testers (e.g. activities, procedures, etc.),
• Eligibility and selection of TAs (i.e. pre-requisites and whether a vote is necessary),
• Procedures for initiation and termination (self- or forced) of TAs,
• The target number of TAs, thus how to divide up the repos between the testing teams,
• Standard testing tools (already refernced KISStester, should anything else be used instead or in addition?)
• The relationship(s) between developers and testing teams (what are members' comments on my suggestions above?)
• What happens if a TA makes an error e.g. passes a poor package, especially if it's a dependency for many other packages and ends up breaking a lot of things!?

... and many besides, I'm sure!

Promotion of tester and TA roles:
Although this thread is getting OK hits, it's quite technical/dry so probably turns off quite a few people. To get this project off the ground, we need to make a massive hullaballoo about how easy it is and how beneficial it is, as an average user, to be a tester*/TA. Otherwise if only a few people do it, we'll be in the same state in quite a short time (e.g. when devices start to give up!).
*tester - there's nothing to say that there cannot be grades of tester, based on <criteria> ranging from, say, novice to super-expert-plus-plus-plus?

Aside: Release Repository:
- Should the release repository (i.e. maemo extras) have finer sub-catgories seeing as there are likely to be a few thousand packages in extras, when things get underway?
- Can packages be cross-referrenced into two+ categories e.g. a note programme could be considered both "Office" and "Utility"; if so, how can this point-to mechanism be made to work?

Re: Supertesters - Make, accept, nominations
 

I think we should complete everything concerning selection and verification of tester admins and supertesters by March 19. And then have any process improvements by April 10. With this order, the selected TAs and supertesters can suggest and comment on the QA improvements. But we should do something before April to free packages which have been stuck in testing.

I think we should stay with TA/super-tester/tester roles unless it proves inadequate. Testing is very important, and should be promoted when possible. I should also say Andre is still on as bugmaster so we should check with him on suggestions for coordinating bug fixes. I do expect the number of new packages except for some projects to be decreasing.

Good ideas for improving release repository. There are other suggestions for having QA ensure that the packages in the repo have good supplemental info for the release repos.

How should communication occur? I don't think the forum is optimum. There is a testing mailing list, but it's been unused for a year. How about resurrecting that?

Re: Supertesters - Make, accept, nominations
 

... I will do it.

Re: Supertesters - Make, accept, nominations
 

Just added my email address to the testing list on the wiki page about lists. Is that the one you were thinking of? If so, who fancies sending out an initial message to get things moving?

Re: Supertesters - Make, accept, nominations
 

I'll send out an initial message this weekend.

And thanks for volunteering vi_

Re: Supertesters - Make, accept, nominations
 

Count me in.

Re: Supertesters - Make, accept, nominations
 

Thanks - any help is welcome.

The nominees so far are
Demolition
vi_
ammyt
ivgalvez

Based on having a package testing karma >100, I nominate the following.
mikkov
jer006
snatcher
vpsych
mve
speedevil

Please note any objections or decline a nomination. I sent a message on the testing squad mailing list. Wiki page about supertesters is here.

Re: Supertesters - Make, accept, nominations
 

Hi SD69!

I'm snatcher. I don't know if I'm a trusted member of this community and honestly I'm not the big Maemo expert.
But I'm glad if I can help...

Regards,
Stephan

Re: Supertesters - Make, accept, nominations
 

Hi everyone,

it's been a while since my last tested package, unfortunately University is not letting me continuing that beloved activity of mine.

Therefore I'm here to give up the supertester status, and to thank all the community for these amazing years together. I sincerely love you all.

Best regards,

Emanuele Cassioli
torpedo48

Re: Supertesters - Make, accept, nominations
 

I'm suggesting ability to delete package from repository. Some packages changes name and package with old name (and old version) should be deleted.

Just open HAM and search for packages with same display name

For example:
http://maemo.org/packages/view/midi-support/
http://maemo.org/packages/view/midisupport/

Re: Supertesters - Make, accept, nominations
 

Another ability that might be useful is to edit bugtrackers links. Most of the packages lack such a link which makes them non able to be promoted, by editing this link and pointing to a proper TMO thread these packages would be able to promote.

Re: Supertesters - Make, accept, nominations
 

Hi,

I still can offer some time for testing.

Re: Supertesters - Make, accept, nominations
 

Wouldn't it be nice if there was short description about differences between supertesters and regular testers and the link to the wiki page http://wiki.maemo.org/Testing_Squad#...ybody_can_help in the OP for the newcomers?

Re: Supertesters - Make, accept, nominations
 

I've edited the OP to link to the wiki page.

Re: Supertesters - Make, accept, nominations
 

@SD69:

Another usefull ability for supertesters: add other developer to maintainer list of some package.

So if old maintainer abandoned package and new maintainer was found, supertester should be able to add new maintainer to the list and new maintainer should then promote packages (e.g supertester will not care about that package for promoting...)

Re: Supertesters - Make, accept, nominations
 

@pali

hey, I get it and that's what I'd like to see too. You've said it before. I share your desire to prevent things like letting absent package maintainers needlessly prevent development. But you have to work with me.

When I suggested a TSG to facilitate CSSU, some people say I'm attempting a hostile takeover. :confused:
I thought that accusation was unfair and over the top. I appreciate that at least freemangordan and joergrw supported my general intentions if not the actual proposal. http://talk.maemo.org/showthread.php?t=82372

Now what you're proposing really is a takeover, an appropriate one I think if a package is abandoned, but goes farther than what I've already proposed. Do the majority of developers support it? Will they say they support it? What do you say to those who will claim it is beyond council's authority?

Trying to fix things one itch at a time isn't going to work in my opinion. I proposed a referendum before to permit council to be more proactive and to have the community's blessing to change the governance and structure of maemo.org so that it functions more smoothly. http://talk.maemo.org/showpost.php?p...66&postcount=4 For that, I got personally attacked - "just because you have a strong opinion about how to save Maemo does not make you Maemo's savior..."

So please don't ask me to fix things all by myself. Please propose the referendum (or one like it) yourself, get a few other developers to openly support it, you'll have my support to get it voted on and we should then be able to carry through with several fixes despite any naysayers.

Re: Supertesters - Make, accept, nominations
 

Humour me while I play DA for a moment:

given the current situation (not enough active testers even with the "supertester" kludge, Fremantle never going to be a mass market OS and existing user base is shrinking), is the whole testing thing still needed/wanted? Has it ever caught a "bad" package before (I recall one case of an author abusing the test process, though the package itself was benign) or is it merely tiger repellant?

Maybe Fremantle extras package maintainers should also be allowed to handle promotion themselves?

Re: Supertesters - Make, accept, nominations
 

Good point. We are lowering the promotion requirements significantly though. What is your assessment of the situation with Diablo?

Re: Supertesters - Make, accept, nominations
 

Another point, a supertester should be able to promote not only from testing but also from devel to Extras.

Re: Supertesters - Make, accept, nominations
 

Disclaimer: I've used maybe 5-10% of what's available, and looked at the source of even less than that.

I don't think we've ever had any malware, even in the wild west days before extras. I guess it's that kind of community, and/or the platform was never an attractive target for script kiddies.

Re: Supertesters - Make, accept, nominations
 

Well, I am not sure I support the idea the crappatches maintainer to be able to directly promote to extras

Re: Supertesters - Make, accept, nominations
 

Some software is very specialisd and even if stable, in itself, it may have the capability of causing problems with other application-software and/or the OS and/or hardware.

There are probably several packages that are highly stable but really not suitable for use by the uninitiated. It could be argued that such packages should remain in the testing repository so as to ward off unsuspecting users, while the package is also given a rating on the basis of its stability/bugginess? Packages on maemo.org/downloads have a stability star rating, so why not introduce for -devel and -testing?

On the other hand, self promotion makes getting packages to release repositories a lot quicker. Therefore more people are able to make use of/comment on/etc. these packages... mmm. Not sure though. If self promotion became a reality, there would need to need to be a way of demoting packages, should they pose Any problem (I think some need to go down already!).

Re: Supertesters - Make, accept, nominations
 

Please, could some Maemo administrator approve me as IPython maintainer ? See the post :
http://talk.maemo.org/showpost.php?p...&postcount=740
I am waiting more than one week after contacting you.

Thanks in advance,

Roberto Colistete Jr.

Re: Supertesters - Make, accept, nominations
 

What about a separate repo for the "initiated users" stable packages?
I certainly think specialised software like this should be kept out of Extras.

Re: Supertesters - Make, accept, nominations
 

I suggest that the discussion be continued on the mailing list with the supertesters and others who will implement the testing/promotion process. Please sign up for the mailing list if you haven't already.

If you want to volunteer, then you can also still do so in the mailing list as well.

https://garage.maemo.org/mailman/lis...tingsquad-list