Hi,

We have been having a private discussion on extras-testing and how it impacts the way software becomes available in the maemo.org Fremantle extras repository.

The thing is that for a piece of software to be available to normal users of the N900 in the Fremantle maemo.org extras repository the software has to go through extras-devel and extras-testing.

To get into extras-devel, the developer uploads the software to the autobuilder queue. If it builds correctly and doesn't break places, it goes to extras-devel, where it is available for the brave who like to live on the edge. Extras-devel isn't really the place for the average user.

When the developer feels that the quirks have been ironed out from the application, she/he can promote it to extras-testing. That is a place where active people who like to test new software can take the application into use, try it out and vote whether it should go to extras. Ten up votes and a certain fixed time in extras-testing and the app goes to extras.

Extras is the final station for software in maemo.org. There the normal user can find things that simply work.

The discussion we have been having is whether the testing phase makes it too hard to get into extras and if it breaks the idea of frequent releases.

To answer those questions, yes, in the beginning when there are only few devices sold getting into extras will be slower. But my guess is that once there are more people with N900's out there trying out new software, it will be the quarantine time that will limit the pass rate.

Also an important thing to note is that the votes are for a particular version of the software. So if version 1.0 was ok, and passed, but 1.1 has a huge bug and get's voted down. The developer simply fixes the bug and promotes the new 1.2 version to testing which starts from a clean slate (and all that time 1.0 has been available for users in extras).

The thing is that the developer can consider the extras-testing phase as a real testing phase for their software.

The person doing testing in extras-testing gets to use the newest software (with the potential that occasionally the software will be a bit unpolished) and help the developer.

And the normal end user, who uses extras get's good quality open software.

One problem I see is that when I see a comment about a bug on my package on extras-testing and the package has some positive ratings already, when I upload a new package with that bug mentioned in that single comment fixed, my package automatically "loses" all other positive ratings.

So right now it would be better for me to not fix any bugs and wait for enough positive ratings to reach Extras instead of fixing bugs one by one (re-uploading the package and re-promoting to Testing) and let testers see if the bugs they reported as comments have been fixed.

Maybe the policy could be to keep the positive ratings and notify all negative voters to re-test the new version? At least for a "minor fix", which could technically be described as just the debian revision number being increased by one (i.e. what "dch -i" does).

I agree that for a "major" version bump, all positive ratings should be cleared and testing should start from the beginning (because new bugs could have been introduced with new features/bigger changes).

Oh, and I forgot.

The testing phase is essentially there for the end user. The person who has no clear idea of what open source software is (because the N900 will find it's way to those pockets too).

We (community, Nokia, developers) want to give a good impression of open source. You only get one shot at giving a first impression, and we want that to be a good one. The normal user should go: "Cool! And that guy made this so that it's open for everyone!"

To do that, we need a bit of testing.

@thomas A simple one line fix can break your package in unexpected ways. Every version should be seen as a separate thing.

I'm pretty sure your new version will get enough votes before the waiting period is over.

If I understand this correctly, the user base is divided into two groups. The power-users on the one hand, who are eager to have the latest apps and install every update. And on the other hand, the regular users, who barely know what an app is, and who would just get frustrated by having to figure out how to use an application that is more complicated to use than e.g. iFart apps.

Those regular users will only get to see the pearls, while the rest of us (which will be the minority) continue to use the N900 as NIT just as we are used to. While IMHO this division of user groups is inevitable to not piss off power-users, only few apps will make it to the majority of users this way, giving them the impression that there is not much available for N900.
Also, power-users have different demands than regular users, so the apps that are of interest for the majority of users may never reach them because they don't get scored high enough by the power-users.

@pycage, sort of. But the people testing will most likely be power users. So they will let the tools they use pass, as long as the tools are in a sensible condition. The number of apps in extras should thus be high.

And the average user is still pretty tech savvy. So it helps if the material in extras is of good quality.

I don't know how an average NIT user uses a NIT, but I would love to see a large portion of the future user base finding extras and seeing how good the apps are there.

Thomas, an app might get positive votes from testers that actually missed bugs A, B, C. Then a tester comes finding those, voting thumb down and explaining the issues in the comments. And then most probably the ones that voted thunbs up would look at the issues and agree on the problems.

It has happened to me already with a couple of packages in extras-testing. That's find, some tester look at certain things and some others look at other things. Almost none of us are professional testers and even those miss things sometimes.

In general my feeling is that the old Extras is now extras-testing. Until Diablo the developer was alone the one that guaranteed the quality of the software. Most tablet users were basically power users and almost everything was sustainable and fine.

With the N900 the most probable is that ol' Maemo users will jump quite fast to extras-testing (just like most long term Debian users end up pretty fast in Unstable) and the newcomers will be mostly happy with the good selection available in Extras.

Most developers will through new releases to extras-testing every two weeks or so, and after a couple of attempts they will make it to Extras, offering to end users in practice an update every month or two. Which is actually a high frequency if you think of it.

Also in practice, getting to Extras for the first time will be more or less tough for most apps, but once you are there you "only" need to be careful not introducing regressions in your newer versions, And documenting the progress with good release notes helping testers knowing what to check.

PS: Isn't this thread more appropriate in Developers? Am I being to pita with these comments about threads in the right forum?

That's true.

I just hope we don't get the same situation with Extras like what I currently have with Ubuntu: There are ~ 9-months-old packages (of gPodder) in Ubuntu's "stable" repository, and most of the time when users report bugs, it's fixed in the latest version that is only available as a single .deb due to my inability to upload new versions to Ubuntu stable (which i kind of understand, though).

We probably all just have to get used to the new process (which should allow for higher-quality packages in the hands of real end users) and forget the "new release today enters extras tomorrow" kind of thing we can currently do in Diablo Extras

Hopefully other developers aren't turned away from uploading to Extras

I agree....