Want more apps? Vote for this bug
 

One holdup for making more applications available is that it takes forever for developers to get programs compiled on the Maemo servers. Things that should take minutes take hour+. This really impedes the workflow for developers, so projects that should be done in an afternoon take a week instead...

A big cause of this is the actual servers building the applications (and hosting things like "brainstorm") are overloaded and sloooooow.

Nokia knows this, and supposedly was getting set up with a new ISP--in October. Will this happen before Christmas? Spring/Fall? 2012?

Let Nokia know you would like to see things sped up... As more users start using the N900, things are just going to slow down *more* if they don't get on the ball with this.

You can vote for this bug as one way to let them know.
https://bugs.maemo.org/show_bug.cgi?id=5818
https://bugs.maemo.org/votes.cgi?act...5818#vote_5818

A bit more info here:
http://wiki.maemo.org/ISP_Move

Thanks!

-Jeff
http://wiki.maemo.org/User:Jebba

Re: Want more apps? Vote for this bug
 

That's a bit unfair, as the guys working on this do not postpone it purposely to save money or something like that.
I hope that it goes live this month. Also see http://wiki.maemo.org/Maemo.org_Sprints/December_09

Re: Want more apps? Vote for this bug
 

That page says it's 2% done and "Work started. The first machines are being installed. Linked to ISP move."

See also:
Checking November:
http://wiki.maemo.org/Maemo.org_Sprints/November_09

Identical text: "Work started. The first machines are being installed. Linked to ISP move." It also reports 2% being done.

In October,
http://wiki.maemo.org/Maemo.org_Sprints/October_09
"Servers will be available in November only. "

-Jeff

Re: Want more apps? Vote for this bug
 

So... Maemo developers can't compile their code locally? I'm confused...

Re: Want more apps? Vote for this bug
 

A night's worth of holdup is next to nothing when compared to the -testing one.

Re: Want more apps? Vote for this bug
 

Yeah, it definitely is confusing... "Must compile apps on Maemo Servers" sounds like a "must distribute app through app store"

WHY?

Re: Want more apps? Vote for this bug
 

"Extras repository process definition" in the wiki may answer some of your questions.

And it is not required that an application be compiled by the autobuilder to be included in Extras. That's only a requirement for "free", and there's also "non-free" for closed source software.*

* Free as freedom, not as in beer.

Re: Want more apps? Vote for this bug
 

You can compile locally and install it on your own fone, for example, but if I want to make a package available for testing in extras-devel, I have to upload the source files to the build server, it gets built there, then pushed into extras-devel. I can't upload binary/pre-compiled .debs.

I also saw in IRC that the build box is qemu-arm (!). Here's it compiling erlang, which has taken something like 16 hours (and still running):

As a point of reference, the guy that submitted the package can compile it in 8 hours on his N900!.


-Jeff

Re: Want more apps? Vote for this bug
 

Okay, this makes sense now. Thanks.

Re: Want more apps? Vote for this bug
 

Isn't it sbox+sbdmock? It will use qemu-arm-user _only_ for build tools that are not in any devkit/toolchain. That doesn't include C/C++ compilers, but of course may include Erlang compilers.

More hardware won't fix this (but will help). Allowing to upload new devkits to the autobuilder would fix that, but would open another can of worms, so...

Re: Want more apps? Vote for this bug
 

I was basing it simply on this:

http://mg.pov.lt/maemo-irclog/%23mae...12-08.log.html

Also, fwiw, he's building erlang itself, not building something with erlang.

Re: Want more apps? Vote for this bug
 

Note that his answer and mine are compatible :)

Re: Want more apps? Vote for this bug
 

Update: that erlang build i mentioned...is still running...

Re: Want more apps? Vote for this bug
 

While I agree that there are serious issues with the build system (especially since Fremantle came online... and it's basically trashing everything because it's being overwhelmed), I've heard that Nokia is also providing several new servers along with a completely overhauled/fixed build setup so that things will be faster. My understanding of it right now is that EVERYTHING (Maemo.org stuff, autobuilder for Chinook, Diablo, Fremantle, Garage, etc) are running off of a single server. That poor little server!

However in the mean time the argument to say "the autobuilder is preventing more applications" is in my mind not accurate at all. Okay, so let's say worst case scenario your app takes 2 days to build in autobuilder... If that's your product release/app release, then 2 extra days for people to wait doesn't exactly mean "there are less apps available because of it". Sure things get backed up a bit, but you're still talking about final releases here. I think the bigger problem is that many people are using autobuilder as their "testing compiler" instead of compiling locally.

For example, when I build a new version of one of my apps, I'll compile locally, test it on my device to make sure it installs/uninstalls properly.. then I'll upload the compiled .deb to the project Garage page, and post links for others to "test install" with. Once they say it's good to go and there are no major problems/changes to be made, THEN I submit to autobuilder... sure it takes a little bit for autobuilder to process and things to show up in extras, but after you finish writing a commerical application it takes a LOT MORE time to get that application published and out to store shelves...

So should things be faster... yea definitely... is the fact that things are slower than ideal holding back the whole community and development process? No way...

Re: Want more apps? Vote for this bug
 

Well mine compiled locally fine in the /scratchbox (I was building asterisk, fwiw) but didn't build in the autobuilder. There are always subtleties and the autobuilder is not identical to the scratchbox. Take a look here and you can see how many FAILS or other errors there are:

https://garage.maemo.org/pipermail/e...er/thread.html

I bet you *all* of those devs built their packages locally, to be honest.

It also does hold things up for more than just two days because you lose momentum. If it took 15 minutes to find out your build failed, you continue on with the project; if it takes you a couple days, well, you've lost all that mindspace and by then 50 other things have come flying at you to work on. It's not just two days wait, it disturbs a clean workflow. I've decided to not bother packaging on there until things get sorted out because it's agonizing to wait, heh.

That and the fact that you can't see any live build logs (like fedora does, for example), so there's no real way to know what state your package is in either.

Anyway, more boxes is obviously going to help. Hopefully they're huge.

-Jeff

Re: Want more apps? Vote for this bug
 

Oh, and by the way, erlang is STILL building.

Re: Want more apps? Vote for this bug
 

Erlang finished! It only took two days, 18 hours:

Re: Want more apps? Vote for this bug
 

The biggest difference I find between scratchbox and autobuilder is that in scratchbox you're not necessarily building a package "from nothing", where as on the autobuilder you are. For example, the first few times I built a package it would compile locally, but when uploading to autobuilder it failed because my source package didn't have "depends" setup properly (it wasn't an issue locally because I already had the packages installed locally; autobuilder doesn't do that though).

What I ended up doing (and found to be the most helpful for troubleshooting/test building) was to get a clean SDK virtual image, and use that as my "local autobuilder"; never install anything on it, never use it for actual devel, but only for testing my source packages to see if they would build on a clean platform. Definitely helped me resolve my packaging problems... should we have to do this? Heck no... I think it's more an issue with the SDK not including that type of "clean build" functionality for testing than anything else though...

Anyways, still looking forward to faster build times on Maemo servers, as I agree completely, delays do tend to take a bit of "the wind out from under you".

Thanks!

-Rob

Re: Want more apps? Vote for this bug
 

Ya, like Fedora has mock so you can simply do a `mock --clean` to get a clean build environment. With scratchbox you pretty much have to blow it out and start over.

I'm investigating sbdmock now.

-Jeff

Re: Want more apps? Vote for this bug
 

If you want more apps you can give a hand in Extras-testing: http://wiki.maemo.org/Extras-testing

Re: Want more apps? Vote for this bug
 

But that has the same problem. Even trying to get an app promoted into extras-testing takes forever. http://maemo.org/packages is dog slow...

Re: Want more apps? Vote for this bug
 

Build server has been out for 14+ hours.

No one has posted anywhere that I can see that they are even aware of the outage.

Apparently it's not a 24/7/365 setup...

Really shoots the weekend when you are hoping to get things built. It creates distractions, mis-directed energy and de-motivating frustration.

A couple days ago, before the current outage, I had a package built. I went to the page to get it into -testing and it just sat there with a spinning cursor. I went to downtown central Buenos Aires, in rush hour, had a meeting, came back (still rush hour) and the thing was *still* spinning. Conclusion?

The servers are slower than a traffic jam in Buenos Aires.

Gah.

-Jeff

Re: Want more apps? Vote for this bug
 

Apparently Nokia admins post updates to this obscure channel:

http://www.qaiku.com/channels/show/maemork/

Re: Want more apps? Vote for this bug
 

That's not a obscure channel, the channel exist for some time now and was properly advertised, can be reached from the wiki front page, and is linked to the monthly sprint tasks assigned to the maemo.org team.
For e.g: http://wiki.maemo.org/Maemo.org_Sprints/December_09

Re: Want more apps? Vote for this bug
 

It's not linked on the front wiki page. The *only* place it's linked to from the wiki is the sprint page, buried at the bottom.

Is that "properly advertised"?

Have you seen how much investigation I've done of this issue? Can't you see I've obviously read *tons* of the wiki and other maemo docs and *finally* stumbled upon this because it was mentioned in IRC? I bet 99% of devs have no idea about the channel.

Even on that #maemork channel it says nothing about the current outage. It mentions some build server may be ready Monday. Great. So is the current outage a planned outage? No mention. Is he even aware of the current outage?

Regardless, the point is that we waste our time on this **** instead of actually packaging applications.

-Jeff

Re: Want more apps? Vote for this bug
 

There's a link to current month's sprint on the Maemo.org Sprints front page but it points to August sprint -> http://wiki.maemo.org/Maemo.org_sprints.

I tried editing it but apparently couldn't...

Re: Want more apps? Vote for this bug
 

As I said this channel is associate with the sprint work, so try under "hot topics", third link. Don't have too much expectations about the things posted there, because they're mostly related to the tasks for each spring and some BAU work.

Well, for the devs that joined the community a month or two ago, you might be right, but you know the community is very big and we have a lot of things, don't expect to learn everything in a few weeks, and we can't put everything that we have in the front page of maemo.org.

This channel is not intended to announce these kind of things, normally those announcements happen in the MLs, but your're right they are missing a lot in the last weeks, but I've to warning you that this is a free service offer to the community, and as you know we're getting new servers (again for free), so expect outage in the next days.

Re: Want more apps? Vote for this bug
 

In the front page: Hot topics -> Current Sprint

Re: Want more apps? Vote for this bug
 

Folks! You've heard it here: just look for comment twenty-seven in a thread linked from the front page of maemo.org linked to talk.maemo.org linked to applications forum to find out about the outage in the "next days"! Everything can't be posted to the front page, you see.

Is there any more info you can point to about this outage? If you know something about it, why not post it to the -devel mailing list or something reasonable?

Also this "for free" thing is a bit disingenuous. That's like Red Hat saying to the Fedora community "hey, you're getting this for free STFU".... I mean, this is greatly in Nokia's interest--they aren't being Mother Teresa here...

-Jeff

Re: Want more apps? Vote for this bug
 

Woo hoo!

The builder is running again. :)

Re: Want more apps? Vote for this bug
 

Do you know you can test your apps WITHOUT the autobuilder, do you? Just asking.

Re: Want more apps? Vote for this bug
 

No Nokia is not mother Teresa, but neither is it telling anyone in the community to shut up either. The idea is that Nokia works together with the community to make maemo.org a good place to be.

As for the new hardware.
There has been a lot of work in both getting new hardware and setting it up. Just be patient a small while more. No promises before Christmas (everyone wants to eat Christmas lunch without watching the server stats on an N900 under the table), but services will start moving to new hardware soon.

The downside is that setting up causes further slowness on the old servers as they need to be copied and synced to the new ones. The decision was not to shut down for a week to do that, but try and keep things running all the time.

The people who do this work (set up and run maemo.org) are not Nokia employees, they are community members who get sponsored to do this work.

Re: Want more apps? Vote for this bug
 

Yes, and I do. It's much easier for *other* people to test them when they are in the maemo repositories though.

I have my own repository set up here:

http://www.freemoe.org/users/jebba/README

-Jeff

Re: Want more apps? Vote for this bug
 

I'm not part of the maemo.org team and I'm not payed by Nokia, so these kind of announcements are not my job, unless someone ask me to do it. If you read carefully my post you can see that I agree with you in the lack of announcements, I already raised my voice, but you can also do it in the MLs, since I suspect that most of the people responsible for the servers don't read TMO in a regular basis.

As for the complains a bit more of patience and gratitude is welcome, we're in the same boat here, those kind of problems are not normal and should be fixed soon, our infrastructure and resources is too small for the community growth, patience please.

Re: Want more apps? Vote for this bug
 

Jebba: please join the maemo-developers mailing list if you haven't yet. You will like it.

Re: Want more apps? Vote for this bug
 

Ya, just actually subscribed. I have been reading the archives though.

Thanks,

-Jeff

Re: Want more apps? Vote for this bug
 

It's been fixed now... ;)

As I wrote I would have edited it myself if I could (I too would like to contribute to the community in any way I can, given the time)... :)

Re: Want more apps? Vote for this bug
 

You can edit the wiki, even without log in.

Re: Want more apps? Vote for this bug
 

I apparently didn't notice the EDIT link on the right (for the whole page), but only the edit links on each section... :rolleyes:

Re: Want more apps? Vote for this bug
 

It's certainly been more than two days. I've been trying on & off to get something built or even uploaded. One time it did make it through the build but somehow never quite made it to maemo.org/packages--only one subpackage fully appeared, the others never did. I tried over & over to merely upload the thing. It has failed in the builder a number of times though through no problem in the packaging just because something else was going on with the builder. With a functioning system I would have uploaded far more packages by now. /me trying again...

-Jeff