maemo.org - Talk

maemo.org - Talk (https://talk.maemo.org/index.php)
-   General (https://talk.maemo.org/forumdisplay.php?f=7)
-   -   Repositories down until Monday 18.1.2010 (https://talk.maemo.org/showthread.php?t=41023)

jscribble 2010-01-18 01:04

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by geneven (Post 478954)
It's possible that so many people are using them they are being overloaded. Anyway, people, it's time for patience. All together ommmmmh.

And pray that none of the people who are going to bring back the repositories are doing something else on Monday's US holiday.

Yeah, that's pretty much a bank holiday. Almost everyone else will be on the job. Until this is fixed, my n810 is basically a browser. Really hoping someone clocks in at the provider. Hopefully someone smart and efficient, because I NEED MY FIX! I'm getting the shakes already!

swagger 2010-01-18 01:18

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by geneven (Post 478954)
It's possible that so many people are using them they are being overloaded. Anyway, people, it's time for patience. All together ommmmmh.

And pray that none of the people who are going to bring back the repositories are doing something else on Monday's US holiday.

two of them are up now but extras devel mirror is down

jebba 2010-01-18 01:59

Re: Repositories down until Monday 18.1.2010
 
Mirror info here:

http://espejo.freemoe.org/

Quote:

Originally Posted by tekojo (Post 477928)
Just got off the phone with the ISP help desk.
The fix goes to tomorrow morning.

Until that repositories and mailing lists are down due to a completely dead NFS mount. :(

This boggles the mind. Even small ISPs have contracts where worst case scenario someone gets on a plane to hand deliver parts. How the hell can something as huge as Akamai be down for a weekend?

I've been in these data centers that these huge players use. They are not a "no one's home for the weekend" kind of place. Here's a description of one I visited a few years back in San Jose: you get in an elevator with no buttons. Security controls the elevator remotely and shoots you up to the data center entrance. Elevator opens, and you are in a secured area. There you get checked through. They have entire floors which just do power, with batteries, generators, etc. If power goes out, diesel trucks get sent (premeptively) from two distinct routes to bring diesel *just in case*. Worst case scenario, the building has a helipad for diesel to be delivered just to keep the place running.

The nameservers that freemoe.org is using is from an ISP that didn't go down for a minute throughout all of the Katrina hurricane disaster and they were in downtown New Orleans.

These data centers are *prepared*. Akamai is one of the biggest (e.g. they do such sites as CNN).

Akamai "gone for the weekend"? Hmmmmmmm. They can't bring up a simple NFS mount??? Hmmmm.

Quote:

Originally Posted by tekojo (Post 477928)
P.P.S. I know a couple of people have set up mirrors of the repos, so if you have just flashed your device and want the software today, please see the server move thread for people who might have a mirror.

I know of my http://espejo.freemoe.org one, but of no other. If someone would point me to one I'd be happy to include it on my site. We obviously need more mirrors, especially for weekends.

Quote:

Originally Posted by swagger (Post 478972)
two of them are up now but extras devel mirror is down

This makes no sense as they are all fed from the same servers. Extras-devel is simply another directory on the server. It's not a separate server. Anyway, all three servers are pushing .debs, you can see the (rough) stats here:
http://mas.freemoe.org/cgi-bin/anlgform.cgi
http://masymas.freemoe.org/cgi-bin/anlgform.cgi
http://masymasymas.freemoe.org/cgi-bin/anlgform.cgi

Sadavyk 2010-01-18 02:14

Re: Repositories down until Monday 18.1.2010
 
i hope everything goes smoothlty it would suck if the clock hit 3:00pm and it's still not working

F2thaK 2010-01-18 02:18

Re: Repositories down until Monday 18.1.2010
 
since when has nokia been on time?!?

KevinSass 2010-01-18 02:21

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by f2thak (Post 478041)
these worked for me





Extras Mirror
http://espejo.freemoe.org/repository.maemo.org/extras/
fremantle
free


xtras Testing Mirror
http://espejo.freemoe.org/repository...xtras-testing/
fremantle
free


Extras Development Mirror
http://espejo.freemoe.org/repository.../extras-devel/
fremantle
free


Tools Mirror
http://espejo.freemoe.org/repository.maemo.org/
fremantle/tools
free

It still dont work for me.. When i press download in the app manager menu i can only choose desktop and then there's only 2 programs.. :(

jscribble 2010-01-18 02:33

Re: Repositories down until Monday 18.1.2010
 
Am I mistaken, or are there no mirrors for anything but Maemo 5?

real_per 2010-01-18 02:36

Re: Repositories down until Monday 18.1.2010
 
I have to say, that this breakdown really shatters my view on Nokia and their commitment to Maemo.
Is this a hobby project, for people to be messing around with, or it this supposed to be a serious work device, that the £500 price tag suggests ?

I have to say that i completely boggles the mind, that the main App store, for Nokias newly introduced big mobile OS is allowed to be down for a whole weekend, a time where most users have the time to play with their devices. Are we supposed to believe that when something goes down at Nokia, they simply phone up the ISP's helpdesk, like some average joe, and gets told "It'll be up on monday", and even worse, that they'll just accept it ?! I mean, wtf ?!

From what I've seen and heard from Nokia before and after launch, Maemo was portrayed as this "next big thing" from Nokia, big the way things are run, it seems more like a garage project, runned by hobbyists... But for someone who bought into Nokias marketing, this just feels like another big slap in the face.

GeneralAntilles 2010-01-18 02:37

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by jebba (Post 479008)
This boggles the mind. Even small ISPs have contracts where worst case scenario someone gets on a plane to hand deliver parts. How the hell can something as huge as Akamai be down for a weekend?

It's not Akamai that's down, it's the (Finnish?) ISP that hosts the primary machines for maemo.org.

jebba 2010-01-18 02:48

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by jscribble (Post 479059)
Am I mistaken, or are there no mirrors for anything but Maemo 5?

The espejo.freemoe.org mirrors are just Maemo 5. I'm not aware of other mirrors.

jebba 2010-01-18 02:52

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by real_per (Post 479064)
From what I've seen and heard from Nokia before and after launch, Maemo was portrayed as this "next big thing" from Nokia, big the way things are run, it seems more like a garage project, runned by hobbyists...

Uh, hobbyists know enough to use mirrors. I bet of the top 500 distros on distrowatch 99% of them have mirrors. Don't dis hobbyist projects, they run much better than this! This isn't just an isolated weekend incident either, but going on for months, as indicated by this bug:
https://bugs.maemo.org/show_bug.cgi?id=5818

For my part, I got my N900 at the beginning of December and there have been continuous outages since then, often extended due to weekends or holidays. This boggles my mind too. It's run very very poorly. I've never seen it so bad in any project, hobbbyist or professional.

jscribble 2010-01-18 02:53

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by jebba (Post 479071)
The espejo.freemoe.org mirrors are just Maemo 5. I'm not aware of other mirrors.

Well, I thank you for those that benefit from those mirrors. This is exactly what makes this community great. It would be great if we could have a contingency plan in place going forward, but I don't think I could be of much help in doing so unfortunately.

jebba 2010-01-18 02:57

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by GeneralAntilles (Post 479065)
It's not Akamai that's down, it's the (Finnish?) ISP that hosts the primary machines for maemo.org.

Ah, well they should be fired and servers moved immediately, if the above story about just getting the help desk and all that is true.

wmarone 2010-01-18 02:59

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by real_per (Post 479064)
Is this a hobby project, for people to be messing around with, or it this supposed to be a serious work device, that the £500 price tag suggests ?

Both, IMO. It's quite capable of that,

Quote:

I have to say that i completely boggles the mind, that the main App store, for Nokias newly introduced big mobile OS is allowed to be down for a whole weekend
To be fair, the Ovi store is still up. What's down are all the community repositories that the Application Manager waves its hands and sends you on your merry way.

Quote:

it seems more like a garage project, runned by hobbyists... But for someone who bought into Nokias marketing, this just feels like another big slap in the face.
Maemo has always been, and hopefully always will be, a project that takes the user community into consideration more than any other. Instead of being given a device and told "You can do X, Y, and Z" and to do more you have to perform contortions (jailbreaking/rooting,) we have a device that's totally open from the get-go. I can tolerate a weekend of downtime if that's what it takes to ramp up the community available resources to handle all the users.

... I'd also pitch a fit at the ISP, and work on getting some good mirrors going like Jebba suggested. If my upstream were better than Comcast provided, I'd host one.

jscribble 2010-01-18 03:02

Re: Repositories down until Monday 18.1.2010
 
Not to discredit any of the points made, but this is essentially a community-run service. Granted, there are some bigger players that have much more control over decisions and actions such as the ones that led us to this, and I certainly feel for anyone who has dealt with this already on their N900, as it is so new. I have been running maemo since shortly after the release of the 770, and there have been a few problems, but overall, I have been amazed at what both maemo.org, and the community continue to pull off.

This is not apple, and I love that. Nokia makes great devices, and with the tablets, they have thrown the reins openly to the community.

Have faith, repos will be back, and better than ever.

Warpt1 2010-01-18 03:03

Re: Repositories down until Monday 18.1.2010
 
If my DataCenter had an outage like this: We would all be layed off.
That said...I just lost power at my home for almost three hours.:mad:

Now back to our regularly scheduled programming

Nathan 2010-01-18 03:07

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by tekojo (Post 477928)
Just got off the phone with the ISP help desk. The fix goes to tomorrow morning.
Until that repositories and mailing lists are down due to a completely dead NFS mount. :(

Can I ask why we have the maemo servers at some two-bit host, that doesn't appear to have a clue at what they are doing?

The host I host my three dedicated machines at -- has people at all their facilities 24/7. Multiple redundant power & internet connections -- so if one set goes down the center is still fully functional. My "down" time for the last few years has been only pretty much because of my own stupidity. ;-)

I can get them to not only to just physically "reboot" my servers at any point-- but to do technical work at any time at night and weekends.

Their is no reason why these machines should be still down at a decently classed host.

Nathan

F2thaK 2010-01-18 03:09

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by jebba (Post 479071)
The espejo.freemoe.org mirrors are just Maemo 5. I'm not aware of other mirrors.

the mirrors above worked for me the other day, havent tried lately

i found everything on there....

jakiman 2010-01-18 03:29

Re: Repositories down until Monday 18.1.2010
 
Yeah, really stange how it cannot be fixed over the weekend for such a major outage. I support clustering application servers with the biggest of companies and it's either "bring it up now" or "work non-stop till it's up". Ah well, I'm glad there are mirrors and that we can still install .deb files directly if need be. =)

swagger 2010-01-18 03:56

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by jebba (Post 479008)
M



This makes no sense as they are all fed from the same servers. Extras-devel is simply another directory on the server. It's not a separate server. Anyway, all three servers are pushing .debs, you can see the (rough) stats here:
http://mas.freemoe.org/cgi-bin/anlgform.cgi
http://masymas.freemoe.org/cgi-bin/anlgform.cgi
http://masymasymas.freemoe.org/cgi-bin/anlgform.cgi

i disabled all the mirrors cept for devel and only got access to two apps the i dsabled devel and enabled the other two and got access to alot more

jebba 2010-01-18 04:03

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by jscribble (Post 479097)
Not to discredit any of the points made, but this is essentially a community-run service.

This is *not* a community run service. As far as I can tell, everyone that has access to the servers is paid directly or indirectly (as consultants or via other companies) by Nokia. The "community" has no access to the servers themselves, nor even the most basic information about them. The community has no access to logs, how many hits, or anything whatsoever. In fact, who is in charge of what is very vague as well--it could perhaps be pieced together through tidbits left here and there, but it certainly isn't clear.

If it were a community run service, we'd be able to get in there and get some things done. Instead we have to wait for the crumbs that Nokia throws us. Compare this to Debian, which is a *real* community run distro. Or even to Fedora, which is corporate sponsored, but has huge community involvement.

Quote:

Originally Posted by jscribble (Post 479097)
Have faith, repos will be back, and better than ever.

Check this out:
https://fedoraproject.org/wiki/Outag...astructure_SOP
https://fedoraproject.org/wiki/Categ...structure_SOPs

Think Nokia will do something like that? We'll just get one way blog posts from Ari Jaaksi or something talking about how great things are.

-Jeff

jscribble 2010-01-18 04:12

Re: Repositories down until Monday 18.1.2010
 
Jeff, you're absolutely right. We do have community involvement, but not at any level which gives us any ability to fix this issue with the servers. However, we can set up our own mirrors and backup plan, which maemo.org and nokia welcome. I'm all for doing anything I can to help set up full mirrors, or anything else that could ease the pain of future issues. I have a limited set of skills, but I would love a chance to help.

Sadavyk 2010-01-18 04:15

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by f2thak (Post 479037)
since when has nokia been on time?!?

if i'm not mistaken Nokia isn't maemo.org
nokia is the one with the updates maemo.org are the guys doing all the other work as far as testing and Repositories with the add ons like IM and others

thefoxtrot 2010-01-18 04:58

Re: Repositories down until Monday 18.1.2010
 
It doesn't appear that Nokia even cares. This is their prime, new device and I had to search forums to find out that on Monday, their device will actually re-gain access to the limited number of apps. I hope they do something to make up for this. Fingers crossed for a stash of new apps when Ovi Store exits beta stage. :)

nashith 2010-01-18 05:25

Re: Repositories down until Monday 18.1.2010
 
I would gladly pay a monthly donation for a real community run server mirroring everything, I often hear issues about slow build servers too. I am pretty sure others would support such an undertaking.

jjx 2010-01-18 05:26

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by thefoxtrot (Post 479194)
It doesn't appear that Nokia even cares. This is their prime, new device and I had to search forums to find out that on Monday, their device will actually re-gain access back to the limited number of apps. I hope they do something to make up for this. Fingers crossed for a stash of new apps when Ovi exits beta stage. :)

That is not accurate.

Nokia's servers haven't gone down, and apps provided on Nokia servers including on Ovi are still available.

Only the maemo.org repositories are down. Do you think Apple cares when non-Apple sites go down, or Google cares when non-Google Android sites go down?

That said, maemo.org is special. It is a rather grey line between Nokia and maemo.org.

Nearly all the useful third-party apps are on maemo.org. Nokia clearly relies on it for their (almost non-existent) marketing.

Nokia pays for maemo.org servers, and maemo.org repos are installed by default. But it's clearly not run by Nokia, nor run the way Nokia would run commercial servers. That comes with pros and cons. A pro being all the good community development and involvement resources that are hosted on maemo.org. A con being... what we saw this weekend.

geneven 2010-01-18 05:56

Re: Repositories down until Monday 18.1.2010
 
Whoever is or isn't at fault, it can't be good for Nokia or Maemo to have this kind of outage when its ace card is an enthusiastic bunch of users who are not rendered more enthusiastic by repositories going down for days.

PolarFox 2010-01-18 06:30

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by f2thak (Post 479037)
since when has nokia been on time?!?

I would give this a Score: 5, Funny ...
I have been annoyed by the repository shortage as much as the next guy. But this just wasn't a problem to warrant paying someone the money that would have been needed to get someone to work on weekend. With Hockey season on and all! :))

This is just IMHO. I'm not working with Nokia or have any real affiliation with maemo.org.. :)

RouterRebooter 2010-01-18 06:37

Re: Repositories down until Monday 18.1.2010
 
Are the servers back online ?
Can't check it up at the moment. Thanks.

DarkFulgore 2010-01-18 06:53

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by RouterRebooter (Post 479271)
Are the servers back online ?
Can't check it up at the moment. Thanks.

not working for me as of yet!!!!

Arabian 2010-01-18 06:55

Re: Repositories down until Monday 18.1.2010
 
if maemo servers use FreeBSD OS, they would be more reliable

totololo 2010-01-18 07:14

Re: Repositories down until Monday 18.1.2010
 
It's back online !!!

arnoldthebat 2010-01-18 07:14

Re: Repositories down until Monday 18.1.2010
 
PingdomAlert UP:
maemo (repository.maemo.org) is UP again at 18/01/2010 07:10:06, after 13h 15m of downtime.


Dont take this as a guarantee of course since there may be teething issues now its back online...

arnoldthebat 2010-01-18 07:17

Re: Repositories down until Monday 18.1.2010
 
http://www.pingdom.com/reports/rjyv7sz3c2t6/ for public uptime on the repos...

By the way, a single website is free from Pingdom. Happy to leave the monitoring for now, but I cannot guarantee I will need it myself later....

Also bear in mind the only check I am placing is the root of the repo. no additional functional checks are placed....

DNS looks like it's propogating so be patient!!

18/01/2010 07:15:06 Up 224 ms London 2, UK
18/01/2010 07:10:06 Up 525 ms Dallas 5, TX
18/01/2010 07:05:05 Down Unknown target Dallas 6, TX
18/01/2010 07:00:05 Down Unknown target Los Angeles, CA
18/01/2010 06:55:05 Down Unknown target Frankfurt, Germany
18/01/2010 06:50:05 Down Unknown target Atlanta, Georgia
18/01/2010 06:45:05 Down Unknown target New York, NY
18/01/2010 06:40:05 Down Unknown target Chicago, IL

KD Epic 2010-01-18 07:18

Re: Repositories down until Monday 18.1.2010
 
Still not working for me in Texas, reset my phone just in case -- still nothing.

Arabian 2010-01-18 07:23

Repositories up now.
 
it's up for me now.

KD Epic 2010-01-18 07:26

Re: Repositories down until Monday 18.1.2010
 
Umm... So what do I do now? lol... If one person can connect shouldn't I be able to as well??

arnoldthebat 2010-01-18 07:31

Re: Repositories down until Monday 18.1.2010
 
Quote:

Originally Posted by KD Epic (Post 479317)
Umm... So what do I do now? lol... If one person can connect shouldn't I be able to as well??

Nope, the vagueness of the internet prevent such things!!:D Just give it a couple of hours...

KD Epic 2010-01-18 07:34

Re: Repositories down until Monday 18.1.2010
 
I'll take your word for it bud ;-) ran an apt-get update and it didn't help. Will try and update y'all in the morning.

dominik 2010-01-18 07:35

Re: Repositories up now.
 
Quote:

Originally Posted by Arabian (Post 479315)
it's up for me now.

For it works as well since a few minutes ago.


All times are GMT. The time now is 13:05.

vBulletin® Version 3.8.8