Re: R&D Mode Control [CLI version]
 

Okay, this is weird. It is definitely the same exact binary, segfaulting on the r&d-virgin N900 at bootup, but not on the development one where it no longer does.

So there is still more work do be done, I am missing something somewhere. But for now, I am out. Sleep. Needs to happen. Besides that, I updated the source once again in case the above post didn't make that obvious. I cleaned up the code that handles the R&D-virgin N900s correctly (there was a cal_finish() call left in one of those code paths from years ago, and I fixed the messages it provided to be more meaningful thanks to me getting new understanding of what they mean myself, and made it actually properly distinguish between the 'R&D virgin' error and the other errors that might be returned - I also made it use the cal.h defined macros for the return values of cal_read_block instead of just going by the numerical value as the old code did..

Edit: P.S. reinob, etc, anyone still left with an N900 that hasn't been R&D mode turned-on, ever?

Re: R&D Mode Control [CLI version]
 

Sure, me, make myself sleep when I have a problem like this.

I know what it is, damn simple: the /dev/shm directory needs to exist before you can mount on it. Apparently, by default, that early in the boot it doesn't.

The reason I didn't notice is that 2 days ago on my dev. N900 I mkdir'ed that directory from inside the @boot console. Since then it hasn't disappeared. But I guess normally is does go away between reboots? Really odd to me. I am done digging for a few days. The solution is right there, but I don't want to just add mkdir and rmdir calls to the code without knowing if that's the 'right way'. I want to know how/why /dev/shm normally gets created and then disappears - and why manually making it at book kept it safe from getting killed through the reboots. Is it some sort of virtual directory that isn't actually written to the flash, normally? If so, I'd rather figure out how to do it properly that way and not chip away at the NAND flash with a a pair of mkdir/rmdir calls every time rdmod runs.

Okay, NOW sleep.

Re: R&D Mode Control [CLI version]
 

I believe my 1st N900 never been in R&D-mode, but I am not sure. But hey, it's worth the try right?

Re: R&D Mode Control [CLI version]
 

@Mentalist Traceur,

/dev is actually not stored in the filesystem, but also a tmpfs.
Going back to /etc/init.d/rcS (I have to someday post details about the whole /etc/init.d in the Maemo5 boot process thread, which only dealt with the upstart part).

it begins by doing start_udev(), which does
In prepare_start_udev the magic happens:
(tmpfs_size = 1MB unless set in /etc/udev/udev.conf, which DOES set it to 10MB)
(note that *my* /dev currently uses up 88.0K, so I'll see if I can modify that back to 1MB (or even less) without breaking anything, and hopefully giving some RAM back to Maemo :)

IN SHORT:
if you run your program during your early boot console, rcS has not run yet, so udev is not running yet and /dev is merely a directory of your rootfs, where you can store your p0rn if wanted (and your semaphores :)

Once rcS runs whatever you had under /dev is not visible anymore.

[EDIT] so yes, feel free to mkdir -p /dev/shm before rcS runs, this way nothing should segfault

[EDIT**2] I just modified /etc/udev/udev.conf and changed the size to "512k". Maemo booted fine (this is my "production" N900). Obviously I cannot tell if less actual RAM is being used or not, but I internally feel better knowing that less RAM-management is taking place :)

Re: R&D Mode Control [CLI version]
 

Hi again,

I have just had a look at @freemangordon's implementation of libcal, and it's a bit weird.

First of all: cal_read_block() and cal_write_block() do *internally* cal_init_ and cal_finish_ (which do the actual set-up and free()ing of the buffers).

So from the point of view of a user application you do not need to call cal_init() or cal_finish(), as they are nops.

The first thing your program does is to read the r&d_mode block. After the call to call_read_block the only thing that is valid is tmp_ptr and len, i.e. cal_s has been freed.

So towards the end of the program, when you actually call cal_write_block (if the mode string is different), cal_s is undefined, but it's OK because it will be cal_init_()'ed again.

If I read correctly, when you want to clear r&d_mode (*rd_mode_string == '\0')) you use sizeof(rd_mode_string) as the length. However the length should be zero. sizeof() gives the actual size of rd_mode_string, which is the same as sizeof(char *) which is namely 4 (for a 32-bit computer). You actually want a zero in there.

Basically you want to drop the "if(*rd_mode_string == '\0') and just use the normal path, where you use strlen(rd_mode_string) (which will be zero when rd_mode_string is empty anyway), however you need to get rid of the "+1" (why is it there anyway?, that's going to cause random segmentation faults).

But cal_write_block_() has a problem when len = 0, because it does malloc(len), which when len is zero may return NULL or a pointer. If NULL however, the function exits with an error (CAL_ERROR). This is something for @freemangordon to fix, unless it is required that when writing len > 0, meaning there should be another function to erase a CAL block.

I've printed cal.c and will read it when I'm on the train. Maybe I spot something.

Re: R&D Mode Control [CLI version]
 

It is not "my implementation" but a RE, so whathever weirdness you found, most probably it is Nokia legacy :) . I even fixed some obvious bugs, but not all of them.

I disagree, those are part of the API and the implementation may change in the future versions.

Correct. And this comes from the Nokia (I'll double-check with IDA again). I think libcal was never meant to be used for block erasure, that is why len == 0 is not checked for. On the other hand it is possible that malloc in our glibc to return non-NULL for zero-sized allocation, so there is no (real)problem. However, len == 0 case should be processed correctly(it is going in my todo list, along with packaging).

Re: R&D Mode Control [CLI version]
 

Ah OK. I didn't know it was RE. I thought it was more "clean-room". But yeah, looking at the mess cal.c is, it lacks the elegance one would expect from carefully written code :) [ shame on you, Nokia :]

Repeat after me: WE are the API :)

Thanks. By the way, I just verified (on a N900 having R&D previously set) that setting and clearing a flag using Mentalist Traceur's program (I used serial-console as example) works fine. Double-checked with cal-tool and everything looks OK.

Re: R&D Mode Control [CLI version]
 

I've compiled libcal with -DDEBUG and run "rd -q". I've attached the output, in case somebody is interested.

I find interesting to see which blocks are stored in the cal (note: libcal is effectively the implementation of a file system).

Will try to spend some time figuring out the content of those blocks.

Re: R&D Mode Control [CLI version]
 

- TL;DR -
Source in front page updated; deemed stable and fully working. Further work - NOT in the near future, will be refactoring of the code, and some logic improvements, namely splitting the parsing of the command line parameters and the actual building of the rd_mode_string. Thanks to everyone who commented as your comments are all either helpful or appreciated. I disagreed with reinob on one point (the one about the '+1' after the strlen when calling cal_write_block()).
- END TL;DR -

None of the above future work hopes really matters as far as normal functionality is concerned. I.e. the logic improvements only matter for performance if you're an efficiency zealot (so like me): the rd_mode_string is rebuilt way more than it needs to be.
'rdmod -e -s no-omap-wd no-ext-wd' will invoke 3 realloc() calls. So it is better to abstractly to through the arguments and build a compact set of what the arguments want, which is just booleans/ints/chars/bitfields/whatever, not wild dynamic memory thrashing as with the current way, then calculate the final string size, malloc once, compose the string - just once - then write. But it's not like the way it is currently is even remotely noticeable as slow though...

Well, if you ran my current first-post source code (well, the compiled binary thereof obviously), it would tell you when you ran stuff (something like "R&D area blank... maybe you never set R&D mode." but worded better), if you never did have it in R&D mode. :) At this point I tested it on my R&D-virgin N900 enough to know it works though, so further testers aren't as desired as I was hoping for when I made that post (I think I was just longing for sleep and kinda implying "someone go test this so I don't have to"...but then I stayed up and tested it anyway last night, and by the time I've written this post I've gone through even more final testing).

Yes plz. Also, this was helpful. I'm snipping the rest, but I REALLY need to get into the habit of checking those scripts. I did earlier today when I came back to digging into it about [strike]two[strike] nine hours ago: It proved helpful in other ways.

Yeah, this was definitely helpful in starting me down the right path. As you say, once the new tmpfs /dev is mounted over the root ubifs or whatever /dev, all the relevant devices nodes disappear.

I wanted to go a 'cleaner' route than simply mkdir-ing a /dev/shm, let alone mkdir-ing it and removing it every time (like a ninja - not leaving any traces in your rootfs is I don't have to - you'll never know I was there - or that rdmod was there in this case).

This presented the next problem: I had to mount the temporary /dev like the N900 did at boot, then mkdir on that. But now the device that cal resides on, /dev/mtd1, disappeared and needed to be somehow brought into the new /dev - fortunately, mknod is a thing. So that's been my process today - getting the code to check if /dev is mounted as a tmpfs filesystem, and if it's not, mount one, then mkdir /dev/shm and mknod /dev/mtd1, and go from there.

I've got that all working fine now, although I've seemingly created a much more scary bug just now - I seem to have somehow straight up wiped the cal area (lock code reset to 12345, r&d settings being seen as R&D-virgin after every write... Woot broken code.) Not sure what the hell I managed to do there, but that's not exactly a good thing. I'll be debugging that later as time permits lol. As typical with bugs in this program, they entail things I wouldn't even know where to begin the investigation of, lol. Correction, before I even made the post (this post has been open for hours... every hour or so I come back to it). I seemingly have fixed it, without fully figuring out the cause. Possible contributing factors:
I was 'fusing' the rd_mode_string and rd_mode_current pointers (setting them to equal each other) in a spot. I don't think this could've initiated the problem because it only effected the R&D-virgin code path.
I was failing to properly unmount/unwind the stacking of tmpfs that I was creating - I accidentally left a copasted "umount" statement in without actually editing the parameter so it was trying to umount /dev/shm twice. This was unlikely to be the issue because that problem doesn't surface until you try to do other normal tasks (like continue booting) - then the kernel flips a **** and panics because /dev is suddenly empty. Really bad that I slipped up like that (but that's why I test before release, to catch dumb **** like that), but the fix was trival and I don't think that was what was causing the issue with the cal area self-wipe either. *Shrug*

Anyway, I fixed those things, double checked that everything else was seemingly okay, recompiled and the issue seemingly went away, so I don't know what exactly it was, but it was weird and potentially serious if it ever came up again. In fact, I just enabled R&D mode for the first time on my main use-as-phone N900, from inside my boot shell, no problems, WOOOOOOOOOO **** yes.

Good to know. I'm not sure that the RAM management is any smaller - I thought tmpfs dynamically resized itself anyway, with the kernel only pushing things out when it needed the room? (As opposed to ramfs which reserves the whole chunk... maybe? I have this VERY hazy memory to that effect from somewhere...)

This is all very informative, thanks for taking the time to dig deeper. For the time being I'll admit I've concerned myself only with the parts of libcal that weren't working on one way or another, so it's nice to see a general critical look at some of it.

You're absolutely right about the sizeof() thing. I've got no idea what I was thinking when I did that - it obviously harkens back to 2010-2011 when I didn't really know what sizeof() truly meant. Good catch. I just tested it with 0 and it seems to work just fine.

Actually, here you're off - there is a very specific reason for the +1: what's a C-string depend on? Terminating null. What's strlen() return? Length without the terminating null. What happens if we write only strlen chars? We omit the terminating null pointer. Now, I went ahead and looked through the code path part of the way into cal_write_block(), into set_header(), and it looks like the CAL block structure DOES store the length (so it's not relying on a null byte to tell it when the block ends - great).

But consider this: what happens if I don't write in the terminating null. When my code goes to load the string, it will get a len that's just long enough to fit the string content, without the c-string null terminator. It will also get a void * containing said string, which it then proceeds to cast to char * and treat as a string.

The outbound end would have to be tweaked to get the string out of cal without a terminating null, which means more work to reassign it to a new char array one size bigger, to fit the null byte. That or the string.h functions become useless without further work.

If I recall correctly the process that went in to the early coding on rdmod 2-3 years ago, then that + 1 is the precisely the result of originally using strlen() by itself, and consequently reading random garbage past the end of the written-in string (and consequently writing trailing garbage back in some of the time). In fact, I'll even put it this way (in a friendly manner mind you, I'm not angry or lecturing here): I challenge you to remove the +1 and see if any segfaults (of which I presently see none, and I assure you I test reasonably thoroughly) disappear, and more importantly, if you see any segfaults or other issues appear.

Now that said what WOULD be a good reason to take the approach of not writing in the c-string terminating null, is if you set R&D mode settings on using Nokia's Flasher, then try to read it with R&D mode control and that gives problems (this being completely not caused by the +1 but by the fact that the strings coming out would have no terminating null for my code then (or the standard c-string functions thereof) to stop at), then I will, for the sake of compatibility and consistency, write the strings without the terminating null to the cal block and compensate for the lack thereof in my own code when pulling the string out.

This to me seems to present a stronger reason in favor of writing strlen()+1, even in the empty case. Which is what the latest code does. Eliminates the separate code path (like you suggested) but keeps the + 1.

Hope that went well for you. I'm always happy to learn so feel free to post and other findings you find interesting.

I think libcal does have a call or two for block erasure, I could've sworn I saw functions titled something like that in there.

I actually would love it if some more formalized effort occurred to standardize some changes/differences/features where appropriate - but I'd want it to actually have some sort of formal way of reaching consensus - random developers simply deciding "this package will do /this/" is fine in general but helps make things less cohesive when it's fairly system-integrated libs we're talking about.

Oh yes absolutely. rdmod can and will erase your R&D setting into being a blank string, and otherwise clearing just individual flags, just fine. That functionality was around and fairly vetted for a pretty long time now. But that said, it's always nice to have more confirmation/testing.

That's a very generous title for what is a basic linked list with some metadata and actual data per node/link. But yeah, in a sense it is. It's worth noting that it is clearly sequential in nature, whereas a filesystem can typically locate files at random pretty easily and efficiently. Filesystems usually don't need to potentially traverse their entire contents to find what they are looking for - libcal may well have to.

Now that libcal is open I may well just write a program to simply allow perusing the data and viewing the data of each block. Unless someone's already done it.

...and if I have time. Given how long it took rdmod to reach the full functionality I wanted out of it to begin with, there's no guarantee I will have that done any time soon - and in fact, I will explicitly say I will do my best not to for a while because I just used all of my free time these last three days or so doing just rdmod work and I have to make sure everything else is taken care of.

Re: R&D Mode Control [CLI version]
 

Would it be too much to assume that CAL in N9 is of similar structure as CAL on N900...?
Even as it's locked in Open Mode on N9 it would still be possible to read it, just for intrest of the content. :D

Re: R&D Mode Control [CLI version]
 

I have no idea, but if I by some unlikely chance have the spare time or motivation, I might look into it. Honestly, I am not the most qualified - I wasn't the one who managed to reverse engineer libcal - my program is just the userspace component for going through libcal to change your R&D mode settings (though with an open source libcal we may be able to just control all of the cal area. I guess a general conceptual pre-announce - there will probably eventually be either several other '*mod' programs or one 'calmod' or something, allowing changes at will to all of the CAL 'areas').

Re: R&D Mode Control [CLI version]
 

Exactly. Your using of the +1 is a kind of workaround for using string functions when libcal actually uses "Pascal-type" strings, i.e. a block and a length, and memcpy instead of strcpy, etc.

If you do rewrite your rd program you might want to go that route, which would sort of fit better with libcal. But this is (likely) a very academic issue. At least R&D mode is essentially treated like a (C-)string.

But if you consider expanding your program to other regions/blocks of cal (think "bme", "phone-info", "wlan-tx-cost3_0", "fmtx_pwl", "lock_code", "lock_period", "lock_enable", etc.) then you'll have to make sure that you read/write exactly as expected by the library (and any other client programs making use of the library..)

Cheers, and congratulations for making it possible to manipulate R&D mode from the recovery console. This is actually seriously cool!

Re: R&D Mode Control [CLI version]
 

You just wrote a good definition of a file system :)

Well, that's the thing of flash-memory. You cannot just overwrite something. So you can only write to blocks which have never been written to, hence the appending of data and using a version number to keep track of the latest version. Only when the space gets full you need to erase an entire erase-block (in our case this is 128Kb, i.e. there are THREE erase-blocks in /dev/mtd1), which can then be written to.

The odd thing about libcal is that it implements the whole fs from the highest level (open "r&d_mode" file in "user" directory) to the lowest level (write a CRC32'ed block of data to
an available NAND block).

Not that I know. There is this "calvaria" program which supposedly does that, but I haven't seen it yet.

Your work is much appreciated!

Re: R&D Mode Control [CLI version]
 

What I meant is that it remained to be seen (at the time I made that comment) if, in spite of the libcal code being made to support abstract data with a given length parameter, the actual data stored was perhaps in C-string format. I.e. libcal supports storing arbitrary data of a specific size (up to a maximum if I understood the code correctly), but one type of such data it could store could well have been C-style strings - and in the case of the R&D mode area, I thought that would actually kinda make sense. Which is why I was saying that one way to figure that out would be to set the R&D mode settings with Flasher then parse the resultant data retrieved from the latest R&D mode block and see.

But I think you're right, anyway, it's not stored in C-string style. I went and ran 'strings' on cal-tool and it calls memcpy (I think this is sound, as my understanding is the reason for such strings being within binary files is that the symbols for calling functions in dynamically loaded libraries use the actual function names used). I still want to do the test eventually with Flasher (it's possible cal-tool opts for one function using memcpy to get the data for any of the blocks that it's supposed to read, or uses it elsewhere), but I think you're absolutely right, the strings are probably stored without the terminating null and the application logic around libcal is supposed to account for the lack of the terminating null if that's the format it needs it in. That would make much more sense - more portable that way, allowing C++ strings or other languages to use libcal and the contents of CAL without having to account for the C-specific string implementation of using terminating nulls.

Actually screw the test, I went ahead and coded in the change and tested it over the course of like 5-10 minutes - it seems to work great. cal-tool -f and rdmod -p don't accidentally run 'too far' and read junk. Near as one can tell... I might as well go ahead and write a special program just to test, making a string "test hi" and then saving it with len passed in as 4. In the meantime, I discovered an interesting flaw (or feature, but doubtful) in cal-tool - if the last substring of the R&D area is not a valid flag, it will print a newline at the end of its input. So rdmod -w "master,shell" will causes cal-tool -f to print:
"master,shell
"
but rdmod -w "master,shell,no-omap-wd" causes cal-tool -f to print:
"master,shell,no-omap-wd"


(And yes, I would like "shell" to be a R&D mode CAL area flag - for specifying that you want a shell at boot or something.)

[another ten minutes later, most of that time being wasted on the actual text file manipulation and typing out the commands and stuff:] Works fine. Pass "test hi" as the data and 4 as the length into cal_write_block (which we knew would store just "test"), and cal-tool -f prints out "test". Funny enough, without a newline after it...

Doing more testing reveals that cal-tool -f does not print an extra newline if you instead do rdmod -w master,shell,poop - so something is special about that string length - rdmod -w master,ploop causes cal-tool -f to print an extra newline, but not say rdmod -w shell

In a similar vein, when the r&d mode string is 1 character long, cal-tool -f prints nothing.

Well whatever. Also, cal-tool has what I'd call a bug - cal-tool -d declares "Enabled" when the r&d string is at all non-empty, regardless of whether "master" is actually in the R&D string.

Source code on front page updated to not assume the string coming in/out needs to be c-style null-terminated.

Thank you very much. Yeah, admittedly the high of the achievement happened a day or so prior, when 'the end' was first in sight, but either way, it is indeed seriously cool, and I am glad I've gotten (with the help of several others, yourself included) to the point of being able to do this.

Also, your remarks about the file system stuff are informative and more importantly made me see how the things I already knew applied to the situation (wear-leveling on flash media for instance, combined with the fact that the nand chip on our N900 SoC doesn't implement its own wear leveling controller, leaving it up to the software to do it [which, as an aside, is how I think it should be done - let the kernel and sufficiently privileged users decide exactly how to wear-level and when to bypass it, don't make it literally hardwired] - I already deduced that the choice to not override existing blocks was made precisely to avoid digging a hole down in the same spot of the flash chip, but your remarks made me go 'well if not as a linked list, the how the frak else would it minimize repetitive writes in the same spots. Any more efficient to search arrangement that I can think of requires more edits of the underlying metadata that would have to actually stay in the same spot). So I stand corrected. It is indeed a file system.

Re: R&D Mode Control [CLI version]
 

Reading over my last post it occurs to me, I need to not do so much stream-of-consciousness when writing posts.

Re: R&D Mode Control [CLI version]
 

Actually, I rather like it.....

Re: R&D Mode Control [CLI version]
 

Interesting. Good to know.

Alright, so now that the rdmod code is stable and rather thoroughly tested, show of hands, who wants me to attach a compiled binary so they don't have to compile it themselves? (I might not get around to it for a day or two, but still, might as well ask now.)

Re: R&D Mode Control [CLI version]
 

I have at hand a compiled version of yesterday's code. I have now a meeting but after that I'll compile[*] today's version and send it over.
[*] I seriously recommend everyone to get a VPS with scratchbox and the Maemo SDK installed on it. Being able to compile just about anything (including kernels) on-the-go without worrying about space and/or speed and/or battery of your N900 is really an affordable luxury.

EDIT: I also like & enjoy your stream-of-conciousness-style of writing.

Re: R&D Mode Control [CLI version]
 

Please find attached (gzip compressed) rd, rd_dbg (rd compiled with -DDEBUG). As a bonus I've added calvaria (which dumps CAL in various ways).

Re: R&D Mode Control [CLI version]
 

Maybe it's time to put it into repos, too?

Re: R&D Mode Control [CLI version]
 

I'm not a repo-fan :), but I've requested permission to upload calvaria. Once I'm approved I might actually do that, so I get used to the whole maemo.org/autobuilder/repo mess.

But note that calvaria is only a single C file (calvaria.c), with no dependencies, and no plans of being updated [ although I might update it once I learn about the exact details of some blobs ]

Re: R&D Mode Control [CLI version]
 

Yeah, I'd say it's about that time finally.

Personally I call it 'rdmod' but matter of taste I guess. Just know that when I do put it in the repos, it'll be under that name. At any rate, thanks for throwing that up. Can you point me to where the calvaria source is that you found?

Really? Why not a fan? (Just whatever you do don't upload two versions of a package with the same reported package version. Last I did that it still had a *****ic bug in the autobuilder python code (idk if actual autobuilder, or technically separate, but it's part of that whole process), which jammed the autobuilder for everyone.)

If you want I can do the maintaining of it in the repo if you don't wish to deal with it (just add me as a maintainer once you get approval for it). I may end up updating it at that point as well, idk (Actually the approval is for upload rights overall, what you upload after doesn't have to match or be anything, so you could just not upload it. That said, if you want to be maintaining it, then it's less work for me which I'm not complaining about - just offering to take calvaria on as well if you wanted.)

Re: R&D Mode Control [CLI version]
 

P.S. I meant to make a joke like "but does yours come compiled with the -fPIC flag" @ reinob's posting of precompiled binaries, but ended up forgetting.

Re: R&D Mode Control [CLI version]
 

@Mentalist Traceur,

I like short names during development :), I'll rename it anyway from rd to rdmod :)
The source for calvaria (a single .c file) can be found.. wait:
https://dev.openwrt.org/browser/pack...src/calvaria.c (text form)
https://dev.openwrt.org/export/38692...src/calvaria.c (downloadable)

If you could take care of maintaining it -- much better, as I don't like the whole packaging/versioning/maintaining stuff. I don't use or like version numbers :)

Re: R&D Mode Control [CLI version]
 

what is this r and d mode ?

Re: R&D Mode Control [CLI version]
 

It's all about luring the dog out of the watch house.... and other stuff.

Re: R&D Mode Control [CLI version]
 

There's supposed to be a wiki page for it, but it seems that the URL is broken.

R&D mode ('Research and Development mode') is a feature built into the N900 for the sake of developers, but which is sometimes useful for other purposes.

Basically, there are a few fairly technical features that the N900 can turn on/off, and by default, the way it does that is that one of the early bootup scripts reads the R&D mode string from the CAL area of the N900 (the CAL area is a special partition/section of the small nand flash chip, on which the rootfs resides), and then the shell scripts see what, if any, R&D mode features to turn on depending on what, if any, flags it reads in the string.

By default, you can only change these settings with the Nokia Flasher utility, which has to run on another computer, and can't do the needed changes locally.

One of the things that turning R&D mode does by default (without actually toggling on any of the R&D flags), is that the N900 turns on a thing where the LEDs of the keyboard flicker in response to system activity - this was meant to let developers see if their apps were causing system activity in the background.


Another thing you can do is disable a few features on the N900 that will reboot the phone if they think they detect a problem (those are the watchdogs you might hear about occasionally), there's also some 'lifeguard' thing that does a similar thing. In rare cases being able to disable these things is good, though admittedly that can be rare.

Then there's some other R&D mode flags that deal with an STI console and disabling sleep on the USB port, etc.

Re: R&D Mode Control [CLI version]
 

libcal semaphore name fixed, debian packaging added (thanks Pali)

https://gitorious.org/community-ssu/...e361372ec8cc2:

Re: R&D Mode Control [CLI version]
 

I know, I know, but the sheer fun of having it in "updated" section of package manager, *if* some update happens, after all - even if I miss info inside thread, or whole thread, at all - is priceless.

Not to mention fun-saving in case of fresh install, without need for thread-and-download-url hunting - repo and name is enough. Every time I prepare N900 for someone, somewhere away from my link collections, and I start link-hunting for all repo-less but unskippable packages, I feel almost like windows user, back again too geeky.

/Estel

Re: R&D Mode Control [CLI version]
 

Well, that seemed to have fixed the prior issue (in a different way than how I did it, but alright), but now another thing is broken (easily worked around, but..):

CAL_ERROR_NOT_FOUND and CAL_OK defines should not have been moved into cal.c - they are return values of libcal functions and one of the points of having them defined in the header is that so code that uses libcal can be written to check for those defined macro values and not need digging in the cal.c file to find what the actual values are.

I am not sure what reasoning led Pali to think they ought to be private.

Besides that though, Pali taking the time to do something about it is appreciated. I would've gladly submitted a fix on gitorious when I got around to it, but I've been busy (I really shouldn't be taking the time to post right now nor should I have been opening this thread to check for new posts at that...for that matter I shouldn't have even been working on rdmod at all this week, but what can I say, I felt really compelled to.. still do).

One last note: Actually, on further glance, I have to lodge a complaint about the entire change made - I don't think several of the things that used to be in the header file should have been moved out of it, the 'struct cal' definition included. For starters, if I wanted to write a program that would allow manual perusing of the CAL area, I could no longer do it with the new version of the header, because the code having access to the struct cal layout is needed to be able to do that (at the minimum; access to some of those other internal functions is probably needed too and I'm not even sure if the compiler can look at just the forward declaration "struct cal;" with no further definition and even know everything it needs to know to work with a struct cal pointer correctly, and more importantly frankly I don't get the rationale for moving so many definitions into cal.c - sure the very limited interface we had from Nokia's cal.h made sense - because Nokia actually wanted to keep the whole thing locked down and access/control of it limited; but that makes no sense for the open source recode - in general making things private to the .c instead of public in the .h tends to mean you're making arbitrarily limiting assumptions about how someone might find it useful to use the functionality in your library; it makes /some/ sense as a minimal protection approach, i.e. if you want to bounds-check values that your code uses, but there's a lot less benefit to doing that in open source code especially of a low-level system library like this [i.e. now if I wanted to write said program I would have to simply copy-paste a good portion of cal.c code into my own code and that's just extra maintenance overhead]). Suffice it to say the update that just happened took away substantial functionality/flexibility that the previous version opened up, when the easiest fix to the problem I had initially reported was simply to add "#include <sys/stat.h>" to cal.h.

Re: R&D Mode Control [CLI version]
 

P.S. Off the top of my head, I think what Pali did was just make the open cal.h header conform to the Nokia-provided cal.h header. While I understand doing this 'automatically', I think in-depth further consideration (summarized in my little ramble in my prior post), suggests that we need and should not be striving to make our open cal.h as limiting as the closed one was.

Re: R&D Mode Control [CLI version]
 

Oh, the reason I had actually gotten on to the forum to make a post here in the first place (completely forgot when I saw the cal.h/cal.c update):

I am going to start using C99 features, so from now on (until/unless GCC finally supports C99 enough for the gnu99 dialect of C to be the default in your version of GCC), rdmod will need "--std=gnu99" to compile - or whatever it takes for your compiler of choice to compile in a C99-compatible manner.

Specifically, the reason I am doing this is to enable the use of the restrict keyword (I could use a less portable compiler extension that does the same thing even in the default gnu89 dialect, but I don't like using compiler-specific things), which allows the compiler to make some optimizations in the compiled end result. But since I am enabling it, I am also considering switching to inline declarations of the counter variable 'i' inside all of the loops, instead of declaring it once in main. I'd have to figure out which one executes faster, if the current way is faster then I'll keep it. That's later though. I really need to focus on other things and stop allowing myself to come back to this.

Re: R&D Mode Control [CLI version]
 

@Mentalist Traceur - please clone libcal on gitorious, make the changes you think should be done and make a merge request. That way we'll be able to keep discussion in one place and will include Pali in it.

Re: R&D Mode Control [CLI version]
 

@freemangordon. Absolutely the better way to do it, I just needed to (and still do) get around to doing this... I will try to soon now that I have a break between semesters.

Re: R&D Mode Control [CLI version]
 

So today I saved myself a lot of frustration and bootlooping with this, verifying that there are corner cases where you absolutely need to alter R&D mode flags from inside very early boot.

See, apparently, as per Pali's understanding of getbootstate here, when getbootstate detects the battery as being in certain states, it simply refuses to boot. I have very strong insults I want to hurl at whoever designed it like this because, over the years, I have had many occasions where it seems that this is the cause for my N900s not booting successfully on first try. Sometimes, it gets stuck in a very persistent failure-to-boot, such that I have to leave it plugged in overnight, and it seems that the tiny amount of charging it gets done in between boot loops (since it boots up from detecting the USB cord, but then almost immediately shuts down because of the whole getbootstate system being *****ic; rinse and repeat on and on), finally resets whatever state the battery hardware keeps internally.

Well, one of my N900s apparently decided to get stuck like this for over 24 hours - I guess it discharged overnight Thursday, was off when I woke up Friday, left it charging... nothing when I got back after a full work day. Left it charging all the way through the night until this morning. Nothing. Did some frustrated digging, found the above linked thread. Used rdmod to enable R&D mode, attempted to reboot once (or maybe twice, forgot already) more, and victory was mine, it booted. Of course, I was then able to use rdmod to disable R&D mode right back again, which was also nice, but not really the point of the story.

TL;DR: First time I've legitimately saved myself with this tool of mine, can't begin to describe how happy that makes me right now.

[edit]I always forget the word 'm0ron' gets censored by this forum's software...[/edit]

Re: R&D Mode Control [CLI version]
 

@Mentalist Traceur,

Nice :)

Just a piece of advice, to everyone: leave R&D turned on, always. You won't regret it.

Re: R&D Mode Control [CLI version]
 

As a bit of advice to people doing this, if you do, remember to edit, erm, I think it was /sbin/preinit, to disable the keyboard flashing - or not, if you like it, but battery life will suffer, probably noticeably. Otherwise in the stock configuration, all enabling of R&D mode causes the keyboard leds to flash whenever there's system activity while the screen is off, or something approximately like that. It's not a major drain on a stock N900 lying around idling for days (my stock N900 with R&D mode on being the only non-stock thing has better idle battery life with those keyboard flashes than my actual heavy-use N900s get lying around equally unused, from what I think I've noticed (but I haven't timed it to be absolutely sure)), but it's probably a major drain on an N900 with a bunch of extra stuff installed that's getting used regularly.

Back on topic, interestingly enough, I'm starting to think that, with sufficient age and without other counter measures, that may be exactly what has to be done. I don't know what the cause is, I suspect too many battery drains/removals after the tiny backup battery/capacitor has discharged somehow effecting the motherboard's battery sensors is the culprit, because it seems that a long time ago that's what happened to my first (now perpetually left-as-stock) N900 - it has to be in R&D mode to boot, and now my second N900 (the one mentioned in the previous post) has seemingly gotten to the same predicament - needing R&D mode on to boot. I can definitely confirm that the battery used does NOT seem to have anything to do with it - I can swap in fully or nearly fully charged batteries and still fail to boot on those without R&D mode on. So it's not any given battery wearing out to the point of being a problem. Strange, eh?