R&D Mode Control [CLI version]
 

Necessary warnings: This writes to a critical area of the N900 - I'm pretty sure it's safe, but this tool also gives you the potential to do things that could be unsafe or haven't been tested yet. (Though you won't hit that if you're just using it to turn R&D flags on/off, probably).

I have written a command-line program for enabling/disabling R&D mode and setting/clearing R&D mode flags on-device on the N900. I used qwerty's R&D Mode Control (which was GUI-only) for reference with regard to how to use the libcal function, since I could find no documentation anywhere on how to use libcal otherwise, so for that, credit goes to him. However, other than than, a couple of the variable names and error message wordings, my code is my own, hence me not including his name in the copyright statement (edit: except for the "based on work by"). If people more knowledgeable in GPL legalities feel this is wrong, please don't hesitate to inform me.

Source Code (Updated 2013-11-07)
Due to the source code growing in size and the forum software being limited to 15000 characters per post (bad coding, indefinite sized posts with splicing in the backend transparent to the user is trivial to do right), I've had to move the remainder of this post to:

http://talk.maemo.org/showpost.php?p...7&postcount=40

Re: R&D Mode Control [CLI version]
 

I think a small hat-tip to q12 would be nice -- but that's at your discretion. I don't suppose you know what other "non-modifiable" information is stored inside CAL? I'd love to have a table with values and address lookups. Maybe I'll find a tattered N900 and start mapping....

Re: R&D Mode Control [CLI version]
 

I edited the source to include some more printf's, indicating when something is enabled/disabled/etc. (I do want to credit qwerty12 in some way, but I'm not sure I want to do it in-code, and I can't really think of a good place in-code-comment to do it, or a proper way to word it. "Credit goes to [qwerty12's name here] for writing the code that was used for libcal reference" is wordy, methinks, but I can't think of an efficient-er way of writing the same meaning.)

Also, a clarification (for those who don't read or understand the code): although you can do something like rdmod -e -e -d -e -d -w abcdefghijklmnopqrstuvwxyz -d -e -s [all flags] -e -d, it won't actually write to the CAL area until it's done processing all the steps - so you'll never get more than at most one write to flash memory per execution, AND, if it finds the original string it reads from cal is identical to the one you entered, it won't write anything, saving flash writes on those stages.

As for Hawaii's question I don't know much about CAL - I know somewhere in there it stores the lock code for the device, and if you look at the original R&D Mode Control thread by qwerty12, someone did some straces of libcal. Another person in some thread I don't remember claimed to have worked on an open source dsme replacement, and had succeeded in reverse-engineering read-only functionality of libcal.

I'll gather the links and sources to everything I can find on the subject to a wiki page and link that here when I get the time.

Re: R&D Mode Control [CLI version]
 

Usually "Based on work by XXXXXX" under your copyright line or as comment in the relevant code should be ok.

Re: R&D Mode Control [CLI version]
 

I like that. Edited accordingly.

Re: R&D Mode Control [CLI version]
 

*Facepalm* I finally got to test it in my boot-shell /sbin/preinit mod, and... it segmentation faults. Works just fine outside of that shell. Just breaks in the utterly basic environment provided by my /sbin/preinit shell.

Anyone mind checking if this works in pali's recovery shell bootmenu item?

- Edit -
I would've tested earlier, even before posting, but the then-current busybox shell wasn't capable of running that early in the boot due to the interim file-in-tmp solution to the history bugs.

Re: R&D Mode Control [CLI version]
 

I'm not sure if this is of any interest to anyone but me, BUT the cal-tool works from the /sbin/preinit shell. So rdmod -p segmentation faults, but cal-tool -f (which does the same thing, prints the R&D mode string) works.

Eventually I'll start comparing straces and see what I can figure out, but it doesn't make sense to me why my program would fail at that stage of boot, as far as I can deduce, at the cal.h derived functions.

Re: R&D Mode Control [CLI version]
 

Been bouncing around ideas in my head about this. First of all, this might be the reason why qwerty12's R&D Mode Control (non-CLI) segfaults when ran in the /sbin/preinit launched shell. (Until I noticed this I blamed it on trying to load graphical elements from within a framebuffer console.)

Another thought - the open source getbootstate binary pali made uses libcal for reading the R&D mode flags. Which suggests somewhere I'm doing something wrong, because getbootstate is executed by /sbin/preinit before my shell runs, if I recall correctly. (Or maybe just after, but they're executed close together.) So it's not libcal by itself at fault, which is what I was thinking might be involved.

I will solve this eventually, when life gives me enough time again, the few users of my code have my word on that. My current plan is to start by looking at Pali's open source getbootstate code and see how he uses the libcal stuff.

Re: R&D Mode Control [CLI version]
 

I'm just going to stream-of-consciousness here, both for personal benefit and because it might help someone else some day.

I noted another thing getting my second N900 set-up: "Failed to read cal area" error message comes out when trying to run my rdmod (which for those not following is what I'm calling the compiled result of the code above on my N900s) binary.

Here's the interesting bit - the N900 is freshly flashed, and has had nothing done to the R&D Mode flag area done since then. Thinking back, I had this occur on my first N900 before, back when I first compiled qwerty12's code. Then I noticed disabling the R&D mode with the flasher tool served to make qwerty12's R&D Mode Control work. I shrugged it off and assumed it meant that my CAL area was messed up earlier at the time. But this happening on a completely newly reflashed N900 suggests that a 'virgin' N900 doesn't have the R&D mode part of the CAL area set up the same as normal. Probably minor irrelevant trivia to most people, but I thought it was interesting. Now the question is, does 'virgin' in this context mean just 'freshly reflashed', 'freshly totally reflashed, eMMC included', or 'never R&D Mode configured ever before'?

Anyone who has N900s which have been freshly fiasco flashed, freshly eMMC flashed, and never put into R&D mode ever, who is interested in helping me by running my code on their device and reporting back, would be greatly appreciated. You don't even have to try to change the R&D flags, just "rdmod -q" will suffice for what I'm looking for. Actually, I'm even willing to post my code pre-compiled for people who are willing to test but haven't the desire to compile, because experience has thus far shown both mine and qwerty12's GUI predecessor to be safe at changing R&D mode flags.

On my side of things when I get the time I'm going to throw caution to the wind here and test what happens if I compile a version of the program with that check removed, and if it fails the check after that, with that check removed too. It's possible those checks are unnecessary in practice for one reason or another.

Re: R&D Mode Control [CLI version]
 

@Mentalist Traceur,

I'd be willing to try that, but better if you provide a compiled binary (I'm just starting to learn to use scratchbox, and anyway have no time when I'm at home..)

My N900 was bought (new) in Jan 2011. Update to PR1.3 was OTA. I have kernel-power v47 installed (no CSSU, no other low-level tweaks).

It's never been reflashed (yet :). The only "flashing" was the OTA 1.3 update and the kernel-power flashing, so I'd assume the R&D area has never been touched.

If that's OK for testing, then please post the rdmod binary!

Re: R&D Mode Control [CLI version]
 

That sounds great for testing, and thank you. Here you go:
Attachment 23165

Extract using the command:
I say cd to root because it's tar'ed with a path to /sbin/rdmod already. Obviously if you want it anywhere else you can move it afterwords. Or you could usesome archive and file managers, of course. Up to you.

Anyway, much appreciated that you are willing to test.

Re: R&D Mode Control [CLI version]
 

i prefer R&D mode, less crashes in intensive file copying and i only have to press the button once to turn it on rather than hold it till the white light comes on

Re: R&D Mode Control [CLI version]
 

I prefer it for the same reasons, though I don't get the point of stating that. Anyone using my command line tool for turning it on/off would obviously be already in favor of using R&D mode for one reason/situation or another.

Re: R&D Mode Control [CLI version]
 

Okay, I spoke to dr_frost_dk and it sounds very likely that my current N900 (that I bought from him) never had R&D Mode turned on ever.

reinob: Did you get around to testing it? Any problems or need for more detailed instructions?

@ Everyone else: Tests on FRESHLY-reflashed N900s that HAVE been in R&D Mode before the reflash but haven't been put into R&D mode since, are also appreciated.

Re: R&D Mode Control [CLI version]
 

Hi all

I will be able to test it tomorrow (monday).

Re: R&D Mode Control [CLI version]
 

Cool. No rush by the way, and I'm sorry if I sounded like I was impatient, I was just curious given the two-day silence if you had seen my post or something had come up or wasn't working out.

Re: R&D Mode Control [CLI version]
 

No prob. Just busy on weekend (kids, guests + building a house :)

Anyway:

~# ./rdmod -q
Failed to read R&D Mode area from CAL.
Segmentation fault
~#

If you want me to strace/ltrace/syslog I might give it a go.. but tomorrow :)

Re: R&D Mode Control [CLI version]
 

Alright, so that's the same error I was getting. Great. No need for more detailed debugging currently... well, I suppose if you didn't I'd end up asking for it later anyway, but it's not urgent, as I currently lack the knowledge to figure out what the contents of strace/ltrace outputs mean anyway.

Re: R&D Mode Control [CLI version]
 

Well, I tried with ltrace and strace but got no useful info. Program is being killed by SISSEGV (Segmentation fault).

I don't know if the source code for the libcal are available. I might have a look at it some day..

Cheers.

Re: R&D Mode Control [CLI version]
 

Nope, libcal is closed. There was someone on this board who claimed they managed to reverse engineer all of the read functionality of it in an open source variant a long while back, though I don't know where the source for that is.

I suspect the segmentation fault is my own fault, perhaps the program isn't handling it correctly. But the error message you get before the segfault is indicative of what's happening (as I understand it it means the R&D Mode area for CAL isn't set up correctly at all yet on the devices where this happens, though I am not sure what that entails from a technical perspective). If you look at my source code posted in the first post, that error shows up at this moment in the code:
My C knowledge was and is just barely enough to write my program based on qwerty12's, so I'm not 100% sure what could cause the segmentation fault, unless there's something wrong with my "return 2" statement.

Re: R&D Mode Control [CLI version]
 

Just a few notes (my C is also a bit rusty, but it's still there :)

With cal_read_block() both tmp and len are initialized. rd_mode_string is, at that point, still not initialized (i.e. not a valid pointer).

Then you do if((len < 1) && !rd_mode_string) { ... }

so in principle this is invalid. I guess you want if( (len<1) || (tmp == NULL))

because it's tmp and len that get initialized/assigned by cal_read_block().

The && is important. It should be || (OR). Because, assuming rd_mode_string is hopefully initialized to NULL, you're only exiting if len < 1.

So it can happen that you're actually trying to read from tmp (rd_mode_string) when it's still NULL or uninitialized -> SIGSEGV!

I can't compile anything now (can only do useful "work" here when I'm at work, i.e. no compiler; at home I have the compiler, but no time :)

Try fixing those bugs.(I only looked at the "-q" code path) and post another compiled binary and I'll give it a try!

Re: R&D Mode Control [CLI version]
 

5 months and much C experience later, I finally return to this. Funny how instantly clear it is why it's segmentation faulting in one of the situations now...To everyone, reinob especially, I apologize for the delay in getting back to this. Life, busy, etc. The usual.

You may notice that right under the problematic (len > 1 && !rd_mod_string) if-then brackets, I had rd_mod_string = (char *)tmp;

My understanding is, originally (in qwerty12's RD Control program, which I derived most of the meat of this program from), that line was in front of that if-then statement, and at some point back when I understood what I was doing even less than I do now, I moved it down not realizing it would mess things up.

After brief though, I've decided to change the if statement to (len > 1 && !tmp). This is, in my opinion, keeping in line with qwerty12's original logic, and at the same time, might possibly save an instruction cycle or two, because I'm not assigning tmp's value to rd_mod_string until after we know tmp is not null. I'm hesitant to switch to an || instead of &&, because if qwerty12 didn't do it, he probably had a decent reason... maybe.

With this fix, it fixes the segmentation fault on 'virgin' N900's - that is, ones that /never/ had their cal areas' R&D mode flags set. But it does /not/ fix the segmentation fault that occurs in my /sbin/preinit ebedded recovery shell.

Re: R&D Mode Control [CLI version]
 

@Mentalist Traceur,

Just some off-line debugging.. maybe my comments don't solve anything, but as a "past" C programmer there are some things that hurt my eyes.

Before calling cal_init() I would first initialize cal_s to NULL. We don't know what libcal does exactly, so better to be on the safe side.

Then, before doing cal_read_block make sure that cal_s is not NULL. Otherwise cal_read_block will most likely segfault.

Same with cal_finish(). What if cal_finish expects a non-NULL pointer but cal_read_block actually handles a NULL pointer by merely returning a negative value? I bet 10$ that cal_finish does free() at some point, and we all know what happens (or what should happen anyway) when you free a NULL pointer.

Now, AND and OR. Obviously we don't know what libcal is doing, but it is supposedly writing something to tmp *and* returning a length, presumably of the data being written to tmp, meaning tmp may not necessarily be NUL-terminated (i.e. not a standard C-type string), meaning strcpy() might freak out and write more than 128 bytes to rd_mode_current, meaning that .. well, I don't have to explain that do I?

So, to be on the safe side, use strncpy making sure that you write at most 128 bytes (or sizeof(rd_mode_current)/sizeof(char)) and preferably len, i.e. min(len, 128).
Then put a zero at the end just for security. rd_mode_current[min(len, 128)] = '\0'.

And before that (AND/OR), I would still do if(len < 1 || (tmp == NULL)), as you probably would only continue if len > 0 and tmp is not NULL.

Namely, if for any reason (did I say we don't know what libcal does?) len > 0 and tmp is NULL strcpy() will also segfault.

That's it for now. I'd suggest you add a lot of printf()'s to aid in debugging. Or use assert() wherever you assume something.

e.g. before doing strcpy(a, "hello") put assert(a != NULL). This way you will see the truck before it hits you :)

Re: R&D Mode Control [CLI version]
 

Slightly off-topic.

I just found this:
Calvaria - Maemo CAL partition variable access tool.
https://dev.openwrt.org/browser/pack...src/calvaria.c

Not exactly R&D mode settings, but can apparently parse the CAL partition.

Re: R&D Mode Control [CLI version]
 

This is interesting to follow. Infrequently having to do code analysis during my day job (albeit on a level more abstract), i appreciate your efforts.

Re: R&D Mode Control [CLI version]
 

@don_falcone,

I also find it very interesting.. but at least we can do code analysis with the source in front of us. In a previous life, I used to do that using a hex editor (long live HIEW and MS-DOS in general!), and I actually developed a sort of "talent" for cracking shareware without even bothering to use a debugger.

I'll see if I can get some time to inspect libcal.so. Nowadays people are very sloppy and leave debugging symbols all over the place..

Re: R&D Mode Control [CLI version]
 

Quick update: When I last posted here, the state of affairs was such:
1. Segfault when using this program from the /sbin/preinit point in the boot process.
2. Unable to set/read R&D mode on an N900 that has NEVER had R&D mode set before in it's lifetime (but at least it no longer segfaulted in such cases, as was the case in the last precompiled binary I posted on this thread).

I have just gotten back to this yesterday, and as of earlier this morning, I was able to turn on R&D mode on the N900 I had which, until that point, never had R&D mode enabled on it. Rebooted to confirm it worked (it did), and then turned R&D mode off and rebooted to confirm that worked. Success. So issue 2 is near as I can tell, fixed, but still need to figure out the cause for issue 1. But I consider this a major achievement, in that it makes this FAR more useable for it's intended usecase of letting people turn on R&D Mode on-device (and if you depend on flasher to do it for the 'first time', that's a large amount of scenarios where this tool would be useless).

I will update the source in the first post once I've cleaned up the logic of the code better (and once I get around to it after that). Right now I just took a gamble that you could write to the cal area even if the other two functions for reading the contents of the libcal area return 'false's, and just commented out the return statements in those if blocks - this seems to work just fine, but as a result, the code needs rearranging, me thinks.

Re: R&D Mode Control [CLI version]
 

IT *IS* A MAJOR ACHIEVEMENT.

Thanks a lot for continuing to work on this. I have an N900 waiting for you to post the program :)

(Oh, and I will do my best to test and debug if/why it works/doesn't work during preinit).

Re: R&D Mode Control [CLI version]
 

Hello all, so the code in the first post is STILL not updated, even though I posted over a year ago saying "I'll update it when I've cleaned up the code". Someday, I swear....

Anyway, I've made headway on the segfault issue, albeit not actually solved it.

This is all thanks to freemangordon having created opensource libcal (which, BTW, works beautifully for me so far - my development N900 is running with that libcal installed over the stock one with no issues for days now [Edit: well... it worked once I fixed a single missing #include in the cal.h file]. One thing to note - the gcc version built for ARM in the sdk repo, when installed on the N900 (as opposed to scratchbox) is NOT capable of building the library, thanks to some bug in that ARM version of GCC with compiling dynamic things; so it needs to be compiled in scratchbox - I have not tested with the gcc-4.6 that's in the extras-devel repo to see if that does any better).

Long story short, libcal uses semaphores, as provided by semaphore.h. The function sem_open(), specifically, is what is causing the segfault. That's in the semaphore.h library, which means the problem is way deeper than my code, or even libcal code. I guess so early in the boot as my preinit shell, the kernel/OS simply isn't set up enough to even allow semaphores to be used properly.

Two potential workarounds come to mind. 1) figure out how to proceed the boot the minimal amount needed to get working semaphores, and package that with the code - catch segserv (or whatever the segfault error signal is), and run that code in response, or just leave it to the user to run the required stuff (either shell script or built into the rdmod binary invoked by commandline flag). OR 2) rewrite the relevant aspects to NOT use semaphores, build that version of libcal statically into rdmod, and then have it invoke the statically included versions of the functions when it detects semaphores aren't available, under the assumption that so early in the boot nothing else that could access rdmod will be running at the same time.

Further investigation path before I do anything else tho: Figure out why the stock cal-tool, that lets you get r&d mode flags, isn't segfaulting still (if it does with the new libcal, that means the new reverse-engineered libcal isn't 1-to-1 with the old libcal; if it doesn't, then it's worth looking into why cal-tool doesn't fail, or if it's even using libcal at all (or if it just doesn't use it at all)).

Don't expect further progress by me for the next week or two (or more, who knows with my busy schedule and unpredictable direction for my bursts of motivation), because collegiate work and actual work (not that anyone cares, but I got a job as a parttime job as a programmer about two weeks ago: which means I have much less time to donate to community but more more money to donate to members thereof, yay).

But yeah, this is promising. And I felt like updating you guys.

Re: R&D Mode Control [CLI version]
 

@Mentalist Traceur - check if you have /dev/shm mounted by the time the segfault occurs ;)

Re: R&D Mode Control [CLI version]
 

This. Right here. You are awesome. That was the missing piece.

For everyone else who's curious, my understanding is: Semaphores need a spot in shared memory that they can be stored in (it can be shared either between different threads in a single process, or between different processes - libcal uses the latter). I guess the default location for that is whatever virtual filesystem /dev/shm is mounted as. At the stage were my preinit shell prompt runs, /dev/shm is not mounted, so the sempahore creation fails.

Thankfully, /dev/shm is a simple tmpfs filesystem in typical Linuxes, so "mount -t tmpfs tmpfs /dev/shm" as root is sufficient; then rdmod runs with no segfaults!

Furthermore, now that I understand this, I can understand more of the straces of rdmod vs. cal-tool. It seems that cal-tool, upon detecting that /dev/shm is missing (it doesn't seem to be catching the segfault, rather I assume it actually checks for it somehow), it seems to call mmap, after which it creates the semaphore in /tmp/ instead of /dev/shm/. (This is an educated guess interpretation based on the fact that there is a stat() call on /dev/shm, followed by an mmap() call, followed by a stat() on /tmp, whereas rdmod just does the first stat and then segfaults. My strace reading skills are still in their early stages.) [Edit]Correction, it's statfs(), not 'stat()', that it calls on the directories, and more importantly, the mmap does not seem to be the thing that matters in this situation, as a closer look shows there's a bunch of mmap calls all over the place leading up to the segfault and I don't think that that one mmap has anything to do with cal-tool managing to switch to using /tmp as a spot for its semaphore.[/Edit]

(In retrospect last year when I was trying to figure this stuff out with strace on cal-tool vs my rdmod I should've known to look into /dev/shm (as it gets stat'ed right before the segfault, which I had seen last year) but at the time I had no context to know what it meant, and I think eithout understanding that semaphores were used I'd've prob'ly not realized the significance of anything I found at the time even if I did think to attribute any significance to /dev/shm.)

The next few steps are to figure out how to either mount /dev/shm from rdmod when needed, or I think even better, do what cal-tool does and use mmap somehow.

Re: R&D Mode Control [CLI version]
 

What really makes this confusing is that I can't find any documentation explaining WHY /dev/shm is the default location where named semaphores go, or how to move/create your named semaphores elsewhere (like cal-tool does). So I don't even really know where to start, when it comes to figuring out how the cal-tool code works to achieve what it does.

Besides that, here's a headscratcher (although the question is just a pure why rather than it getting in the way of a solution at the moment): /dev/shm DOES show up as mounted if you run 'mount'. rdmod segfaults. Then you run 'mount -t tmpfs tmpfs /dev/shm'. /dev/shm shows up in the same exact way in 'mount' output again. No change what so ever. Yet rdmod stops segfaulting.

I want to know how @ paragraph 1 and why @ paragraph 2. But for the time being, I am content with the fact that some way of making rdmod finally work at early boot is possible.

Re: R&D Mode Control [CLI version]
 

Partial answer to my own question: /proc/mounts is where the actually accurate mounts data is, 'mount' with no arguments seems to print out a list that doesn't reflect the current mounts, at least at that stage of boot. *Shrug*

Re: R&D Mode Control [CLI version]
 

You just opened the "/etc/mtab vs /proc/mounts (or /proc/self/mounts)" pandora box :)

The actual information regarding mounted filesystems (as seen by the kernel) is in /proc/mounts (or /proc/self/mounts if you want to be chroot-compatible).

The program "mount" writes to /etc/mtab. The program "umount" writes (deletes lines from) /etc/mtab.

Normally /etc/mtab contains the same information as (or rather, information consistent with) /proc/mounts.

However if the computer crashes it doesn't get a chance to delete /etc/mtab, so upon reboot /etc/mtab will contain stuff that possibly is not relevant anymore. For that reason some people make /etc/mtab be a symlink to /proc/[self/]mounts. However this breaks other stuff.

Ideally /etc/mtab should be wiped during (early) boot, i.e. before anything is mounted. I don't know (at the moment) if/how Maemo5 handles this.

EDIT: In /etc/init.d/rcS you have
and a bit later
and for completeness, if I look in my /dev/shm I have a 20-byte file called "sem.nokiacal" containing, in hex:
"01000000 00000000 00000000 00000000"

/usr/bin/cal-tool uses libphread.so, so I guess it uses sem_open(3) to create a named semaphore. It is (I think) the kernel which takes care of creating the named file under /dev/shm.

so basically sem_open("/my_semaphoe", ...) will create /dev/shm/sem.my_semaphore

Re: R&D Mode Control [CLI version]
 

Oh okay, that explains that. Thanks.

Ohhh, okay. That's why my /etc/mtab and mount output is completely meaningless - as people might recall my 'at boot' shell of choice is literally virtually the first moment it's possible to boot during. /etc/init.d/* doesn't happen until later.

I'm particularly glad you looked up the exact mount command that's run to get /dev/shm up - my current solution to the segfaults seems to work regardless of it, but if it ever seemed to be an issue, that could come in handy.

On this point you're absolutely right, but this wasn't in question anymore: In case you missed it, freemangordon got us an opensource recode of libcal, which means we can look at what the (presumably nearly identical) version does here:
https://gitorious.org/community-ssu/libcal/

So from that code we already knew that it was using a semaphore, calling sem_open(), etc. That's how I even got to the point where I knew it was segfaulting at the semaphore creation level, which is what led freemangordon to give the hint that he gave that solved our segfault troubles.

But that's not the question. The question is the behavior of cal-tool at that early boot situation where /dev/shm is not yet mounted. Somehow, it still manages to create a semaphore, located at: /tmp/sem.nokiacal

Nonetheless, your statement says more concisely/clearly what cal-tool and other libcal programs do with regard to semaphore use, and either way this remark is appreciated just as your prior ones were.

---Regarding updated source---
I am hoping to have the updated source out in the next 24 hours - it was technically ready to go as it was but it wasn't the best way to do it (I decided to not screw with detecting /dev/shm and simply mount a tmpfs on /dev/shm before cal_init() and umount it after, and it worked just fine, but in principle it's better to actually check if there's a /dev/shm already mounted, and only mount a new one if needed. Until this post I figured /etc/mtab, mount, and /proc/mounts was a crapshoot, but now that I know that /proc/mounts is actually word-of-god (s/god/kernel/ same thing for our purposes), I figured it's worth it to go ahead and see if /proc/mounts has a /dev/shm in it already, and only mount/unmount a new one for the duration of the program if not.

So once THAT is coded up, I will post an updated source, finally.

Re: R&D Mode Control [CLI version]
 

Code updated in the main page.

It works. It finally works. [edit]In every way that I have always wanted it to that is - it obviously worked almost entire before, but now the segfault at early boot issue is solved too, so it fully works as I've wanted ever since I started on this like 2-3 years ago.[/edit] I have one more N900 with a never-touched-R&D-mode-area, on which it needs a tiny bit of testing (to make sure nothing I changed between last year when I fixed that problem and this year when I fixed this one, or nothing that I forgot to clean up, causes any issues there). But I am pretty confident that even as is, it's usable in all of the scenarios it needed to be usable in.

It's frakking awesome. It scans through /proc/self/mounts to check if a /dev/shm is listed in there, and if it is, proceeds. If it reaches the end of file without having found one, it mounts one before proceeding and then unmounts it later when it's done (only if it mounted it prior of course). For academic purposes I still want to know how cal-tool redirects its semaphore placement to /tmp, but I like my way better unless their way is vastly more efficient.

There are is at least one spot where I want to go over the code to A. verify that I didn't mess anything up with the R&D cal area 'virgin' N900s, and B. tweak the actual output that it makes. But I do believe that should work. (expect an edit within 24 hours (probably, who knows with me) to the front post once more, but after that it should stabilize for a while again)

In the long run, there will be refactoring. What people might not realize is this was effectively my first meaningful C program. It has grown along with me over the years (and, quiet poetically, it is the first piece of code where I used a goto (two even); I've always been against the anti-goto dogma, being an efficiency zealot deep down, but I have also always held off on using it until it seemed like it was notably the best option for what I wanted. In a silly, irrational way, I am glad that it was this piece of code that it first came up).

Anyway, I would ultimately like to make the commandline parameter parsing more robust and more informative when something wrong is done. I would also like to make it do a full pass over all of the commandline parameters before beginning to construct the actual string to write into the R&D mode area of CAL, so that it doesn't was time realloc-ing and strcpy-ing all over the place only to reach later commandline flags that negate what the earlier flags specified. (Not that that's a good use of the program anyway so that's the user's fault, but there is a few possible spots where being able to parse the parameters in advance may in theory be useful.

I also need to get around to submitting the bugfix(es?*) to freemangordon's libcal.

*I noticed this before but at the time didn't understand why it might be an issue: Nokia's libcal uses a named semaphore "nokiacal". The open libcal uses a named semaphore "nokiacal3". So far it seems that nothing that uses the cal area on the N900 does so by other means than libcal, but if by some chance some system/program has "nokiacal" as the named semaphore hardcoded into it, and doesn't go through libcal, we could have a problem of concurrent access to the cal area, which is exactly what the semaphores are presumably meant to protect against. Furthermore, just strictly speaking a recode of some system library (or in general any software) doesn't need to make changes to the name of a semaphore that it uses (or in general, anything it uses) unless there are technical reasons why that's an improvement.

P.S. I expect tears will be shed by those reading my code who are non-believers in the goto. The goto will forgive you, and welcome you with open arms when you come to accept the goto wisdom. (What I seriously mean to say is, those who have a problem with where I used goto calls are ultimately welcome to give me a rewritten version that is qualitatively easier to understand, and we can discuss from there. I am not likely to be convinced by just the typical anti-goto mentality though if it's just preachy.)

Re: R&D Mode Control [CLI version]
 

I renamed the semaphore while testing the library and forget to rename it back before pushing to gitorious. feel free to make a merge request on gitorious fixing that.

also, libcal is missing debian packaging, I guess we'll address that when the time comes (neo900 :) )

Re: R&D Mode Control [CLI version]
 

Thinking out loud here about the issues with the 'virgin' N900s originally having had issues with cal_read_block:

cal_read_block invokes find_block_type, which then returns a struct cal_block *, which if null, causes cal_read_block to return -2, which is what it returns on 'virgin' N900s.

find_block_type then does a bitwise & against a CAL_FLAG_USER (as it so happens, we do pass in that exact constant in the last field of cal_read_block) which determines where it looks in the cal area for the relevant block it seems. It then calls find_block.

find_block simply iterates through the struct cal_block *, which is a linked list basically. It heads onwards through until it finds a block with a name that matches the name passed in. The difference between CAL_FLAG_USER and not is which of the block lists find_block_type selects from the cal area - there is a user_block_list and a wp_block_list variable inside the main cal struct, and those are presumably heads to the lists. I suspect this is largely irrelevant to my situation, but may be worth understanding more what the two lists mean later (i.e. what belongs in which).

I think I will write code some day to peruse these cal area lists and see what's where. I make the tentative conjecture that the cal area is 'sprayed' over some region of the NAND flash and that new settings are added on to the tail of the linked list instead of overriding old ones (I say this because there is a comment implying that if multiple blocks with the same name are found, the one with the highest version must take priority, suggesting there may be more than one "r&d_mode"-named block in the cal area at a given time). But maybe I am wrong, hence the interest in eventually digging deeper.

Anyway, I think the case is pretty clear then: a 'virgin' N900 which has never had R&D mode settings toggled on/off simply has no block in the user_block_list named "r&d_mode". The linked list perusing code of find_block goes all the way to the end until the pointer becomes null, at which point it returns it (that's all it is, a while(block) {} loop, with block being the struct cal_block *, and inside the loop it breaks if the name matches the one requested, or iterates by doing block = block->next. When the loop is over, it returns whatever the current block value is, in this case null).

That is returned by find_block_type without any further consideration. cal_read_block checks if it got a valid pointer, sees that it didn't, and returns a predefined negative value to signify the error.

And having walked through that, I think I have a pretty good idea of how to proceed with finalizing the code cleanly (I don't want to ignore the other error it can give or hypothetical future implemented errors, just that not-found-block one is safe to disregard).

Re: R&D Mode Control [CLI version]
 

Quick note, segfaulting at boot on my main, still 'r&d-virgin' N900, not sure what's up. I quite possibly just f'ed something up copying it over from my development N900, but this might indicate a deeper issue and that my code is not as ready as I had hoped.

Re: R&D Mode Control [CLI version]
 

[THIS IS A PASTE OF THE END OF MY FIRST POST, FORUM DEMANDED THAT I TRUNKATE IT DUE TO THE SOURCE GETTING TOO LONG]

Compiling
I coded and compiled and tested all of this on the N900 itself - there, I compiled this using The "-lcal" flag is needed to get the libcal functions to compile in correctly. I did not try using "-Wall", as far as I remember, so if anyone sees anything they don't like while using the "-Wall" flag, speak up, by all means.

But I'm don't wanna compile, give me a binary, or better yet a .deb file
I didn't want to compile back when I first had to either, but I figured it out and my life is better for it (and I never intended to program back then, I just really wanted aircrack-ng and qwerty's gui equivalent of this). I'm not offering it in a precompiled binary form right now because this is mainly for power users, and because I don't really feel like packaging it and sticking it into extras-devel right now (not sure if I could, given the dependency on Nokia's libcal and build-dependency on libcal-dev), however, I will happily place it inside extras-devel (with an all-caps warning in the description, possibly in a non-user category), when I have more time, so long as no one finds horrible flaws with it. Experience using qwerty's program for around almost a year suggests to me that this is a perfectly safe way of modifying R&D Mode flags, but in spite somewhat heavy testing it's possible I missed something. I would also like to see if the -w command is safe. As far as I've seen, correct flags separated by commas are read, extraneous text after that is ignored, but I haven't messed with it too much, or tried writing in strings longer than the R&D CAL area was presumably intended to take at the most (see below, where I talk about -w).

I should probably also tweak the output of -q to be a little more at-a-glance readable.

Why would you want to use this over or along with qwerty12's, if you can compile both already?
The only problem with qwerty12's was that it was a GUI-only program. This is good enough for most uses, and is a bit more intuitive to use, but in my experience, the most crucial (not necessarily most common, but definitely most crucial) situation where you would want to enable R&D Mode on-device is if you're having some problem early in the boot process, and suspect that changing the R&D Mode flags (to disable watchdogs or lifeguard-reset, or boot in spite of getbootstate complaining about the battery even when you know the battery to be a reliable one) will let you boot up, either to repair the initial problem, or leave the device in that state.

Out of the box, the N900 can't get a shell early enough in the boot to do that anyway, BUT, with the addition of framebuffer console to the latest power-kernel versions (thanks to pali), and the use of it to get at-boot recovery shells by pali and myself, the possibility of being able to edit R&D mode flags from on-device in the earliest stages of boot-up is very feasible.

However, since qwerty12's R&D Mode Control was GUI-only, it simply cannot do anything when ran from a console in a framebuffer environment. Also, the use of a command-line tool makes for convenient opportunities to add a couple of other 'features', such as seeing what actual string is written inside the R&D mode CAL area (useful for understanding how the R&D mode flags are stored if like me, you didn't originally know that cal-tool provides the same ability).

-w
I also added the ability to write a string inside the R&D Mode CAL area manually, using the -w flag. This isn't really coded to check against 'stupid' input, so if you typo something like "[rdmod or whatever you named the binary after compiling] -w -w "so much text", it will write "-w" to the R&D CAL area, not "so much text", because it will see the first "-w", expect the next argument to be the string it needs to write, and thus not process the second -w as an actual command. Then since "so much text" doesn't actually match any of the commands, it'll ignore that argument completely.

This isn't recommended by me for normal use by ANY means, but I thought some power users and people who wanted to mess around while slightly risking the bricking or unpredictable behavior of their N900 could benefit from such a feature - mainly because, if tested enough and proven safe, power users could use this tool for their own scripts to store custom R&D mode flags inside the R&D CAL area - note, however, that this too is untested - the default R&D mode flags, if all are turned on, take up 117 characters including the C string terminating Null byte. Anything bigger might cause undesirable behavior (although I have occasionally accidentally written 119 character into the CAL area, and this did not seem to break anything, but it's always slightly possible it managed to write garbage over something supposedly important - though my N900 is working fine so far).

At the same time, if/when the maximum size limit is found, the code can be updated to include a maximum character limit.

Other warnings
-s will write flags in even if R&D mode isn't on ("master" isn't already written in) - I suspect this to be safe and I THINK it'll just be ignored when booting, but I haven't rigorously tested, so when you're turning on R&D mode AND setting some flags at once, use "-e" followed by "-s". (-s followed by -e will simply make -e cancel out -s; I'll modify the behavior one way or the other accordingly when I have more time to see if R&D mode flags can be left set without the master flag being set as I suspect or not).

Everything is executed in order, so you can write -e and -d in the same command, and R&D mode will be enabled or disabled according to whatever was last. Same with -s and -c. This also means that -e, -d, and -w completely override whatever preceded them. There's no logical reason to combine commands in a way that this would arise, ever, but it's always possible someone will, so I wanted them to understand the behavior.