I have no idea, but if I by some unlikely chance have the spare time or motivation, I might look into it. Honestly, I am not the most qualified - I wasn't the one who managed to reverse engineer libcal - my program is just the userspace component for going through libcal to change your R&D mode settings (though with an open source libcal we may be able to just control all of the cal area. I guess a general conceptual pre-announce - there will probably eventually be either several other '*mod' programs or one 'calmod' or something, allowing changes at will to all of the CAL 'areas').

Exactly. Your using of the +1 is a kind of workaround for using string functions when libcal actually uses "Pascal-type" strings, i.e. a block and a length, and memcpy instead of strcpy, etc.

If you do rewrite your rd program you might want to go that route, which would sort of fit better with libcal. But this is (likely) a very academic issue. At least R&D mode is essentially treated like a (C-)string.

But if you consider expanding your program to other regions/blocks of cal (think "bme", "phone-info", "wlan-tx-cost3_0", "fmtx_pwl", "lock_code", "lock_period", "lock_enable", etc.) then you'll have to make sure that you read/write exactly as expected by the library (and any other client programs making use of the library..)

Cheers, and congratulations for making it possible to manipulate R&D mode from the recovery console. This is actually seriously cool!

You just wrote a good definition of a file system

Well, that's the thing of flash-memory. You cannot just overwrite something. So you can only write to blocks which have never been written to, hence the appending of data and using a version number to keep track of the latest version. Only when the space gets full you need to erase an entire erase-block (in our case this is 128Kb, i.e. there are THREE erase-blocks in /dev/mtd1), which can then be written to.

The odd thing about libcal is that it implements the whole fs from the highest level (open "r&d_mode" file in "user" directory) to the lowest level (write a CRC32'ed block of data to
an available NAND block).

Not that I know. There is this "calvaria" program which supposedly does that, but I haven't seen it yet.

Your work is much appreciated!

What I meant is that it remained to be seen (at the time I made that comment) if, in spite of the libcal code being made to support abstract data with a given length parameter, the actual data stored was perhaps in C-string format. I.e. libcal supports storing arbitrary data of a specific size (up to a maximum if I understood the code correctly), but one type of such data it could store could well have been C-style strings - and in the case of the R&D mode area, I thought that would actually kinda make sense. Which is why I was saying that one way to figure that out would be to set the R&D mode settings with Flasher then parse the resultant data retrieved from the latest R&D mode block and see.

But I think you're right, anyway, it's not stored in C-string style. I went and ran 'strings' on cal-tool and it calls memcpy (I think this is sound, as my understanding is the reason for such strings being within binary files is that the symbols for calling functions in dynamically loaded libraries use the actual function names used). I still want to do the test eventually with Flasher (it's possible cal-tool opts for one function using memcpy to get the data for any of the blocks that it's supposed to read, or uses it elsewhere), but I think you're absolutely right, the strings are probably stored without the terminating null and the application logic around libcal is supposed to account for the lack of the terminating null if that's the format it needs it in. That would make much more sense - more portable that way, allowing C++ strings or other languages to use libcal and the contents of CAL without having to account for the C-specific string implementation of using terminating nulls.

Actually screw the test, I went ahead and coded in the change and tested it over the course of like 5-10 minutes - it seems to work great. cal-tool -f and rdmod -p don't accidentally run 'too far' and read junk. Near as one can tell... I might as well go ahead and write a special program just to test, making a string "test hi" and then saving it with len passed in as 4. In the meantime, I discovered an interesting flaw (or feature, but doubtful) in cal-tool - if the last substring of the R&D area is not a valid flag, it will print a newline at the end of its input. So rdmod -w "master,shell" will causes cal-tool -f to print:
"master,shell
"
but rdmod -w "master,shell,no-omap-wd" causes cal-tool -f to print:
"master,shell,no-omap-wd"


(And yes, I would like "shell" to be a R&D mode CAL area flag - for specifying that you want a shell at boot or something.)

[another ten minutes later, most of that time being wasted on the actual text file manipulation and typing out the commands and stuff:] Works fine. Pass "test hi" as the data and 4 as the length into cal_write_block (which we knew would store just "test"), and cal-tool -f prints out "test". Funny enough, without a newline after it...

Doing more testing reveals that cal-tool -f does not print an extra newline if you instead do rdmod -w master,shell,poop - so something is special about that string length - rdmod -w master,ploop causes cal-tool -f to print an extra newline, but not say rdmod -w shell

In a similar vein, when the r&d mode string is 1 character long, cal-tool -f prints nothing.

Well whatever. Also, cal-tool has what I'd call a bug - cal-tool -d declares "Enabled" when the r&d string is at all non-empty, regardless of whether "master" is actually in the R&D string.

Source code on front page updated to not assume the string coming in/out needs to be c-style null-terminated.

Thank you very much. Yeah, admittedly the high of the achievement happened a day or so prior, when 'the end' was first in sight, but either way, it is indeed seriously cool, and I am glad I've gotten (with the help of several others, yourself included) to the point of being able to do this.

Also, your remarks about the file system stuff are informative and more importantly made me see how the things I already knew applied to the situation (wear-leveling on flash media for instance, combined with the fact that the nand chip on our N900 SoC doesn't implement its own wear leveling controller, leaving it up to the software to do it [which, as an aside, is how I think it should be done - let the kernel and sufficiently privileged users decide exactly how to wear-level and when to bypass it, don't make it literally hardwired] - I already deduced that the choice to not override existing blocks was made precisely to avoid digging a hole down in the same spot of the flash chip, but your remarks made me go 'well if not as a linked list, the how the frak else would it minimize repetitive writes in the same spots. Any more efficient to search arrangement that I can think of requires more edits of the underlying metadata that would have to actually stay in the same spot). So I stand corrected. It is indeed a file system.

Reading over my last post it occurs to me, I need to not do so much stream-of-consciousness when writing posts.

Actually, I rather like it.....

Interesting. Good to know.

Alright, so now that the rdmod code is stable and rather thoroughly tested, show of hands, who wants me to attach a compiled binary so they don't have to compile it themselves? (I might not get around to it for a day or two, but still, might as well ask now.)

I have at hand a compiled version of yesterday's code. I have now a meeting but after that I'll compile[*] today's version and send it over.
[*] I seriously recommend everyone to get a VPS with scratchbox and the Maemo SDK installed on it. Being able to compile just about anything (including kernels) on-the-go without worrying about space and/or speed and/or battery of your N900 is really an affordable luxury.

EDIT: I also like & enjoy your stream-of-conciousness-style of writing.

Please find attached (gzip compressed) rd, rd_dbg (rd compiled with -DDEBUG). As a bonus I've added calvaria (which dumps CAL in various ways).

Maybe it's time to put it into repos, too?