Active Topics
| The Following 12 Users Say Thank You to iDont For This Useful Post: | ||
 |
|
2012-06-26
, 19:25
|
|
Posts: 5,028 |
Thanked: 8,613 times |
Joined on Mar 2011
|
#262
|
Thanks a lot, iDont! I've updated to new busybox from day first, and indeed, I don't have anything to report.
As for busybox-thumb - maybe it's good idea to sent it to freemangordon, for inclusion in thumb repository?
/Estel
As for busybox-thumb - maybe it's good idea to sent it to freemangordon, for inclusion in thumb repository?
/Estel
__________________
N900's aluminum backcover / body replacement
-
N900's HDMI-Out
-
Camera cover MOD
-
Measure battery's real capacity on-device
-
TrueCrypt 7.1 | ereswap | bnf
-
Hardware's mods research is costly. To support my work, please consider donating. Thank You!
N900's aluminum backcover / body replacement
-
N900's HDMI-Out
-
Camera cover MOD
-
Measure battery's real capacity on-device
-
TrueCrypt 7.1 | ereswap | bnf
-
Hardware's mods research is costly. To support my work, please consider donating. Thank You!
| The Following 5 Users Say Thank You to Estel For This Useful Post: | ||
|
|
2012-06-29
, 14:10
|
|
Posts: 268 |
Thanked: 1,053 times |
Joined on May 2010
@ The Netherlands
|
#263
|
Originally Posted by Estel
Great to hear that you haven't got anything to say
Thanks a lot, iDont! I've updated to new busybox from day first, and indeed, I don't have anything to report.
.But seriously though, this is a useful confirmation on the software's stability/maturity.
Originally Posted by Estel
Thanks for the pointer! I spoke to freemangordon or #maemo yesterday: there is no problem with including busybox-power in community-thumb. Of course, it won't get pulled in by cssu-thumb or anything, the repository will just provide a place for it to sit in.
As for busybox-thumb - maybe it's good idea to sent it to freemangordon, for inclusion in thumb repository?
Now I just need to get around to cleaning some things up (version string, thumb-related dependencies, ..) before I can send the sources to freemangordon for upload. Well, the weekend is just around the corner
.Update: the package is in community thumb!
Last edited by iDont; 2012-07-21 at 17:45.
| The Following 4 Users Say Thank You to iDont For This Useful Post: | ||
|
|
2012-07-18
, 06:14
|
|
Posts: 2,225 |
Thanked: 3,822 times |
Joined on Jun 2010
@ Florida
|
#264
|
Originally Posted by iDont
I had a thought a couple of days ago about this - haven't had time to look at the code, but I figured I'd type it 'out loud' here in case anyone wants to look into it: busybox is written in C, correct? And in C, a file is accessed using a variable of type FILE *, if I recall correctly?
As of now there is just one issue with busybox-power: when ash is killed -for the lack of a better term- "unnicely", there is a small chance ~/.ash_history could get garbled up. ...
Well, given how C is, it's fairly possible that writing too large of a sequence of values to a given address, or an inappropriately assigned pointer somewhere, could result in rare cases in the overriding of the FILE *, no?
If you're referring to the bug I think you're referring to, in my experience, when it has happened, I ended up with lines of XML as my 'shell history' instead of actual commands. Suggesting that at some point the wrong file got copied over into .ash_history. I'll be honest, I know nothing about how 'FILE' is implemented, on our device or otherwise, but I would think that, if a FILE pointer's value got changed, it would either point to a different file, or to a non-existent file, depending on the exect bytes at the newly pointed to location. C being C would assume you knew what you were doing and would interpret the data at the new location as a typical FILE struct (I think it's a struct, idk). If it was a corner case resulting from some rarely occuring overwrite of a FILE pointer, the compiler would probably not catch it?
(If I am wrong, if someone could let me know how it actually works, it would be greatly appreciated.)
| The Following 6 Users Say Thank You to Mentalist Traceur For This Useful Post: | ||
 |
|
2012-07-19
, 18:36
|
|
Posts: 4,118 |
Thanked: 8,901 times |
Joined on Aug 2010
@ Ruhrgebiet, Germany
|
#265
|
Exact description.
That will be the consequence, and also explains the absolutely random structure of text content of history (sometimes XML or other config or even email content).
But not yet the reason, we (iDont
) have to find ...
But if it would be that (having history written from FILE to a file, I do not understand why .... wait just last 100 lines are written from FILe to history file. OK, fully agree.
That will be the consequence, and also explains the absolutely random structure of text content of history (sometimes XML or other config or even email content).
But not yet the reason, we (iDont
) have to find ...But if it would be that (having history written from FILE to a file, I do not understand why .... wait just last 100 lines are written from FILe to history file. OK, fully agree.
| The Following User Says Thank You to peterleinchen For This Useful Post: | ||
|
|
2012-07-20
, 07:15
|
|
Posts: 1,808 |
Thanked: 4,272 times |
Joined on Feb 2011
@ Germany
|
#266
|
Originally Posted by Mentalist Traceur
The idea sounds good. But honestly the chances of a random FILE* pointer (e.g. one having been used for cat'ing an XML file..) being overwritten so as to end up having the exact value of the FILE* pointer of .ash_history are close to zero.
If you're referring to the bug I think you're referring to, in my experience, when it has happened, I ended up with lines of XML as my 'shell history' instead of actual commands. Suggesting that at some point the wrong file got copied over into .ash_history. I'll be honest, I know nothing about how 'FILE' is implemented, on our device or otherwise, but I would think that, if a FILE pointer's value got changed, it would either point to a different file, or to a non-existent file, depending on the exect bytes at the newly pointed to location.
C being C would assume you knew what you were doing and would interpret the data at the new location as a typical FILE struct (I think it's a struct, idk). If it was a corner case resulting from some rarely occuring overwrite of a FILE pointer, the compiler would probably not catch it?
(If I am wrong, if someone could let me know how it actually works, it would be greatly appreciated.)
I'm not 100% sure now but I think busybox keeps the history in RAM until it exists, where the history file is actually written to. Meaning [ assuming that what I say is true ] that at some point, for some reason, the RAM buffer is being overwritten (partially) with other data (XML, whatever).
I'll see if I can have a look at busybox's source code and figure out what's going on, but my severe lack of time may prevent me from actually achieving anything anytime soon.
| The Following 3 Users Say Thank You to reinob For This Useful Post: | ||
|
|
2012-07-20
, 13:36
|
|
Posts: 2,225 |
Thanked: 3,822 times |
Joined on Jun 2010
@ Florida
|
#267
|
For the record, we don't need to use XML files as the example - yesterday I got the bug again on my testing N900 when playing with the twl4030 watchdog, and the new histories I got were nothing at all for the root shell (the one you get to by using 'sudo gainroot', not the one gotten to by using 'root', if that makes a difference), and something which was probably a mozilla config file for the 'user' user shell.
I wasn't suggesting, as I think you interpreted, that by using the shell to manipulate file X, I somehow managed to cause that file's pointer to point to the .ash_history file. Rather, I was saying that the .ash_history file struct may have gotten overwritten by random 'garbage' data - and the depending on your luck it might either end up with a handle to a non-existing file, perhaps explaining the 'blank' root user shell history I got in my last instance of the bug, or the file handle at the C FILE struct might end up being one pointing to an actual file somewhere - explaining why people have different experiences with what file contents end up overriding the shell history.
Either way though, you know C more than me, so you probably got more insight into this than me.
Originally Posted by reinob
Well, you don't need a random FILE* to get the value of the .ash_history FILE*, which would indeed be vanishingly unlikely. You'd just need the value of the .ash_history FILE* to get the value of any random file on the system - a much more likely possibility.
The idea sounds good. But honestly the chances of a random FILE* pointer (e.g. one having been used for cat'ing an XML file..) being overwritten so as to end up having the exact value of the FILE* pointer of .ash_history are close to zero.
I wasn't suggesting, as I think you interpreted, that by using the shell to manipulate file X, I somehow managed to cause that file's pointer to point to the .ash_history file. Rather, I was saying that the .ash_history file struct may have gotten overwritten by random 'garbage' data - and the depending on your luck it might either end up with a handle to a non-existing file, perhaps explaining the 'blank' root user shell history I got in my last instance of the bug, or the file handle at the C FILE struct might end up being one pointing to an actual file somewhere - explaining why people have different experiences with what file contents end up overriding the shell history.
Originally Posted by reinob
This should indeed be the case - after all, it was in this thread that I originally suggested to iDont that he implement a buffering of history in RAM (so that devices with flash memory don't waste flash chip write cycles with every press of the enter key. And I remember being very happy about him saying that this got implemented upstream.I'm not 100% sure now but I think busybox keeps the history in RAM until it exists, where the history file is actually written to. Meaning [ assuming that what I say is true ] that at some point, for some reason, the RAM buffer is being overwritten (partially) with other data (XML, whatever).
Either way though, you know C more than me, so you probably got more insight into this than me.
| The Following 4 Users Say Thank You to Mentalist Traceur For This Useful Post: | ||
|
|
2012-07-20
, 20:52
|
|
Posts: 1,808 |
Thanked: 4,272 times |
Joined on Feb 2011
@ Germany
|
#268
|
Originally Posted by Mentalist Traceur
True. But then the other file would be overwritten with the .ash_history contents (which would actually be a major bug). Unless the rewriting would happen after the history is read but before it is written to.
Well, you don't need a random FILE* to get the value of the .ash_history FILE*, which would indeed be vanishingly unlikely. You'd just need the value of the .ash_history FILE* to get the value of any random file on the system - a much more likely possibility.
I wasn't suggesting, as I think you interpreted, that by using the shell to manipulate file X, I somehow managed to cause that file's pointer to point to the .ash_history file. Rather, I was saying that the .ash_history file struct may have gotten overwritten by random 'garbage' data - and the depending on your luck it might either end up with a handle to a non-existing file, perhaps explaining the 'blank' root user shell history I got in my last instance of the bug, or the file handle at the C FILE struct might end up being one pointing to an actual file somewhere - explaining why people have different experiences with what file contents end up overriding the shell history.
So to me a plausible explanation would be that the garbage comes from data that has been read/processed by busybox at some point (hence my example of cat'ing an XML file).
But this is only a wild guess.
Either way though, you know C more than me, so you probably got more insight into this than me.
(and even if/when I look at the code: I find it hard to read (=like) what others have written)
| The Following 2 Users Say Thank You to reinob For This Useful Post: | ||
|
|
2012-07-20
, 23:09
|
|
Posts: 4,118 |
Thanked: 8,901 times |
Joined on Aug 2010
@ Ruhrgebiet, Germany
|
#269
|
Originally Posted by reinob
Some comments (from bottom up):
The idea sounds good. But honestly the chances of a random FILE* pointer (e.g. one having been used for cat'ing an XML file..) being overwritten so as to end up having the exact value of the FILE* pointer of .ash_history are close to zero.
Well, a FILE structure essentially contains a file handle (so to speak, the kernel's equivalent of C's FILE) as well as some pointers to various read/write buffers, etc.
I'm not 100% sure now but I think busybox keeps the history in RAM until it exists, where the history file is actually written to. Meaning [ assuming that what I say is true ] that at some point, for some reason, the RAM buffer is being overwritten (partially) with other data (XML, whatever).
My basic knowledge of Linux memory handling would say me that it is almost impossible that some other process overwrite bb owned memory.
Possibly is that bb overwrites its own RAM space for history.
But another thing is that also bb on its own screws the FILE pointer itself, pointing to any space in RAM (reading will be allowed out of non-belonging space?). This would also explain the totally random content of history.
|
|
2012-07-21
, 03:38
|
|
Posts: 2,225 |
Thanked: 3,822 times |
Joined on Jun 2010
@ Florida
|
#270
|
Originally Posted by peterleinchen
That's what I've been trying to get at: not one file overwriting another file, but Busybox's own code messing up and writing some other values to the location of it's own FILE*. I.e. suppose busybox is running, it's storing new changes to history somewhere in RAM, etc, etc. Maybe it's using a temp file in the /tmp/ part of the file system, for instance (or not, either way).
But another thing is that also bb on its own screws the FILE pointer itself, pointing to any space in RAM (reading will be allowed out of non-belonging space?). This would also explain the totally random content of history.
Anyway, at some point when it's supposed to close, it'll have a FILE* sitting in the code - perhaps one that has been declared but not initialized. Maybe it's uninitialized so it has whatever other bytes were at that location instead of a 'real' FILE* - after all, garbage at the low level of C doesn't have to be put in from anywhere, the contents of uninitialized memory are typically garbage too, no? Or, if, say, I point an int* at a location in RAM containing 4 chars, if the compiler doesn't catch it (because it's not predictable to the compiler for watever reason), the code will just interpret the 'garbage' data at that location as if it were an int. Sure, the data isn't really garbage in a technical sense, but it is in context of reading that spot in RAM if you're interpreting it as an int.
So what if the busybox-is-not-getting-killed-nicely scenario occurs? Shell tries to save history, as described by iDont - so what if the FILE* that's supposed to correspond to the ~/.ash_history.$PID.new file gets overwritten - pointer itself, causing it to point to any other segment of memory which Busybox can access - then when the time comes to copy/move ~/.ash_history.$PID.new over to ~/.ash_history, bam, your FILE* is no longer pointing at ~/.ash_history.$PID.new, but may very well be pointing at one file in the OS or another.
Meh, all of the above is still speculation until I've looked at the code, but I think it's a likely explanation.
| The Following 4 Users Say Thank You to Mentalist Traceur For This Useful Post: | ||
Initial thumb ISA support is also available in busybox-power's source tree, if you want to compile it yourself. You'll need a thumb-enabled target in Scratchbox (instructions). Once the target is set up, compiling is as simple as running `sh build.sh thumb` from your local busybox-power git repository.
Furthermore, a new busybox-power version has been pushed to Maemo's repositories last Sunday. It's mostly a bugfix release, incorporating upstream's hotfixes, but it also has a nice new feature for the vi users amongst us: when vi is ran in a terminal emulator (i.e. X Terminal), your terminal's output is restored after closing vi.
Lastly, I must say that the transition from BusyBox' 1.19.x to 1.20.x series went extremely smooth; there have been zero new issues reported since busybox-power 1.20.0power1 has been released!
-
As of now there is just one issue with busybox-power: when ash is killed -for the lack of a better term- "unnicely", there is a small chance ~/.ash_history could get garbled up. I still can't find any problematic code in the source, any help is greatly appreciated. Look for save_history() in libbb/lineedit.c in upstream's latest stable BusyBox release. Unless someone else finds an issue in the source, I'm assuming the issue is not related to BusyBox itself.
Currently, history saving is handled as follows: all new history is appended to ~/.ash_history. After that, the history file gets reloaded via load_history(), which effectively trims it to n lines (n = MAX_HISTORY). Finally, the newly trimmed history is written out to ~/.ash_history.$PID.new, which will be moved to ~/.ash_history upon finishing writing it out.
Last edited by iDont; 2012-06-26 at 18:30.