Reply
Thread Tools
Capt'n Corrupt's Avatar
Posts: 3,524 | Thanked: 2,958 times | Joined on Oct 2007 @ Delta Quadrant
#161
Originally Posted by attila77 View Post
QPointer and friends *sort of* do this. My code is mostly devoid of class* thingies because it's easier and safer to use Qt's smart pointer classes. While this does not do object movement in memory (*now* - as said above, benefits of that can be ambiguous), it does affect memory management, as it provides reference counting, CoW operations, shared data, etc.
Smart pointers sound interesting, I will look into this class. QPointers are also interesting. However, my idea was to mimic the pointer system via class that controlled a contiguous block (or blocks) and methods for controlling it. This is somewhat different than guarded pointers.

Out of curiosity (and don't answer if you don't want to), what type of programming do you do?
 
Posts: 3,319 | Thanked: 5,610 times | Joined on Aug 2008 @ Finland
#162
Originally Posted by Capt'n Corrupt View Post
Smart pointers sound interesting, I will look into this class. QPointers are also interesting. However, my idea was to mimic the pointer system via class that controlled a contiguous block (or blocks) and methods for controlling it. This is somewhat different than guarded pointers.
Yes, the 'base' QPointer is just a template to make a guarded pointer, but QShared*Pointers are very close to what you describe - they are fully aware of the memory (sizes) assigned to them and are able to allocate/free/reallocate - so while not necessarily the same thing (I did say 'sorf of' ) it shows that at least the general approach is applicable.

Out of curiosity (and don't answer if you don't want to), what type of programming do you do?
Type as in... ? I do all sorts of stuff, from system level to applications (mostly Qt), from C++ to Python...
__________________
Blogging about mobile linux - The Penguin Moves!
Maintainer of PyQt (see introduction and docs), AppWatch, QuickBrownFox, etc
 

The Following 3 Users Say Thank You to attila77 For This Useful Post:
Posts: 1,341 | Thanked: 708 times | Joined on Feb 2010
#163
Originally Posted by attila77 View Post
Which brings things us to the (repeated) bottom line - C++ as such does not *require* a certain type of memory management, it's not part of the language, so if you have a fancier, smarter way of dealing with allocated memory, give it a go.
The fancier smarter way, like defragmentation when needed, will be more inefficient in C++ than with Java JIT. Java JIT can use real pointers in its JIT'ed code, but C++ with a "fancy" memory management should have all pointers then indirect (or do code morphing). When GC in JVM notices there is fragmentation or the ratio of pagefaults has increased, it can re-arrange the objects and re-JIT the affected hotspots.
 
Posts: 376 | Thanked: 511 times | Joined on Aug 2009 @ Greece
#164
Originally Posted by zimon View Post
The fancier smarter way, like defragmentation when needed,
For shake... Do you have any clue *when* or *where* memory defragmentation is needed when using C/C++? Do you have at least one linux-based use case? (we're still talking about Maemo and Android and thus we're talking about Linux... right?).

Sorry for sounding offensive. I needed to emphasize that a bit since (to my knowledge) this conversation is based on a claim (memory loss/usage because of fragmentation) that is wrong.

In fact, it is Java that may experience the memory fragmentation problem.
 
Posts: 1,341 | Thanked: 708 times | Joined on Feb 2010
#165
Read for example this:
http://en.wikipedia.org/wiki/Garbage...vs._non-moving
At first, a moving GC strategy may seem inefficient and costly compared to the non-moving approach, since much more work would appear to be required on each cycle. In fact, however, the moving GC strategy leads to several performance advantages, both during the garbage collection cycle itself and during actual program execution:

* No additional work is required to reclaim the space freed by dead objects; the entire region of memory from which reachable objects were moved can be considered free space. In contrast, a non-moving GC must visit each unreachable object and somehow record that the memory it alone occupied is available.
* Similarly, new objects can be allocated very quickly. Since large contiguous regions of memory are usually made available by the moving GC strategy, new objects can be allocated by simply incrementing a 'free memory' pointer. A non-moving strategy may, after some time, lead to a heavily fragmented heap, requiring expensive consultation of "free lists" of small available blocks of memory in order to allocate new objects.
* If an appropriate traversal order is used (such as cdr-first for list conses), objects that refer to each other frequently can be moved very close to each other in memory, increasing the likelihood that they will be located in the same cache line or virtual memory page. This can significantly speed up access to these objects through these references.
And in all OOP-programs which have many variable size variable life time objects, you cannot really archive both high object locality and small fragmentation ratio, unless you use some kind of "moving-GC".

The usefull method to try to keep heap memory fragmentation small, is to use different memory pools for different sizes of objects, but with that method you easily loose the locality of the objects and get more pagefaults because in some hotspot needed objects are scattered around in different memory pools.

I wonder have you really read the URIs given in the start of this thread where they talk about opimizing VM, GC and why eventually Java can (and will) be faster than C++.
 
Posts: 3,319 | Thanked: 5,610 times | Joined on Aug 2008 @ Finland
#166
Originally Posted by zimon View Post
The fancier smarter way, like defragmentation when needed, will be more inefficient in C++ than with Java JIT. Java JIT can use real pointers in its JIT'ed code, but C++ with a "fancy" memory management should have all pointers then indirect (or do code morphing). When GC in JVM notices there is fragmentation or the ratio of pagefaults has increased, it can re-arrange the objects and re-JIT the affected hotspots.
I understand what you say and disagree. Sorry. C++ can also be JIT-ed, garbage collected, and VM memory management is by no means a holy grail (if nothing else, the VM does not know about physical memory layout and architecture, which will always be an issue). C++ has moved past the '80s and Stroustrup, honest, and mobile contexts are a fairly unexplored area for JIT and proactive fancypants memory management.
__________________
Blogging about mobile linux - The Penguin Moves!
Maintainer of PyQt (see introduction and docs), AppWatch, QuickBrownFox, etc
 
Posts: 1,341 | Thanked: 708 times | Joined on Feb 2010
#167
Originally Posted by attila77 View Post
Sorry. C++ can also be JIT-ed, garbage collected, and VM memory management is by no means a holy grail (if nothing else, the VM does not know about physical memory layout and architecture, which will always be an issue). C++ has moved past the '80s and Stroustrup, honest, and mobile contexts are a fairly unexplored area for JIT and proactive fancypants memory management.
Why VM wouldn't know about physical memory layout and architecture? There is not reason VM couldn't be clever like that. In the (in)famous "Java will be faster and C++" tells also that VM can also know how many and what kind of processors (how many registers, what is the size of L2 cache) the platform where it is running has. It can monitor page fault rate especially in multi-core platforms or if a program is interactive and sometimes is not doing anything.

On the other hand, fully compiled C++ code cannot be optimized so well because it has to fit to some minimum common architecture or have multiple compilations. Before also in this thread was an example why C++ compiler cannot do register allocations in some cases where Java JIT can. Same goes if there would be different number of registers on platforms the code is compiled for. Java VM can know how many registers and JIT the bytecode to use all available registers efficiently.

Of course C++ can be also JITed and interpreted, but then why use C++ after that when there is Java which seem to be more productive than C++, easier to maintain, easier to read, easier to fix and find bugs?

It is probably true, not yet for example Dalvik has all these fancy optimizations, but if it lives long enough it will, especially now when mobile platforms have started to get multiple CPU cores.
 
Posts: 376 | Thanked: 511 times | Joined on Aug 2009 @ Greece
#168
Originally Posted by zimon View Post
Read for example this:
And in all OOP-programs which have many variable size variable life time objects, you cannot really archive both high object locality and small fragmentation ratio, unless you use some kind of "moving-GC".
Assuming that this is a reply to my post regarding memory fragmentation (please include a part of the original message or use @):

Where do you base the above claim?

Please re-read and consider this post when talking about memory fragmentation in C.

Can you provide a sample program in C/C++ that doesn't implement its own memory management and memory becomes fragmented when compiled under a modern linux/glibc system?
 
Posts: 1,341 | Thanked: 708 times | Joined on Feb 2010
#169
@v13: I think we are talking about little different heap memory fragmentation problems and effects here. I am mainly concerned (in this Java vs C++ issue) the increased pagefaults due to increasing heap memory fragmentation in long living OOP programs like www-browser. The problem with fragmentation is that locality of the objects is getting worse.

I do see Firefox still taking too much available RAM when it is runnign long time, but that is not why I restart it every few days, because the machine do not start swapping yet. But firefox seem to get slow, and restarting it makes a real difference. I notice this also if I disable flash. Javascript engine in Firefox has pretty good GC nowadays, so I don't think that is a problem but C++ memory management itself.
 
Posts: 376 | Thanked: 511 times | Joined on Aug 2009 @ Greece
#170
Originally Posted by zimon View Post
@v13: I think we are talking about little different heap memory fragmentation problems and effects here. I am mainly concerned (in this Java vs C++ issue) the increased pagefaults due to increasing heap memory fragmentation in long living OOP programs like www-browser. The problem with fragmentation is that locality of the objects is getting worse.
I believe that we're talking about the same thing (but I may be wrong). First, I don't understand how pagefaults are related to memory fragmentation. In fact, I'd think that pagefaults are an inverse function of memory fragmentation since with fragmentation it would be possible to reuse the same memory without needing a pagefault to extend the process's VM.

Originally Posted by zimon View Post
I do see Firefox still taking too much available RAM when it is runnign long time, but that is not why I restart it every few days, because the machine do not start swapping yet. But firefox seem to get slow, and restarting it makes a real difference. I notice this also if I disable flash. Javascript engine in Firefox has pretty good GC nowadays, so I don't think that is a problem but C++ memory management itself.
As it was mentioned on other posts, Firefox implements its own memory management and thus it suffers from memory fragmentation. It is really a bad example since (to my knowledge) it doesn't use the underlying OS's libraries for memory management.

Also, since Firefox is a really really complex and very big program, you should not use it as a reference. At least not unless you compare it with a browser written in java.

So the question stands: Do you have any (comparable with java) example of memory fragmentation in C under Linux?
 
Reply

Tags
bada rox, dalvik, future, java haters, meego, meego?fail, nokia, sandbox sucks


 
Forum Jump


All times are GMT. The time now is 13:53.