gcc-4.6 and Fremantle, WIP
 

This is about c-compiler gcc. Discussion have started on other threads off-topic, it might be better to have one dedicated.

Release dates:
GCC 4.2.1 release (2007-06-18)
GCC 4.2.4 release (2008-05-19)
GCC 4.4.6 release (2011-04-16)
GCC 4.6.2 release (2011-08-26)

Maemo5 has gcc-4.2.1. We have already encountered programs, which need newer compiler. And there are some bug fixed on ARM-version of gcc-4.4 (atomic types are in 4.2 but actually not working on ARM until 4.4). Third reason is 4.6 has and will have more optimization on ARM (by e.g. Linaro, http://www.linaro.org)

Some developers are already using Tom Tanner's gcc-4.4 (
http://n900.tannerlab.com/pool/free/g/gcc-4.4/ ), but it is not in extras (-devel).

--
I have started to push new compiler to the extras-devel. There are needed libraries and newer tools (e.g. bison-2.4 and make-3.81) on repository (and autobuilder can use them). Seems to me that 4.2 can compile 4.6, so we can skip 4.4.

Some random thoughts:
*On i386-target multilib should not be used.
*Virtualized/emulated environments lacks RAM/SWAP, so -j1 might be the only way (I think issue is generated insn-attrtab.c)
*It might be must to use --disable-bootstrap (bootstrap="compiler test can it compile itself during process" (seems to be memory issue))
*I had some problems to not compile java

---
When program is compiled with newer gcc, it needs also newer libstdc++ (Is this true with only c++ applications?) .Do old programs work with new libstdc++ or should there be both then installed on device? (until we can recompile every program)
---
Should this be packages gcc-4.6 (building) and libstdc++6-4.6 (runtime)? Or could this overwrite gcc and libstdc++6?
---
This is Work In Progress, join discussion if you have something to contribute.

----------------
Edit:
Reason's for newer compiler:
*There are more and more applications which can't be built with old compiler, due bugs in old compiler or new features on new compiler. This is proven to be true. (But we can still claim that those new applications are not needed, or we can port them to use old compiler)
*Possible better optimization: E.g. Linaro is making gcc 'better' for ARM speaking code size and execution speed. It is not (yet) proved that this happens on N900.
*THUMB2
http://www.cnx-software.com/2011/04/...e-performance/
(I have not seen tests for N900)
*NEON (whatever this means...)

Re: gcc-4.6 and Fremantle, WIP
 

Can it be assumed that...
- there are a number of bug fixes i.e. code compiled with 4.6 is more likely to behave as expected compared with code compiled with 4.2?
- there are some optimisations so code compiled with 4.6 should run faster than that compiled with 4.2?

These might seem unnecessary questions but I'd be interested to have clear answers all the same:
- can maemo make full use of software compiled with gcc/g++ 4.6?
- are additional/alternative libraries needed to run software compiled with 4.6 compared with 4.2? (seen note re: libc - anything else?)
- what happens when linking in other, existing libraries, frameworks or dependencies?
- if multiple library versions are required, will they fit on the device?

Re: gcc-4.6 and Fremantle, WIP
 

I have scratchbox (armel target) with gcc 4.6.2 and latest binuils which I used for compiling thumb2 binaries (Qt, gtk, ...). Newer libstdc++ should be copied to n900, but it seems there are no problems caused by that.

EDIT:

I compiled gcc and binutils outside scratchbox (statically linked) and setup a new target using /scratchbox/compilers/(gcc462_dir)

And so far the only new library needed is libstdc++

Re: gcc-4.6 and Fremantle, WIP
 

Any chances for CSSU interest in this? I may be wrong, but it seems like perfect candidate for CSSU, even more than any Qt thing.

/Estel

Re: gcc-4.6 and Fremantle, WIP
 

i have a new libstdc++ from debian repo
and it's working fine
i need it for launching online game servers

but do we need to compile a new one ?

Re: gcc-4.6 and Fremantle, WIP
 

gcc is only a compiler, so whether you use 4.2 or 4.6 should be irrelevant for the program you're compiling, so it will work equally good or bad :)

The only issue is with libraries that are internal to gcc (such as libstdc++).

The problem would be if you now wanted to upgrade to a newer glibc, but that's another story...

Re: gcc-4.6 and Fremantle, WIP
 

Incorrect, gcc 4.2.1 for ARM has bugs which are resolved in newer versions. Look at the case with CSSU, Qt 4.7.4 and raster engine - it ended that it is -O3 flag that breaks it. And newer gcc and binutils have workarounds for ARM erratas, better vectoriser (for NEON), produce more optimized code, etc...

Re: gcc-4.6 and Fremantle, WIP
 

So far never had problems with programs compiled with Tanner's 4.4 (also no reports of programs failing due to libstdc++ incompatibility, but sample size is unreliable at best). It does seem that bugs from 4.2 are quite common and workarounds for those are non-trivial (in some cases -Ox does impact succesfull compilation still).

@Estel: unlikely this would get full attention from CSSU team, as binutils (and rest of compilation packages) is not part of standard system setup. Currently it is even required to add 'risky' SDK repos to get 4.2 running.

Re: gcc-4.6 and Fremantle, WIP
 

Still, what I wrote is strictly correct. A compiler is only a compiler :)

If a new version of a compiler provides better optimization, etc. then a new recompilation of a program will perhaps work faster, but not differently. If a new version of a compiler generates code that is *not* compatible with other programs compiled with other versions of the compiler or even with other compilers (who says everything has to be gcc?), then something's fckd up with the compiler.

Obviously, I know you're talking about thumb2 and neon. But these are just optimization options which should work regardless of the underlying system (as long as the processor supports those instruction sets).

Using aggresive optimization options is something that has to be done VERY carefully. Of course a compiler should alwayws produce correct code, but in the real world this is not the case.

I think -O3 enables the -fstrict-aliasing option (or at least it used to do that), and enabling that option implies that *you* (developer) make *damn* sure that your program is OK with that.

I can bet a year's a salary (OK, make it a month) that the combination of CSSU, QT 4.7.4 and whatever raster engine you were compiler DOES NOT fulfill that (very strict) restriction, meaning that using strict aliasing is an unsafe option.

Re: gcc-4.6 and Fremantle, WIP
 

Wait, why should we use -O3 by default? Many if not all distros frown upon this, as the extra flags break some programs (explained on the Gentoo wiki iirc).

Anyways, GCC 4.6 is a stable release, by all means yes, we should push for this and packages to be recompiled with this (plus NEON, afaik THUMB is broken on the N900's SoC, and the workaround's overhead > benefit of THUMB)

While we're on the subject, why not upgrade other open components from upstream?

Re: gcc-4.6 and Fremantle, WIP
 

Yes, for example Tracker, PulseAudio, Dbus... some of the components could bring bug fixes or better performance, while others could bring new features (and new bugs!) to play with.

Re: gcc-4.6 and Fremantle, WIP
 

[OT]
Will you be more specific on the above, i.e. which workaround overhead aganst which benefit. I will appreciate any data on the matter.
[/OT]

Re: gcc-4.6 and Fremantle, WIP
 

The overhead for the 430973 bug against Thumb's purported reduced code size.

Re: gcc-4.6 and Fremantle, WIP
 

Not all packages can be automatically recompiled with gcc-4.6, there are some stricter rules which needs hand tuning.
---

How much used binutis affects resulting binary (size/speed)? I made test compilation with gcc-4.6.1 and with old binutils and I got bigger and slower binaries (run on N900) than just using old compiler.

What are needed flags for thumb et cetera for compiling most optimized (as speed e.g.) code?


I have all ready started this, libglib: http://talk.maemo.org/showthread.php?p=1155248

About *programs*: some programs has new features on upstream and we could just take and use them (I would say we should do it). But new versions can be using new version of libraries, so we must upgrade libraries too.

Some libraries we can just upgrade from upstream (so just do it). But some libraries aren't backward compatible and they cause recompilation of packages. And if some of them are closed source, we must drop that package. There are 355 packages waiting for alternatives (or judgement to be dropped entirely): http://wiki.maemo.org/Fremantle_closed_packages

Is there something in non-free extras we should worry?

Re: gcc-4.6 and Fremantle, WIP
 

linkbot: http://cateee.net/lkddb/web-lkddb/AR...TA_430973.html

Q
 

Well, I am pretty aware of that errata, I was asking for some real data, not what your expectations are.

In short - what is the overhead from workaround for ARM errata 430973.

Because I can give some hard values for thumb2:

from only Qt, gtk, hildon-desktop, hildon-home and hildon-status-menu thumb2 compiled the reduction of code size is exactly 19MB. If you wish I can search on #maemo-ssu for how is memory consumption reduced (for several Qt/QML applications), but it varies between 2 and 6 %. And from my personal experience helium-moble-browser is much faster and less laggy when run on top of thumb2 compiled Qt.

Have in mind that I am not arguing whether it is better to have thumb2 (with errata workaround) or not, I am missing data to do the evaluation, thus my question.

@AapoRantalainen - if you feel this is totally OT, my apologies, I will start a new thread, just say it.

Re: gcc-4.6 and Fremantle, WIP
 

Can't vouch for Aapo, but as long this is on-topic in regards to optimizations (available/possible in near future) and compiling seems totally fine for me. For on-device compilators this would be an awesome improvement - more space for libs, more memory (less restarts when compiling things that take >700mb of memory).
Having no-clue really about the underlying mechanics, but the few pdfs out there do hint at ~20% more space (or ~6% with speed increase), so would love to see this happen (all the above posts point to this being enormous project though requiring refresh of the whole system). If someone has an idea how to get this done, please share. I am willing to test all the closed packages compatibility with newer libstdc++/gcc, if someone knows how to approach this.

Re: gcc-4.6 and Fremantle, WIP
 

Here is some results, I hope somebody could give some hint.

--
Binutils.
Seems recent binutils is needed and will not itself has hard dependencies, so I started from it.
Binutils-2.22 from debian packed fine
modifications/configured:
* with_multiarch := no (compiling for i386 will be easier)
* # with_gold = yes (there are bug in gcc-4.2 and it can't compile gold (internal compiler error). "gold = A new, faster, ELF only linker, still in beta test." Seems to be optional).
* using 'tar -c -z' instead of 'tar -c --xz' (Maemo's tar can't handle xz)

Packages can be installed and they seems work, but linking causes warnings:
/usr/bin/ld: BFD (GNU Binutils for Debian) 2.22 assertion fail ../../bfd/elf32-arm.c:11467
which is:
And sometimes I must add "-lm" even sources were compiled earlier without it.

Linked binaries work on device.

-----------
Gcc
gcc-4.6.1-9 from Ubuntu 11.10.
Modifications:
*version's of dependencies dropped (seems ubuntu is over rated them)
dpkg-dev (1.16.0 -> 1.14.15)
gcc (4.4 -> 4.2)
libmpfr (3.0.0 -> 2.4.2)
libgmp-dev 5.0.1 -> libgmp3-dev 4.3.1
libelf0 0.8.12 -> 0.8.10
doxygen (1.7.2 -> 1.5.1)

*multi_arch =no
*uses binutils-2.22)

Packages can be installed, compilation works, but linking NOT. There are many errors like this:
File /usr/include/sys/sysmacros.h is coming from libc6-dev.

---------
Libc
eglibc-2.13-20 from Ubuntu 11.10
Modifications:
* Maemo's patch doesn't understand: " patch: unrecognized option `--reject-format=unified' "
* Some dependencies were too high

Compilation fails with rpcgen. According to the FAQ

-----------
So do I must compile libc6 also? Doest this cause harm when running on device? Is my compilation order correct?

Re: gcc-4.6 and Fremantle, WIP
 

can't recall off the top of my head. normally i rebuild them only when creating a toolchain for building distros, not updating an existing one. having said that, i would still use the same order you are using. with respect to bootstrapping, other than comparing builds, it helps remove traces of old gcc from build by compiling itslef with newly generated binary:
1st build: gcc 4.6 built with 4.2
2nd build: 4.6 built with 4.6 built with 4.2
3rd build 4.6 built with 4.6 built with 4.6

are you sure you're making use of the new binutils?

libc6 (provided in glibc) in ubuntu used to depend on libgcc1, might still do, so you might have circular dependancies using current build method. what happens if you try to build+boostrap just gcc c compiler, then come back and do c and c++ without bootstrap after glibc?

make sure libtool using correct paths after gcc build. it might still be hardcoded to old version of gcc.

Re: gcc-4.6 and Fremantle, WIP
 

I installed new binutils and new gcc and checked gcc-4.6 --version -v
if there are anything suspicious (or even not-perfect) in configurations, please comment.

Seems I'm doing this already:
compiled by GNU C version 4.6.1, GMP version 4.3.2, MPFR version 2.4.2-p1, MPC version 0.9

GNU assembler (GNU Binutils for Debian) 2.22
GNU ld (GNU Binutils for Debian) 2.22

And this gcc-packaging uses 'ld --sysroot', which is not working with old binutils, so I'm sure it is compiled with new binutils.
(It is possible to compile gcc-4.6 with binutils-2.18, just disabling --sysroot. Truly speaking I do not know what this means.)


libtool is pointing to the gcc. I have /usr/bin/gcc -> gcc-4.6
(and g++ -> g++-4.6)
--
I made more application tests with gcc-4.6 and binutils-2.22
A) multifile complex c-application which is using sin (libm.so)

B) c-application without sin:
works

c) c++ application without sin:
works

d) naive sin-test: works:
---
So I'm still wondering is there need for compiling libc6?

My fault was start testing new compiler with huge and complex application. It caused lots of warning relating /usr/include/sys/stat.h and libm.so, which come from libc6-dev and libc6. (Same test compiles under Ubuntu with gcc-4.6 so it is not just deprecation of gcc-4.2). Maybe I should just compile everything lying on hard disk with gcc-4.6 and collect similarities from every failing applications.

Re: gcc-4.6 and Fremantle, WIP
 

i'm wondering if there was a change between gcc 4.4 and 4.6. there are a few libm related issues floating, google turns up a few reports about missing libm. Maybe it was previously assumed and automatically added.

Re: gcc-4.6 and Fremantle, WIP
 

So far it seems that -lm is needed in place where earlier doesn't needed.

--
About multiple definition of `gnu_dev_major' etc. It is not warning, but error.

I found this post which I think be the most relevant:
http://us.generation-nt.com/answer/g...170398461.html

Summary:
"
The glibc support for the C99 inline semantics was added in glibc-2.6. Gcc-4.3 and later use C99 inline semantics in -std=gnu99 mode. To use any earlier version of glibc with Gcc-4.3 you need at least to backport the large patch.
"
(We have: libc6: 2.5.1-1eglibc27+0m5)

'-std=gnu99 mode' doesn't sound bad, but seems to me that it is hard to know if using some libraries triggers this.

Re: gcc-4.6 and Fremantle, WIP
 

apologies for needing to ask, but if I write the following, won't it work?
Are we allowed to use inline to write code for Maemo5? Maybe I'm missing your point? Why does do we need more recent compiler and if you convince us, what are its dependencies/pre-requesites?

Re: gcc-4.6 and Fremantle, WIP
 

Reasons are multiple, but from my perspective the most important one is 4.2 has a lot of bugs. If you try to compile most of latest libtcod using programs for example you end up having to rewrite lots of code. 4.3 and above compile without any problems (at least for libtcod, guessing 4.6 will fix even more). Autobuilder for repos uses 4.2 so porting is not easy using SDK build-essentials package. If 4.6 got into the repos you could then use it to build packages. With time more and more packages will be unbuildable on 4.2, so this will prolong N900 life immensely (until we hit a wall when newer libc/libstdc++ and rest is needed which would require updating whole maemo, which in turn with closed source components might be impossible)

Re: gcc-4.6 and Fremantle, WIP
 

I just tested to recompile some packages and they can't be compiled with newer compiler, and I'm struggling what is problem. There are also packages I haven't wrote and I have no idea are they using inlines, or are there some libraries using inlines.

Good question, I updated first post. Shortly: a) some applications need it. b) It *may* be 'better'.

I'm just solving this locally, and I'm planning to push them to the extras-devel:
make3.81 (extras-devel)
bison-2.4 (extras-devel)
binutils-2.22 (I have locally working, and maybe could be omitted)
-> gcc-4.6 can be compiled and it 'sometimes works' (as I wrote earlier).

We might need new libc6 for really using new compiler. Seems it can compile some packages, but not all. (But maybe there are no need for compile all?)

Re: gcc-4.6 and Fremantle, WIP
 

If it works, why not. Have you also seen http://stackoverflow.com/questions/4...e-math-library

According to this one: http://stackoverflow.com/questions/1...h-library-in-c the C++ runtime libstdc++ requires libm, so if you compile a C++ program with GCC (g++), you will automatically get libm linked in.

Finally, http://stackoverflow.com/questions/7...n-ubuntu-11-10 some versions require -lm to be in a certain place in command sequence or it could be because a default flag in the linker.

Theres another interesting piece of information on that page

Does the old version of GCC have TLS enabled?

Re: gcc-4.6 and Fremantle, WIP
 

It seems some other libraries need also manually added, e.g. with gcc-4.2 -lSDL_net was enough, now it must be -lSDL_net -lSDL.

gcc-4.2 -v --version doesn't response enough. I don't know answer or way to solve it.

Re: gcc-4.6 and Fremantle, WIP
 

Finally got around to reading through this entire thread, so now a question:

Why is there hesitance to recompile and upgrade libc6? I know just about every C program depends on it, but would the version difference be enough to break that much? If that's unknown, I'd be happy to use one of my N900s as a testing device (I have two, one used just for development/testing, that I'm happy to reflash).

Also, I just realized that this is why newest Subversion versions were segfaulting on the N900 - the default config/make configuration was to compile with NEON support (and it worked when you compiled without NEON), which judging by comments on this thread, was a feature that wasn't added in until a more recent version of GCC for this platform that what is currently default.

Re: gcc-4.6 and Fremantle, WIP
 

Building gcc-4.6 on autobuilder might be near impossible.
Builder won't build package with same name on SDK, e.g binutils. "Package binutils provides binary package binutils which is also available on the device or Nokia repository. Build of this package has been prevented."

Even there are package bison-2.4 which replaces bison, autobuilder wont' install it.
"E: Packages need to be removed but remove is disabled."
Solution: package 'bison-2.4' doesn't conflicts with 'bison' but contains file 'bison-2.4', and it must be called by that name.

There are now binutils2.22 on extras, and it will replace binutils, so builder won't install it -> there must be ld-2.22, ar-2.22 and so on....

**
Another finding: gcc-4.2 depends binutils *with* version, so there must be package named exactly binutils, providing it is not enough. (7.5 on http://www.debian.org/doc/debian-pol...tionships.html ). So gcc-4.2 and binutils2.22 can't be installed same time (and that was exactly my plan to compile gcc-4.6 on builder).

***

My goal was make this transparently and I though autobuilder is transparent; public build-logs, accessible source and binary packages.


Any ideas what is the second best way? Locally built and dumb to the non-free section (autobuilder might then could use gcc-4.6, but not binutils2-22)? How to get builder's policy changed? How to get builder working with CSSU-repository?

Re: gcc-4.6 and Fremantle, WIP
 

i think the reluctance is only beacuse of the risk of breaking dynamically linked closed source apps. for desktop some used to come with their own version of libc and run using a modified path to ensure its use (maybe original quake 3?). as long as it don't break compatibility, by all means testing would be much appreciated if you have the device to spare.

with regards to autobuilder, you know more than me. maybe its time for cssu-sdk repo :)

Re: gcc-4.6 and Fremantle, WIP
 

Maybe go Tanner's way and just use a private repository? Marmistrz would definitely host it, maybe CSSU repo too? Not sure how transparent it would be, but including compilable source would definitely help in that matter (don't trust .deb, compile it yourself).

Re: gcc-4.6 and Fremantle, WIP
 

My dream was just keep pushing sources of interesting applications to the autobuilder (where they get compiled with new compiler) and binary packages go to the extras-devel.

Currently there are no way to accomplish this.

Re: gcc-4.6 and Fremantle, WIP
 

keep on building outside of autobuilder for now then and post links to files here. in the mean time try contacting some of the admins/council to see if they have a solution. When you have a stable, working set of libraries it would put you in a better position to get them included, however it depends on how much control they have over autobuilder.

reason i mentioned cssu before was i remember talk about rebuilding a number of packages with thumb2 support when the kernel support is fixed. this would be a good use of your packages to bring in more recent compiler optimizations.

have you tried contacting pali or freemangordan, to see if they have any ideas. remember something similar with powertop.

Re: gcc-4.6 and Fremantle, WIP
 

Deb-packages for armel-maemo5, bintutils 2.22 and gcc 4.6
http://cc.oulu.fi/~rantalai/fremantle/

Install binutils:
Install gcc:
Symlink gcc is still pointing to gcc-4.2 (and g++ to g++-4.2). Use e.g. CC=gcc-4.6 or tune symlinks
-----------------
How is binutils made:
-------------
How is gcc made:

Re: gcc-4.6 and Fremantle, WIP
 

and this is supposed to be installed in scratchbox?

Re: gcc-4.6 and Fremantle, WIP
 

This is native ARM to ARM compiler. It works on scratchbox, but if you have working cross-compilation, this is not optimal solution.

It works on sb2 without hazzling.

Downloading packages:
On sb1, it needs some tricks to get just installed binaries to work, because they are pointing to the sb_gcc_wrapper. Seems to me it needs tuning outside of scratchbox, something like this (for every binary?):
And installing on sb1 differs from sb2: gcc-4.6-base must be forced (I don't understand, they both have dpkg 1.14.25maemo3+0m5 )

Re: gcc-4.6 and Fremantle, WIP
 

"This is the error that happens when trying to install a partially
downloaded package."



You missed one row:
But test first:
Or
Or if you are not interested in to get it packaged, but only compiled, you can try: download source code.
unpack source code
cd gcc*
mkdir build
cd build
../configure
make

(Yes, it must be configured and built out of the source tree).

Re: gcc-4.6 and Fremantle, WIP
 

Removed pointless posts, after DLing correctly and following all steps: It works!
Compiled cataclysm, no warnings/errors, but on linking:

g++-4.6 -o cataclysm -g obj/artifact.o obj/bionics.o obj/bodypart.o obj/calendar.o obj/catacurse.o obj/color.o obj/computer.o obj/construction.o obj/crafting.o obj/event.o obj/faction.o obj/field.o obj/game.o obj/help.o obj/inventory.o obj/inventory_ui.o obj/item.o obj/itypedef.o obj/iuse.o obj/keypress.o obj/line.o obj/main.o obj/map.o obj/mapgen.o obj/mapitemsdef.o obj/melee.o obj/mission.o obj/missiondef.o obj/mission_end.o obj/mission_fail.o obj/mission_place.o obj/mission_start.o obj/monattack.o obj/mondeath.o obj/mongroupdef.o obj/monitemsdef.o obj/monmove.o obj/monster.o obj/mtypedef.o obj/mutation.o obj/mutation_data.o obj/newcharacter.o obj/npc.o obj/npcmove.o obj/npctalk.o obj/output.o obj/overmap.o obj/player.o obj/posix_time.o obj/ranged.o obj/rng.o obj/settlement.o obj/setvector.o obj/skill.o obj/texthash.o obj/trapdef.o obj/trapfunc.o obj/weather.o obj/wish.o -lncurses
/usr/bin/ld: BFD (GNU Binutils for Debian) 2.22 assertion fail ../../bfd/elf32-arm.c:11467
/usr/bin/ld: BFD (GNU Binutils for Debian) 2.22 assertion fail ../../bfd/elf32-arm.c:11467
/usr/bin/ld: BFD (GNU Binutils for Debian) 2.22 assertion fail ../../bfd/elf32-arm.c:11467

these 3 assertion fails showed up. Binary is there and runs ok, so not sure what to think about them. Just thought would let you know.
THANK YOU!

Re: gcc-4.6 and Fremantle, WIP
 

Just compiled TRN (libtcod 1.5.0 based SDL tron/roguelike(?) game), above 3 fails happen on both CC=g++ (4.4) and g++-4.6, both builds successful though, so just annoying warning it seems. BTW, any chance this can be optified? I vaguely remember Tanner having problem with symlinks, not all could be moved (libs mostly IIRC), but it's a 35mb of precious rootfs space. Going to try symlinking and check results, part of it definitely can be moved.

Re: gcc-4.6 and Fremantle, WIP
 

Then again no need to reinvent the wheel, here's [2|user@Nokia-N900|/usr/lib/gcc/arm-linux-gnueabi] 4.4 (~50kb) vs 4.6 (35mb):
4.6 seems to have a few more files though.

EDIT: hope [code] tag works as the above was a mess, attached recursive ls result