Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[lethosor]

DFHack already distributes a libstdc++ from GCC 4.5.4. I've been able to use a libstdc++ from GCC 4.8, 4.9, and 5.1 without issues, as far as I can tell, so they seem to be compatible with older compilers. The issue right now is that nothing newer than GCC 4.5 can be used to produce a distributable build, since end users don't have a new enough libstdc++.

[mifki]

DFHack already distributes a libstdc++ from GCC 4.5.4. I've been able to use a libstdc++ from GCC 4.8, 4.9, and 5.1 without issues, as far as I can tell, so they seem to be compatible with older compilers. The issue right now is that nothing newer than GCC 4.5 can be used to produce a distributable build, since end users don't have a new enough libstdc++.

I don't understand how "end users don't have a new enough libstdc++" is a problem if "DFHack already distributes a libstdc++" anyway.

[lethosor]

DFHack (and DF) already distributes a libstdc++ from GCC 4.5. TheBloke was able to compile GCC 4.9 on OS X 10.11, but the resulting DFHack build can't be distributed because end users don't have a new enough libstdc++ (libstdc++ from GCC 4.5 won't work with code compiled with GCC 4.9).
A libstdc++ from GCC 4.9, however, should work with anything compiled by GCC <= 4.9, but requires some extra effort to work on older OS X versions.

[TheBloke]

I'm still confused too - if we distribute the libstdc++ from 4.9 with a DFHack compiled in 4.9, what breaks?    If an end user on, let's say, 10.8, adds DFHack-with-libstdc++-from-4.9 to his DF, what happens?

Because if we're supplyin the libstdc++ from 4.9, why does it matter what the end user has?  We're already giving him the only file he needs, I thought?

[mifki]

A libstdc++ from GCC 4.9, however, should work with anything compiled by GCC <= 4.9, but requires some extra effort to work on older OS X versions.

Are you sure?

Code: [Select]

Vit-iMac:df4024 vit$ cp /usr/local/Cellar/gcc49/4.9.3/lib/gcc/4.9/libstdc++.6.dylib ./libs/
Vit-iMac:df4024 vit$ ./df
Vit-iMac:df4024 vit$ dyld: Symbol not found: __ZNSt8__detail12__prime_listE
  Referenced from: /Volumes/df/df4024/./dwarfort.exe
  Expected in: /usr/lib/libstdc++.6.dylib


[TheBloke]

A libstdc++ from GCC 4.9, however, should work with anything compiled by GCC <= 4.9, but requires some extra effort to work on older OS X versions.

Are you sure?

Code: [Select]

Vit-iMac:df4024 vit$ cp /usr/local/Cellar/gcc49/4.9.3/lib/gcc/4.9/libstdc++.6.dylib ./libs/
Vit-iMac:df4024 vit$ ./df
Vit-iMac:df4024 vit$ dyld: Symbol not found: __ZNSt8__detail12__prime_listE
  Referenced from: /Volumes/df/df4024/./dwarfort.exe
  Expected in: /usr/lib/libstdc++.6.dylib


However:

Code: [Select]

tomj@Marvin ~/games/df_osx $ ls -al hack/libstdc*                                                                                                                 [ruby-2.1.3p242]
lrwxr-xr-x  1 tomj  staff    64B 24 Nov 04:19 hack/libstdc++.6.dylib -> /usr/local/Cellar/gcc49/4.9.3/lib/gcc/4.9/i386/libstdc++.6.dylib
tomj@Marvin ~/games/df_osx $ ./dfhack                                                                                                                             [ruby-2.1.3p242]
dfhack: attempting to hook in
Loading bindings from data/init/interface.txt
Resetting textures

The dfhack.init configuration file is missing. To customize
your DFHack installation, rename the dfhack.init-example file
to dfhack.init and edit it to suit your needs.

For now, dfhack.init-example will be used instead.

DFHack is ready. Have a nice day!
DFHack version 0.40.24-r4 (development build 0.40.24-r4-47-gbe93a91)
Type in '?' or 'help' for general help, 'ls' to see all commands.
[DFHack]#

And the game works too of course.  That's with DFHack compiled with 4.9, and putting libstdc++ from 4.9 into hack/ - and not changing the libstdc++ in libs/.

This is what I am still not understanding - we compiled DFHack with 4.9; we supply libstdc++ from 4.9; the game runs, at least it does for me.  So when/why/for whom does it not run?   Will the libstdc++ from 4.9 not work on older systems?  (That is something I will be able to test for sure soon on 10.6.)

But it definitely doesn't seem to break DF itself - at least not yet in a brief test of the game.

[mifki]

And the game works too of course.  That's with DFHack compiled with 4.9, and putting libstdc++ from 4.9 into hack/

So you have 4.5 lib in ./libs and 4.9 lib in ./hack. Yep, it may work in this case, but I really don't like it - who knows what changed between this versions and how it will work if part of the code uses one library and part uses another. That's why I think continuing to use GCC 4.5 and having an extra copy of Xcode 6 for it is simpler than to deal with all the possible issues.

[lethosor]

A libstdc++ from GCC 4.9, however, should work with anything compiled by GCC <= 4.9, but requires some extra effort to work on older OS X versions.

Are you sure?

Code: [Select]

Vit-iMac:df4024 vit$ cp /usr/local/Cellar/gcc49/4.9.3/lib/gcc/4.9/libstdc++.6.dylib ./libs/
Vit-iMac:df4024 vit$ ./df
Vit-iMac:df4024 vit$ dyld: Symbol not found: __ZNSt8__detail12__prime_listE
  Referenced from: /Volumes/df/df4024/./dwarfort.exe
  Expected in: /usr/lib/libstdc++.6.dylib


You copied in a 64 bit library.

[TheBloke]

Yeah:

Code: [Select]

tomj@Marvin ~/games/df_osx $ ls -al libs/libstdc* hack/libstdc*                                                                                                   [ruby-2.1.3p242]
lrwxr-xr-x  1 tomj  staff    64B 24 Nov 04:19 hack/libstdc++.6.dylib -> /usr/local/Cellar/gcc49/4.9.3/lib/gcc/4.9/i386/libstdc++.6.dylib
lrwxr-xr-x  1 tomj  staff    64B 24 Nov 04:49 libs/libstdc++.6.dylib -> /usr/local/Cellar/gcc49/4.9.3/lib/gcc/4.9/i386/libstdc++.6.dylib
tomj@Marvin ~/games/df_osx $ ./dfhack                                                                                                                             [ruby-2.1.3p242]
dfhack: attempting to hook in
Loading bindings from data/init/interface.txt
Resetting textures

The dfhack.init configuration file is missing. To customize
your DFHack installation, rename the dfhack.init-example file
to dfhack.init and edit it to suit your needs.

For now, dfhack.init-example will be used instead.

DFHack is ready. Have a nice day!
DFHack version 0.40.24-r4 (development build 0.40.24-r4-47-gbe93a91)
Type in '?' or 'help' for general help, 'ls' to see all commands.
[DFHack]#

According to GCC ABI policy, they are meant to be forward compatible.  Ie code compiled against libstdc++.so.6.0.14 (GCC 4.5.0) is is intended to be forward compatible with any future libstdc++.so.6.0.X, which at the time of writing is up to GCC 5.1.0: libstdc++.so.6.0.21.

That said, we know GCC 5.x is NOT compatible because of name mangling issues.  But maybe that comes under a separate rule, I'm not sure?

I do see your point mifki that some developer pain is better than any risk to end user.  However, it does appear that everything should work, at least providing a 4.x libstdc++ instead of 4.5.4.   I don't know enough about it to know why GCC 5.x doesn't work despite the forward guarantee.

[TheBloke]

Actually the GCC 5 issue is probably explained by : http://developerblog.redhat.com/2015/02/05/gcc5-and-the-c11-abi/ - changes related to C++11 and name mangling alterations required as a result.

I'm going to test a GCC 5 build now to confirm that's the same that we're seeing.  (And I have same vague recollection that lethosor already mentioned this before, last time it was discussed in #dfhack)

If so, then that's further confirmation of the forward policy applying usually - the above speaks of a specific case where they chose to break forward ABI compatibility without a version bump in libstdc++ (the soname), and detail why they did that, and emphasising that this is not the usual case.  This is apparently the first time since GCC 3.x that the ABI changed in this way, and the first time they did it without a soname version bump to the shared libraries.

This gives me further confidence that distributing gcc 4.9 into an application that was built against 4.5 is "no big thing".   Though as always I caveat such statements with "I don't know a huge amount about it"

[mifki]

Haha, ok, that was stupid.

Well, if the library in ./libs doesn't need to be replaced, then it's fine to use GCC 4.9 I guess.
What's the problem with GCC 4.9 lib on 10.6 ?

Still we'll need to test it thoroughly - I was building TWBT with Clang all the time actually, and only when I started to use some of the file IO-related stuff, it started to crash.

[TheBloke]

I don't know if there necessarily is a problem with 4.9 on 10.6 - just, we don't know if there is?

It needs to be tested on 10.6 is the main point, I think. 

I am getting a 10.6 VM up and will try: a) using a 4.9 libstdc++ built on 10.11;  b) building 4.9 on 10.6 itself.  If both work, I don't think we have any problems; if a) fails but b) works, then I can confirm that the 10.6 built libstdc++ also works on all later versions, then we could just distribute the 10.6 compiled version.

[TheBloke]

It would be awesome if we could have some automated tests

lethosor, I saw references to Travis in the DFHack repo - is there some kind of test server in operation?  something we could add more tests to over time, and get a suite eventually?   Then such changes could be auto tested with a new libstdc++ dropped in, or whatever else is changing.

[TheBloke]

DFHack compiled with GCC 5.2 & working

Code: [Select]

tomj@Marvin ~/games/df_osx $ ls -al hack/libstdc*                                                                                                                 [ruby-2.1.3p242]
lrwxr-xr-x  1 tomj  staff    61B 24 Nov 05:42 hack/libstdc++.6.dylib -> /usr/local/Cellar/gcc5/5.2.0/lib/gcc/5/i386/libstdc++.6.dylib
tomj@Marvin ~/games/df_osx $ ./dfhack                                                                                                                             [ruby-2.1.3p242]
dfhack: attempting to hook in
Loading bindings from data/init/interface.txt
Resetting textures
...

DFHack is ready. Have a nice day!
DFHack version 0.40.24-r4 (development build 0.40.24-r4-47-gbe93a91)
Type in '?' or 'help' for general help, 'ls' to see all commands.
[DFHack]# plug
                     Name      State Cmds  Enabled
                  3dveins     loaded    1      n/a
     RemoteFortressReader     loaded    0      n/a
...
                      zone     loaded    3  enabled
[DFHack]#
Pull request coming in a moment lethosor to re-enable GCC 5 building with the appropriate arg, which was -D_GLIBCXX_USE_CXX11_ABI=0; simple as that.

(I don't know if this means we should switch to 5.2 instead of 4.9, if we do switch.  But at least it's possible!)

[TheBloke]

And to answer my own question re Travis - I see that pull requests get auto built to test compilation at least.   Nice.