GCC 3.2.3 / 3.4.6 - almost there!

iDork · December 17, 2011, 01:11:40 PM

Please, excuse my ignorance on the matter, but is't it easier to build a cross-compiler that targets NeXTSTEP, rather than porting GCC and all dependencies. OK, maybe not easier, but definitely less time consuming.

t-rexky · December 19, 2011, 04:28:05 AM

I am pretty ignorant on this subject as well, but trying to learn... From what I understand, the challenge with a cross-compiler is that one needs the "binary tools" to create the native object files and executable files. So we would need full sources for 'as' and 'ld' to make executables that run in the cross-compile environment to produce native NS/OS code.

Normally this is not an issue because the GNU binutils are used for the cross-compile environment. However, the NS/OS as well as the current OS X use heavily modified or completely custom 'as' and 'ld'. The original NS and OS Developer distributions included source for 'as' as it was based on gas-1.38, but no source whatsoever for 'ld'. Apple started supplying the source for 'ld' some time in the Darwin era. At this point the executable format and object file format was changed and will not be compatible on NS/OS.

Now, perhaps it would be possible to glue together a linker that is compatible with NS/OS from the more modern Apple 'ld' sources, but it would probably be much more of a challenge than getting an updated assembler that I was able to glue together.

What I am really itching to do is port a much more modern GCC compiler and perhaps the glibc libraries to allow porting of modern software. But of course this is way beyond my skill level. However, since I now managed to create working executables for gcc-3.2.3 on m68k and i386, it can obviously be done by someone who is more familiar with gcc.

I am currently side-tracked into compiling recent versions of bash, bison, sed, gawk and binutils. I have also been struggling to understand why I can build Perl 5.6.2 that passes all but 3 tests fine in the build directory, but fails dynamic loading once installed into /usr/local. (I have the same problem with the kb7sqi's Perl binary package on my m68k NS33 and i386 OS42).

However, once the above is done (or I give up on it), I plan on getting back to gcc-3.2.3 to get the C++ libraries ported.

Quote from: "iDork"Please, excuse my ignorance on the matter, but is't it easier to build a cross-compiler that targets NeXTSTEP, rather than porting GCC and all dependencies. OK, maybe not easier, but definitely less time consuming.

linmag7 · January 13, 2012, 04:41:24 AM

Quote from: "t-rexky"
I reconfigured the 3.2.3 build to only supply the C compiler, and I have to report that it works fine. In fact, I used it to build a number of other GNU tools and everything seems to be OK. The one thing that I have not yet tested is full support for the (broken) POSIX...

This is really great news! a more recent compiler is really the key to building a bunch of new stuff. How m68k specific is this? is it a big step to get gcc-3.2.3 working for sparc/intel as well?

A while back I worked on some ports of unix software to NextSTEP I put together a "libnext" with a collection of posix functions that were missing or faulty in nextstep, it wasn't much but fixed a lot of unresolved symbols when linking standard unix utilities...

t-rexky · January 14, 2012, 08:30:37 AM

I cannot say about sparc, but I managed to hack an Intel build almost as far as m68k. There was a fundamental challenge with the Intel build because the assembly code generated by gcc-3.2.3 was breaking the OPENSTEP supplied 'as'. I managed however to hack together a new 'as' by combining the OPENSTEP 'as' sources with Apple's cctools-622.9. And shockingly it works and produces good code. So I was able to build and successfully compare gcc-3.2.3 binaries for C only. But I stopped there since oneNeXT offered to help with the Intel port. I will submit some of what I have done once I have access.

I did follow everything here from you and also from kb7sqi - thank you for all your efforts! I also spent an incredible amount of time going through the old Usenet posts, which provide a wealth of information. In fact it would be great to produce an indexed archive of all NeXT Usenet info with the Librarian, but that's a task for a different time. I am not a programmer, but I used to do a bit of hacking in the past and I enjoy this kind of puzzles :-). I managed to build a few fairly recent GNU tools and they pass most if not all of the supplied tests. I also almost finished TCL 8.4.19 that at this point passes all but 11 tests! And I almost have latest expect working as well, all in an effort to try to get dejagnu working...

I found that once I have gone through a few ports things got much easier, as the issues are very similar. The biggest challenge now is that things compile and run, but the NeXT libraries are so out of date and sometimes broken that the tests fail for mysterious reasons. For example, with TCL I had to replace the NeXT string libraries (sprintf, fprintf, sscanf) with strio string libraries, and that made most of the test failures disappear.

It would be spectacular if we could get enough interest in the community to finish a more recent GCC and GLIBC port! I do realize that GLIBC would be a huge effort, but it would also provide a huge benefit. I think the above would allow most of the recent software to be compiled cleanly, or mostly so.

I would be interested in what you have included in your libnext as well as any notes you might have on porting. I have been putting together a similar collection of functions and notes, but it's also very limited.

Quote from: "linmag7"

This is really great news! a more recent compiler is really the key to building a bunch of new stuff. How m68k specific is this? is it a big step to get gcc-3.2.3 working for sparc/intel as well?

A while back I worked on some ports of unix software to NextSTEP I put together a "libnext" with a collection of posix functions that were missing or faulty in nextstep, it wasn't much but fixed a lot of unresolved symbols when linking standard unix utilities...

linmag7 · February 26, 2012, 01:49:42 AM

Hi,

Sorry for the delay i my response I've been less active for the past year or so, maybe its time to check in more often.
A more recent glibc along with a recent gcc would really bring nextstep into 2012 in a big way, maybe also a posix-threads library would be required for some compiles.

I agree the porting glibc to NeXT is no small task, not only does it requires some true skill but also a significant amount of time.

my "next library" contains an implementation of a very small subset of the common c-library functions that are either missing or broken in NeXTSTEP. The following functions are implemented:

I'd be happy to make the library available to you if you'd like to use it or add more implementations to it.

oneNeXT · February 26, 2012, 11:49:55 AM

Quote
... maybe also a posix-threads library would be required for some compiles.

There is a small pthreads library implemenentation that use cthreads. it is maybe no complete but it may be used for starting a libpthread.

I'm also particulary interrested by your implementation of snprintf, would it be possible to get it ?

linmag7 · February 26, 2012, 06:10:46 PM

I'll put "libnext" on an ftp sometime tomorrow, PM me if you want the login to grab it. Its not well documented and not even fully tested, but it has been useful to may a few software packages build. Its mostly glibc stuff that I've used.

t-rexky · February 28, 2012, 04:32:13 PM

Hi,

That sounds awesome. I was able to collect a small subset of the functions you listed, but a more complete library would be great. If you agree, I can submit this to Nitro to have it posted in the New Ports area of the file archive...

One thing I was struggling with is how to structure the replacement headers and library such that it is almost transparent to the code. Right now I jus redefine the proper functions names to something unique, but it gets a bit hairy with packages like Perl and TCL that load their own modules against the built-in and library functions.

Thanks again!

Quote from: "linmag7"Hi,

Sorry for the delay i my response I've been less active for the past year or so, maybe its time to check in more often.
A more recent glibc along with a recent gcc would really bring nextstep into 2012 in a big way, maybe also a posix-threads library would be required for some compiles.

I agree the porting glibc to NeXT is no small task, not only does it requires some true skill but also a significant amount of time.

my "next library" contains an implementation of a very small subset of the common c-library functions that are either missing or broken in NeXTSTEP. The following functions are implemented: [...]

I'd be happy to make the library available to you if you'd like to use it or add more implementations to it.

linmag7 · March 01, 2012, 03:16:16 AM

I'm not sure what the best approach to header files is. When I was porting stuff to nextstep I collected missing header stuff into new headers that I included into whatever source-files needed them. This was in order to leave the original nextstep header files intact.

Maybe a different approach would be to create a patch-kit that could be applied to original nexstep headerfiles in order to add/fix missing stuff to those files?

t-rexky · March 01, 2012, 06:10:32 PM

I am actually tempted to do at least some cleaning-up on the NeXT header files, if nothing else then just to prevent multiple includes... It is quite a bit of work though, if done manually - the "fixed" headers that I created for gcc-3.2.3 took almost a day of work.

Quote from: "linmag7"I'm not sure what the best approach to header files is. When I was porting stuff to nextstep I collected missing header stuff into new headers that I included into whatever source-files needed them. This was in order to leave the original nextstep header files intact.

Maybe a different approach would be to create a patch-kit that could be applied to original nexstep headerfiles in order to add/fix missing stuff to those files?

gregwtmtno · March 14, 2012, 06:44:30 AM

Sure thing. I'll give your suggestion a try when I get a chance. Right now I've messed my build chain up something fierce, so It's gonna take some serious work just to get it back where it was. I'll keep you updated.

t-rexky · March 18, 2012, 10:15:06 AM

I was finally able to complete the build process of g++ on m68k NS33, including building and (manually for now) linking abi_check.cc! I was getting undefined symbols during the linking process because libsupc++ is not being automatically linked by g++ the way it should.

As a side note, I now also have gcc-3.2.3 (C only) working on intel OS42, but there are some really "strange" things happening that I do not understand during the build process. Irrespective, the final binaries appear to be working just fine.

Remaining to do, from m68k perpective:
[list=1]

Fix the libstdc++ configuration such that libsupc++ is automatically linked by g++
Figure out how to implement section attribute, since NS33 and OS42 do support it - this will allow use of the original crtstuff.c
Customize the configuration files of libstdc++ for the NeXT
Finish the port of recent tcl, expect and dejagnu so that we can run the testsuite (I have tcl and expect mostly working)
Get a volunteer to test the C++ port since I don't know anything about C++[/list:o]
I am also in the process of studying the configuration changes that were made to gcc after 3.2.3 such that an even more recent version can be ported. I am hoping that this is doable for 3.4.6 or even perhaps 4.3.6.

After that it would be really nice to get a small group of people to perhaps try to port a recent libgcc and gdb so that all the good new code can be debugged natively on the NeXT. And to finish the i386 port.

gregwtmtno · March 19, 2012, 12:20:16 PM

Incredible work.

t-rexky · March 19, 2012, 02:51:14 PM

Thank you, I just wish I knew more about gcc and programming in general, although I am learning quite a bit throughout this exercise.

Are there any volunteers to try porting glibc to NS and OS? :wink:

t-rexky · April 13, 2012, 11:01:41 PM

I spent a bit more time on the compiler and made some good progress.

The libstdc++ issues were related to libtool and a 15 character limit for object file names on NS33 that 'ar' simply truncates. Once truncated, the libtool does not pick them up since they no longer have the ".o" extension. I implemented a workaround (thank you Daniel Richard G.) that fixes this and libstdc++ builds correctly. I am now able to build C++ executables that link and execute.

I was also able to finish my port of recent tcl, expect and dejagnu that is robust enough to be able to run automated test cases. The good news is that it works, and the bad news is that libstdc++ fails some tests, presumably because of its reliance on the outdated NeXT system libraries.

I put the implementation of arbitrary sections aside for now, as it is not critical to the overall function of the compiler with the tweaked crtstuff.c.

I also have some very unexpected good news - I am no longer working on version 3.2.3. I spent a few days tinkering with the GCC configuration files and I now have a bootstrapped & fully working gcc-3.4.6 that includes c, c++, objc and f77! I am currently running a 'make check' on the C portion, and then will see how badly the C++ portion behaves.

My plan is to spend some more time on 3.4.6 to make it as solid as possible, and then to move the configuration over to gcc4 and see how much damage I can do with it :-). After that I would like to try a proper OPENSTEP build on m68k that includes Mach-O PIC support. Then perhaps Intel, unless someone else picks up the flag.

I will also need to revise my fixed system headers, since I naively used non ANSI definitions in the #ifdef statements (such as m68k instead of __m68k__) and compiling with the -ansi flag does not pick up the required headers.

I still want to get a few people interested to port a recent (e)glibc so that we can more easily port all modern GNU tools.