GCC 3.2.3 / 3.4.6 - almost there!

[t-rexky]

I'm not sure what the best approach to header files is. When I was porting stuff to nextstep I collected missing header stuff into new headers that I included into whatever source-files needed them. This was in order to leave the original nextstep header files intact.

Maybe a different approach would be to create a patch-kit that could be applied to original nexstep headerfiles in order to add/fix missing stuff to those files?

I have been doing quite a bit of tinkering (still) and lots of learning.  In the process, I came up with an idea for "blending in" some updated functions and introducing new functions into the existing NEXTSTEP libraries.  This will also work on OPENSTEP, but perhaps there is a better way with shared libraries that I don't (yet) know much about?  In any case, here are my thoughts and I would appreciate some comments on the viability of this approach from real programmers.

[list=1]

• Custom-configure GCC such that INCLUDE_DEFAULTS defines new header directories located at /LocalDeveloper/LibnxHeaders just before /NextDeveloper/Headers.  This will allow us to install modified or new headers into this new directory without affecting the NeXT cc tools.  The NeXT cc will continue using the factory directories, while our new gcc will satisfy its header dependencies from the new directories before searching the factory directories.
  
• Create a new libnext.a library with new functions as well as replacement for existing but broken/incompatible functions.  This would include all the functions that linmag7 supplied in his library, plus full replacement for printf/scanf functions, strcoll(), mmap() & munmap() and many others.  All these functions would have nx_ prepended to them, for example strcoll() would be nx_strcoll().  This new namespace will allow linking against the existing NeXT libraries without duplicate symbol warnings.
  
• Change the GCC LIB_SPEC to automatically link against libnext.a when -posix argument is not specified to gcc.  I think we can safely ignore doing anything with -posix argument since the implementation is so broken on NEXTSTEP and does not exist on OPENSTEP.  Our new library will provide some some POSIX support by default.
  
• Introduce new header files into /LocalDeveloper/LibnxHeaders that define all new and overriding features plus all the existing NeXT library features that we want to use.  Here we can use the wonderful GCC #pragma redefined_extname oldname newname, to replace function names at assembly level.  For example:
  
  Code Select Expand
  #pragma redefined_extname strcoll nx_strcoll
  placed in new string.h will replace all references in our newly compiled code to strcoll() with nx_strcoll().  I noticed that gcc-4.x uses this approach with libmudflap and a light bulb illuminated in my head.
  
• And that's all folks!  It is quite a bit of work, but I can continue chipping away at it in my spare time.[/list:o]
  
  I would appreciate some comments on the above proposal.  If it works and is feasible, it would allow us to fix some of the biggest deficiencies in the NEXTSTEP / OPENSTEP libraries without the need for porting a completely new library such as glibc / eglibc.
  
  Looking forward to some comments...

[gregwtmtno]

Just wanted to say thanks for the hard work. I wish I could give insightful comments, but you've really way beyond my level of understanding on this one.

[t-rexky]

Just wanted to say thanks for the hard work. I wish I could give insightful comments, but you've really way beyond my level of understanding on this one.

Thanks!

I spent quite a bit more time with gcc and I was able to translate the target configuration files over to gcc-4.6.  I am able to build a stage 1 compiler that appears to work, except for the table jumps that are used in switch/case statements.  They cause bus errors and I am not certain if it is the code generation or the NeXT assembler itself.

I spent a considerable amount of time on the m68k debian ports mailing list and with the help of the knowledgable people there I was able to find a bug in the newer gcc codebase that was causing internal compiler errors with the NeXT target configuration.

Unfortunately at this point I am a bit stuck due to my lack of knowledge of the MIT m68k assembly and gcc internals...  Still trying, but not hopeful that I will be able to create a patch - that part of gcc is just way beyond my knowledge level.

NB - the same problem happens on gcc-4.4 and 4.2.  I have not tried 4.1 yet as it needs yet another change to the target configuration files and it will take me some time.

t-r

- UPDATE -

Of course shortly after I posted the above I was able to figure out what was ailing gcc-4.2.4.  It is now (finally) happily crunching through the stage 2 build and jump tables are working.  Fingers crossed!

[gregwtmtno]

Excellent. Keep us updated. I feel like getting a modern compiler going could open the door to a lot more ports.

- UPDATE -

Of course shortly after I posted the above I was able to figure out what was ailing gcc-4.2.4.  It is now (finally) happily crunching through the stage 2 build and jump tables are working.  Fingers crossed!

[t-rexky]

I feel the same way about availability of a more modern compiler.  Which is exactly why I am obsessing to get one going...

I now have gcc-4.2.4 completing the bootstrap process on NS33 m68k.  There are still some weird issues with the compiler that I do not understand, but just yesterday I was able to make a small step in the right direction in identifying the root cause.  For anyone interested, here are some posts that I made to the gcc list looking for some help with the strange warning that my current build generates:

http://gcc.gnu.org/ml/gcc/2012-06/msg00167.html

On a related note, many thanks to Thorsten Glaser, Vincent Riviere and Andreas Schwab on debian-68k for their help!  I would not be able to identify the jump table issue without them.

On m68k, gcc-4.2.4 is the most recent version that I can get going without diving into the machine description files for the m68k cpu.  This is because the NeXT was the only gcc target that used absolute case vectors while all other machines used PC relative case vectors.  The m68k.md was substantially rewritten at gcc-4.3 and the compiler generates some PC relative case vector assembly that the NeXT assembler does not understand, causing bus errors.  One way forward with this would be to attempt a port of a more modern gas, but it would be a very significant effort.  Another possibility would be to patch the m68k.md and m68k.c files, but that's way beyond me.

The "small step forward" yesterday allowed me to determine that the warning messages I have been getting from stage 2 & 3 compilers are related to some of the optimization options.  Now I have to determine which one, but it will take a while.  This is because the bootstrap process of gcc-4.2.4 takes on the order of 3 days on my Turbo Color :-).  And the gdb version supplied with NS33 is too ancient to cope with much of the code within gcc, it would seem.  So here is another project - porting a newer version of gdb :-)

t-r

[oneNeXT]

I feel like getting a modern compiler going could open the door to a lot more ports.

and save a lot of work and time.

Replacing designated initializers is a pain.

[t-rexky]

After quite a bit of tinkering I was able to cleanly build quad-fat cctools-435.4 from OS X 10.2.8 on NEXTSTEP 3.3.  The only significant item that does not build due to lack of static system libraries is 'dyld'.  This is not a big loss however, since NEXTSTEP uses static libraries and shared (shlib) libraries.  Perhaps one day we will be able to build dyld as well...

I consider this HIGHLY EXPERIMENTAL and would describe the "port" as OS X cctools running on NEXTSTEP rather than a true port of cctools to NEXTSTEP.  However, the fascinating aspect of OS X lineage is that the Mach-O files produced and manipulated by these tools appear to function perfectly on NEXTSTEP.  I have done some rudimentary testing on i386 and m68k architectures but I do not have access to SPARC or HPPA hardware.  I was able to successfully build numerous packages using these tools, including rebuilding the tools themselves and building the NeXT cc compiler.  This exercised 'as', 'libtool' and 'ld'.  I also did some ad-hoc testing using 'otool', 'strings', 'nm', and others with good success.

This package has been built on NEXTSTEP 3.3 with NeXT cc compiler modified to support __private_extern__ and -dynamic compiler features.  A reasonably recent version of GNU make is also required, such as Apple's gnumake-108.  This package will not build with the standard NEXTSTEP 3.3 compiler without additional code modifications.

I tried even more recent cctools but they were producing executable files that failed to run on NEXTSTEP (i386).  However, just a few days ago I was able to determine the source of this issue and I also have cctools-622.9 from OS X 10.4.11 creating object files and executables that appear to function correctly on NEXTSTEP 3.3.  This newer version, however, will take more time to clean-up and package.

For the adventurous types, I will be uploading the package to the software archive here and it should eventually find its way into the New_Ports directory.

[aragon]

:shock:  Congratulations! This sounds really great and quite exciting!

Let us keep posted with your progress.

[gregwtmtno]

wow! More incredible work t-rexky. I can't wait to try it out.

[barcher174]

Hi T-rexky,

Has your most recent GCC port been made available for download?

[t-rexky]

Hi,

The most recent files that I submitted are in the following directory:  http://www.nextcomputers.org/NeXTfiles/Software/New_Ports/

I have not submitted anything more recent because I am a bit stuck in a chicken-and-egg situation with the ancient NeXT libraries.  I was also not able to connect with Nitro for renewed ftp access.

I believe that the library is responsible for a number of failures that I see with the test suite and I would like to fix this.   I hope to "blend-in" a significant portion of BSD 4.4 library functions into the NS 3.3 library.  However, this is a much larger endeavour.

I am hoping to put together a somewhat more complete build of gcc-3 this winter when I have a little bit more time.  This would likely be without any refreshed libraries.  I need to collect all the notes and the various test builds that I have on my slab and move it all onto an SVN server that I recently set-up.  This will help significantly with revision control.

With respect to making some of the stuff available, I have been trying to put a simple web site together but with no prior experience it has been a bit of a struggle.  I resorted to iWeb, but I think it's too awkward for what I wanted, for example hierarchical menus.  I am also trying to find an inexpensive hosting service.

In addition to all the above I have been dealing with some ongoing medical issues so it has been very hard for me to concentrate and stay focused on the tinkering.  But I have not lost the drive to do it!

Peter.

[barcher174]

Hi t-rexky

Thanks for the update. I will give your previous version a try. I have a shared hosting account, so if you are unable to contact nitro maybe we can mirror anything you want there until you get access here. I'll look forward to your progress. I've been considering the possibility of creating something similar to busybox for the next. Basically a single low resource binary with scaled down versions of modern shell tools.

[t-rexky]

Hi t-rexky

Thanks for the update. I will give your previous version a try. I have a shared hosting account, so if you are unable to contact nitro maybe we can mirror anything you want there until you get access here. I'll look forward to your progress. I've been considering the possibility of creating something similar to busybox for the next. Basically a single low resource binary with scaled down versions of modern shell tools.

Thank you very much for the offer!  For now I created a Dropbox account and will use it to "host" some of the files.

I just placed my build of Apple's cctools-435.4 for NEXTSTEP 3.3 at the following link, for those who would like to tinker with it.  Please let me know if these work for you, particularly on the hppa and sparc platforms.

https://www.dropbox.com/s/n028mygoss5iay2/cctools-435.4%7Ens1.NIHSs.tar.gz

cctools-435.4~ns1.NIHSs.tar.gz

A note of caution that this port is highly experimental, but the tools seem to work fine for me on m68k and i386.  The Installer package will overwrite the corresponding system files but will keep a backup in the Receipts directory.  When uninstalled, your system will be restored to the original pre-install configuration.  The patch file is included as well, but the source will not compile cleanly with stock NS3.3 compiler due to lack of support for __private_extern__ and -dynamic compiler features, as explained in one of my earlier posts.

Once again, if anyone with a real programming background is interested in helping with any of the porting efforts it would be much appreciated!  It is hard to keep the enthusiasm going if it's just me, myself and I :wink:

[gregwtmtno]

Glad to see you're still working on this. I, on the other hand, have messed up by system beyond repair. I'll try again from scratch one of these days.

[oneNeXT]

Thanks for sharing it, I hope I will be soon able to contact you from a custom app ...