NeXT Computers Forum Index NeXT Computers
www.NeXTComputers.org
 
Log in to check your private messagesLog in to check your private messages

Log inLog in  RegisterRegister


Profile  Search  Memberlist  FAQ  Usergroups
I am able to build NS shared libraries

 
Post new topic   Reply to topic    NeXT Computers Forum Index -> Porting New Software
View previous topic :: View next topic  
Author Message
t-rexky



Joined: 09 Jan 2011
Posts: 272
Location: Snowy Canada

PostPosted: Tue Jan 22, 2013 9:46 pm    Post subject: I am able to build NS shared libraries Reply with quote

This forum seems to have been pretty quiet in the last while, so perhaps this might bring some excitement in Smile.

As part of my ongoing struggles with porting a reasonably recent GCC (and other GNU stuff), I discovered that one of the largest sources of issues on NS and OS are the ancient system libraries. Even things as simple as outdated printf() formatting strings make many tools fail, such as Perl for example. Consequently, I have been trying to figure out how to refresh some of the functions within the NeXT libraries without breaking the existing system.

Unfortunately it looks like there is no help forthcoming from Apple in the form of the original source code, so we are likely going to be on our own. I spent hours searching through the old usenet posts trying to determine how to patch or to tweak the NS3.3 shlibs, only to discover that no one other than NeXT themselves were ever able to build shared libraries. And then I finally had a bit of an eureka moment when I came across old references to a "mysterious" mkshlib tool. As it turns out the source code to mkshlib has been released by Apple in the Darwin cctools package, and it is almost unchanged since the dark ages.

With a few minor tweaks I was able to almost completely compile cctools-384.1 on OS i386 ('dyld' does not build of course, and 'ar' chokes as well). I then studied the mkshlib manpage and continued searching the net for other references and examples. I ultimately discovered that the mkshlib tool seems to be related to Sys V Unix and there is reasonable amount of information available on the SCO site at http://osr507doc.sco.com/en/tools/ShLib.html.

Armed with all that info I set off to test the concept. I created a replacement object file for the frexp() function on my NS m68k, using some assembly code I found a while ago. I then created the required library spec file, moved everything to the OS i386 machine and after a few final tweaks I was staring at two output files: libsysnew_s.A.shlib and libsysnew_s.a. I quickly moved the files into /lib and /usr/shlib on my NS m68k and built a quick frexp() test program with the "factory" libraries. I then used 'ar' to remove the NeXT version of frexp() from /lib/libsys_s.a and rebuilt the test program against /lib/libsys_s.a and /lib/libsysnew_s.a. And it worked just fine!!! Inspecting the executable with otool showed that it was pulling objects from the NeXT libsys shared library as well as from my newly built shared library.

Because of how the NeXT shlib scheme works, removing an object reference from the .a "host" library of a "target" shlib will have no impact on the existing executables and will only impact newly created & linked executables. This mechanism therefore appears to allow substitution of new versions of existing functions without the need to change their names with #define of #pragma. I am really thrilled that this might provide a reasonably decent solution to the system library refresh dilemma.

One of the challenges with this approach is a need for a centralized control of the memory space in order to prevent different shared libraries from overlapping. I already mapped out all the existing NS 3.3 shlib memory space, however, so that I know where new shlibs can be slotted in.

Next I will do a bit more work to build cctools natively on NS. This could also potentially open the doors to building a cross-compiler for the m68k and perhaps a BSD 4.4 based 'ar' with support for long object names...

...now time to get some sleep.
Back to top
View user's profile Send private message
Shuren



Joined: 31 Oct 2006
Posts: 48
Location: Italy

PostPosted: Wed Jan 23, 2013 7:09 am    Post subject: Reply with quote

Well done!
_________________
Blog (in Italian): http://greedyandlounge.blogspot.com
Back to top
View user's profile Send private message MSN Messenger
pitz



Joined: 14 Aug 2006
Posts: 122
Location: Oregon, USA

PostPosted: Wed Jan 23, 2013 11:06 am    Post subject: Re: I am able to build NS shared libraries Reply with quote

This is exciting. Would I dare say the seeds for an OPENSTEP 4.3?

t-rexky wrote:

One of the challenges with this approach is a need for a centralized control of the memory space in order to prevent different shared libraries from overlapping. I already mapped out all the existing NS 3.3 shlib memory space, however, so that I know where new shlibs can be slotted in.


Details soon please.

Thanks for all the hard work,
/pitz
Back to top
View user's profile Send private message
Noth



Joined: 04 Apr 2008
Posts: 146
Location: Lausanne, Switzerland

PostPosted: Thu Jan 24, 2013 12:03 pm    Post subject: Reply with quote

Excellent work!
_________________
NeXT Cube 040 (NeXTSTEP 3.3), SUN SparcStation5 (NeXTSTEP 3.3), SGI Indigo2 R10000 (IRIX 6.5.22), SGI VSW320 (Windows 2000/Slakware 9.1)
Back to top
View user's profile Send private message
gregwtmtno



Joined: 28 Aug 2011
Posts: 19

PostPosted: Sun Feb 03, 2013 10:11 am    Post subject: Reply with quote

I have no idea what most of this post means, but it sounds very impressive.
Correct me if I'm wrong, but what you're saying is that you've made progress getting a modern cctools to build. If so, excellent work
Back to top
View user's profile Send private message
t-rexky



Joined: 09 Jan 2011
Posts: 272
Location: Snowy Canada

PostPosted: Sun Feb 03, 2013 4:13 pm    Post subject: Reply with quote

Thanks for the encouragement guys. Here is a bit of an update.

I spent a bit more time tinkering on the slab over the weekend and was able to partially build cctools-358 (from OS X 10.0.4) on NeXTstep 3.3 m68k. It took quite a bit of elbow grease since there have been many system level changes (libraries, compilers, etc.). I had to first build cc-471.7 from prerelease OPENSTEP 4.0 as it supports the __private_extern__ compiler keyword that is (kind of) required to build cctools with minimal changes. I first tried to rebuild the 3.3 compiler cc-437.2.6 with added support for __private_extern__ but as it turns out this endeavour was not as easy as I had hoped and the resultant compiler is flaky. In addition, the cctools build many of the libraries and code with the -dynamic compiler option that does not work on the NS33 compiler.

To make a long story short, I was able to get cc-471.7 running on NS33 and with it I was able to build majority of cctools-358. I now have the following cctools binaries running natively on NS33:

Code:
./as/a68_dir/as
./file/file
./gprof/gprof
./ld/ld_dir/ld
./misc/nm
./misc/lipo
./misc/size
./misc/strings
./misc/libtool
./misc/checksyms
./misc/indr
./misc/strip
./misc/nmedit
./misc/segedit
./misc/kern_tool
./misc/cmpdylib
./misc/dylib_pcsampler
./misc/pagestuff
./misc/redo_prebinding
./misc/seg_addr_table
./misc/check_dylib
./misc/seg_hack
./misc/atom
./mkshlib/mkshlib
./mkshlib/cmpshlib
./mkshlib/hack_libgcc
./otool/otool
./profileServer/dylibprof
./profileServer/profileServer


I was particularly interested in the linker, so I immediately substituted it for the NeXT linker and so far everything seems to be working well! I am having some issues getting 'ar' to build because it uses a lot of BSD4.4 specific functions and features, requiring a lot of hacking. I am at the point where I now need to build a complete replacement for sprintf() because 'ar' breaks when using the NeXT library version on NS33 and OS42...

However, all of the above means that building a cross compiler is now definitely a much closer possibility since the assembler and linker can be build from sources. Of course, there have been many changes to the linker from NS33 to OS X 10.0.4 so I am shocked that it seems to work and I am expecting it to croak really soon Smile.

@gregwtmtno: The shlib stuff is NeXTstep specific as on OPENSTEP shlibs were pretty much replaced by dylibs and bundles, akin to OS X. The advantage of being able to build shlibs for NS is that a large library including fixes and enhancements to the existing NS system library functions can be built as a shared library as opposed to a static library. With a static library, every running executable will load into memory its own copy of the library functions (such as sprintf() for example), using up precious RAM. If sprintf() resides in a shlib, on the other hand, every executable in memory will only use one and the same instance of it - it resides in RAM only once.
Back to top
View user's profile Send private message
barcher174



Joined: 07 Dec 2012
Posts: 559

PostPosted: Sun Feb 03, 2013 5:30 pm    Post subject: Reply with quote

Brilliant work!
Back to top
View user's profile Send private message
t-rexky



Joined: 09 Jan 2011
Posts: 272
Location: Snowy Canada

PostPosted: Mon Feb 04, 2013 8:54 pm    Post subject: Reply with quote

I had a bit of a revelation. A few quick tweaks of the printf formatting strings and I now appear to have 'ar' working without the need for custom printf functions. And yes, it does support long file (object) names without the 15 character limit of NS and OS 'ar' which should make life much easier with newish gcc library builds.

Now I have to do a bit of testing of all this new stuff.
Back to top
View user's profile Send private message
t-rexky



Joined: 09 Jan 2011
Posts: 272
Location: Snowy Canada

PostPosted: Tue Mar 12, 2013 7:19 pm    Post subject: NEXTSTEP build of OS X 10.2.8 cctools-435.4 Reply with quote

I now have a quad fat package put together of cctools-435.4 from OS X 10.2.8 that will be submitted to the file archive here as soon as Nitro resets my access. No point in cross-posting so I will provide more info in my gcc-3.2.3 thread...
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    NeXT Computers Forum Index -> Porting New Software All times are GMT - 7 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2017 phpBB Group