Taking the NeXT step (mach kernel upgrades)

[neozeed]

The earliest x86 Darwin image that I have is darwinx86-141.iso.  I have DarwinOS-0.3.toast, DarwinOS-1.0.toast and DarwinOS-1.0.2.dmg, but those are most certainly PPC binaries...

I have the 141 from Apple's site (its still there but the download page is missing....)  I don't have the 1.0.2 dmg though.. that my have something interesting !?

[NCommander]

On HFS segfaulting, do we have the GDB stub in this version of mach to attach a debugger? I believe it requires a kernel command line option to be enabled, but you can then attach gdb over a serial line.

It might want a userland component and that's why its making a sad.

[neozeed]

I'll have to learn and bumble through trying to setup a kernel debugger.  I've never done it before, so I'm sure it'll be fun.

I suspect I'll have to debug from Rhapsody as it has a working gdb.

At least Virtual machines are cheap!!

[neozeed]

I tried this from Rhapsody's yellowbox on NT

Code Select Expand

C:\temp\cc-1.obj\x>sh /Apple/proj/cc-1/cc/configure --host=i386-nextpdo-winnt3.5 --target=i386-next-nextstep3 --build=i386-nextpdo-winnt3.5 --srcdir=/Apple/proj/cc-1/cc
Using `/Apple/proj/cc-1/cc/config/i386/next.c' to output insns.
Using `/Apple/proj/cc-1/cc/config/i386/i386.md' as machine description file.
Using `/Apple/proj/cc-1/cc/config/i386/next.h' as target machine macro file.
Using `/Apple/proj/cc-1/cc/config/i386/xm-winnt.h' as host machine macro file.
Merged winnt/x-winnt.
Merged next/t-next.
Merged c++ objective-c++ fragment(s).
Created `./Makefile'.
Merged winnt/x-winnt.
Merged next/t-next.
Created `cp/Makefile'.
Merged winnt/x-winnt.
Merged next/t-next.
Created `obcp/Makefile'.
Links are now set up to build a cross-compiler for i386-next-nextstep3
 from i386-nextpdo-winnt3.5.

sure looks good, right?

Code Select Expand

/Apple/proj/cc-1/cc/config/next/nextstep.h(928): warning: #warning ****** Disabled objcunique *******
/Apple/proj/cc-1/cc/c-decl.c: In function `grokdeclarator':
/Apple/proj/cc-1/cc/c-decl.c:4771: `stdcallp' undeclared (first use this function)
/Apple/proj/cc-1/cc/c-decl.c:4771: (Each undeclared identifier is reported only once
/Apple/proj/cc-1/cc/c-decl.c:4771: for each function it appears in.)
/Apple/proj/cc-1/cc/c-decl.c:4839: `RID_STDCALL' undeclared (first use this function)
make: *** [c-decl.o] Error 1

So weird.  Guess this stuff wasn't meant to cross compile this way.  Or it just needs more hammering.

I was thinking as this is possible:

Code Select Expand

C:\temp\cc-1.obj\y>type hi.c
int main(){
printf("hi from Darwin0.3 cc-1\n");
return 0;}

C:\temp\cc-1.obj\y>xgcc -v  -S hi.c
gcc version 2.7.2.1 for Apple PDO
cpp-precomp -smart -lang-c -v -undef -D__GNUC__=2 -D__GNUC_MINOR__=7 -Di386 -DWIN32 -D_WIN32 -Dwinnt -DWINNT -D_M_IX86=300 -D_X86_=1 -D__STDC__=0 -DALMOST_STDC -DNeXT_PDO -D_MT -D_DLL -D__LITTLE_ENDIAN__ -D__ARCHITECTURE__="i386" -D__i386__ -D__WIN32__ -D_WIN32 -D__winnt__ -D__WINNT__ -D_M_IX86=300 -D_X86_=1 -D__STDC__=0 -D__ALMOST_STDC__ -D__NeXT_PDO__ -D_MT -D_DLL -D__LITTLE_ENDIAN__ -D__ARCHITECTURE__="i386" -D__i386 -D__WIN32 -D__winnt -D__WINNT -D__ALMOST_STDC -D__NeXT_PDO -Asystem(winnt) -Acpu(i386) -Amachine(i386) hi.c C:\Users\jason\AppData\Local\Temp\cca00672.i
GNU CPP version 2.6.1 (i386-applepdo-winnt3.51)
#include "..." search starts here:
#include <...> search starts here:
./Library/Frameworks/System.framework/PrivateHeaders
./Library/Frameworks/System.framework/Headers
./Developer/Headers
./Local/Developer/Headers
End of search list.
cc1obj C:\Users\jason\AppData\Local\Temp\cca00672.i -quiet -dumpbase hi.c -version -o hi.s
GNU Obj-C version 2.7.2.1 (i386-applepdo-winnt3.51) compiled by GNU C version 2.7.2.1.


C:\temp\cc-1.obj\y>type hi.s
       .file   "hi.c"
gcc2_compiled.:
___gnu_compiled_c:
       .const
LC0:
       .ascii "hi from Darwin0.3 cc-1\12\0"
.text
       .align 4
.globl _main
_main:
       pushl %ebp
       movl %esp,%ebp
       call ___main
       pushl $LC0
       call _printf
       addl $4,%esp
       xorl %eax,%eax
       jmp L1
       .align 2,0x90
L1:
       leave
       ret

C:\temp\cc-1.obj\y>sh /Apple/proj/cc-1/cc/configure --host=i386-nextpdo-winnt3.5 --target=i386-nextpdo-winnt3.5 --build=i386-nextpdo-winnt3.5 --srcdir=/Apple/proj/cc-1/cc

So it ought to be possible to use OPENSTEP to cross compile, meaning we could hash stuff around so building using NT (aka Windows 10) and using all CPU cores so we can greatly speed stuff up...

[rooprob]

This truly is awesome work guys - and as someone who's "had a go" at modernising OpenStep (on black hardware) Id really love a modern bsd 4.4 compatible kernel.

That being said, I don't want to take away from your great work, but would attacking this problem from the other side be more practical use of time and obvious smarts.

I understand the juice is in cracking the problem, but there's an equal opportunity here too. Take the NetBSD kernel and userland and try port the necessary layer required to run NS/OS apps? With the ready available source and dev tools, how easy to open a window from the kernel side?

[neozeed]

This truly is awesome work guys - and as someone who's "had a go" at modernising OpenStep (on black hardware) Id really love a modern bsd 4.4 compatible kernel.

That being said, I don't want to take away from your great work, but would attacking this problem from the other side be more practical use of time and obvious smarts.

I understand the juice is in cracking the problem, but there's an equal opportunity here too. Take the NetBSD kernel and userland and try port the necessary layer required to run NS/OS apps? With the ready available source and dev tools, how easy to open a window from the kernel side?

To make NetBSD more like NS you'll need to have it loading frameworks and the like... and of course talking to them in Objective C linking...

for now I want to merge in some better filesystem, then try to tackle the missing syscalls for NeXTSTEP to Rhapsody.... that is the other fun thing, they changed a lot of weird stuff... but if I can reliably get NS stuff running it ought to be in a better place.

fleshing out i386 ought to be easier, then looking at something like 68k.  Although the other thing I'm worried about is looking back at the 0.8 release they were all in on the 68030 + 68881 setup.  And there is a very high chance that they emitted a LOT of instructions the 68040 omitted.  I don't know how NetBSD handles it when someone builds something with -m68020 -m68030 -m68881 -mno-soft-float  and then runs it on a 68040.

It's my suspicion that we'll actually need a (partial) 68881 emulator..

Pure speculation of course, but outside of that, I also want to see if I can do drivers in the 'bsd' kernel and dump the driverkit.

[NCommander]

It's looking like I'll have my life back come end of June and can pitch in on this.

GCC cross-compiler: ... um, theorically possible, but good look. The ObjC frontend has all sorts of pain and suffering that makes that impractical at best. Apple's fat binary solution is a nifty hack around the problem. Based on what I've seen, NeXT essentially never did "true" cross-compiling, but just fat binary *everything*, and the PBMakefiles suggest they had a NT on NeXT target which was probably how they bootstrapped the thing.

If you compile mach with a debug option and/or boot switch, it should dump a stack track when it goes belly up. You can use gdb on mach_kernel or just check with objtools to determine where it code base it went boom.

EDIT: Re-reading through the backlog, libsystem is dynamically built from several other libraries at once. The exact bits of magic escape me, but I know where to recover them. Dylib files are (fortunately) position independent, and I belive they're also modular, which means they can be disassembled, modified, and reassembled relatively easily. The libsystem buildscript is a fairly decent example of this.

As a potential "option B" however, there might be a cleaner way to handle this. The dynamic link loader is in mach (with another part I think in libc), and we've got some handy fields on the mach-o. We can forcibly include different libraries in the search path, or if we're got a Rhaspody binary, load the Rhaspody sysroot. Got OPENSTEP? Get Openstep, etc.

I've already proven we can run NeXT, OPEN, Rhaspody, and Darwin 0.x binaries all on the same kernel, and you proved from a DisplayServer bit that it should all work.

[t-rexky]

It's gotten awfully quiet here lately.  I am suffering from withdrawal symptoms.  Hands shaking, body tremors, dreams of magnesium cubes...

[neozeed]

It's gotten awfully quiet here lately.  I am suffering from withdrawal symptoms.  Hands shaking, body tremors, dreams of magnesium cubes...

works been kicking my ass.  I did make a slight discovery, the NS CD's actually have a disklabel on them.  I think the filesystem is almost 4.3BSD but with weird sizes.  I think it's a matter of driving mkfs directly to create a filesystem, although you can't just mount one read write, I guess there is a flag to prevent that?  I haven't dived deep enough (time).

I also put up cvs.nextstep33.info with my latest buildable stuff, although clock drift is going to mess stuff up for make.  /debs has binaries though which is an easy way to pull down a release.

I'll try to put together some steps, and a hacked up 'disk' command to chain to the new mkfs/boot blocks instead of the system stuff so it ought to be semi-workable.  Although I don't think Rhapsody can read an 8GB UFS disk on it's own.

Also I see that in NS 3.1 for the x86 there is a fsck diskette, aka a tiny root filesystem that fits onto a floppy.

[NCommander]

Poking my head back in. I have very little free time, and very little motivation to work on tech projects on it, but I'll weigh in a bit here.

The UFS format used on the CDs is slightly different than what's used on the HDDs at a 2k block size vs 512; this is noted somewhere in mach's source, and I think I found it originally in the Linux UFS kernel source which has at least some support for NeXTstep volume formats.

From a NeXT perspective, raw disk images exist as the "h" partition of a device, so if you want to dump the entire disk, you can dd if=/dev/srXh or something like that and get the whoel thing

[Rob Blessin Black Hole]

Hello: When I use DD to micro sd cards this works very well
file.iso = your disk image !
you will want to unmount the sd disk as well before using dd
diskutil unmountDisk /dev/disk1

sudo dd if=/dev/rdisk1 of=/file.iso bs=512b then to copy the file back to the disk sudo dd if=/file.iso of=/dev/rdisk1    using /dev/rdisk1 instead of just /dev/disk save significantly on time , also works for making iso files of NeXT CD's

Also the slower 10mb/s sd cards used to take hours to make clone dd , go with extreme pro a 95mb/s 16Gb microsd card takes 5 minutes or less!  What is cool about this is if I hose something up on black or intel  hardware I can just pop in a backup sd and go and repair the hosed sd.

So it may help you all here trying different incarnations of this project ....

[neozeed]

Im not dead, work is trying to consume my life, and when I get home, the last thing Ive been wanting to do is touch a compiler.  I've been treking across the wastelands and blowing stuff up.

I'll get in the mood again though.

[NCommander]

Im not dead, work is trying to consume my life, and when I get home, the last thing Ive been wanting to do is touch a compiler.  I've been treking across the wastelands and blowing stuff up.

I'll get in the mood again though.

I'm unfortunately in the same boat. Being the one full-stack developer at a startup means I lack time.

Looking back through the thread, NetBSD and Linux can both do software emulation of missing opcodes so it's theorically possible to run 68040 and later on newer. That being said, I honestly can't remember off hand how much new was added to the later m68ks from a userland perspective. I think a lot of it came from extensions of the address bus.

[neozeed]

Ok, I know all the cool kids use git or whatever, but since Rhapsody has CVS built in, let's use that to pull stuff down.

the date & timestamps will be munged and thusly the build process will want to do further build stuff, and it wont work as there is some missing tools (gprof) which is bombing as I haven't self hosted the c++ stuff just yet.

Code Select Expand


niutil -create . /machines/cvs.nextstep33.info
niutil -createprop . /machines/cvs.nextstep33.info ip_address 14.136.113.162
cvs -d:pserver:[email protected]:/cvsroot login


There is no passowrd.  And I just created a record for my machine in case you don't have DNS or whatever.

I have 2 things to snag, the source:

Code Select Expand

cvs -d:pserver:[email protected]:/cvsroot checkout darwin

And the pre-compiled binaries, aka the debs

Code Select Expand

cvs -d:pserver:[email protected]:/cvsroot checkout debs

The source is 388MB, and the debs are 36MB.

[neozeed]

Vintage apple pages....

https://archive.li/uFEzI

https://archive.li/S8eQn