layout: post
title: "Catching Up"
subtitle: "with a new toolchain"
tags: [osdev]

Well... I've been bad at updating the blog on my process again...

Last time I missed updating, I made a branch from an earlier git commit,
and threw away a bit of work and messy code rather than trying to catch
up.

This time, however, I'm actually quite happy that I didn't write too
much, because I've recently changed a lot of what I would have written
about.

So instead, I'll describe one of my most recent changes here - a new
cross compiler and a newlib port - and then in subsequent posts go over
chosen parts of the other things I've done.

So let's jump into it.

### Why a cross compiler
The be-or-not-to-be of cross compilers has been discussed at
length and it seems every time someone posts a description
on how to build one they are told that you don't need one if
you know how to use your compiler or whatever. For examples,
read
[the osdev wiki - Why do I need a Cross Compiler? talk page](http://wiki.osdev.org/Talk:Why_do_I_need_a_Cross_Compiler%3F)
(I don't recommend it if you're in a bad mood).

Well, I'm developing on OSX but have chosen to build an X86 kernel
in elf format due to the availability of documentation, support and
expertize. That's reason enough for a cross compiler build of binutils
at least.

Clang offers excellent support for cross compilation out of the box, so
I didn't use to have a need for a gcc cross compiler as illustrated in
[this post](/blog/2013/02/New-Environment/). As it turns out, however,
compiling newlib with that setup is rather annoying.

I'm also a fan of clean makefiles. Take a look at this:

    :::make
    VPATH := ../src

    CC := i586-pc-myos-gcc

    TARGETS := $(shell find ../src -name "*.c")
    TARGETS := $(notdir $(TARGETS))
    TARGETS := $(patsubst %.c, %, $(TARGETS))

    all: $(TARGETS)

    clean:
        -rm $(TARGETS) 2>/dev/null

That's the makefile for the entire `/bin` directory in my os.

So let's move on...

### Building newlib and gcc
For my toolchain build, I followed the tutorial at
[the osdev wiki](http://wiki.osdev.org/OS_Specific_Toolchain) with a few
changes. I chose to build everything inside the scope of
[Homebrew](http://brew.sh) to keep track of all files and eventually
make a formula for it. So after applying the patches described in the
post (I even kept the name `i586-pc-myos` since I don't have a working
name for my kernel besides an iteration number...) I did

    :::bash
    $ export TARGET=i586-pc-myos
    $ export PREFIX=/usr/local/Cellar/osdev/1.0
    # Configure, build and install binutils
    $ brew link osdev
    # Configure, build and install gcc and libgcc
    $ brew unlink osdev
    $ brew link osdev

And that prepared me for building newlib.

### Building newlib
Now this was an adventure...

Newlib uses a directory structure that hasn't been supported by
automake and autoconf since a couple of versions. More specifically,
you need automake version 1.12 or earlier and autoconf version 2.64.
Unfortunately, those versions are not available through Homebrew, so ...

    :::bash
    $ curl -O http://ftp.gnu.org/gnu/automake/automake-1.12.tar.gz
    $ tar -zxf automake-1.12.tar.gz
    $ mkdir -p build-automake
    $ pushd build-automake
    $ ../automake-1.12/configure --prefix=/usr/local/Cellar/automake/1.12
    $ make all -j
    $ make install
    $ popd
    $ curl -O http://ftp.gnu.org/gnu/autoconf/autoconf-2.64.tar.gz
    $ tar -zxf autoconf-2.64.tar.gz
    $ pushd build-autoconf
    $ ../autoconf-2.64/configure --prefix=/usr/local/Cellar/autoconf/2.64
    $ make all -j
    $ make install
    $ popd
    $ brew switch automake 1.12
    $ brew switch autoconf 2.64

Those last two lines tells Homebrew that you want to use those specific
versions for now.

Now for the neat part. I followed the wiki post and used the
[syscall interface](/blog/2013/06/System-Calls) i've described earlier
but I also wrapped `crt0.S` and `syscalls.c` in

    :::c
    #ifndef KERNEL_MODE
    ...
    #endif

Then I built it all through

    :::bash
    $ pushd build-newlib
    $ ../newlib/configure --target=$TARGET --prefix=$PREFIX
    $ export CPPFLAGS_FOR_TARGET=-DKERNEL_MODE
    $ make -j
    $ make install
    $ mv $PREFIX/$TARGET/lib/libc.a $PREFIX/$TARGET/lib/libkernel.a
    $ rm -rf *
    $ ../newlib/configure --target=$TARGET --prefix=$PREFIX
    $ export CPPFLAGS_FOR_TARGET=
    $ make -j
    $ make install
    $ popd

This gives me two versions of the newlib c library. One with all syscalls
defined and one without. The latter is suitable for linking into the
kernel and allows me to use a standard C library with things like
`sprintf` and even a ready made `malloc`.

"But wait," you say. "Doesn't malloc make an sbrk() syscall? You can't
make a syscall from inside the kernel, can you?"

Well. Remember [my last post](/blog/2013/06/System-Calls) where I said
that keeping versions of every syscall function inside the kernel would
turn out to have some really cool advantages? Here they are.

I need an `sbrk` anyway, so all I have to do differently is make it
check whether the request came from a user process or from the kernel
and handle them a slight bit differently (i.e. make sure the `brk` is
above `0xC0000000` for kernel and below for user mode...).

Similar checks can be made for all syscalls and some doesn't actually
need any changing at all.

Needless to say, this revelation changed my kernel structure
quite a bit and I entirely threw away my own
[kernel heap manager](/blog/2012/07/Memory-Heap).

### Using this
Having the cross compiler toolchain and library setup allows me to
compile the kernel using a very simple makefile.

Somewhat simplified:

    :::make
    OBJECTS := $(shell find . -name "*.S")
    OBJECTS += $(shell find . -name "*.c")
    OBJECTS := $(OBJECTS:%.S=%.o)
    OBJECTS := $(OBJECTS:%.c=%.o)

    CC := i586-pc-myos-gcc
    LDFLAGS := -nostdlib -T linkfile.ld
    LDLIBS := -lkernel

    kernel: $(OBJECTS)
        $(LINK.o) $^ $(LOADLIBES) $(LDLIBS) -o $@

and everything else is taken care of by the default rules of gnu make,
including preprocessing and assembling .S files.

For executables running under my operating system it's even easier

    :::make
    CC := i586-pc-myos-gcc

That's all.

### GAS
On a final note, the automatic rules for building object files out of .S
files runs them through `CC`, which means they are assembled by
`i586-pc-myos-as` which uses AT&T syntax rather than NASM which uses
Intel syntax. I actually converted all my assembly code to AT&T syntax,
but you might want to use the `.intel_syntax` directive instead.


### Git
The methods described in this post has been implemented in the directory
`toolchain` of git commit
[f09bd57b8e](https://github.com/thomasloven/os5/tree/f09bd57b8eb8c27607dab8250574c8eaed2939a0/toolchain).