diff --git a/pages/2014-04-15-Dito-Framework.md b/pages/2014-04-15-Dito-Framework.md deleted file mode 100644 index f78cc98..0000000 --- a/pages/2014-04-15-Dito-Framework.md +++ /dev/null @@ -1,177 +0,0 @@ -layout: post -title: "DITo - Framework" -subtitle: "the Disk Image TOols" -tags: [osdev, filesystems] - -In my osdeving, I was starting to reach the point where a disk driver -seemed like the obvious next step. This was pretty much entirely unknown -territory for me. In fact, my only experience from disks and filesystems -were from when I got started in osdeving and found some tutorial in pdf -form which described how to write a bootloader in asm that read a kernel -from a FAT12 floppy disk. - -Since then, whenever I needed a disk image for testing, I'd go through -a painful process of finding an image with GRUB preinstalled, mounting -it using a discontinued third party application, copy stuff to it, hope -I would be able to unmount it without the entire computer freezing up -and finally pray that it worked when I started the emulator. In short, -trying to manage a disk image from the command line in OSX sucks. - -That's when I realized I could kill two birds with one stone. By writing -a tool for managing files in a disk image without mounting it, I could -gain understanding and experience of working with filesystems. If I -wrote it well, I would probably be able to reuse much of the code for -my kernel as well. At the time I had just finished my master thesis and -had all but signed the contract for my current employment, so I had some -free time on my hands while the paperwork fell through. - -The result was [DITo - Disk Image -Tools](https://github.com/thomasloven/dito), a c library and set of -applications for creating and handling disk images from the command -line. - -Recently, I actually did copy some of the code from DITo into my kernel. -Immagine my surprise when it actually worked like a charm after changing -only a few function calls. - -I've since realized a couple of mistakes though, and decided to rewrite -some parts from scratch. Let's go! - -###Drive operations - -The basic operations of DITo are reading from or writing to image files -or disk drives. Each drive type has a driver - - :::c - typedef struct drive_driver - { - int (*open)(struct drive_t *d, int flags); - int (*close)(struct drive_t *d, int flags); - int (*read)(struct drive_t *d, void *buffer, size_t length, off_t offset); - int (*write)(struct drive_t *d, void *buffer, size_t length, off_t offset); - } drive_driver_t; - -The drive type contains a pointer to the driver and a pointer to some -arbitrary data used by the driver. - - :::c - typedef struct drive_t - { - struct drive_driver *d; - void *data; - } drive_t; - -Then there are some wrapper functions for performing the required -operations: - - :::c - int drive_open(struct drive_t *d, int flags) - { - if(d->d->open) - return d->d->open(d, flags); - else - return 0; - } - -and simmilar for `drive_close`, `drive_read` and `drive_write`. - -###Filesystem operations - -The next important part of DITo is the filesystem handling. After -thinking about it, the important primitive functions for all file -operations I could think about are all in a filesystem driver struct: - - :::c - struct fs_driver - { - INODE (*open)(struct fs_t *fs, const char *path, int flags); - int (*close)(struct fs_t *fs, INODE ino); - int (*read)(struct fs_t *fs, INODE ino, void *buffer, size_t length, off_t offset); - int (*write)(struct fs_t *fs, INODE ino, void *buffer, size_t length, off_t offset); - int (*truncate)(struct fs_t *fs, INODE ino, off_t length); - int (*stat)(struct fs_t *fs, INODE ino, struct stat *st); - - int (*touch)(struct fs_t *fs, const char *path, struct stat *st); - int (*link)(struct fs_t *fs, const char *path1, const char *path2); - int (*unlink)(struct fs_t *fs, const char *path); - dirent_t *(*readdir)(struct fs_t *fs, INODE dir, unsigned int num); - }; - -The `fs_t` type contains a pointer to the driver, a pointer to the drive -and a general data pointer. - - :::c - typedef struct fs_t - { - struct fs_driver *driver; - drive_t *d; - void *data; - } fs_t; - -The wrapper functions `fs_open`, `fs_close` and so on work the same way -as the `drive_*` functions. - -The `INODE` type is a pointer to a struct containing a pointer to the -filesystem, a unique inode number and a pointer to arbitrary data. - - :::c - struct ino_st - { - fs_t *fs; - unsigned int ino; - void *data; - }; - - typedef struct ino_st * INODE; - -And that's the basic framework. As you probably notice, the same `fs_t` -pointer is passed to most functions twice. Once as `fs` and once as -`ino->fs`. I decided to keep it this way to get the function interface -consistant, and also for the possible sanity check `fs == ino->fs`. - -The idea behind the framework is that the same functions should be -usable for all kinds of filesystems on all kinds of drives. For example, -if I have one image of an FAT floppy disk with a file I want copied to -the ext2 formated second partition of a hard drive image, I could do -someting like this: - - :::c - drive_t *fat_disk = image_drive("floppy.img"); - drive_open(fat_disk, READ_FLAG); - drive_t *ext2_disk = image_drive("harddrive.img"); - drive_open(ext2_disk, READ_WRITE_FLAG); - drive_t *ext2_partition = mbr_drive(ext2_disk, 2); - drive_open(ext2_partition, READ_WRITE_FLAG); - - fs_t *fat = fat_fs(fat_disk); - fs_t *ext2 = ext2_fs(ext2_partition); - - INODE source = fs_open(fat, "/path/to/file", READ_FLAG); - struct st *st = malloc(sizeof(struct st)); - fs_struct(fat, source, st); - - fs_touch(ext2, "/new/path", st); - INODE destination = fs_open(ext2, "/new/path", WRITE_FLAG); - - void *buffer = malloc(BUFER_SIZE); - off_t offset = 0; - off_t add = 0; - while(add = fs_read(fat, source, buffer, BUFFER_SIZE, offset)) - { - fs_write(ext2, destination, buffer, BUFFER_SIZE, offset); - offset += add; - } - - fs_close(destination); - fs_close(source); - - drive_close(ext2_partition); - drive_close(fat_disk); - -Which of couse will eventually become its own tool so that the actual -work the end user has to do becomes: - - :::bash - $ dito-cp floppy.img:/path/to/file harddrive.img:2:/new/path - - diff --git a/pages/2014-04-16-Dito-Drives.md b/pages/2014-04-16-Dito-Drives.md deleted file mode 100644 index 9db25c5..0000000 --- a/pages/2014-04-16-Dito-Drives.md +++ /dev/null @@ -1,130 +0,0 @@ -layout: post -title: "DITo - Drives" -subtitle: "Exploring the MBR" -tags: [osdev, filesystems] - -Let's write a few drive drivers. - -### Pure image - -The simplest image we'd have to handle is a plain image file, such as -a floppy disk. There's only a single partition and a one-to-one mapping -from the image to the drive. - -First of all, let's think of what data we'd need to store. The filename -might be nice to have, and probably a pointer to the opened file. Would -hate to lose that... - - :::c - struct im_data - { - char *filename; - FILE *file; - }; - -The functions required for the driver would then make use of the -c standard library: - - :::c - int im_open(drive_t *d, int flags) - { - struct im_data *data = d->data; - data->file = fopen(data->filename, flags); - return 1; - } - int im_close(drive_t *d, int flags) - { - struct im_data *data = d->data; - fclose(data->file); - return 1; - } - int im_read(drive_t *d, void *buffer, size_t length, off_t offset) - { - struct im_data *data = d->data; - fseek(data->file, offset, SEEK_SET); - return fread(buffer, length, 1, data->file); - } - int im_write(drive_t *d, void *buffer, size_t length, off_t offset) - { - struct im_data *data = d->data; - fseek(data->file, offset, SEEK_SET); - return fwrite(buffer, length, 1, data->file); - } - -The function for setting up the drive is equally simple: - - :::c - drive_driver_t im_driver = - { - im_open, - im_close, - im_read, - im_write, - }; - - drive_t *image_drive(const char *filename) - { - drive_t *d = calloc(1, sizeof(drive_t)); - struct im_data *data = calloc(1, sizeof(struct im_data)); - - d->driver = &im_driver; - d->data = data; - data->filename = strndup(filename, FILENAME_MAX_LENGTH); - - return d; - } - -As always, I've stripped away all sanity checking and error handling. - -###MBR formated disks - -In order to use a harddrive for different things (like file storage or -swap space), it can be split into several partitions. One standard for -doing this is through a table in the MBR or Master Boot Record of the -disk. - -The MBR starts with 436 bytes of space for a bootloader, then there's a -10 byte disk ID followed by 4 partition table entries. - -The entries of the table has the following structure: - - :::c - struct MBR - { - uint8_t boot_indicator; - uint8_t start_chs[3]; - uint8_t system_id; - uint8_t end_chs[3]; - uint32_t start_lba; - uint32_t num_sectors; - }__attribute__((packed)); - -_CHS_ and _LBA_ are different ways of addressing sectors of the disk. - -CHS - or Cylinder Sector Head - addresses the sectors by their physical -location on the disk. The format in this table is first 8 bytes for -the wanted head (two heads per harddrive platter), then 6 bits for the -sector (angle on the platter) and finally 10 bits for the cylinder -(distance from platter center). This method requires knowledge of the -actual physical structure of the drive and maxes out at around 8 GB with -512 byte sectors. - -LBA - or Logical Block Addressing - leaves all that to the drive and -addresses the sectors by a single number. With 512 byte sectors, this is -useful up to about 2 GB. - -For our purposes, the fields `start_lba` and `num_sectors` are all we -need (and `system_id` to check if the partition table entry is used). - -###MBR driver - -The driver for reading from a partition is pretty alike the plain image -driver. Principally, the only difference is that 512 times `start_lba` -has to be added to the offset for reading and writing, and that care -must be taken not to allow reading or writing outside of the partition -boundaries. - -I even wrote the driver to piggyback on a different driver. I.e. the -function for creating the drive object requires a drive object as -argument (as in the [example in the previous -post](/blog/2014/04/Dito-Framework/)). diff --git a/pages/2014-04-17-Dito-Ext2.md b/pages/2014-04-17-Dito-Ext2.md deleted file mode 100644 index d9b1679..0000000 --- a/pages/2014-04-17-Dito-Ext2.md +++ /dev/null @@ -1,8 +0,0 @@ -layout: post -title: "DITo - Ext2" -subtitle: "A beginning..." -tags: [osdev, filesystems] - -It's finally time to start looking at an actual filesystem. - -I chose to begin with ext2.