Remove unpublished articles from master branch
This commit is contained in:
parent
ae0c4ee4ab
commit
bf9e821772
@ -1,177 +0,0 @@
|
||||
layout: post
|
||||
title: "DITo - Framework"
|
||||
subtitle: "the Disk Image TOols"
|
||||
tags: [osdev, filesystems]
|
||||
|
||||
In my osdeving, I was starting to reach the point where a disk driver
|
||||
seemed like the obvious next step. This was pretty much entirely unknown
|
||||
territory for me. In fact, my only experience from disks and filesystems
|
||||
were from when I got started in osdeving and found some tutorial in pdf
|
||||
form which described how to write a bootloader in asm that read a kernel
|
||||
from a FAT12 floppy disk.
|
||||
|
||||
Since then, whenever I needed a disk image for testing, I'd go through
|
||||
a painful process of finding an image with GRUB preinstalled, mounting
|
||||
it using a discontinued third party application, copy stuff to it, hope
|
||||
I would be able to unmount it without the entire computer freezing up
|
||||
and finally pray that it worked when I started the emulator. In short,
|
||||
trying to manage a disk image from the command line in OSX sucks.
|
||||
|
||||
That's when I realized I could kill two birds with one stone. By writing
|
||||
a tool for managing files in a disk image without mounting it, I could
|
||||
gain understanding and experience of working with filesystems. If I
|
||||
wrote it well, I would probably be able to reuse much of the code for
|
||||
my kernel as well. At the time I had just finished my master thesis and
|
||||
had all but signed the contract for my current employment, so I had some
|
||||
free time on my hands while the paperwork fell through.
|
||||
|
||||
The result was [DITo - Disk Image
|
||||
Tools](https://github.com/thomasloven/dito), a c library and set of
|
||||
applications for creating and handling disk images from the command
|
||||
line.
|
||||
|
||||
Recently, I actually did copy some of the code from DITo into my kernel.
|
||||
Immagine my surprise when it actually worked like a charm after changing
|
||||
only a few function calls.
|
||||
|
||||
I've since realized a couple of mistakes though, and decided to rewrite
|
||||
some parts from scratch. Let's go!
|
||||
|
||||
###Drive operations
|
||||
|
||||
The basic operations of DITo are reading from or writing to image files
|
||||
or disk drives. Each drive type has a driver
|
||||
|
||||
:::c
|
||||
typedef struct drive_driver
|
||||
{
|
||||
int (*open)(struct drive_t *d, int flags);
|
||||
int (*close)(struct drive_t *d, int flags);
|
||||
int (*read)(struct drive_t *d, void *buffer, size_t length, off_t offset);
|
||||
int (*write)(struct drive_t *d, void *buffer, size_t length, off_t offset);
|
||||
} drive_driver_t;
|
||||
|
||||
The drive type contains a pointer to the driver and a pointer to some
|
||||
arbitrary data used by the driver.
|
||||
|
||||
:::c
|
||||
typedef struct drive_t
|
||||
{
|
||||
struct drive_driver *d;
|
||||
void *data;
|
||||
} drive_t;
|
||||
|
||||
Then there are some wrapper functions for performing the required
|
||||
operations:
|
||||
|
||||
:::c
|
||||
int drive_open(struct drive_t *d, int flags)
|
||||
{
|
||||
if(d->d->open)
|
||||
return d->d->open(d, flags);
|
||||
else
|
||||
return 0;
|
||||
}
|
||||
|
||||
and simmilar for `drive_close`, `drive_read` and `drive_write`.
|
||||
|
||||
###Filesystem operations
|
||||
|
||||
The next important part of DITo is the filesystem handling. After
|
||||
thinking about it, the important primitive functions for all file
|
||||
operations I could think about are all in a filesystem driver struct:
|
||||
|
||||
:::c
|
||||
struct fs_driver
|
||||
{
|
||||
INODE (*open)(struct fs_t *fs, const char *path, int flags);
|
||||
int (*close)(struct fs_t *fs, INODE ino);
|
||||
int (*read)(struct fs_t *fs, INODE ino, void *buffer, size_t length, off_t offset);
|
||||
int (*write)(struct fs_t *fs, INODE ino, void *buffer, size_t length, off_t offset);
|
||||
int (*truncate)(struct fs_t *fs, INODE ino, off_t length);
|
||||
int (*stat)(struct fs_t *fs, INODE ino, struct stat *st);
|
||||
|
||||
int (*touch)(struct fs_t *fs, const char *path, struct stat *st);
|
||||
int (*link)(struct fs_t *fs, const char *path1, const char *path2);
|
||||
int (*unlink)(struct fs_t *fs, const char *path);
|
||||
dirent_t *(*readdir)(struct fs_t *fs, INODE dir, unsigned int num);
|
||||
};
|
||||
|
||||
The `fs_t` type contains a pointer to the driver, a pointer to the drive
|
||||
and a general data pointer.
|
||||
|
||||
:::c
|
||||
typedef struct fs_t
|
||||
{
|
||||
struct fs_driver *driver;
|
||||
drive_t *d;
|
||||
void *data;
|
||||
} fs_t;
|
||||
|
||||
The wrapper functions `fs_open`, `fs_close` and so on work the same way
|
||||
as the `drive_*` functions.
|
||||
|
||||
The `INODE` type is a pointer to a struct containing a pointer to the
|
||||
filesystem, a unique inode number and a pointer to arbitrary data.
|
||||
|
||||
:::c
|
||||
struct ino_st
|
||||
{
|
||||
fs_t *fs;
|
||||
unsigned int ino;
|
||||
void *data;
|
||||
};
|
||||
|
||||
typedef struct ino_st * INODE;
|
||||
|
||||
And that's the basic framework. As you probably notice, the same `fs_t`
|
||||
pointer is passed to most functions twice. Once as `fs` and once as
|
||||
`ino->fs`. I decided to keep it this way to get the function interface
|
||||
consistant, and also for the possible sanity check `fs == ino->fs`.
|
||||
|
||||
The idea behind the framework is that the same functions should be
|
||||
usable for all kinds of filesystems on all kinds of drives. For example,
|
||||
if I have one image of an FAT floppy disk with a file I want copied to
|
||||
the ext2 formated second partition of a hard drive image, I could do
|
||||
someting like this:
|
||||
|
||||
:::c
|
||||
drive_t *fat_disk = image_drive("floppy.img");
|
||||
drive_open(fat_disk, READ_FLAG);
|
||||
drive_t *ext2_disk = image_drive("harddrive.img");
|
||||
drive_open(ext2_disk, READ_WRITE_FLAG);
|
||||
drive_t *ext2_partition = mbr_drive(ext2_disk, 2);
|
||||
drive_open(ext2_partition, READ_WRITE_FLAG);
|
||||
|
||||
fs_t *fat = fat_fs(fat_disk);
|
||||
fs_t *ext2 = ext2_fs(ext2_partition);
|
||||
|
||||
INODE source = fs_open(fat, "/path/to/file", READ_FLAG);
|
||||
struct st *st = malloc(sizeof(struct st));
|
||||
fs_struct(fat, source, st);
|
||||
|
||||
fs_touch(ext2, "/new/path", st);
|
||||
INODE destination = fs_open(ext2, "/new/path", WRITE_FLAG);
|
||||
|
||||
void *buffer = malloc(BUFER_SIZE);
|
||||
off_t offset = 0;
|
||||
off_t add = 0;
|
||||
while(add = fs_read(fat, source, buffer, BUFFER_SIZE, offset))
|
||||
{
|
||||
fs_write(ext2, destination, buffer, BUFFER_SIZE, offset);
|
||||
offset += add;
|
||||
}
|
||||
|
||||
fs_close(destination);
|
||||
fs_close(source);
|
||||
|
||||
drive_close(ext2_partition);
|
||||
drive_close(fat_disk);
|
||||
|
||||
Which of couse will eventually become its own tool so that the actual
|
||||
work the end user has to do becomes:
|
||||
|
||||
:::bash
|
||||
$ dito-cp floppy.img:/path/to/file harddrive.img:2:/new/path
|
||||
|
||||
|
||||
@ -1,130 +0,0 @@
|
||||
layout: post
|
||||
title: "DITo - Drives"
|
||||
subtitle: "Exploring the MBR"
|
||||
tags: [osdev, filesystems]
|
||||
|
||||
Let's write a few drive drivers.
|
||||
|
||||
### Pure image
|
||||
|
||||
The simplest image we'd have to handle is a plain image file, such as
|
||||
a floppy disk. There's only a single partition and a one-to-one mapping
|
||||
from the image to the drive.
|
||||
|
||||
First of all, let's think of what data we'd need to store. The filename
|
||||
might be nice to have, and probably a pointer to the opened file. Would
|
||||
hate to lose that...
|
||||
|
||||
:::c
|
||||
struct im_data
|
||||
{
|
||||
char *filename;
|
||||
FILE *file;
|
||||
};
|
||||
|
||||
The functions required for the driver would then make use of the
|
||||
c standard library:
|
||||
|
||||
:::c
|
||||
int im_open(drive_t *d, int flags)
|
||||
{
|
||||
struct im_data *data = d->data;
|
||||
data->file = fopen(data->filename, flags);
|
||||
return 1;
|
||||
}
|
||||
int im_close(drive_t *d, int flags)
|
||||
{
|
||||
struct im_data *data = d->data;
|
||||
fclose(data->file);
|
||||
return 1;
|
||||
}
|
||||
int im_read(drive_t *d, void *buffer, size_t length, off_t offset)
|
||||
{
|
||||
struct im_data *data = d->data;
|
||||
fseek(data->file, offset, SEEK_SET);
|
||||
return fread(buffer, length, 1, data->file);
|
||||
}
|
||||
int im_write(drive_t *d, void *buffer, size_t length, off_t offset)
|
||||
{
|
||||
struct im_data *data = d->data;
|
||||
fseek(data->file, offset, SEEK_SET);
|
||||
return fwrite(buffer, length, 1, data->file);
|
||||
}
|
||||
|
||||
The function for setting up the drive is equally simple:
|
||||
|
||||
:::c
|
||||
drive_driver_t im_driver =
|
||||
{
|
||||
im_open,
|
||||
im_close,
|
||||
im_read,
|
||||
im_write,
|
||||
};
|
||||
|
||||
drive_t *image_drive(const char *filename)
|
||||
{
|
||||
drive_t *d = calloc(1, sizeof(drive_t));
|
||||
struct im_data *data = calloc(1, sizeof(struct im_data));
|
||||
|
||||
d->driver = &im_driver;
|
||||
d->data = data;
|
||||
data->filename = strndup(filename, FILENAME_MAX_LENGTH);
|
||||
|
||||
return d;
|
||||
}
|
||||
|
||||
As always, I've stripped away all sanity checking and error handling.
|
||||
|
||||
###MBR formated disks
|
||||
|
||||
In order to use a harddrive for different things (like file storage or
|
||||
swap space), it can be split into several partitions. One standard for
|
||||
doing this is through a table in the MBR or Master Boot Record of the
|
||||
disk.
|
||||
|
||||
The MBR starts with 436 bytes of space for a bootloader, then there's a
|
||||
10 byte disk ID followed by 4 partition table entries.
|
||||
|
||||
The entries of the table has the following structure:
|
||||
|
||||
:::c
|
||||
struct MBR
|
||||
{
|
||||
uint8_t boot_indicator;
|
||||
uint8_t start_chs[3];
|
||||
uint8_t system_id;
|
||||
uint8_t end_chs[3];
|
||||
uint32_t start_lba;
|
||||
uint32_t num_sectors;
|
||||
}__attribute__((packed));
|
||||
|
||||
_CHS_ and _LBA_ are different ways of addressing sectors of the disk.
|
||||
|
||||
CHS - or Cylinder Sector Head - addresses the sectors by their physical
|
||||
location on the disk. The format in this table is first 8 bytes for
|
||||
the wanted head (two heads per harddrive platter), then 6 bits for the
|
||||
sector (angle on the platter) and finally 10 bits for the cylinder
|
||||
(distance from platter center). This method requires knowledge of the
|
||||
actual physical structure of the drive and maxes out at around 8 GB with
|
||||
512 byte sectors.
|
||||
|
||||
LBA - or Logical Block Addressing - leaves all that to the drive and
|
||||
addresses the sectors by a single number. With 512 byte sectors, this is
|
||||
useful up to about 2 GB.
|
||||
|
||||
For our purposes, the fields `start_lba` and `num_sectors` are all we
|
||||
need (and `system_id` to check if the partition table entry is used).
|
||||
|
||||
###MBR driver
|
||||
|
||||
The driver for reading from a partition is pretty alike the plain image
|
||||
driver. Principally, the only difference is that 512 times `start_lba`
|
||||
has to be added to the offset for reading and writing, and that care
|
||||
must be taken not to allow reading or writing outside of the partition
|
||||
boundaries.
|
||||
|
||||
I even wrote the driver to piggyback on a different driver. I.e. the
|
||||
function for creating the drive object requires a drive object as
|
||||
argument (as in the [example in the previous
|
||||
post](/blog/2014/04/Dito-Framework/)).
|
||||
@ -1,8 +0,0 @@
|
||||
layout: post
|
||||
title: "DITo - Ext2"
|
||||
subtitle: "A beginning..."
|
||||
tags: [osdev, filesystems]
|
||||
|
||||
It's finally time to start looking at an actual filesystem.
|
||||
|
||||
I chose to begin with ext2.
|
||||
Loading…
x
Reference in New Issue
Block a user