Remove unpublished articles from master branch
This commit is contained in:
parent
ae0c4ee4ab
commit
bf9e821772
@ -1,177 +0,0 @@
|
|||||||
layout: post
|
|
||||||
title: "DITo - Framework"
|
|
||||||
subtitle: "the Disk Image TOols"
|
|
||||||
tags: [osdev, filesystems]
|
|
||||||
|
|
||||||
In my osdeving, I was starting to reach the point where a disk driver
|
|
||||||
seemed like the obvious next step. This was pretty much entirely unknown
|
|
||||||
territory for me. In fact, my only experience from disks and filesystems
|
|
||||||
were from when I got started in osdeving and found some tutorial in pdf
|
|
||||||
form which described how to write a bootloader in asm that read a kernel
|
|
||||||
from a FAT12 floppy disk.
|
|
||||||
|
|
||||||
Since then, whenever I needed a disk image for testing, I'd go through
|
|
||||||
a painful process of finding an image with GRUB preinstalled, mounting
|
|
||||||
it using a discontinued third party application, copy stuff to it, hope
|
|
||||||
I would be able to unmount it without the entire computer freezing up
|
|
||||||
and finally pray that it worked when I started the emulator. In short,
|
|
||||||
trying to manage a disk image from the command line in OSX sucks.
|
|
||||||
|
|
||||||
That's when I realized I could kill two birds with one stone. By writing
|
|
||||||
a tool for managing files in a disk image without mounting it, I could
|
|
||||||
gain understanding and experience of working with filesystems. If I
|
|
||||||
wrote it well, I would probably be able to reuse much of the code for
|
|
||||||
my kernel as well. At the time I had just finished my master thesis and
|
|
||||||
had all but signed the contract for my current employment, so I had some
|
|
||||||
free time on my hands while the paperwork fell through.
|
|
||||||
|
|
||||||
The result was [DITo - Disk Image
|
|
||||||
Tools](https://github.com/thomasloven/dito), a c library and set of
|
|
||||||
applications for creating and handling disk images from the command
|
|
||||||
line.
|
|
||||||
|
|
||||||
Recently, I actually did copy some of the code from DITo into my kernel.
|
|
||||||
Immagine my surprise when it actually worked like a charm after changing
|
|
||||||
only a few function calls.
|
|
||||||
|
|
||||||
I've since realized a couple of mistakes though, and decided to rewrite
|
|
||||||
some parts from scratch. Let's go!
|
|
||||||
|
|
||||||
###Drive operations
|
|
||||||
|
|
||||||
The basic operations of DITo are reading from or writing to image files
|
|
||||||
or disk drives. Each drive type has a driver
|
|
||||||
|
|
||||||
:::c
|
|
||||||
typedef struct drive_driver
|
|
||||||
{
|
|
||||||
int (*open)(struct drive_t *d, int flags);
|
|
||||||
int (*close)(struct drive_t *d, int flags);
|
|
||||||
int (*read)(struct drive_t *d, void *buffer, size_t length, off_t offset);
|
|
||||||
int (*write)(struct drive_t *d, void *buffer, size_t length, off_t offset);
|
|
||||||
} drive_driver_t;
|
|
||||||
|
|
||||||
The drive type contains a pointer to the driver and a pointer to some
|
|
||||||
arbitrary data used by the driver.
|
|
||||||
|
|
||||||
:::c
|
|
||||||
typedef struct drive_t
|
|
||||||
{
|
|
||||||
struct drive_driver *d;
|
|
||||||
void *data;
|
|
||||||
} drive_t;
|
|
||||||
|
|
||||||
Then there are some wrapper functions for performing the required
|
|
||||||
operations:
|
|
||||||
|
|
||||||
:::c
|
|
||||||
int drive_open(struct drive_t *d, int flags)
|
|
||||||
{
|
|
||||||
if(d->d->open)
|
|
||||||
return d->d->open(d, flags);
|
|
||||||
else
|
|
||||||
return 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
and simmilar for `drive_close`, `drive_read` and `drive_write`.
|
|
||||||
|
|
||||||
###Filesystem operations
|
|
||||||
|
|
||||||
The next important part of DITo is the filesystem handling. After
|
|
||||||
thinking about it, the important primitive functions for all file
|
|
||||||
operations I could think about are all in a filesystem driver struct:
|
|
||||||
|
|
||||||
:::c
|
|
||||||
struct fs_driver
|
|
||||||
{
|
|
||||||
INODE (*open)(struct fs_t *fs, const char *path, int flags);
|
|
||||||
int (*close)(struct fs_t *fs, INODE ino);
|
|
||||||
int (*read)(struct fs_t *fs, INODE ino, void *buffer, size_t length, off_t offset);
|
|
||||||
int (*write)(struct fs_t *fs, INODE ino, void *buffer, size_t length, off_t offset);
|
|
||||||
int (*truncate)(struct fs_t *fs, INODE ino, off_t length);
|
|
||||||
int (*stat)(struct fs_t *fs, INODE ino, struct stat *st);
|
|
||||||
|
|
||||||
int (*touch)(struct fs_t *fs, const char *path, struct stat *st);
|
|
||||||
int (*link)(struct fs_t *fs, const char *path1, const char *path2);
|
|
||||||
int (*unlink)(struct fs_t *fs, const char *path);
|
|
||||||
dirent_t *(*readdir)(struct fs_t *fs, INODE dir, unsigned int num);
|
|
||||||
};
|
|
||||||
|
|
||||||
The `fs_t` type contains a pointer to the driver, a pointer to the drive
|
|
||||||
and a general data pointer.
|
|
||||||
|
|
||||||
:::c
|
|
||||||
typedef struct fs_t
|
|
||||||
{
|
|
||||||
struct fs_driver *driver;
|
|
||||||
drive_t *d;
|
|
||||||
void *data;
|
|
||||||
} fs_t;
|
|
||||||
|
|
||||||
The wrapper functions `fs_open`, `fs_close` and so on work the same way
|
|
||||||
as the `drive_*` functions.
|
|
||||||
|
|
||||||
The `INODE` type is a pointer to a struct containing a pointer to the
|
|
||||||
filesystem, a unique inode number and a pointer to arbitrary data.
|
|
||||||
|
|
||||||
:::c
|
|
||||||
struct ino_st
|
|
||||||
{
|
|
||||||
fs_t *fs;
|
|
||||||
unsigned int ino;
|
|
||||||
void *data;
|
|
||||||
};
|
|
||||||
|
|
||||||
typedef struct ino_st * INODE;
|
|
||||||
|
|
||||||
And that's the basic framework. As you probably notice, the same `fs_t`
|
|
||||||
pointer is passed to most functions twice. Once as `fs` and once as
|
|
||||||
`ino->fs`. I decided to keep it this way to get the function interface
|
|
||||||
consistant, and also for the possible sanity check `fs == ino->fs`.
|
|
||||||
|
|
||||||
The idea behind the framework is that the same functions should be
|
|
||||||
usable for all kinds of filesystems on all kinds of drives. For example,
|
|
||||||
if I have one image of an FAT floppy disk with a file I want copied to
|
|
||||||
the ext2 formated second partition of a hard drive image, I could do
|
|
||||||
someting like this:
|
|
||||||
|
|
||||||
:::c
|
|
||||||
drive_t *fat_disk = image_drive("floppy.img");
|
|
||||||
drive_open(fat_disk, READ_FLAG);
|
|
||||||
drive_t *ext2_disk = image_drive("harddrive.img");
|
|
||||||
drive_open(ext2_disk, READ_WRITE_FLAG);
|
|
||||||
drive_t *ext2_partition = mbr_drive(ext2_disk, 2);
|
|
||||||
drive_open(ext2_partition, READ_WRITE_FLAG);
|
|
||||||
|
|
||||||
fs_t *fat = fat_fs(fat_disk);
|
|
||||||
fs_t *ext2 = ext2_fs(ext2_partition);
|
|
||||||
|
|
||||||
INODE source = fs_open(fat, "/path/to/file", READ_FLAG);
|
|
||||||
struct st *st = malloc(sizeof(struct st));
|
|
||||||
fs_struct(fat, source, st);
|
|
||||||
|
|
||||||
fs_touch(ext2, "/new/path", st);
|
|
||||||
INODE destination = fs_open(ext2, "/new/path", WRITE_FLAG);
|
|
||||||
|
|
||||||
void *buffer = malloc(BUFER_SIZE);
|
|
||||||
off_t offset = 0;
|
|
||||||
off_t add = 0;
|
|
||||||
while(add = fs_read(fat, source, buffer, BUFFER_SIZE, offset))
|
|
||||||
{
|
|
||||||
fs_write(ext2, destination, buffer, BUFFER_SIZE, offset);
|
|
||||||
offset += add;
|
|
||||||
}
|
|
||||||
|
|
||||||
fs_close(destination);
|
|
||||||
fs_close(source);
|
|
||||||
|
|
||||||
drive_close(ext2_partition);
|
|
||||||
drive_close(fat_disk);
|
|
||||||
|
|
||||||
Which of couse will eventually become its own tool so that the actual
|
|
||||||
work the end user has to do becomes:
|
|
||||||
|
|
||||||
:::bash
|
|
||||||
$ dito-cp floppy.img:/path/to/file harddrive.img:2:/new/path
|
|
||||||
|
|
||||||
|
|
||||||
@ -1,130 +0,0 @@
|
|||||||
layout: post
|
|
||||||
title: "DITo - Drives"
|
|
||||||
subtitle: "Exploring the MBR"
|
|
||||||
tags: [osdev, filesystems]
|
|
||||||
|
|
||||||
Let's write a few drive drivers.
|
|
||||||
|
|
||||||
### Pure image
|
|
||||||
|
|
||||||
The simplest image we'd have to handle is a plain image file, such as
|
|
||||||
a floppy disk. There's only a single partition and a one-to-one mapping
|
|
||||||
from the image to the drive.
|
|
||||||
|
|
||||||
First of all, let's think of what data we'd need to store. The filename
|
|
||||||
might be nice to have, and probably a pointer to the opened file. Would
|
|
||||||
hate to lose that...
|
|
||||||
|
|
||||||
:::c
|
|
||||||
struct im_data
|
|
||||||
{
|
|
||||||
char *filename;
|
|
||||||
FILE *file;
|
|
||||||
};
|
|
||||||
|
|
||||||
The functions required for the driver would then make use of the
|
|
||||||
c standard library:
|
|
||||||
|
|
||||||
:::c
|
|
||||||
int im_open(drive_t *d, int flags)
|
|
||||||
{
|
|
||||||
struct im_data *data = d->data;
|
|
||||||
data->file = fopen(data->filename, flags);
|
|
||||||
return 1;
|
|
||||||
}
|
|
||||||
int im_close(drive_t *d, int flags)
|
|
||||||
{
|
|
||||||
struct im_data *data = d->data;
|
|
||||||
fclose(data->file);
|
|
||||||
return 1;
|
|
||||||
}
|
|
||||||
int im_read(drive_t *d, void *buffer, size_t length, off_t offset)
|
|
||||||
{
|
|
||||||
struct im_data *data = d->data;
|
|
||||||
fseek(data->file, offset, SEEK_SET);
|
|
||||||
return fread(buffer, length, 1, data->file);
|
|
||||||
}
|
|
||||||
int im_write(drive_t *d, void *buffer, size_t length, off_t offset)
|
|
||||||
{
|
|
||||||
struct im_data *data = d->data;
|
|
||||||
fseek(data->file, offset, SEEK_SET);
|
|
||||||
return fwrite(buffer, length, 1, data->file);
|
|
||||||
}
|
|
||||||
|
|
||||||
The function for setting up the drive is equally simple:
|
|
||||||
|
|
||||||
:::c
|
|
||||||
drive_driver_t im_driver =
|
|
||||||
{
|
|
||||||
im_open,
|
|
||||||
im_close,
|
|
||||||
im_read,
|
|
||||||
im_write,
|
|
||||||
};
|
|
||||||
|
|
||||||
drive_t *image_drive(const char *filename)
|
|
||||||
{
|
|
||||||
drive_t *d = calloc(1, sizeof(drive_t));
|
|
||||||
struct im_data *data = calloc(1, sizeof(struct im_data));
|
|
||||||
|
|
||||||
d->driver = &im_driver;
|
|
||||||
d->data = data;
|
|
||||||
data->filename = strndup(filename, FILENAME_MAX_LENGTH);
|
|
||||||
|
|
||||||
return d;
|
|
||||||
}
|
|
||||||
|
|
||||||
As always, I've stripped away all sanity checking and error handling.
|
|
||||||
|
|
||||||
###MBR formated disks
|
|
||||||
|
|
||||||
In order to use a harddrive for different things (like file storage or
|
|
||||||
swap space), it can be split into several partitions. One standard for
|
|
||||||
doing this is through a table in the MBR or Master Boot Record of the
|
|
||||||
disk.
|
|
||||||
|
|
||||||
The MBR starts with 436 bytes of space for a bootloader, then there's a
|
|
||||||
10 byte disk ID followed by 4 partition table entries.
|
|
||||||
|
|
||||||
The entries of the table has the following structure:
|
|
||||||
|
|
||||||
:::c
|
|
||||||
struct MBR
|
|
||||||
{
|
|
||||||
uint8_t boot_indicator;
|
|
||||||
uint8_t start_chs[3];
|
|
||||||
uint8_t system_id;
|
|
||||||
uint8_t end_chs[3];
|
|
||||||
uint32_t start_lba;
|
|
||||||
uint32_t num_sectors;
|
|
||||||
}__attribute__((packed));
|
|
||||||
|
|
||||||
_CHS_ and _LBA_ are different ways of addressing sectors of the disk.
|
|
||||||
|
|
||||||
CHS - or Cylinder Sector Head - addresses the sectors by their physical
|
|
||||||
location on the disk. The format in this table is first 8 bytes for
|
|
||||||
the wanted head (two heads per harddrive platter), then 6 bits for the
|
|
||||||
sector (angle on the platter) and finally 10 bits for the cylinder
|
|
||||||
(distance from platter center). This method requires knowledge of the
|
|
||||||
actual physical structure of the drive and maxes out at around 8 GB with
|
|
||||||
512 byte sectors.
|
|
||||||
|
|
||||||
LBA - or Logical Block Addressing - leaves all that to the drive and
|
|
||||||
addresses the sectors by a single number. With 512 byte sectors, this is
|
|
||||||
useful up to about 2 GB.
|
|
||||||
|
|
||||||
For our purposes, the fields `start_lba` and `num_sectors` are all we
|
|
||||||
need (and `system_id` to check if the partition table entry is used).
|
|
||||||
|
|
||||||
###MBR driver
|
|
||||||
|
|
||||||
The driver for reading from a partition is pretty alike the plain image
|
|
||||||
driver. Principally, the only difference is that 512 times `start_lba`
|
|
||||||
has to be added to the offset for reading and writing, and that care
|
|
||||||
must be taken not to allow reading or writing outside of the partition
|
|
||||||
boundaries.
|
|
||||||
|
|
||||||
I even wrote the driver to piggyback on a different driver. I.e. the
|
|
||||||
function for creating the drive object requires a drive object as
|
|
||||||
argument (as in the [example in the previous
|
|
||||||
post](/blog/2014/04/Dito-Framework/)).
|
|
||||||
@ -1,8 +0,0 @@
|
|||||||
layout: post
|
|
||||||
title: "DITo - Ext2"
|
|
||||||
subtitle: "A beginning..."
|
|
||||||
tags: [osdev, filesystems]
|
|
||||||
|
|
||||||
It's finally time to start looking at an actual filesystem.
|
|
||||||
|
|
||||||
I chose to begin with ext2.
|
|
||||||
Loading…
x
Reference in New Issue
Block a user