233 lines
7.4 KiB
Markdown
233 lines
7.4 KiB
Markdown
layout: post
|
|
title: "TAR Filesystem"
|
|
subtitle: "Almost a useful system"
|
|
tags: [osdev, filesystems]
|
|
|
|
It's finally time to implement an actuall filesystem, and all the hard
|
|
work with the VFS framework will pay off.
|
|
|
|
For my first filesystem, I chose the tape archive format - also known as
|
|
a tarball. It's a bit like what
|
|
[James Molloy](http://www.jamesmolloy.co.uk/tutorial_html/8.-The%20VFS%20and%20the%20initrd.html)
|
|
did but with support for directories.
|
|
|
|
###Tar file format
|
|
The tar file format - as the name implies - was designed for storing
|
|
files on magnetic tape, which can only be read and written sequentially.
|
|
Therefore, there is no index of the files in the archive, but each file
|
|
is preceded by a data block in human-readable format.
|
|
|
|
In the POSIX standard of 1988, the data block has the following format:
|
|
|
|
:::c
|
|
typedef struct
|
|
{
|
|
unsigned char name[100];
|
|
unsigned char mode[8];
|
|
unsigned char uid[8];
|
|
unsigned char gid[8];
|
|
unsigned char size[12];
|
|
unsigned char mtime[12];
|
|
unsigned char checksum[8];
|
|
unsigned char type[1];
|
|
unsigned char linkname[100];
|
|
unsigned char tar_indicator[6];
|
|
unsigned char tar_version[2];
|
|
unsigned char owner[32];
|
|
unsigned char group[32];
|
|
unsigned char device_major[8];
|
|
unsigned char device_minor[8];
|
|
unsigned char prefix[155];
|
|
}__attribute__((packed)) tar_header_t;
|
|
|
|
The data block is 500 bytes long, but the data starts 512 (one standard
|
|
disk sector) bytes after the start of the data block
|
|
The data itself is also padded so that the next data block starts at
|
|
a 512 byte boundary.
|
|
|
|
For historical reasons all numerical values (i.e. `uid`, `gid`, `size`,
|
|
`mtime`, `checksum`, `device_major` and `device_minor`) are in __octal__
|
|
ascii format (ascii `'0'` to `'7'`) padded with leading zeroes and
|
|
terminated with a `null` character (`0x00`) or space (`0x20`).
|
|
|
|
###Reshaping the tree
|
|
The TAR format is very good for storing data on tape where read and
|
|
write operations are sequential, but it makes less sense once the files
|
|
are loaded into ram and you might want to access them randomly.
|
|
|
|
Instead, I reshape the file list into a file tree:
|
|
|
|
:::c
|
|
tree_t *build_tar_tree(tar_header_t *tar)
|
|
{
|
|
...
|
|
while(tar->name[0])
|
|
{
|
|
tartree_add_node(tree, tar, (char *)&tar->name);
|
|
uint32_t size;
|
|
sscanf((char *)&tar->size, "%o", &size);
|
|
tar = (tar_header_t *)((size_t)tar + size + 512);
|
|
if((size_t)tar % 512)
|
|
tar = (tar_header_t *)((uint32_t)tar + 512 - ((uint32_t)tar%512))
|
|
}
|
|
...
|
|
}
|
|
|
|
void tartree_add_node(tree_t *tree, tar_header_t *tar, char *path)
|
|
{
|
|
...
|
|
tree_node_t *node = tree->root;
|
|
char *p;
|
|
for(p = strtok(path, "/"); p; p = strtok(NULL, "/"))
|
|
{
|
|
int found = 0;
|
|
list_t *l;
|
|
for_each_in_list(&node->children)
|
|
{
|
|
...
|
|
if(!strcmp(entry->name, p))
|
|
{
|
|
found = 1;
|
|
node = tn;
|
|
break;
|
|
}
|
|
}
|
|
if(!found)
|
|
{
|
|
...
|
|
tarfs_entry_t *n = malloc(sizeof(tar_entry_t));
|
|
n->name = strdup(p);
|
|
n->tar = tar;
|
|
...
|
|
tree_make_child(node, new);
|
|
node = new;
|
|
}
|
|
}
|
|
}
|
|
|
|
Note that this assumes that the files and directories of the tar archive
|
|
are in top-down order, e.g. that `/bin` is before `/bin/echo`, otherwise
|
|
strange things happen. Deterministic and expected things, mind you, but
|
|
strange.
|
|
|
|
###Mounting the filesystem
|
|
The tarfs driver makes use of the data field in the vfs node to store
|
|
the tar tree.
|
|
|
|
:::c
|
|
INODE tarfs_init(tar_header_t *tar)
|
|
{
|
|
vfs_node_t *node = calloc(1, sizeof(vfs_node_t));
|
|
strcpy(node->name, "tarfs");
|
|
node->d = &tarfs_driver;
|
|
node->type = FS_DIRECTORY;
|
|
|
|
tree_t *tar_tree = build_tar_tree(tar);
|
|
node->data = tar_tree->root;
|
|
free(tar_tree);
|
|
|
|
return node;
|
|
}
|
|
|
|
I then add the following to my kernel initialization function:
|
|
|
|
:::c
|
|
tar_header_t *tarfs_location = assert_higher((tar_header_t *)mods[0].mod_start);
|
|
vfs_mount("/", tarfs_init(tarfs_location));
|
|
|
|
where `mods` is the multiboot modules table as loaded by grub.
|
|
|
|
###The tarfs driver
|
|
Finally, the tarfs driver functions. For now I only need to implement
|
|
`read()` and `finddir()`.
|
|
|
|
:::c
|
|
INODE tar_finddir(INODE dir, const char *name)
|
|
{
|
|
tree_node_t *tn = (tree_node_t *)dir->data;
|
|
list_t *l;
|
|
for_each_in_list(&tn->children, l)
|
|
{
|
|
tree_node_t *cn = list_entry(l, tree_node_t, siblings);
|
|
...
|
|
if(!strcmp(entry->name, name)
|
|
{
|
|
INODE node = calloc(1, sizeof(vfs_node_t));
|
|
strcpy(node->name, entry->name);
|
|
node->d = &tarfs_driver;
|
|
node->data = (void *)cn;
|
|
sscanf((char *)&entry->tar->size, "%o", &node->length);
|
|
if(entry->tar->type[0] == TAR_TYPE_DIR)
|
|
node->type = FS_DIRECTORY;
|
|
else
|
|
node->type = FS_FILE;
|
|
return node;
|
|
}
|
|
}
|
|
return 0;
|
|
}
|
|
|
|
`Finddir` allocates space for a new inode for each file that is searched
|
|
for. It's up to the caller to free the inode when it's done with it.
|
|
This should be true for all `finddir` functions, but I think I've missed
|
|
it at a few places... Someday I will clean up all my memory leaks.
|
|
|
|
Anyway, `finddir` also finds the right node in the tarfs tree and puts
|
|
it in the `data` field of the inode.
|
|
|
|
:::c
|
|
uint32_t read_tar(INODE node, void *buffer, uint32_t size, uint32_t offset)
|
|
{
|
|
tree_node_t *tn = (tree_node_t *)node->data;
|
|
tarfs_entry_t *te = (tarfs_entry_t *)tn->item;
|
|
tar_header_t *tar = te->tar;
|
|
|
|
uint32_t tsz;
|
|
sscanf((char *)&tar->size, "%o", &tsz);
|
|
if(offset > tsz) return EOF;
|
|
|
|
if((size + offset) > tsz) size = tsz - offset;
|
|
|
|
offset = offset + (uint32_t)tar + 512;
|
|
memcpy(buffer, (void *)offset, size);
|
|
if(size == tsz - offset)
|
|
((char *)buffer)[size] = EOF;
|
|
|
|
return size;
|
|
}
|
|
|
|
###Using it
|
|
Now, all I need to do in order to make read-only files accessible to my
|
|
kernel is put them in a directory and run
|
|
|
|
:::bash
|
|
$ tar -cf tarfs.tar tarfs/*
|
|
|
|
and then make sure `tarfs.tar` is loaded as a multiboot module by qemu
|
|
|
|
:::bash
|
|
$ qemu-system-i386 -kernel kernel -initrd tarfs.tar
|
|
|
|
or by adding a line to the grub `menu.lst` file.
|
|
|
|
module /boot/tarfs.tar
|
|
|
|
###Final note
|
|
While writing this post, I got back to polishing this code and added
|
|
`readdir` for the tarfs. I also added buffering of inodes, so that if
|
|
the same file is asked for twice, the same inode is returned. This meant
|
|
I had to add a `flush` function to the driver structure.
|
|
|
|
I also rewrote a bit of both `vfs_finddir` and `vfs_readdir`. This in
|
|
turn forced me to go back on a design decision. The VFS can now only
|
|
mount new devices onto already existing paths (excluding `/`).
|
|
|
|
Just in case I'd write something in coming posts that might confuse
|
|
you...
|
|
|
|
### Git
|
|
I finally decided to push upstream, so here's the latest commit:
|
|
[843520405e](https://github.com/thomasloven/os5/tree/843520405eb950413c75b2ca2a0ee638f15e7f4e).
|
|
It still contains some stuff I haven't written about, though.
|
|
|