After having finished my 2A academic term at UWaterloo, I’ve taken a few weeks off before beginning my next co-op at Shopify as a backend intern. Over this break, I finally found the time to explore a technology that has interested me over the past few months. FUSE, which stands for Filesystem in Userspace, is software that allows for non-privileged users to create userspace filesystems. Some examples of FUSE projects are EncFS, an encrypted virtual filesystem, and a personal favourite, SSHFS that gives access to a remote filesystem over ssh.
I used Bazil’s Go FUSE package. It seems development of it has slowed down over the past year, but the package worked great for me. From what I found, the package and its interface have evolved steadily over time, but the documentation hasn’t kept up. Tommi Virtanen’s talk from 2013 was a helpful overview of FUSE, but the API shown has since changed.
A Basic Filesystem
With Bazil’s FUSE package I built a very simple filesystem. It supports CRUD operations on files and directories. I hardcoded a few files and directories to get started. My sample project was based off of the clockfs example, you can check out my final version here.
Above are the core structs used. In the
Dir struct, the pointer to the slice was required to fix a map-related crash within Bazil and the pointers within the slice fix the problem that modifications made to some
Files obviously don’t get persisted in pass-by-value methods. Almost all of the operations will be implemented by either
File through Bazil’s exported interfaces.
The most important interface in Bazil’s package is the
Node interface. This interface is the provides the core information for parts of the filesystem, in our case,
Dir. Implementing it is straightforward, requiring only one method:
File, I implemented it as follows:
One important note here, you see it uses a
context.Context. This is not the context package included in Go 1.7, but it actually expects
golang.org/x/net/context. Likely because Bazil’s package hasn’t been updated recently.
Reading a File
To read a file, the
File struct implements
Editing a File
To be able to edit and write to a file,
The important documentation note is that
resp.Size must be set for the amount of data actually written to your internal implementation.
When I tried this in vim, I kept getting “Fsync error E667”. Fsync is a library function in vim that flushes a file to disk, making sure that it is safely written. Luckily, fixing this was as easy as implementing another Bazil interface,
Creating a File
This operation will run on a
NodeCreater provides a
fuse.CreateRequest struct that contains the file metadata including the new name.
Reading a Directory
Given a string
NodeStringLookuper’s Lookup method provides the Node that matches that name, otherwise, return
fuse.ENOENT. It could be either a
File or a sub-
Dir, so I loop through both.
ReadDirAll is the broader method. Here you create a slice of
fuse.Dirent for all
Dirs in the provided
Creating a Directory
Making a new directory is very similar to creating a file, as seen above.
NodeMkdirer (I think you’re getting the pattern now) passes the
fuse.MkdirRequest struct with the new directory’s metadata.
Removing Files and Directories
The last operation I’ll cover is the deletion of
Dirs. Both are handled by the same method, from the
NodeRemover interface. The passed
fuse.RemoveRequest struct has a bool, also called
Dir, that informs you if the remove request is for a directory or not. If the request cannot be handled, a
fuse.ENOENT will be returned.
Here we just filter out the name provided from the existing files and directories within the provided
Starting the filesystem
The last interface to implement is the
FS interface on our
FS struct. Again, just one key method. Here, we return the root
To startup your new filesystem, you must first use FUSE to mount your mountpoint, then create a
fs.Server that serves an instance of your filesystem. In this case, a
You can find the full sample project on my Github. I implemented some more interfaces and added logging so I can see how my interactions with the filesystem translates into FUSE calls. You can find the GoDoc for Bazil’s FUSE here, but most of the interfaces mentioned above are documented in the fs GoDoc.
FUSE is a really cool technology that I think is underutilized. One idea that I’ve been kicking around was an extension to my Hack The North project,
nfinite.space. Ideally a FUSE integration with nfinite.space would provide a seamless way to integrate the service into everyone’s filesystem. The extension could transparently manage sharded fileparts and communicate in the background with the central service. That’d be a good start.
So that’s an overview of Bazil’s FUSE package. It provides an easy, extensible interface for FUSE in Go. Of course, you can find FUSE bindings in many other languages. Try experimenting with FUSE in your next project!