v9fs: Plan 9 Resource Sharing for Linux

About

v9fs is a Unix implementation of the Plan 9 9p remote filesystem protocol.

This software was originally developed by Ron Minnich <rminnich@sandia.gov> and Maya Gokhale. Additional development by Greg Watson <gwatson@lanl.gov> and most recently Eric Van Hensbergen <ericvh@gmail.com>, Latchesar Ionkov <lucho@ionkov.net> and Russ Cox <rsc@swtch.com>.

The best detailed explanation of the Linux implementation and applications of the 9p client is available in the form of a USENIX paper:

Other applications are described in the following papers:

Usage

For remote file server:

mount -t 9p 10.10.1.2 /mnt/9

For Plan 9 From User Space applications (http://swtch.com/plan9):

mount -t 9p `namespace`/acme /mnt/9 -o trans=unix,uname=$USER

For server running on QEMU host with virtio transport:

mount -t 9p -o trans=virtio <mount_tag> /mnt/9

where mount_tag is the tag generated by the server to each of the exported mount points. Each 9P export is seen by the client as a virtio device with an associated “mount_tag” property. Available mount tags can be seen by reading /sys/bus/virtio/drivers/9pnet_virtio/virtio<n>/mount_tag files.

USBG Usage

To mount a 9p FS on a USB Host accessible via the gadget at runtime:

mount -t 9p -o trans=usbg,aname=/path/to/fs <device> /mnt/9

To mount a 9p FS on a USB Host accessible via the gadget as root filesystem:

root=<device> rootfstype=9p rootflags=trans=usbg,cache=loose,uname=root,access=0,dfltuid=0,dfltgid=0,aname=/path/to/rootfs

where <device> is the tag associated by the usb gadget transport. It is defined by the configfs instance name.

USBG Example

The USB host exports a filesystem, while the gadget on the USB device side makes it mountable.

Diod (9pfs server) and the forwarder are on the development host, where the root filesystem is actually stored. The gadget is initialized during boot (or later) on the embedded board. Then the forwarder will find it on the USB bus and start forwarding requests.

In this case the 9p requests come from the device and are handled by the host. The reason is that USB device ports are normally not available on PCs, so a connection in the other direction would not work.

When using the usbg transport, for now there is no native usb host service capable to handle the requests from the gadget driver. For this we have to use the extra python tool p9_fwd.py from tools/usb.

Just start the 9pfs capable network server like diod/nfs-ganesha e.g.:

$ diod -f -n -d 0 -S -l 0.0.0.0:9999 -e $PWD

Optionaly scan your bus if there are more then one usbg gadgets to find their path:

$ python $kernel_dir/tools/usb/p9_fwd.py list

Bus | Addr | Manufacturer     | Product          | ID        | Path
--- | ---- | ---------------- | ---------------- | --------- | ----
  2 |   67 | unknown          | unknown          | 1d6b:0109 | 2-1.1.2
  2 |   68 | unknown          | unknown          | 1d6b:0109 | 2-1.1.3

Then start the python transport:

$ python $kernel_dir/tools/usb/p9_fwd.py --path 2-1.1.2 connect -p 9999

After that the gadget driver can be used as described above.

One use-case is to use it as an alternative to NFS root booting during the development of embedded Linux devices.

Options

trans=name

select an alternative transport. Valid options are currently:

unix

specifying a named pipe mount point

tcp

specifying a normal TCP/IP connection

fd

used passed file descriptors for connection (see rfdno and wfdno)

virtio

connect to the next virtio channel available (from QEMU with trans_virtio module)

rdma

connect to a specified RDMA channel

usbg

connect to a specified usb gadget channel

uname=name

user name to attempt mount as on the remote server. The server may override or ignore this value. Certain user names may require authentication.

aname=name

aname specifies the file tree to access when the server is offering several exported file systems.

cache=mode

specifies a caching policy. By default, no caches are used. The mode can be specified as a bitmask or by using one of the preexisting common ‘shortcuts’. The bitmask is described below: (unspecified bits are reserved)

0b00000000

all caches disabled, mmap disabled

0b00000001

file caches enabled

0b00000010

meta-data caches enabled

0b00000100

writeback behavior (as opposed to writethrough)

0b00001000

loose caches (no explicit consistency with server)

0b10000000

fscache enabled for persistent caching

The current shortcuts and their associated bitmask are:

none

0b00000000 (no caching)

readahead

0b00000001 (only read-ahead file caching)

mmap

0b00000101 (read-ahead + writeback file cache)

loose

0b00001111 (non-coherent file and meta-data caches)

fscache

0b10001111 (persistent loose cache)

NOTE: only these shortcuts are tested modes of operation at the moment, so using other combinations of bit-patterns is not known to work. Work on better cache support is in progress.

IMPORTANT: loose caches (and by extension at the moment fscache) do not necessarily validate cached values on the server. In other words changes on the server are not guaranteed to be reflected on the client system. Only use this mode of operation if you have an exclusive mount and the server will modify the filesystem underneath you.

debug=n

specifies debug level. The debug level is a bitmask.

0x01

display verbose error messages

0x02

developer debug (DEBUG_CURRENT)

0x04

display 9p trace

0x08

display VFS trace

0x10

display Marshalling debug

0x20

display RPC debug

0x40

display transport debug

0x80

display allocation debug

0x100

display protocol message debug

0x200

display Fid debug

0x400

display packet debug

0x800

display fscache tracing debug

rfdno=n

the file descriptor for reading with trans=fd

wfdno=n

the file descriptor for writing with trans=fd

msize=n

the number of bytes to use for 9p packet payload

port=n

port to connect to on the remote server

noextend

force legacy mode (no 9p2000.u or 9p2000.L semantics)

version=name

Select 9P protocol version. Valid options are:

9p2000

Legacy mode (same as noextend)

9p2000.u

Use 9P2000.u protocol

9p2000.L

Use 9P2000.L protocol

dfltuid

attempt to mount as a particular uid

dfltgid

attempt to mount with a particular gid

afid

security channel - used by Plan 9 authentication protocols

nodevmap

do not map special files - represent them as normal files. This can be used to share devices/named pipes/sockets between hosts. This functionality will be expanded in later versions.

directio

bypass page cache on all read/write operations

ignoreqv

ignore qid.version==0 as a marker to ignore cache

noxattr

do not offer xattr functions on this mount.

access
there are four access modes.
user

if a user tries to access a file on v9fs filesystem for the first time, v9fs sends an attach command (Tattach) for that user. This is the default mode.

<uid>

allows only user with uid=<uid> to access the files on the mounted filesystem

any

v9fs does single attach and performs all operations as one user

clien

ACL based access check on the 9p client side for access validation

cachetag

cache tag to use the specified persistent cache. cache tags for existing cache sessions can be listed at /sys/fs/9p/caches. (applies only to cache=fscache)

Behavior

This section aims at describing 9p ‘quirks’ that can be different from a local filesystem behaviors.

  • Setting O_NONBLOCK on a file will make client reads return as early as the server returns some data instead of trying to fill the read buffer with the requested amount of bytes or end of file is reached.

Resources

Protocol specifications are maintained on github: http://ericvh.github.com/9p-rfc/

9p client and server implementations are listed on http://9p.cat-v.org/implementations

A 9p2000.L server is being developed by LLNL and can be found at http://code.google.com/p/diod/

There are user and developer mailing lists available through the v9fs project on sourceforge (http://sourceforge.net/projects/v9fs).

News and other information is maintained on a Wiki. (http://sf.net/apps/mediawiki/v9fs/index.php).

Bug reports are best issued via the mailing list.

For more information on the Plan 9 Operating System check out http://plan9.bell-labs.com/plan9

For information on Plan 9 from User Space (Plan 9 applications and libraries ported to Linux/BSD/OSX/etc) check out https://9fans.github.io/plan9port/