XFS status update for 2011

XFS status update for December 2011

December saw further stabilization of the Linux 3.2 release candidates. For XFS that meant two important fixes for sync() data integrity, to work around issues introduced in the VFS sync code in the past few kernel releases. These fixes have also been backported to the 3.0-stable release.

Development for the next merge windows continue in fast pace, although only a relatively small amount of patches was merged into the development tree for the Linux 3.3 window. The most interesting topic in December probably was further development of the SEEK_DATA / SEEK_HOLE support, including defining the exact semantics in presence of unwritten extents and proper test coverage.

On the user space side December was fairly quiet, with about a handful fixes commit to xfsprogs, two new test cases and a couple of fixes in xfstests, and no activity in xfsdump.

XFS status update for November 2011

November saw stabilization of the Linux 3.2 release candidates, including a few fixes for XFS. In addition a lot of bug fixes were backported to the 3.0 long term stable and 3.1-stable releases for users not on bleeding edge kernels.

At the same time development for Linux 3.3 went on at a fast pace, although no pages were merged into the development tree yet. The highlights are:

- further versions of the patches to log all file size updates instead of
  relying the the flaky VM writeback code for them
- an initial version of SEEK_HOLE/SEEK_DATA support
- removal of the old non-delaylog logging code, and cleanups resulting
  from the removal
- large updates for the quota code

Userspace development was even more busy:

Xfsprogs saw the rushed 3.1.7 release which contains Debian packaging fixes, a polish translation update and a xfs_repair fix. In the meantime a lot of xfs_repair fixes were posted but mostly not reviewed and commit yet.

Xfsdump grew support for using pthreads to write backup streams to multiple tapes in parallel, and SGI_XFSDUMP_SKIP_FILE which has been deprecated in favor of the nodump flag has finally been removed. Xfstests saw an enormous amount of updates. The fsstress tool saw major updates to exercise even more system calls, and found numerous bugs in all major Linux filesystems, additional ENOSPC tests, a new test for btrfs-specific functionality and the usual amount of bug fixes and small cleanups. Also a series to clean up the very large filesystem testing, including extending the support to ext4 was posted but not committed yet.

XFS status update for October 2011

October finally saw the delayed release of Linux 3.1, which is a fairly boring release as XFS is concerned. In addition to a few bug fixes and cleanups the biggest item is an XFS-internal re organization of the source files, dropping all sub directories under fs/xfs.

Due to the long Linux 3.1 release cycle development for 3.3 has already started full steam in October while adding a few more small optimization and fixes to the development tree for Linux 3.2, and merging that tree into mainline.

Notable items for Linux 3.2 are speedup for parallel O_DIRECT reads and writes on high IOPS devices, optimizations for fsync(2) on directories and sync(2) latency, as well as further small improvements for metadata performance on highly parallel workloads.

On the user space side xfsprogs saw a few more xfs_repair fixes, as well as some updates of mount point handling for the xfs_quota tools, which together with the updates from the last months was published in form of the xfsprogs 3.1.6 release. This was accompanied by an xfsdump 3.0.6 release, which does not include any new updates in October, but lots of work from the previous month. Xfstests saw two additional test cases and various fixes, and it's first versioned release ever.

XFS status update for September 2011

August saw further release candidates of Linux 3.1, which had been completely uneventful with just a single small regression fix being merged.

In the meantime developments for the Linux 3.2 kernel went on with the merge of a large series that completely refactors the XFS-internal xfs_bmapi interfaces for simpler interfaces and less stack usage, as well as various smaller cleanups and fixes.

September also was a very busy month for userspace development. In xfsprogs we saw various error handling fixes to libxcmd, libxfs, mkfs.xfs, xfs_quota and xfs_repair, xfsdump saw a few smaller changes finishing up the large work done in August. Xfstests saw 4 new test cases contributed from various developers, and the usual handful of bug fixes.

XFS status update for August 2011

August saw further release candidates of Linux 3.1, which had been quiet for XFS except for the bulk renaming of many XFS source files so that all source files are now located directly underneath the fs/xfs/ directory.

A lot of development for the Linux 3.2 kernel series was going on, including an overhaul of the data I/O completion handler, further buffer cache speedups and cleanups, a major refactoring around xfs_bmapi, quota locking changes, and optimization for direct I/O on high IOP solid state devices.

On the userspace side xfsdump saw a lot of cleanups in preparation of porting the multithreaded dump and restore code from IRIX. Xfsprogs saw a few fix to the xfs_quota tool and mkfs.xfs as well as a man page update. This month xfstests did not see any new test cases, but it got the usual amount of fixes and grew support for jfs and NFS v4.

XFS status update for July 2011

July finally saw the release of Linux 3.0, including a relatively small XFS update:

 37 files changed, 1168 insertions(+), 847 deletions(-)

The primary news in this release is a complete rework of the busy extent tracking, which speeds up allocation heavy multithreaded workloads. This feature also allowed adding support for discard support at transaction commit time using the discard mount option. While the implementation of discards in XFS is state of the art it should be considered mostly a technology preview until various efficiency issue in the block layer discard support are sorted out. Another important feature visible to users is that XFS now supports using external logs even when using volatile write caches, although the implementation is not fully optimized yet, the rest of the changes consists of the usual pile of bug fixes and a relatively small set of cleanups. After the release of Linux 3.0 the merge window for Linux 3.1 also fell mostly into July. The XFS merge for 3.1 included further speedups for the AIL code, and a huge amount of cleanups.

On the userspace side the biggest item was the merge of the libxfs resync with the Linux 2.6.39 kernel code. In addition to that xfsprogs saw small xfs_repair updates, xfstests saw various fixes to fsx, and various build system fixes were commit to all userspace repositories.

XFS status update for June 2011

In June we saw more release candidates for Linux 3.0, which contain a few XFS fixes, but no major updates. No updates were committed to the XFS development tree for Linux 3.1 either, although the mailing list has been rather busy with updates for that merge window.

On the user space side the xfsprogs and xfsdump repositories didn't see any updates, while xfstests has been rather busy with a lot of fixes to various test cases.

XFS status update for May 2011

May finally saw the release of Linux 2.6.39, which was a little more calm than usual for XFS, and only contains about half the amount of the changes we are used to see:

 58 files changed, 1660 insertions(+), 1912 deletions(-)

The most visible change is an overhaul of the XFS-internal interfaces to print kernel messages, which makes all messages from XFS look slightly different from before by always providing information about which device these messages relate to. In addition to that support for the RT subvolume, which had been broken for a while has been resurrect, the XFS buffer cache switched away from using the Linux pagecache to improve performance on metadata intensive workloads, and all but one of the XFS kernel threads have been switched to the new concurrent managed workqueue infrastructure that is present in more recent Linux 2.6 releases.

In the meantime development for the release now known as Linux 3.0 went ahead full steam up to the merge of the XFS tree into Linux 3.0-rc1. News in that release contain support for vastly improved busy extent tracking, support for online discard (aka TRIM) and the usual amount of bug fixes.

On the user space side the xfsprogs saw a fix for a corner case in xfs_repair, and xfstests saw a few bug fixes as well as a new test case to test btrfs-specific functionality.

XFS status update for April 2011

April saw further stabilization work on the Linux 2.6.39 kernel, including a number of XFS bug fixes. Most importantly a series of patches fixes various OOM problems due to bad interactions between the generic writeback code and XFS inode reclaim, but there also were other patches for various smaller issues. In the meantime the XFS development tree saw the addition of the optimized busy extent tracking, which allows large speedups for multi-threaded meta data heavy workloads, and lays the groundwork for discard support on transaction commit, and a few other smaller patches.

On the user space side the xfsprogs and xfsdump repositories saw a very quiet month with no applied patches, although a few were posted and discussed on the mailing list. The xfstests repository on the other hand saw a new test cases exercising the xfs_metadump functionally as well as a fixes to existing tests.

XFS status update for March 2011

March saw the release of Linux 2.6.38, which included a sizable XFS update. The most prominent new features of XFS in Linux 2.6.39 is support for the FITRIM ioctl that allows discarding unused space on the filesystem periodically, better handling of persistent preallocations especially on NFS servers, and further scalability improvements in the buffer cache and log code. In additions to that the release includes a wide range of fixes and cleanups to the code base. The diff stat for XFS in the Linux 2.6.39 release is:

 57 files changed, 2964 insertions(+), 2528 deletions(-)

Which means the XFS code base actually had a minor growth in code size this time around. In the second half of March the XFS development tree got merged into Linus' tree for Linux 2.6.39. Linux 2.6.39 is going to be a rather quiet release for XFS, mostly concentrating on settling the large changes that went into the last releases and smaller cleanups. The only user visible change will be that the delaylog option which improves metadata performance and scalability is now turned on by default, and a couple of fixes that make the realtime subvolume support usable again.

On the user space side both xfsprogs and xfsdump saw new releases in March. The xfsprogs 3.1.5 release contains various smaller updates to xfs_repair, xfs_metadump and xfs_quota, as well as support for the new generic hole punching in the falloc system call in the xfs_io tool. The xfsdump 3.0.5 release now supports up to 4 billion directory entries, has much better performance for large dumps, and some improvements to the inventory code and dumping of quota information, as well as long overdue updates to the build system. The xfstests repository has seen various build system improvements, better FIEMAP testing, falloc support for fsx and a few cleanups.

XFS status update for February 2011

February saw the stabilization of the Linux 2.6.38 tree, with just two small XFS fixes going into Linus' tree, and the XFS development tree has been similarly quiet with just a few cleanups, and the delaylog option propagated to the default operation mode. A few more patches for the 2.6.39 merge window have been posted and/or discussed on the mailing list, but February was a rather quiet month in general.

On the user space side xfsprogs saw a few bug fixes, and a speedup for phase2 of xfs_repair, xfsdump saw a bug fix and support for pruning the inventory by session id, and xfstests saw it's usual stream of bug fixes as well as two new test cases.

XFS status update for January 2011

On the 4th of January we saw the release of Linux 2.6.37, which contains a large XFS update:

   67 files changed, 1424 insertions(+), 1524 deletions(-)

User visible changes are the new XFS_IOC_ZERO_RANGE ioctl which allows to convert already allocated space into unwritten extents that return zeros on a read, and support for 32-bit wide project IDs. The other large item are various changes to improve metadata scalability even further, by changes to the the buffer cache, inode lookup and other parts of the filesystem driver.

After that the XFS development tree for 2.6.38 was merged into mainline, with an even larger set of changes. Notable items include support for the FITRIM ioctl to discard unused space on SSDs and thinly provisioned storage systems, a buffer LRU scheme to improve hit rates for metadata, an overhaul of the log subsystem locking, dramatically improving scalability in that area, and much smarter handling of preallocations, especially for files closed and reopened frequently, e.g. by the NFS server.

User space development has been very quiet, with just a few fixes committed to the xfstests repository, although various additional patches for xfsprogs and xfstests that haven't been committed yet were discussed on the mailing list.