XFS status update for 2012

XFS status update for October 2012

October saw the merge of the XFS development tree for Linux 3.7 into mainline as well as a few additional fixes for this new work.

Linux 3.7 will be a fairly boring release as far as XFS is concerned, the biggest user visible changes are an intelligent implementation of the lseek SEEK_HOLE/SEEK_DATA calls, and finally the switch to use the inode64 allocator by default.

The XFS development tree for Linux 3.8 started getting commits in October as well, most notably a large cleanup of the periodic sync code.

On the user space side we saw release candidate for the next releases of the XFS tools. For xfsprogs the 3.1.9-rc1 release brings various fixes in xfs_repair, especially dealing with directories as well as various improvements in xfs_db and mkfs.xfs, three new commands for xfs_io and a few user visible fixes to xfs_quota. The xfsdump 3.1.1-rc1 release adds a German string translation and support for 32-bit project IDs. xfstests saw its usual stream of fixes, but also a new ext4-specific test case and a switch to use the util-linux version of fstrim instead of a bundled one. All user space projects saw various improvements to the build system.

XFS status update for September 2012

September saw the release of Linux 3.6 which included a slightly smaller than usual XFS update:

51 files changed, 2607 insertions(+), 2455 deletions(-)

The bulk of the changes in Linux 3.6 is made up by native support for discontinuous buffers, with various minor bug fixes and cleanups making up the rest. In the meantime the mailing list was busy with multiple revisions of the sync/unmount code rewrite, and the patch series to make the inode64 mount option remountable and the default.

On the user space side xfsprogs saw new support for the preadv, pwritev and sync_file_range system calls in xfs_io as well as minor fixes to libxcmd and mkfs.xfs, xfsdump grew support for dumping and restoring 32-bit project ids and and saw an update to the German translation. Xfstests saw a large number of fixes, and a single new test case.

XFS status update for August 2012

After the release of Linux 3.5 in July, August saw the big XFS merge for the Linux 3.6 merge window. In the meantime the XFS tree has seen low activity, with refined support for SEEK_HOLE/SEEK_DATA that takes unwritten extents into account as the main feature. A couple of fixes also went in and mostly got back ported to mainline for Linux 3.5 as well.

The mailing list had a lot of traffic about interesting updates that didn't make it into the tree yet, with the most interesting being: remount support for the inode64 option, a patch series allowing to trim back large preallocations when the filesystem is out of space, and a large rewrite of the XFS-internal sync code.

On the user land side xfs_db saw a fix for a very long standing bug go into the tree and xfstests saw a few fixes. The mailing list list saw various other interesting updates including a German translation for the xfsdump messages, 32-bit project ID support in xfsdump, a lot of xfstests updates including a patch to use the upstream util-linux version of fstrim in xfstests, support for tmpfs in xfstests, various ext3 and btrfs updates, but none of them made it into the git tree. There also was a lot of discussion of older patch to reorganize the xfstests source tree.

XFS status update for July 2012

July saw the release of Linux 3.5, which included an average sized XFS update:

   81 files changed, 2633 insertions(+), 3004 deletions(-)

Noteworthy updates in Linux 3.5 include the removal of the xfsbufd background thread which writes metadata buffers, background processing of CIL flushes and thus lower average latency for metadata heavy workloads, as well a a reduction of lock hold times and frequency during reads and writes. The large XFS update for the Linux 3.6 merge window did not get merged yet by the end of July, but will bring better support for large directory block sizes, a faster inode allocator and a large amount of bug fixes in different areas of the code. Development on the list has been fairly busy on the patchsets to support group and project quotas at the same time, as well as improved SEEK_HOLE/SEEK_DATA support but otherwise mostly limited to bug fixes and smaller cleanups.

Userpace development has been rather slow with a single fix for xfs_db and another single fix in xfstests.

XFS status update for June 2012

June has been a pretty slow month for XFS in Linus tree, as June saw the stabilization of the Linux 3.5 and thus only a handful of important XFS fixes. Development of the tree destined for the next merge window has been a bit more busy with the addition of discontinuous buffers that replace the dabuf layer which has performance issues and isn't easy to deal with in the upcoming metadata checksumming as well as a few smaller cleanups.

The list saw various other patches not applied yet, most notable a refactoring and speedup of the inode allocator, and a heavily discussed rewrite of the mount option parsing.

On the user space side a small number of fixes to the handling of fragmented directories were applied to xfs_repair, and xfstests saw a fix for handling different versions of xfsdump.

XFS status update for May 2012

May saw the release of Linux 3.4, including a decent sized XFS update. Remarkable XFS features in Linux 3.4 include moving over all metadata updates to use transactions, the addition of a work queue for the low-level allocator code to avoid stack overflows due to extreme stack use in the Linux VM/VFS call chain, better xattr operation tracing, fixes for a long-standing but hard to hit deadlock when using the XFS real time subvolume, and big improvements in disk quota scalability.

The final diffstat for XFS in Linux 3.4 is:

61 files changed, 1692 insertions(+), 2356 deletions(-)

In the meantime the merge window for Linux 3.5 opened, and another large update has been merged into Linus' tree. Interesting changes in Linux 3.5-rc1 include improved error handling on buffer write failures, a drastic reduction of locking overhead when doing high-IOPS direct I/O, removal of the old xfsbufd daemon in favor of writing most run-time metadata from the xfsaild daemon, deferral of CIL pushes to decouple user space metadata I/O from log writeback, and last but not least the addition of the SEEK_DATA/SEEK_HOLE lseek arguments that allow user space programs to deal with sparse files efficiently. Traffic on the mailing list has been a bit quiet in May, mostly focusing on a wide range of bug fixes, but little new features.

On the user space side xfs_repair saw a few bug fixes posted to the list that didn't make it to the repository yet, while xfstests saw it's usual amount of minor bug fixes.

XFS status update for April 2012

April was a relatively quite month in XFS development, partially because most XFS developers traveled to San Francisco for the Linux Filesystem and Storage Summit and the first XFS developers meeting co hosted at the Linux Foundation Collaboration Summit.

The XFS development tree only saw a series to reduce ilock hold times and a few patches merged. Traffic on the mailing list list has been a bit more busy, with a few patchset being reposted again and again after various updates. Features still scheduled for Linux 3.5 include: removal of the xfsbufd daemon, various other cleanups of the buffer layer, moving CIL commits into a background thread and thus reducing the impact on threads causing the commits, as well as a bigger than usual set of cleanups and fixes.

The user space side as been extremely slow with no patches applied and very few posted to the mailing list.

XFS status update for March 2012

March saw the release of Linux 3.3, which saw an XFS update slightly smaller than usual, and consisting largely of code removal and cleanups, with the only user visible change being the removal of the old logging code, and thus the "nodelaylog" mount option. This is reflected in a diffstat with more than twice as many deletions as additions:

 45 files changed, 954 insertions(+), 2182 deletions(-)

After that Linux 3.4 merge window opened, and a big XFS updates was merged into mainline, again removing a lot more code than adding it. The highlight are quota changes to make in-memory quota caching a lot faster and more scalable, as well as changes to log all metadata updates on a live XFS filesystem, thus finishing the consolidation of all metadata I/O code pathes to just after many years of hard work.

Development for the Linux 3.5 merge window also were in full steam, including a series to kill the xfsbufd daemon in favour of on-stack delayed buffer writeback lists.

On the user space side we finally saw the long awaited xfsprogs 3.1.8 and xfsdump 3.1.0 releases. The new xfsprogs release features a larger number of repair fixes and speedups, as well as smaller updates to the other tools. All users are recommended to upgrade to this version. Major updates in xfsdump 3.1.0 is support for dumping multiple I/O streams in parallel, a new dump format to support all 32 bits of the inode generation number, removal of support for the old SGI_XFSDUMP_SKIP_FILE extended attribute, as well as various smaller fixes.

Development of xfstests included at its normal pace, with a few new test cases for xfsdump, as well as various fixes for existing test cases and the addition of a new "dangerous" group to skip tests that potentially crash the system if an too old kernel is used.

XFS status update for February 2012

February has been been busy with development for Linux 3.3, although only little new code made it to the development tree yet, mostly a refactor of the log grant code. Patch traffic on the mailing list included a rewrite of disk quota caching, yielding dramatic performance improvements for systems with lots of quotas, a series to make all inode updates and thus all live metadata updates transactional. Last but not least a version to fix a long standing bug in support of the real time subvolume was posted, showing that even this long deprecated feature still has a user base.

On the userspace side development has been quite, with just two fixes to xfs_io committed to xfsprogs, while waiting for the long overdue 3.1.8 release. Xfsdump has been entirely quite this month, again waiting for a release, and xfstests saw only two commits either, although there are still lots of outstanding patches on the list that haven't been reviewed and/or applied.

XFS status update for January 2012

January saw the release of Linux 3.2, which as usual included a large number of XFS changes, most notably an large speedup for removing files that have external attribute blocks, speedups and livelock fixes for sync while doing heavy I/O, and large internal cleanups of the inode block map handling. The diffstat for XFS in Linux 3.2 is:

54 files changed, 2414 insertions(+), 2625 deletions(-)

which is slightly below the average of the last releases.

In the meantime development for 3.3 went ahead full speed, including the removal of the deprecated pre-delaylog logging code, various quota cleanups, a shrink of the inode, a great simplification of the file write path as well as the usual batch of fixes and cleanups.

On the userland side xfs_repair saw various major fixes and speedups, with few other fixes thrown in, while xfsdump got two commits fixing longer standing issues recently reported on the mailing list. For xfstests January was an extremely slow month, seeing only two new test cases and less than a handful of other updates.