XFS status update for 2012

XFS status update for July 2012

July saw the release of Linux 3.5, which included an average sized XFS update:

   81 files changed, 2633 insertions(+), 3004 deletions(-)

Noteworthy updates in Linux 3.5 include the removal of the xfsbufd background thread which writes metadata buffers, background processing of CIL flushes and thus lower average latency for metadata heavy workloads, as well a a reduction of lock hold times and frequency during reads and writes. The large XFS update for the Linux 3.6 merge window did not get merged yet by the end of July, but will bring better support for large directory block sizes, a faster inode allocator and a large amount of bug fixes in different areas of the code. Development on the list has been fairly busy on the patchsets to support group and project quotas at the same time, as well as improved SEEK_HOLE/SEEK_DATA support but otherwise mostly limited to bug fixes and smaller cleanups.

Userpace development has been rather slow with a single fix for xfs_db and another single fixe in xfstests.

XFS status update for June 2012

June has been a pretty slow month for XFS in Linus tree, as June saw the stabilization of the Linux 3.5 and thus only a handful of important XFS fixes. Development of the tree destined for the next merge window has been a bit more busy with the addition of discontinuous buffers that replace the dabuf layer which has performance issues and isn't easy to deal with in the upcoming metadata checksumming as well as a few smaller cleanups.

The list saw various other patches not applied yet, most notable a refactoring and speedup of the inode allocator, and a heavily discussed rewrite of the mount option parsing.

On the user space side a small number of fixes to the handling of fragmented directories were applied to xfs_repair, and xfstests saw a fix for handling different versions of xfsdump.

XFS status update for May 2012

May saw the release of Linux 3.4, including a decent sized XFS update. Remarkable XFS features in Linux 3.4 include moving over all metadata updates to use transactions, the addition of a work queue for the low-level allocator code to avoid stack overflows due to extreme stack use in the Linux VM/VFS call chain, better xattr operation tracing, fixes for a long-standing but hard to hit deadlock when using the XFS real time subvolume, and big improvements in disk quota scalability.

The final diffstat for XFS in Linux 3.4 is:

61 files changed, 1692 insertions(+), 2356 deletions(-)

In the meantime the merge window for Linux 3.5 opened, and another large update has been merged into Linus' tree. Interesting changes in Linux 3.5-rc1 include improved error handling on buffer write failures, a drastic reduction of locking overhead when doing high-IOPS direct I/O, removal of the old xfsbufd daemon in favor of writing most run-time metadata from the xfsaild daemon, deferral of CIL pushes to decouple user space metadata I/O from log writeback, and last but not least the addition of the SEEK_DATA/SEEK_HOLE lseek arguments that allow user space programs to deal with sparse files efficiently. Traffic on the mailing list has been a bit quiet in May, mostly focusing on a wide range of bug fixes, but little new features.

On the user space side xfs_repair saw a few bug fixes posted to the list that didn't make it to the repository yet, while xfstests saw it's usual amount of minor bug fixes.

XFS status update for April 2012

April was a relatively quite month in XFS development, partially because most XFS developers traveled to San Francisco for the Linux Filesystem and Storage Summit and the first XFS developers meeting co hosted at the Linux Foundation Collaboration Summit.

The XFS development tree only saw a series to reduce ilock hold times and a few patches merged. Traffic on the mailing list list has been a bit more busy, with a few patchset being reposted again and again after various updates. Features still scheduled for Linux 3.5 include: removal of the xfsbufd daemon, various other cleanups of the buffer layer, moving CIL commits into a background thread and thus reducing the impact on threads causing the commits, as well as a bigger than usual set of cleanups and fixes.

The user space side as been extremely slow with no patches applied and very few posted to the mailing list.

XFS status update for March 2012

March saw the release of Linux 3.3, which saw an XFS update slightly smaller than usual, and consisting largely of code removal and cleanups, with the only user visible change being the removal of the old logging code, and thus the "nodelaylog" mount option. This is reflected in a diffstat with more than twice as many deletions as additions:

 45 files changed, 954 insertions(+), 2182 deletions(-)

After that Linux 3.4 merge window opened, and a big XFS updates was merged into mainline, again removing a lot more code than adding it. The highlight are quota changes to make in-memory quota caching a lot faster and more scalable, as well as changes to log all metadata updates on a live XFS filesystem, thus finishing the consolidation of all metadata I/O code pathes to just after many years of hard work.

Development for the Linux 3.5 merge window also were in full steam, including a series to kill the xfsbufd daemon in favour of on-stack delayed buffer writeback lists.

On the user space side we finally saw the long awaited xfsprogs 3.1.8 and xfsdump 3.1.0 releases. The new xfsprogs release features a larger number of repair fixes and speedups, as well as smaller updates to the other tools. All users are recommended to upgrade to this version. Major updates in xfsdump 3.1.0 is support for dumping multiple I/O streams in parallel, a new dump format to support all 32 bits of the inode generation number, removal of support for the old SGI_XFSDUMP_SKIP_FILE extended attribute, as well as various smaller fixes.

Development of xfstests included at its normal pace, with a few new test cases for xfsdump, as well as various fixes for existing test cases and the addition of a new "dangerous" group to skip tests that potentially crash the system if an too old kernel is used.

XFS status update for February 2012

February has been been busy with development for Linux 3.3, although only little new code made it to the development tree yet, mostly a refactor of the log grant code. Patch traffic on the mailing list included a rewrite of disk quota caching, yielding dramatic performance improvements for systems with lots of quotas, a series to make all inode updates and thus all live metadata updates transactional. Last but not least a version to fix a long standing bug in support of the real time subvolume was posted, showing that even this long deprecated feature still has a user base.

On the userspace side development has been quite, with just two fixes to xfs_io committed to xfsprogs, while waiting for the long overdue 3.1.8 release. Xfsdump has been entirely quite this month, again waiting for a release, and xfstests saw only two commits either, although there are still lots of outstanding patches on the list that haven't been reviewed and/or applied.

XFS status update for January 2012

January saw the release of Linux 3.2, which as usual included a large number of XFS changes, most notably an large speedup for removing files that have external attribute blocks, speedups and livelock fixes for sync while doing heavy I/O, and large internal cleanups of the inode block map handling. The diffstat for XFS in Linux 3.2 is:

54 files changed, 2414 insertions(+), 2625 deletions(-)

which is slightly below the average of the last releases.

In the meantime development for 3.3 went ahead full speed, including the removal of the deprecated pre-delaylog logging code, various quota cleanups, a shrink of the inode, a great simplification of the file write path as well as the usual batch of fixes and cleanups.

On the userland side xfs_repair saw various major fixes and speedups, with few other fixes thrown in, while xfsdump got two commits fixing longer standing issues recently reported on the mailing list. For xfstests January was an extremely slow month, seeing only two new test cases and less than a handful of other updates.