Pibroch: New page: == XFS status update for December 2011 == December saw further stabilization of the Linux 3.2 release candidates. For XFS that meant two important fixes for sync() data integrity, to work...

2012-03-19T06:12:09Z

New page: == XFS status update for December 2011 == December saw further stabilization of the Linux 3.2 release candidates. For XFS that meant two important fixes for sync() data integrity, to work...

New page

== XFS status update for December 2011 ==

December saw further stabilization of the Linux 3.2 release candidates.
For XFS that meant two important fixes for sync() data integrity, to
work around issues introduced in the VFS sync code in the past few
kernel releases. These fixes have also been backported to the 3.0-stable
release.

Development for the next merge windows continue in fast pace, although
only a relatively small amount of patches was merged into the development
tree for the Linux 3.3 window. The most interesting topic in December
probably was further development of the SEEK_DATA / SEEK_HOLE support,
including defining the exact semantics in presence of unwritten extents
and proper test coverage.

On the user space side December was fairly quiet, with about a handful
fixes commit to xfsprogs, two new test cases and a couple of fixes in
xfstests, and no activity in xfsdump.

== XFS status update for November 2011 ==

November saw stabilization of the Linux 3.2 release candidates, including
a few fixes for XFS. In addition a lot of bug fixes were backported to
the 3.0 long term stable and 3.1-stable releases for users not on
bleeding edge kernels.

At the same time development for Linux 3.3 went on at a fast pace, although
no pages were merged into the development tree yet. The highlights are:

- further versions of the patches to log all file size updates instead of
relying the the flaky VM writeback code for them
- an initial version of SEEK_HOLE/SEEK_DATA support
- removal of the old non-delaylog logging code, and cleanups resulting
from the removal
- large updates for the quota code

Userspace development was even more busy:

Xfsprogs saw the rushed 3.1.7 release which contains Debian packaging fixes,
a polish translation update and a xfs_repair fix. In the meantime a lot of
xfs_repair fixes were posted but mostly not reviewed and commit yet.

Xfsdump grew support for using pthreads to write backup streams to multiple
tapes in parallel, and SGI_XFSDUMP_SKIP_FILE which has been deprecated in
favor of the nodump flag has finally been removed.
Xfstests saw an enormous amount of updates. The fsstress tool saw major
updates to exercise even more system calls, and found numerous bugs in
all major Linux filesystems, additional ENOSPC tests, a new test for
btrfs-specific functionality and the usual amount of bug fixes and small
cleanups. Also a series to clean up the very large filesystem testing,
including extending the support to ext4 was posted but not committed yet.

== XFS status update for October 2011 ==

October finally saw the delayed release of Linux 3.1, which is a fairly
boring release as XFS is concerned. In addition to a few bug fixes
and cleanups the biggest item is an XFS-internal re organization of the
source files, dropping all sub directories under fs/xfs.

Due to the long Linux 3.1 release cycle development for 3.3 has already
started full steam in October while adding a few more small optimization
and fixes to the development tree for Linux 3.2, and merging that tree
into mainline.

Notable items for Linux 3.2 are speedup for parallel O_DIRECT reads and
writes on high IOPS devices, optimizations for fsync(2) on directories
and sync(2) latency, as well as further small improvements for metadata
performance on highly parallel workloads.

On the user space side xfsprogs saw a few more xfs_repair fixes, as well
as some updates of mount point handling for the xfs_quota tools, which
together with the updates from the last months was published in form
of the xfsprogs 3.1.6 release. This was accompanied by an xfsdump
3.0.6 release, which does not include any new updates in October, but
lots of work from the previous month. Xfstests saw two additional
test cases and various fixes, and it's first versioned release ever.

== XFS status update for September 2011 ==

August saw further release candidates of Linux 3.1, which had been
completely uneventful with just a single small regression fix being
merged.

In the meantime developments for the Linux 3.2 kernel went on with the merge
of a large series that completely refactors the XFS-internal xfs_bmapi
interfaces for simpler interfaces and less stack usage, as well as various
smaller cleanups and fixes.

September also was a very busy month for userspace development. In xfsprogs
we saw various error handling fixes to libxcmd, libxfs, mkfs.xfs, xfs_quota
and xfs_repair, xfsdump saw a few smaller changes finishing up the large
work done in August. Xfstests saw 4 new test cases contributed from
various developers, and the usual handful of bug fixes.

== XFS status update for August 2011 ==

August saw further release candidates of Linux 3.1, which had been quiet
for XFS except for the bulk renaming of many XFS source files so that all
source files are now located directly underneath the fs/xfs/ directory.

A lot of development for the Linux 3.2 kernel series was going on,
including an overhaul of the data I/O completion handler, further buffer
cache speedups and cleanups, a major refactoring around xfs_bmapi, quota
locking changes, and optimization for direct I/O on high IOP solid state
devices.

On the userspace side xfsdump saw a lot of cleanups in preparation of porting
the multithreaded dump and restore code from IRIX. Xfsprogs saw a few fix
to the xfs_quota tool and mkfs.xfs as well as a man page update. This month
xfstests did not see any new test cases, but it got the usual amount of fixes
and grew support for jfs and NFS v4.

== XFS status update for July 2011 ==

July finally saw the release of Linux 3.0, including a relatively small XFS
update:

37 files changed, 1168 insertions(+), 847 deletions(-)

The primary news in this release is a complete rework of the busy extent
tracking, which speeds up allocation heavy multithreaded workloads. This
feature also allowed adding support for discard support at transaction
commit time using the discard mount option. While the implementation of
discards in XFS is state of the art it should be considered mostly a
technology preview until various efficiency issue in the block layer
discard support are sorted out. Another important feature visible to
users is that XFS now supports using external logs even when using volatile
write caches, although the implementation is not fully optimized yet,
the rest of the changes consists of the usual pile of bug fixes and a
relatively small set of cleanups. After the release of Linux 3.0 the
merge window for Linux 3.1 also fell mostly into July. The XFS merge
for 3.1 included further speedups for the AIL code, and a huge amount
of cleanups.

On the userspace side the biggest item was the merge of the libxfs resync
with the Linux 2.6.39 kernel code. In addition to that xfsprogs saw small
xfs_repair updates, xfstests saw various fixes to fsx, and various build
system fixes were commit to all userspace repositories.

== XFS status update for June 2011 ==

In June we saw more release candidates for Linux 3.0, which contain a few
XFS fixes, but no major updates. No updates were committed to the XFS
development tree for Linux 3.1 either, although the mailing list has been
rather busy with updates for that merge window.

On the user space side the xfsprogs and xfsdump repositories didn't see
any updates, while xfstests has been rather busy with a lot of fixes
to various test cases.

== XFS status update for May 2011 ==

May finally saw the release of Linux 2.6.39, which was a little more calm
than usual for XFS, and only contains about half the amount of the changes
we are used to see:

58 files changed, 1660 insertions(+), 1912 deletions(-)

The most visible change is an overhaul of the XFS-internal interfaces
to print kernel messages, which makes all messages from XFS look slightly
different from before by always providing information about which device
these messages relate to. In addition to that support for the RT subvolume,
which had been broken for a while has been resurrect, the XFS buffer cache
switched away from using the Linux pagecache to improve performance on
metadata intensive workloads, and all but one of the XFS kernel threads have
been switched to the new concurrent managed workqueue infrastructure that
is present in more recent Linux 2.6 releases.

In the meantime development for the release now known as Linux 3.0 went
ahead full steam up to the merge of the XFS tree into Linux 3.0-rc1. News
in that release contain support for vastly improved busy extent tracking,
support for online discard (aka TRIM) and the usual amount of bug fixes.

On the user space side the xfsprogs saw a fix for a corner case in xfs_repair,
and xfstests saw a few bug fixes as well as a new test case to test
btrfs-specific functionality.

== XFS status update for April 2011 ==

April saw further stabilization work on the Linux 2.6.39 kernel, including
a number of XFS bug fixes. Most importantly a series of patches fixes various
OOM problems due to bad interactions between the generic writeback code
and XFS inode reclaim, but there also were other patches for various smaller
issues. In the meantime the XFS development tree saw the addition of the
optimized busy extent tracking, which allows large speedups for multi-threaded
meta data heavy workloads, and lays the groundwork for discard support on
transaction commit, and a few other smaller patches.

On the user space side the xfsprogs and xfsdump repositories saw a very quiet
month with no applied patches, although a few were posted and discussed on
the mailing list. The xfstests repository on the other hand saw a new test
cases exercising the xfs_metadump functionally as well as a fixes to existing
tests.

== XFS status update for March 2011 ==

March saw the release of Linux 2.6.38, which included a sizable XFS update.
The most prominent new features of XFS in Linux 2.6.39 is support for the
FITRIM ioctl that allows discarding unused space on the filesystem
periodically, better handling of persistent preallocations especially on
NFS servers, and further scalability improvements in the buffer cache
and log code. In additions to that the release includes a wide range
of fixes and cleanups to the code base. The diff stat for XFS in the
Linux 2.6.39 release is:

57 files changed, 2964 insertions(+), 2528 deletions(-)

Which means the XFS code base actually had a minor growth in code size
this time around. In the second half of March the XFS development
tree got merged into Linus' tree for Linux 2.6.39. Linux 2.6.39 is going
to be a rather quiet release for XFS, mostly concentrating on settling
the large changes that went into the last releases and smaller cleanups.
The only user visible change will be that the delaylog option which
improves metadata performance and scalability is now turned on by default,
and a couple of fixes that make the realtime subvolume support usable
again.

On the user space side both xfsprogs and xfsdump saw new releases in March.
The xfsprogs 3.1.5 release contains various smaller updates to xfs_repair,
xfs_metadump and xfs_quota, as well as support for the new generic hole
punching in the falloc system call in the xfs_io tool. The xfsdump 3.0.5
release now supports up to 4 billion directory entries, has much better
performance for large dumps, and some improvements to the inventory code
and dumping of quota information, as well as long overdue updates to the
build system. The xfstests repository has seen various build system
improvements, better FIEMAP testing, falloc support for fsx and a few
cleanups.

== XFS status update for February 2011 ==

February saw the stabilization of the Linux 2.6.38 tree, with just two
small XFS fixes going into Linus' tree, and the XFS development tree
has been similarly quiet with just a few cleanups, and the delaylog option
propagated to the default operation mode. A few more patches for the 2.6.39
merge window have been posted and/or discussed on the mailing list, but February
was a rather quiet month in general.

On the user space side xfsprogs saw a few bug fixes, and a speedup for
phase2 of xfs_repair, xfsdump saw a bug fix and support for pruning the
inventory by session id, and xfstests saw it's usual stream of bug fixes
as well as two new test cases.

== XFS status update for January 2011 ==

On the 4th of January we saw the release of Linux 2.6.37, which contains a
large XFS update:

67 files changed, 1424 insertions(+), 1524 deletions(-)

User visible changes are the new XFS_IOC_ZERO_RANGE ioctl which allows
to convert already allocated space into unwritten extents that return
zeros on a read, and support for 32-bit wide project IDs. The other large
item are various changes to improve metadata scalability even further,
by changes to the the buffer cache, inode lookup and other parts of the
filesystem driver.

After that the XFS development tree for 2.6.38 was merged into mainline,
with an even larger set of changes. Notable items include support for the
FITRIM ioctl to discard unused space on SSDs and thinly provisioned storage
systems, a buffer LRU scheme to improve hit rates for metadata, an
overhaul of the log subsystem locking, dramatically improving scalability
in that area, and much smarter handling of preallocations, especially
for files closed and reopened frequently, e.g. by the NFS server.

User space development has been very quiet, with just a few fixes committed
to the xfstests repository, although various additional patches for xfsprogs
and xfstests that haven't been committed yet were discussed on the mailing list.

XFS status update for 2011 - Revision history

Pibroch: New page: == XFS status update for December 2011 == December saw further stabilization of the Linux 3.2 release candidates. For XFS that meant two important fixes for sync() data integrity, to work...