XFS Status Updates: Difference between revisions

From xfs.org
← Older editNewer edit →
VisualWikitext
February update
Pibroch (talk | contribs)
28 edits
No edit summary
Line 1: Line 1:
== XFS status update for February 2012 ==
== XFS Status Updates ==


February has been been busy with development for Linux 3.3, although only
* [[XFS status update for 2012]] ([[XFS_status_update_for_2012#XFS status update for February 2012|Latest]])
little new code made it to the development tree yet, mostly a refactor
* [[XFS status update for 2011]] ([[XFS_status_update_for_2011#XFS status update for December 2011|Latest]])
of the log grant code.  Patch traffic on the mailing list included a rewrite
* [[XFS status update for 2010]] ([[XFS_status_update_for_2010#XFS status update for December 2010|Latest]])
of disk quota caching, yielding dramatic performance improvements for systems
* [[XFS status update for 2009]] ([[XFS_status_update_for_2009#XFS status update for December 2009|Latest]])
with lots of quotas, a series to make all inode updates and thus all live
* [[XFS status update for 2008]] ([[XFS_status_update_for_2008#XFS status update for December 2008|Latest]])
metadata updates transactional.  Last but not least a version versions to
* [[OLD_News|XFS status update before 2008]]
fix a long standing bug in support of the real time subvolume was posted,
showing that even this long deprecated feature still has a user base.
 
On the userspace side development has been quite, with just two fixes to
xfs_io committed to xfsprogs, while waiting for the long overdue 3.1.8
release.  Xfsdump has been entirely quite this month, again waiting for
a release, and xfstests saw only two commits either, although there are
still lots of outstanding patches on the list that haven't been reviewed
and/or applied.
 
== XFS status update for January 2012 ==
 
January saw the release of Linux 3.2, which as usual included a large
number of XFS changes, most notably an large speedup for removing files
that have external attribute blocks, speedups and livelock fixes for
sync while doing heavy I/O, and large internal cleanups of the inode
block map handling.  The diffstat for XFS in Linux 3.2 is:
 
54 files changed, 2414 insertions(+), 2625 deletions(-)
 
which is slightly below the average of the last releases.
 
In the meantime development for 3.3 went ahead full speed, including
the removal of the deprecated pre-delaylog logging code, various
quota cleanups, a shrink of the inode, a great simplification of the file
write path as well as the usual batch of fixes and cleanups.
 
On the userland side xfs_repair saw various major fixes and speedups,
with few other fixes thrown in, while xfsdump got two commits fixing
longer standing issues recently reported on the mailing list.  For
xfstests January was an extremely slow month, seeing only two new test cases
and less than a handful of other updates.
 
== XFS status update for December 2011 ==
 
December saw further stabilization of the Linux 3.2 release candidates.
For XFS that meant two important fixes for sync() data integrity, to
work around issues introduced in the VFS sync code in the past few
kernel releases.  These fixes have also been backported to the 3.0-stable
release.
 
Development for the next merge windows continue in fast pace, although
only a relatively small amount of patches was merged into the development
tree for the Linux 3.3 window.  The most interesting topic in December
probably was further development of the SEEK_DATA /  SEEK_HOLE support,
including defining the exact semantics in presence of unwritten extents
and proper test coverage.
 
On the user space side December was fairly quiet, with about a handful
fixes commit to xfsprogs, two new test cases and a couple of fixes in
xfstests, and no activity in xfsdump.
 
== XFS status update for November 2011 ==
 
November saw stabilization of the Linux 3.2 release candidates, including
a few fixes for XFS.  In addition a lot of bug fixes were backported to
the 3.0 long term stable and 3.1-stable releases for users not on
bleeding edge kernels.
 
At the same time development for Linux 3.3 went on at a fast pace, although
no pages were merged into the development tree yet.  The highlights are:
 
- further versions of the patches to log all file size updates instead of
  relying the the flaky VM writeback code for them
- an initial version of SEEK_HOLE/SEEK_DATA support
- removal of the old non-delaylog logging code, and cleanups resulting
  from the removal
- large updates for the quota code
 
Userspace development was even more busy:
 
Xfsprogs saw the rushed 3.1.7 release which contains Debian packaging fixes,
a polish translation update and a xfs_repair fix.  In the meantime a lot of
xfs_repair fixes were posted but mostly not reviewed and commit yet.
 
Xfsdump grew support for using pthreads to write backup streams to multiple
tapes in parallel, and SGI_XFSDUMP_SKIP_FILE which has been deprecated in
favor of the nodump flag has finally been removed.
Xfstests saw an enormous amount of updates.  The fsstress tool saw major
updates to exercise even more system calls, and found numerous bugs in
all major Linux filesystems, additional ENOSPC tests, a new test for
btrfs-specific functionality and the usual amount of bug fixes and small
cleanups.  Also a series to clean up the very large filesystem testing,
including extending the support to ext4 was posted but not committed yet.
 
== XFS status update for October 2011 ==
 
October finally saw the delayed release of Linux 3.1, which is a fairly
boring release as XFS is concerned.  In addition to a few bug fixes
and cleanups the biggest item is an XFS-internal re organization of the
source files, dropping all sub directories under fs/xfs.
 
Due to the long Linux 3.1 release cycle development for 3.3 has already
started full steam in October while adding a few more small optimization
and fixes to the development tree for Linux 3.2, and merging that tree
into mainline.
 
Notable items for Linux 3.2 are speedup for parallel O_DIRECT reads and
writes on high IOPS devices, optimizations for fsync(2) on directories
and sync(2) latency, as well as further small improvements for metadata
performance on highly parallel workloads.
 
On the user space side xfsprogs saw a few more xfs_repair fixes, as well
as some updates of mount point handling for the xfs_quota tools, which
together with the updates from the last months was published in form
of the xfsprogs 3.1.6 release.  This was accompanied by an xfsdump
3.0.6 release, which does not include any new updates in October, but
lots of work from the previous month.  Xfstests saw two additional
test cases and various fixes, and it's first versioned release ever.
 
== XFS status update for September 2011 ==
 
August saw further release candidates of Linux 3.1, which had been
completely uneventful with just a single small regression fix being
merged.
 
In the meantime developments for the Linux 3.2 kernel went on with the merge
of a large series that completely refactors the XFS-internal xfs_bmapi
interfaces for simpler interfaces and less stack usage, as well as various
smaller cleanups and fixes.
 
September also was a very busy month for userspace development. In xfsprogs
we saw various error handling fixes to libxcmd, libxfs, mkfs.xfs, xfs_quota
and xfs_repair, xfsdump saw a few smaller changes finishing up the large
work done in August.  Xfstests saw 4 new test cases contributed from
various developers, and the usual handful of bug fixes.
 
== XFS status update for August 2011 ==
 
August saw further release candidates of Linux 3.1, which had been quiet
for XFS except for the bulk renaming of many XFS source files so that all
source files are now located directly underneath the fs/xfs/ directory.
 
A lot of development for the Linux 3.2 kernel series was going on,
including an overhaul of the data I/O completion handler, further buffer
cache speedups and cleanups, a major refactoring around xfs_bmapi, quota
locking changes, and optimization for direct I/O on high IOP solid state
devices.
 
On the userspace side xfsdump saw a lot of cleanups in preparation of porting
the multithreaded dump and restore code from IRIX.  Xfsprogs saw a few fix
to the xfs_quota tool and mkfs.xfs as well as a man page update.  This month
xfstests did not see any new test cases, but it got the usual amount of fixes
and  grew support for jfs and NFS v4.
 
== XFS status update for July 2011 ==
 
July finally saw the release of Linux 3.0, including a relatively small XFS
update:
 
  37 files changed, 1168 insertions(+), 847 deletions(-)
 
The primary news in this release is a complete rework of the busy extent
tracking, which speeds up allocation heavy multithreaded workloads.  This
feature also allowed adding support for discard support at transaction
commit time using the discard mount option.  While the implementation of
discards in XFS is state of the art it should be considered mostly a
technology preview until various efficiency issue in the block layer
discard support are sorted out.  Another important feature visible to
users is that XFS now supports using external logs even when using volatile
write caches, although the implementation is not fully optimized yet,
the rest of the changes consists of the usual pile of bug fixes and a
relatively small set of cleanups.  After the release of Linux 3.0 the
merge window for Linux 3.1 also fell mostly into July.  The XFS merge
for 3.1 included further speedups for the AIL code, and a huge amount
of cleanups.
 
On the userspace side the biggest item was the merge of the libxfs resync
with the Linux 2.6.39 kernel code.  In addition to that xfsprogs saw small
xfs_repair updates, xfstests saw various fixes to fsx, and various build
system fixes were commit to all userspace repositories.
 
 
 
== XFS status update for June 2011 ==
 
In June we saw more release candidates for Linux 3.0, which contain a few
XFS fixes, but no major updates.  No updates were committed to the XFS
development tree for Linux 3.1 either, although the mailing list has been
rather busy with updates for that merge window.
 
On the user space side the xfsprogs and xfsdump repositories didn't see
any updates, while xfstests has been rather busy with a lot of fixes
to various test cases.
 
== XFS status update for May 2011 ==
 
May finally saw the release of Linux 2.6.39, which was a little more calm
than usual for XFS, and only contains about half the amount of the changes
we are used to see:
 
  58 files changed, 1660 insertions(+), 1912 deletions(-)
 
The most visible change is an overhaul of the XFS-internal interfaces
to print kernel messages, which makes all messages from XFS look slightly
different from before by always providing information about which device
these messages relate to.  In addition to that support for the RT subvolume,
which had been broken for a while has been resurrect, the XFS buffer cache
switched away from using the Linux pagecache to improve performance on
metadata intensive workloads, and all but one of the XFS kernel threads have
been switched to the new concurrent managed workqueue infrastructure that
is present in more recent Linux 2.6 releases.
 
In the meantime development for the release now known as Linux 3.0 went
ahead full steam up to the merge of the XFS tree into Linux 3.0-rc1. News
in that release contain support for vastly improved busy extent tracking,
support for online discard (aka TRIM) and the usual amount of bug fixes.
 
On the user space side the xfsprogs saw a fix for a corner case in xfs_repair,
and xfstests saw a few bug fixes as well as a new test case to test
btrfs-specific functionality.
 
== XFS status update for April 2011 ==
 
April saw further stabilization work on the Linux 2.6.39 kernel, including
a number of XFS bug fixes.  Most importantly a series of patches fixes various
OOM problems due to bad interactions between the generic writeback code
and XFS inode reclaim, but there also were other patches for various smaller
issues.  In the meantime the XFS development tree saw the addition of the
optimized busy extent tracking, which allows large speedups for multi-threaded
meta data heavy workloads, and lays the groundwork for discard support on
transaction commit, and a few other smaller patches.
 
On the user space side the xfsprogs and xfsdump repositories saw a very quiet
month with no applied patches, although a few were posted and discussed on
the mailing list. The xfstests repository on the other hand saw a new test
cases exercising the xfs_metadump functionally as well as a fixes to existing
tests.
 
== XFS status update for March 2011 ==
 
March saw the release of Linux 2.6.38, which included a sizable XFS update.
The most prominent new features of XFS in Linux 2.6.39 is support for the
FITRIM ioctl that allows discarding unused space on the filesystem
periodically, better handling of persistent preallocations especially on
NFS servers, and further scalability improvements in the buffer cache
and log code.  In additions to that the release includes a wide range
of fixes and cleanups to the code base.  The diff stat for XFS in the
Linux 2.6.39 release is:
 
  57 files changed, 2964 insertions(+), 2528 deletions(-)
 
Which means the XFS code base actually had a minor growth in code size
this time around.  In the second half of March the XFS development
tree got merged into Linus' tree for Linux 2.6.39.  Linux 2.6.39 is going
to be a rather quiet release for XFS, mostly concentrating on settling
the large changes that went into the last releases and smaller cleanups.
The only user visible change will be that the delaylog option which
improves metadata performance and scalability is now turned on by default,
and a couple of fixes that make the realtime subvolume support usable
again.
 
On the user space side both xfsprogs and xfsdump saw new releases in March.
The xfsprogs 3.1.5 release contains various smaller updates to xfs_repair,
xfs_metadump and xfs_quota, as well as support for the new generic hole
punching in the falloc system call in the xfs_io tool.  The xfsdump 3.0.5
release now supports up to 4 billion directory entries, has much better
performance for large dumps, and some improvements to the inventory code
and dumping of quota information, as well as long overdue updates to the
build system.  The xfstests repository has seen various build system
improvements, better FIEMAP testing, falloc support for fsx and a few
cleanups.
 
== XFS status update for February 2011 ==
 
February saw the stabilization of the Linux 2.6.38 tree, with just two
small XFS fixes going into Linus' tree, and the XFS development tree
has been similarly quiet with just a few cleanups, and the delaylog option
propagated to the default operation mode.  A few more patches for the 2.6.39
merge window have been posted and/or discussed on the mailing list, but February
was a rather quiet month in general.
 
On the user space side xfsprogs saw a few bug fixes, and a speedup for
phase2 of xfs_repair, xfsdump saw a bug fix and support for pruning the
inventory by session id, and xfstests saw it's usual stream of bug fixes
as well as two new test cases.
 
== XFS status update for January 2011 ==
 
On the 4th of January we saw the release of Linux 2.6.37, which contains a
large XFS update:
 
    67 files changed, 1424 insertions(+), 1524 deletions(-)
 
User visible changes are the new XFS_IOC_ZERO_RANGE ioctl which allows
to convert already allocated space into unwritten extents that return
zeros on a read, and support for 32-bit wide project IDs.  The other large
item are various changes to improve metadata scalability even further,
by changes to the the buffer cache, inode lookup and other parts of the
filesystem driver.
 
After that the XFS development tree for 2.6.38 was merged into mainline,
with an even larger set of changes.  Notable items include support for the
FITRIM ioctl to discard unused space on SSDs and thinly provisioned storage
systems, a buffer LRU scheme to improve hit rates for metadata, an
overhaul of the log subsystem locking, dramatically improving scalability
in that area, and much smarter handling of preallocations, especially
for files closed and reopened frequently, e.g. by the NFS server.
 
User space development has been very quiet, with just a few fixes committed
to the xfstests repository, although various additional patches for xfsprogs
and xfstests that haven't been committed yet were discussed on the mailing list.
 
== XFS status update for December 2010 ==
 
The release process of the Linux 2.6.37 kernel with it's large XFS updates
was in it's final days in December, which explains why we only saw a single
one-liner regression fix for XFS in Linus' tree.  The XFS development tree
finally saw some updates when the writeback updates and some small cleanups
to the allocator and log recovery code were merged, but the large metadata
scalability updates that have been posted to the list multiple times are
still missing.  In addition to this on-going work the list also saw patches
that fix smaller issues, which are also still waiting to be merged.
 
On the userspace side xfsprogs and xfsdump development has been quit, with
no commits to either repository in December, although a large series of
updates to the metadump command has been reposted near the end of the month.
The xfstests repository saw a new regression test for a btrfs problem,
and various updates to existing tests.
 
== XFS status update for November 2010 ==
 
From looking at the kernel git commits November looked like a pretty
slow month with just two hand full fixes going into the release candidates
for Linux 2.6.37, and none at all going into the development tree.
But in this case git statistics didn't tell the whole story - there
was a lot of activity on patches for the next merge window on the list.
The focus in November was still at metadata scalability, with various
patchsets that improves parallel creates and unlinks again, and also
improves 8-way dbench throughput by 30%.  In addition to that there
were patches to improve preallocation for NFS servers, to simplify
the writeback code, and to remove the XFS-internal percpu counters
for free space for the generic kernel percpu counters, which just needed
a small improvement.
 
On the user space side we saw the release of xfsprogs 3.1.4, which
contains various accumulated bug fixes and Debian packaging updates.
The xfsdump tree saw a large update to speed up restore by using
mmap for an internal database and remove the limitation of ~ 214
million directory entries per dump file.  The xfstests test suite
saw three new testcases and various fixes, including support for the
hfsplus filesystem.
 
== XFS status update for October 2010 ==
 
Near the end of the month we finally saw the release of Linux 2.6.36.
Just a single fix made it into mainline in this month, showing that the
stabilization period before has worked very well.
 
Linux 2.6.36 has been another impressive release for XFS, seeing
various performance improvements in the new delayed logging code,
for direct I/O and the sync system call, a few bug fixes, and lots
of cleanups, resulting in a net removal of over 2000 lines of code:
 
        89 files changed, 1998 insertions(+), 4279 deletions(-)
 
The merge window for Linux 2.6.37 opened just a few days after the
release of Linux 2.6.36 and already contains another large XFS update
at the end of October.  Highlights of the XFS tree merged into 2.6.37-rc1
are another large set of metadata scalability patches, support for 32-bit
wide project IDs, and support for the new XFS_IOC_ZERO_RANGE ioctl,
which allows to punch a whole and convert it to an unwritten extent
in a single atomic operation.
 
The metadata scalability changes improve 8-way fs_mark of 50 million files
by over 15% and removal of those files by over 100%, with further
improvements expected by the next round of XFS metadata scalability
and VFS scalability improvements targeted at Linux 2.6.38.
 
On the user space side October was a rather quit month for xfsprogs, which
only saw the addition of 32-bit project ID handling, and a fix for
parsing the mount table in fsr when used together with disk encryption
tools.  A few patches for xfsdump were posted on the list, but none
was applied, leaving the majority of the user space activity to
xfstests, which saw very active development.  Various patches went
into xfstests to improve portability to filesystems with a limited
feature set, and to move more filters to generic code.  In addition
various cleanups to test cases in test programs were applied.
 
== XFS status update for September 2010 ==
 
Mainline activity has been rather low in September while with only
two more fixes going into the 2.6.36 release candidates after the
large merge activity in August.  Development for the next merge
window has been more active.  The largest item was the inclusion
of the metadata scalability patch series, which provides very large
speedups for parallel metadata operations.  In addition a new
ioctl to punch holes and convert the whole to an unwritten extent
was added and a small number of cleanups also made it into the tree.
 
Patches to add support for 32bit wide project ID identifiers and
using group and project quotas concurrently were posted to the list
and discussed but not yet included.
 
Userspace development has been rather quiet again, with a single fix
committed to xfsprogs and xfsdump each.  The xfstests test suite grew
a new test case and received a few additional fixes.  Last but not least
the [http://xfs.org/docs/xfsdocs-xml-dev/XFS_User_Guide//tmp/en-US/html/index.html XFS Users Guide]
was updated with various factual corrections and spelling fixes.
 
== XFS status update for August 2010 ==
 
At the first of August we finally saw the release of Linux 2.6.35,
which includes a large XFS update.  The most prominent feature in
Linux 2.6.35 is the new delayed logging code which provides massive
speedups for metadata-intensive workloads, but there has been
a large amount of other fixes and cleanups, leading to the following
diffstat:
 
        67 files changed, 4426 insertions(+), 3835 deletions(-)
 
Given the early release of Linux 2.6.35 the merge window for the
next release fully fell into the month of August.  The XFS updates
for Linux 2.6.36 include various additional performance improvements
in the delayed logging code, for direct I/O writes and for avoiding
synchronous transactions, as well as various fixed and large amount
of cleanups, including the removal of the remaining dead DMAPI
code.
 
On the userspace side we saw the 3.1.3 release of xfsprogs, which includes
various smaller fixes, support for the new XFS_IOC_ZERO_RANGE ioctl and
Debian packaging updates.  The xfstests package saw one new test case
and a couple of smaller patches, and xfsdump has not seen any updates at
all.
 
The XMLified versions of the XFS users guide, training labs and filesystem
structure documentation are now available as on the fly generated html on
the xfs.org website and can be found at [[XFS_Papers_and_Documentation|Papers & Documentation]].
 
== XFS status update for July 2010 ==
 
July saw three more release candidates for the Linux 2.6.35 kernel, which
included a relatively large number of XFS updates.  There were two security
fixes, a small one to prevent swapext to operate on write-only file
descriptors, and a much larger one to properly validate inode numbers
coming from NFS clients or userspace applications using the bulkstat or
the open-by-handle interfaces.  In addition to that another relatively
large patch fixes the way inodes get reclaimed in the background, and
avoids inode caches growing out of bounds.
 
In the meantime the code for the Linux 2.6.36 got the last touches before
the expected opening of the merge window, by merging a few more last
minute fixes and cleanups.  The most notable one is a patch series
that fixes in-memory corruption when concurrently accessing unwritten
extents using the in-kernel AIO code.
 
The userspace side was still quite slow, but some a bit more activity
than June.  In xfsprogs the xfs_db code grew two bug fixes, as did
the xfs_io script.  The xfstests package saw one new test cases and
various fixes to existing code.  Last but not least a few patches
affecting the build system for all userspace tools were committed.
 
== XFS status update for June 2010 ==
 
The month of June saw a few important bug fixes for the Linux 2.6.35
release candidates.  That includes ensuring that files used for the
swapext ioctl are writable to the user, and doing proper validation
of inodes coming from untrusted sources, such as NFS exporting and
the open by handle system calls.  The main work however has been
focused on development for the Linux 2.6.36 merge window, including
merging various patches that have been out on the mainline list
for a long time.  Highlights include further performance improvements
for sync heavy metadata workloads, stack space reduction in the
writeback path and improvements of the XFS tracing infrastructure.
Also after some discussion the remaining hooks for DMAPI are going
to be dropped in mainline.  As a replacement a tree containing
full DMAPI support with a slightly cleaner XFS interaction will be
hosted by SGI.
 
On the userspace side June was a rather slow month, with no updates
to xfsprogs and xfsdump at all, and just one new test case and a cleanup
applied to xfstests.
 
== XFS status update for May 2010 ==
 
In May 2010 we saw the long awaited release of Linux 2.6.34, which includes
a large XFS update.  The most important features appearing in 2.6.34 was the
new inode and quota flushing code, which leads to much better I/O patterns
for metadata-intensive workloads.  Additionally support for synchronous NFS
exports has been improved to give much better performance, and performance
for the fsync, fdatasync and sync system calls has been improved slightly.
A bug when resizing extremely busy filesystems has been fixed, which required
extensive modification to the data structure used for looking up the
per-allocation group data.  Last but not least there was a steady flow of
minor bug fixes and cleanups, leading to the following diffstat from
2.6.33 to 2.6.34:
 
  86 files changed, 3209 insertions(+), 3178 deletions(-)
 
Meanwhile active development aimed at 2.6.35 merge progressed.  The
major feature for this window is the merge of the delayed logging code,
which adds a new logging mode that dramatically reduces the bandwidth
required for log I/O.  See the
[http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/filesystems/xfs-delayed-logging-design.txt;h=96d0df28bed323d5596fc051b0ffb96ed8e3c8df;hb=HEAD documentation] for details.  Testers
for this new code are welcome.
 
In userland xfsprogs saw the long awaited 3.1.2 release, which can be
considered a bug fix release for xfs_repair, xfs_fsr and mkfs.xfs.  After
the release a few more fixes were merged into the development tree.
The xfstests package saw various new tests, including many tests to
exercise the quota code, and a few fixes to existing tests.
 
== XFS status update for April 2010 ==
 
In April 2.6.34 still was in the release candidate phase, with
a hand full of XFS fixes making it into mainline.  Development for
the 2.6.35 merge window went ahead full steam at the same time.
 
While a fair amount of patches hit the development tree these were
largely cleanups, with the real development activity happening on
the mailing list.  There was another round of patches and following
discussion on the scalable busy extent tracking and delayed logging
features mentioned last month.  They are expected to be merged in
May and queue up for the Linux 2.6.35 window.  Last but not least
April saw a large number of XFS fixes backported to the 2.6.32 and
2.6.33 -stable series.
 
In user land xfsprogs has seen few but important updates, preparing
for a new release next month.  The xfs_repair tool saw a fix to
correctly enable the lazy superblock counters on an existing
filesystem, and xfs_fsr saw updates to better deal with dynamic
attribute forks.  Last but not a least a port to Debian GNU/kFreeBSD
got merged. The xfstests test suite saw two new test cases and various
smaller fixes.
 
== XFS status update for March 2010 ==
 
The merge window for Linux 2.6.34 closed in the first week of March,
with the important XFS features already landing in February.  Not
surprisingly the XFS merge activity in March has been rather slow,
with only about a dozen bug fixes patches making it towards Linus'
tree in that time.
 
On the other hand active development for the 2.6.35 merge window has
been very active.  Most importantly there was a lot of work on the
transaction and log subsystems.  Starting with a large patchset to
clean up and refactor the transaction subsystem and introducing more
flexible I/O containers in the low-level logging code work is
progressing to a new, more efficient logging implementation.  While
this preparatory work has already been merged in the development tree,
the actual delayed logging implementation still needs more work after
the initial public posting.  The delayed logging implementation which
is very briefly modeled after the journaling mode in the ext3/4
and reiserfs filesystems allows to accumulated multiple asynchronous
transactions in memory instead of possibly writing them out
many times.  Using the new delayed logging mechanism I/O bandwidth
used for the log decreases by orders of magnitude and performance
on metadata intensive workloads increases massively.
 
In addition to that a new version of the discard (aka TRIM) support
has been posted, this time entirely contained in kernel space
and without the need of a userspace utility to drive it.  Last but
not least the usual steady stream of cleanups and bug fixes has not
ceased this month either.
 
Besides the usual flow of fixes and new test cases in the xfstests
test suite development on the userspace side has been rather slow.
Xfsprogs has only seen a single fix for SMP locking in xfs_repair
and support for building on Debian GNU/kFreeBSD, and xfsdump
has seen no commit at all.
 
== XFS status update for February 2010 ==
 
February saw the release of the Linux 2.6.33 kernel, which includes
a large XFS update.  The biggest user-visible change in Linux 2.6.33
is that XFS now support the generic Linux trace event infrastructure,
which allows tracing lots of XFS behavior with a normal production
built kernel.  Except for this Linux 2.6.33 has been mostly a bug-fix
release, fixing various user reported bugs in previous releases.
The total diffstat for XFS in Linux 2.6.33 looks like:
 
  84 files changed, 3023 insertions(+), 3550 deletions(-)
 
In addition to that the merge window for Linux 2.6.34 opened and the
first merge of the XFS tree made it into Linus tree.  Unlike Linux
2.6.33 this merge window includes major feature work.  The most
important change for users is a new algorithm for inode and quota
writeback that leads to better I/O locality and improved metadata
performance.  The second big change is a rewrite of the per-allocation
group data lookup which fixes a long-standing problem in the code
to grow a life filesystem and will also ease future filesystem
shrinking support.  Not merged through the XFS tree, but of great
importance for embedded users is a new API that allows XFS to properly
flush cache lines on it's log and large directory buffers, making
XFS work properly on architectures with virtually indexed caches,
such as parisc and various arm and mips variants.  Last but not
least there is an above average amount of cleanups that went into
Linus tree in this cycle.
 
There have been more patches on the mailing list that haven't made
it to Linus tree yet, including an optimized implementation of
fdatasync(2) and massive speedups for metadata workloads on
NFS exported XFS filesystems.
 
On the userspace side February has been a relatively quiet month.
Lead by xfstests only a moderate amount of fixes made it into
the respective trees.
 
== XFS status update for January 2010 ==
 
January saw additional release candidates of the Linux 2.6.33 kernel,
including a couple of bug fixes for XFS.  In the meantime the XFS tree
has been growing a large number of patches destined for the Linux 2.6.34
merge window: a large rework of the handling of per-AG data, support for
the quota netlink interface, and better power saving behavior of the
XFS kernel threads, and of course various cleanups.
 
A large patch series to replace the current asynchronous inode writeback
with a new scheme that uses the delayed write buffers was posted to
the list.  The new scheme, which allows archive better I/O locality by
dispatching meta-data I/O from a single place has been discussed
extensively and is expected to be merged in February.
 
On the userspace side January saw the 3.1.0 and 3.1.1 releases of xfsprogs,
as well as the 3.0.4 release of xfsdump.  The biggest changes in xfsprogs
3.1.0 were optimizations in xfs_repair that lead to a much lower memory
usage, and optional use of the blkid library for filesystem detection
and retrieving storage topology information.  The 3.1.1 release contained
various important bug fixes for these changes and a various improvements to
the build system.  The major feature of xfsdump 3.0.4 were fixes for
time stamp handling on 64-bit systems.
 
The xfstests package also lots of activity including various new testcases
and an improved build system.
 
== XFS status update for December 2009 ==
 
December finally saw the long awaited release of Linux 2.6.32, which for
XFS is mostly a bug fix release, with the major changes being various
improvement to the sync path, including working around the expectation
from the grub boot loader where metadata is supposed to be after a sync()
system call.  Together with a refactoring of the inode allocator this
gives a nice diffstat for this kernel release:
 
46 files changed, 767 insertions(+), 1048 deletions(-)
 
In the meantime development for the 2.6.33 has been going strong.  The
new event tracing code that allows to observe the inner workings of XFS
in production systems has finally been merged, with another patch to
reduce the size of the tracing code by using new upstream kernel features
posted for review.  Also a large patch series has been posted which
changes per-AG data to be looked up by a radix tree instead of the
existing array.  This works around possible deadlocks and user after
free issues during growfs, and prepares for removing a global (shared)
lock from the free space allocators.  In addition to that a wide range
of fixes has been posted and applied.
 
Work on the userspace packages has been just as busy.  In mkfs.xfs the
lazy superblock counter feature has now been enabled by default for the
upcoming xfsprogs 3.1.0 release, which will require kernel 2.6.22 for
the default mkfs invocation.  Also for mkfs.xfs as patch was posted
to correct the automatic detection of 4 kilobyte sector drivers which
are expected to show up in large quantities the real work soon.  The
norepair mode in xfs_repair has been enhanced with additional freespace
btree correction checks from xfs_db and is now identical to xfs_check in
filesystem consistency checking coverage.  A temporary file permission
problems has been fixed in xfs_fsr, and the libhandle library has been
fixed to better deal with symbolic links.  In xfs_io a few commands
that were added years ago have finally been wired up to actually be
usable.  And last but not least xfsdump saw a fix to the time stamp
handling in the backup format and some usability and documentation
improvements to xfsinvutil.
 
== XFS status update for November 2009 ==
 
November was a relatively slow month for XFS development.  The XFS tree
that is destined for the Linux 2.6.33 merge window saw a few fixes and
cleanups applied to it, and few important fixes still made it into the
last Linux 2.6.32 release candidates.  A few more patches including a
final version of the event tracing support for XFS were posted but not
reviewed yet.
 
On the userspace side there has been a fair amount of xfsprogs activity.
The repair speedup patches have finally been merged into the main development
branch and a couple of other fixes to the various utilities made it in, too.
The xfstests test suite saw another new regression test suite and a build
system fix up.
 
== XFS status update for October 2009 ==
In October we saw the Linux 2.6.32 merge window with a major XFS update.
This update includes a refactoring of the inode allocator which also
allows for speedups for very large filesystems, major sync fixes, updates
to the fsync and O_SYNC handling which merge the two code paths into a single
and more efficient one, a workaround for the VFS time stamp behavior,
and of course various smaller fixes.  A couple of additional fixes have been
queued up for the next merge window.
 
On the userspace side there has been a healthy activity on xfsprogs:  mkfs can
now discard unused sectors on SSDs and thinly provisioned storage devices and
use the more generic libblkid for topology information and filesystems detection
instead of the older libdisk, and the build system gained some updates to
make the source package generation simpler and shared for different package
types.  A patch has been out to the list but yet committed to add symbol
versioning to the libhandle library to make future ABI additions easier.
The xfstests package only saw some minor activity with a new test case
and small build system fixes.
 
New minor releases of xfsprogs and xfsdump were tagged but not formally
released after additional discussion.  Instead a new major xfsprogs release
is planned for next month.
 
== XFS status update for September 2009 ==
 
In September the Linux 2.6.31 kernel was finally released, including another
last minute XFS fix for the swapext (defragmentation) compat ioctl handler.
The final patch from 2.6.30 to 2.6.31 shows the following impressive diffstat
for XFS:
 
  55 files changed, 1476 insertions(+), 2269 deletions(-)
 
The 2.6.32 merge window started with a large XFS merge that included changes
to the inode allocator, and a few smaller fixes.  New versions of the sync
and time stamp fixes as well as the event tracing support have been posted
in September but not yet merged into the XFS development tree and/or mainline.
 
On the userspace side a large patch series to reduce the memory usage in
xfs_repair to acceptable levels was posted, but not yet merged.  A new xfs_df
shell script to measure use of the on disk space was posted but not yet
merged pending some minor review comments and a missing man page.  In addition
we saw the usual amount of smaller fixes and cleanups.
 
Also this month Felix Blyakher resigned from his post as XFS maintainer and handed off to Alex Elder.
 
== XFS status update for August 2009 ==
 
In August the Linux 2.6.31 kernel has still been in the release candidate
stage, but a couple of important XFS fixes made it in time for the release,
including a fix for the inode cache races with NFS workloads that have
plagued us for a long time.
 
The list saw various patches destined for the Linux 2.6.32 merge window,
including a merge of the fsync and O_SYNC handling code to address various
issues with the latter, a workaround for deficits in the timestamp handling
interface between the VFS and filesystems, a repost of the sync improvements
patch series and various smaller patches.
 
August also saw the minor 3.0.3 release of xfsprogs which collects smaller
fixes to the various tools and most importantly a fix to allow xfsprogs to
work again on SPARC and other strict alignment handling which regressed a
few releases ago.  The xfstests repository saw a few new test cases and a
various small improvements.
 
== XFS status update for July 2009 ==
 
As a traditional summer vacation month July has not seen a lot of XFS
activity.  The mainline 2.6.31 kernel made it to the 5th release candidate
but besides a few kernel-wide patches touching XFS the only activity were
two small patches fixing a bug in FIEMAP and working around writeback
performance problems in the VM.
 
A few more patches were posted to the list but haven't been merged yet.
Two big patch series deal with theoretically possible deadlocks due to
locks taken in reclaim contexts, which are now detected by lockdep.
 
The pace on the userspace side has been slow.  There have been a couple
of fixes to xfs_repair and xfs_db, and xfstests grew a few more testcases.
 
== XFS status update for June 2009 ==
 
On June 9th we finally saw the release of Linux 2.6.30.  For XFS
this release mostly contains the improved ENOSPC handling, but also
various smaller bugfixes and lots of cleanups.  The code size of XFS
decreased again by 500 lines of code in this release.
 
The Linux 2.6.31 merge opened in the mid of the month and some big XFS
changes have been pushed: A removal of the quotaops
infrastructure which simplifies the quota implementation, the switch
from XFS's own Posix ACL implementation to the generic one shared
by various other filesystems which also supports in-memory caching of
ACLs and another incremental refactoring of the sync code.
 
A patch to better track dirty inodes and work around issues in the
way the VFS updates the access time stamp on inodes has been reposted
and discussed. Another patch to converting the existing XFS tracing
infrastructure to use the ftrace even tracer has been posted.
 
On the userspace side there have been a few updates to xfsprogs, including
some repair fixes and a new fallocate command for xfs_io.  There were
major updates for xfstests:  The existing aio-dio-regress testsuite has
been merged into xfstests, and various changes went into the tree to make
xfstests better suitable for use with other filesystems.
 
The attr and acl projects which have been traditionally been hosted
as part of the XFS userspace utilities have now been split into a separate
project maintained by Andreas Gruenbacher, who has been doing most of
the work on it, and moved to the Savannah hosting platform.
 
== XFS status update for May 2009 ==
 
In May Linux 2.6.30 was getting close to be released, and various
important XFS fixes made it during the latest release candidates.
In the meantime some big patch series to rework the sync code and
the inode allocator have been posted for the next merge window.
 
On the userspace side xfsprogs and xfsdump 3.0.1 were finally released,
quickly followed by 3.0.2 releases with updated Debian packaging.
After that various small patches that were held back made it into xfsprogs.
A patch to add the xfs_reno tool which allows to move inodes around to
fit into 32 bit inode number space has been posted which is also one
central aspect of future online shrinking support.
 
There has been major activity on xfstests including adding generic
filesystems support to allow running tests that aren't XFS-specific on
any Linux filesystems.
 
== XFS status update for April 2009 ==
 
In April development for Linux 2.6.30 was in full motion.  A patchset to correct flushing of delayed allocations with near full filesystems has been committed in early April, as well as various smaller fixes. A patch series to improve the behavior of sys_sync has been posted but is waiting for VFS changes queued for Linux 2.6.31.
 
On the userspace side xfsprogs and xfsdump 3.0.1 have managed to split their release dates into May again after a lot of last-minute build system updates.
 
== XFS status update for March 2009 ==
 
Linux 2.6.29 has been released which includes major XFS updates like the
new generic btree code, a fully functional 32bit compat ioctl implementation
and the new combined XFS and Linux inode.  (See previous status reports
for more details). A patch series to improve correctness and performance
has been posted but not yet applied.  Various minor fixes and cleanups
have been sent to Linus for 2.6.30 which looks like it will be a minor
release for XFS after the big churn in 2.6.29.
 
On userspace a lot of time has been spent on fixing and improving the
build system shared by the various XFS utilities as well as various smaller
improvements leading to the xfsprogs and xfsdump 3.0.1 releases which are
still outstanding.
 
== XFS status update for February 2009 ==
 
In February various smaller fixes have been sent to Linus for 2.6.29,
including a revert of the faster vmap APIs which don't seem to be quite
ready yet on the VM side.  At the same time various patches have been
queued up for 2.6.30, with another big batch pending.  There also has
been a repost of the CRC patch series, including support for a new,
larger inode core.
 
SGI released various bits of work in progress from former employees
that will be extremely helpful for the future development of XFS,
thanks a lot to Mark Goodwin for making this happen.
 
On the userspace side the long awaited 3.0.0 releases of xfsprogs and
xfsdump finally happened early in the month, accompanied by a 2.2.9
release of the dmapi userspace.  There have been some issues with packaging
so a new minor release might follow soon.
 
The xfs_irecover tool has been relicensed so that it can be merged into
the GPLv2 codebase of xfsprogs, but the actual integration work hasn't
happened yet.
 
Important bits of XFS documentation that have been available on the XFS
website in PDF form have been released in the document source form under
the Creative Commons license so that they can be updated as a community
effort, and checked into a public git tree.
 
== XFS status update for January 2009 ==
 
January has been an extremely busy month on the userspace front.  Many
smaller and medium updates went into xfsprogs, xfstests and to a lesser
extent xfsdump.  xfsprogs and xfsdump are ramping up for getting a 3.0.0
release out in early February which will include the first major re-sync
with the kernel code in libxfs, a cleanup of the exported library interfaces
and the move of two tools (xfs_fsr and xfs_estimate) from the xfsdump
package to xfsprogs.  After this the xfsprogs package will contain all
tools that use internal libxfs interfaces which fortunately equates to those
needed for normal administration.  The xfsdump package now only contains
the xfsdump/xfsrestore tools needed for backing up and restoring XFS
filesystems.  In addition it grew a fix to support dump/restore on systems
with a 64k page size.  A large number of acl/attr package patches was
posted to the list, but pending a possible split of these packages from the
XFS project these weren't processed yet.
 
On the kernel side the big excitement in January was an in-memory corruption
introduced in the btree refactoring which hit people running 32bit platforms
without support for large block devices.  This issue was fixed and pushed
to the 2.6.29 development tree after a long collaborative debugging effort
at linux.conf.au.  Besides that about a dozen minor fixes were pushed to
2.6.29 and the first batch of misc patches for the 2.6.30 release cycle
was sent out.
 
At the end of December the SGI group in Melbourne which the previous
XFS maintainer and some other developers worked for has been closed down
and they will be missed greatly.  As a result maintainership has been passed
on in a way that has been slightly controversial in the community, and the
first patchset of work in progress in Melbourne have been posted to the list
to be picked up by others.
 
The xfs.org wiki has gotten a little facelift on it's front page making it
a lot easier to read.
 
== XFS status update for December 2008 ==
 
On Christmas Eve the 2.6.28 mainline kernel was release, with only minor XFS
bug fixes over 2.6.27.
 
On the development side December has been busy but unspectacular month.
All lot of misc fixes and improvements have been sent out, tested and committed
especially on the user land side.
 
== XFS status update for November 2008 ==
 
The mainline kernel is now at 2.6.28-rc6 and includes a small number of
XFS fixes.  There have been no updates to the XFS development tree during
November.  Without new regressions that large number of changes that
missed 2.6.28 has thus stabilized to be ready for 2.6.29.  In the meantime
kernel-side development has been slow, with the only major patch set
being a wide number of fixes to the compatibility for 32 bit ioctls on
a 64 bit kernel.
 
In the meantime there has been a large number of commits to the user space
tree, which mostly consist of smaller fixes.  xfsprogs is getting close
to have the 3.0.0 release which will be the first full resync with the
kernel sources since the year 2005.
 
== XFS status update for October 2008 ==
 
Linux 2.6.27 released with all the bits covered in last month's report.  It
did however miss two important fixes for regressions that a few people hit.
2.6.27.3 or later are recommended for use with XFS.
 
In the meantime the generic btree implementation, the sync reorganization
and after a lot of merge pain the XFS and VFS inode unification hit the
development tree during the time allocated for the merge window.  No XFS
updates other than the two regression fixes also in 2.6.27.3 have made it
into mainline as of 2.6.28-rc3.
 
The only new feature on the list in October is support for the fiemap
interface that has been added to the VFS during the 2.6.28 merge window.
However there was lot of patch traffic consisting of fixes and respun
versions of previously known patches.  There still is a large backlog of
patches on the list that is not applied to the development tree yet.
 
== XFS status update for September 2008 ==
 
With Linux 2.6.27 still not released but only making slow progress from 2.6.27-rc5 to 2.6.27-rc8 XFS changes in mainline have been minimal in September with only about half a dozen bug fixes patches.
 
In the meantime the generic btree patch set has been committed to the development tree, but not many other updates yet. On the user space side xfsprogs 2.10.1 has been released on September 5th with a number of important bug fixes. Following the release of xfsprogs 2.10.1 open season for development of the user space code has started. The first full update of the shared kernel / user space code in libxfs since 2005 has been committed. In addition to that the number of headers installed for the regular devel package has been reduced to the required minimum and support for checking the source code for endianess errors using sparse has been added.
 
The patch sets to unify the XFS and Linux inode structures, and rewrite various bits of the sync code have seen various iterations on the XFS list, but haven't been committed yet. A first set of patches implementing CRCs for various metadata structures has been posted to the list.
 
== XFS status update for August 2008 ==
 
With the 2.6.27-rc5 release the 2.6.27 cycle is nearing it's end. The major XFS feature in 2.6.27-rc5 is support for case-insensitive file names. At this point it is still limited to 7bit ASCII file names, with updates for utf8 file names expected to follow later. In addition to that 2.6.27-rc5 fixes a long-standing problem with non-EABI arm compiler which pack some XFS data structures wrongly. Besides this 2.6.27-rc5 also contains various cleanups, most notably the removal of the last bhv_vnode_t instances, and most uses of semaphores. As usual the diffstat for XFS from 2.6.26 to 2.6.26-rc5 is negative:
 
      100 files changed, 3819 insertions(+), 4409 deletions(-)
 
On the user space front a new minor xfsprogs version is about to be released containing various fixes including the user space part of arm packing fix.
 
Work in progress on the XFS mailing list are a large patch set to unify the alloc, inobt and bmap btree implementation into a single that supports arbitrarily pluggable key and record formats. These btree changes are the first major preparation for adding CRC checks to all metadata structures in XFS, and an even larger patch set to unify the XFS and Linux inode structures, and perform all inode write back from the btree uses instead of an inode cache in XFS.
 
== Updates before 2008 ==
 
News up to 2007 can be found on a separate page: [[OLD_News]]

Revision as of 10:54, 19 March 2012

XFS Status Updates

Retrieved from "https://xfs.org/index.php?title=XFS_Status_Updates&oldid=2463"