Improving inode Caching - Revision history

Sandeen at 16:06, 12 April 2016

2016-04-12T16:06:37Z

← Older revision		Revision as of 16:06, 12 April 2016
Line 1:		Line 1:
	Future Directions for XFS		= Future Directions for XFS - Inode Subsystems =

			From http://oss.sgi.com/archives/xfs/2008-09/msg00799.html


	== Improving Inode Caching and Operation in XFS ==		== Improving Inode Caching and Operation in XFS ==

Dgc: /* Compressed Inode Cache */

2010-12-23T03:51:48Z

Compressed Inode Cache

← Older revision		Revision as of 03:51, 23 December 2010
Line 30:		Line 30:

	== Compressed Inode Cache ==		== Compressed Inode Cache ==
	~~-----------------------------~~

	The XFs inode cache uses a lot of memory. We can avoid this problem by making		The XFs inode cache uses a lot of memory. We can avoid this problem by making

Dgc: /* Combining XFS and VFS inodes */

2010-12-23T03:51:34Z

Combining XFS and VFS inodes

← Older revision		Revision as of 03:51, 23 December 2010
Line 26:		Line 26:

	== Combining XFS and VFS inodes ==		== Combining XFS and VFS inodes ==
	~~----------------------------~~

	Status: Done (October 2008)		Status: Done (October 2008)

Dgc: /* Reclaim Optimizations */

2010-12-23T03:51:08Z

Reclaim Optimizations

← Older revision		Revision as of 03:51, 23 December 2010
Line 304:		Line 304:

	Tracking inodes for reclaim in radix tree: Done		Tracking inodes for reclaim in radix tree: Done

	Using RCU for radix tree reclaim walks: Done		Using RCU for radix tree reclaim walks: Done

	Non-blocking background reclaim: Done		Non-blocking background reclaim: Done

	Parallelised shrinker based reclaim: Done		Parallelised shrinker based reclaim: Done

Dgc: /* Accelerated Reclaim of buftarg Page Cache for Inodes */

2010-12-23T03:50:49Z

Accelerated Reclaim of buftarg Page Cache for Inodes

← Older revision		Revision as of 03:50, 23 December 2010
Line 369:		Line 369:

	Per-buftarg buffer LRU reclaim: Done		Per-buftarg buffer LRU reclaim: Done

	Per-buftarg shrinker: Done		Per-buftarg shrinker: Done

	Per-buffer type reclaim prioritisation: Done		Per-buffer type reclaim prioritisation: Done

Dgc: /* Accelerated Reclaim of buftarg Page Cache for Inodes */

2010-12-23T03:50:37Z

Accelerated Reclaim of buftarg Page Cache for Inodes

@@ Line 367: / Line 367: @@
 == Accelerated Reclaim of buftarg Page Cache for Inodes ==
 ----------------------------------------------------
 For single use inodes or even read-only inodes, we read them in, use them, then
@@ Line 376: / Line 380: @@
 advantageous to accelerate the reclaim of these pages so that they do not build
 up unneccessarily.
 A better method would appear to be to leverage the delayed read queue
@@ Line 392: / Line 388: @@
 manage the page cache. This seems much simpler and a much more holistic
 approach to solving the problem than implementing page LRUs.
 == Killing Bufferheads (a.k.a "Die, buggerheads, Die!") ==

Dgc: /* Reclaim Optimizations */

2010-12-23T03:48:26Z

Reclaim Optimizations

← Older revision		Revision as of 03:48, 23 December 2010
Line 303:		Line 303:
	== Reclaim Optimizations ==		== Reclaim Optimizations ==

			Tracking inodes for reclaim in radix tree: Done
			Using RCU for radix tree reclaim walks: Done
			Non-blocking background reclaim: Done
			Parallelised shrinker based reclaim: Done

	Now that we have efficient unlink, we've got to handle the reclaim of all the		Now that we have efficient unlink, we've got to handle the reclaim of all the
Line 360:		Line 364:
	inode, not have to wait, re-allocate and uncompress it again. These		inode, not have to wait, re-allocate and uncompress it again. These
	are all solvable issues - they just add complexity.		are all solvable issues - they just add complexity.


	== Accelerated Reclaim of buftarg Page Cache for Inodes ==		== Accelerated Reclaim of buftarg Page Cache for Inodes ==

Dgc: /* Avoiding the Generic pdflush Code */

2010-12-23T03:42:48Z

Avoiding the Generic pdflush Code

← Older revision		Revision as of 03:42, 23 December 2010
Line 76:		Line 76:
	== Avoiding the Generic pdflush Code ==		== Avoiding the Generic pdflush Code ==

			Writeback of inodes via AIL: Done

	For pdflush driven writeback, we only want to write back data; all other inode		For pdflush driven writeback, we only want to write back data; all other inode
Line 89:		Line 90:
	requirements to avoid data writeback (and hence delayed allocation) being		requirements to avoid data writeback (and hence delayed allocation) being
	limited to the throughput of a single CPU per filesystem.		limited to the throughput of a single CPU per filesystem.

	~~As it is, once data writeback is separated from inode writeback, we could~~
	~~simply use pushing the AIL as a method of writing back metadata in the~~
	~~background. There is no good reason for writing the inode immediately after~~
	~~data if the inode is in the AIL - it will get written soon enough as the tail~~
	~~of the AIL gets moved along. If we log all inode changes, then we'll be~~
	~~unlikely to write the inode multiple times over it's dirty life-cycle as it~~
	~~will continue to be moved forward in the AIL each time it is logged...~~


	== Improving Inode Writeback ==		== Improving Inode Writeback ==

Dgc: /* Bypassing the Linux Inode Cache */

2010-12-23T03:41:39Z

Bypassing the Linux Inode Cache

← Older revision		Revision as of 03:41, 23 December 2010
Line 60:		Line 60:
	== Bypassing the Linux Inode Cache ==		== Bypassing the Linux Inode Cache ==

			Lookups: Done (October 2008)

	~~With a larger number of cached~~ inodes ~~that the linux inode cache could possibly~~		Tracking dirty inodes: Done
	~~hold, it makes sense to completely remove the linux inode cache from the lookup~~
	~~path. That is, we do all our inode lookup based on the XFS cache, and if we find~~
	~~a compressed inode we uncompress it and turn it into a combined linux+XFS inode.~~
	~~If we also fast-path igrab() to avoid the inode_lock in the common case~~
	~~(refcount > 0) then we will substantially reduce the traffic on the inode_lock.~~

	~~If we have not hashed the inode in the linux inode cache, we now have to take~~		Writeback of dirty inodes: Done
	~~care or tracking~~ dirty inodes ~~ourselves - unhashed inodes are not added to the~~
	~~superblock dirty inode list by __mark_inode_dirty(). However, we do get a~~		Writeback of dirty pages: still executed the by VFS
	~~callout (->dirty_inode) that allows us to do this ourselves. We can use this~~
	~~callout and a tag in the inode radix tree to track all~~ dirty ~~inodes, or even~~
	~~just use~~ the ~~superblock list ourselves. Either way, we now have a mechanism that~~
	~~allows us to track all dirty inodes our own way.~~

	Now that we can track dirty inodes ourselves, we can pretty much isolate		Now that we can track dirty inodes ourselves, we can pretty much isolate
Line 81:		Line 73:
	structure with the current writeback guidelines, we can do writeback within		structure with the current writeback guidelines, we can do writeback within
	those guidelines in the most optimal fashion for XFS.		those guidelines in the most optimal fashion for XFS.


	== Avoiding the Generic pdflush Code ==		== Avoiding the Generic pdflush Code ==

Dgc: /* Combining XFS and VFS inodes */

2010-12-23T03:37:35Z

Combining XFS and VFS inodes

← Older revision		Revision as of 03:37, 23 December 2010
Line 28:		Line 28:
	----------------------------		----------------------------

	~~If we combine the XFS and the Linux inode, we'll increase memory usage for~~		Status: Done (October 2008)
	~~cached inodes. However, combining them has some distinct advantages~~:

	- ~~one less memory allocation~~		== Compressed Inode Cache ==
	- ~~simpler reclaim code~~		-----------------------------
	- ~~removal of a significant amount of duplicate information~~
	- ~~a single reference count for the entire inode data~~
	- ~~solves several issues with RCU locking on the cache and~~
	~~reclaim races~~

	~~Of course, the downside is the~~ memory ~~usage~~. We can avoid this problem by making		The XFs inode cache uses a lot of memory. We can avoid this problem by making
	use of the compressed inode cache - only the active inodes are held in a		use of the compressed inode cache - only the active inodes are held in a
	non-compressed form, hence most inodes will end up being cached in compressed		non-compressed form, hence most inodes will end up being cached in compressed
Line 50:		Line 45:
	this compressed cache is discussed further later on in the reclaim context.]		this compressed cache is discussed further later on in the reclaim context.]

	It is worth noting that for embedded systems it may be worth while allowing		== Fixed Inode Cache Size ==

			It is worth noting that for embedded systems and appliances it may be worth while allowing
	the size of the caches to be fixed. Also, to prevent memory fragmentation		the size of the caches to be fixed. Also, to prevent memory fragmentation
	problems,~~we~~ could simply allocate that memory to the compressed cache slab.		problems, we could simply allocate that memory to the compressed cache slab.
	In effect, this would become a 'static slab' in that it has a bound maximum		In effect, this would become a 'static slab' in that it has a bound maximum
	size and never frees and memory. When the cache is full, we reclaim an		size and never frees and memory. When the cache is full, we reclaim an
Line 60:		Line 57:
	Such an extension to the slab caches does not look difficult to implement,		Such an extension to the slab caches does not look difficult to implement,
	and would allow such customisation with minimal deviation from mainline code.		and would allow such customisation with minimal deviation from mainline code.

	~~Also, it would be a nice-to-have to be able to make use of the 'init_once'~~
	~~slab cache optimisation that the linux inode uses for idempotent fields~~
	~~in the inode. This would reduce the allocation overhead of the combined~~
	~~inode because we wouldn't need to zero the whole structure and then~~
	~~re-initialise everything in it. We'd only need to initialise the changing~~
	~~structures in this case - it means a little more code but should use~~
	~~less CPU time for allocation (especially if the inodes are already hot~~
	~~in cache).~~

	== Bypassing the Linux Inode Cache ==		== Bypassing the Linux Inode Cache ==