| Age | Commit message (Collapse) | Author |
|---|
|
Fix the nlinks now too.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Copy-pasta the online quotacheck code to check inode link counts too.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Report on the health of the inode link counts.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Add some experimental flags to drop inodes from the cache after a scan.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Introduce a variant on XFS_SCRUB_METADATA that allows for vectored mode.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Forward everything scrub whines about to dmesg.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Track some basic statistics on how hard we're pushing the defer ops.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
When we have a workload that does open/write/close on files with
extent size hints set in parallel with other allocation, the file
becomes rapidly fragmented. This is due to close() calling
xfs_release() and removing the preallocated extent beyond EOF. This
occurs for both buffered and direct writes that append to files with
extent size hints.
The existing open/write/close hueristic in xfs_release() does not
catch this as writes to files using extent size hints do not use
delayed allocation and hence do not leave delayed allocation blocks
allocated on the inode that can be detected in xfs_release(). Hence
XFS_IDIRTY_RELEASE never gets set.
In xfs_file_release(), we can tell whether the inode has extent size
hints set and skip EOF block truncation. We add this check to
xfs_can_free_eofblocks() so that we treat the post-EOF preallocated
extent like intentional preallocation and so are persistent unless
directly removed by userspace.
Before:
Test 2: Extent size hint fragmentation counts
/mnt/scratch/file.0: 1002
/mnt/scratch/file.1: 1002
/mnt/scratch/file.2: 1002
/mnt/scratch/file.3: 1002
/mnt/scratch/file.4: 1002
/mnt/scratch/file.5: 1002
/mnt/scratch/file.6: 1002
/mnt/scratch/file.7: 1002
After:
Test 2: Extent size hint fragmentation counts
/mnt/scratch/file.0: 4
/mnt/scratch/file.1: 4
/mnt/scratch/file.2: 4
/mnt/scratch/file.3: 4
/mnt/scratch/file.4: 4
/mnt/scratch/file.5: 4
/mnt/scratch/file.6: 4
/mnt/scratch/file.7: 4
Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
When we have a workload that does open/read/close in parallel with other
allocation, the file becomes rapidly fragmented. This is due to close()
calling xfs_release() and removing the speculative preallocation beyond
EOF.
The existing open/*/close heuristic in xfs_release() does not catch this
as a sync writer does not leave delayed allocation blocks allocated on
the inode for later writeback that can be detected in xfs_release() and
hence XFS_IDIRTY_RELEASE never gets set.
In xfs_file_release(), we know more about the released file context, and
so we need to communicate some of the details to xfs_release() so it can
do the right thing here and skip EOF block truncation. This defers the
EOF block cleanup for synchronous write contexts to the background EOF
block cleaner which will clean up within a few minutes.
Before:
Test 1: sync write fragmentation counts
/mnt/scratch/file.0: 919
/mnt/scratch/file.1: 916
/mnt/scratch/file.2: 919
/mnt/scratch/file.3: 920
/mnt/scratch/file.4: 920
/mnt/scratch/file.5: 921
/mnt/scratch/file.6: 916
/mnt/scratch/file.7: 918
After:
Test 1: sync write fragmentation counts
/mnt/scratch/file.0: 24
/mnt/scratch/file.1: 24
/mnt/scratch/file.2: 11
/mnt/scratch/file.3: 24
/mnt/scratch/file.4: 3
/mnt/scratch/file.5: 24
/mnt/scratch/file.6: 24
/mnt/scratch/file.7: 23
Signed-off-by: Dave Chinner <dchinner@redhat.com>
[darrick: wordsmithing, fix commit message]
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Certain workloads fragment files on XFS very badly, such as a software
package that creates a number of threads, each of which repeatedly run
the sequence: open a file, perform a synchronous write, and close the
file, which defeats the speculative preallocation mechanism. We work
around this problem by only deleting posteof blocks the /first/ time a
file is closed to preserve the behavior that unpacking a tarball lays
out files one after the other with no gaps.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Enable quotas for the realtime device.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
When growing the realtime bitmap or summary inodes, use
xfs_trans_alloc_inode to reserve quota for the blocks that could be
allocated to the file. Although we never enforce limits against the
root dquot, making a reservation means that the bmap code will update
the quota block count, which is necessary for correct accounting.
Found by running xfs/521.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Make chown's quota adjustments work with realtime files.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Add the necessary alignment checking code to the reflink remap code to
ensure that remap requests are aligned to rt extent boundaries if the
realtime extent size isn't a power of two. The VFS helpers assume that
they can use the usual (blocksize - 1) masking to avoid slow 64-bit
division, but since XFS is special we won't make everyone pay that cost
for our weird edge case.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Make the necessary tweaks to the reflink remapping code to support
remapping on the realtime volume when the rt extent size is larger than
a single rt block. We need to check that the remap arguments from
userspace are aligned to a rt extent boundary, and that the length
is always aligned, even if the kernel tried to round it up to EOF for
us. XFS can only map and remap full rt extents, so we have to be a
little more strict about the alignment there.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
CoW extent size hints are not allowed on filesystems that have large
realtime extents because we only want to perform the minimum required
amount of write-around (aka write amplification) for shared extents.
On filesystems where rtextsize > 1, allocations can only be done in
units of full rt extents, which means that we can only map an entire rt
extent's worth of blocks into the data fork. Hole punch requests become
conversions to unwritten if the request isn't aligned properly.
Because a copy-write fundamentally requires remapping, this means that
we also can only do copy-writes of a full rt extent. This is too
expensive for large hint sizes, since it's all or nothing.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
If we have shared realtime files and the rt extent size is larger than a
single fs block, we need to extend writeback requests to be aligned to
rt extent size granularity because we cannot share partial rt extents.
The front end should have set us up for this by dirtying the relevant
ranges.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
As noted in the previous patch, XFS can only unmap and map full rt
extents. This means that we cannot stop mid-extent for any reason,
including stepping around unwritten/written extents. Second, the
reflink and CoW mechanisms were not designed to handle shared unwritten
extents, so we have to do something to get rid of them.
If the user asks us to remap two files, we must scan both ranges
beforehand to convert any unwritten extents that are not aligned to rt
extent boundaries into zeroed written extents before sharing.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Copy on write encounters a major plot twist when the file being CoW'd
lives on the realtime volume and the realtime extent size is larger than
a single filesystem block. XFS can only unmap and remap full rt
extents, which means that allocations are always done in units of full
rt extents, and a request to unmap less than one extent is treated as a
request to convert an extent to unwritten status.
This behavioral quirk is not compatible with the existing CoW mechanism,
so we have to intercept every path through which files can be modified
to ensure that we dirty an entire rt extent at once so that we can remap
a full rt extent. Use the existing VFS unshare functions to dirty the
page cache to set that up.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
In anticipation of enabling reflink on the realtime volume where the
allocation unit is larger than a page, create an iomap function to dirty
arbitrary parts of a file's page cache so that when we dirty part of a
file that could undergo a COW extent, we can dirty an entire allocation
unit's worth of pages.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Make it so that filesystems can pass an explicit blocksize to the remap
prep function. This enables filesystems whose fundamental allocation
units are /not/ the same as the blocksize to ensure that the remapping
checks are aligned properly.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Enable reflink for realtime devices, sort of.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Plumb knowledge of refcount btrees into the inode core repair code.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Port the data device's refcount btree repair code to the realtime
refcount btree.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Walk the realtime refcount btree to find the CoW staging extents when
we're rebuilding the realtime rmap btree.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
When we're rebuilding the data device rmap, if we encounter a "refcount"
format fork, we have to walk the (realtime) refcount btree inode to
build the appropriate mappings.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
When we're rebuilding the realtime bitmap, check the proposed free
extents against the rt refcount btree to make sure we don't commit any
grievous errors.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
If we encounter a directory that has been configured to pass on a CoW
extent size hint to a new realtime file and the hint isn't an integer
multiple of the rt extent size, we should flag the hint for
administrative review and/or turn it off because that is a
misconfiguration.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Update the quota scrubber to allow dquots where the realtime block count
exceeds the block count of the rt volume if reflink is enabled.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Use the realtime refcount btree to cross-reference other data
structures.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Add code to scrub realtime refcount btrees.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Whenever we encounter corrupt realtime refcount btree blocks, we should
report that to the health monitoring system for later reporting.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
If we're adding a realtime section to the filesystem, create the rt
refcount btree inode before we start adding rt space.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
The size of filesystem transaction reservations depends on the maximum
height (maxlevels) of the realtime btrees. Since we don't want a grow
operation to increase the reservation size enough that we'll fail the
minimum log size checks on the next mount, constrain growfs operations
if they would cause an increase in the rt refcount btree maxlevels.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Wire up the copy-on-write extent size hint for realtime files, and
connect it to the rt allocator so that we avoid fragmentation on rt
filesystems.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
The copy-on-write extent size hint is subject to the same alignment
constraints as the regular extent size hint. Since we're in the process
of adding reflink (and therefore CoW) to the realtime device, we must
apply the same scattered rextsize alignment validation strategies to
both hints to deal with the possibility of rextsize changing.
Therefore, fix the inode validator to perform rextsize alignment checks
on regular realtime files, and to remove misaligned directory hints.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Currently, we (ab)use xfs_get_extsz_hint so that it always returns a
nonzero value for realtime files. This apparently was done to disable
delayed allocation for realtime files.
However, once we enable realtime reflink, we can also turn on the
alwayscow flag to force CoW writes to realtime files. In this case, the
logic will incorrectly send the write through the delalloc write path.
Fix this by adjusting the logic slightly.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Scan the realtime refcount tree at mount time to get rid of leftover
CoW staging extents.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Now that we can share blocks between realtime files, allow this
combination.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Update the remapping routines to be able to handle realtime files.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Update our write paths to support copy on write on the rt volume. This
works in more or less the same way as it does on the data device, with
the major exception that we never do delalloc on the rt volume.
Because we consider unwritten CoW fork staging extents to be incore
quota reservation, we update xfs_quota_reserve_blkres to support this
case. Though xfs doesn't allow rt and quota together, the change is
trivial and we shouldn't leave a logic bomb here.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Hoist all quota updates for reflink into a helper function, since things
are about to become more complicated.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Compute the maximum possible height of the realtime rmap btree when
reflink is enabled.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Don't error out on CoW staging extent records when realtime reflink is
enabled.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Variable names in this code module are inconsistent and confusing.
xfs_phys_extent describe physical mappings, so rename them "pmap".
xfs_refcount_intents describe refcount intents, so rename them "ri".
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Create a library routine to allocate and initialize an empty realtime
refcountbt inode. We'll use this for growfs, mkfs, and repair.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Wire up realtime refcount btree cursors wherever they're needed
throughout the code base.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Plumb in the pieces we need to embed the root of the realtime refcount
btree in an inode's data fork, complete with new fork type and
on-disk interpretation functions.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
|
Reserve some free blocks so that we will always have enough free blocks
in the data volume to handle expansion of the realtime refcount btree.
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
|
