summaryrefslogtreecommitdiff
path: root/fs/overlayfs
AgeCommit message (Collapse)AuthorFilesLines
2018-12-04Revert "ovl: relax permission checking on underlying layers"Miklos Szeredi1-13/+4
This reverts commit 007ea44892e6fa963a0876a979e34890325c64eb. The commit broke some selinux-testsuite cases, and it looks like there's no straightforward fix keeping the direction of this patch, so revert for now. The original patch was trying to fix the consistency of permission checks, and not an observed bug. So reverting should be safe. Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-11-21ovl: fix decode of dir file handle with multi lower layersAmir Goldstein1-3/+3
When decoding a lower file handle, we first call ovl_check_origin_fh() with connected=false to get any real lower dentry for overlay inode cache lookup. If the real dentry is a disconnected dir dentry, ovl_check_origin_fh() is called again with connected=true to get a connected real dentry and find the lower layer the real dentry belongs to. If the first call returned a connected real dentry, we use it to lookup an overlay connected dentry, but the first ovl_check_origin_fh() call with connected=false did not check that the found dentry is under the root of the layer (see ovl_acceptable()), it only checked that the found dentry super block matches the uuid of the lower file handle. In case there are multiple lower layers on the same fs and the found dentry is not from the top most lower layer, using the layer index returned from the first ovl_check_origin_fh() is wrong and we end up failing to decode the file handle. Fix this by always calling ovl_check_origin_fh() with connected=true if we got a directory dentry in the first call. Fixes: 8b58924ad55c ("ovl: lookup in inode cache first when decoding...") Cc: <stable@vger.kernel.org> # v4.17 Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-11-19ovl: fix missing override creds in link of a metacopy upperAmir Goldstein1-1/+13
Theodore Ts'o reported a v4.19 regression with docker-dropbox: https://marc.info/?l=linux-fsdevel&m=154070089431116&w=2 "I was rebuilding my dropbox Docker container, and it failed in 4.19 with the following error: ... dpkg: error: error creating new backup file \ '/var/lib/dpkg/status-old': Invalid cross-device link" The problem did not reproduce with metacopy feature disabled. The error was caused by insufficient credentials to set "trusted.overlay.redirect" xattr on link of a metacopy file. Reproducer: echo Y > /sys/module/overlay/parameters/redirect_dir echo Y > /sys/module/overlay/parameters/metacopy cd /tmp mkdir l u w m chmod 777 l u touch l/foo ln l/foo l/link chmod 666 l/foo mount -t overlay none -olowerdir=l,upperdir=u,workdir=w m su fsgqa ln m/foo m/bar [ 21.455823] overlayfs: failed to set redirect (-1) ln: failed to create hard link 'm/bar' => 'm/foo':\ Invalid cross-device link Reported-by: Theodore Y. Ts'o <tytso@mit.edu> Reported-by: Maciej Zięba <maciekz82@gmail.com> Fixes: 4120fe64dce4 ("ovl: Set redirect on upper inode when it is linked") Cc: <stable@vger.kernel.org> # v4.19 Signed-off-by: Amir Goldstein <amir73il@gmail.com> Acked-by: Vivek Goyal <vgoyal@redhat.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-11-02Merge tag 'xfs-4.20-merge-2' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linuxLinus Torvalds2-22/+27
Pull vfs dedup fixes from Dave Chinner: "This reworks the vfs data cloning infrastructure. We discovered many issues with these interfaces late in the 4.19 cycle - the worst of them (data corruption, setuid stripping) were fixed for XFS in 4.19-rc8, but a larger rework of the infrastructure fixing all the problems was needed. That rework is the contents of this pull request. Rework the vfs_clone_file_range and vfs_dedupe_file_range infrastructure to use a common .remap_file_range method and supply generic bounds and sanity checking functions that are shared with the data write path. The current VFS infrastructure has problems with rlimit, LFS file sizes, file time stamps, maximum filesystem file sizes, stripping setuid bits, etc and so they are addressed in these commits. We also introduce the ability for the ->remap_file_range methods to return short clones so that clones for vfs_copy_file_range() don't get rejected if the entire range can't be cloned. It also allows filesystems to sliently skip deduplication of partial EOF blocks if they are not capable of doing so without requiring errors to be thrown to userspace. Existing filesystems are converted to user the new remap_file_range method, and both XFS and ocfs2 are modified to make use of the new generic checking infrastructure" * tag 'xfs-4.20-merge-2' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux: (28 commits) xfs: remove [cm]time update from reflink calls xfs: remove xfs_reflink_remap_range xfs: remove redundant remap partial EOF block checks xfs: support returning partial reflink results xfs: clean up xfs_reflink_remap_blocks call site xfs: fix pagecache truncation prior to reflink ocfs2: remove ocfs2_reflink_remap_range ocfs2: support partial clone range and dedupe range ocfs2: fix pagecache truncation prior to reflink ocfs2: truncate page cache for clone destination file before remapping vfs: clean up generic_remap_file_range_prep return value vfs: hide file range comparison function vfs: enable remap callers that can handle short operations vfs: plumb remap flags through the vfs dedupe functions vfs: plumb remap flags through the vfs clone functions vfs: make remap_file_range functions take and return bytes completed vfs: remap helper should update destination inode metadata vfs: pass remap flags to generic_remap_checks vfs: pass remap flags to generic_remap_file_range_prep vfs: combine the clone and dedupe into a single remap_file_range ...
2018-11-01ovl: automatically enable redirect_dir on metacopy=onMiklos Szeredi1-7/+29
Current behavior is to automatically disable metacopy if redirect_dir is not enabled and proceed with the mount. If "metacopy=on" mount option was given, then this behavior can confuse the user: no mount failure, yet metacopy is disabled. This patch makes metacopy=on imply redirect_dir=on. The converse is also true: turning off full redirect with redirect_dir= {off|follow|nofollow} will disable metacopy. If both metacopy=on and redirect_dir={off|follow|nofollow} is specified, then mount will fail, since there's no way to correctly resolve the conflict. Reported-by: Daniel Walsh <dwalsh@redhat.com> Fixes: d5791044d2e5 ("ovl: Provide a mount option metacopy=on/off...") Cc: <stable@vger.kernel.org> # v4.19 Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-10-31ovl: check whiteout in ovl_create_over_whiteout()Miklos Szeredi1-0/+4
Kaixuxia repors that it's possible to crash overlayfs by removing the whiteout on the upper layer before creating a directory over it. This is a reproducer: mkdir lower upper work merge touch lower/file mount -t overlay overlay -olowerdir=lower,upperdir=upper,workdir=work merge rm merge/file ls -al merge/file rm upper/file ls -al merge/ mkdir merge/file Before commencing with a vfs_rename(..., RENAME_EXCHANGE) verify that the lookup of "upper" is positive and is a whiteout, and return ESTALE otherwise. Reported by: kaixuxia <xiakaixu1987@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com> Fixes: e9be9d5e76e3 ("overlay filesystem") Cc: <stable@vger.kernel.org> # v3.18
2018-10-30vfs: plumb remap flags through the vfs dedupe functionsDarrick J. Wong1-1/+2
Plumb a remap_flags argument through the vfs_dedupe_file_range_one functions so that dedupe can take advantage of it. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
2018-10-30vfs: plumb remap flags through the vfs clone functionsDarrick J. Wong2-4/+4
Plumb a remap_flags argument through the {do,vfs}_clone_file_range functions so that clone can take advantage of it. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
2018-10-30vfs: make remap_file_range functions take and return bytes completedDarrick J. Wong2-9/+9
Change the remap_file_range functions to take a number of bytes to operate upon and return the number of bytes they operated on. This is a requirement for allowing fs implementations to return short clone/dedupe results to the user, which will enable us to obey resource limits in a graceful manner. A subsequent patch will enable copy_file_range to signal to the ->clone_file_range implementation that it can handle a short length, which will be returned in the function's return value. For now the short return is not implemented anywhere so the behavior won't change -- either copy_file_range manages to clone the entire range or it tries an alternative. Neither clone ioctl can take advantage of this, alas. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Dave Chinner <david@fromorbit.com>
2018-10-30vfs: combine the clone and dedupe into a single remap_file_rangeDarrick J. Wong1-13/+17
Combine the clone_file_range and dedupe_file_range operations into a single remap_file_range file operation dispatch since they're fundamentally the same operation. The differences between the two can be made in the prep functions. Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Dave Chinner <david@fromorbit.com>
2018-10-27ovl: using posix_acl_xattr_size() to get size instead of posix_acl_to_xattr()Chengguang Xu1-3/+2
There is no functional change but it seems better to get size by calling posix_acl_xattr_size() instead of calling posix_acl_to_xattr() with NULL buffer argument. Additionally, remove unnecessary assignments. Signed-off-by: Chengguang Xu <cgxu519@gmx.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-10-27ovl: abstract ovl_inode lock with a helperAmir Goldstein2-12/+23
The abstraction improves code readabilty (to some). Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-10-27ovl: remove the 'locked' argument of ovl_nlink_{start,end}Amir Goldstein3-28/+25
It just makes the interface strange without adding any significant value. The only case where locked is false and return value is 0 is in ovl_rename() when new is negative, so handle that case explicitly in ovl_rename(). Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-10-27ovl: relax requirement for non null uuid of lower fsAmir Goldstein2-4/+31
We use uuid to associate an overlay lower file handle with a lower layer, so we can accept lower fs with null uuid as long as all lower layers with null uuid are on the same fs. This change allows enabling index and nfs_export features for the setup of single lower fs of type squashfs - squashfs supports file handles, but has a null uuid. This change also allows enabling index and nfs_export features for nested overlayfs, where the lower overlay has nfs_export enabled. Enabling the index feature with single lower squashfs fixes the unionmount-testsuite test: ./run --ov --squashfs --verify As a by-product, if, like the lower squashfs, upper fs also uses the generic export_encode_fh() implementation to export 32bit inode file handles (e.g. ext4), then the xino_auto config/module/mount option will enable unique overlay inode numbers. Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-10-27ovl: fold copy-up helpers into callersMiklos Szeredi1-108/+67
Now that the workdir and tmpfile copy up modes have been untagled, the functions become simple enough that the helpers can be folded into the callers. Add new helpers where there is any duplication remaining: preparing creds for creating the object. Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-10-27ovl: untangle copy up call chainAmir Goldstein1-86/+159
In an attempt to dedup ~100 LOC, we ended up creating a tangled call chain, whose branches merge and diverge in several points according to the immutable c->tmpfile copy up mode. This call chain was hard to analyse for locking correctness because the locking requirements for the c->tmpfile flow were very different from the locking requirements for the !c->tmpfile flow (i.e. directory vs. regulare file copy up). Split the copy up helpers of the c->tmpfile flow from those of the !c->tmpfile (i.e. workdir) flow and remove the c->tmpfile mode from copy up context. Suggested-by: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-10-27ovl: relax permission checking on underlying layersMiklos Szeredi1-4/+13
Make permission checking more consistent: - special files don't need any access check on underling fs - exec permission check doesn't need to be performed on underlying fs Reported-by: "J. Bruce Fields" <bfields@fieldses.org> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-10-27ovl: fix recursive oi->lock in ovl_link()Amir Goldstein1-0/+4
linking a non-copied-up file into a non-copied-up parent results in a nested call to mutex_lock_interruptible(&oi->lock). Fix this by copying up target parent before ovl_nlink_start(), same as done in ovl_rename(). ~/unionmount-testsuite$ ./run --ov -s ~/unionmount-testsuite$ ln /mnt/a/foo100 /mnt/a/dir100/ WARNING: possible recursive locking detected -------------------------------------------- ln/1545 is trying to acquire lock: 00000000bcce7c4c (&ovl_i_lock_key[depth]){+.+.}, at: ovl_copy_up_start+0x28/0x7d but task is already holding lock: 0000000026d73d5b (&ovl_i_lock_key[depth]){+.+.}, at: ovl_nlink_start+0x3c/0xc1 [SzM: this seems to be a false positive, but doing the copy-up first is harmless and removes the lockdep splat] Reported-by: syzbot+3ef5c0d1a5cb0b21e6be@syzkaller.appspotmail.com Fixes: 5f8415d6b87e ("ovl: persistent overlay inode nlink for...") Cc: <stable@vger.kernel.org> # v4.13 Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-10-27ovl: clean up error handling in ovl_get_tmpfile()Miklos Szeredi1-3/+2
If security_inode_copy_up() fails, it should not set new_creds, so no need for the cleanup (which would've Oops-ed anyway, due to old_creds being NULL). Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-10-27ovl: fix error handling in ovl_verify_set_fh()Amir Goldstein1-1/+3
We hit a BUG on kfree of an ERR_PTR()... Reported-by: syzbot+ff03fe05c717b82502d0@syzkaller.appspotmail.com Fixes: 8b88a2e64036 ("ovl: verify upper root dir matches lower root dir") Cc: <stable@vger.kernel.org> # v4.13 Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-10-04ovl: fix format of setxattr debugMiklos Szeredi1-2/+2
Format has a typo: it was meant to be "%.*s", not "%*s". But at some point callers grew nonprintable values as well, so use "%*pE" instead with a maximized length. Reported-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com> Fixes: 3a1e819b4e80 ("ovl: store file handle of lower inode on copy up") Cc: <stable@vger.kernel.org> # v4.12
2018-10-04ovl: fix access beyond unterminated stringsAmir Goldstein1-1/+1
KASAN detected slab-out-of-bounds access in printk from overlayfs, because string format used %*s instead of %.*s. > BUG: KASAN: slab-out-of-bounds in string+0x298/0x2d0 lib/vsprintf.c:604 > Read of size 1 at addr ffff8801c36c66ba by task syz-executor2/27811 > > CPU: 0 PID: 27811 Comm: syz-executor2 Not tainted 4.19.0-rc5+ #36 ... > printk+0xa7/0xcf kernel/printk/printk.c:1996 > ovl_lookup_index.cold.15+0xe8/0x1f8 fs/overlayfs/namei.c:689 Reported-by: syzbot+376cea2b0ef340db3dd4@syzkaller.appspotmail.com Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com> Fixes: 359f392ca53e ("ovl: lookup index entry for copy up origin") Cc: <stable@vger.kernel.org> # v4.13
2018-09-25ovl: make symbol 'ovl_aops' staticWei Yongjun1-1/+1
Fixes the following sparse warning: fs/overlayfs/inode.c:507:39: warning: symbol 'ovl_aops' was not declared. Should it be static? Fixes: 5b910bd615ba ("ovl: fix GPF in swapfile_activate of file from overlayfs over xfs") Signed-off-by: Wei Yongjun <weiyongjun1@huawei.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-09-24vfs: swap names of {do,vfs}_clone_file_range()Amir Goldstein2-2/+2
Commit 031a072a0b8a ("vfs: call vfs_clone_file_range() under freeze protection") created a wrapper do_clone_file_range() around vfs_clone_file_range() moving the freeze protection to former, so overlayfs could call the latter. The more common vfs practice is to call do_xxx helpers from vfs_xxx helpers, where freeze protecction is taken in the vfs_xxx helper, so this anomality could be a source of confusion. It seems that commit 8ede205541ff ("ovl: add reflink/copyfile/dedup support") may have fallen a victim to this confusion - ovl_clone_file_range() calls the vfs_clone_file_range() helper in the hope of getting freeze protection on upper fs, but in fact results in overlayfs allowing to bypass upper fs freeze protection. Swap the names of the two helpers to conform to common vfs practice and call the correct helpers from overlayfs and nfsd. Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-09-24ovl: fix freeze protection bypass in ovl_clone_file_range()Amir Goldstein1-1/+1
Tested by doing clone on overlayfs while upper xfs+reflink is frozen: xfs_io -f /ovl/y fsfreeze -f /xfs xfs_io> reflink /ovl/x Before the fix xfs_io enters xfs_reflink_remap_range() and blocks in xfs_trans_alloc(). After the fix, xfs_io blocks outside xfs code in ovl_clone_file_range(). Fixes: 8ede205541ff ("ovl: add reflink/copyfile/dedup support") Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-09-24ovl: fix freeze protection bypass in ovl_write_iter()Amir Goldstein1-0/+2
Tested by re-writing to an open overlayfs file while upper ext4 is frozen: xfs_io -f /ovl/x xfs_io> pwrite 0 4096 fsfreeze -f /ext4 xfs_io> pwrite 0 4096 WARNING: CPU: 0 PID: 1492 at fs/ext4/ext4_jbd2.c:53 \ ext4_journal_check_start+0x48/0x82 After the fix, the second write blocks in ovl_write_iter() and avoids hitting WARN_ON(sb->s_writers.frozen == SB_FREEZE_COMPLETE) in ext4_journal_check_start(). Fixes: 2a92e07edc5e ("ovl: add ovl_write_iter()") Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-09-24ovl: fix memory leak on unlink of indexed fileAmir Goldstein1-1/+2
The memory leak was detected by kmemleak when running xfstests overlay/051,053 Fixes: caf70cb2ba5d ("ovl: cleanup orphan index entries") Cc: <stable@vger.kernel.org> # v4.13 Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-09-10ovl: fix oopses in ovl_fill_super() failure pathsMiklos Szeredi1-12/+14
ovl_free_fs() dereferences ofs->workbasedir and ofs->upper_mnt in cases when those might not have been initialized yet. Fix the initialization order for these fields. Reported-by: syzbot+c75f181dc8429d2eb887@syzkaller.appspotmail.com Signed-off-by: Miklos Szeredi <mszeredi@redhat.com> Cc: <stable@vger.kernel.org> # v4.15 Fixes: 95e6d4177cb7 ("ovl: grab reference to workbasedir early") Fixes: a9075cdb467d ("ovl: factor out ovl_free_fs() helper")
2018-09-03ovl: add ovl_fadvise()Amir Goldstein1-0/+20
Implement stacked fadvise to fix syscalls readahead(2) and fadvise64(2) on an overlayfs file. Suggested-by: Miklos Szeredi <mszeredi@redhat.com> Fixes: d1d04ef8572b ("ovl: stack file ops") Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-08-30ovl: fix GPF in swapfile_activate of file from overlayfs over xfsAmir Goldstein2-3/+6
Since overlayfs implements stacked file operations, the underlying filesystems are not supposed to be exposed to the overlayfs file, whose f_inode is an overlayfs inode. Assigning an overlayfs file to swap_file results in an attempt of xfs code to dereference an xfs_inode struct from an ovl_inode pointer: CPU: 0 PID: 2462 Comm: swapon Not tainted 4.18.0-xfstests-12721-g33e17876ea4e #3402 RIP: 0010:xfs_find_bdev_for_inode+0x23/0x2f Call Trace: xfs_iomap_swapfile_activate+0x1f/0x43 __se_sys_swapon+0xb1a/0xee9 Fix this by not assigning the real inode mapping to f_mapping, which will cause swapon() to return an error (-EINVAL). Although it makes sense not to allow setting swpafile on an overlayfs file, some users may depend on it, so we may need to fix this up in the future. Keeping f_mapping pointing to overlay inode mapping will cause O_DIRECT open to fail. Fix this by installing ovl_aops with noop_direct_IO in overlay inode mapping. Keeping f_mapping pointing to overlay inode mapping will cause other a_ops related operations to fail (e.g. readahead()). Those will be fixed by follow up patches. Suggested-by: Miklos Szeredi <mszeredi@redhat.com> Fixes: f7c72396d0de ("ovl: add O_DIRECT support") Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-08-30ovl: respect FIEMAP_FLAG_SYNC flagAmir Goldstein1-0/+4
Stacked overlayfs fiemap operation broke xfstests that test delayed allocation (with "_test_generic_punch -d"), because ovl_fiemap() failed to write dirty pages when requested. Fixes: 9e142c4102db ("ovl: add ovl_fiemap()") Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-08-22ovl: set I_CREATING on inode being createdMiklos Szeredi1-0/+4
...otherwise there will be list corruption due to inode_sb_list_add() being called for inode already on the sb list. Signed-off-by: Miklos Szeredi <mszeredi@redhat.com> Fixes: e950564b97fd ("vfs: don't evict uninitialized inode") Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2018-07-20ovl: Enable metadata only featureVivek Goyal1-3/+0
All the bits are in patches before this. So it is time to enable the metadata only copy up feature. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Do not do metacopy only for ioctl modifying file attrVivek Goyal1-1/+1
ovl_copy_up() by default will only do metadata only copy up (if enabled). That means when ovl_real_ioctl() calls ovl_real_file(), it will still get the lower file (as ovl_real_file() opens data file and not metacopy). And that means "chattr +i" will end up modifying lower inode. There seem to be two ways to solve this. A. Open metacopy file in ovl_real_ioctl() and do operations on that B. Force full copy up when FS_IOC_SETFLAGS is called. I am resorting to option B for now as it feels little safer option. If there are performance issues due to this, we can revisit it. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Do not do metadata only copy-up for truncate operationVivek Goyal1-1/+8
truncate should copy up full file (and not do metacopy only), otherwise it will be broken. For example, use truncate to increase size of a file so that any read beyong existing size will return null bytes. If we don't copy up full file, then we end up opening lower file and read from it only reads upto the old size (and not new size after truncate). Hence to avoid such situations, copy up data as well when file size changes. So far it was being done by d_real(O_WRONLY) call in truncate() path. Now that patch has been reverted. So force full copy up in ovl_setattr() if size of file is changing. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: add helper to force data copy-upVivek Goyal2-0/+6
Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Check redirect on index as wellVivek Goyal3-37/+64
Right now we seem to check redirect only if upperdentry is found. But it is possible that there is no upperdentry but later we found an index. We need to check redirect on index as well and set it in ovl_inode->redirect. Otherwise link code can assume that dentry does not have redirect and place a new one which breaks things. In my testing overlay/033 test started failing in xfstests. Following are the details. For example do following. $ mkdir lower upper work merged - Make lower dir with 4 links. $ echo "foo" > lower/l0.txt $ ln lower/l0.txt lower/l1.txt $ ln lower/l0.txt lower/l2.txt $ ln lower/l0.txt lower/l3.txt - Mount with index on and metacopy on. $ mount -t overlay -o lowerdir=lower,upperdir=upper,workdir=work,\ index=on,metacopy=on none merged - Link lower $ ln merged/l0.txt merged/l4.txt (This will metadata copy up of l0.txt and put an absolute redirect /l0.txt) $ echo 2 > /proc/sys/vm/drop/caches $ ls merged/l1.txt (Now l1.txt will be looked up. There is no upper dentry but there is lower dentry and index will be found. We don't check for redirect on index, hence ovl_inode->redirect will be NULL.) - Link Upper $ ln merged/l4.txt merged/l5.txt (Lookup of l4.txt will use inode from l1.txt lookup which is still in cache. It has ovl_inode->redirect NULL, hence link will put a new redirect and replace /l0.txt with /l4.txt - Drop caches. echo 2 > /proc/sys/vm/drop_caches - List l1.txt and it returns -ESTALE $ ls merged/l0.txt (It returns stale because, we found a metacopy of l0.txt in upper and it has redirect l4.txt but there is no file named l4.txt in lower layer. So lower data copy is not found and -ESTALE is returned.) So problem here is that we did not process redirect on index. Check redirect on index as well and then problem is fixed. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Set redirect on upper inode when it is linkedVivek Goyal1-0/+8
When we create a hardlink to a metacopy upper file, first the redirect on that inode. Path based lookup will not work with newly created link and redirect will solve that issue. Also use absolute redirect as two hardlinks could be in different directores and relative redirect will not work. I have not put any additional locking around setting redirects while introducing redirects for non-dir files. For now it feels like existing locking is sufficient. If that's not the case, we will have add more locking. Following is my rationale about why do I think current locking seems ok. Basic problem for non-dir files is that more than on dentry could be pointing to same inode and in theory only relying on dentry based locks (d->d_lock) did not seem sufficient. We set redirect upon rename and upon link creation. In both the paths for non-dir file, VFS locks both source and target inodes (->i_rwsem). That means vfs rename and link operations on same source and target can't he happening in parallel (Even if there are multiple dentries pointing to same inode). So that probably means that at a time on an inode, only one call of ovl_set_redirect() could be working and we don't need additional locking in ovl_set_redirect(). ovl_inode->redirect is initialized only when inode is created new. That means it should not race with any other path and setting ovl_inode->redirect should be fine. Reading of ovl_inode->redirect happens in ovl_get_redirect() path. And this called only in ovl_set_redirect(). And ovl_set_redirect() already seemed to be protected using ->i_rwsem. That means ovl_set_redirect() and ovl_get_redirect() on source/target inode should not make progress in parallel and is mutually exclusive. Hence no additional locking required. Now, only case where ovl_set_redirect() and ovl_get_redirect() could race seems to be case of absolute redirects where ovl_get_redirect() has to travel up the tree. In that case we already take d->d_lock and that should be sufficient as directories will not have multiple dentries pointing to same inode. So given VFS locking and current usage of redirect, current locking around redirect seems to be ok for non-dir as well. Once we have the logic to remove redirect when metacopy file gets copied up, then we probably will need additional locking. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Set redirect on metacopy files upon renameVivek Goyal1-20/+46
Set redirect on metacopy files upon rename. This will help find data dentry in lower dirs. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Do not set dentry type ORIGIN for broken hardlinksVivek Goyal1-1/+2
If a dentry has copy up origin, we set flag OVL_PATH_ORIGIN. So far this decision was easy that we had to check only for oe->numlower and if it is non-zero, we knew there is copy up origin. (For non-dir we installed origin dentry in lowerstack[0]). But we don't create ORGIN xattr for broken hardlinks (index=off). And with metacopy feature it is possible that we will install lowerstack[0] but ORIGIN xattr is not there. It is data dentry of upper metacopy dentry which has been found using regular name based lookup or using REDIRECT. So with addition of this new case, just presence of oe->numlower is not sufficient to guarantee that ORIGIN xattr is present. So to differentiate between two cases, look at OVL_CONST_INO flag. If this flag is set and upperdentry is there, that means it can be marked as type ORIGIN. OVL_CONST_INO is not set if lower hardlink is broken or will be broken over copy up. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Add an inode flag OVL_CONST_INOVivek Goyal2-0/+5
Add an ovl_inode flag OVL_CONST_INO. This flag signifies if inode number will remain constant over copy up or not. This flag does not get updated over copy up and remains unmodifed after setting once. Next patch in the series will make use of this flag. It will basically figure out if dentry is of type ORIGIN or not. And this can be derived by this flag. ORIGIN = (upperdentry && ovl_test_flag(OVL_CONST_INO, inode)). Suggested-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Treat metacopy dentries as type OVL_PATH_MERGEVivek Goyal1-1/+2
Right now OVL_PATH_MERGE is used only for merged directories. But conceptually, a metacopy dentry (backed by a lower data dentry) is a merged entity as well. So mark metacopy dentries as OVL_PATH_MERGE and ovl_rename() makes use of this property later to set redirect on a metacopy file. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Check redirects for metacopy filesVivek Goyal1-2/+2
Right now we rely on path based lookup for data origin of metacopy upper. This will work only if upper has not been renamed. We solved this problem already for merged directories using redirect. Use same logic for metacopy files. This patch just goes on to check redirects for metacopy files. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Move some dir related ovl_lookup_single() code in else blockVivek Goyal1-10/+11
Move some directory related code in else block. This is pure code reorganization and no functionality change. Next patch enables redirect processing on metacopy files and needs this change. By keeping non-functional changes in a separate patch, next patch looks much smaller and cleaner. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Do not expose metacopy only dentry from d_real()Vivek Goyal1-2/+5
Metacopy dentry/inode is internal to overlay and is never exposed outside of it. Exception is metacopy upper file used for fsync(). Modify d_real() to look for dentries/inode which have data, but also allow matching upper inode without data for the fsync case. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Open file with data except for the case of fsyncVivek Goyal1-9/+30
ovl_open() should open file which contains data and not open metacopy inode. With the introduction of metacopy inodes, with current implementaion we will end up opening metacopy inode as well. But there can be certain circumstances like ovl_fsync() where we want to allow opening a metacopy inode instead. Hence, change ovl_open_realfile() and and add extra parameter which specifies whether to allow opening metacopy inode or not. If this parameter is false, we look for data inode and open that. This should allow covering both the cases. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Add helper ovl_inode_realdata()Vivek Goyal2-0/+13
Add an helper to retrieve real data inode associated with overlay inode. This helper will ignore all metacopy inodes and will return only the real inode which has data. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Store lower data inode in ovl_inodeVivek Goyal6-6/+27
Right now ovl_inode stores inode pointer for lower inode. This helps with quickly getting lower inode given overlay inode (ovl_inode_lower()). Now with metadata only copy-up, we can have metacopy inode in middle layer as well and inode containing data can be different from ->lower. I need to be able to open the real file in ovl_open_realfile() and for that I need to quickly find the lower data inode. Hence store lower data inode also in ovl_inode. Also provide an helper ovl_inode_lowerdata() to access this field. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Fix ovl_getattr() to get number of blocks from lowerVivek Goyal3-1/+51
If an inode has been copied up metadata only, then we need to query the number of blocks from lower and fill up the stat->st_blocks. We need to be careful about races where we are doing stat on one cpu and data copy up is taking place on other cpu. We want to return stat->st_blocks either from lower or stable upper and not something in between. Hence, ovl_has_upperdata() is called first to figure out whether block reporting will take place from lower or upper. We now support metacopy dentries in middle layer. That means number of blocks reporting needs to come from lowest data dentry and this could be different from lower dentry. Hence we end up making a separate vfs_getxattr() call for metacopy dentries to get number of blocks. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
2018-07-20ovl: Add helper ovl_dentry_lowerdata() to get lower data dentryVivek Goyal2-0/+14
Now we have the notion of data dentry and metacopy dentry. ovl_dentry_lower() will return uppermost lower dentry, but it could be either data or metacopy dentry. Now we support metacopy dentries in lower layers so it is possible that lowerstack[0] is metacopy dentry while lowerstack[1] is actual data dentry. So add an helper which returns lowest most dentry which is supposed to be data dentry. Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
generated by cgit v1.2.3 (git 2.39.0) at 2024-12-30 10:45:15 +0300