d6fbfc3545
The folder already knew how to stop properly, but the fs.Walk() didn't and can potentially take a very long time. This adds context support to Walk and the underlying scanning stuff, and passes in an appropriate context from above. The stop channel in model.folder is replaced with a context for this purpose. To test I added an infiniteFS that represents a large amount of data (not actually infinite, but close) and verify that walking it is properly stopped. For that to be implemented smoothly I moved out the Walk function to it's own type, as typically the implementer of a new filesystem type might not need or want to reimplement Walk. It's somewhat tricky to test that this actually works properly on the actual sendReceiveFolder and so on, as those are started from inside the model and the filesystem isn't easily pluggable etc. Instead I've tested that part manually by adding a huge folder and verifying that pause, resume and reconfig do the right things by looking at debug output. GitHub-Pull-Request: https://github.com/syncthing/syncthing/pull/4117
90 lines
2.9 KiB
Go
90 lines
2.9 KiB
Go
// Copyright 2009 The Go Authors. All rights reserved.
|
|
// Use of this source code is governed by a BSD-style
|
|
// license that can be found in the LICENSE file.
|
|
|
|
// This part copied directly from golang.org/src/path/filepath/path.go (Go
|
|
// 1.6) and lightly modified to be methods on BasicFilesystem.
|
|
|
|
// In our Walk() all paths given to a WalkFunc() are relative to the
|
|
// filesystem root.
|
|
|
|
package fs
|
|
|
|
import "path/filepath"
|
|
|
|
// WalkFunc is the type of the function called for each file or directory
|
|
// visited by Walk. The path argument contains the argument to Walk as a
|
|
// prefix; that is, if Walk is called with "dir", which is a directory
|
|
// containing the file "a", the walk function will be called with argument
|
|
// "dir/a". The info argument is the FileInfo for the named path.
|
|
//
|
|
// If there was a problem walking to the file or directory named by path, the
|
|
// incoming error will describe the problem and the function can decide how
|
|
// to handle that error (and Walk will not descend into that directory). If
|
|
// an error is returned, processing stops. The sole exception is when the function
|
|
// returns the special value SkipDir. If the function returns SkipDir when invoked
|
|
// on a directory, Walk skips the directory's contents entirely.
|
|
// If the function returns SkipDir when invoked on a non-directory file,
|
|
// Walk skips the remaining files in the containing directory.
|
|
type WalkFunc func(path string, info FileInfo, err error) error
|
|
|
|
type WalkFilesystem struct {
|
|
Filesystem
|
|
}
|
|
|
|
func NewWalkFilesystem(next Filesystem) *WalkFilesystem {
|
|
return &WalkFilesystem{next}
|
|
}
|
|
|
|
// walk recursively descends path, calling walkFn.
|
|
func (f *WalkFilesystem) walk(path string, info FileInfo, walkFn WalkFunc) error {
|
|
err := walkFn(path, info, nil)
|
|
if err != nil {
|
|
if info.IsDir() && err == SkipDir {
|
|
return nil
|
|
}
|
|
return err
|
|
}
|
|
|
|
if !info.IsDir() {
|
|
return nil
|
|
}
|
|
|
|
names, err := f.DirNames(path)
|
|
if err != nil {
|
|
return walkFn(path, info, err)
|
|
}
|
|
|
|
for _, name := range names {
|
|
filename := filepath.Join(path, name)
|
|
fileInfo, err := f.Lstat(filename)
|
|
if err != nil {
|
|
if err := walkFn(filename, fileInfo, err); err != nil && err != SkipDir {
|
|
return err
|
|
}
|
|
} else {
|
|
err = f.walk(filename, fileInfo, walkFn)
|
|
if err != nil {
|
|
if !fileInfo.IsDir() || err != SkipDir {
|
|
return err
|
|
}
|
|
}
|
|
}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// Walk walks the file tree rooted at root, calling walkFn for each file or
|
|
// directory in the tree, including root. All errors that arise visiting files
|
|
// and directories are filtered by walkFn. The files are walked in lexical
|
|
// order, which makes the output deterministic but means that for very
|
|
// large directories Walk can be inefficient.
|
|
// Walk does not follow symbolic links.
|
|
func (f *WalkFilesystem) Walk(root string, walkFn WalkFunc) error {
|
|
info, err := f.Lstat(root)
|
|
if err != nil {
|
|
return walkFn(root, nil, err)
|
|
}
|
|
return f.walk(root, info, walkFn)
|
|
}
|