webdesus / fs_extra Goto Github PK
View Code? Open in Web Editor NEWExpanding opportunities standard library std::fs and std::io
License: MIT License
Expanding opportunities standard library std::fs and std::io
License: MIT License
let mut path = result_path.as_os_str().to_os_string().into_string()?;
if path.find("\\\\?\\") == Some(0) {
path = path[4..].to_string();
}
First, this is relatively inefficient, as it creates a copy of the path and then searches entire path.
But most importantly, this isn't safe. UNC paths can change their meaning if they're reinterpreted as DOS paths.
Consider using https://lib.rs/crates/dunce which strips the prefix only when it is safe to do so.
It's pretty hard to display what is the result of the ls
command. It would be cool to be able to print it.
I am using standard example from documentation.
dir::copy(&src, &dst, &dir_options)?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `std::convert::From<fs_extra::error::Error>` is not implemented for `std::io::Error`
Same for file::copy
.
thread 'it_copy_exist_overwrite' panicked at 'called Result::unwrap()
on an Err
value: Error { kind: AlreadyExists, message: "entity already exists" }', /buildslave/rust-buildbot/slave/stable-dist-rustc-linux/build/src/libcore/result.rs:837
I would like to see a special copy option to transform symbolic links using absolute file paths to symbolic links relative to another path. The reason I am looking for such an option is that I am working on a tool to create sysroots for the purpose of cross-compiling based on the contents of docker containers (https://github.com/devolutions/albatross-rs). Just like chroot environments, the absolute symbolic links inside the docker container will not work when copied as-is for usage outside of the container.
For instance, if I copy the contents of a 32-bit Linux container, I get symbolic links like this:
/this-is-my-new-sysroot-dir/usr/lib/i386-linux-gnu/libz.so -> /lib/i386-linux-gnu/libz.so.1.2.11
where /lib/i386-linux-gnu/libz.so.1.2.11 is obviously something that only exists relative to the root of the original file system contained in the docker container. The same issue occurs with trying to copy files from a chroot for usage on the host.
The easiest way to work around the issue would be to provide the old root and modify all symbolic links to relative links (libz.so -> libz.so.1.2.11, no absolute path). Another option would be to simply "rebase" all symbolic links to another absolute path based on where the files will be copied, but I would rather go with the relative symbolic links option.
I'd appreciate it if you could make a new point release containing #38. This would help in evaluating the prevalence of macro trailing semicolons in the Rust ecosystem, as I'll be able to bump the fs_extra
dependency in various places.
Thanks!
It is Result<u32>
, not Result<()>
, but the documentation does not tell what that number (u32
) means.
Hello! ๐
While scanning crates.io. we noticed a document-code mismatch issue in this crate, we think the doc examples are generally meant to illustrate why someone would want to use the item.
Location
documentation for fs_extra::file::move_file_with_progress
(https://docs.rs/fs_extra/1.3.0/fs_extra/file/fn.move_file_with_progress.html)
Summary
The example of the document does not mention the move_file_with_progress
function, but mention move_file
, is it a doc-code mismatch?
Thank you for checking out this issue! ๐๐ค
If there's any reason you don't think this is a mismatch issue, please let me know, we're doing an empirical study of documentation in the Rust ecosystem. Your answers will help us and better promote the development of the Rust ecosystem.
It would very nice to have an exclude paths in CopyOptions. For example, I want to copy one folder recursively but excluding some files / directories.
Thanks in advance for your work ๐
By default, the cp
command on Linux will attempt to reflink on Copy-on-Write filesystems (--reflink=auto
). I think the CopyOptions
struct should have similar option.
For example, if CopyOptions::reflink
is set to RefLink::Auto
, the copy
function will use the reflink_or_copy
function from the reflink
crate to copy files.
The advantages of reflink:
The latest release produces forwards-compatibility errors, fixed in 5f54a1a (Feb 2021) but never released. Could another release be published soon-ish?
if let Err(e) = fs_extra::dir::move_dir(&thing, &destination_file, &fs_extra::dir::CopyOptions::new()) {
messages += format!("Failed to move folder {}, reason {}\n", thing, e).as_str();
continue 'next_result;
}
such code prints(error contains file name)
Failed to move file /home/rafal/TESTTT/TT/AutoSave_0091.sav, reason Path "/home/rafal/TESTTT/TT/AutoSave_0091.sav" is exist
but such code from std::fs
don't print name of file/folder which is I think better, because I can put this name in any other place
if let Err(e) = fs::metadata("/hehe") {
messages += format!("HEHE {}, reason {}\n", thing, e).as_str();
continue 'next_result;
}
HEHE /hehe reason No such file or directory (os error 2)
is
fs_extra::dir_copy("a/b", "c/d");
goint to create c/d/b
, or just c/d
? Does it depend on c/d
existing or not? The documentation is very unclear.
"Copies the directory contents from one place to another using recursive method. This function will also copy the permission bits of the original files to destionation files (not for directories)."
Explains non of it.
I'm responsible for maintaining the Fedora Linux packages for many Rust crates, the fs_extra crate among them. I tried updating our package to the recently released v1.3.0, but it appears that the test suite produces a lot of errors (all of which originating in tests/dir.rs
):
failures:
---- it_copy_progress_work stdout ----
thread 'it_copy_progress_work' panicked at 'assertion failed: `(left == right)`
left: `55`,
right: `57`', tests/dir.rs:853:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---- it_copy_with_progress_inside_no_overwrite_work_target_dir_exist_with_source_dir_exist stdout ----
thread 'it_copy_with_progress_inside_no_overwrite_work_target_dir_exist_with_source_dir_exist' panicked at 'assertion failed: `(left == right)`
left: `57`,
right: `59`', tests/dir.rs:3970:21
---- it_copy_with_progress_inside_work_target_dir_exist_with_no_source_dir_named_sub_dir stdout ----
thread 'it_copy_with_progress_inside_work_target_dir_exist_with_no_source_dir_named_sub_dir' panicked at 'assertion failed: `(left == right)`
left: `57`,
right: `59`', tests/dir.rs:3881:21
---- it_copy_with_progress_inside_work_target_dir_not_exist stdout ----
thread 'it_copy_with_progress_inside_work_target_dir_not_exist' panicked at 'assertion failed: `(left == right)`
left: `55`,
right: `57`', tests/dir.rs:3800:21
---- it_copy_with_progress_inside_overwrite_work_target_dir_exist_with_source_dir_exist stdout ----
thread 'it_copy_with_progress_inside_overwrite_work_target_dir_exist_with_source_dir_exist' panicked at 'assertion failed: `(left == right)`
left: `57`,
right: `59`', tests/dir.rs:4058:21
---- it_get_dir_content stdout ----
thread 'it_get_dir_content' panicked at 'assertion failed: `(left == right)`
left: `56`,
right: `58`', tests/dir.rs:2936:5
---- it_copy_with_progress_work_dif_buf_size stdout ----
thread '<unnamed>' panicked at 'assertion failed: `(left == right)`
left: `56`,
right: `58`', tests/dir.rs:980:13
thread 'it_copy_with_progress_work_dif_buf_size' panicked at 'assertion failed: `(left == right)`
left: `56`,
right: `58`', tests/dir.rs:1002:9
---- it_get_dir_content_many_levels stdout ----
thread 'it_get_dir_content_many_levels' panicked at 'assertion failed: `(left == right)`
left: `140`,
right: `202`', tests/dir.rs:2998:5
---- it_move_dir_with_progress_inside_no_overwrite_work_target_dir_exist_with_source_dir_exist stdout ----
thread 'it_move_dir_with_progress_inside_no_overwrite_work_target_dir_exist_with_source_dir_exist' panicked at 'assertion failed: `(left == right)`
left: `57`,
right: `59`', tests/dir.rs:4632:21
---- it_move_dir_with_progress_inside_work_target_dir_exist_with_no_source_dir_named_sub_dir stdout ----
thread 'it_move_dir_with_progress_inside_work_target_dir_exist_with_no_source_dir_named_sub_dir' panicked at 'assertion failed: `(left == right)`
left: `57`,
right: `59`', tests/dir.rs:4533:21
---- it_move_dir_with_progress_inside_work_target_dir_not_exist stdout ----
thread 'it_move_dir_with_progress_inside_work_target_dir_not_exist' panicked at 'assertion failed: `(left == right)`
left: `55`,
right: `57`', tests/dir.rs:4446:21
---- it_move_dir_with_progress_inside_overwrite_work_target_dir_exist_with_source_dir_exist stdout ----
thread 'it_move_dir_with_progress_inside_overwrite_work_target_dir_exist_with_source_dir_exist' panicked at 'assertion failed: `(left == right)`
left: `57`,
right: `59`', tests/dir.rs:4727:21
---- it_move_progress_work stdout ----
thread 'it_move_progress_work' panicked at 'assertion failed: `(left == right)`
left: `55`,
right: `57`', tests/dir.rs:2357:21
---- it_move_with_progress_work_dif_buf_size stdout ----
thread '<unnamed>' panicked at 'assertion failed: `(left == right)`
left: `56`,
right: `58`', tests/dir.rs:2501:13
thread 'it_move_with_progress_work_dif_buf_size' panicked at 'assertion failed: `(left == right)`
left: `56`,
right: `58`', tests/dir.rs:2523:9
failures:
it_copy_progress_work
it_copy_with_progress_inside_no_overwrite_work_target_dir_exist_with_source_dir_exist
it_copy_with_progress_inside_overwrite_work_target_dir_exist_with_source_dir_exist
it_copy_with_progress_inside_work_target_dir_exist_with_no_source_dir_named_sub_dir
it_copy_with_progress_inside_work_target_dir_not_exist
it_copy_with_progress_work_dif_buf_size
it_get_dir_content
it_get_dir_content_many_levels
it_move_dir_with_progress_inside_no_overwrite_work_target_dir_exist_with_source_dir_exist
it_move_dir_with_progress_inside_overwrite_work_target_dir_exist_with_source_dir_exist
it_move_dir_with_progress_inside_work_target_dir_exist_with_no_source_dir_named_sub_dir
it_move_dir_with_progress_inside_work_target_dir_not_exist
it_move_progress_work
it_move_with_progress_work_dif_buf_size
test result: FAILED. 68 passed; 14 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s
I originally saw these failures in our containerized build environment (which has limited OS access in some ways), but I also reproduced it by just cloning this git repo and just running cargo test
.
Tracing back the errors to where they originate in test sources, it appears that the calculation of expected total directory size is off - the assertion failures seem to point to lines like assert_eq!(get_dir_size() * 2 + 17, process_info.total_bytes);
. I don't understand what this assertion does, but in most cases, it seems to be off by 2 (or in one case, by 62).
Additional information:
What is the meaning of those flags?
Standard library std::fs
not have method receiving information about folder size.
Hey! Thx for lib, but I have noticed that directory copy behaves in non-idempotent way.
Suppose we have:
let pathbufs = [PathBuf::from("directory1"), PathBuf::from("directory2"), PathBuf::from("directory3")];
let options = fs_extra::dir::CopyOptions {
overwrite: true,
copy_inside: true,
..Default::default()
};
let to = PathBuf::new("current");
for p in pathbufs {
if let Err(e) = fs_extra::dir::copy(&p, &to, &options) {
eprintln!(
"Failed to copy directory: {:?} path: {}",
e,
p.display()
);
}
}
First time running copying content of the directory1
in the root of the current
path, and the second time it works as I want - I have all 3 dirs there but with directory1
files from previous run. Is there a way to copy all 3 directories if there are no current/
directory created previously?
Result after 1st run:
current/
|-- file1.txt
|-- file11.txt
|-- directory2
| |-- ...
|-- directory3
` |--...
Result after 2nd run
current/
|-- directory1
| |-- file1.txt
| `-- file11.txt
|-- file1.txt
|-- file11.txt
|-- directory2
| |-- ...
|-- directory3
` |--...
I understand that this is exactly the behavior of cp -r
on Linux, but maybe there could be a flag for this case?
If the directory contains symlinks, then dir::get_size
follows the symlinks and counts the sizes of those files too. If the targets of those symlinks are inside that same directory structure, then it effectively means the file sizes of some files are counted twice. Or if the targets of those symlinks are not inside that directory structure, then it means the returned size isn't really that of the requested directory, but also includes that of arbitrary other locations on the filesystem, which doesn't seem desirable.
mkdir test-dir
truncate -s 1M test-dir/large-file
ln -sr test-dir/large-file test-dir/symlink-to-large-file
fs_extra
v1.2.0
):
fn main() {
let size = fs_extra::dir::get_size("test-dir").expect("IO error calculating size");
println!("size = {size}");
}
fs_extra::dir::get_size()
returns the same size as ls
would, eg:
ls -al test-dir/
total 0
drwxr-xr-x 4 emorley staff 128 Jul 1 11:42 .
drwxr-xr-x 9 emorley staff 288 Jul 1 11:42 ..
-rw-r--r-- 1 emorley staff 1048576 Jul 1 11:41 large-file
lrwxr-xr-x 1 emorley staff 10 Jul 1 11:42 symlink-to-large-file -> large-file
...so 1048576 + 10 (for the symlink) + 128 (for the directory itself) = 1048714
ie something like:
$ cargo run -q
size = 1048714
fs_extra::dir::get_size()
follows the symlink, so counts the size of that file twice. It also doesn't seem to count the size of the directory entry itself.
$ cargo run -q
size = 2097152
I believe the cause is that fs_extra::dir::get_size()
uses std::fs::metadata
not std::fs::symlink_metadata
, and that the former follows symlinks. Also, use of std::fs::metadata
is actually unnecessary since the std::fs::DirEntry
returned by std::fs::read_dir
already has the metadata we need.
I believe there is a race condition when recursively copying:
Lines 604 to 609 in 1754296
create_all
will fail when the directory already exists. Having two threads copying files to the same directory, this may result in an a File exists
error.
The my understanding tokio
compensates for this by ignoring this case as an error: https://docs.rs/tokio/latest/tokio/fs/fn.create_dir_all.html
Notable exception is made for situations where any of the directories specified in the path could not be created as it was being created concurrently. Such cases are considered to be successful. That is, calling
create_dir_all
concurrently from multiple threads or processes is guaranteed not to fail due to a race condition with itself.
I think it makes sense to implement the same logic here.
Currently fs_extra::remove_items
does not act on symlinks that point to not-existing targets. It is not clear from the documentation what the intended behavior is, so I am not sure if this asks for fixing, or just documenting.
Assume a directory that contains symlinks, both to existing and non-existing targets:
mkdir dir/
touch dir/file-1
ln -s dir/file-1 dir/link-1
ln -s dir/file-2 dir/link-2
A minimal rust program for reproduction:
fn main() {
for arg in std::env::args().skip(1) {
let path = Path::new(&arg);
println!("Removing path {:?}", path);
match fs_extra::remove_items(&[path]) {
Ok(_) => println!(" -> OK"),
Err(err) => println!(" -> Err: {}", err)
}
}
}
And then:
$ tree dir
dir
โโโ file-1
โโโ link-1 -> file-1
โโโ link-2 -> file-2
1 directory, 3 files
$ cargo run -- dir/link*
Removing path "dir/link-1"
-> OK
Removing path "dir/link-2"
-> OK
$ tree dir
dir
โโโ file-1
โโโ link-2 -> file-2
1 directory, 2 files
(above tested on MacOS and Linux)
Up to the maintainers of the library:
link-1
and link-2
are both removed. file-1
stays (unchanged).remove_items
has caveat (or maybe more broadly: recommend not to use with symlinks?)I think (1) makes the most sense. Alas this was my expectation when using remove_items
.
remove_items
uses file::remove
, in case the item is not a directoryfile::remove
only removes when the path existsstd::path::Path::exists
, that evaluates the symlink target (which here is missing) and not the symlink itselfSome files can be bigger but we want to change only one little part of this file. Will be a stupid load all full file data in a memory.
It's will be interesting for people who work on own format.
I want to make two operations at the current moment:
I'm getting the following error:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { kind: NotFound, message: "No such file or directory (os err
or 2)" }', src/main.rs:4:82
although it should work (I think).
Create test directory:
mkdir -p /tmp/rofl1/yeet
touch /tmp/rofl1/test.txt
touch /tmp/rofl1/yeet/bruh.txt
Cargo.toml
[package]
name = "rust_tmp"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
fs_extra = "1.2.0"
src/main.rs
use fs_extra::dir::CopyOptions;
fn main() {
fs_extra::dir::copy("/tmp/rofl1", "/tmp/rofl2", &CopyOptions::new()).unwrap();
}
What am I doing wrong?
Err(Error { kind: NotFound, message: "Path "/Users/edison/Downloads/123\ aaaa/ๅฝๆกฃ(5)/panda" does not exist or you don't have access!" })
Hello, thanks for the great crate! It really helps cut down on boilerplate for common fs
tasks.
I was wondering if you would be open to me adding the builder pattern for fs_extra::dir::CopyOptions
and fs_extra::file::CopyOptions
? They should both be valid for the full time they're being built so that means it should be rather simple to add and it allows for
use fs_extra::dir;
let mut opts = dir::CopyOptions::new();
opts.overwrite = true;
to be represented as
use fs_extra::dir;
let opts = dir::CopyOptions::new()
.overwrite(true);
There's not too much difference here, but getting to drop the mut
is slightly nicer since it means that refactoring away whatever uses the thing using opts
will let the rust compiler pick up that opts
is unused in the second and not in the first.
Nothing too major, but might be a bit of a nicety that people are used to. Let me know what you think!
How can I create pull request to start contributing to this repo?
this might already be supported, but to be honest from the docs I can't tell. The equivalent of what I want to do on the command line is.
โฏ ln -s shared/.gitattributes
โฏ cp --dereference .gitattributes gitattr
โฏ ls -la
lrwxrwxrwx - xeno 25 Jan 13:54 .gitattributes -> shared/.gitattributes
.rw-r--r-- 68 xeno 25 Jan 13:54 gitattr
Move should use rename, not copy and remove.
Especially, that now, virtually whole code of move is a copy paste of copy with minor differences that even if it to stay in current form should be refactored to get rid of these duplicate codes.
hey. I want to copy all files and folders from "a" folder to "b" folder. I tried to use dir::get_dir_content
and iterate with ::copy_item
, but I couldn't came up with a simple solution. I was able to copy "a" folder inside the "b" folder, but this is not what I want to do.
Is there a simple solution to this problem? If yes, I would love to see it in examples. If no, an additional function would be great.
Add support for copying files while preserving timestamps (do not change the last modified time), just like the cp command -p or --preserve option: http://man7.org/linux/man-pages/man1/cp.1.html
-p same as --preserve=mode,ownership,timestamps
--preserve[=ATTR_LIST]
preserve the specified attributes (default:
mode,ownership,timestamps), if possible additional attributes:
context, links, xattr, all
I am currently working on a tool to reuse cargo build artifacts from the "target" directory, and it turns out that cargo checks the last modified time on files and triggers a rebuild if it changed.
In my application, I need to get the size of a directory, and in order to avoid having to re-invent the wheel I was planning to use the dir::get_size
method provided by this library (thanks for your work!).
Unfortunately though, the results I am getting differ from the ones expected (I am targeting a Linux machine). This is because the implementation does not return the size of a directory, but the sum of all the sizes of its files, without taking into account the size of the directory structure itself.
You can quickly test this behaviour on a Linux machine by running the mention method on a given directory, and then running for the same directory the command du --bytes -s {dir_path}
.
If this is the expected behaviour, I would simply suggest to explicitly state it in the documentation in a clear way.
If this is not the expected behaviour, it can be easily fixed by slightly changing the implementation:
pub fn get_size<P>(path: P) -> Result<u64>
where
P: AsRef<Path>,
{
let mut result = 0;
if path.as_ref().is_dir() {
for entry in read_dir(&path)? {
let _path = entry?.path();
if _path.is_file() {
result += _path.metadata()?.len();
} else {
result += get_size(_path)?;
}
}
// add the directory data structure size
result += path.as_ref().metadata()?.len();
} else {
result = path.as_ref().metadata()?.len();
}
Ok(result)
}
I understand this will also require some work to update the tests to reflect the new implementation. Please let me know if support is needed for this (can't promise when I'll be able to have a deeper look into this in case it's required).
Hi, we use fs_extra
in Nushell; thanks for writing it!
We recently had a scary bug report, and it looks like the root cause is fs_extra::dir::move_dir()
. If you use move_dir()
to change the case of a directory name on Windows (ex: "Test" -> "test"), the directory is deleted and fs_extra
reports success.
For example:
// This deletes the directory named Test and does not panic!
fs_extra::dir::move_dir("Test", "test", &options).unwrap();
I've put together a minimal repro: https://github.com/rgwood/fs-extra-repro
fs-extra
v1.2.0
Windows 11 (can't reproduce on Linux)
rustc v1.63.0
I have been using the fs_crate for file copying in my project, and I've found it to be a valuable tool. However, I noticed that the crate currently provides options to either overwrite or skip existing files/folders, but it lacks the functionality to copy files with sequential naming.
This feature would be particularly useful in scenarios where users want to duplicate files to a destination directory but need to avoid overwriting existing files. The sequential naming option ensures that each copied file has a unique name without the risk of overwriting.
tests/file.rs and test/dir.rs rewrite with use Path.
Hi, I have this code:
static DIR_MOVE_OPTIONS: fs_extra::dir::CopyOptions = fs_extra::dir::CopyOptions {
overwrite: false,
skip_exist: true,
buffer_size: 64000,
copy_inside: true,
content_only: true,
depth: 0
};
static FILE_MOVE_OPTIONS: fs_extra::file::CopyOptions = fs_extra::file::CopyOptions {
overwrite: false,
skip_exist: true,
buffer_size: 64000
};
...
fn move_items(&self, from: &Path, to: &Path) -> Result<(), String> {
/*let mut renamed_from = from.to_owned();
if from.file_name().unwrap() != to.file_name().unwrap() {
// move_dir doesn't rename directories when moving(?), so we'll have to do it ourselves
renamed_from.pop();
renamed_from.push(&to.file_name().unwrap());
match fs::rename(from, &renamed_from) {
Err(error) => { return Err(String::from(format!("{:?}", &error))) },
Ok(_) => {}
}
}*/
match match from.is_dir() {
true => fs_extra::dir::move_dir(from, to, &DIR_MOVE_OPTIONS),
false => fs_extra::file::move_file(from, to, &FILE_MOVE_OPTIONS)
} {
Err(error) => Err(String::from(format!("{:?}", &error))),
Ok(_) => Ok(())
}
}
(#24 is why I have that comment block there)
Error { kind: PermissionDenied, message: \"Access is denied. (os error 5)\" }
thread 'main' panicked at 'Blackhole PANIC: Failed to move file/folder to Blackhole ("Error { kind: PermissionDenied, message: \"Access is denied. (os error 5)\" }")
"D:\\Servers\\Garry\'s Mod\\GarrysMod\\addons\\rrrrrrrrrrrrrrrrrrrr\\_bkeypads-4.0.3" -> "C:\\Users\\billy\\$BLACKHOLE\\_bkeypads-4.0.3 (14)"', src\show.rs:7:13
stack backtrace:
0: 0x7ff6270dff49 - std::backtrace_rs::backtrace::dbghelp::trace
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\..\..\backtrace\src\backtrace\dbghelp.rs:98
1: 0x7ff6270dff49 - std::backtrace_rs::backtrace::trace_unsynchronized
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\..\..\backtrace\src\backtrace\mod.rs:66
2: 0x7ff6270dff49 - std::sys_common::backtrace::_print_fmt
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\sys_common\backtrace.rs:79
3: 0x7ff6270dff49 - std::sys_common::backtrace::_print::{{impl}}::fmt
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\sys_common\backtrace.rs:58
4: 0x7ff6270f69ab - core::fmt::write
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\core\src\fmt\mod.rs:1080
5: 0x7ff6270db8f8 - std::io::Write::write_fmt<std::sys::windows::stdio::Stderr>
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\io\mod.rs:1516
6: 0x7ff6270e2a44 - std::sys_common::backtrace::_print
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\sys_common\backtrace.rs:61
7: 0x7ff6270e2a44 - std::sys_common::backtrace::print
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\sys_common\backtrace.rs:48
8: 0x7ff6270e2a44 - std::panicking::default_hook::{{closure}}
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\panicking.rs:208
9: 0x7ff6270e2628 - std::panicking::default_hook
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\panicking.rs:227
10: 0x7ff6270e32ff - std::panicking::rust_panic_with_hook
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\panicking.rs:577
11: 0x7ff6270e2e65 - std::panicking::begin_panic_handler::{{closure}}
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\panicking.rs:484
12: 0x7ff6270e082f - std::sys_common::backtrace::__rust_end_short_backtrace<closure-0,!>
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\sys_common\backtrace.rs:153
13: 0x7ff6270e2e19 - std::panicking::begin_panic_handler
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\panicking.rs:483
14: 0x7ff6270e2dcc - std::panicking::begin_panic_fmt
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\panicking.rs:437
15: 0x7ff62705044e - blackhole::show::show::Show::panic
at C:\Users\billy\Documents\GitHub\blackhole\src\show.rs:7
16: 0x7ff627040761 - blackhole::blackhole::blackhole::Blackhole::send
at C:\Users\billy\Documents\GitHub\blackhole\src\blackhole.rs:149
17: 0x7ff62703c731 - blackhole::main
at C:\Users\billy\Documents\GitHub\blackhole\src\main.rs:70
18: 0x7ff627038b2b - core::ops::function::FnOnce::call_once<fn(),tuple<>>
at C:\Users\billy\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\ops\function.rs:227
19: 0x7ff62704472b - std::sys_common::backtrace::__rust_begin_short_backtrace<fn(),tuple<>>
at C:\Users\billy\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\sys_common\backtrace.rs:137
20: 0x7ff6270451f1 - std::rt::lang_start::{{closure}}<tuple<>>
at C:\Users\billy\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\rt.rs:66
21: 0x7ff6270e3503 - core::ops::function::impls::{{impl}}::call_once
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\library\core\src\ops\function.rs:280
22: 0x7ff6270e3503 - std::panicking::try::do_call
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\panicking.rs:381
23: 0x7ff6270e3503 - std::panicking::try
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\panicking.rs:345
24: 0x7ff6270e3503 - std::panic::catch_unwind
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\panic.rs:382
25: 0x7ff6270e3503 - std::rt::lang_start_internal
at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4\/library\std\src\rt.rs:51
26: 0x7ff6270451c3 - std::rt::lang_start<tuple<>>
at C:\Users\billy\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\std\src\rt.rs:65
27: 0x7ff62703c940 - main
28: 0x7ff6270fb394 - invoke_main
at d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
29: 0x7ff6270fb394 - __scrt_common_main_seh
at d:\agent\_work\63\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
30: 0x7ffe75ec7c24 - BaseThreadInitThunk
31: 0x7ffe77e6d721 - RtlUserThreadStart
dir::move_dir
successfully creates all the directories and nested directories:
The only file copied is the first file in .git/objects/00
:
It does appear to contain data:
.git
folder stops the issue from happeningCurrently DirInfo.directories returns all directories recursively, but I need only to loop through children directories. I thought it was weird that it returns a recursive result by default. Can you add child_directories or change it to get_directories(recursive: bool)?
Error { kind: InvalidFile, message: "Path \"/tmp/state/0/data/ipc\" is not a file!" }
I wish it would just skip copying it...
If one of the file entries being copied by fs_extra::dir::copy
happens to be a symlink to a directory, then that symlink is converted to a real directory as part of copying, resulting in (a) duplicate file contents in the destination directory, (b) non-identical contents between source and destination (which defeats the point of copying a directory verbatim).
Note: Symlinks to files are preserved, this is just about symlinks that target directories.
Using fs_extra = "=1.2.0"
with rustc 1.63.0-beta.3
on macOS 12.4, run the following:
use fs_extra::dir::CopyOptions;
use std::fs::{self, File};
use std::path::{Path, PathBuf};
fn main() {
let source_dir = PathBuf::from("source");
let destination_dir = PathBuf::from("destination");
if source_dir.exists() {
fs::remove_dir_all(&source_dir).unwrap();
}
if destination_dir.exists() {
fs::remove_dir_all(&destination_dir).unwrap();
}
let subdir = source_dir.join("subdir");
fs::create_dir_all(&subdir).unwrap();
File::create(&subdir.join("file.txt")).unwrap();
create_dir_symlink("subdir", &source_dir.join("symlink-to-subdir")).unwrap();
let options = &CopyOptions {
content_only: true,
..CopyOptions::default()
};
fs_extra::dir::copy(source_dir, destination_dir, options).unwrap();
}
#[cfg(target_family = "unix")]
fn create_dir_symlink<P: AsRef<Path>, Q: AsRef<Path>>(original: P, link: Q) -> std::io::Result<()> {
std::os::unix::fs::symlink(original.as_ref(), link.as_ref())
}
#[cfg(target_family = "windows")]
fn create_dir_symlink<P: AsRef<Path>, Q: AsRef<Path>>(original: P, link: Q) -> std::io::Result<()> {
std::os::windows::fs::symlink_dir(original.as_ref(), link.as_ref())
}
The contents of source/
and destination/
to be identical, and for any symlinks targetting directories to be preserved as symlinks.
The source/
and destination/
directories have different contents, with the symlinked directory now being a real directory containing duplicate files.
Source:
$ ls -alR source/
source/:
total 0
drwxr-xr-x 4 emorley staff 128 Jul 6 14:03 .
drwxr-xr-x 10 emorley staff 320 Jul 6 14:03 ..
drwxr-xr-x 3 emorley staff 96 Jul 6 14:03 subdir
lrwxr-xr-x 1 emorley staff 6 Jul 6 14:03 symlink-to-subdir -> subdir
source/subdir:
total 0
drwxr-xr-x 3 emorley staff 96 Jul 6 14:03 .
drwxr-xr-x 4 emorley staff 128 Jul 6 14:03 ..
-rw-r--r-- 1 emorley staff 0 Jul 6 14:03 file.txt
Destination:
$ ls -alR destination/
destination/:
total 0
drwxr-xr-x 4 emorley staff 128 Jul 6 14:03 .
drwxr-xr-x 10 emorley staff 320 Jul 6 14:03 ..
drwxr-xr-x 3 emorley staff 96 Jul 6 14:03 subdir
drwxr-xr-x 3 emorley staff 96 Jul 6 14:03 symlink-to-subdir
destination/subdir:
total 0
drwxr-xr-x 3 emorley staff 96 Jul 6 14:03 .
drwxr-xr-x 4 emorley staff 128 Jul 6 14:03 ..
-rw-r--r-- 1 emorley staff 0 Jul 6 14:03 file.txt
destination/symlink-to-subdir:
total 0
drwxr-xr-x 3 emorley staff 96 Jul 6 14:03 .
drwxr-xr-x 4 emorley staff 128 Jul 6 14:03 ..
-rw-r--r-- 1 emorley staff 0 Jul 6 14:03 file.txt
Inside /my/source/.git
there is a non-bare git repo with objects, history, etc... Then I want to copy the contents of that directory to a whole other directory at /my/dest/.git
, which is empty.
let source = PathBuf::from("/my/source/.git");
let dest = PathBuf::from("/my/dest/.git");
fs_extra::dir::create(&dest, true); // create with erase
let mut copts = fs_extra::dir::CopyOptions::new();
copts.overwrite = true;
copts.copy_inside = true;
let _ = fs_extra::dir::copy(&source, &dest, &copts);
After this I would expect to see the following inside /my/dest/.git
:
/
my/
dest/
.git/
refs/
objects/
...
But instead I see (note nested .git
inside .git
)
/
my/
dest/
.git/
.git/
refs/
objects/
...
This looks like the dest path should be the parent directory to where I want to copy the first directory? Documentation states:
Copies the directory contents from one place to another using recursive method.
I would assume this means that the contents from source directory become the contents of destination directory, while the directory itself is not copied anywhere?
I'm using fs_extra 1.1.0 and latest stable Rust.
Hey, what's the difference between SkipExisting and Overwrite? I mean doesn't overwrite selected mean don't skip existing and and not checked means skip existing?
fs_extra::dir::CopyOptions
I hope fs_extra's copy for directory can simulate cp
command. e.g., cp d1 d2
, if d2
does not exist it copy the whole directory d1
to d2
(in this ase d1
and d2
in same folder with same substructure); if d2
exists, copy d1
into d2
(as what has been implemented). The default behavior can be changed by modifying CopyOptions
.
Realize the function copy/move list items. Where item its will be path to folder or file.
A move should be instantaneous when files/folders are on the same filesystem.
But instead, a full copy/remove is done when doing move_*
Not sure if this is within the scope of this project, but I'll make an issue anyways
Hi I have some questions regarding this crate.
Add new option for copydeep
how much more files will be copy by levels of directory.
This options will be disabled for move function, cuz it's imposible.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.