Coder Social home page Coder Social logo

ossutil's Introduction

Alibaba Cloud OSSUTIL

GitHub Version Build Status Coverage Status

About

  • This tool is developed with Go and built on the official GO SDK of OSS Alibaba Cloud Object Storage Service.
  • OSS is a cloud storage service provided by Alibaba Cloud, featuring massive capacity, security, low cost, and high reliability.
  • OSS can store any type of files. It applies to various websites, development enterprises and developers.
  • This tool aims to provide a convenient-to-use command line for users to manage data in OSS.
  • The current version provides to list and delete multipart upload tasks.
  • The current version does not support bucket management. The feature will be available in future versions.

Version

  • Current version: v1.7.19

Run environment

  • Linux
  • Windows
  • Mac

Dependent libraries

  • goopt (github.com/droundy/goopt)
  • configparser (github.com/alyu/configparser)
  • leveldb (github.com/syndtr/goleveldb/leveldb)
  • oss (github.com/aliyun/aliyun-oss-go-sdk/oss)
  • gopkg.in/check.v1 (gopkg.in/check.v1)

Quick use

Get the command list

    ./ossutil
    or ./ossutil help

View the help documentation for a command

    ./ossutil help cmd 

Configure OSSUTIL

    ./ossutil config

List buckets

    ./ossutil ls
    or ./ossutil ls oss://

List objects and multipart upload tasks

    ./ossutil ls -a
    or ./ossutil ls oss:// -a

Upload a file

    ./ossutil cp localfile oss://bucket

Others

You can use ./ossutil help cmd to view the help documentation for the command you want to use.

Notes

Run OSSUTIL

  • First, configure your Go project directory.
  • Use go get to get the library that ossutil depends on.
  • Run go get github.com/aliyun/ossutil.
  • Enter the src directory under the Go project directory and build to generate the OSSUTIL tool. For example, on Linux, you can run go build github.com/aliyun/ossutil/ossutil.go.
  • Refer to the example above to run the OSSUTIL tool.

Test

  • Enter the src directory under the Go project directory and modify the endpoint, AccessKeyId, AccessKeySecret and STSToken configuration items in the github.com/aliyun/ossutil/lib/command_test.go.
  • Run go test under the lib directory.

Contact us

Author

License

ossutil's People

Contributors

abo969 avatar baiyubin avatar baiyubin2020 avatar dengwu12 avatar dependabot[bot] avatar guoxing-z avatar hangzws avatar huiguangjun avatar jacksontian avatar kkuai avatar mars-coder avatar yangzong18 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ossutil's Issues

time is too large

root@has:~# date "+%F %T"
2017-08-03 22:50:21

root@has:~# curl 'http://api.k780.com:88/?app=life.time&appkey=10003&sign=b59bc3ef6191eb9f747dd4e83c99f2a4&format=json'
{"success":"1","result":{"timestamp":"1501771932","datetime_1":"2017-08-03 22:52:12","datetime_2":"2

root@has:~# ossutil ls
Error: oss: service returned error: StatusCode=403, ErrorCode=RequestTimeTooSkewed, ErrorMessage=The difference between the request time and the current time is too large., RequestId=598338A88CEE3190462CAA94!

root@has:~#

不能上传超过4GB的文件

通过 ossutil 1.4(Linux x86_64版,Ubuntu 17.10 64位) 版无法上传超过4GB的文件。同样的环境,使用 osscmd 刚正常。错误信息如下:

Total num: 1, size: 4,639,948,800. Dealed num: 0, OK size: 0, Progress: 0%
Total num: 1, size: 4,639,948,800. Dealed num: 0, Transfer size: 0. When error happens.
Error: Put http://XXXX.oss-cn-shenzhen.aliyuncs.com/V834394-01-7.3.iso?partNumber=3&uploadId=XXXXDB06B4D64B4B9DD9BA359FD8XXXX: EOF, File=/PATH/OracleLinux/V834394-01-7.3.iso!

版本错误

官网和go get下的版本都是1.0.0版本,为什么文档里面都说当前版本是:1.2.0

下载时会产生.temp文件,能否增加一个参数来定义这个文件的产生路径?

比如--temp_dir=TEMP_DIR

原因:
我的场景是这样,使用ossutil下载oss的大量文件到本地目录,这个目录由logstash的file插件监控着,并且logstash配置为不允许读取.temp结尾的文件,ossutil产生.temp文件时,logstash是会扫描到并记录这个文件路径和inode信息以及偏移量,当ossutil将.temp后缀改名后,文件inode是不会改变的,logstash再读取这个文件时,偏移量就不是从0开始的,所以导致了丢失数据

只能查看第一个空间的文件,其他空间都是403返回

Error: oss: service returned error: StatusCode=403, ErrorCode=AccessDenied, ErrorMessage=The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint., RequestId=5B8E81775CC26B5BBA097D54, Bucket=test!

静默输出

你好,我们在CI环境中使用 ossutil 上传文件,上传过程的日志可以有办法隐藏吗?

我们只需要知道是否上传成功就可以了

ossutil copy to local_dir adds all prefix

OSSUTIL VERSION: 1.4.0

When using the copy command to recursively import a specific directory, it adds full prefix to local_dir:

Example:
I have a directory with images files oss://bucket/dir1/dir2/dir3/*.png
Using:
ossutil cp oss://bucket/dir1/dir2/dir3 /home/ubuntu/dir3 -r -u -f
will copy all the png files in oss://bucket/dir1/dir2/dir3 to /home/ubuntu/dir3/dir1/dir2/dir3/*.png
when I would like to copy to /home/ubuntu/dir3/*.png

ossutil不能在alpine镜像内运行

~ # ls -ltr
total 9820
-rwxr-xr-x 1 root root 10053808 Aug 14 08:32 ossutil64
~ # cat /etc/issue
Welcome to Alpine Linux 3.10
Kernel \r on an \m (\l)

~ # bash ./ossutil64
./ossutil64: ./ossutil64: cannot execute binary file

Request Entity Too Large

osscmd 可以设置retry count,
而ossutil传输大文件不方便。
./ossutil64 cp传输大文件不方便。
retry count:9:upload file:data/StereoRecons-5d6fbf86d4c2b807c939fa0f/tutorial_out/reconstruction_sequential/sfm_data.json.

提点小需求

能不能增加一个传输的时候排除某个文件或者文件夹选项
或者传输文件的时候支持通配符或者正则

请在 github release 上发布构建后的版本

https://github.com/aliyun/ossutil/releases
目前这里只有源码,而构建后的在 https://help.aliyun.com/document_detail/50452.html 这里。

文档那个页面上的链接,在国外访问非常不顺畅,在 Travis CI 需要构建后部署的场景无法顺利的下载下来,像这样。。

$ curl --location --retry 5 --output ossutil 'http://docs-aliyun.cn-hangzhou.oss.aliyun-inc.com/assets/attach/50452/intl_zh/1488523315888/ossutil'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  8 8150k    8  668k    0     0   2371      0  0:58:39  0:04:48  0:53:51  8344
curl: (18) transfer closed with 7660849 bytes remaining to read
The command "curl --location --retry 5 --output ossutil 'http://docs-aliyun.cn-hangzhou.oss.aliyun-inc.com/assets/attach/50452/intl_zh/1488523315888/ossutil'" failed and exited with 18 during .
Your build has been stopped.

如果在 github release 上提供一个构件好的版本,在国外访问会顺畅很多

cp 时可不可以禁止生成 report file

access_key & access_secret 是写在环境变量里的,ossutil -i $ACCESS_KEY -k $ACCESS_SECRET cp xxx 这样;
生成 report file 会把敏感信息也直接保存在里面,非常不安全,可不可以禁止生成 report file?

Can `ossutil cp` support exclude dir?

I met problem when I tried to upload local folder(a git repo) to OSS bucket.
I just want to upload the normal files only, but couldn't find an option to exclude the '.git' dir.

当object name是 `oss://mingdi-hz-2/y9n/g/%E9%98%B4%E9%98%B3%E5%B8%88-%E9%A3%9F%E6%A2%A6%E8%B2%98.jpg`时,显示ls命令显示object name错误

当object name是 oss://mingdi-hz-2/y9n/g/%E9%98%B4%E9%98%B3%E5%B8%88-%E9%A3%9F%E6%A2%A6%E8%B2%98.jpg时,显示ls命令显示如下:
2016-12-16 14:54:18 +0800 CST 11903 75209F561F61BADFA0F2B112D61BE7EE oss://mingdi-hz-2/y9n/g/%!E(MISSING)9%B4%!E(MISSING)9%B3%!E(MISSING)5%!B(MISSING)8%!-(MISSING)%!E(MISSING)9%!A(MISSING)3%!F(MISSING)%!E(MISSING)6%!A(MISSING)2%!A(MISSING)6%!E(MISSING)8%!B(MISSING)2%!j(MISSING)pg

ubuntu软件包

我想把ossutil打包,放到ppa上,这样在添加了ppa源的ubuntu系统中安装、更新、uninstall etc 都比较方便,不知道 aliyun有没有开 自己的ppa,或者可以让用户自己弄。

./ossutil ls oss:// 报错

./ossutil ls oss://
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x6463fb]

goroutine 1 [running]:
github.com/aliyun/aliyun-oss-go-sdk/oss.Conn.doRequest(0xc4201760f0, 0xc4200118c0, 0xc4200118f0, 0x7b59ba, 0x3, 0x0, 0xc420016e4e, 0x1, 0x0, 0x0, ...)
/home/zubo.yzb/go/src/github.com/aliyun/aliyun-oss-go-sdk/oss/conn.go:192 +0x17b
github.com/aliyun/aliyun-oss-go-sdk/oss.Conn.Do(0xc4201760f0, 0xc4200118c0, 0xc4200118f0, 0x7b59ba, 0x3, 0x0, 0x0, 0x0, 0x0, 0xc4200119b0, ...)
/home/zubo.yzb/go/src/github.com/aliyun/aliyun-oss-go-sdk/oss/conn.go:59 +0x276
github.com/aliyun/aliyun-oss-go-sdk/oss.Client.do(0xc4201760f0, 0xc4200d2fa0, 0x7b59ba, 0x3, 0x0, 0x0, 0xc4200119b0, 0x0, 0x0, 0x0, ...)
/home/zubo.yzb/go/src/github.com/aliyun/aliyun-oss-go-sdk/oss/client.go:800 +0xea
github.com/aliyun/aliyun-oss-go-sdk/oss.Client.ListBuckets(0xc4201760f0, 0xc4200d2fa0, 0xc42004f8a0, 0x2, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/home/zubo.yzb/go/src/github.com/aliyun/aliyun-oss-go-sdk/oss/client.go:143 +0x12c
github.com/aliyun/ossutil/lib.(*ListCommand).ossListBucketsRetry(0x9b62e0, 0xc420041cb0, 0xc42004f8a0, 0x2, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/home/zubo.yzb/go/src/github.com/aliyun/ossutil/lib/ls.go:461 +0x11f
github.com/aliyun/ossutil/lib.(*ListCommand).listBuckets(0x9b62e0, 0x0, 0x0, 0x0, 0x7ffd1db8d6fc)
/home/zubo.yzb/go/src/github.com/aliyun/ossutil/lib/ls.go:411 +0x67d
github.com/aliyun/ossutil/lib.(*ListCommand).RunCommand(0x9b62e0, 0x79cde0, 0x9b62e0)
/home/zubo.yzb/go/src/github.com/aliyun/ossutil/lib/ls.go:380 +0x156
github.com/aliyun/ossutil/lib.(*CommandManager).RunCommand(0xc42004fe18, 0x7ffd1db8d6f9, 0x2, 0xc42005a590, 0x1, 0x3, 0xc420011380, 0x29, 0xc4200185d0, 0x23)
/home/zubo.yzb/go/src/github.com/aliyun/ossutil/lib/command_manager.go:89 +0x12b
github.com/aliyun/ossutil/lib.RunCommand(0xc42005a580, 0x2, 0x4, 0xc420011380, 0x0, 0x0, 0x7d3da8)
/home/zubo.yzb/go/src/github.com/aliyun/ossutil/lib/command_manager.go:63 +0x106
github.com/aliyun/ossutil/lib.ParseAndRunCommand(0xc420014ba0, 0x14)
/home/zubo.yzb/go/src/github.com/aliyun/ossutil/lib/command_manager.go:27 +0x101
main.main()
/home/zubo.yzb/go/src/github.com/aliyun/ossutil/ossutil.go:12 +0x26

Unable to include files in ls

Doesn't matter how the ossutil command with ls switch is used it doesn't work.

./ossutilmac64 ls oss://macos-backup --include=*.jpg
Error: invalid usage of "ls" command, reason: the command does not support option: "include", please try "help ls" for more information!

./ossutilmac64 ls oss://macos-backup --include= *.jpg
Error: invalid usage of "ls" command, reason: the command needs at most 1 argument, please try "help ls" for more information!

./ossutilmac64 ls oss://macos-backup --include = *.jpg
Error: invalid usage of "ls" command, reason: the command needs at most 1 argument, please try "help ls" for more information!

./ossutilmac64 ls oss://macos-backup --include =*.jpg
Error: invalid usage of "ls" command, reason: the command does not support option: "include", please try "help ls" for more information!

./ossutilmac64 ls oss://macos-backup --include="*.jpg"
Error: invalid usage of "ls" command, reason: the command does not support option: "include", please try "help ls" for more information!

./ossutilmac64 ls oss://macos-backup --include= "*.jpg"
Error: invalid usage of "ls" command, reason: the command needs at most 1 argument, please try "help ls" for more information!

/ossutilmac64 ls oss://macos-backup --include = "*.jpg"
Error: invalid usage of "ls" command, reason: the command needs at most 1 argument, please try "help ls" for more information!

Fetch credentials from environment variable

Hi, would it be possible to fetch credentials from environment variables ALICLOUD_ACCESS_KEY and ALICLOUD_SECRET_KEY.

That would make it easy to run from within Docker.

Thanks !

Remove large number of files

When trying to ossutil rm a large number of files, I get the following error:
Total 1002 objects. Removed 100 objects. when error happens.
Error: oss: service returned error: StatusCode=400, ErrorCode=MalformedXML, ErrorMessage="The XML you provided was not well-formed or did not validate against our published schema.", RequestId=5D3F0D47C1E6C9CDCBBD3B8F

What is the suggested way to delete a large number of objects ?
Thanks

Version still reads 1.0.0

Possible cause : at line 87 lib/const.go
Version string = "1.0.0"
This misleads the update checker.

ossutil cp 路经检测不正确

ossutil cp -r oss://mycard/mobile oss://mycard/mobile3
Error: "oss://mycard/mobile3" include "oss://mycard/mobile", it's not allowed, recursivlly copy should be avoided!

显然mobile3和mobile是两个不同的目录,并没有包含关系

树莓派上下载文件panic 空指针

shellus@raspberrypi ~ $ ossutil cp "oss://wawa-disk/Website Backup/shuoxingba/db-2016-10-05.sql.tgz" ./
Total num: 1, size: 229,522,629. Dealed num: 0, OK size: 0, Progress: 0%

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x4 pc=0x11580]

goroutine 1 [running]:
sync/atomic.addUint64(0x49903c, 0x271668, 0x0, 0x76f6e708, 0x0)
C:/Go/src/sync/atomic/64bit_arm.go:31 +0x4c
github.com/aliyun/ossutil/lib.(*CPMonitor).updateTransferSize(0x49901c, 0x271668, 0x0)
C:/Users/shellus/go/src/github.com/aliyun/ossutil/lib/monitor.go:428 +0x34
github.com/aliyun/ossutil/lib.(*OssProgressListener).ProgressChanged(0x10868440, 0x1093dd40)
C:/Users/shellus/go/src/github.com/aliyun/ossutil/lib/cp.go:88 +0x6c
github.com/aliyun/aliyun-oss-go-sdk/oss.publishProgress(0x470ca0, 0x10868440, 0x1093dd40)
C:/Users/shellus/go/src/github.com/aliyun/aliyun-oss-go-sdk/oss/progress.go:43 +0x40
github.com/aliyun/aliyun-oss-go-sdk/oss.Bucket.downloadFileWithCp(0x1080a1b0, 0x10800760, 0x7e9018b1, 0x9, 0x7e9018bb, 0x2f, 0x10746ec0, 0x31, 0x271668, 0x0, ...)
C:/Users/shellus/go/src/github.com/aliyun/aliyun-oss-go-sdk/oss/download.go:474 +0x608
github.com/aliyun/aliyun-oss-go-sdk/oss.Bucket.DownloadFile(0x1080a1b0, 0x10800760, 0x7e9018b1, 0x9, 0x7e9018bb, 0x2f, 0x10746ec0, 0x31, 0x271668, 0x0, ...)
C:/Users/shellus/go/src/github.com/aliyun/aliyun-oss-go-sdk/oss/download.go:42 +0x1f8
github.com/aliyun/ossutil/lib.(*CopyCommand).ossResumeDownloadRetry(0x498f48, 0x10800780, 0x7e9018bb, 0x2f, 0x10746ec0, 0x31, 0xdae3cc5, 0x0, 0x271668, 0x0, ...)
C:/Users/shellus/go/src/github.com/aliyun/ossutil/lib/cp.go:1831 +0xb8
github.com/aliyun/ossutil/lib.(*CopyCommand).downloadSingleFile(0x498f48, 0x10800780, 0x7e9018bb, 0x2f, 0xffffffff, 0xffffffff, 0xd0ff7ce3, 0xe, 0x195f6c23, 0x49aa38, ...)
C:/Users/shellus/go/src/github.com/aliyun/ossutil/lib/cp.go:1776 +0x67c
github.com/aliyun/ossutil/lib.(*CopyCommand).downloadSingleFileWithReport(0x498f48, 0x10800780, 0x7e9018bb, 0x2f, 0xffffffff, 0xffffffff, 0xd0ff7ce3, 0xe, 0x195f6c23, 0x49aa38, ...)
C:/Users/shellus/go/src/github.com/aliyun/ossutil/lib/cp.go:1717 +0x40
github.com/aliyun/ossutil/lib.(*CopyCommand).downloadFiles(0x498f48, 0x7e9018ab, 0x3f, 0x7e9018b1, 0x9, 0x7e9018bb, 0x2f, 0x7e9018eb, 0x2, 0xf, ...)
C:/Users/shellus/go/src/github.com/aliyun/ossutil/lib/cp.go:1680 +0x1dc
github.com/aliyun/ossutil/lib.(*CopyCommand).RunCommand(0x498f48, 0x0, 0x0)
C:/Users/shellus/go/src/github.com/aliyun/ossutil/lib/cp.go:1077 +0x6e4
github.com/aliyun/ossutil/lib.(*CommandManager).RunCommand(0x1072ff28, 0x7e9018a8, 0x2, 0x1070f8c8, 0x2, 0x3, 0x1070f8a0, 0xc4, 0x3204e7, 0xf7)
C:/Users/shellus/go/src/github.com/aliyun/ossutil/lib/command_manager.go:89 +0xe0
github.com/aliyun/ossutil/lib.RunCommand(0x1070f8c0, 0x3, 0x4, 0x1070f8a0, 0x0, 0x0, 0x1)
C:/Users/shellus/go/src/github.com/aliyun/ossutil/lib/command_manager.go:63 +0xbc
github.com/aliyun/ossutil/lib.ParseAndRunCommand(0x0, 0x1070f640)
C:/Users/shellus/go/src/github.com/aliyun/ossutil/lib/command_manager.go:27 +0xec
main.main()
C:/Users/shellus/go/src/github.com/aliyun/ossutil/ossutil.go:11 +0x14

文件是在windows7上使用go get 下载源码,然后使用GOOS=linux GOARCH=arm go build来编译到的。
然后在树莓派上运行。ls 之类的命令正常。但是cp时,空指针异常。

ossutil上传文件时,是否可以保留文件的修改时间

举例如下:
a目录下有
a-2018-06-17.bak (2018-06-17创建)
a-2018-06-18.bak (2018-06-18创建)
a-2018-06-19.bak (2018-06-19创建)

当我在2018-06-25号使用ossutil cp命令将这些文件上传到oss上时,文件的更新时间都变成上传时刻的时间2018-06-25,请问ossutil是否有选项,在上传或下载时可以保留原来文件的创建时间,以方便在批量备份时实现oss的生命周期控制。

issue 19的fix需要重新fix

issue 19 的fix如下

Pull Request: #23
Update atomic.AddInt64() to RWMutex to avoid atomic.AddInt64() crash
Local mutex不能避免并发访问。需要根据go官方advise重新fix

Example of using ossutil with ECS RAM Role

I don't see any help or example of using ossutil on an ECS instance that has a RAM role attached to.

When I try running ossutil64 on such an instance where the attached RAM role has AliyunOSSFullAccess permission, I still see the following error message :-

.ossutilconfig: no such file or directory

Can an example be added to the documentation stating how to use ossutil on ECS instance with valid RAM role ?

似乎无法正常完成DNS解析?

运行ossutil ls报错:

Error: Get http:/?marker=&prefix=: dial tcp: lookup oss-cn-beijing.aliyuncs.com on 10.10.0.1:53: read udp 10.10.5.76:52088->10.10.0.1:53: i/o timeout!

环境CentOS 7,iptables全ACCEPT,selinux已关。

Same command with different responses

wangwen@ZhangLab:~/project/ChIP/inhouse/20181017$ ossutil64 cp oss://pangoo-bj/customer-CLefpCrT/ /mnt/Storage/home/wangwen/project/ChIP/inhouse/20181017/ -r –f --jobs 3 --parallel 2
Error: invalid url: /mnt/Storage/home/wangwen/project/ChIP/inhouse/20181017/, multiple source url in download operation!
wangwen@ZhangLab:~/project/ChIP/inhouse/20181017$ ossutil64 cp oss://pangoo-bj/customer-CLefpCrT/ /mnt/Storage/home/wangwen/project/ChIP/inhouse/20181017/ -r -f --jobs 3 --parallel 2
Total num: 51, size: 31,094,361,103. Dealed num: 4(download 4 objects), OK size: 130, Progress: 0%^C

Publishing binary pkg

Hi, I'm mosuke5. I'm using this tool usually.
By the way why don't you publish binary pkg about this tool?
I think you should publish it.

Error messages with stderr

Hi,
Would it please be possible to direct error messages to stderr, and not stdout ?

Such as:
b'Error: oss: service returned error: StatusCode=404, ErrorCode=NoSuchBucket, ErrorMessage=The specified bucket does not exist., RequestId=5B4C9536C7E7649D66BC1CE0, Bucket=notexisting-123!\n'

Published binary not working in docker ubuntu 18.04 image

The binary release from here doesn't work in docker ubuntu 18.04 image, when building this dockerfile:

FROM ubuntu:18.04

RUN apt-get update && apt-get install -y curl
RUN curl -o ossutil http://gosspublic.alicdn.com/ossutil/1.4.2/ossutil64 && chmod +x ossutil
RUN ./ossutil -v

Output:

Step 4/4 : RUN ./ossutil -v
 ---> Running in 872483d5f935
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x6b4981]

goroutine 1 [running]:
github.com/aliyun/ossutil/lib.DecideConfigFile(0x0, 0x0, 0x7a8017, 0x7)
        /Users/fengyu/go/src/github.com/aliyun/ossutil/lib/config_helper.go:57 +0x51
The command '/bin/sh -c ./ossutil -v' returned a non-zero code: 2

But it works fine if I build it from scratch:

FROM ubuntu:18.04

WORKDIR /root

RUN apt-get update && apt-get install -y curl git
RUN curl -O https://dl.google.com/go/go1.10.3.linux-amd64.tar.gz \
 && tar xvf go1.10.3.linux-amd64.tar.gz

ENV GOROOT=/root/go
ENV GOPATH=/root/work
ENV PATH=$PATH:$GOROOT/bin:$GOPATH/bin

RUN go get github.com/aliyun/ossutil \
 && cd $GOPATH/src \
 && go build github.com/aliyun/ossutil/ossutil.go

RUN ossutil -v

Output:

Step 9/9 : RUN ossutil -v
 ---> Running in 24c75ef99721
ossutil version: 1.4.2

Error: mkdir .ossutil_checkpoint: permission denied!

执行下面这操作时提示错误:

ali@ljp:/var/backups$ ossutil64 cp /var/backups/alihome.tgz oss://alicode/`date +%Y%m%d`/ -f
Error: mkdir .ossutil_checkpoint: permission denied!

上传文件要使用到mkdir?
在ali用户的home目录下执行这个命令是正常的。

上传速度显示错误

ossutile version:1.6.3

cp 上传进度错误
QQ截图20190624174633
size,progree,speed都显示错误,speed还会出现负值。

咋文件不支持磁盘路径呢?

比如windows 磁盘的路径\.\PhysicalDrive1 不支持上传。。
ossutil32.exe cp \.\PhysicalDrive1 oss://evidence-image-shanghai/hello
并且我看了sdk。。这种路径skd也不支持。。只有append的方法适合这种情形,然后这种方法还限制最大上传5个G。。没考虑过支持上传整个磁盘么?。。取证界可是刚需。。现在我该咋搞呢,貌似只能自己修改源码了?

拷贝时,某些文件错误会导致复制停止

ossutil cp -r -j 10 -u oss://zblz/imgs/ ./

Error occurs, message: oss: service returned error: StatusCode=404, ErrorCode=NoSuchStyle, ErrorMessage=No Such Image Style., RequestId=5D104B9408F4BE32886E0689, Bucket=zblz, Object=imgs/1502568338814.jpg@!300x. See more information in file: ossutil_output/ossutil_report_20190624_120316.report
Scanned 54300 objects. Dealed num: 32453(download 645 objects, skip 31808 objects), OK size: 20,498,633,633(transfer: 354,494,013, skip: 20,144,139,620). When error happens.
Error: oss: service returned error: StatusCode=403, ErrorCode=AccessDenied, ErrorMessage=You do not have read permission on this object., RequestId=5D104BA21E7FB62C9F7EDCB5, Bucket=zblz, Object=imgs/1502890551777.jpg!

这些异常文件不能复制就算了,希望能有个参数可以直接忽略这些错误,跳过文件后继续复制

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.