Comments (9)
Please try new version from branch assumeRole_debug
, it's should works (It's working for me).
from s3sync.
Changes merged to master.
from s3sync.
Hello.
Thanks for feedback;).
Are you checked the s3sync with this setup? AWS SDK supports AWS_PROFILE
and should read credentials from it.
So you try to run s3sync and report a result.
Please use latest version (2.15). I was build create this release few min ago. Prev release was not contains latest changes for some auth methods.
from s3sync.
@larrabee thanks for the quick reply. Sorry for not mentioning this in my initial post - of course I did test the setup with s3sync, but it did not work.
I now used the new release, but that also fails:
INFO[0000] Starting sync
DEBU[0006] S3 listing failed with error: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
DEBU[0006] Pipeline step: ListSource finished
DEBU[0006] Recv pipeline err: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
ERRO[0006] Sync error: pipeline step: 0 (ListSource) failed with error: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors, terminating
DEBU[0006] Pipeline step: ACLUpdater finished
DEBU[0006] Pipeline step: LoadObjData finished
DEBU[0006] Pipeline step: UploadObj finished
DEBU[0006] Pipeline step: Terminator finished
DEBU[0006] All pipeline steps finished
DEBU[0006] Pipeline terminated
INFO[0006] 0 ListSource: Input: 0; Output: 0 (0 obj/sec); Errors: 1
INFO[0006] 1 LoadObjData: Input: 0; Output: 0 (0 obj/sec); Errors: 0
INFO[0006] 2 ACLUpdater: Input: 0; Output: 0 (0 obj/sec); Errors: 0
INFO[0006] 3 UploadObj: Input: 0; Output: 0 (0 obj/sec); Errors: 0
INFO[0006] 4 Terminator: Input: 0; Output: 0 (0 obj/sec); Errors: 0
INFO[0006] Duration: 6.27037978s
ERRO[0006] Sync Failed
from s3sync.
Hello again;)
Are you using AWS_PROFILE
correctly?
AWS_PROFILE
should contain account name from credentials file. File location can be specified by variable AWS_SHARED_CREDENTIALS_FILE
.
For example:
export AWS_PROFILE="default"
export AWS_SHARED_CREDENTIALS_FILE="/home/test/aws_creds"
>> cat /home/test/aws_creds
[default]
aws_access_key_id = KEY
aws_secret_access_key = SECRET
Is AWS regions correct? By default it's connecting to us-east-1
region. You can set region by args --sr
and --tr
.
from s3sync.
@larrabee yes, I did all that:
$ env | grep AWS
AWS_PROFILE=my.account.test
AWS_SHARED_CREDENTIALS_FILE=/home/myuser/.aws/credentials
The credentials file (/home/myuser/.aws/credentials
) looks like this:
[default]
aws_access_key_id = REDACTED
aws_secret_access_key = REDACTED
The difference in my case is that I want to "assume a role". The credentials file contains my access credentials, but then a file ~/.aws/config
exists containing:
[profile my.account.test]
role_arn = arn:aws:iam::1234567890:role/myRoleInTheOtherAccount
source_profile = default
region = eu-central-1
So I assume the role of myRoleInTheOtherAccount
which actually allows access to the S3 bucket, not the role / account I have when using the aws_access_key_id/aws_secret_access_key.
In short, the aws_secret_access_key authenticate me as a human, the role is the role I have related to the particular account. This is quite common in larger / enterprise usage of AWS accounts.
from s3sync.
I'm commit changes to branch assumeRole_debug
.
Can you build and test it?
from s3sync.
Hey @larrabee thanks for really digging into this! Really appreciate it.
Are we talking about this commit: ded3296 ? What is that intended to do as the line ded3296#diff-44bbcc9d983da65f32aa64529eb190e2R50 is commented out ...
I built s3sync in branch assumeRole_debug with the commit ... but nothing has changed unfortunately.
from s3sync.
Yeah! It works splendidly.
from s3sync.
Related Issues (20)
- Out of Memory and process killed after skip many files. HOT 1
- adding --delete HOT 1
- No docker image for arm
- No sync happend - no error HOT 8
- Preserve file permissions
- Manual retry logic is needed for "read: connection reset by peer"
- Feat: Automatically read the default profile and keys from .aws config directory HOT 2
- Nothing is saved to TARGET HOT 1
- Add --filter-not-empty option
- Journal HOT 1
- Gzipped files in s3 buckets get unzipped
- content downloaded in gzip format HOT 2
- Sync error: pipeline step: 0 (ListSource) HOT 3
- token rotation iam roles.
- [Feature] Exclude directory when sync bucket HOT 1
- Panic syncing from S3 to local FS HOT 1
- README: offers streaming or limited to files fitting in RAM? HOT 1
- s3.ListObjects maybe cause an infinite loop HOT 1
- [Feature] Support Azure blobstore
- Retransmission with minimal changes: performance?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from s3sync.