Parameter validation failed using result_storage about aws

[Bladrak]

Could you print out the versions of the packages you're using? I'll try to reproduce this.

[eduherraiz]

My requirements is:

argparse==1.2.1
envtpl==0.4.1
boto==2.38.0
dateutils==0.6.6
numpy==1.9.2
pyremotecv==0.5.0
remotecv==1.0.2
opencv-engine==1.0.1
redis==2.4.12
thumbor==5.2.1
git+https://github.com/APSL/thumbor_aws.git
tc_aws==2.0.11

The difference is when put the image to storage use:

path

And when use the result_storage:

self.context.request.url

[Bladrak]

Thanks, I'll try to take a look at this whenever I get the chance.

[eduherraiz]

@Bladrak Consider i'm using a http_loader to get a external image via URL.

[Bladrak]

@eduherraiz I think the issue might come from requiring boto; tc_aws now uses botocre directly (not boto), hence there might be some conflicts here.

[Bladrak]

Could you do a pip freeze to list the installed dependencies? Here's mine:

backports-abc==0.4
backports.ssl-match-hostname==3.5.0.1
botocore==1.2.0
certifi==2015.11.20.1
colour==0.1.2
derpconf==0.7.3
docutils==0.12
futures==3.0.4
jmespath==0.7.1
libthumbor==1.3.1
numpy==1.10.4
opencv-engine==1.0.1
pexif==0.15
Pillow==2.9.0
pycrypto==2.6.1
pycurl==7.19.5.3
python-dateutil==2.4.2
python-magic==0.4.10
redis==2.10.5
shortuuid==0.4.3
singledispatch==3.4.0.3
six==1.10.0
statsd==3.2.1
tc-aws==2.0.12
tc-core==0.3.0
tc-shortener==0.2.2
thumbor==5.2.1
tornado==4.3
tornado-botocore==1.0.2

[eduherraiz]

I tried removing the boto package but is still failing.
The list of packages installed now is;

backports-abc==0.4
backports.ssl-match-hostname==3.5.0.1
botocore==1.2.0
certifi==2015.11.20.1
colour==0.1.2
dateutils==0.6.6
derpconf==0.7.3
docutils==0.12
envtpl==0.4.1
futures==3.0.4
itty==0.8.2
Jinja2==2.8
jmespath==0.7.1
libthumbor==1.3.1
MarkupSafe==0.23
numpy==1.9.2
opencv-engine==1.0.1
pexif==0.15
Pillow==2.9.0
pycrypto==2.6.1
pycurl==7.19.5.3
pyremotecv==0.5.0
pyres==1.2
pystache==0.5.4
python-dateutil==2.4.2
python-magic==0.4.10
pytz==2015.7
redis==2.4.12
remotecv==1.0.2
setproctitle==1.1.9
shortuuid==0.4.3
simplejson==3.8.1
singledispatch==3.4.0.3
six==1.10.0
statsd==3.2.1
tc-aws==2.0.12
tc-core==0.3.0
tc-shortener==0.2.2
thumbor==5.2.1
thumbor-aws==1
tornado==4.3
tornado-botocore==1.0.2
virtualenv==13.1.2
wheel==0.26.0

And for more information i'm trying to get the thumbor URL:
http://localhost/unsafe/100x100/avatars0.githubusercontent.com/u/469968

[Bladrak]

Ok, could you print your thumbor config as well?

[eduherraiz]

Of course!

THUMBOR_LOG_FORMAT = '%(asctime)s %(name)s:%(levelname)s %(message)s'
THUMBOR_LOG_DATE_FORMAT = '%Y-%m-%d %H:%M:%S'
MAX_WIDTH = 0
MAX_HEIGHT = 0
MIN_WIDTH = 1
MIN_HEIGHT = 1
ALLOWED_SOURCES =  []
QUALITY = 80
WEBP_QUALITY = None
AUTO_WEBP = False
MAX_AGE = 86400
MAX_AGE_TEMP_IMAGE = 0
RESPECT_ORIENTATION = False
IGNORE_SMART_ERRORS = False
PRESERVE_EXIF_INFO = False
ALLOW_ANIMATED_GIFS = True
USE_GIFSICLE_ENGINE = False
USE_BLACKLIST = False
LOADER = 'thumbor.loaders.http_loader'
STORAGE = 'tc_aws.storages.s3_storage'
RESULT_STORAGE = 'tc_aws.result_storages.s3_storage'
ENGINE = 'thumbor.engines.pil'
SECURITY_KEY = 'MY_SECURE_KEY'
ALLOW_UNSAFE_URL = True
ALLOW_OLD_URLS = True
AWS_ACCESS_KEY = '---'
AWS_SECRET_KEY = '---'
FILE_LOADER_ROOT_PATH = '/data/loader'
HTTP_LOADER_CONNECT_TIMEOUT = 5
HTTP_LOADER_REQUEST_TIMEOUT = 20
HTTP_LOADER_FOLLOW_REDIRECTS = True
HTTP_LOADER_MAX_REDIRECTS = 5
HTTP_LOADER_FORWARD_USER_AGENT = False
HTTP_LOADER_DEFAULT_USER_AGENT = 'Thumbor/5.0.3'
HTTP_LOADER_PROXY_HOST = None
HTTP_LOADER_PROXY_PORT = None
HTTP_LOADER_PROXY_USERNAME = None
HTTP_LOADER_PROXY_PASSWORD = None
STORAGE_EXPIRATION_SECONDS = 2592000
STORES_CRYPTO_KEY_FOR_EACH_IMAGE = False
FILE_STORAGE_ROOT_PATH = '/data/storage'
UPLOAD_MAX_SIZE = 0
UPLOAD_ENABLED = False
UPLOAD_PHOTO_STORAGE = 'tc_aws.storages.s3_storage'
UPLOAD_DELETE_ALLOWED = False
UPLOAD_PUT_ALLOWED = False
UPLOAD_DEFAULT_FILENAME = 'image'
MONGO_STORAGE_SERVER_HOST = 'mongo'
MONGO_STORAGE_SERVER_PORT = 27017
MONGO_STORAGE_SERVER_DB = 'thumbor'
MONGO_STORAGE_SERVER_COLLECTION = 'images'
REDIS_STORAGE_SERVER_HOST = 'redis'
REDIS_STORAGE_SERVER_PORT = 6379
REDIS_STORAGE_SERVER_DB = 0
REDIS_STORAGE_SERVER_PASSWORD = None
MEMCACHE_STORAGE_SERVERS = ['localhost:11211']
MIXED_STORAGE_FILE_STORAGE = 'thumbor.storages.no_storage'
MIXED_STORAGE_CRYPTO_STORAGE = 'thumbor.storages.no_storage'
MIXED_STORAGE_DETECTOR_STORAGE = 'thumbor.storages.no_storage'
META_CALLBACK_NAME = None
DETECTORS = ['thumbor.detectors.feature_detector', 'thumbor.detectors.face_detector']
FACE_DETECTOR_CASCADE_FILE = 'haarcascade_frontalface_alt.xml'
OPTIMIZERS = []
JPEGTRAN_PATH = '/usr/bin/jpegtran'
PROGRESSIVE_JPEG = True
FILTERS = ['thumbor.filters.brightness', 'thumbor.filters.contrast', 'thumbor.filters.rgb', 'thumbor.filters.round_corner', 'thumbor.filters.quality', 'thumbor.filters.noise', 'thumbor.filters.watermark', 'thumbor.filters.equalize', 'thumbor.filters.fill', 'thumbor.filters.sharpen', 'thumbor.filters.strip_icc', 'thumbor.filters.frame', 'thumbor.filters.grayscale', 'thumbor.filters.rotate', 'thumbor.filters.format', 'thumbor.filters.max_bytes', 'thumbor.filters.convolution', 'thumbor.filters.blur', 'thumbor.filters.extract_focal', 'thumbor.filters.no_upscale']
RESULT_STORAGE_EXPIRATION_SECONDS = 0
RESULT_STORAGE_FILE_STORAGE_ROOT_PATH = '/data/result_storage'
RESULT_STORAGE_STORES_UNSAFE = True
REDIS_QUEUE_SERVER_HOST = 'redis'
REDIS_QUEUE_SERVER_PORT = 6379
REDIS_QUEUE_SERVER_DB = 0
REDIS_QUEUE_SERVER_PASSWORD = None
SQS_QUEUE_KEY_ID = None
SQS_QUEUE_KEY_SECRET = None
SQS_QUEUE_REGION = 'us-east-1'
USE_CUSTOM_ERROR_HANDLING = False
ERROR_HANDLER_MODULE = 'thumbor.error_handlers.sentry'
ERROR_FILE_LOGGER = None
ERROR_FILE_NAME_USE_CONTEXT = False
SENTRY_DSN_URL = ''
TC_AWS_REGION = 'eu-west-1' # AWS Region
TC_AWS_STORAGE_BUCKET = 'thumbor-travis-test' # S3 bucket for Storage
TC_AWS_STORAGE_ROOT_PATH = '/storage' # S3 path prefix for Storage bucket
TC_AWS_LOADER_BUCKET = '' #S3 bucket for loader
TC_AWS_LOADER_ROOT_PATH = '' # S3 path prefix for Loader bucket
TC_AWS_RESULT_STORAGE_BUCKET = 'thumbor-travis-test' # S3 bucket for result Storage
TC_AWS_RESULT_STORAGE_ROOT_PATH = '/result_storage' # S3 path prefix for Result storage bucket
TC_AWS_STORAGE_SSE = 'False'
TC_AWS_STORAGE_RRS = 'False'
TC_AWS_ENABLE_HTTP_LOADER = 'False'
TC_AWS_ALLOWED_BUCKETS = 'False' # List of allowed bucket to be requested
TC_AWS_STORE_METADATA = 'False' # Store result with metadata (for instance content-type)

I tried to change TC_AWS_ENABLE_HTTP_LOADER to True, but didn't solve the problem.

[Bladrak]

I think you may have an issue with your configuration, when you set the value False, you put it between quotes, maybe try removing them?
This doesn't mean that there's no issue, given that TC_AWS_STORE_METADATA should otherwise work properly...

[Bladrak]

Commit cf418fa should fix the issue. I've published version 2.1.1 with this fix.

[Bladrak]

@eduherraiz feel free to close this issue once you tested the fix :)

[eduherraiz]

Thank you!! I will test it soon.

[eduherraiz]

The fix works fine. Thank you @Bladrak !!