Coder Social home page Coder Social logo

silverstripe-cloudassets's People

Contributors

gitter-badger avatar gordonbanderson avatar markguinn avatar obj63mc avatar scrutinizer-auto-fixer avatar simonwelsh avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

nathanbrauer souldigital gitter-badger helpfulrobot gordonbanderson useallfive timezoneone obj63mc smarcet

silverstripe-cloudassets's Issues

'Sync Files' removes cloud assets

Hi, I've got this working using Heroku and the S3 driver. The only issue I've noticed is that clicking 'Sync Files' is okay while the files are still on disk, but after a deploy/restart, clicking that will remove all cloud assets from the file browser. The button doesn't really make sense with this type of setup, so it would probably make sense to hide it, but I'm wondering if it's just a problem with my set up?

CMS upload fails for images when LocalCopy=false

The problem is that UploadField creates an Image object, which is then wrapped to CloudImage, but the initial reference is still an Image. This is a weakness in the architecture that I don't know how to overcome at this stage. The following code then fails in UploadField.php (currently line 911 - the call to SetRatioSize returns null because the width and height are not set):

    /**
     * @param File $file
     * @return string
     */
    protected function getThumbnailURLForFile(File $file) {
        if ($file->exists() && file_exists(Director::baseFolder() . '/' . $file->getFilename())) {
            $width = $this->getPreviewMaxWidth();
            $height = $this->getPreviewMaxHeight();
            if ($file->hasMethod('getThumbnail')) {
                return $file->getThumbnail($width, $height)->getURL();
            } elseif ($file->hasMethod('getThumbnailURL')) {
                return $file->getThumbnailURL($width, $height);
            } elseif ($file->hasMethod('SetRatioSize')) {
                return $file->SetRatioSize($width, $height)->getURL();
            } else {
                return $file->Icon();
            }
        }
        return false;
    }

Options:

  1. Implement getThumbnail in CloudFileExtension. This would cover the above case but I'm uncomfortable with it b/c it feels hack-y.
  2. Set $this->width and $this->height on the source image object during the wrapping process. This would cause SetRatioSize to at minimum return a CloudImageMissing placeholder. Again, not ideal.
  3. Do the upload offline - either by requiring the QueuedJobs module or an additional cron job. I really don't want to have to do this.
  4. Try to extend UploadField.

Disappearing Images

I'm not sure if anybody else has had this issue, but for me, as soon as my EC2 instance rebooted or scaled, images were lost.

I tracked it down to line 122 of CloudImage, the getFormattedImage function (Worth also adding to the doc that this ONLY works with that perhaps? E.G - CroppedImage etc all just fall back to standard Image.)

So that line seems to check if the file exists locally before doing anything else, which in most cases, it doesn't.

    if ($this->ID && $this->Filename && Director::fileExists($this->Filename)) {

I changed that to

    if ($this->ID && $this->Filename)) {

and now this works fine for me. As soon as I clear my assets folder on disk, it downloads every item again from S3, then resamples them and stores them correctly.

Error: Upload File

Hi, when i try to upload file with $has_one 'VideoFile' => 'File' i've this error:
Object->__call(): the method 'parent' does not exist on 'File'

How can i fix it?

Issue setting up add-on

Hi,
I am trying to setup this add-on, but when I am uploading an image in the CMS I am getting:
SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
Not sure how to solve that.
Also on https://github.com/edlinklater/silverstripe-cloudassets-s3 the .yml is setup with api key and api secret and this add-on is using username and apikey. which one should I use?
thanks you!

Add RequirementsBackend that uses CDN links

Would need:

  • RequirementsBackend
  • Controller or ViewableData extension with $CDNLink method for templates
  • Task for keeping theme in sync
    • store a small file on the cdn bucket with the last sync date and then use the file mtime
    • also, may not even need that as Rackspace API has a syncFolder method (can we expect others to have that as well?)
  • Could be part of the bucket map with the assets

When uploading via HTMLEditorField "Warning Line 247 in /framework/api/RestfulService.php"

Below is the dump, this is originating from /admin/pages/edit/EditorToolbar/viewfile but only happens when the asset is uploaded to S3

ERROR [Warning]: curl_setopt(): supplied argument is not a valid File-Handle resource
IN GET /admin/pages/edit/EditorToolbar/viewfile?ID=92
Line 247 in /Users/drye/Development/Crossroads/Milacron/SS-CMS/framework/api/RestfulService.php

Source
======
  238:      curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  239:      curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
  240:      curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $this->getConnectTimeout());
  241:      if(!ini_get('open_basedir')) curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);
  242:      curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);
  243: 
  244: 
  245:      // Write headers to a temporary file
  246:      $headerfd = tmpfile();
* 247:      curl_setopt($ch, CURLOPT_WRITEHEADER, $headerfd);
  248: 
  249:      // Add headers
  250:      if($this->customHeaders) {
  251:          $headers = array_merge((array)$this->customHeaders, (array)$headers);
  252:      }
  253: 

Trace
=====
curl_setopt(Resource id #429,10029,)
RestfulService.php:247

RestfulService->curlRequest(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png,GET,,,Array)
RestfulService.php:195

RestfulService->request()
Oembed.php:107

Oembed::autodiscover_from_url(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png)
Oembed.php:147

Oembed::get_oembed_from_url(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png)
HtmlEditorField.php:928

HtmlEditorField_Embed->__construct(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png,CloudImage)
HtmlEditorField.php:516

HtmlEditorField_Toolbar->viewfile(SS_HTTPRequest)
RequestHandler.php:288

RequestHandler->handleAction(SS_HTTPRequest,viewfile)
RequestHandler.php:200

RequestHandler->handleRequest(SS_HTTPRequest,DataModel)
RequestHandler.php:222

RequestHandler->handleRequest(SS_HTTPRequest,DataModel)
Controller.php:153

Controller->handleRequest(SS_HTTPRequest,DataModel)
LeftAndMain.php:443

LeftAndMain->handleRequest(SS_HTTPRequest,DataModel)
AdminRootController.php:93

AdminRootController->handleRequest(SS_HTTPRequest,DataModel)
Director.php:370

Director::handleRequest(SS_HTTPRequest,Session,DataModel)
Director.php:153

Director::direct(/admin/pages/edit/EditorToolbar/viewfile,DataModel)
main.php:177

ERROR [Warning]: rewind() expects parameter 1 to be resource, boolean given
IN GET /admin/pages/edit/EditorToolbar/viewfile?ID=92
Line 284 in /Users/drye/Development/Crossroads/Milacron/SS-CMS/framework/api/RestfulService.php

Source
======
  275:          curl_setopt_array($ch, $this->proxy);
  276:      }
  277: 
  278:      // Set any custom options passed to the request() function
  279:      curl_setopt_array($ch, $curlOptions);
  280: 
  281:      // Run request
  282:      $body = curl_exec($ch);
  283: 
* 284:      rewind($headerfd);
  285:      $headers = stream_get_contents($headerfd);
  286:      fclose($headerfd);
  287: 
  288:      $response = $this->extractResponse($ch, $headers, $body);
  289:      curl_close($ch);
  290: 

Trace
=====
rewind()
RestfulService.php:284

RestfulService->curlRequest(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png,GET,,,Array)
RestfulService.php:195

RestfulService->request()
Oembed.php:107

Oembed::autodiscover_from_url(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png)
Oembed.php:147

Oembed::get_oembed_from_url(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png)
HtmlEditorField.php:928

HtmlEditorField_Embed->__construct(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png,CloudImage)
HtmlEditorField.php:516

HtmlEditorField_Toolbar->viewfile(SS_HTTPRequest)
RequestHandler.php:288

RequestHandler->handleAction(SS_HTTPRequest,viewfile)
RequestHandler.php:200

RequestHandler->handleRequest(SS_HTTPRequest,DataModel)
RequestHandler.php:222

RequestHandler->handleRequest(SS_HTTPRequest,DataModel)
Controller.php:153

Controller->handleRequest(SS_HTTPRequest,DataModel)
LeftAndMain.php:443

LeftAndMain->handleRequest(SS_HTTPRequest,DataModel)
AdminRootController.php:93

AdminRootController->handleRequest(SS_HTTPRequest,DataModel)
Director.php:370

Director::handleRequest(SS_HTTPRequest,Session,DataModel)
Director.php:153

Director::direct(/admin/pages/edit/EditorToolbar/viewfile,DataModel)
main.php:177

ERROR [Warning]: stream_get_contents() expects parameter 1 to be resource, boolean given
IN GET /admin/pages/edit/EditorToolbar/viewfile?ID=92
Line 285 in /Users/drye/Development/Crossroads/Milacron/SS-CMS/framework/api/RestfulService.php

Source
======
  276:      }
  277: 
  278:      // Set any custom options passed to the request() function
  279:      curl_setopt_array($ch, $curlOptions);
  280: 
  281:      // Run request
  282:      $body = curl_exec($ch);
  283: 
  284:      rewind($headerfd);
* 285:      $headers = stream_get_contents($headerfd);
  286:      fclose($headerfd);
  287: 
  288:      $response = $this->extractResponse($ch, $headers, $body);
  289:      curl_close($ch);
  290: 
  291:      return $response;

Trace
=====
stream_get_contents()
RestfulService.php:285

RestfulService->curlRequest(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png,GET,,,Array)
RestfulService.php:195

RestfulService->request()
Oembed.php:107

Oembed::autodiscover_from_url(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png)
Oembed.php:147

Oembed::get_oembed_from_url(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png)
HtmlEditorField.php:928

HtmlEditorField_Embed->__construct(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png,CloudImage)
HtmlEditorField.php:516

HtmlEditorField_Toolbar->viewfile(SS_HTTPRequest)
RequestHandler.php:288

RequestHandler->handleAction(SS_HTTPRequest,viewfile)
RequestHandler.php:200

RequestHandler->handleRequest(SS_HTTPRequest,DataModel)
RequestHandler.php:222

RequestHandler->handleRequest(SS_HTTPRequest,DataModel)
Controller.php:153

Controller->handleRequest(SS_HTTPRequest,DataModel)
LeftAndMain.php:443

LeftAndMain->handleRequest(SS_HTTPRequest,DataModel)
AdminRootController.php:93

AdminRootController->handleRequest(SS_HTTPRequest,DataModel)
Director.php:370

Director::handleRequest(SS_HTTPRequest,Session,DataModel)
Director.php:153

Director::direct(/admin/pages/edit/EditorToolbar/viewfile,DataModel)
main.php:177

ERROR [Warning]: fclose() expects parameter 1 to be resource, boolean given
IN GET /admin/pages/edit/EditorToolbar/viewfile?ID=92
Line 286 in /Users/drye/Development/Crossroads/Milacron/SS-CMS/framework/api/RestfulService.php

Source
======
  277: 
  278:      // Set any custom options passed to the request() function
  279:      curl_setopt_array($ch, $curlOptions);
  280: 
  281:      // Run request
  282:      $body = curl_exec($ch);
  283: 
  284:      rewind($headerfd);
  285:      $headers = stream_get_contents($headerfd);
* 286:      fclose($headerfd);
  287: 
  288:      $response = $this->extractResponse($ch, $headers, $body);
  289:      curl_close($ch);
  290: 
  291:      return $response;
  292:  }

Trace
=====
fclose()
RestfulService.php:286

RestfulService->curlRequest(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png,GET,,,Array)
RestfulService.php:195

RestfulService->request()
Oembed.php:107

Oembed::autodiscover_from_url(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png)
Oembed.php:147

Oembed::get_oembed_from_url(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png)
HtmlEditorField.php:928

HtmlEditorField_Embed->__construct(https://s3.amazonaws.com/crds-cms-uploads/Screen-Shot-2014-11-21-at-3.15.24-AM.png,CloudImage)
HtmlEditorField.php:516

HtmlEditorField_Toolbar->viewfile(SS_HTTPRequest)
RequestHandler.php:288

RequestHandler->handleAction(SS_HTTPRequest,viewfile)
RequestHandler.php:200

RequestHandler->handleRequest(SS_HTTPRequest,DataModel)
RequestHandler.php:222

RequestHandler->handleRequest(SS_HTTPRequest,DataModel)
Controller.php:153

Controller->handleRequest(SS_HTTPRequest,DataModel)
LeftAndMain.php:443

LeftAndMain->handleRequest(SS_HTTPRequest,DataModel)
AdminRootController.php:93

AdminRootController->handleRequest(SS_HTTPRequest,DataModel)
Director.php:370

Director::handleRequest(SS_HTTPRequest,Session,DataModel)
Director.php:153

Director::direct(/admin/pages/edit/EditorToolbar/viewfile,DataModel)
main.php:177

Corrupted images causes

When there is an upload error due to network issue or abrupt termination of upload (incomplete/corrupted image), it causes a read error and my app returns a internal server error.

CloudImage->getDimensions() probably should just return a 0x0 dimensions in this case

getimagesize(/Volumes/MainStorage/Development/Projects/htdocs/mmyapp/assets/Uploads/Member/45678/Items/14559942-240C-4542-8C2F-54A5F3DC699C-2.png) Image.php:1009 Image->getDimensions(string) CloudImage.php:87 CloudImage->getDimensions(0) Image.php:1022 Image->getWidth() Image.php:705 Image->isWidth(1200) Image.php:377 Image->ScaleWidth(1200) Image.php:526 Image->SetWidth(1200) ViewableData.php:466 ViewableData->obj(SetWidth,Array,1,,) SSViewer.php:99 SSViewer_Scope->getObj(SetWidth,Array,1,,) SSViewer.php:625 SSViewer_DataPresenter->getObj(SetWidth,Array,1,,) SSViewer.php:119 SSViewer_Scope->obj(SetWidth,Array,1,,) SSViewer.php:619

Clean up logic in CloudFileExtension

This is a low priority task but needs to be done at some point. That logic is difficult to understand and therefore at greater risk of bugs being introduced.

Abandoned dependency

edlinklater/silverstripe-cloudassets-s3 is now abandoned and silverstripe/s3 has replaced it. Can we get this updated?

Images treated as "from the web"

Images are being processed as "from the web" so users see the hotlink warning and there is a bug in SS that doesn't respect the height and width set on images "from the web" I have a fix for the later, but it seems to me that this shouldn't be happening. This also seems to mean that the _resampled isn't really being used.

Error shows up with resized images when flush=1 is present. 

CloudAssetsTest::testMetadataRestoredOnCachedImageIfNotPresent
getimagesize(): Read error!
getimagesize(/Users/guinns/Dropbox/Sites/daywind/assets/FileTest-folder1/_resampled/SetWidth10-test-png32.png)
GD.php:456

GDBackend->writeTo(/Users/guinns/Dropbox/Sites/daywind/assets/FileTest-folder1/_resampled/SetWidth10-test-png32.png)
Image.php:475

call_user_func_array(Array,Array)
CloudImage.php:102

CloudImage->getFormattedImage(SetWidth,10)
Image.php:249

Image->SetWidth(10)
CloudAssetsTest.php:214

Improved error handling

At this stage, error handling is largely silent such that if there's an upload or download error it just serves from the local assets as much as possible.

I'd like to see this taken up a notch:

  • Turn on logging via Monolog with fallback to Debug::log
  • Should be able to crash on error in some contexts (like possibly a build task)
  • Configurable via yml

Allow multiple CDN urls for parallel loading

You should be able to give an array of CDN urls and have it pick one randomly or round-robin each time Link() is called. This would allow you to set up multiple CNAME records to the CDN and utilise parallel connections better.

Something like:

    BaseURL:
      - http://media1.mysite.com
      - http://media2.mysite.com
    SecureURL:
      - https://media3.mysite.com
      - https://media4.mysite.com

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.