how to render relative images? about prerender

[thoop]

If you're proxying the requests through your webserver (the recommended way), all relative URLs should work correctly.

I am trying to use the nginx conf here https://gist.github.com/thoop/8165802

the node server is running on 3000

there's no mention of port 3000 in that conf. is there a step I'm missing?

[Stanback]

If you're using your own instance of Prerender on the same server as Nginx, you'll want to replace the following:

proxy_pass http://service.prerender.io;

with:

proxy_pass http://localhost:3000;

@Stanback

I tried that but when I navigate to mydomain.com/http://somewhere.com, it only shows the index.html. The site is not being rendered by the prerender server.

When I check the running prerender node server on 3000, there's nothing triggere, it just says 'running on port 3000'

I've tried using the server's ip, 127.0.0.1, 0.0.0.0 but the result is same.

Any ideas on what else I can try? does my user agent have to be baiduspider or twitterspider? (not sure what that line does)

server {
    listen 80;
listen [::]:80 ipv6only=on default_server;
server_name mydomain.com;

root   /home/me/www/;
index  index.html;

location / {
    try_files $uri @prerender;
}

location @prerender {
    #proxy_set_header X-Prerender-Token YOUR_TOKEN;

    set $prerender 0;
    if ($http_user_agent ~* "baiduspider|twitterbot") {
        set $prerender 1;
    }
    if ($args ~ "_escaped_fragment_") {
        set $prerender 1;
    }
    if ($http_user_agent ~ "Prerender") {
        set $prerender 0;
    }
    if ($uri ~ ".js|.css|.xml|.less|.png|.jpg|.jpeg|.gif|.pdf|.doc|.txt|.ico|.rss|.zip|.mp3|.rar|.exe|.wmv|.doc|.avi|.ppt|.mpg|.mpeg|.tif|.wav|.mov|.psd|.ai|.xls|.mp4|.m4a|.swf|.dat|.dmg|.iso|.flv|.m4v|.torrent") {
      set $prerender 0;
    }

    if ($prerender = 1) {
        rewrite .* /$scheme://mydomain.com$request_uri? break;
        proxy_pass http://0.0.0.0:3000;
    }
    if ($prerender = 0) {
        rewrite .* /index.html break;
    }
}

}

[Stanback]

The line with "baiduspider|twitterbot" is a case-insensitive, pipe-delimited list (regular expression) of user agents that Nginx should serve prerendered pages to. The idea is that Nginx will return the prerendered version of the page to crawlers while normal browsers will get the Javascript version.

To test if prerender is working, you can change your browser's user agent to something in that list. In addition, I've setup my Nginx config to show the prerendered version when I append "?prerender=1" to the URL. This can be done by replacing:

    if ($args ~ "_escaped_fragment_") {
        set $prerender 1;
    }

with:

    if ($args ~ "_escaped_fragment_|prerender=1") {
        set $prerender 1;
    }

If you would like to always serve prerendered pages, replace the following:

    set $prerender 0;
    if ($http_user_agent ~* "baiduspider|twitterbot") {
        set $prerender 1;
    }
    if ($args ~ "_escaped_fragment_") {
        set $prerender 1;
    }

with:

    set $prerender 1;

[thoop]

You said you tried to access mydomain.com/http://somewhere.com, which sounds like you're just trying to set up a Prerender server that you can access at mydomain.com.

Are you trying to get your website set up so that it renders HTML pages for crawlers? Or are you trying to just set up a Prerender server?

@thoop yes im trying to setup a prerender server I can access at mydomain.com

so update, the prerender seems to be starting when i hit
http://mydomain.com/http://asdf.com

however, it just displays the index.html

[thoop]

So, that nginx config is only for a website that is trying to use a Prerender server to generate the static HTML.

If you're just trying to set up a Prerender server at http://mydomain.com, then you don't need that config at all. Just start the Prerender server on whatever port you need to start it on, and just go back to a normal nginx config. You don't need all that Prerender stuff in the nginx config anymore.

export PORT=80

[thoop]

Also, that won't fix the original problem of not rendering relative images. Once you set up your website to proxy requests to your new Prerender server, they'll work for your website.

If you're trying to use the Prerender server for a different purpose, you could write your own Prerender plugin to add a tag to the page, pointing to the root of the website. That will make relative links work for other web pages when you access them directly from your Prerender server.

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/base

so I created a plugin called absolute.js however relative image and assets are still broken and checking source code there is no <base> tag inside <head>. The jquery code was tested to be working.

module.exports = {
    afterPhantomRequest: function(phantom, context, next) {
      phantom.includeJs("http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js", function() {

        phantom.evaluate(function() {
          var pathArray = window.location.href.split( '/' );
          var protocol = pathArray[0];
          var host = pathArray[2];
          var url = protocol + '://' + host;
          console.log(url);                                            
          $('head').prepend('<base href="' + url + '" target="_blank">');              
        });

    });
    next();
 }
}

This is my server.js

server.use(prerender.logger());
//server.use(prerender.removeScriptTags());
server.use(prerender.httpHeaders());
//server.use(prerender.inMemoryHtmlCache());
//server.use(prerender.s3HtmlCache());
server.use(prerender.absolute());

server.start();

[thoop]

Where you able to get this working correctly?

[johnzw89]

If I am using the prerender.io service, how can I setup the proxy through my servers to get the relative image paths working? I am using the prerender_rails gem. Does that not setup the middleware?

Sorry if this is a totally noob question...

[thoop]

You should proxy the request through your website using the Prerender.io middleware. That way relative images will work fine without changing anything since the domain is correct. If you are just testing directly against your prerender server, the relative images won't work since the domain is different but that shouldn't matter since it will work once proxied through your site.

[benbonnet]

@thoop how do you proxy those css/images/etc… ?

[thoop]

You shouldn't proxy static files through the Prerender service. You should serve those directly from your server. Email us at [email protected] if you have any questions about getting that to work with the middleware.

[scher200]

Dear @thoop I like to add that on my angular-cli project I experience the same error. Relative non-external paths within css files where not loaded correctly:

background-image: url('../images/a-green-field.jpg');

I was forced to rewrite my urls with the full path https://www.live-domain.com/images/a-green-field.jpg.
This happens during local testing, but also at the online service prerender.io

While testing locally, I tried the following path:

http://localhost:3000/http://localhost:4200/images/a-green-field.jpg

And the image showed up.

Please let me know if there is any trick to inform the prerender io know that the images are available:

[thoop]

You shouldn't need to make that change. Relative links will work just fine when you proxy the HTML through your middleware. What you are seeing is the following:

When you access a Prerender server directly in your browser, like:

http://localhost:3000/http://localhost:4200/

The prerender server will render the http://localhost:4200/ page and return the HTML to your browser. Since your browser is viewing your HTML through the domain of http://localhost:3000/, that means any relative links in the HTML of http://localhost:4200/ will be loaded from http://localhost:3000/ by your browser. Since those files don't live on the Prerender server, you'll see 504s for those relative URLs.

That's expected and isn't an issue because once that HTML is proxied through your Prerender middleware, the HTML will be shown under the domain of http://localhost:4200/, and that's where the relative links will work correctly.

So testing directly against your Prerender server might not show css or images that are relative links but serving the HTML through your middleware will fix that without needing to make any changes.

[thoop]

Please feel free to create a new issue if anyone has questions about anything else.