greengerong / prerender_asp_mvc

Prerende client for asp.net mvc

Prerender.io middleware for ASP.NET projects

Are you using backbone, angular, emberjs, etc, but you're unsure about the SEO implications?

Use this filter that prerenders a javascript-rendered page using an external service and returns the HTML to the search engine crawler for SEO.

Note: If you are using a # in your urls, make sure to change it to #!. View Google's ajax crawling protocol

Note: Make sure you have more than one webserver thread/process running because the prerender service will make a request to your server to render the HTML.

Demo project moved to Prerender_asp_mvc_demo.

Installing the middleware

1: Do a build of this project and include and reference the DLL in your web application

2: Add the http module to your web.config:

<httpModules>
	<add name="Prerender" type="Prerender.io.PrerenderModule, Prerender.io, Version=1.0.0.2, Culture=neutral, PublicKeyToken=null"/>
</httpModules>

3: Sign up to get a token and add a prerender section to the web.config file containing your token:

<configSections>
	<section name="prerender" type="Prerender.io.PrerenderConfigSection, Prerender.io, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
</configSections>

<prerender 
	token="[YOURTOKEN]">
</prerender>

4: You can add following additional attributes to the prerender section to override or add to the custom settings (see PrerenderModule.cs):

• prerenderServiceUrl
• stripApplicationNameFromRequestUrl
• whitelist
• blacklist
• extensionsToIgnore
• crawlerUserAgents

5: Create a new class called PreApplicationStartCode in the App_Start folder:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.Web.Infrastructure.DynamicModuleHelper;
using Microsoft.Web.WebPages.OAuth;
using Demo.Models;

namespace Demo
{
    public static class PreApplicationStartCode
    {
        private static bool _isStarting;

        public static void PreStart()
        {
            if (!_isStarting)
            {
                _isStarting = true;

                DynamicModuleUtility.RegisterModule(typeof(Prerender.io.PrerenderModule));
            }
        }
    }
}

6: Add this line to the bottom of the AssemblyInfo.cs file( PreApplicationStartCode):

[assembly: PreApplicationStartMethod(typeof(Demo.PreApplicationStartCode), "PreStart")]

7: Build and publish you web application.

8: To make shure the middleware works correctly, you can create a first request by surfing to [YOURURL]?_escaped_fragment_=, then log into your Prerender.io account. If there are errors shown in red, something went wrong. Otherwise, you're good to go!

How it works / Testing

1. Check to make sure we should show a prerendered page
   1. Check if the request is from a crawler (_escaped_fragment_ or agent string)
   2. Check to make sure we aren't requesting a resource (js, css, etc...)
   3. (optional) Check to make sure the url is in the whitelist
   4. (optional) Check to make sure the url isn't in the blacklist
2. Make a GET request to the prerender service(phantomjs server) for the page's prerendered HTML
3. Return that HTML to the crawler

OR

Mac:

1. Open the Developer Tools in Chrome (Cmd + Atl + J)
2. Click the Settings gear in the bottom right corner.
3. Click "Overrides" on the left side of the settings panel.
4. Check the "User Agent" checkbox.
5. Choose "Other..." from the User Agent dropdown.
6. Type googlebot into the input box.
7. Refresh the page (make sure to keep the developer tools open).

Windows:

1. Open the Developer Tools in Chrome (Ctrl + shift + i)
2. Open settings (F1)
3. Click "Devices" on the left side of the settings panel.
4. Click "Add custom device..."
5. Choose a name (eg. Googlebot), screen size and enter the following User agent string:

   Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
   

6. Make sure the new device is checked.
7. You can now choose it from the device dropdown in the Developer Tools screen.

Using your own prerender service

If you've deployed the prerender service on your own, set the PRERENDER_SERVICE_URL environment variable so that this package points there instead. Otherwise, it will default to the service already deployed at http://service.prerender.io

$ export PRERENDER_SERVICE_URL=<new url>

Or on heroku:

$ heroku config:add PRERENDER_SERVICE_URL=<new url>

As an alternative, you can pass prerender_service_url in the options object during initialization of the middleware

License

The MIT License (MIT)

TODO:

• upgrade version.(wait for my local env.)

prerender_asp_mvc's Issues

Blank page

Hi, I have everything already installed but when I build the project it's a blank page.

I read in the pretender.io site that is a common problem get <html><head></head><body></body></html> and that: You are probably on Rails or some other framework that uses blocking I/O

I am having this issue on a .NET app and I think that it doesn't use blocking I/O. Do you know what can be happening?

Thanks!

Using Prenrender with .Net Core

Hi,

How can I use prerender reference in my .Net Core app?

Prerender_asp_mvc stopped working correctly on my website

It had been functioning properly about 6 months. But suddenly we noticed that it did not render correctly the meta tags of the page.

The only change we made was to redirect to https and our web config is like this.

<configuration>
  <configSections>
    <section name="prerender" type="Prerender.io.PrerenderConfigSection, Prerender.io, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />
  </configSections>
  <prerender token="[Token]">
  </prerender>
  <appSettings>
    ...
  </appSettings>
  <system.web>
    <compilation debug="true" targetFramework="4.5.2" />
    <httpRuntime targetFramework="4.5.2" maxUrlLength="2048" />
    <httpModules>
      ...
      <add name="Prerender" type="Prerender.io.PrerenderModule, Prerender.io, Version=1.0.0.2, Culture=neutral, PublicKeyToken=null" />
    </httpModules>
  </system.web>
  <runtime>
    ...
  </runtime>
  <system.webServer>
    <httpProtocol>
      ...
    </httpProtocol>
    <staticContent>
      ...
    </staticContent>
    <rewrite>
      <rules>
        <rule name="httpsredirect" stopProcessing="true">
          <match url="(.*)" />
          <conditions>
            <add input="{HTTPS}" pattern="off" ignoreCase="true" />
          </conditions>
          <action type="Redirect" redirectType="Permanent" url="https://{HTTP_HOST}/{R:1}" />
        </rule>
      </rules>
    </rewrite>
    <security>
      <requestFiltering allowDoubleEscaping="true" />
    </security>
    <validation validateIntegratedModeConfiguration="false" />
    <modules>
      ...
    </modules>
  </system.webServer>
  <system.codedom>
    ..
  </system.codedom>
</configuration>

We went back to update the library from nuget package, we even downloaded the project and compiled it and used the dll and it has not worked again.

Note:
the urls of the site do not use # or !# They appear like these https://github.com/vinaygopinath/ngMeta for example.

using this in angularjs router
$locationProvider.html5Mode(true);
and
<base href="/" /> in th html

I reiterate, the project before worked properly, which means that it was configured correctly, as indicated in the tutorial, but now it no longer render the code js in the searches.

Any suggestions? Thanks in advance.

Confused

Hi,

I built an application using AngularJS and Web API and hosted in IIS . I want to make my application SEO friendly. To do this i have done all the below steps

1. Install Nuget Package Prerender.io
2. Added meta tag in index.html
3. Added $locationProvider.hashPrefix('!'); in the config file.
4. Updated my token in Web.Config file

But, still I'm seeing
"You need to install your Prerender token before you can start caching your pages."
And unable to cache my pages.

Could you please help me.

Prevent prerender from intercepting error code responses

We need to stop the prerender.io middleware package from intercepting errors (3xx, 4xx, 5xx) and forwarding them through prerender.io.

The way how it's currently implemented it intercepts every response and forwards it through prerender.io. This caused us to rack up additional fees this last month. So it would be great to make that configurable.

Also the list of bot user-agents is hard-coded. So, we are not able to configure this, which is what we would "like" to be able to do.

Add support for ASP.NET 5 and owin

hi,
great efforts, any chance to upgrade it to latest asp.net 5

thanks in advanced :)

Getting 504 on all page and also after implementing this google search console not able to crawl

All the uri is being appended with :5905 port and getting 504 error.

We used asp.net and MVC in single website some pages are in .aspx and some are in MVC. Also there are some component build on VUE.js that are being used.

We want to prerender the pages where VUE.js components are being loaded. Also after pushing changes on production it stopped allowing google to crawl the website. Also sent mail to support for the same but no response from prerender team. Any one can help me. (Implemented ASP.NET MVC Middleware)

504 for https

Hi!
I use prerender on my own server and have
respond 504 for https and 200 for http.

Could you please help.

Fix the demo project to work with what is in the documentation.

Tried using it with your default settings .

the demo just fails on PrerenderModule.cs
Line 96

Should not remove _escaped_fragment_ from the URL

the prerender service actually manages this part, so no need to remove on the module side

Potentially dangerous request.path

Having some trouble getting this implemented into my project. As soon as I add the PreApplicationStartMethod to my AssemblyInfo.cs, I get a stack trace of: Potentially dangerous request.path was detected from the client.

Is there something I need to add to web.config to get this running?

Add gzip support

The latest version of the Prerender server will return gzipped content to the middleware if the Accept-Encoding: gzip header is provided.

In our Node/Rails middleware, we currently always send the Accept-Encoding: gzip header and always unzip the response when it comes back from the server. That way, we save on bandwidth.

Supporting gzip with the ASP.NET middleware would be great! You could also decide to not unzip the response if the user sent the Accept-Encoding: gzip header in order to leave it compressed.

Can you give examples of how to use the black list?

I need to exclude URL's that have certain query strings (i.e. Google's ?utm_...) as well as certain paths on my site.

I see there is a blacklist option but I can't find any documentation about it.

Nuget package problem

NuGet package version 1.0.0.2 still downloads the outdated Assembly 1.0.0.0

Nuget package

Hi @greengerong,

I see you are an "Author" on this page:

https://www.nuget.org/packages/Prerender.io/

Does that give you access to publish newer versions? If so, could you publish a new one? Or would you like me to give you a version to publish?

Or what permissions does an "Author" give you on that repository? That's a very old package and we get emails about it needing to be updated so it would be nice if we could get a newer version out there.

Thanks so much for your help here.

Add NuGet support

I took the liberty of putting the package available via nuget.
To do this i change the namespace from Prerender_asp_net to Prerender.io.
https://www.nuget.org/packages/Prerender.io/
Maybe ia a good idea divide in two projects, the module and the demo.

What am I doing wrong?

Hi guys,

I'd really like to get this working, I'm not sure what else to try. Here are my steps so far:

I'm using angular in an MVC 5 project, configured to use #! urls. It's hosted as an azure web app/site.

I Downloaded and referenced the project from github (also tried using the nuget package as an alternative ), signed up to preprender.io and got my token. I've tried using the preapplicationstart method and using the web.config httpmodule setting.

I wasn't sure when/where to set my token so I used default value, is this correct?

[ConfigurationProperty("token", DefaultValue = "dfkjdkfjdkf.........")] 

I'm still getting: "You need to install your Prerender token before you can start caching your pages."

Setting my user agent to googlebot makes no difference to the rendered page. I've looked through the demo MVC project and as far as I can tell I've followed all the steps.

Many thanks for your time

HTTPS urls being redirected to HTTPSS

The Prerender module is not taking into consideration HTTPS and as such is failing to crawl them as it is adding an extra 's' onto the end of the protocol.

Whilst it looks like you've addressed this in code, nuget package for this was last updated in 2014.

I've found a more up to date package by sjdirect with a separate nuget package which we're going to try and use in place of this. Is the form supposed to replace this repo? If so, you really should highlight this in your readme.

Thanks,

Joe

URL-Rewrite

Just a thought is there a reason this isn't using the IIS re-write module? Did you try and develop using this? Or is there any limitations or reasons this wont work?

http://www.iis.net/downloads/microsoft/url-rewrite

Information on how to create your own prerender service

Is there information on how to create and utilize your own prerender service, beyond just creating your own prerender url?