Use latest in the link for the reference documentation about mapstruct.org

[gunnarmorling]

That's a very good idea. I'd name it "stable" only instead of "latest".

We also should prevent any doc versions besides "stable" from being indexed: https://support.google.com/webmasters/answer/93710?hl=en

[filiphr]

"stable" is also fine. Just to have something which is not linked to a specific version 😄.

I was also looking into the noindex meta tags. For this we will have to add <meta name="robots" content="noindex" /> to the reference documentation during the generation. Should we do the same for the api documents as well?

I think that we will either need to add/remove the meta tags manually or make it configurable in our build, so they are either added or not added to the pages

[agudian]

Another alternative would be to use a robots.txt - we wouldn't have to alter the html files (there are a lot of them in the javadoc subdir), but that wouldn't make google forget that it had the older versions indexed already. Not sure if that would be a problem, though.

[gunnarmorling]

I don't think we can have a robots.txt on GH Pages. AFAICS the way forward is to add the meta tag mentioned by filip to the non-stable reference guide and API docs. Ideally it's automated in some way.

[agudian]

Really? Don't the crawlers visit through www.mapstruct.org and not via http://mapstruct.github.io/ or something?

But yes, we could add a script to add those headers to the older files as well...

[filiphr]

I think that I found some way that we can add/remove the needed tags. I propose the following solution:

• We generate the api documentation and the reference documentation we are going to add <meta name="robots" content="noindex" /> which will tell the crawlers not to index those sides.
• We will have a stable folder where the stable version will be located, before we do the publish in our publish.sh we add this line find public/documentation/stable -type f -name "*.html" | xargs sed -i '/<meta name="robots" content="noindex".*\/>/d' before we do the commit

We will of course have to regenerate the documentation for 1.0.0.Final and 1.1.0.Final so we can add the noindex meta tag.

WDYT? Should I make a PR proposition?

[gunnarmorling]

Yes, that sounds great! Am 28.01.2017 6:15 nachm. schrieb "Filip Hrisafov" <[email protected]

…

: I think that I found some way that we can add/remove the needed tags. I propose the following solution: - We generate the api documentation and the reference documentation we are going to add <meta name="robots" content="noindex" /> which will tell the crawlers not to index those sides. - We will have a stable folder where the stable version will be located, before we do the publish in our publish.sh we add this line find public/documentation/stable -type f -name "*.html" | xargs sed -i '/<meta name="robots" content="noindex".*\/>/d' before we do the commit We will of course have to regenerate the documentation for 1.0.0.Final and 1.1.0.Final so we can add the noindex meta tag. WDYT? Should I make a PR proposition? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#43 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABvxE3ZDTCPXpDeuIr36NMVIMIlWTMoks5rW4WlgaJpZM4LvGgA> .

[gunnarmorling]

Closing; thanks, @filiphr!