Coder Social home page Coder Social logo

edgeryders / websites-from-discourse Goto Github PK

View Code? Open in Web Editor NEW
2.0 5.0 0.0 13.03 MB

Middleman-based framework for semi-static websites that are edited via Discourse wikis.

License: MIT License

Ruby 4.37% CSS 71.08% JavaScript 3.94% HTML 20.62%
middleman discourse discourse-api ruby markdown

websites-from-discourse's Introduction

NGI Forward Platform

A web-based exchange platform for a co-design and participation process. Developed by Edgeryders OÜ for the NGI Forward project. The platform can be seen live under ngi.edgeryders.eu (preliminary domain, will change later).

Content

1. Overview

2. Installation

3. Usage

4. Software architecture

5. Licence


1. Overview

This platform uses the static website framework Middleman and gets its content mostly from the open source forum software Discourse. This allows using Discourse as a basic CMS to edit the content on a website.

Unlike a typical static website, this one allows quasi-dynamic content because Middleman is installed server-side and re-generates the website every few minutes. The platform supports multiple language versions and like in Discourse, all content uses Markdown for formatting, while HTML is als supported.

The result is a fast web platform with dynamic content that has no or minimal potential security issues because it's a static website. Also the source code turns out to be easy and comfortable to maintain, as it is very compact due to the content being managed externally in Discourse rather than in the source files.

2. Installation

For deployment, you need to install the following on your server:

  • git
  • Ruby
  • bundler, the Ruby dependency manager
  • npm, needed for Bootstrap
  • Middleman
  • the code from this repository, by cloning the repository

After the basic installation is done, configure it as follows:

  1. Make the DOCUMENT_ROOT of your website point to the build/ subdirectory of your copy of this repository.

  2. Create a build script deploy.sh and place it into the root directory of your copy of this repository. Content will usually be like this:

    #!/bin/bash
    git pull https://github.com/edgeryders/ngi-forward-platform;
    LC_ALL="en_US.UTF-8" bundle exec middleman build;
    
  3. Execute your deploy.sh and check if the website is now accessible.

  4. Configure a cron job that runs deploy.sh regularly, for example once per hour or once in 15 minutes, depending on how close to realtime you need the dynamic information on the site.

    Configuring cron jobs can be horrible if you need to provide certain environment for your command to run. For example, if you use chruby or another mechanism to select one of multiple installed Ruby versions, your deploy.sh depends on $PATH etc. changes as set up by that system. In the case of chruby, we had to do the following steps until the cron job started to work:

    1. Adding the environment changes done by chruby (as seen in /etc/profile.d/chruby.sh) to the top of the deploy.sh script:

      source /usr/local/share/chruby/chruby.sh
      source /usr/local/share/chruby/auto.sh
      
    2. Configuring the cron job in /etc/crontab as follows:

      */15 * * * * username bash --login -c '/path/to/deploy.sh' >>/path/to/cron.log 2>>/path/to/cron_error.log
      

3. Usage

Editing content. Edit the relevant Discourse posts defined in the mapping file. It makes sense to place all this content into one Discourse category. We used the Internet of Humans → Web Content sub-category. It is non-public to prevent redundant public content on the Internet (which may lead to user confusion and SEO penalties).

Editing templates, i18n string translations or code. These parts of the website are directly hosted inside this Github repository. Edit them with the usual pull/edit/commit/push git workflow. For small edits, you can also directly change the file using the Github web interface.

Rebuilding the site. The website will rebuild itself automatically at every cron job run, which includes updating the repository from Github, all content and all other dynamic elements. When you are editing and immediately want to see how the results look, you can trigger a rebuilding via SSH as follows:

ssh [email protected]

web32@server:~$ cd /var/www/clients/client12/ngi.edgeryders.eu/web
web32@server:~$ ./deploy.sh

(Later, we will also provide a URL switch for the same purpose.)

4. Software architecture

This website uses the static website framework Middleman. However, unlike any "normal" Middleman based website, the website is automatically rebuilt in regular intervals (like, hourly) by a Middleman installation on the server. This allows to integrate dynamic content via API calls (if it does not need updates too often).

Currently, this website integrates the following dynamic elements:

  • Formatted content. The content blocks incl. images used on the various pages can be managed in an external CMS-like system. We use non-public, wiki-type posts in the open source forum software Discourse for this purpose, because our main website is made with Discourse. Each text block maps to one topic in Discourse, and each translation of that text block to one post in that topic. The mapping is defined in data/discourse_sources.yml.

  • Statistics. The number of topics, posts and users active in a Discourse category.

  • Thread lists. Detailed lists of Discourse forum topics in a certain category.

  • Events list. Obtained from all Discourse topics in a certain category that have a date and time attached (via the Discourse events plugin). Each event is shown with date, time, thumbnail image, title and teaser of the main text.

  • Other. It is easily possible to embed any external content that can be accessed by API or simply via HTTP and that can be converted to the publishable form with some code. The result should be HTML with CSS and optionally JavaScript. For example, data visualizations such as graphs about interactions on a Discourse forum can be embedded in this way.

Other major software technologies and components used:

  • Bootstrap. A mature HTML+CSS template framework developed by Twitter.

  • Markdown. Like in Discourse, all content uses Markdown as the markup language. It is converted to HTML during the Middleman build process.

  • Ruby. Because Middleman is written in Ruby. It fits nicely because Discourse itself is written in Ruby as well. This is pure Ruby code, without the Ruby on Rails framework.

5. Licence

In general, the MIT Licence applies to this software (see file LICENSE). The following exceptions apply:

  • Partner organization logos. All files assets/images/logo-* are copyrighted and not licensed under MIT License. For the purpose of the NGI Forward Platform (which is website on a specific domain that we operate), permission to use them has been obtained, see #13. If you want to use or publish them for other purposes, you have to obtain permission from the respective organizations yourselves.

  • EU emblem. The EU emblem in file assets/images/eu-emblem.png is protected as a trade mark or by similar means (it's not clear so far) and its use is subject to the rules in "The use of the EU emblem in the context of EU programmes".

websites-from-discourse's People

Contributors

albertocottica avatar damingo avatar tanius avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

websites-from-discourse's Issues

Use more funky formatting for headings

This applies to the H2 to H4 headings used in the site content. Each level should have a consistent style, but more "funky" than now :-)

Some inspirations can be found on our OpenVillage website. At the same time, the design of the headings should stay compatible with the design used on the ngi.eu platform. Which is somewhat of a challenge.

Let's use a quick solution for now (colors plus dotted border for example) as we'll most probably have to adapt it again later.

Render <a title="…"> elements as links

Sometimes, we use <a title="…"> … </a> elements to provide a "see details on hover" effect. See for example the link "famous study" in this post.

In the build output, the on-hover effect appears, but the text is rendered like normal text. It should be shown like a hyperlink instead.

Provide CSS formatting for blockquote elements

Markdown quotes (> …) are already properly translated into HTML <blockquote> elements, but these elements are not yet distinguished from normal text in the rendering. To be fixed :-)

Fix that in-page search is very slow in Chrome

Tested in Chrome 73 under Linux. When searching for words in the page using the standard Ctrl + F search box, both search and the display of search results is very slow. It took 10-20 s here.

It seems that the JavaScript animation is interfering with the (probably likewise JavaScript based) in-page search. This is not an issue with all these JavaScript particle animations but possibly just a configuration issue, as it does not happen over at the OpenVillage website.

Detect and handle Discourse 404 errors

When accessing the Discourse API and the result is a HTTP 404 error, the build script should render nothing and handle the error appropriately.

Right now, it will show the "Oops! That page doesn’t exist or is private." page content returned by the Discourse API for a 404 case (example). That page also contains lists of "Popular" and "Recent" posts that mess up the content of the rendered website in such a case.

Adapt the menu font

The menu font uses the text size of normal text paragraphs in the page right now. That's a bit small, and also it's "gray on gray".

Instead, let's switch to using the same menu font as used on ngi.eu, which is also the best bet to match the more detailed design requirements we'll get:

  • all caps
  • color: white
  • font-weight: 700
  • font-size: 14px (ngi.eu has 12px but that's a bit on the small side)

Show the intro paragraph as white text on the "10010111" background

The intro text (as taken from here) and the "Share your Experience" button below it should not have the white page background but rather should be shown in white text color over the dark page background and particle animation. So, in the same way as the "A community working together …" site title is shown right now, but in adequate font for an intro text of course.

This is a cosmetic change, so has a rather low priority.

Separate the main content sections more prominently

Currently, the main sections start with H2 headers and are contained in a <section id="…" …> element each. Since they are the main sections, they should be separated from the other main sections with the most obvious visual separation technique that will be used on the website.

Alternatives include (among other suitable options):

  • A gap in the white background, through which one would see the "101100100101000101011110010" background.

  • Different text background colors, as seen on openvillage.edgeryders.eu. However, as we also use the "101100100101000101011110010", this might create a visual mess.

Apply the new green / yellow color scheme

We got the final logo for the NGI Forward project, and it has already been applied on this site everywhere.

What's left to do is adapting the color scheme of this site based on the logo colors. This has only been done for the primary color (as configurable in Bootstrap), the link color (automatically based on the primary color with 5% darkening) and the "Share your experience" button.

We should apply it to the headings (currently black), sub-headings (currently red) etc. in one way or another.

The colors to use are these:

green yellow gray white
RGB Hex #009c8e #ffdb6a #6f9aa8 #ffffff
RGB Decimal 0/156/142 255/219/106 111/154/168 255/255/255
CMYK 80/13/51/1 1/14/67/0 67/27/28/6 0/0/0/0

Add a separator before the footer element

Currently, it is not visually obvious where the site's main content ends and the "This project has received funding from …" footer begins.

Please use a light gray, full-width background as seen on consultation.ngi.eu. And since the content area of the site is not full-width, let's separate it from that content area with a bit of gap through which the "10101000100111100110" background can be seen.

So in principle it will look like the top navbar mirrored on the bottom.

This seems to be a suitable choice that fits into the official design guide, while avoiding large areas of "that green NGI color" which can be "a bit too much". So, we'd not want to do it exactly the same way as the footer on ngi.eu itself.

Mark links to edgeryders.eu with a tooltip

I thin we should no longer force opening any links in a new tab (currently applied to the "Share your experience" and "Events" menu items). In Discourse, they consider this "a mild form of browser hijacking, as it prevents the user from opening links the way they like to do it.

Instead, let's add a title="…" attribute to all links that point to edgeryders.eu, including in the "Activity" table. That will create a tooltip before the user clicks on the link. The tooltip text should be the same for all tooltips, translated via the translation file mechanism. The initial English text should be:

Link goes to our content creation platform.

Make "Share your Experience" a page inside this website

To not confuse people too early with switching over to the Edgeryders website for posting, the menu item "Share your Experience" should point to a new page inside this website.

That page would have the following content:

  • An intro paragraph imported from Discourse posts, with the usual technique we use.
  • Another "Share your Experience" button, as used on the frontpage.
  • A table with the recent posts in the "Tell your Story" category. So basically the same "Activity" table as used on the frontpage, but the data source limited to just this one "Tell your Story" Discourse category.
  • A bit of statistics about the "Tell your Story" category, maybe displayed over the "recent posts" table as a few cards with one bit number and a small description each. The statistics could include:
    • number of users who posted in the category ("121 people sharing")
    • number of posts in the category ("200 stories")
    • number of comments in the category, which here means "number of posts minus number of topics" ("567 comments")
    • … something else maybe? If there's any other good metric, just add it.

Add the particle animation in the topmost content section

Would be great to have a particle network animation (as seen here) in the topmost content section "A community working together to build a human centric next generation Internet".

There are multiple repos providing code for this kind of thing. At first glance, the most suitable seems to be this repo because (1) it seems to be the most well-known and most maintained and (2) it can be installed as a gem.

Let's only provide a few, relatively slow-moving particles (means, so that a low framerate is fine). Otherwise the site would hog people's CPU, which is really annoying …

And if adding this gimmick turns out to be somehow difficult, let's leave it for later.

For now, apply the same color scheme as used on edgeryders.eu

Don't put in much effort for this though, as we'll have to change it again.

Our Discourse color scheme is here.

The header background color and page background color should still use the dark gray like now. But other elements should be switched to Discourse-aligned colors, including:

  1. The background color of menu navigation links should be #e45735 on-click and when being the currently active menu item (the latter only if supported by the Middleman framework, we don't want to make it complicated …).

Replace the NGI logo with a SVG version

Because otherwise it will always look fuzzy on high-resolution monitors. Github for example also uses a SVG logo in the top left (the octocat).

We should however wait with this until it's clear that this is the logo we'll continue to use.

Provide a "Stakeholders" menu

We have a need for another menu item in the main navigation menu bar of this website, as follows:

  • menu item position: left aligned, after "Events"

  • menu item name: "Stakeholder"

  • menu item content: when clicking on the menu item, a dropdown menu should appear with the following other menu items, linked to the URLs as given here:

These are just tag searches in the project category on edgeryders.eu and will be filled by tagging the topics accordingly.

Support the named icons used in Discourse

Currently, named icons used in Discourse appear as plain source text (for example :slight_smile:) in the rendered output of this website. They should be rendered as icons.

Anyway, this is for a later time when this might become a more complete, generic way to manage websites from within Discourse.

Fix the size of the footer image

Currently, there is an image /assets/images/OC-EC-H2020.jpg at the bottom of the https://ngi.edgeryders.eu/ pages.

When shown in 100% scale as done by default, on small screen resolutions like 1024×768 px it will appear with too large text and also introduce a horizontal scrollbar.

It should made to fit the page content width, including on smartphone screens.

Embed the "Storyboard slides" presentation into section "How it works"

This should rather not be done via the Discourse editing mechanism, as it's about embedding an iframe. Instead, the following embed code should go directly into a Middleman template:

<iframe src="https://docs.google.com/presentation/d/e/2PACX-1vQlqM-aeE3MrzBn3ROGojQbSzKFtGG-LunZfrcltwDdmjzSx4-uBU7gh144FKmNdtwCw8tZAmZKdV4n/embed?start=false&loop=false&delayms=3000" frameborder="0" width="960" height="569" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true"></iframe>

You can adapt the width="960" height="569" values proportionally to something else, as needed.

Fix that the lower storyboard slide navigation has an offset of +1

In the embedded storyboard presentation, there is a row of gray rectangles at the bottom to navigate through the presentation. Clicking on one will however navigate to the slide that corresponds to the rectangle next to it on the right.

To be fixed if / when development on this website resumes.

Rename the eu-banner* CSS classes

The currently used CSS class eu-banner in the footer section triggers a cookie-consent-killer extension like "I don't care about cookies" to hide the HTML element to which it is assigned.

Now such extensions are very useful 😄 … so let's keep them functional and rather adapt our CSS. This means:

  • rename CSS class eu-banner to eu-emblem
  • rename CSS class eu-banner-info to eu-emblem-info

Detect storyboard slides automatically

To configure which storyboard slides will be shown, currently one has to add a line such as the following

<% slides = %w[01 02 03 04 05 06] %>

Would be more comfortable if all slides based on some file naming pattern would be shown automatically.

Remove all HTML comments from the content texts

When managing the content for this website in Discourse topics, we occasionally use <!-- … --> HTML comments for editorial comments, TODO items etc..

For some reason (maybe the Markdown engine used in Middleman here?), HTML comments that are their own paragraph are removed in the build output, but all other "inline" HTML comments will appear as plain text in the output. Discourse however removes all HTML comments from the cooked posts, so that is what a user expects here, too.

For now, I have edited all HTML comments to be their own paragraphs, so this issue is not really urgent.

Support all markup supported by Discourse

Currently, all standard Markdown syntax is supported for writing content, but some special Discourse markup that would be useful is not supported yet:

  • hiding details with [details=summary text]…[/details]
  • Markdown formatting inside HTML tags, as supported by the Markdown standard (which states that Markdown and HTML can be freely mixed)
  • (some other useful stuff that is probably found in our Discourse text formatting manual)

Anyway, this is for a later time when this might become a more complete, generic way to manage websites from within Discourse. (And at that time, the repo would probably be split in two, one for the re-usable code and one for the code that only applies to the NGI Forward platform.)

The clean way to support all the Discourse markup would be to extract the post-to-HTML rendering code from Discourse and use that.

In the Activity table, show when the last activity happened

This would be a new column, with the timestamp shown in time-ago format. So this would be as in category listings of Discourse, just that it would consistently use the time-ago format (22min, 2h, 4d etc.).

The column should be in the rightmost position, and use "Last Activity" as its header.

Provide a way to trigger rebuilding the site

There should be a mechanism that, when triggered, forces a rebuild of the site outside of the usual cron-triggered rebuilding actions. That will enable authors to test / see their changes on the website after editing content in Discourse.

The mechanism could be:

  • A public page that is linked in the footer as "Rebuild site". That page would contain just one button "Rebuild", and may ask the user to enter an auth key that would be published in the access protected category's "About the … category" in Discourse. Alternatively, it would not require authorization, but rate-limit the frequency of actions by IP address to prevent potential server overload issues. A button (issuing a POST request) is technically cleaner than a GET parameter, because GET requests should not result in state changes on the server.

  • A secret GET parameter like ?rebuild_auth_key=123456. The full URL incl. that parameter would be linked from the access protected category's "About the … category" in Discourse.

  • Something else.

Anyway, this is for a later time when this might become a more complete, generic way to manage websites from within Discourse.

Fill the "About" dropdown menu according to the H3 headings in the text

The dropdown menu is in general already made from the relevant H3 headings. But one menu item is missing and others are in a different order.

The correct, full list of menu items with their links is:

  • Overview, #overview
  • Consortium, #consortium
  • Privacy & Consent, #privacy
  • Legal information, #legals
  • Project Deliverables, #deliverables
  • Resources, #resources

(Also, I just added the relevant id attributes to the content over at Discourse so that the links do now work.)

Make "Events" a page inside this website

Like the new "Share your experience" page (see #31), the menu item "Events" should point to a new page inside this website.

That page would have the following content: one card for each upcoming event.

Upcoming events are topics in category Internet of Humans that are tagged with tag upcoming-event (which is not used yet, but will be). This excludes topics about events after they happened and also topics with reflections about events, speculative planning for events etc.. We could further limit to category Internet of Humans → Events, but better don't because sometimes people will place event invitations into other sub-categories by mistake.

Each card should contain:

  • Teaser image. Take the first image from the first post of the topic; if there is no image in the first post, take a default placeholder image.

  • Event title. Take the title of the topic. By convention, this would also always contain the date of the event.

  • Event description. Take the first paragraph of the first post of the topic. Usually, this paragraph will contain all the essentials about the event (location, date, how to attend, short description). If the paragraph is very short, better include another one, until a certain minimum length is met.

  • "Read More" link. This will point to the topic on edgeryders.eu, with the usual on-hover note "Link goes to our content creation platform".

Fill section "Partners" with the partner organization logos

This refers to this section: https://ngi.edgeryders.eu/#partners

The process to solve this issue is as follows:

  1. Obtain the logos and put them into a private folder in a uniform size (here, 130 px high). This is done by now.

  2. @maribati: please agree with each of the partner organizations on logo use, in writing. This includes:

    • Obtain written permission from each organization that we can use their logo. You can send them the logo as we intend to use it, and a link to the location where we intend to use it: https://ngi.edgeryders.eu/#partners
    • Ask them to provide a different logo file, ideally in a vector format, if we should use a different one.
    • Ask them for the link where the logo should point to. This can be nothing (no link), the frontpage of their website, or a sub-page they have set up about their involvement in the NGI Forward project.
    • Post relevant excerpts of their answers as a comment to this issue. So we have a public record that we use the logos properly.
  3. @damingo: Once the above two steps are done, please add the logos at https://ngi.edgeryders.eu/#partners . The logos should be presented in a reasonable layout, such as rows with 3 or 4 logos each. Each logo should be linked to the link target given by the partner organization, namely:

  4. @tanius: Create a prominent notice in README.md and in a special readme in the logos' directory that the logos are not covered by the open source licence of this repository, but are copyrighted materials licenced for use on https://ngi.edgeryders.eu/ only.

Render @mentions in the Events section with absolute links

If an @mention appears in the start of a topic that is published in the Events section, it is correctly rendered as a hyperlink. Just, it has a relative target /u/username, which will result in 404 Not Found. It needs an absolute URL https://edgeryders.eu/u/username.

Fix that the "Last Activity" column of the Discourse post lists ignores activity from edits

To reproduce, edit the last post of a public topic anywhere in https://edgeryders.eu/c/ioh . Re-generate the static website content and have a look at the tables with Discourse topics.

The table is supposed to show the n Discourse topic with the most recent "Latest Activity", starting with the one that had the most recent activity. The topic you edited is at the top, as expected. But its "Last Activity" column calculates its value from when the last post in the topic was posted (which could be months ago), instead of when it was edited as here.

The "Activity" column on https://edgeryders.eu/c/ioh which serves as the template here gets this thing right. So the necessary information will also be available in https://edgeryders.eu/c/ioh.json, serving as the data source here.

Remove the borders to the left and right of the jumbotron element

The topmost content section "A community working together …" is inside a <div class="jumbotron"> … </div> element. That's nice except for the small white gaps at the left and right where the text background shines through. So let's just make the jumbotron as wide as the content area.

Get rid of npm

We want to completely remove using npm from this system, and all code currently managed by it, because it is the most messed up packaging system under the sun. For a tiny software like this, we must read:

added 975 packages from 617 contributors in 72.289s

Hahaha 🤣

In this project, npm seems to be only integrated as the "external pipeline" in config.rb. See this portion:

 activate :external_pipeline, 
   name: :webpack, 
   command: build? ? 'npm run build' : 'npm run watch', 
   source: ".tmp/dist", 
   latency: 1

If there is a need to have any package manager beyond gem, let's rather use yarn. Because yarn will be the standard package manager for the Edgeryders organization for all JavaScript related projects.

Use only black text in the "Activity" table

Seems better for readability. Since Discourse itself uses no hyperlink color in such a table either, it should still be obvious enough that rows of that table are links.

Improve the rebuilding speed during development by not talking to Discourse

There should be a command line argument that disabled fetching new content from Discourse, because this takes a lot of the rebuilding time and is not needed when just wanting to see design or structure changes during development.

However, this would then require caching the Discourse API responses into some kind of JSON files. And also, rebuilding is fast during development when using the integrated development server. So this might not be worth the effort after all.

Add a Github repo link in the footer

Since the website is open source and its system is quite interesting and "nice", it would be good to point from the footer area to this repository, using the typical Github octocat icon.

This can be located in the right bottom corner, with the same paddings and margins as already used on the left side. It should be prepared for adding linked social media icons later.

(Also, please use the title="Our Github repo. Please report any problem with this website as an issue there." as an attribute on the link.)

Set up a system to update translation files

There should be a proper system for people to comfortably create and update translations stored within the translation files in the Github repository.

That system would recognize from git commits which translations got updates in the English version and show them to translators to change the translation as well, if it needs it. It would also ask them to translate the texts for newly added translation keys. Since this is a very usual problem, there is probably a solution around that is integrated with Github and can be used by non-developers.

Anyway, this is for a later time when this might become a more complete, generic way to manage websites from within Discourse.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.