apache / camel-website

Apache Camel Website

Apache Camel Website

This is a site generator project for Apache Camel. It generates static HTML and resources that are to be published.

Tools used to generate the website:

• Git a source code management tool used to fetch document sources from different github repositories.
• Node.js a JavaScript runtime used to build the website. You will need to use Node.js version 10.
• yarn a blazing fast dependency and package manager tool used to download and manage required libraries.
• (installed via yarn) Gulp a task automation tool. Used to build the Camel Antora UI theme.
• (installed via yarn) Hugo a static site generator. Simplified, it takes the documentation from the content directory and applies templates from layouts directory and together with any resources in static directory generates output in the public directory.
• (installed via yarn) Antora a documentation site generator. It uses Asciidoc documents from different sources in the Camel, Camel K and Camel Quarkus repositories where user manual and component reference documentation resides and renders them for inclusion in this website.
• (optional) Maven a build tool used to run the complete website generating process.

Build with Node and yarn

You can build the website locally using the tools Node.js and yarn.

If you can not use these tools on your local machine for some reason you can also build the website using Maven as described in section "Build with Maven".

Preparing the tools

Chocolatey

For windows users, a beginning step to install yarn and nvm on your local system is through installing chocolatey.

An easy step to step guide to install chocolatey on your local system is as follows:

1. Open cmd/powershell and run it as administrator.

2. Install with cmd.exe

    > @"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command " [System.Net.ServicePointManager]::SecurityProtocol = 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"
   

3. Install with PowerShell.exe

    > Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
   

Node

Make sure that you have Node.js (herein "Node") installed.

$ node --version

If this command fails with an error, you do not have Node installed.

This project requires the Node LTS version 14 (e.g., v14.15.0).

Please make sure to have a suitable version of Node installed. You have several options to install Node on your machine.

Installation of nvm on Linux/Mac OS

• Install using the Node version manager nvm
• Install using Homebrew and Node formulae
• Install from official Node packages

An easy step to step guide to install nvm and install node v14 on your local system is as follows:

$ touch ~/.bash_profile
$ curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.36.0/install.sh | bash
$ source ~/.nvm/nvm.sh
$ nvm install 14

Note - If you have different Node version other than Node LTS version 14 you can use following command to make Node LTS version 14 as default Node version.

$ nvm use 14

Installation of nvm on Windows

Note - The following steps need to be ran on cmd as administrator only.

An easy step to step guide to install nvm and install node v14 on your local system is as follows:

> choco install nvm
> nvm install 14

Note - If you have different Node version other than Node LTS version 14 you can use following command to make Node LTS version 14 as default Node version.

> nvm use 14

Now that you have Node 14 installed, you can proceed with checking the Yarn installation.

Yarn

Follow the documentation on installing Yarn for your Operating system.

Note: For windows users, run on cmd as administrator and install yarn through chocolatey.

Clone and Initialize the project

Clone the Apache Camel Website project using git:

$ git clone https://github.com/apache/camel-website.git

The command above clones the Apache Camel Website project. After that you can switch to the new project directory on your filesystem.

Build the website and Antora theme

Some of the content for the website is derived from the data received from GitHub API and rate limits can cause build failures. For that reason it is necessary to set either the following environment variables:

• HUGO_PARAMS_GitHubUsername=<GitHub username>
• HUGO_PARAMS_GitHubToken=<GitHub token>

Or the following environment variable:

• GITHUB_TOKEN=<personal access token>

These values are used by Hugo when building or running in development mode (yarn preview:hugo) or building the website (yarn build:hugo or yarn build-all) to access GitHub API with a higher rate limit.

NOTE: When you clone the camel-website repository the binaries present in the Yarn PnP cache or in the .yarn/unplugged directories are built for x86-64 Linux, so if you're running on a different architecture or OS run yarn workspaces foreach rebuild to rebuild the binaries.

We're using yarn workspaces to build both the theme and the website run build-all script, for example:

$ yarn build-all

That will build the Antora theme (from antora-ui-camel directory) and the website. Result of the build can be seen in the public directory.

Build the Antora Camel UI theme

The theme sources are located inside Project root directory/antora-ui-camel. So first switch to that directory:

$ cd antora-ui-camel

In that directory execute:

$ yarn build   # to perform the ui theme build

You should see the Antora theme bundle generated in in antora-ui-camel/build/ui-bundle.zip.

In case yarn build raises error, run yarn format to format the code and re-run yarn build to build your bundle successfully.

The Camel Antora UI theme should not be a subject to change very frequently. So you might execute this once and never come back.

Build the website content

Building the website requires the built Antora Camel UI theme bundle from above. Please check that the theme bundle exists in antora-ui-camel/build/ui-bundle.zip.

To build the website go to the project root directory and run:

$ yarn build   # to perform the build

In case yarn build throws the error: JavaScript heap out of memory, the issue can be resolved by increasing the memory used by node.js by setting NODE_OPTIONS environment variable to include --max_old_space_size, for example to increase the old space to 4GB do:

$ export NODE_OPTIONS="--max_old_space_size=4096"

This should fetch doc sources for Camel and Camel K and generate the website with Hugo. You should see the generated website in the public directory.

Preview website locally

You can preview the Apache Camel website on your local machine once you have the generated website available in the public directory.

Hugo can start a simple web server serving the generated site content so you can view it in your favorite browser.

Simply call

$ yarn preview

and you will be provided with a web server running the site on http://localhost:1313/

Point your favorite browser to http://localhost:1313/ and you will see the Apache Camel website.

Changes that are made to the content managed by Hugo (i.e. content, layouts, config.toml) are applied automatically and reloaded in the browser. To make changes to the content managed by Antora, a rebuild needs to be done. The same is true for the CSS changes in the antora-ui-camel. To rebuild you can run, in another terminal window, from the root directory of the website:

$ (cd antora-ui-camel && yarn build) && yarn antora  --clean --fetch antora-playbook.yml

This will build the antora-ui-camel which holds all the CSS and JavaScript, and then rebuild the documentation, resulting in an updated content in the documentation directory.

To iterate quickly, it's easier to make changes directly in the browser tooling and then bring the changes over to the CSS files after the fact.

Working on documentation (asciidoc) content

To build the documentation we pull the content of git repositories (see antora-playbook.yml), so to make changes locally and have them built or previewed without those changes being merged to Camel git repositories you need to adapt the antora-playbook.yml file.

For example to work on the user manual locally change the content, sources to point to HEAD of your local git repository, in this example located in ../camel:

content:
  sources:

    - url: ../camel
      branches: HEAD
      start_paths:
        # manual
        - docs/user-manual

Now you can run yarn build:antora or yarn preview and see the locally made changes. More details on this you can find in the Antora documentation.

Typical workflow is to run the yarn preview in one command line session and then rebuild the Antora documentation by running yarn build:antora for the documentation to be refreshed.

TIP: We pull in several git repositories and build several versions (branches) of documentation from them, time can be saved by removing sources for the documentation not worked on. Though be careful about inter-dependencies, for example several documents in the component reference point to the user manual.

CAMEL_ENV environment variable

Setting the CAMEL_ENV changes the output of the website build slightly, possible values are development (set by default if CAMEL_ENV is unset), production or netlify.

To run the optimizations, which currently consist of running htmlmin to reduce the size of generated HTML documents, set the CAMEL_ENV environment variable to production, for example:

$ CAMEL_ENV=production yarn build

When build is performed on Netlify, we set it to netlify to add the link to Netlify required by Netlify's open source policy.

Contribute changes

The Apache Camel website is composed of different sources. So where to add and contribute changes in particular?

Changes on the website

Menu

The site main menu is defined in the top level configuration config.toml. You can add/change menu items there.

Content

The basic website content is located in content. You can find several different directorys representing different areas of the website:

• docs: Getting started, user manual, component reference
• download: Download Camel artifacts
• blog: Blog posts
• community: Support, contributing, articles, etc.
• projects: Subproject information (e.g. Camel K)
• security: Security information and advisories
• releases: Release notes

Adding new blog post

Use the blog archetype to create a new markdown content file in content/blog:

$ yarn hugo new --kind blog blog/YYYY/MM/PostName/index.md # replace YYYY with the year, MM with the month and PostName with the actual name

Put a nice featured image in content/blog/YYYY/MM/PostName/featured.png and edit content/blog/YYYY/MM/PostName/index.md filling in the details.

The final generated URL would be something like https://camel.apache.org/blog/2020/05/MyNewPost/.

Don't forget to remove draft: true to publish the blog post.

Adding new security advisory content

Use the security-advisory archetype to create a new markdown content file in content/security:

$ yarn run hugo new --kind security-advisory security/CVE-YYYY-NNNNN # replace YYYY-NNNNN with the CVE number

This will create a content/security/CVE-YYYY-NNNNN.md file which you need to edit to and fill in the required parameters. The content of the created markdown file is added to the Notes section.

Place the signed PGP advisory in plain text as content/security/CVE-YYYY-NNNNN.txt.asc.

Make sure that you set the draft: false property to have the page published.

Adding new release note

Release of Apache Camel (core)

Use the release-note archetype to create a new markdown content file in content/releases:

$ yarn run hugo new --kind release-note releases/release-x.y.z.md # replace x.y.z with the release version

This will create a content/release-x.y.z.md file which you need to edit to and fill in the required parameters. The content of the created markdown file is added to the New and Noteworthy section.

Make sure that you set the draft: false property to have the page published.

Sub-project release

Use the release-{category} archetype to create a new markdown content file in content/releases/{category}-{version with underscores}/index.md.

For {category} you can use: k k-runtime, ckc, or q.

For example, to create Camel Kafka Connector release note:

$ yarn run hugo new --kind release-ckc releases/ckc/release-x.y.z.md # replace x.y.z with the release version (use underscores)

This will create a content/releases/ckc/release-x.y.z.md file which you need to edit to and fill in the required parameters. The content of the created markdown file is added to the New and Noteworthy section.

Make sure that you set the draft: false property to have the page published.

Layout and templates

Layout related changes go into layout directory where you will find HTML templates that define the common layout of the different page categories including footer and header templates.

Changes in Antora UI theme

The Antora UI theme basically defines the look and feel of the website. You can find the theme sources within this repository in antora-ui-camel.

You need to rebuild the Antora UI theme in order to see your changes reflected locally.

Changes for Antora generated content

Edits to individual pages can generally be made from the "edit this page" link at the top left of each antora-generated page. However, this will not yet work for Camel component documentation pages.

For more extensive changes, use of the Intellij Asciidoctor plugin is extremely convenient as it provides extensive syntax checking and understands a great deal of Antora-specific asciidoc syntax such as xrefs. The plugin works with all Intellij editors including the free IDEA CE.

Consult the Antora documentation and Asciidoctor documentation for complete information on Antora and AsciidDoc/Asciidoctor.

The Apache Camel website includes documentation sources from other github repositories. Content sources are defined in antora-playbook.yml.

Your changes in these repositories will automatically be included in the website after a site rebuild.

Links between pages in Antora content

Links to other pages in Antora content are defined using the xref: inline macro. This takes the form xref:<optional #fragment>[]. The default link text for non-fragment links is the target page title.

Antora documentation for page ids is here. A small project showing the effect of all possible forms of page id is here.

The first step is to determine the component, version, and module of both the source and target pages.

To determine the component name and version of a page, look in the antora.yml file next to the "modules" folder the page source is in. The "module" is the folder name under "modules". The "default" module is "ROOT": leaving out the module name will always take you to the ROOT module.

Several repositories and start paths in a repository can contribute to the same component/version. For instance, in the main camel repository, the "components" component has sources under docs/components (dataformats, languages, other, and ROOT (components) modules) and core/camel-core-engine/src/main/docs (eips module). The easiest way to determine the current set of components and their versions is to consult the website and look in the "drawer" at the bottom left, showing the current component/version. When opened it lists the components and for each component the versions. The main camel repository contains both the "components" and "user manual" Antora components. Other components are in the obvious repository.

• To link to another page in the same component/version/module use the absolute path from "pages": xref:path/to/page.adoc[]. For instance, a link from one camel component to another would be xref:jms-component.adoc[]. A link from any camel-quarkus page to a camel-quarkus extension would be xref:reference/extensions/jms.adoc[].
• To link to a page in another module in the same component/version use the module name and absolute path from "pages": xref:module:path/to/page.adoc[]. For instance, a link from a dataformat to a language would be xref:languages:jsonpath-language.adoc[]. A link from a dataformat to an eip would be xref:eips:aggregate-eip.adoc[].

Xref links that do not specify the component or version stay within the same component/version. If only the (Antora) component is specified, the link will be to the "latest" version of the component. This is most likely what you want.

• For instance, a link from camel-quarkus to the latest version of a component would be xref:components::jms-component.adoc[].
• A link from a component, eip, or other "components" module to the user manual would be xref:manual::architecture.adoc[].
• A link from a component, eip, etc, or the user manual to a camel-quarkus extension would be xref:camel-quarkus::reference/extensions/jms.adoc[].
• Note that specifying the same component as the source page will link to the latest version of the target page: e.g. xref:camel-quarkus::reference/extensions/jms.adoc[] from a version 2.0.0 camel-quarkus page will link to the latest jms.adoc version, not the 2.0.0 version.

If you need to link to a specific non-latest version of a page, specify the version in the xref.

• A link from anywhere to a camel-quarkus 2.0.0 page would be xref:2.0.0@camel-quarkus::reference/extensions/jms.adoc[].

Build with Yarn in Docker

To avoid having to install everything on your local machine or if you are a Mac or Windows user, you still have the possibility to launch the yarn commands directly inside a Docker container in which all the required tools have been preinstalled.

Assuming that Docker is properly installed on your local machine and the Docker daemon is started, all you need to do is to call the script yarn-in-docker.sh or yarn-in-docker.bat with the arguments that you would like to pass to the yarn command.

For example to launch yarn with build-all as argument inside a Docker container, the corresponding command is then:

$ ./yarn-in-docker.sh build-all

Equivalent for Windows

> yarn-in-docker.bat build-all

Note - The environment variables GITHUB_TOKEN and CAMEL_ENV locally set are automatically propagated to the Docker container.

Build with Maven

The project provides a simple way to build the website sources locally using the build tool Maven.

The Maven build automatically downloads the tool binaries such as node and yarn for you. You do not need to install those tools on your host then. The binaries are added to the local project sources only and generate the website content.

As the Maven build uses pinned versions of node and yarn that are tested to build the website you most likely avoid build errors due to incompatible versions of Node.js tooling installed on your machine.

Preparing Maven

Make sure that you have Maven installed.

$ mvn --version

If this command fails with an error, you do not have Maven installed.

Please install Maven using your favorite package manager (like Homebrew) or from official Maven binaries

Building from scratch

When building everything from scratch the build executes following steps:

• Download yarn and Node.js binaries to the local project
• Load required libraries to the local project using yarn
• Build the Antora Camel UI theme (antora-ui-camel)
• Fetch the doc sources from Camel and Camel K github reporsitories
• Build the website content using Hugo

You can do all of this with one single command:

$ mvn package

The whole process takes up to five minutes (time to grab some coffee!)

When the build is finished you should see the generated website in the public directory.

Rebuild website

When rebuilding the website you can optimize the build process as some of the steps are only required for a fresh build from scratch. You can skip the ui theme rendering (unless you have changes in the theme itself).

$ mvn package -Dskip.theme

This should save you some minutes in the build process. You can find the updated website content in the public directory.

Clean build

When rebuilding the website the process uses some cached content (e.g. the fetched doc sources for Camel and Camel K or the Antora ui theme). If you want to start from scratch for some reason you can simply add the clean operation to the build which removes all generated sources in the project first.

$ mvn clean package

Of course this then takes some more time than an optimized rebuild (time to grab another coffee!).

Checks, publishing the website

The content of the website, as built by the Camel.website job, is served from the asf-site branch and served by ASF Infrastrucuture team.

For the site to be published a number of checks need to pass, these include two levels of link checking: Antora xref and HTML link checker; HTML validation. In local development those checks can be run with yarn checks, to increase the local turnaround time some lengthy checks are not run unless CAMEL_ENV=production environment variable is set. To run all checks use:

$ CAMEL_ENV=production yarn checks

Publishing the website is done by ASF Jenkins this includes running all the checks. It is common that a check might fail there that hasn't failed when the change was made, this could be for any number of reasons, but most commonly there was a change in one of the subproject's documentation, and most common issue is an introduction of a broken or absolute link towards camel.apache.org domnain.

The configuration of the HTML validation rules is in .htmlvalidate.json, with exclusions of checks listed in .htmlvalidateignore file, custom rules are in rules.js for mandating relative links to camel.apache.org domain, JSON-LD schema validation, and mandating that the HTML title be set.

Pull request previews are powered by Netlify

This website is hosted by Apache Software foundation. Pull request previews and checks are powered by Netlify.

camel-website's Issues

Add Java version to the release files and display on download page

We need a column in the downloads for each of our projects, that has title Java
Then we can list the java version supported such as: 8, 11, 17

Camel 2.x will be: 8
Camel 3.x will be: 8, 11

Camel Kafka Connector will be: 8, 11

All the others will be: 11

Investigate how to keep old doc versions in the website without building them on every build.

It would be great to keep all the old doc versions accessible in our website:

• links from old blog posts would keep working
• external links would keep working
• Someone stuck on an old version could find the docs

However...

• building an ever growing set of doc versions on every build will soon take too long to be practical
• Without a way to hide obsolete doc versions, the component explorer will become unwieldy

In any case, I believe we have to rebuild the entire site when the UI changes or Antora is significantly upgraded.

At the moment we are using the standard Antora component explorer, which includes the whole explorer in every page. There are other solutions such as putting this stuff in a separate json document and rendering the json on each page. If we stick with the default component explorer I think we have to rebuild the entire site whenever we add a doc version, so the explorer on old pages can get to all versions.

For all other site builds, I think it should be possible to do something using a site manifest similar to the subproject-specific partial doc builds to only build "active" doc branches while using the already-built pages for the "inactive" doc branches. Together with setting up the partial builds, we'll need to figure out how to publish a complete site when only part of it has been "just built". Either we have to remove only the "old, active" pages when publishing the site, or find an independent way to archive the inactive pages and add them to the "built site" before committing.

Blog posts should always link to concrete versions of docs

There are many blog posts linking to "next" version, which is always wrong and will often cause broken links as the "next" docs evolve.

We should change all existing blog posts to link to concrete versions, or delink and note the version is no longer documented.

This might apply to the rest of the hugo-generated website, but I'm only aware of problems from the blog posts.

Some problem with social media overview

I've just published a tweet announcing the last blog published: https://twitter.com/Squ4k3/status/1461290119575621636

However the overview does not point to the description, and, in fact, the meta is falling back to the main page description instead:

<meta property="og:description" content="Camel is an open source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data.">

What is the proper way to handle the specific description?

/docs/ page should direct to latest, not next, versions of docs

Directing users to the 'next' documentation implies that the unreleased 'next' version of code can be used for purposes other than development of camel, against Apache policy.

Up to date data on the security page

On this page
https://camel.apache.org/security/

I wonder if we can add a "None reported" or something for 2021 so users can see that the table seems up to date. Otherwise they may wonder why there are nothing for 2021.

@zregvart

Enforce that blog posts must only link to concrete versions

Along with fixing the existing blog post links (#779), we need a check that will fail the build if a blog post links to "next' or "latest" (or unversioned).

Blog for releases stats Camel 3

Turned the stats on a thread on the Camel mailing list into a blog

See: https://lists.apache.org/thread/3tbw6qr4vpz47vby2gnnmyxt6nzf1997

Add link to Security Advisories more prominent

This page
https://camel.apache.org/security/

Is not easy to find, we should have a link so end users easily can find the list of known security advisories for Camel

@zregvart maybe you have a good idea to improve this

Move the published website to a separate repo

The local/partial builds idea requires everyone working on the docs (hopefully all camel devs :-) to clone and pull the camel-website repo moderately often. The main branch is pretty tiny compared with the published site, and although it's possible to just clone a single branch it's rather tricky.

It's confusing to pull and see a big update as the published site is completely replaced when there is no change in main.

It's a pretty simple change to publish to a separate repo rather than a branch in the current repo.

Publishing to a separate repo offers the ability to keep website history rather than only the current version (by squashing commits) without slowing down everything in sight, but whether to do that is a separate question.

Tune image output for social links

Right now the result would be like this:

We must work on improving the image (which is required to be 1080p to render automatically).

Originally posted by @squakez in #767 (comment)

Publish site manifest from actual website (cf local/partial builds)

If we publish the site manifest used in partial builds on the main website, one main problem with doing a quick local xref-check build is removed, namely the need to do a full local build at least once first.

This will require some changes to the local-build.sh scripts, including another "goal".

Default values are no longer shown in the options tables

Looking at
https://camel.apache.org/components/latest/aws2-s3-component.html

Then all the default values are no longer in the table. This is on the pre-release version (eg main branch).

@djencks wonder if you have seen this too.

Use next instead of pre-release display label

We have latest / pre-release today as indicator for the next version.

If we use pre-release then we have to keep updating the version, and the display is long.
Instead we can just use next which is what we mean. The next upcoming version.

Strange spacing for code label descriptions

When <1>,<2>,... is used in docs there's some strange formatting between the labels and their descriptions.
For example in infinispan and knative.
There's different spacing between the labels and descriptions in each list:

Reproducible on Firefox and Chromium, happens on multiple computers with different resolutions.

Weird error building website about logo-camel-medium.png

See
https://ci-builds.apache.org/job/Camel/job/Camel.website/job/main/507/console

@zregvart @djencks not sure if you have seen this or have any clue what the error may be?

Add more prominent link to the official Apache Camel group on linkedin

The official group
https://www.linkedin.com/groups/2447439/

We should add a link to this group in a more prominent place.
Maybe next to twitter icon in the top right corner, if we can find a icon in orange color scheme

Camel components doc page is scrolling when user is typing

When user is typing in the searchbar the page scrolls a little bit, it's not horrible but it's kinda funny.
See gif:

higlight.js deprecation warning

We have this warning displayed in the browser console:

Deprecated as of 10.6.0. initHighlightingOnLoad() deprecated. Use highlightAll() now.
https://camel.apache.org/_/js/vendor/highlight-cd2936b302.js:1

Faceted search

Algolia has the ability to support faceted search, we could take advantage of that to help website visitors to further refine the search results. Some ideas on likely facets:

• documentation type (user manual, component reference, particular sub-project)
• documentation version

For this it would be best that we replace the custom UI with Algolia provided UI.

Point to commits of repositories that go into the website

When we build the website we pull in files from several git repositories and several branches within them. The result of the website build is pushed to the asf-site branch on the camel-website repository and it is now squashed to conserve storage space in git. The commit message in the asf-site branch points to the commit on main branch of the camel-website repository that was the source of the build.

It would be useful to have that commit message point to all of the repositories (branches?) that contributed to the website at that point.

blog about dropping Java 8

We should make a little blog to say that java 8 is now dropped from all Camel projects / sub-projects going forward.

Camel 3.14.x LTS is the last release to support java 8 and that it has 2-years instead of 1-year lifecycle.

Add download for camel-k osx arm binary

https://camel.apache.org/download/

There is only x86 mac download on the download page at Apache

But on github we have also arm mac downloads
https://github.com/apache/camel-k/releases

`camel-kamelets` links broken

It appears that links which contains camel-kamelets are wrongly redirected. Ie https://camel.apache.org/camel-kamelets/next/ is redirecting to https://camel.apache.org/camel-k/next/amelets/next/ which is a 404.

I think it can be related to the change in redirect rules introduced in 93b65c8 wdyt @zregvart ?

Show a released kamelets catalog in the docs

At the moment, the docs strongly imply that kamelets are usable only by camel developers, since there's no docs for any released version. That's what I thought until recently. We should have the docs for every kamelet catalog version that is used by other subprojects, which certainly includes the latest release.

invalid links in docs

[check:links ] Found invalid urls in components/next/sql-component.html:
[check:links ] Fragment #sql-component-header-values at components/next/sql-component.html does not exist!
[check:links ] Found invalid urls in components/next/mongodb-component.html:
[check:links ] Fragment #_endpoint_header%3C/em%3Eid at components/next/mongodb-component.html does not exist!
[check:links ] Found invalid urls in camel-k/next/configuration/dependencies.html:
[check:links ] Fragment #configuring-registry at camel-k/next/cli/cli.html does not exist!
[check:links ] Found invalid urls in camel-k/next/cli/cli.html:
[check:links ] Fragment #configuring-registry at camel-k/next/installation/registry/registry.html does not exist!
[check:links ] Found invalid urls in blog/2022/04/camel-textual-debug-0.1.0/index.html:
[check:links ] Linked file at path blog/2022/04/camel-textual-debug-0.1.0/blog/2022/02/camel-textual-debug-eclipse-vscode does not exist!

Yarn cached dependencies are not correct for Jenkins build, and there are no instructions on how to update them

I don't understand yarn pnp/zero install and am getting results I don't understand running yarn install, so it would be a bad idea for me to try to update the yarn cache to match the recent changes in package.json.

It would be great if there was easy-to-find documentation about how to do this cache update, and whether it needs to be done on a linux system.

The existing website build instructions and mvn build seem to be out of date; mvn appears to be using yarn 1.22 and node 10.

Show kamelets on the download page

Leaving the latest release off the download page strongly implies that the subproject has never been released and should only be used for camel development. That was certainly my conclusion until quite recently.

Visual regression testing

A set of issues like #718 could be prevented by visual regression testing. There are SaaS offerings that have free test minutes and can integrate with GitHub actions like https://percy.io/. We could test few key pages like frontpage, a blog post, downloads page and some documentation pages.

Some pages are showing as 'white background' only

https://camel.apache.org/manual/getting-started.html
https://camel.apache.org/components/3.13.x/index.html
https://camel.apache.org/manual/writing-components.html

are all returning proper HTML but appear as white background...

Tried from multiple browsers (Chrome, Firefox, Edge)
and from multiple machines

other pages seem to look ok (homepage included)

Referential integrity among camel subprojects

1. In every subproject component descriptor, add attributes for the versions of dependency subprojects. Also add attributes indicating prerelease status and LTS expiration date as appropriate.
2. Make sure that all xrefs use these attributes to specify version when referring to another subproject.
3. Add something like this on every subproject index page after the header:

[NOTE]
--
This version ({page-component-display-version}) of {page-component-name} depends on:

* xref:{camel-version}@components::index.adoc[Camel] at version {camel-version}
* xref:{camel-k-version}@camel-k::index.adoc[] at version {camel-k-version}
* xref:{camel-kamelets-version}@camel-kamelets::index.adoc[] at version {camel-kamelets-version}

ifdef::lts[This long term service release will be supported until {lts}.]
ifndef::lts[]
ifdef::prerelease[This is the development version of{page-component-name}. It should not be used in production.]
ifndef::prerelease[This release will not be updated, but rather replaced by a new release.]
endif::[]
--

Provide script run during subproject release process that updates antora.yml and source-map.yml

camel-quarkus has tooling/scripts/update-antora-yaml.groovy which despite its uninformative name now updates the version and related info in antora.yml and the branch in source-map.yml during the release process.

We should add this to all the subprojects, it will considerably simplify the release process. It might need to be rewritten in go for camel-k.

This will also involve updating https://camel.apache.org/manual/release-guide-website.html.

User has to have a component category open for the search bar to work

When searching for a specific component in the docs you have to have the respective category open otherwise the search functionality won't find it. This is a big hassle when searching for something new and you are not sure what "category" in Camel it is.
See gif:

Kamelets missing from search

I've noticed that anything related to Kamelets is not showing up in search. This is because we added Kamelets documentation component without updating the docsearch configuration.

ckc release - wrong folder

This does not look right with that svn import folder

.

Upgrade Antora to 3.0.0-rc.2

There are some breaking changes to the newest Antora rc, requiring updating some of the extensions.

This will pave the way for a POC of quick partial local builds.

Remove outdated sub project versions in the documentation

We have outdated versions for CKC, Quarkus etc that should be removed

Create a check to make sure URLs remain backward compatible

When we re-organize content on the website we might accidentally shuffle the URLs around. We do check if within the website links are not broken, but we could also check if we changed any of the URLs so we know we broke links from other websites pointing to Camel and that we might need to add a redirect from old to the new content.
Perhaps looking up the current sitemap and the built sitemap on the pull request might be a way to check for any changes in the URL.

Update for release 3.13

Involves:
antora-playbook.yml

Camel:
main (instructions update)
3.13.x (update versions in antora.yml)
Camel spring boot:
3.13.x

Camel-karaf:
3.13.x (update versions in antora.yml)

Camel quarkus: (update camel release in pom.xml, generated changes)
main
2.4.x

Share buttons disappeared in Blog

I noticed Share buttons disappeared from a blog article. Is this something intentional or because some work is in progress on the website? Otherwise, I'd like to see them recovered.

https://camel.apache.org/blog/2021/12/api-management-infra/

Use cache bursting hash on resources from the Antora UI

Recently we updated the antora-ui-camel/src/img/brand-logos.svg, but the effect of that change might not be seen by all user agents depending on their cache state. The cache expires after 1 month, making the user agent skip fetching the resource until it expires, which could be a ~30 days from now.
For CSS we use cache bursting hash, we should use it for other resources within the Anotra UI so changes are immediately in effect.

Provide a way to do partial local subproject builds

Subprojects need to be able to check their documentation reasonably quickly without doing a full Antora build. The current approach of having customized playbooks for each subproject doesn't work.

One clear requirement is that local partial builds must be based on the actual camel-website full build.

I have a POC that I will discuss on the dev list.

Fix some yarn problems

• explicitly set nodeLinker to "pnp" to avoid interference from .yarnrc.yml in parent directories
• fail pr builds if yarn.lock has changed on install (using --immutable)
• allow checkins of dependency changes from non-linux systems by adding a yarn rebuild step to jenkinsfile.
  (the promise of yarn pnp seems to be partial: stuff in .yarn/unplugged is platform specific and breaks the build on other platforms)

I expect to add some scripts to root package.json to make some of this easier to maintain.

GitHub API rate limits on GitHub Actions

Seems that we're hitting the GitHub API rate limits on GitHub Actions, from a recent build:

➤ YN0000: ERROR 2022/02/09 07:47:51 Failed to get JSON resource "[api.github.com/repos/apache/camel-quarkus/issues?state=closed&milestone=10](https://api.github.com/repos/apache/camel-quarkus/issues?state=closed&milestone=10)": Failed to retrieve remote file: Forbidden, body: "{\"message\":\"API rate limit exceeded for 52.191.164.91. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)\",\"documentation_url\":\"[docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting\](https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting\)"}"
➤ YN0000: If you feel that this should not be logged as an ERROR, you can ignore it by adding this to your site config:
➤ YN0000: ignoreErrors = ["error-remote-getjson"]

We might need to set HUGO_PARAMS_GitHubUsername and HUGO_PARAMS_GitHubToken to a read-only GitHub token.

Re-order sub projects in version picker popup menu in the website

In the bottom left corner you can choose versions of docs. However the order should be improved

User Manual
Components
Camel Spring Boot
Camel Quarkus
Camel K
Camel Kafka Connector
Camel Karaf
Kamelet Catalog

Assemble .htaccess file from static component, Antora generated .htaccess, and derived rules

This

• Uses Antora to generate an .htaccess file, including 302 redirects from latest to
• Generates some more 302 redirects, basically from to
• Inserts the result into the static .htaccess

This allows us to:

• redirect from e.g. /components/activemq-component.html to /components/3.13.x/activemq-component.html
• drop the out of date 301s copied from an old camel-quarkus .htaccess
• use page-aliases everywhere.

I've updated test/redirect.sh to be correct and more complete and provided a script to run the httpd server easily.

Links in Kamelets yaml are part of the link checker job

In the last website build we have

Found invalid urls in camel-kamelets/next/jslt-action.html:
	Linked file at path /template.json does not exist!
Found invalid urls in camel-kamelets/next/jolt-transformation-action.html:
	Linked file at path /template.json does not exist!
Found invalid urls in camel-kamelets/next/json-schema-validator-action.html:
	Linked file at path /template.json does not exist!
Found invalid urls in camel-kamelets/next/json-patch-action.html:
	Linked file at path /template.json does not exist!
Found invalid urls in camel-kamelets/next/freemarker-template-action.html:
	Linked file at path /template.ftl does not exist!
Found invalid urls in camel-kamelets/next/jsonata-action.html:
	Linked file at path /template.spec does not exist!
ERROR: "check:links" exited with 1.

But those are just examples
https://github.com/apache/camel-kamelets/blob/main/kamelets/jslt-action.kamelet.yaml#L42

We need to skip checking or workaround the problem.

cc @zregvart

Commit the built Antora ui bundle so it's always available

The local/partial build setup needs the Antora UI bundle to be available.

Building it locally is at best a nuisance and on MacOS requires running in Docker or running yarn install to set up native dependencies for MacOS rather than the checked-in linux versions.

Another solution was proposed in #537, along with some variants, but I believe publishing the UI bundle as proposed there is significantly more complex for every use of the bundle with no redeeming advantages.

I'm happy with just manually committing it whenever it changes and we notice, since the official builds and previews will continue to build it just before using it, but it would be even better for the PR build or jenkins build to open a PR when it notices that the UI bundle has changed.

Use aggregate-collector to avoid symlinks; generate pages from json files

• Using @djencks/aggregate-collector we can avoid needing symlinks or copying: Antora can scan the project repos, find the relevant files, and map them into the Antora appropriate location so they are added to the content catalog.
• Using @djencks/asciidoctor-antora-indexer indexPage Antora extension we can generate fully-generated pages directly from json files and a template.

The initial set of PRs applies this to camel-spring-boot to avoid copying and to camel-kafka-connector to avoid page generation.
There are many other projects this can be used with.
At the moment the aggregate-collector can't eliminate all symlinks in main camel because the existing gulp file code scans the adoc sources for examples and symlinks only the code that is used. I think the aggregate-collector can be extended to do this scanning but that is for the future.

Blog about various debugging possibilities

• Camel Route Textual Debugging
• graphical with Eclipse Desktop Fuse Tooling and hawt.io
• classical Java debug: Processor usage and some specific place to place breakpoints in Camel source code
• classical Java debug with Camel K in VS Code and in other IDEs
• (adding trace log?)
• using jconsole directly?

mention differences between Camel 2.x, 3.0+<3.15 and 3.15+
mention when possible to do it remotely in a pre-production environment (and how)