neon-mmd / websurfx

:rocket: An open source alternative to searx which provides a modern-looking :sparkles:, lightning-fast :zap:, privacy respecting :disguised_face:, secure :lock: meta search engine

Readme | Discord | Instances | User Showcase | GitHub | Documentation



A modern-looking, lightning-fast, privacy-respecting, secure meta search engine (pronounced as websurface or web-surface /wɛbˈsɜːrfəs/.) written in Rust. It provides a quick and secure search experience while completely respecting user privacy.

Table of Contents

• Getting Started

  • 🔭 Preview
  • 🚀 Features
  • 🔗 Instances
  • 🛠️ Installation and Testing
  • 🔧 Configuration

• Feature Overview

  • 🎨 Theming
  • 🌍 Multi-Language Support

• Community

  • 📊 System Requirements
  • 🗨️ FAQ (Frequently Asked Questions)
  • 📣 More Contributors Wanted
  • 💖 Supporting Websurfx
  • 📘 Documentation
  • 🛣️ Roadmap
  • 🙋 Contributing
  • 📜 License
  • 🤝 Credits

Preview 🔭

Home Page

Search Page

404 Error Page

⬆️ Back to Top

Instances 🔗

For a full list of publicly available community driven websurfx instances to test or for daily use. see Instances

⬆️ Back to Top

Features 🚀

• 🎨 Make Websurfx uniquely yours with the twelve color schemes provided by default. It also supports the creation of custom themes and color schemes in a quick and easy way, so unleash your creativity!
• 🚀 Easy to setup with Docker or on bare metal with various installation and deployment options.
• ⛔ Search filtering to filter search results based on four different levels.
• 💾 Different caching levels focusing on reliability, speed and resiliancy.
• 🔐 Fast, private, and secure
• 🆓 100% free and open source
• 💨 Ad-free and clean results
• 🌟 and lots more...

⬆️ Back to Top

Installation and Testing 🛠️

For full setup instructions, see: Installation

Before you can start building websurfx, you will need to have Cargo installed on your system. You can find the installation instructions here.

To get started with Websurfx, clone the repository, edit the config file, which is located in the websurfx/ directory, and install the Redis server by following the instructions located here and then run the websurfx server and redis server using the following commands:

git clone https://github.com/neon-mmd/websurfx.git
cd websurfx
git checkout stable
cargo build -r
redis-server --port 8082 &
./target/release/websurfx

Once you have started the server, open your preferred web browser and navigate to http://127.0.0.1:8080 to start using Websurfx.

Note

1. The project is no longer in the testing phase and is now ready for production use.
2. There are many features still missing, like support for image search, different categories, quick apps, etc., but they will be added soon as part of future releases.

⬆️ Back to Top

Configuration 🔧

For full configuration instructions, see: Configuration

Websurfx is configured through the config.lua file, located at websurfx/config.lua.

⬆️ Back to Top

Theming 🎨

For full theming and customization instructions, see: Theming

Websurfx comes loaded with several themes and color schemes, which you can apply and edit through the config file. It also supports custom themes and color schemes using CSS, allowing you to make it truly yours.

⬆️ Back to Top

Multi-Language Support 🌍

Note

Currently, we do not support other languages, but we will start accepting contributions regarding language support in the future. We believe language should never be a barrier to entry.

⬆️ Back to Top

System Requirements 📊

At present, we only support x86_64 architecture systems, but we would love to have contributions that extend to other architectures as well.

⬆️ Back to Top

FAQ (Frequently Asked Questions) 🗨️

Why Websurfx?

The primary purpose of the Websurfx project is to create a fast, secure, and privacy-focused meta-search engine. There are numerous meta-search engines available, but not all guarantee the security of their search engines, which is critical for maintaining privacy. Memory flaws, for example, can expose private or sensitive information, which is understandably bad. There is also the added problem of spam, ads, and inorganic results, which most engines don't have a full-proof answer to. Until now. With Websurfx, I finally put a full stop to this problem. Websurfx is based on Rust, which ensures memory safety and removes such issues. Many meta-search engines also lack important features like advanced picture search, required by graphic designers, content providers, and others. Websurfx improves the user experience by providing these and other features, such as proper NSFW blocking and micro-apps or quick results (providing a calculator, currency exchanges, etc. in the search results).

Why AGPLv3?

Websurfx is distributed under the AGPLv3 license to keep the source code open and transparent. This helps keep malware, telemetry, and other dangers out of the project. AGPLv3 is a strong copyleft license that ensures the software's source code, including any modifications or improvements made to the code, remains open and available to everyone.

Why Rust?

Websurfx is based on Rust due to its memory safety features, which prevent vulnerabilities and make the codebase more secure. Rust is also faster than C++, contributing to Websurfx's speed and responsiveness. Finally, the Rust ownership and borrowing system enables secure concurrency and thread safety in the program.

⬆️ Back to Top

More Contributors Wanted 📣

We are looking for more willing contributors to help grow this project. For more information on how you can contribute, check out the project board and the CONTRIBUTING.md file for guidelines and rules for making contributions.

⬆️ Back to Top

Supporting Websurfx 💖

For full details and other ways you can help out, see: Contributing

If you use Websurfx and would like to contribute to its development, we're glad to have you on board! Contributions of any size or type are always welcome, and we will always acknowledge your efforts.

Several areas that we need a bit of help with at the moment are:

• Better and more color schemes: Help fix color schemes and add other famous color schemes.
• Improve evasion code for bot detection: Help improve code related to evading IP blocking and emulating human behaviors located in everyone's engine file.
• Logo: Help create a logo for the project and website.
• Docker Support: Help write a Docker Compose file for the project.
• Submit a PR to add a new feature, fix a bug, update the docs, add a theme, widget, or anything else.
• Star Websurfx on GitHub.

⬆️ Back to Top

Documentation 📘

Note

We welcome any contributions to the documentation as this will benefit everyone who uses this project.

⬆️ Back to Top

Roadmap 🛣️

Coming soon! 🙂.

⬆️ Back to Top

Contributing 🙋

Contributions are welcome from anyone. It doesn't matter who you are; you can still contribute to the project in your own way.

Not a developer but still want to contribute?

Check out this video by Mr. Nick on how to contribute.

Developer

If you are a developer, have a look at the CONTRIBUTING.md document for more information.

⬆️ Back to Top

License 📜

Websurfx is licensed under the AGPLv3 license.

⬆️ Back to Top

Credits 🤝

We would like to thank the following people for their contributions and support:

Contributors

Stargazers

⬆️ Back to Top

---

Thank you for Visiting

websurfx's Issues

Save all the configuration setting from the setting UI to a cookie when save button is pressed

Add functionality to save all the configurations configured by the user as a cookie when the save button is pressed on the settings page.

Ability to put the config file under different paths

Add the ability to put the config files under different paths like ~/.config/websurfx/config.lua, /etc/xdg/websurfx/config.lua and under the project folder path under websurfx/ and also add code to properly handle each case like if the file is not available under ~/.config/websurfx/ then it fallback to using /etc/xdg/websurfx/config.lua if it is not available there then it fallback to using the project folder's config.lua which is located at webusrfx/config.lua and finally if it is not available there then it displays an appropriate error.

The code for this should look roughly like this:

fn handle_file_paths() -> Result<&'static str, Box<dyn Error>> {
    if Path::new(
        format!(
            "{}/.config/websurfx/config.lua",
            std::env::var("HOME").unwrap()
        )
        .as_str(),
    )
    .exists()
    {
        Ok("~/.config/websurfx/config.lua")
    } else if Path::new("/etc/xdg/websurfx/config.lua").exists() {
        Ok("/etc/xdg/websurfx/config.lua")
    } else if Path::new("./websurfx/config.lua").exists() {
        Ok("./websurfx/config.lua")
    } else {
        Err(format!("Config file not found!!").into())
    }
}

This will allow the project to be more easily deployable and installable on different Linux distros as well as it allows the project to be easily packaged for different Linux distros.

🔧 Cache next page on search

Why?

When searching, the backend only requests the current page, but it should also get the result for the next page. This would avoid extra loading time when going to the next page as results are already cached.

How?

We can simply create an async task that is run after the requests is done, which will fetch the results, cache them, so that they can be served on requested without delay.

Put the project logo in the website mainpage

Put a logo in the project main page in the area highlighted in the image below by first adding the image under this path public/images and then editing the file public/templates/index.html to replace the image path from images/fps_logo.png to images/websurfx_logo.png.

✨ Disallow user from searching content disallowed via `safesearch`

Provide code to disallow users from searching sensitive content or any content disallowed by the user. When safe search is turned on and set to level 4, either from the config file or in the settings page in the UI (Something similar to how Swisscows search engine disallows users to search sensitive content).

For example:
When you search some sensitive content in Swisscows search engine, the content is blocked and an appropriate error message displayed to the user as to why it was blocked (as shown in the screenshot below).

Patch and improve `Docker` deployment process

Improve and fix the docker deployment by editing the Dockerfile in the project folder (codebase) to make the workdir path relative and fix the locations of the files to be copied from and to.

FROM rust:latest AS chef
# We only pay the installation cost once,
# it will be cached from the second build onwards
RUN cargo install cargo-chef

-WORKDIR app
+WORKDIR /app

FROM chef AS planner
COPY . . 
RUN cargo chef prepare --recipe-path recipe.json

FROM chef AS builder
COPY --from=planner /app/recipe.json recipe.json
# Build dependencies - this is the caching Docker layer!
RUN cargo chef cook --release --recipe-path recipe.json

# Build application
COPY . .
RUN cargo install --path .

# We do not need the Rust toolchain to run the binary!
FROM gcr.io/distroless/cc-debian11
-COPY --from=builder ./public/ ./public/
-COPY --from=builder ./websurfx/ ./websurfx/
+COPY --from=builder /app/public/ /opt/websurfx/public/
+COPY --from=builder /app/websurfx/config.lua /etc/xdg/websurfx/config.lua
COPY --from=builder /usr/local/cargo/bin/* /usr/local/bin/
CMD ["websurfx"]

Note

1. To get started contributing make sure to read the contributing.md file for the guidlines on how to contribute in this project
2. To contribute first fork this project by following this video tutorial if you
   are not familliar with process and add your changes and make a pull request with the changes to this repository and if you
   are new to GitHub then follow this video tutorial to get started
   contributing 🙂 .

Improve async multithreading for aggregation

Currently, the source code does fetch results from upstream engines concurrently but it does not take into account if more engines are added in the future and also it doesn't take into account limited resources, So by using tokio::spawn add code to run asynchronous code in parallel.

Here is the code snippet that needs attention it is located in the file src/search_results_handler/aggregator.rs:

// --- code above ---
    
    // Add a random delay before making the request.
    let mut rng = rand::thread_rng();
    let delay_secs = rng.gen_range(1..10);
    std::thread::sleep(Duration::from_secs(delay_secs));

    // fetch results from upstream search engines simultaneously/concurrently.
    let (ddg_map_results, searx_map_results) = join!(
        duckduckgo::results(query, page, &user_agent),
        searx::results(query, page, &user_agent)
    );

    let ddg_map_results: HashMap<String, RawSearchResult> = ddg_map_results?;
    let searx_map_results: HashMap<String, RawSearchResult> = searx_map_results?;

//  --- code below ---

Include `Criterion` crate in the toml file

Add the criterion crate in the Cargo.toml file located in the codebase (project folder).

[package]
name = "websurfx"
version = "0.13.0"
edition = "2021"
description = "An open-source alternative to Searx that provides clean, ad-free, and organic results with incredible speed while keeping privacy and security in mind."
repository = "https://github.com/neon-mmd/websurfx"
license = "AGPL-3.0"

[dependencies]
reqwest = {version="*",features=["json"]}
tokio = {version="*",features=["full"]}
serde = {version="*",features=["derive"]}
handlebars = { version = "4.3.6", features = ["dir_source"] }
scraper = {version="*"}
actix-web = {version="4.3.1"}
actix-files = {version="0.6.2"}
serde_json = {version="*"}
fake-useragent = {version="*"}
env_logger = {version="0.10.0"}
log = {version="0.4.17"}
rlua = {version="*"}
redis = {version="*"}
md5 = {version="*"}
rand={version="*"}
once_cell = {version="*"}
error-stack = {version="0.3.1"}

[dev-dependencies]
rusty-hook = "^0.11.2"
+ criterion = "0.5.1"

[profile.dev]
opt-level = 0
debug = true
split-debuginfo = '...'
debug-assertions = true
overflow-checks = true
lto = false
panic = 'unwind'
incremental = true
codegen-units = 256
rpath = false

[profile.release]
opt-level = 3
debug = false
split-debuginfo = '...'
debug-assertions = false
overflow-checks = false
lto = 'thin'
panic = 'unwind'
incremental = false
codegen-units = 16
rpath = false
strip = "debuginfo"

Put three dashes in the first line

Put 3 dashes at the top of all the YAML files which are listed below:

• .github/workflows/contributors.yml
• .github/workflows/hello.yml
• .github/workflows/labels.yml
• .github/workflows/mega-linter.yml
• .github/workflows/rust.yml
• .github/workflows/rust_format.yml
• .github/workflows/stale.yml
• .mega-linter.yml
• docker-compose.yml

Example on how to add 3 dashes at the top:

In the .gitpod.yml the 3 dashes are added in the top and then after that all the code starts.

Note

1. To get started contributing make sure to read the contributing.md file for the guidlines on how to contribute in this project
2. To contribute first fork this project by following this video tutorial if you
   are not familliar with process and add your changes and make a pull request with the changes to this repository and if you
   are new to GitHub then follow this video tutorial to get started
   contributing 🙂 .

Ability to have themes on different paths

Add the ability to have the themes folder (public folder) be placed on different paths which include /opt/websurfx and project folder (codebase) with fallbacks as a way to ensure nothing fails similar to issue #44 but for themes folder.

Replace hashes with appropriate links

Replace the hashes with appropriate links which will redirect the user to the expected places in the file public/templates/footer.html in the codebase (project folder).

-<footer>
-    <div>
-       <span>Powered By <b>Websurfx</b></span><span>-</span><span>a lightening fast, privacy respecting, secure meta
-            search engine</span>
-    </div>
-    <div>
-        <ul>
-            <li><a href="#">Source Code</a></li>
-            <li><a href="#">Issues/Bugs</a></li>
-        </ul>
-    </div>
-</footer>
-<script src="static/settings.js"></script>
-</body>
-
-</html>
+<footer>
+    <div>
+       <span>Powered By <b>Websurfx</b></span><span>-</span><span>a lightening fast, privacy respecting, secure meta
+            search engine</span>
+    </div>
+    <div>
+        <ul>
+            <li><a href="https://github.com/neon-mmd/websurfx">Source Code</a></li>
+            <li><a href="https://github.com/neon-mmd/websurfx/issues">Issues/Bugs</a></li>
+        </ul>
+    </div>
+</footer>
+<script src="static/settings.js"></script>
+</body>
+
+</html>

Note

1. To get started contributing make sure to read the contributing.md file for the guidlines on how to contribute in this project
2. To contribute first fork this project by following this video tutorial if you
   are not familliar with process and add your changes and make a pull request with the changes to this repository and if you
   are new to GitHub then follow this video tutorial to get started
   contributing 🙂 .

Create a settings page to allow users to select the search engines from the UI

Create an Ui to select different configuration options like themes, colorschemes and different engines to choose from with the help of checkboxes and also provide a select all and deselect all button to select/deselect all engines at once.

Remove warnings about unused assignments

There is a warning message page_url is assigned but never read, we need to resolve it.

Update the docs to explain the new logging option and bump up the version.

Update the docs under docs/ folder to explain the newly added config file option logging and also bump the app version.

Improve and restructure readme to look attractive

Need to rewrite and reformat readme to look more attractive with icons, images and gifs and with styling like putting images in center.

Example projects with awesome readmes:

• https://github.com/Lissy93/dashy/blob/master/README.md
• https://github.com/vlang/v/blob/master/README.md

Setup a config file for `Gitpod`

Write a Gitpod config file to allow contributors to quickly setup a full-fleged coding environment for making contributing to the project easier and smoother experience.

Handle "reqwest" Errors

Use an enum to handle reqwest errors by adding an enum something like this:

enum ReqwestError{
  NotFound,
  Timeout,
  Forbidden,
  AccessDenied,
  TooManyRequests
}

Getting the Html content from the upstream and grabbing the data and if an error occurs or if an error is found on the page then returning an appropriate enum variant.

Provide a debug option in the config

Provide a debug option in the config file. This option when set to true will give a verbose log and also it will disable random delays.

debug = true

Provide more famous colorschemes

Provide more famous colorschemes like Tokyonight by adding them under the public/static/colorschemes folder located in the codebase (project folder).

To add more colorschemes you will need to add the following code in the new colorscheme file:

:root{
  --bg: <background color>;
  --fg: <foreground color (text color)>;
  --1: <color 1>;
  --2: <color 2>;
  --3: <color 3>;
  --4: <color 4>;
  --5: <color 5>;
  --6: <color 6>;
  --7: <color 7>;
}

Change the variable values with the colors you want to add in the above-provided code snippet.

Automate Contributor list generation

At present the contributor list is being provided by contrib.rocks but this solution seems to have problems like it doesn't update that often which makes the contributor list under the credits section of Readme lag behind as shown in the image below. Provide a way to automate the GitHub contributors list using an appropriate Github action that runs every day on a fixed schedule for example at 10 pm every day.

Here you can see the actual number of contributors is 8 but in the Readme, it shows only 5 contributors:

Note: The Github you choose should provide the ability to provide rounded avatars in the contributors' list.

The process of generating random user agents is abnormally time-consuming.

I added logs before and after random_user_agent() and found that its processing time even exceeded 10 seconds. Is this expected?

Make improvements to `colorschemes` to make it look right

Currently, there are 12 colorschemes available for the website, located under the public/static/colorschemes/ folder in the project. These colorschemes allow the user to change the color of the theme being used on the website. They all are written in CSS and have 9 variables (as shown below with an example colorscheme) in each colorscheme file to play with. At present all colorschemes except the five colorschemes catppuccin-mocha, dracula, tokyo-night, one-dark and dark-chocolate requires a fix.

:root {
  --bg: #44475a;
  --fg: #8be9fd;
  --1: #ff5555;
  --2: #50fa7b;
  --3: #ffb86c;
  --4: #bd93f9;
  --5: #ff79c6;
  --6: #94a3a5;
  --7: #ffffff;
}

For reference, you can take a look at the color palettes located here and here.

Note

1. To get started contributing make sure to read the contributing.md file for the guidlines on how to contribute in this project
2. To contribute first fork this project by following this video tutorial if you
   are not familliar with process and add your changes and make a pull request with the changes to this repository and if you
   are new to GitHub then follow this video tutorial to get started
   contributing 🙂 .

Add code of conduct

Need to add a code of conduct file to ensure safety of all people contributing to this project.

Rename variables' name to make them kebab cased

Rename all variables' name located in all the files located under public/static/colorschemes/ folder and also rename all the places where they are being used (referenced) in the file public/static/themes/.

The variables that need changing their name are:

• --fg change it to --foreground-color
• --bg change it to --background-color
• --1 change it to --color-one
• --2 change it to --color-two
• --3 change it to --color-three
• --4 change it to --color-four
• --5 change it to --color-five
• --6 change it to --color-six
• --7 change it to --color-seven

Note

1. To get started contributing make sure to read the contributing.md file for the guidlines on how to contribute in this project
2. To contribute first fork this project by following this video tutorial if you
   are not familliar with process and add your changes and make a pull request with the changes to this repository and if you
   are new to GitHub then follow this video tutorial to get started
   contributing 🙂 .

Provide logging option in the config

Provide a logging option that when set to true enables logging in standard output but when set to false disables it. This option is important for privacy reasons as this will give users the ability to select servers that have these options set to false which will give them a sense of relief.

logging = true

Optimise code for large scale server use

Optimise code by using tokio async to aggregate search results from each upstream server concurrently/simultaneously with random delay in between each simultaneous requests.

For example:
If there are 4 threads than four simultaneous requests could be made to each server and fetch their result scrape them and aggregate their results and then add a random delay and continue with another 4 and so and so forth.

Example code:

use std::error::Error;
use tokio::time::delay_for;
use reqwest::Client;

#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
    let client = Client::new();
    let urls = vec![
        "https://example.com/page1",
        "https://example.com/page2",
        "https://example.com/page3",
    ];

    let mut tasks = Vec::new();
    for url in urls {
        tasks.push(tokio::spawn(async move {
            let resp = client.get(url).send().await?;
            let body = resp.text().await?;
            // Process the HTML body here
            Ok(())
        }));
    }

    for task in tasks {
        task.await??;
    }

    Ok(())
}

Add status badges in the readme

Currently the readme (which is written in markdown) does not have any status badges but the readme should clearly show a few status badges these are:

• Build status (showing the github workflow status)
• Issue and pull request trackers (showing the number of open issues and pull requests )
• License (showing which license is the project using)
• Rating (showing the number of stars for this project on github)
• Repo size (showing the size of this repository)

These all build statuses can be added using the shields.io website.

In markdown file html code can be added something like this:

<a href="https://github.com/neon-mmd/websurfx/graphs/contributors">
   <img src="https://contrib.rocks/image?repo=neon-mmd/websurfx" />
</a>

Note: All contributions should go into the rolling branch of this repository and for before you contribute fork this project add your appropriate changes to your fork and then add a PR mentioning this issue.

Automate all checks for `linting/formatting` for different language using a single GitHub action

Automate the process of checking all formatting, linting and spelling issues using a GitHub action called MegaLinter.

The linters/formatters/spell checker that should be included:

• Linters
  • Lua -- luacheck
  • Rust -- clippy
  • JavaScript -- eslint
• Formatters
  • Css -- stylelint
  • Markdown -- markdownlint
  • Yaml -- yamllint
  • Html -- djlint
• Tooling
  • actions -- actionlint
  • dockerfile -- hadolint
• Spell Checker
  • spelling -- cspell

Readme Revision

Revise the readme to keep it on par with the current changes. These include adding and changing the following things:

• Remove unnecessary shields from the Readme
• Update the first preview image in the readme if possible replace the images with a Gif
• Fix Back to Top button by updating the link to where it points.
• Update the Why Websurfx section content under the FAQ section from this:

The primary purpose of the Websurfx project is to create a fast, secure, and privacy-focused meta search engine. While there are numerous meta search engines available, not all of them guarantee the security of their search engine, which is critical for maintaining privacy. Memory flaws, for example, can expose private or sensitive information, which is never a good thing. Rust is used to write Websurfx, which ensures memory safety and removes such issues. Many meta search engines also lack important features like advanced picture search, which is required by many graphic designers, content providers, and others. Websurfx attempts to improve the user experience by providing these and other features, such as proper NSFW blocking.

to:

The primary purpose of the Websurfx project is to create a fast, secure, and privacy-focused meta-search engine. While there are numerous meta-search engines available, not all of them guarantee the security of their search engine, which is critical for maintaining privacy. Memory flaws, for example, can expose private or sensitive information, which is never a good thing. Also, there is the added problem of Spam, ads, and unorganic results which most engines don't have the full-proof answer to it till now but with Websurfx I finally put a full stop to this problem, also, Rust is used to write Websurfx, which ensures memory safety and removes such issues. Many meta-search engines also lack important features like advanced picture search, which is required by many graphic designers, content providers, and others. Websurfx attempts to improve the user experience by providing these and other features, such as proper NSFW blocking and Micro-apps or Quick results (like providing a calculator, currency exchanges, etc in the search results).

I faced an issue when I tried to build this project.

I cloned this project and run the following commands.

git clone https://github.com/neon-mmd/websurfx.git
cd websurfx
git checkout stable
cargo build -r

I faced error when I run this command.
cargo build -r

What happened?

error[E0635]: unknown feature proc_macro_span_shrink

Can anybody help me with my issue?

Revise the docs to explain the new debug option.

Update the docs under docs/ folder to explain the newly added config file option debug proposed in issue #45 .

Why is Lua used for writing configurations?

Why are we using Lua for configuration instead of other formats like YAML? I don't have any experience with Lua, so I'm curious about the benefits of using it for configuration files.

For example, I would like to introduce configurations similar to the following:

aggregator:
  random_delay: true
  engines:
    duckduckgo
    searx

I am unsure how to implement this with Lua, and it may be more straightforward with YAML.

Patch cross site scripting (XSS) vulnerability in HTML

Fix two cross-site scripting (XSS) vulnerabilities that have been found to be rooted in the file src/public/templates/search.html, making the necessary modifications to resolve the issue without disrupting the functionality of the search page on the website.

Provide a fix for `Contributor List` github action

Provide a fix to the GitHub action to generate a contributors' list by fixing the problems which have appeared in the logs and in the image shown below.

Logs: https://github.com/neon-mmd/websurfx/actions/runs/5161278677/jobs/9298133910

Correct the spelling in the comments

Fix the spelling of the word finishes in the comments (which all starts with a double slashes in front of the sentences) in the file public/static/cookies.js in the codebase (project folder).

Note

1. To get started contributing make sure to read the contributing.md file for the guidlines on how to contribute in this project
2. To contribute first fork this project by following this video tutorial if you
   are not familliar with process and add your changes and make a pull request with the changes to this repository and if you
   are new to GitHub then follow this video tutorial to get started
   contributing 🙂 .

Optimize build profiles for development and production

Configure different cargo.toml profile options to optimize builds for releases and also to optimize the build for development purposes, also add necessary information under the package section of cargo.toml.

Improve `error-handling` by using `error-stack` crate

At present, The code to handle errors associated with the engines contains a lot of duplication and so crates like error-stack will help reduce a lot of the boilerplate code also using crates like error-stack will allow the app to provide more verbose logs as proposed in #83 and also provide better, prettier and cleaner for contributors to read and work on.

Rename getter functions' name

Rename all the getter functions' name in accordance to the rust naming convention, and also rename the places where they are being used (referenced).

The functions that need changing their name are:

• get_cached_json() located in file ‎src/cache/cacher.rs change it to cached_json()
• get_config_path() located in file src/config/parser.rs change it to config_path()
• get_results() located in file ‎src/server/routes.rs change it to results()
• get_public_path() located in file src/handler/public_paths.rs change it to public_path()

Note

1. To get started contributing make sure to read the contributing.md file for the guidlines on how to contribute in this project
2. To contribute first fork this project by following this video tutorial if you
   are not familliar with process and add your changes and make a pull request with the changes to this repository and if you
   are new to GitHub then follow this video tutorial to get started
   contributing 🙂 .

Documentation Revision

Revise the documentation to make the following changes:

• In the installation page add a section about installation instructions for Archlinux users.
• Update the configuration page to add and explain new options recently added.
• Fix the Go back to home button in the documentation pages.
• Update the Why Websurfx? section on the faq page to reflect the same changes as in #56.
• Update the theming page to show the code about the About Page under the section Themes > Custom.

Change the font case

Change the font case for the word duckduckgo from Duckduckgo to DuckDuckGo in the file public/templates/engines_tab.html in the codebase (project folder).

Note

1. To get started contributing make sure to read the contributing.md file for the guidlines on how to contribute in this project
2. To contribute first fork this project by following this video tutorial if you
   are not familliar with process and add your changes and make a pull request with the changes to this repository and if you
   are new to GitHub then follow this video tutorial to get started
   contributing 🙂 .

Error: Cannot Find Git Executable in Gitpod Workspace

Encountered an error when trying to start a Gitpod workspace. The error message states, Oh, no! Something went wrong! Cannot find git executable, make sure it is installed as part of Gitpod image: exec: 'git': executable file not found in $PATH.

Gitpod config - https://github.com/neon-mmd/websurfx/blob/rolling/.gitpod.yml

Write a docker compose file to allow self-hosting of the server in docker container

Need to write a docker compose file which allows users to setup their server in a docker container.

Make about page centered

Put some margins on the right and left of the content of the About Page to make it centered on the website to make it visually look better by using percentage values for the margins instead of using fixed values like rem, em, px, etc.

The target file that needs to be changed is located under public/static/themes and it is called simple.css.

Should we move the documentation from the wiki section to the "docs" folder in the codebase??

I think having the documentation under the wiki section makes it go unnoticed most of the time because of that sometimes it remains out of sync with the changes that are being made which is bad as you know so I feel having it under the docs folder will make it more noticeable. What do you say @xffxff @alamin655 ??.

Write an about page for the project in the website with appropriate css styling

At present the about page of the website just contains dummy text and it needs to show a clear and concise description about the project in the website. The about page's html code (which uses handlebars templating) is located under the public/templates/ folder in the project and its css stylesheet is located under public/static/themes folder.

Note: All contributions should go into the rolling branch of this repository and before you contribute make sure you have cc-cli installed on your system otherwise you can install by following the installation instructions here fork this project add your appropriate changes to your fork and then add a PR mentioning this issue.

Add documentation pages in the wiki

Need to write different documentation pages about theming, installation, configuration, etc in the wiki and also make empty links in the project to point to their respective sections int the wiki

Make logs more verbose when put in debug mode

Provide more verbose logs to view in-depth what is happening in the app when the app is put in debug mode by setting debug option to true in the config file.

For debug logging put code something similar to this:

    if config.debug {
        log::info("{}","write an appropriate message")
    }

This will only allow some logs to be displayed when the app is put in debug mode.

Improve code to better evade bot detection while scraping

The code needs some extra improvements like adding cookie header with appropriate cookies for each upstream search engine and also adding other headers to better spoof and protect against bot detection and better emulate human behavior and also evade IP blocking/banning.

Files seeking changes are:

• src/engines/duckduckgo.rs
• src/engines/searx.rs

Add format checking to the CI job?

We currently don't verify formatting in our CI jobs, but perhaps it would be beneficial to incorporate this into our CI jobs

Reorder config file options and make headings stand out

Put the option production_use under the config file's server section and make config file section headings stand out so that they are more visible to users as a heading in the config file rather than regular comments.