toltec-dev / toltec

Community-maintained repository of free software for the reMarkable tablet.

Toltec

Toltec is a community-maintained repository of free software for the reMarkable tablet.

Installation and Use

Please refer to the Toltec website for up-to-date information on how to install and use Toltec.

Building

This Git repository contains all the tools and recipes required to build the packages published on the package repository. This repository is automatically built and published every time that a commit is pushed to Git, using Github Actions. Since all the packaged software in Toltec is free, you can also build them from source yourself instead of using the pre-built binaries. The build process is fully reproducible, which means that you can verify that the published packages have not been tampered with during the automated build process.

Learn how to build the repository from source →

Contributing

Your contribution is welcome for adding new packages, updating existing ones or improving the build tooling.

Learn how to contribute to Toltec →

toltec's Issues

Add usage documentation

• How to install opkg
• How to add the toltec repository to opkg
• How to update the list of packages
• How to install, remove, upgrade packages
• Which channel to choose (stable/testing)

Add draft configuration file and icon to calculator

Use HEAD request instead of GET when checking if package exists on remote server

this will save many megabytes

Define and automatically enforce a code style

• Define a common code style for the project (e.g. as a shfmt config).
• Add a make lint target for checking code style issues and optionally automatically fixing them (e.g. using shfmt).
• Add a step in the pr workflow for automatically checking code style.
• Create a contributing guide in which this code style is mentioned.

harmony package should likely not be in `utils` section

what section does it belong in?

Remove conflict declaration with xochitl.service from remux

They can run alongside each other perfectly fine, so they should not be declared as conflicting. In fact, the Conflicts= declaration in this case is inconsistent with the After= declaration which also contains xochitl.service.

feature: prevent installation of rM2 incompatible apps on rM2

remux package should not recommend people disable xochitl, this caused someone on discord to break their rM2

it should be possible to prevent packages from being installed on rM2 if they are incompatible

Announce the repo

From what I can tell, we should probably do a proper rollout of this to the larger community. Here is what I think we need to do first before we can "announce":

• Flush out documentation on how to create packages (#18).
• Flush out user documentation for how to install/update (#19).
• Get the stable repo actually stable.
• Sort out the process (and document it) for getting things from testing to stable (#20).
• Update the community wiki (remarkablewiki.com) with information on this (And probably some of the standards we are starting to form).
• Document philosophy/principles (#16).
• Add security policy (#56 )
• Add privacy policy (#58)

After that we should probably do the following to "announce":

• Put together a nicer front page (or have it redirect to this github) (#21, #22).
• Post to the Subreddit.
• Post to the Facebook group.

Add packaging guidelines

• Description of the ipk format.
• Description of the recipe format.
• Choosing a correct version number.
• Choosing a section.
• How to build and test a package.

Split rmkit in sub-packages

Package requests

• koreader (AGPL-3.0)
• recrossable (GPL-2.0) #90
• remarkable-splash (MIT)
• rmWacomToMouse (MIT)
• VNSee (GPL-3.0)
• reStream (MIT) #184
• keywriter (MIT) #195
• Xochitl templates (#222)
  • https://github.com/bliekp/remarkable-templates (CC0-1.0)
  • https://github.com/oisinmulvihill/reMarkableTemplates (GPL-3.0)
• ddvk-hacks (MIT) #247
• wikipedia (MIT) #309 #311
• gocryptfs (MIT) #318
• remarkable-stylus (Lamy Pen) (LGPL-3.0) #383
• 1bitpaint (MIT) #83
• uinput (GPL-2.0) (Added to base OS)
• ReMarkable.NET (MIT) parzivail/ReMarkable.NET#1
• rM-vnc-server (GPL-2.0) (pl-semiotics/rM-vnc-server#7)
• pocket2rm (MIT)
• Crazy Cow (MIT)
• remarkable_news (GPL-3.0) Evidlo/remarkable_news#21
• remarkable-touchgestures (MIT)
• RemarkableLamyEraser (GPL-3.0)
• mmc-utils #490
• remarkable-encryption (MIT) RedTeamPentesting/remarkable-encryption#1
• DirectFB (LGPL-2.1)
• srvfb (Apache 2.0) (discontinued)
• regenesis (MIT)
• remarkable-calendar-creator (MIT)

Add icon to nao

Add icon to harmony

rename generic apps

• calculator
• minesweeper
• ???

Define a process for moving packages from testing to stable

• See how Debian does it.
• See how Arch Linux does it.

How to list packages from remarkable sections?

we should have a quick way of only listing packages from toltec repo, this is first step towards UI for package management

Maintainer scripts vs. systemd services

i think these scripts should not contain only "echo" statements, still. i'd like to re-enable them for remux, since remux
only starts/stops the remux service (and not xochitl)

Add a project home page

Visiting https://toltec.delab.re should show content similar to this Git repository’s README.

Add builddepends metadata field to recipes

(As proposed by @Eeems on Discord.)

Some recipes require specific packages that are not provided by the Toltec Docker images. Currently, these are calculator and plato. We should add a builddepends field, similar to Arch’s makedepends, which contain be a list of Debian packages required for the build() function.

Support partial rebuilds

eventually, build times are going to balloon. It would be nice to have the build server retrieve the list of already built packages and only build changed packages

Add draft configuration and icon to fingerterm

Add support for per package docker images

Add philosophy/principles

there are a few principled decisions about the repo that should be made clear / explicit in the repository's README. i'm thinking about things like:

• only free (with free software licenses) software
• the repo is aiming to build software without relying on rm's toolchain so all packages should build without it
• ???

Prevent accidental inclusion of source archive in final packages

Having the zip file with the source inside of the source directory will be problematic when having binary distributions as they usually just zip the entire src folder and don't realize that their src folder has this file added.

This issue first was found when adding koreader #44 (comment) . It seems to have been forgotten and the zip is currently still included in the .ipk package on testing.

We should consider either removing the file or moving it e.g. one level above to the final .ipk file.

package koreader

Move toltec to a GitHub organization

I also propose that the individual repos should go under a single organization name (vs. putting them into matteo namespace).
Originally posted by @raisjnn in #29 (comment)

---

I also propose that the individual repos should go under a single organization name (vs. putting them into matteo namespace).

I agree that it’s best if the repository is not tied to my personal GitHub account. Do you suggest creating an org specific to toltec or joining an existing org like @reHackable (if that’s even possible)?

Originally posted by @matteodelabre in #29 (comment)

check that pull request build logs have the tree output in them

(I haven't been able to find it yet, but I see tree output when building locally)

Replace recipe metadata variables by associative array

Currently, recipes declare a handful of metadata variables such as pkgname, url and source which have more or less generic names. This causes two problems:

• We must take care of not overwriting them inside the build scripts.
• shellcheck is not able to automatically detect that those variables are defined, unless we declare them manually.

I propose placing all these variables in a single Bash associative array to solve the above two problems. Here’s an example of how that could look like for plato.

Before:

pkgname=plato
pkgdesc="Document reader"
url=https://github.com/LinusCDE/plato
pkgver=0.9.1-8
timestamp=2020-09-21T18:40Z
section=readers
maintainer="Linus K. <[email protected]>"
license=AGPL-3.0-or-later

image=rust:v1.1
source=(https://github.com/LinusCDE/plato/archive/9b18b76769751e7aa760ae3c31ff8dd263911b8d.zip)
sha256sums=(478333e0f32096471950ef3f1f76d7cc42f6802dc417227518b6ffc5593b4457)

After:

declare -A metadata=(
    [name]=plato
    [description]="Document reader"
    [url]=https://github.com/LinusCDE/plato
    [version]=0.9.1-8
    [timestamp]=2020-09-21T18:40Z
    [section]=readers
    [maintainer]="Linus K. <[email protected]>"
    [license]=AGPL-3.0-or-later

    [image]=rust:v1.1
    [sources]=https://github.com/LinusCDE/plato/archive/9b18b76769751e7aa760ae3c31ff8dd263911b8d.zip
    [sha256sums]=478333e0f32096471950ef3f1f76d7cc42f6802dc417227518b6ffc5593b4457
)

Then we can:

• Retrieve a piece of metadata: ${metadata[name]}
• Check if an attribute is defined: [[ -v metadata[nonexisting] ]]
• Define a default value for an attribute: [[ ! -v metadata[nonexisting] ]] && metadata[nonexisting]="default value".

Remove old package versions from published repository

move bsdtar installation into docker image

it seems like bsdtar is being installed in the github actions / workflow, but this missing dep is also required for local builds

Add toltec-bootstrap package to allow updating bootstrap code

Right now if we modify the bootstrap script, we have to somehow ask users to get the latest and to run it. If we made it a package that gets installed by the base install, it would allow us to push out changes to the script without worry. Packages could also depend upon the toltec-bootstrap package if they rely on new features we push out with it.

Rework pre/post scripts

I think we should abstract away a bunch of the complexity of the pre/post scripts. Ideally we should have the following scripts that a package can define:

• preinstall
• postinstall
• preremove
• postremove
• preupgrade
• postupgrade

The tooling that builds a package should automatically create the wrapper script that would be required to determine which of these scripts should be run at what time (see #55 (comment))

Sort the sections alphabetically in the generated index page

The list of existing sections is computed using a Bash associative array. The keys of that array are then enumerated to generate the list of sections in the index page. The ordering appears to change with each build, which is probably because Bash is using a non-deterministic hashtable of some kind for storing associative arrays. It would be best to ensure that the order is consistent across builds by sorting the list of sections alphabetically.

Plato build is not reproducible

Plato generates a different package when building locally and when building using GitHub Actions. The offending file seems to be /opt/etc/draft/icons/plato.png.

Verify source integrity through SHA-256 sums

Currently, the source code of each package is fetched by cloning a Git repository and checking out a specific commit referenced by its SHA-1 identifier. This SHA-1 sum serves both as an identifier of the targeted revision and as an integrity check of the fetched source. Three caveats:

• By cloning a Git repository, we also clone its complete history, which is not needed for building the package.
• We might encounter apps that do not distribute their source code through Git.
• SHA-1 can be attacked.

An approach which solves these three points is to use the same approach for fetching sources as chosen by Arch (see PKGBUILD#Sources and PKGBUILD#Integrity), as suggested by @Eeems in https://github.com/matteodelabre/toltec/issues/8#issuecomment-689049849.

Move the Docker images to a separate Git repository

Oops, I didn’t foresee that building from a different branch than stable or testing could cause this kind of problem. The package build script uses an heuristic that pulls the Docker image that has the same tag as the current Git branch (so if you build from stable, it’ll use toltec-qt:stable, and from testing it’ll use toltec-qt:testing). The reasoning behind that was that changing the Dockerfiles in testing should not immediately affect the build in stable. Here it seems that you’re building from a branch called oxide, but of course there’s no Docker image tagged like that in the published repository, which explains the error.

I think this could be solved by moving the images to a different Git repo, versioning them properly and including that version into the image= field of each package. I’ve been thinking about doing that for a few days anyway, because a nice side effect of this strategy is that it makes it easier for other projects to reuse the images.

Do you or @raisjnn have any opinion on this?

Originally posted by @matteodelabre in #29 (comment)

---

@matteodelabre I think that's a good idea.

Originally posted by @Eeems in #29 (comment)

---

I think that idea is fine, but there is a step "publish-images" that will fail for everyone who doesn't have access to those repos.

Originally posted by @raisjnn in #29 (comment)

---

I think that idea is fine, but there is a step "publish-images" that will fail for everyone who doesn't have access to those repos.

I assume you’re referring to the GitHub workflow job that builds and publishes the images whenever a Dockerfile is updated. This part of the workflow will be moved to the new repo hosting the images, so this should not cause any failure. Pulling the pre-built images will always work even for people who don’t have committer access to the repo.

Originally posted by @matteodelabre in #29 (comment)

Automatically open issues when upstream publishes a new release

See #16 (comment)

build step doesn't run if there is no image

Either we need to update our documentation to be clearer that build only happens if image= is populated, or still run it for simple things like patching files.

Pull from sources other than Git (including binary sources)

Instead of using git repos, what if a dev wants to publish binaries + their checksums? This might be useful down the line but not just yet

Automate re-enabling of Toltec after system upgrades

when opkg/toltec is installed and there is a system upgrade, we need a way to re-enable opkg. the entware package comes with entware_reinstall which re-creates the opt mounting system service. i believe our bootstrap script (thanks linusCDE!) automatically re-enables the opkg as well. (is the bootstrap script saved locally for re-use after upgrade?)

currently, we do not have any hooks into the remarkable's upgrade process to automatically re-enable opkg.

options:

• try to get hook in upgrade process
• add something to .bashrc (or similar) that notifies user when they ssh in that opkg isn't enabled and print way to re-enable it
• ???

Create a nicer index page for repositories

Visiting https://toltec.delab.re/stable or https://toltec.delab.re/testing should show a nicely formatted list of packages sorted by category, with the following information for each package:

• Name and link to project home page
• License
• Download link
• Size
• SHA-256 sum

This index page should be automatically generated through GitHub Actions.

[bug] Description of packages aren't showing up in opkg info of package

other packages have their description showing up, unsure what's going on here

Handle building KOReader

We may need to push out EBFs specific to the reMarkable outside the normal KOReader release cycle. In order to do that, it would be good to have reproducible builds instead of relying on their binary releases.

Explore packaging the kernel

Since we can access the kernel and dtb file under /boot, we technically could deploy updates to the kernel with opkg. With recovery mode and the second partition, we have options to save the device if something goes wrong.

This will allow us to add modules to the kernel, or to tweak the device tree to allow for things like disabling certain buttons before suspending.

Add icon to mines

feature: allow pointing to custom github names in scripts/lib image-name()

currently matteodelabre is hardcoded, preventing others from mirroring this repository (and having the github actions work). this should be configurable by user somehow (github secrets?)

Add CI to PRs

it would be really nice if when there is a PR, it kicks off a github action that figures out what packages are affected and builds them, verifying that there are no obvious build errors.

a package is affected if 1) it was modified or 2) its image (or image ancestor) was modified

Automatically check that PRs increment the version number of packages they affect

oxide v2.0~alpha is not reproducible

make repo-check indicates a difference inside the /opt/bin/erode and /opt/bin/oxide binaries.

Split packages

Sometimes several packages can be generated from a single source. For example, harmony, nao, mines, remux and simple all come from the rmkit Git repo. It would make packaging simpler if we could combine the four recipes into a single one, à la Arch split packages.