sharktheone / arch-mojo

Install Mojo on Arch

Hello 👋 I am Shark

Skills

arch-mojo's Issues

Install Script Depends On Modular

In the install.py script modular is needed. Doesn't the script install modular?

Reading the MODULAR_AUTH from argument or .env

Hi, this is a PR not an issue.

I think it could be better if the {MODULAR_AUTH} is read as an argument similar to the prposed way at Modular wesite, or from an ENV file, somethink like the below:

import os
import sys

# Check if the token is provided as a command line argument
if "--modular-token" in sys.argv:
    token_index = sys.argv.index("--modular-token") + 1
    if token_index < len(sys.argv):
        token = sys.argv[token_index]
    else:
        print("Error: No token provided after --modular-token")
        exit(1)
else:
    # Check if the token is set in the environment variable
    token = os.environ.get("MODULAR_AUTH")

    # If not, check if there's a .env file with MODULAR_AUTH variable
    if token is None and os.path.isfile(".env"):
        with open(".env", "r") as env_file:
            for line in env_file:
                key, value = line.strip().split("=")
                if key == "MODULAR_AUTH":
                    token = value

# If token is still not obtained, notify the user and exit
if token is None:
    print("Error: Modular auth token not provided.")
    print("Please provide the token using --modular-token, set the MODULAR_AUTH environment variable, or include it in a .env file.")
    exit(1)

# authenticate in modular
authenticated = "user.id" in subprocess.run(["modular", "config-list"], capture_output=True).stdout.decode("utf-8")

# rest of the script

Ehat do you think?

Dockerfile

I'm trying to build a docker file for mojo lang, it was done smoothly using Ubuntu vase image as:

FROM ubuntu:latest

ARG DEFAULT_TZ=Asia/Gaza
# A random default token
ARG AUTH_KEY=mut_a2fxxxxxxxxxxxxxx18f
ARG MODULAR_HOME="/root/.modular"

ENV MODULAR_HOME=$MODULAR_HOME
ENV PATH="$PATH:$MODULAR_HOME/pkg/packages.modular.com_mojo/bin"

RUN apt-get update \
   && DEBIAN_FRONTEND=noninteractive TZ=$DEFAULT_TZ apt-get install -y \
    tzdata \
    curl \
    git \
    wget && \
    rm -rf /var/lib/apt/lists/*

# Download the latest version of minicoda py3.8 for linux x86/x64.
RUN curl -fsSL https://repo.anaconda.com/miniconda/$( wget -O - https://repo.anaconda.com/miniconda/ 2>/dev/null | grep -o 'Miniconda3-py38_[^"]*-Linux-x86_64.sh' | head -n 1) > /tmp/miniconda.sh \
       && chmod +x /tmp/miniconda.sh \
       && /tmp/miniconda.sh -b -p /opt/conda

ENV PATH=/opt/conda/bin:$PATH

RUN conda init

RUN curl https://get.modular.com | sh - && \
    modular auth $AUTH_KEY 

RUN pip install --upgrade pip
RUN modular clean
RUN modular install mojo

EXPOSE 3000

# Change permissions to allow for Apptainer/Singularity containers
RUN chmod -R a+rwX /root

ENTRYPOINT ["tail"]
CMD ["-f","/dev/null"]

And I tried to do the same based on archlinux base image, but I struggled in that, I updated the install.py file to be:

import os
import shutil
import subprocess
import sys
import urllib.request

# TODO use shutil to copy files

arch = "x86_64-linux-gnu"


def param(name: str):
    try:
        return os.environ[name]
    except:
        return None


home = param("HOME")
if home is None:
    home = "~"

WORKING_DIR = "~/.local/arch-mojo/"

mojo_lib_path_from_home = ".local/lib/mojo"
mojo_lib_path = f"{home}/{mojo_lib_path_from_home}"
token = "mut_a2xxxxxxxxxxxxxxxxxxxxxxx18f"

modular = shutil.which("modular") is not None

authenticated = False
if modular:
    authenticated = "user.id" in subprocess.run(["modular", "config-list"], capture_output=True).stdout.decode("utf-8")

WORKING_DIR = WORKING_DIR.replace("~", param("HOME"))
if WORKING_DIR[-1] != "/":
    WORKING_DIR += "/"

try:
    os.makedirs(WORKING_DIR)
except FileExistsError:
    pass
 
# install modular if not installed
if not modular:
    # download PKGBUILD
    urllib.request.urlretrieve("https://raw.githubusercontent.com/Sharktheone/arch-mojo/main/PKGBUILD",
                               f"{WORKING_DIR}PKGBUILD")
    os.system(f"cd {WORKING_DIR} && makepkg -si")

# authenticate in modular
if not authenticated:
    os.system(f"LD_LIBRARY_PATH=$LD_LIBRARY_PATH:{mojo_lib_path} modular auth {token}")

# download ncurses lib

urllib.request.urlretrieve("https://ftp.debian.org/debian/pool/main/n/ncurses/libncurses6_6.4-4_amd64.deb",
                           f"{WORKING_DIR}libncurses.deb")

urllib.request.urlretrieve("https://ftp.debian.org/debian/pool/main/libe/libedit/libedit2_3.1-20221030-2_amd64.deb",
                           f"{WORKING_DIR}libedit.deb")

os.system(f"cd {WORKING_DIR} && ar -vx libncurses.deb && tar -xf data.tar.xz")
os.system(f"cd {WORKING_DIR} && ar -vx libedit.deb && tar -xf data.tar.xz")


# copy libs
os.system(f"sudo cp {WORKING_DIR}lib/{arch}/libncurses.so.6.4 /lib/libncurses.so.6.4")
os.system(f"sudo cp {WORKING_DIR}usr/lib/{arch}/libform.so.6.4 /usr/lib/libform.so.6.4")
os.system(f"sudo cp {WORKING_DIR}usr/lib/{arch}/libpanel.so.6.4 /usr/lib/libpanel.so.6.4")
os.system(f"sudo cp {WORKING_DIR}usr/lib/{arch}/libedit.so.2.0.70 /usr/lib/libedit.so.2.0.70")

os.system("sudo ln -s /lib/libncurses.so.6.4 /lib/libncurses.so.6")
os.system("sudo ln -s /usr/lib/libform.so.6.4 /usr/lib/libform.so.6")
os.system("sudo ln -s /usr/lib/libpanel.so.6.4 /usr/lib/libpanel.so.6")
os.system("sudo ln -s /usr/lib/libedit.so.2.0.70 /usr/lib/libedit.so.2")

# install mojo
mojo = shutil.which(f"PATH=$PATH:{home}/.modular/pkg/packages.modular.com_mojo/bin/ mojo") is not None
if mojo:
    print("Mojo is already installed... cleaning up")
    os.system(f"PATH=$PATH:{home}/.modular/pkg/packages.modular.com_mojo/bin/ modular clean")

os.system(f"LD_LIBRARY_PATH=$LD_LIBRARY_PATH:{mojo_lib_path} modular install mojo")

# fix crashdb directory not found:
os.makedirs(f"{home}/.modular/crashdb", exist_ok=True)


def rc_path():
    return f"{param('HOME')}/.bashrc"

rc_pth = rc_path()

if rc_pth is None:
    print("Skipping rc file installation")
    exit(0)

rc_file = open(rc_pth, "a")
shell_rc = open(rc_pth, "r").read()

# check if exports are already in rc file
if param("LD_LIBRARY_PATH") is None or \
        f"~/{mojo_lib_path_from_home}" not in param("LD_LIBRARY_PATH") \
        or mojo_lib_path not in param("LD_LIBRARY_PATH"):
    rc_file.write(f"export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/{mojo_lib_path_from_home}\n")

if param("PATH") is None \
        or "~/.modular/pkg/packages.modular.com_mojo/bin/" not in param("PATH") \
        or f"{home}.modular/pkg/packages.modular.com_mojo/bin/" not in param("PATH"):
    rc_file.write("export PATH=$PATH:~/.modular/pkg/packages.modular.com_mojo/bin/\n")
rc_file.close()

And wrote the Dockerfile as:

# Use an existing docker image as a base
FROM archlinux/archlinux:base

# Set timezone
ARG DEFAULT_TZ=Asia/Gaza
ENV TZ=$DEFAULT_TZ

# Update Archlinux core and extra / Install necessary tools
RUN pacman -Syu --noconfirm curl

# Install tzdata and set timezone
RUN pacman -Syu --noconfirm tzdata && \
    ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone && \
    pacman -Scc --noconfirm

# Install necessary tools
RUN pacman -Syu --noconfirm curl wget grep python

# Download the latest version of minicoda py3.8 for linux x86/x64.
RUN curl -fsSL https://repo.anaconda.com/miniconda/$( wget -O - https://repo.anaconda.com/miniconda/ 2>/dev/null | grep -o 'Miniconda3-py38_[^"]*-Linux-x86_64.sh' | head -n 1) > /tmp/miniconda.sh \
       && chmod +x /tmp/miniconda.sh \
       && /tmp/miniconda.sh -b -p /opt/conda

ENV PATH=/opt/conda/bin:$PATH

RUN conda init

RUN conda create -n venv python=3.10

# Upgrade pip and install necessary Python packages
RUN python -m ensurepip --upgrade && \
    pip install --upgrade pip setuptools wheel

# Define an environment variable for the auth token
ENV MODULAR_TOKEN=mut_a2ff2b6187ae4aa885abe2010053518f

# Define an environment variable for MODULAR_HOME
ENV MODULAR_HOME /home/tempuser/.modular

# Install necessary tools
RUN pacman -S fakeroot binutils sudo --noconfirm
# python python-pip
# Create a new user, switch to it, and set a password
RUN useradd -m tempuser
RUN echo 'tempuser:password' | chpasswd

# Add tempuser to the wheel group
RUN usermod -aG wheel tempuser

# Grant sudo privileges to the wheel group
RUN echo '%wheel ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers

USER tempuser

# Copy the install.py file from your local machine to the Docker image
COPY install.py .

# Use the auth token in the command
# Use 'yes' to automatically answer 'Y' to any prompts
RUN yes | python install.py

#RUN python -m venv /home/tempuser/venv
#RUN source /home/tempuser/venv/bin/activate
RUN modular clean
RUN modular install mojo

EXPOSE 3000

# Change permissions to allow for Apptainer/Singularity containers
RUN chmod -R a+rwX /root

ENTRYPOINT ["tail"]
CMD ["-f","/dev/null"]

But i got the below at the last stage:

111.3 [notice] A new release of pip is available: 23.0.1 -> 23.3.2
111.3 [notice] To update, run: pip install --upgrade pip
115.3 [mojo] Testing: `mojo build test_mandelbrot.mojo` [FAIL]
115.3 [mojo] Testing: `mojo build test_python.mojo` [FAIL]
115.3 [mojo] Testing: `mojo repl` [FAIL]
115.3 [mojo] Some components may have been installed successfully, but others may not work as expected. Please submit an issue to https://github.com/modularml/mojo and include the full output of the command you just ran.
117.2 modular: error: failed to run python: 
117.2 # Found release for https://packages.modular.com/mojo @ 0.6.1
117.2 # Installing to /home/tempuser/.modular/pkg/packages.modular.com_mojo
117.2 # Downloading artifacts. Please wait...
117.2 # Downloads complete, setting configs...
117.2 # Configs complete, running post-install hooks...
------
Dockerfile.arch:65
--------------------
  63 |     #RUN source /home/tempuser/venv/bin/activate
  64 |     RUN modular clean
  65 | >>> RUN modular install mojo
  66 |     
  67 |     EXPOSE 3000
--------------------
ERROR: failed to solve: process "/bin/sh -c modular install mojo" did not complete successfully: exit code: 1

 *  The terminal process "/usr/bin/bash '-c', 'docker build --pull --rm -f "Dockerfile.arch" -t mojo-arch:latest "."'" terminated with exit code: 1. 
 *  Terminal will be reused by tasks, press any key to close it.

Can you help in this please. thanks

Update modular to `0.3.0`

Currently modular 0.3.0 has some issues with AMD CPUs. It outputs

modular: error: no packages found for the current target hardware - please run `modular host-info` and file a ticket at https://github.com/modularml/mojo with your hardware information

The they are currently investigating this in modularml/mojo#1492

I'll update as soon as this is fixed.

ERROR: Failure while downloading - curl: (22) The requested URL returned error: 404

Looks like modular has moved the resource?

python <(curl -sSL https://raw.githubusercontent.com/Sharktheone/arch-mojo/main/install.py) 
==> Making package: modular 0.4.1-1 (Sat 17 Feb 2024 08:41:44)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Downloading modular-0.4.1-amd64.deb...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0   324    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404
==> ERROR: Failure while downloading https://dl.modular.com/public/installer/deb/debian/pool/any-version/main/m/mo/modular_0.4.1/modular-0.4.1-amd64.deb
    Aborting...
Please enter your Modular auth token:

Authentication not working.

Trying to run the script (had to modify the command to look like this: curl -sSL https://raw.githubusercontent.com/Sharktheone/arch-mojo/main/install.py | python) gives the following error:

==> Installing package modular with pacman -U...
loading packages...
resolving dependencies...
looking for conflicting packages...

Packages (1) modular-0.2.0-1

Total Installed Size:  45,74 MiB

:: Proceed with installation? [Y/n] ==> WARNING: Failed to install built package(s).
Please enter your Modular auth token: Traceback (most recent call last):
  File "<stdin>", line 77, in <module>
EOFError: EOF when reading a line

I am on manjaro linux.

==> ERROR: install file (modular.install) does not exist or is not a regular file. /bin/sh: line 1: modular: command not found Failed to authenticate modular using token: ''

I am using arch linux.

I used python <(curl -sSL https://raw.githubusercontent.com/Sharktheone/arch-mojo/main/src/install.py) but got the error
==> ERROR: install file (modular.install) does not exist or is not a regular file.
/bin/sh: line 1: modular: command not found
Failed to authenticate modular using token: ''

Starting from clean install (manually deleted all modular/mojo related folders) it fails somehow. Maybe somebody is encountering the same issue

Starting from clean install (manually deleted all modular/mojo related folders) it fails somehow. Maybe somebody is encountering the same issue

$ mkdir -p /tmp/arch-mojo && curl -sSL https://raw.githubusercontent.com/Sharktheone/arch-mojo/main/install.py -o /tmp/arch-mojo/install.py && python /tmp/arch-mojo/install.py
Do you have modular already installed? (y/n): n
Do you want to install the libraries globally (for all users)? (y/n): n
Do you want to automatically use a venv when running modular install/update? (y/n): y
Where do you want to create the venv?: zsh

(...)

* `/home/mrk/.modular/pkg/packages.modular.com_mojo/bin/mojo`...
TEST: `mojo --help`... OK
TEST: `mojo run --help`... OK
TEST: `mojo build test_mandelbrot.mojo`... OK
TEST: `mojo build test_python.mojo`... OK
TEST: `mojo demangle`... OK
TEST: `mojo format`... OK
TEST: `mojo package`... OK
TEST: `mojo test_mandelbrot.mojo`... OK
TEST: `mojo test_python.mojo`... OK
== stdout ==

== stderr ==
/home/mrk/.modular/pkg/packages.modular.com_mojo/bin/lldb: error while loading shared libraries: libpanel.so.6: cannot open shared object file: No such file or directory

(...)

Originally posted by @mdmrk in #1 (comment)

Issue with the symbolic links for the libraries used globally

Hi, As I mentioned before the script is working fine with me, I installed it in multiple tries, and all are fine, but just once once I read the code, I understood the reason why out of the sudden I started getting the below everytime I install or remove an app, it looks it is related to this installation code:

(1/1) removing go                                                                  [###############################################] 100%
ldconfig: /usr/lib/libpanel.so.6 is not a symbolic link

ldconfig: /usr/lib/libmenu.so.6 is not a symbolic link

ldconfig: /usr/lib/libform.so.6 is not a symbolic link

ldconfig: /usr/lib/libedit.so.2 is not a symbolic link

ldconfig: /usr/lib/libncurses.so.6 is not a symbolic link

I was able to fix this by:

1. Remove the existing non-symbolic link files as:

sudo rm libpanel.so.6 libmenu.so.6 libform.so.6 libedit.so.2 libncurses.so.6

2. Recreating symbolic links using the ln command:

sudo ln -s libpanel.so.6.0 libpanel.so.6
sudo ln -s libmenu.so.6.0 libmenu.so.6
sudo ln -s libform.so.6.0 libform.so.6
sudo ln -s libedit.so.2.0 libedit.so.2
sudo ln -s libncurses.so.6.0 libncurses.so.6

3. After recreating the symbolic links, running ldconfig to update the library cache, as:

sudo ldconfig

Can you fix the code to handle this properly?

Note
I asked the same to ChatGPT, and it proposed the below:

import os
import shutil
import subprocess
import sys
import urllib.request

# ... (previous code)

# Function to create symbolic links for shared libraries
def create_symbolic_links(lib_name, lib_version, lib_path):
    lib_file = f"{lib_name}.so.{lib_version}"
    lib_link = f"{lib_name}.so.{lib_version.split('-')[0] or lib_version}"
    lib_full_path = f"{lib_path}/{lib_file}"

    # Check if the symbolic link already exists
    if not os.path.exists(f"{lib_path}/{lib_link}"):
        os.system(f"sudo ln -s {lib_full_path} {lib_path}/{lib_link}")

# ... (previous code)

# copy libs
if install_global:
    os.system(f"sudo cp {WORKING_DIR}lib/{arch}/* /lib/")
    os.system(f"sudo cp {WORKING_DIR}usr/lib/{arch}/* /usr/lib/")
    os.system(f"sudo cp {WORKING_DIR}lib/{arch}/* /usr/lib/")
else:
    os.system(f"mkdir -p {mojo_lib_path}")

    os.system(f"cp {WORKING_DIR}lib/{arch}/libncurses.so.6.4 {mojo_lib_path}/libncurses.so.6")
    os.system(f"cp {WORKING_DIR}/usr/lib/{arch}/libform.so.6.4 {mojo_lib_path}/libform.so.6")
    os.system(f"cp {WORKING_DIR}/usr/lib/{arch}/libpanel.so.6.4 {mojo_lib_path}/libpanel.so.6")
    os.system(f"cp {WORKING_DIR}/usr/lib/{arch}/libedit.so.2.0.70 {mojo_lib_path}/libedit.so.2")

    # Create symbolic links for shared libraries
    create_symbolic_links("libncurses", "6.4-4", mojo_lib_path)
    create_symbolic_links("libform", "6.4", mojo_lib_path)
    create_symbolic_links("libpanel", "6.4", mojo_lib_path)
    create_symbolic_links("libedit", "2.0.70", mojo_lib_path)

# ... (rest of the code)

Failure writing output to destination

Hi, I'm at Manjaro, and got the mentioned error:

 $ python --version 
Python 3.11.5
 $ curl -sSL https://raw.githubusercontent.com/Sharktheone/arch-mojo/main/install.py -o /tmp/arch-mojo/install.py && python /tmp/arch-mojo/install.py

encountered an error

I encountered an error can you help me
system :- arch linux
shell :- fish
after running this got an error when checking mojo -v it says unknown command please help

input

curl -sSL https://raw.githubusercontent.com/Sharktheone/arch-mojo/main/src/install.py > /tmp/install.py
python /tmp/install.py --dir=/tmp/arch-mojo

output

# Please visit this URL in your browser: https://developer.modular.com/device?userCode=HTRP-FSVT
# Waiting for confirmation...
x - debian-binary
x - control.tar.xz
x - data.tar.xz
x - debian-binary
x - control.tar.xz
x - data.tar.xz
/bin/sh: line 1: /home/abhinav/.modular/pkg/packages.modular.com_mojo/bin/modular: No such file or directory
Mojo is already installed... cleaning up⏎   

Error related to virtual environment

Hey, first of all thanks for creating this script! Its really a pity the developers of mojo don't distribute platform independent.

I wanted to use your script to install mojo and issues the command

mkdir -p /tmp/arch-mojo && curl -sSL https://raw.githubusercontent.com/Sharktheone/arch-mojo/main/install.py -o /tmp/arch-mojo/install.py && python /tmp/arch-mojo/install.py
Do you have modular already installed? (y/n): n
Do you want to install the libraries globally (for all users)? (y/n): y
Do you want to automatically use a venv when running modular install/update? (y/n): y
Where do you want to create the venv?: ~/Apps/arch-mojo

which lead to the following error

modular: error: failed to run python: Executable "$_self.package_path/venv/bin/python" doesn't exist!

Has anybody faced a similar issue and maybe even got a solution at hand?

Failed to build modular. One or more files did not pass the validity check.

Pamac update log:

Preparing...
Synchronizing package databases...
Cloning modular build files...
Generating modular information...
Checking modular dependencies...
Resolving dependencies...
Checking inter-conflicts...

Building modular...
==> Making package: modular 0.5.1-1 (Sun 03 Mar 2024 09:48:05 PM EET)
==> Checking runtime dependencies...
==> Checking buildtime dependencies...
==> Retrieving sources...
  -> Found modular-v0.5.1-amd64.deb
==> Validating source files with sha256sums...
    modular-v0.5.1-amd64.deb ... FAILED
==> ERROR: One or more files did not pass the validity check!
Failed to build modular

reading ~/.zshrc fails

When running the script, it fails to read ~/.zshrc.

Traceback (most recent call last):
File "/home/zyper/.cache/yay/mojo-git/src/install.py", line 187, in
shell_rc = open(rc_pth, "r").read()
^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xab in position 10263: invalid start byte

Because (my) .zshrc is an ISO-8859 text file, which is not compatible with utf-8, hence the error.
However, writing still works, so I suggest removing the line, as shell_rc isn't used.

Working with Jupyter notebook

I'm not sure where shall I post this, but thought this repo may be the most proper one.
After installing mojo ar archcraft using this repo and getting it running, I was able to access the mojo kernel through the jupyter notebook as well, but once I tried accessing the same kernel through jupyter plugin in the VS code I got the error:
OSError: libpanel.so.6: cannot open shared object file: No such file or directory. as hown in the screenshoot below: