ERROR: Failed building wheel for gym: wheel.vendored.packaging._tokenizer.ParserSyntaxError: Expected end or semicolon (after version specifier) about super-ml-pets

[priyamalart]

gym installation works ==> gym@https://github.com/andreped/gym/releases/download/v0.21.0-binary/gym-0.21.0-py3-none-any.whl

Any procedure involved to release your change?

[priyamalart]

I misunderstood, the correction will be published to https://pypi.org/project/gym/#history.
Now, all issues are sorted out. Thanks for your support.

[andreped]

Yes, this looks identical to the issue mentioned here.

I could try to see if I could fix gym myself on my own fork.

[andreped]

Went down a small rabbit hole and found something surprising. There is a breaking issue with gym>=0.22 (see here), which sb3 depends on. Hence sb3 has a strict dependency to only support gym==0.21, which seems to have all kinds of fun issues.

Hence, upgrading gym is not an option.

However, setting setuptools==66 should resolve the issue. I observed something similar on colab and I managed to fix it by installing pip install setuptools==66 before running pip install -r requirements.txt (see here for more information).

It works on colab now but I will see if it passes the unit tests. Could you try to install the correct version of setuptools before trying to install the remainder of the deps?

[andreped]

I made a gist here for you to try. At least it works there.

[andreped]

All CI jobs were successful (see here). It also works fine on google colab (see here).

Are you sure you ran pip install setuptools==66 before installing any dependencies? Did you create a virtual environment?

Lastly, it might be that upgrading pip resolves the issue. In the colab, I'm using a very new version of pip. Try upgrading to latest:

pip install --upgrade pip

[priyamalart]

@andreped : setuptools were added in requirement.txt. It was installed alng with other python modules.
And, Iam using python:3.8 container image, And we havent created any virtual env.

Let me try to upgrade the pip.

[priyamalart]

Working Solution:
pip3 install --no-cache-dir --no-deps -r requirements.txt --timeout 60 --> Here, setuptools==66 was used. pip upgrade was not done.

Non-Working Solution:
pip3 wheel --no-cache-dir --no-deps --wheel-dir /wheels -r requirements.txt --timeout 60

Why we use wheels?
Machine learning based container images are very huge.
So, we use the python image to install wheels and then only wheels are copied to final image. This reduces the final image size.

Ex:

FROM python:3.8 as builder
RUN mkdir -p /ws/
WORKDIR "/ws/"
COPY src/requirements.txt requirements.txt
RUN pip3 install --upgrade pip
&& pip3 install --no-cache-dir --no-deps setuptools==66 --timeout 60
&& pip3 wheel --no-cache-dir --no-deps --wheel-dir /wheels -r requirements.txt --timeout 60
&& rm -rf /ws
WORKDIR "/"

FROM python:3.8-slim as runner
RUN mkdir -p /ws/
WORKDIR "/ws/"
COPY --from=builder /wheels /wheels
RUN pip install --no-cache-dir --no-deps /wheels/*
&& rm -rf /wheels
&& rm -rf /ws

[andreped]

We use docker as well when we run the CI tests. It is still strange to me why it didn't work for you while it seems to work most other places.

As you use docker, it might be that you are missing some key dependency that resolves this.

[priyamalart]

But, in our case, Error is seen in this step:
pip3 wheel --no-cache-dir --no-deps --wheel-dir /wheels -r requirements.txt --timeout 60

Even for us, when same gym module is installed through "pip3 install -r requirements.txt", installation is okay.
But, using wheels fails.

[andreped]

Aaah, I see. But is it only through the Dockerfile-install that it fails or does it work outside docker?

[andreped]

Just tried running this line on my windows laptop:

pip3 wheel --no-cache-dir --no-deps --wheel-dir /wheels -r requirements.txt --timeout 60

Works fine. However, in my case most of the wheels were already.

Perhaps it is easier if I build a wheel for sapai-gym myself and use that one instead in super-ml-pets. Then you do not need to compiled your own wheels during installation. I'm testing this now here.

[priyamalart]

Aaah, I see. But is it only through the Dockerfile-install that it fails or does it work outside docker?

I didnt try outside docker. All our builds are with Docker only.

[andreped]

I have now built precompiled wheel and made a new release for sapai-gym (see here).

All CIs seem to work (see here).

Hence, if you try cloning/pull latest commit, I believe it might work. At least building gym should not be an issue any longer.

Let me know how it goes, @priyamalart. I might be less responsive throughout easter.

[priyamalart]

Within docker, we use 2 OS.

1. Ubuntu --> Installation is okay
2. Python3.8 images based on debian. --> Issue is seen here.
   I might have to understand, the package difference between these base OS and python modules.

[andreped]

Did you try pulling latest and trying again? Do you get a different error?

We don't build gym anymore. It is precompiled and distributed in the sapai-gym wheel, if I did everything correctly.

---

EDIT: If you could share the exact same Dockerfile you are using, I could try to see if I could reproduce the error and fix it on my end.

[andreped]

Just tried to setup super-ml-pets on a personal windows laptop and I ran into the same issue still. Preinstalling setuptools==66 resolves the issue.

Thus, precompiling wheel for sapai-gym is not sufficient. We will need to precompile wheel for gym. I could make an attempt.

[andreped]

I have now precompiled wheels for sapai-gym and gym, so maybe you will be more lucky this time around. Try to pull latest and try again.

[andreped]

pip3 wheel --no-cache-dir --no-deps --wheel-dir /wheels -r requirements.txt --timeout 60

I also tried running this command locally and it works fine. From the verbose it is also clear that it downloads the precompiled wheels (and does not build gym).

[priyamalart]

I always tried to install setuptool==66 and gym==0.21.0. With this error persists.
With latest gym version, issue doesnt occur.

[andreped]

What do you mean by latest gym version? Newer than 0.21? Those are not compatible with simple-basline3 (sb3).

I have precompiled the 0.21 wheel now and released it in my own fork. I believe that should work, as building during install is no longer required. Was it that you tried?

[priyamalart]

Yes, gym==0.26.2 installation works.
Off course, We also can't migrate to latest gym - 0.26.2.

What should be tried here? How to do this or any links to follow?
I believe that should work, as building during install is no longer required. Was it that you tried?

[andreped]

I have precompiled the 0.21 wheel now and released it in my own fork. I believe that should work, as building during install is no longer required. Was it that you tried?

^again. I should have fixed this in my own release of gym, which my own sapai-gym depends on (see here).

In order to properly get that fix, you will likely need to do --no-cache-dir to ensure that you get the updated wheel of sapai-gym, as I only updated the wheel itself and did not make a new release.

So please, try setting up the docker image again, if thats what you are trying to do.

If all this does not work, I will need a reproducible bash script demonstrating what you are trying to do, including the Dockerfile.

[andreped]

Great! What do you mean with "release your change"?

It is already part of super-ml-pets, as the the gym wheel above is part of the sapai-gym dependency here (which has been recently updated). You can see that it is in use here.

Does everything work now for you?