Coder Social home page Coder Social logo

Comments (19)

adriangibanelbtactic avatar adriangibanelbtactic commented on September 22, 2024 1

It's fixed at c9de527 and will be released in next major release. @adriangibanelbtactic and @shooding thank you for the detailed report.

Thank you very much @igwyd !

BTW, don't forget about the https://github.com/ONLYOFFICE/document-server-package/ repo.
It has some issues that need some attention too.

Thank you again.

from build_tools.

shooding avatar shooding commented on September 22, 2024 1

I've managed to
build for v8.0.1 [OK]
Host OS: Ubuntu 22.04 (x64) + make image from build_tools/Dockerfile (From Ubuntu:16.04)

with several speed up techniques allow you to continue the build. Here's the Dockerfile that pre-install openjdk. You can omit this if network is very stable. Also change sources.list to the one closet helps.

FROM ubuntu:16.04

ENV TZ=Etc/UTC
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone

# Update the sources.list to use the mirror closet to you
RUN sed -i 's|http://archive.ubuntu.com/ubuntu/|http://free.nchc.org.tw/ubuntu/|g' /etc/apt/sources.list

RUN apt-get -y update && \
    apt-get -y install software-properties-common && \
    add-apt-repository ppa:openjdk-r/ppa && \
    apt-get -y update && \
    DEBIAN_FRONTEND=noninteractive apt-get -y install python \
                                    python3 \
                                    sudo \
                                    openjdk-11-jdk \
                                    openjdk-11-jdk-headless \
                                    openjdk-11-jre \
                                    openjdk-11-jre-headless \
                                    git \
                                    openssh-client

RUN rm /usr/bin/python && ln -s /usr/bin/python2 /usr/bin/python

VOLUME ["/work"]
WORKDIR /work

# Set the default command
CMD ["bash"]
export VERSION=8.0.1
cd build_tools && docker build -t documentserver-develop:$VERSION .

Note that i mounted ssh key into container so that we can force using ssh protocol to avoid "RPC failed; curl 18 transfer closed with outstanding read data remaining container" with unstable network (e.g. WiFi congestion).
Note that in Dockerfile, using a volume /work to share the output with host machine, and you can edit/delete repos either within the container or host machine. This is helpful because sometimes you'll need to rm -rf core/Common/3dParty/boost/boost_1_72_0 to clean up intermediate build.

docker run -it --platform=linux/amd64 -p 80:80 \
-e PRODUCT_VERSION=$VERSION \
-e BUILD_NUMBER=50 \
-v ~/.ssh:/root/.ssh \
-v $WORKFOLDER:/work \
documentserver-develop:$VERSION

You'll enter the container then copy & paste these command to have smooth cloning process.

git config --global url."[email protected]:".insteadOf "https://github.com/" && ssh-keyscan github.com >> /root/.ssh/known_hosts && eval "$(ssh-agent -s)" && ssh-add /root/.ssh/id_rsa

If you github ssh key is encrypted, here you enter password. In my senario, i just want to change some stuff in sdjks and web-apps, therefore building server is enough, but still want consistency so pre-cloned these repos and checkout to the correct tag e.g. 8.0.1.50 accordinly, then mount to /work volume before building.

# To share with host machine. Mount -v /path/from/host:/work
# ├── build_tools
# ├── core
# ├── core-fonts
# ├── dictionaries
# ├── qt-everywhere-opensource-src-5.9.9 (pre download and tar -xf )
# ├── sdkjs
# ├── server
# ├── web-apps
# ├── document-server-integration
# ├── ...

You can check the function def get_repositories() in scripts/base.py and comment out those repos pre-cloned. Like:

def get_repositories():
  result = {}
  # result["core"] = [False, False]
  # result["sdkjs"] = [False, False]
  result.update(get_sdkjs_addons())
  result["onlyoffice.github.io"] = [False, False]
  # result["web-apps"] = [False, False]
  result.update(get_web_apps_addons())
  # result["dictionaries"] = [False, False]
  # result["core-fonts"] = [False, False]

Why need to comment out manually? while def update_repositories(repositories) should've done the git update ?
This is because core repo is a submodule of DocumentServer and the building script will check it's parent and error: fatal: Not a git repository: ../.git/modules/core

Then i ran with

cd build_tools/tools/linux/ && python3 ./automate.py server

Build [OK]

from build_tools.

igwyd avatar igwyd commented on September 22, 2024

Hello @adriangibanelbtactic, you are right, i got the same error. I created ticket #68017

from build_tools.

shooding avatar shooding commented on September 22, 2024

build_tools base repo version:
v8.0.1.50
Operating system:
Ubuntu 16.04

similar issue at building v8

return f'.{gcs_file_name}{MIGRATION_TOGGLE_FILE_SUFFIX}' ^ SyntaxError: invalid syntax

Due to need of python3.8

from build_tools.

igwyd avatar igwyd commented on September 22, 2024

It's fixed at c9de527 and will be released in next major release. @adriangibanelbtactic and @shooding thank you for the detailed report.

from build_tools.

shooding avatar shooding commented on September 22, 2024

@igwyd release/v8.1.0 still gets error when building v8 and cipd

Cloning into '/core/Common/3dParty/v8_89/_gclient_v8_gdy3m7cc'...
remote: Sending approximately 789.65 MiB ...
remote: Counting objects: 7459, done
remote: Finding sources: 100% (3/3)
remote: Total 1098892 (delta 920986), reused 1098890 (delta 920986)
Receiving objects: 100% (1098892/1098892), 789.52 MiB | 9.11 MiB/s, done.
Resolving deltas: 100% (920986/920986), done.
Checking connectivity... done.

[0:03:50] Still working on:
[0:03:50]   v8
1>Syncing projects:   0% ( 0/ 2)
[0:03:52] Still working on:
[0:03:52]   v8
Syncing projects: 100% (38/38), done.

________ running 'python3 third_party/depot_tools/download_from_google_storage.py --no_resume --no_auth -u --bucket v8-wasm-spec-tests -s test/wasm-spec-tests/tests.tar.gz.sha1' in '/core/Common/3dParty/v8_89/v8'
  File "third_party/depot_tools/download_from_google_storage.py", line 51
    return f'.{gcs_file_name}{MIGRATION_TOGGLE_FILE_SUFFIX}'
                                                           ^
SyntaxError: invalid syntax
Error: Command 'python3 third_party/depot_tools/download_from_google_storage.py --no_resume --no_auth -u --bucket v8-wasm-spec-tests -s test/wasm-spec-tests/tests.tar.gz.sha1' returned non-zero exit status 1 in /core/Common/3dParty/v8_89/v8
  File "third_party/depot_tools/download_from_google_storage.py", line 51
    return f'.{gcs_file_name}{MIGRATION_TOGGLE_FILE_SUFFIX}'
                                                           ^
SyntaxError: invalid syntax

Subprocess failed with return code 2.
./cipd: line 137: ./depot_tools/cipd_client_version.digests: No such file or directory
Platform linux-amd64 is not supported by the CIPD client bootstrap: there's no pinned SHA256 hash for it in the *.digests file.
error: unknown option `type'

from build_tools.

adriangibanelbtactic avatar adriangibanelbtactic commented on September 22, 2024

It's fixed at c9de527 and will be released in next major release. @adriangibanelbtactic and @shooding thank you for the detailed report.

Wait a moment... that commit has a python3.tar.gz binary.
That's not right to add binaries to a source repo.
And, well, I can stand the many third party binaries you download because we can inspect which urls you are getting them from... but... Why we should trust a random binary?

I mean... that python3.tar.gz's python3/bin/python3.10 binary might have a trojan after all.

There should be another fix that does not end up throwing a binary on the repo.

Thank you for considering an alternative.

from build_tools.

igwyd avatar igwyd commented on September 22, 2024

@shooding, I cheked branch release/v8.1.0 4 days ago in docker 16.04 this error is gone, but branch release/v8.1.0 still in development and other error may appear.

@adriangibanelbtactic I think it's about trust to this repo, you are you are an experienced user and you can check binary files or replase it.
I will show to developers issues from https://github.com/ONLYOFFICE/document-server-package/, thank you.

from build_tools.

shooding avatar shooding commented on September 22, 2024

Struggling for several days

build for v7.5.1 [OK] serveral months ago.
Host OS: Windows (x64) + make image from build_tools/Dockerfile (From Ubuntu:16.04)

build for v8.0.1 [FAILED]
Host OS: macBook pro m1 (arm64) + make image from build_tools/Dockerfile (From Ubuntu:16.04)
Host OS: Ubuntu 22.04 (x64) + make image from build_tools/Dockerfile (From Ubuntu:16.04)
Host OS: Ubuntu 22.04 (x64) + make image from build_tools/Dockerfile (From Ubuntu:20.04)

Several common issues like:

  1. "RPC failed; curl 18 transfer closed with outstanding read data remaining container"
  2. missing header when building with clang++ (v19) but missing #include header, needs
  3. remained build files from previous failure needs to clean up e.g. "core/Common/3dParty/boost/boost_1_72_0"

After all those fails l tried to update Dockerfile and automate.py "--update", "0" and
mount pre-cloned repos like sdkjs and web-apps into the container to boost build speed in case downloading them many times.

In the updated Dockerfile pre-installed
openjdk-11-jdk openjdk-11-jdk-headless openjdk-11-jre openjdk-11-jre-headless.
Download qt_source_5.9.9.tar.xz, unzip and mount as qt-everywhere-opensource-src-5.9.9.

Even though tried configure.py with "--update", "0", still get error 1 when building core common modules like boost or v8.
Should i just find a Ubuntu 16 machine as host and run python3 ./automate.py or
find a Windows machine as host + docker build ?

from build_tools.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.