oca / oca-ci Goto Github PK
View Code? Open in Web Editor NEWContainer image to run OCA CI tests
License: MIT License
Container image to run OCA CI tests
License: MIT License
In this line:
https://github.com/sbidoul/oca-ci/blob/fdbcf377efb00f94102ee5ed373ad1feab91f439/Dockerfile#L108
There is a git clone. Just wondering why not curl directly a tarball
like https://api.github.com/repos/odoo/odoo/tarball/14.0
The issue first occurred in coopiteasy/cie-custom#109, which contains custom modules.
Given:
spp_pos_mustard
in https://github.com/coopiteasy/cie-custom/tree/12.0 depends on pos_shift_partner_can_shop
.pos_shift_partner_can_shop
exists in our repository https://github.com/carmenbianca/circular-rebel-example.other_module
in our repository depends on spp_pos_mustard
. (edit: spp_pos_mustard
is added to test-requirements.txt)pos_shift_partner_can_shop
is a rebel module as per https://github.com/OCA/oca-addons-repo-template.Then:
pos_shift_partner_can_shop
will pass, correctly installing itself and spp_pos_mustard
.other_module
will fail, failing to install spp_pos_mustard
because the dependency pos_shift_partner_can_shop
is not met.See https://github.com/carmenbianca/circular-rebel-example for example GitHub CI runs.
Affected versions: Any
Described above with example repository.
Expected behavior
I expect pos_shift_partner_can_shop
to be pip-installed always so that all modules in test-requirements.txt
can be installed without error.
Additional context
N/A
In sbidoul/runboat#48 we noticed that doing an apt-get install could fail due to package information being outdated.
We do an apt-get update in the Dockerfile. Even though we rebuild the images nightly, due to layer caching, there can be a long delay (up to 2 weeks IIRC) between the apt-get update and apt-get install.
The question I have is the following. Is the problem we had in sbidoul/runboat#48 a one-off incident or could it become prevalent ? In which case we'll need to do something about the installation of addons deb dependencies here and here.
When we look at http://archive.ubuntu.com/ubuntu/pool/main/r/rsync/ we can see that a lot of versions are there, which tend to imply that published version remain available "forever". http://archive.ubuntu.com/ubuntu/pool/main/r/rsync/rsync_3.1.3-8ubuntu0.1_amd64.deb is however absent.
So I suspect it was published then removed and we just played bad luck as we cached an apt index that still included it.
I'm not sure about this explanation and if we should do something about it. If there are people around with deb packaging expertise who could shed light on this it would be very welcome. @gurneyalex maybe ?
Is your feature request related to a problem?
Python libraries, such as PyCurl depend on apt/deb dependencies, unfortunately, oca_install_addons
gets into a chicken and egg issue where oca_list_external_dependencies
won't run until the Python deps are installed, but the Python deps cannot be installed until libcurl4-(openssl|nss|gnutls)-dev apt package is installed.
i.e.
DEBIAN_FRONTEND=noninteractive apt-get install -qq --no-install-recommends $(oca_list_external_dependencies deb)
not found in addons path: component,component_event,connector,queue_job
Aborted!
Describe the solution you'd like
Either:
Describe alternatives you've considered
N/A
Additional context
This seems to be a common issue where the workaround has been to modify the Dockerfile and add external dependencies, such as xmlsec1?
I am happy to do the leg work on this issue, but I'm not clear if it's an OCA-CI or a manifestoo issue, or both? Before I do any work I'd like some guidance or input in the better way to tackle this :)
Images are not being refreshed.
@sbidoul Last action executed is from 12 days ago
Add odoo.cfg
[options]
/etc/odoo.cfg
(?)ODOO_RC
and OPENERP_SERVER
env vars pointing to itIs your feature request related to a problem?
I've develop a werkzeurg middle-ware to redirect http to https in the following PR OCA/server-tools#2558
Before running test (therefor lauching odoo server) unit test require to add module in the server_wide_modules list
Describe the solution you'd like
I'm wondering if there is a proper approach to consider already implemented or to implement that would avoid breaking maintenance task on github action files ?
Or probably this something in copier template https://github.com/OCA/oca-addons-repo-template instead of this project ?
Describe alternatives you've considered
As working alternative I've update the odoo.cfg file running a shell command https://github.com/OCA/server-tools/pull/2558/files#diff-faff1af3d8ff408964a57b2e475f69a6b7c7b71c9978cccc8f471798caac2c88R76
manifestoo does everything we need.
When addons in the repo being tested depend on addons from other repos which themselve depend on other addons in the repo being tested, these dependent addons are installed from PyPI instead of from the repo under test.
This repository has no specified licence, making it difficult to re-use. Is it possible to add a licence?
│ Traceback (most recent call last): │
│ File "/opt/odoo-venv/bin/odoo", line 7, in <module> │
│ exec(compile(f.read(), __file__, 'exec')) │
│ File "/opt/odoo/setup/odoo", line 5, in <module> │
│ import odoo │
│ File "/opt/odoo/odoo/__init__.py", line 113, in <module> │
│ from . import modules │
│ File "/opt/odoo/odoo/modules/__init__.py", line 8, in <module> │
│ from . import db, graph, loading, migration, module, registry │
│ File "/opt/odoo/odoo/modules/graph.py", line 10, in <module> │
│ import odoo.tools as tools │
│ File "/opt/odoo/odoo/tools/__init__.py", line 7, in <module> │
│ from . import pdf │
│ File "/opt/odoo/odoo/tools/pdf.py", line 17, in <module> │
│ from odoo.tools.misc import file_open │
│ File "/opt/odoo/odoo/tools/misc.py", line 34, in <module> │
│ from lxml import etree │
│ ImportError: /opt/odoo-venv/lib/python3.6/site-packages/lxml/etree.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PyFPE_jbuf
Hi @sbidoul. Do you have any suggestions or examples on how to add Odoo enterprise modules among available modules for testing in Github Action?
Originale request here: #23 (comment)
Currently oca-ci does not have an Odoo log parser and relies on the Odoo exit code... and would really like to avoid adding one.
This seems to work well, but there are apparently some exceptions. This issue is to track them.
Searchpanel multi with style Hierarchy like default searchpanel (many2one)
not a modules repository
The tests hang on
❯ docker run -v $PWD/tests:/mnt/tests ghcr.io/oca/oca-ci/py3.8-odoo14.0:latest /mnt/tests/runtests.sh -v
Unable to find image 'ghcr.io/oca/oca-ci/py3.8-odoo14.0:latest' locally
latest: Pulling from oca/oca-ci/py3.8-odoo14.0
eaead16dc43b: Pull complete
69623f897469: Pull complete
cb04cdf8294b: Pull complete
1cdf5e92710b: Pull complete
8bcb80089e13: Pull complete
0127f8309be7: Pull complete
9eaa49ee8f9e: Pull complete
e9d854d08099: Pull complete
d4d3858bb4a9: Pull complete
c88165554b6f: Pull complete
2074cac1f696: Pull complete
9b40702c51ca: Pull complete
23554f0dbe2d: Pull complete
be60c6f99f89: Pull complete
ee4d20d20c14: Pull complete
1e8d4dcb3909: Pull complete
341ccbaa2c55: Pull complete
74775fe9453f: Pull complete
b769083296ea: Pull complete
c486ab10f179: Pull complete
1a3d2699c705: Pull complete
cc98a71c2060: Pull complete
702d4c016a8e: Pull complete
3078e2daea60: Pull complete
Digest: sha256:430787fcf2f71049a02ebad84733678f6bf5d0dd33670053a3e9060aeeb7d331
Status: Downloaded newer image for ghcr.io/oca/oca-ci/py3.8-odoo14.0:latest
created virtual environment CPython3.8.10.final.0-64 in 91ms
creator CPython3Posix(dest=/tmp/testvenv, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
added seed packages: pip==22.2.2, setuptools==65.3.0, wheel==0.37.1
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
Looking in indexes: https://wheelhouse.odoo-community.org/oca-simple-and-pypi
Collecting pytest
Downloading pytest-7.2.0-py3-none-any.whl (316 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 316.8/316.8 kB 2.1 MB/s eta 0:00:00
Collecting pluggy<2.0,>=0.12
Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting attrs>=19.2.0
Downloading attrs-22.1.0-py2.py3-none-any.whl (58 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.8/58.8 kB 593.7 kB/s eta 0:00:00
Collecting packaging
Downloading packaging-22.0-py3-none-any.whl (42 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.6/42.6 kB 80.6 kB/s eta 0:00:00
Collecting iniconfig
Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting tomli>=1.0.0
Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting exceptiongroup>=1.0.0rc8
Downloading exceptiongroup-1.0.4-py3-none-any.whl (14 kB)
Installing collected packages: iniconfig, tomli, pluggy, packaging, exceptiongroup, attrs, pytest
Successfully installed attrs-22.1.0 exceptiongroup-1.0.4 iniconfig-1.1.1 packaging-22.0 pluggy-1.0.0 pytest-7.2.0 tomli-2.0.1
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.2.0, pluggy-1.0.0 -- /tmp/testvenv/bin/python
cachedir: .pytest_cache
rootdir: /mnt/tests
collecting ... collected 17 itemsmnt/tests/test_addons_path.py::test_addons_path PASSED [ 5%]
mnt/tests/test_list_external_dependencies.py::test_list_external_dependencies PASSED [ 11%]
mnt/tests/test_list_external_dependencies.py::test_list_external_dependencies_transitive PASSED [ 17%]
mnt/tests/test_no_addons.py::test_no_addons
Affected versions:
master
Steps to reproduce the behavior:
docker run -v $PWD/tests:/mnt/tests ghcr.io/oca/oca-ci/py3.8-odoo14.0:latest /mnt/tests/runtests.sh -v
Expected behavior
The tests should run
Additional context
Looking at the processes, the tests are waiting for postgres:
maybe postgres should be configured somehow before running the tests?
Is your feature request related to a problem?
Yes, some modules require pip
packages that require deb
packages in order to be installed.
An example is in https://github.com/OCA/server-tools/blob/10.0/base_external_dbsource_odbc: it depends on pyodbc
(https://pypi.org/project/pyodbc) that needs unixodbc-dev
, as explained in its documentation https://github.com/mkleehammer/pyodbc/wiki/Install#installing-on-linux
Describe the solution you'd like
Configure apt
packages on a per-repo basis that will be installed before installing Python libraries.
Describe alternatives you've considered
A step can be added in the github actions that installs the apt
packages before installing pip
packages, but as far I as understand, that is discouraged.
Currently there is nothing similar for runboat
Additional context
Similar to #33.
I ran into this while trying to add https://github.com/OCA/oca-addons-repo-template to https://github.com/OCA/server-tools/tree/10.0 in OCA/server-tools#2486.
There is some discussion already in https://discord.com/channels/737652535149592587/761225067798462465/1067129843130581083
As reported by @pedrobaeza in OCA/sale-workflow#3232 (comment), it seems that Odoo 17 now requires the pg_trgm
extension to be installed.
This requirement was introduced recently in this commit odoo/odoo@85b2ede
@pivi-odoo as author of that commit, maybe you can say if it was intended to add pg_trgm
as a hard dependency in 17.0?
If yes, we can likely do it in oca_init_test_database
.
Hello @sbidoul
I believe the daily workflow to build the images is not running anymore:
As a consequence oca tests are running with a slightly outdated codebase, which is blocking in some cases.
Could you help to re-enable that workflow?
Thanks!
PS: I didn't find a way to prevent it from being disabled again after 60d of inactivity on the repo :/
Hello @sbidoul
This is a follow-up to #57
Disclaimer: I might be missing the bigger picture here 😅
Context: OCA/bank-payment#1236
Adding a non-merged dependency from same repository in test-requirements.txt
leads to 2 issues:
Both cases raise the exception NoScmFound
in whool, here
Workarounds:
Relevant logs:
(1)
Building wheel for odoo-addon-account_payment_order (pyproject.toml): started
Building wheel for odoo-addon-account_payment_order (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Building wheel for odoo-addon-account_payment_order (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [31 lines of output]
fatal: detected dubious ownership in repository at '/__w/bank-payment/bank-payment'
To add an exception for this directory, call:
git config --global --add safe.directory /__w/bank-payment/bank-payment
Traceback (most recent call last):
File "/tmp/pip-build-env-16qd0cbq/overlay/lib/python3.10/site-packages/whool/buildapi.py", line 41, in _scm_ls_files
subprocess.check_output(
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'ls-files']' returned non-zero exit status 128.
(2)
Building wheel for odoo-addon-account_payment_order (pyproject.toml): started
Building wheel for odoo-addon-account_payment_order (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Building wheel for odoo-addon-account_payment_order (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [29 lines of output]
fatal: not a git repository (or any parent up to mount point /mnt)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Traceback (most recent call last):
File "/tmp/pip-build-env-7452fbi7/overlay/lib/python3.10/site-packages/whool/buildapi.py", line 41, in _scm_ls_files
subprocess.check_output(
File "/usr/lib/python3.10/subprocess.py", line 421, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'ls-files']' returned non-zero exit status 128.
Firstly, thanks for this amazing tools.
I would like to know how to add Odoo enterprise modules among available modules for testing in Github Action?
I have a private repo and the modules depend on the odoo enterprise modules. So, when I added the modules that depend on enterprise modules, CI test is failed? I found these #23 (comment). I don't know how to approach.
Hi.
First, thanks a lot for this tools !
I tried to change the CI of my company for my custom repositories, using oca-ci with github workflow actions.
for the first one : (https://github.com/grap/grap-odoo-incubator) : all is great. (I had to implement that fix OCA/oca-addons-repo-template#102, thanks @sbidoul).
note : the grap-odoo-incubator repo only depends on odoo and oca modules.
for the second one (https://github.com/grap/grap-odoo-business) : I have an error during the installation of the dependencies.
https://github.com/legalsylvain/grap-odoo-business/runs/4869488630?check_suite_focus=true#step:4:205
The CI can't install a module (produt_print_category
) that is present in the first repo (grap-odoo-incubator, not odoo nor oca). it tries via pip. how can I change the call of oca_install_addons
to have the possiblity to install modules present in non OCA repositories ? The repo is correctly referenced in the oca_dependencies.txt
.
thanks in advance, and sorry if my question is not relevant.
oca-ci/py3.8-odoo15.0 package
Tests fail on GitHub for both Odoo tests and OCB tests while they pass on Odoo.sh and a local dev instance.
Affected versions: Unsure (currently latest)
Steps to reproduce the behavior:
odoo-bin -c <config file> -d <new db name> --test-enable -i purchase_delivery_split_date --stop-after-init
. All tests should pass. Note that running these tests with the version currently on OCA's repo fails 9 tests on dev machine but passes all tests in the GitHub action.Expected behavior
Test results should be consistent between Odoo's test suite and the tests run by OCA-CI for the same major version.
Additional context
Tests run locally on MacOS with Python 3.8.18, both using only Odoo community edition and full Enterprise edition sources (same result). Tests run on Odoo.sh all passing.
Is your feature request related to a problem?
Yes: some code might fail in Windows so it would be nice to test in Windows too.
Describe the solution you'd like
At least one CI should run the tests on Windows.
Describe alternatives you've considered
Adapt the code or files to work both in Ubuntu and Windows.
Additional context
For example, if a file name contains :
(colon) the repository cannot be cloned in Windows (that is what happened in OCA/l10n-italy#3805).
A docker-compose.yml
that provides a postgres and the appropriate environment variables to the odoo-ci container should help running the tests with docker-compose run
.
Done in 9da97ae
Say you have a repo with addons A and B.
Say you have a PR with changes to B that depend on a changes in an unmerged PR to A . In that PR, test-requirements.txt has a git reference to A.
In that case, it is unclear if the PR to A will have precendence over A found in the addons path (it is deterministic but fragile and not guaranteed to not change in the future).
When using editable installation mode (#62) there will likely be a conflict between the PR to A and -e A
that pip will not be able to resolve.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.