Coder Social home page Coder Social logo

dgroc's Introduction

dgroc

Author

Pierre-Yves Chibon <[email protected]>

dgroc: Daily Git Rebuild On Copr

This project aims at easily provide daily build of a project tracked via git and made available via copr.

Get it running

  • Retrieve the sources:

    git clone https://github.com/pypingou/dgroc.git
  • Create the configuration file ~/.config/dgroc
  • Fill the configuration file, for example:

    [main]
    username = me
    email = [email protected]
    copr_url = https://copr.fedoraproject.org/
    upload_command = cp %s /var/www/html/subsurface/
    upload_url = http://my_server/subsurface/%s
    #no_ssl_check = True  # No longer required now that copr has a valid cert
    
    [subsurface]
    git_url = git://subsurface.hohndel.org/subsurface.git
    git_folder = /tmp/subsurface/
    spec_file = ~/GIT/subsurface/subsurface.spec
    
    [guake]
    copr = subsurface
    git_url = https://github.com/Guake/guake.git
    git_folder = /tmp/guake/
    spec_file = ~/GIT/guake/guake.spec
    patch_files = ~/GIT/guake/guake-0.2.2-fix_vte.patch,
                  ~/GIT/guake/0001-Remove-vte-check-in-the-configure.ac.patch

The Main section

username The name to use in the changelog of the spec file when updating it.

email The email to use in the changelog of the spec file when updating it.

copr_url The url of copr to use.

upload_command The command to run to make the source rpm (src.rpm, srpm) available to copr. This can be a copy command (cp) or a copy over ssh (scp). Note that the %s is important, it will be replaced by the full path to the source rpm created.

upload_url The url of the source rpm once it has been uploaded. Note that here as well the %s is important as it will be replaced by the source rpm file name.

For example, if you upload your source rpm onto your fedorapeople space, your upload_command might be: scp %s fedorapeople:public_html/srpms/ and your upload_url might be: https://pingou.fedorapeople.org/srpms/%s.

no_ssl_check Simple boolean to check the ssl certificate when starting the build on copr. At the moment the ssl certificate is self-signed and thus invalid. So using the https version of copr_url will require a no_ssl_check set to True.

copr_config The path to a config file with details about COPR connection. Go to the API page on your COPR instance to get the contents. This is useful if you want to use dgroc with different copr instances. Defaults to ~/.config/copr.

The project section

For each project, only three options are required:

git_url The url to the git repository, that is only required if the git repo is not already cloned on the disk (see git_folder).

git_folder The location of the local clone of the git repository to build.

spec_file The location of the spec file for the project to build.

patch_files A comma separated list of patches required to build the project. These files will be copied over to the rpm sourcedir to be present when building the source rpm.

copr The optional name of the copr repository to build the package within. When not set, the project name (from []) is used.

Note

The spec file should be fully functionnal as all dgroc will do is update the Source0, Release and add an entry in the Changelog.

Note

You might have to set in your spec file the %setup line to:

%setup -q -n <projectname>

Run the project

From the sources, it requires few steps:

  • Install dependencies:

    yum install libgit2-devel python-virtualenvwrapper
  • Create a virtual env:

    mkvirtualenv --system-site-packages dgroc
  • Install the python dependencies:

    pip install -r requirements.txt
  • Run dgroc:

    python dgroc.py

For more information/output run python dgroc.py --debug

Run dgroc daily

The easiest way to run dgroc daily is to simply rely on cron

Here is an example cron entry that you will need to adjust for your setup:

30 10 * * * python /home/pingou/dgroc/dgroc.py

This cron will run every day at 10:30 am and call the dgroc.py script within the dgroc clone

dgroc's People

Contributors

endle avatar hroncok avatar lubomir avatar pypingou avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

dgroc's Issues

patch_files option should support globs

Currently, I would have to use this crazy config:

[python3-nightly]
scm = hg
hg_url = http://hg.python.org/cpython
hg_folder = /tmp/python-hg/
spec_file = ~/rpmbuild/fedora-scm/python3-keep/python3.spec
patch_files = ~/rpmbuild/fedora-scm/python3-keep/python3-arm-skip-failing-fragile-test.patch, ~/rpmbuild/fedora-scm/python3-keep/Python-3.1.1-rpath.patch, ~/rpmbuild/fedora-scm/python3-keep/python-3.1.2-fix-expat-issue9054.patch, ~/rpmbuild/fedora-scm/python3-keep/python-3.2a1-apply-our-changes-to-expected-shebang-for-test_imp.patch, ~/rpmbuild/fedora-scm/python3-keep/python-3.2a1-debug-build.patch, ~/rpmbuild/fedora-scm/python3-keep/python-3.2a1-fix-parallel-make.patch, ~/rpmbuild/fedora-scm/python3-keep/python-3.2b2-fix-ppc-debug-build.patch, ~/rpmbuild/fedora-scm/python3-keep/python-3.2rc1-s390-tsc.patch, ~/rpmbuild/fedora-scm/python3-keep/temporarily-disable-tests-requiring-SIGHUP.patch, ~/rpmbuild/fedora-scm/python3-keep/00003-remove-mimeaudio-tests.patch, ~/rpmbuild/fedora-scm/python3-keep/00055-systemtap.patch, ~/rpmbuild/fedora-scm/python3-keep/00102-lib64.patch, ~/rpmbuild/fedora-scm/python3-keep/00104-lib64-fix-for-test_install.patch, ~/rpmbuild/fedora-scm/python3-keep/00111-no-static-lib.patch, ~/rpmbuild/fedora-scm/python3-keep/00113-more-configuration-flags.patch, ~/rpmbuild/fedora-scm/python3-keep/00125-less-verbose-COUNT_ALLOCS.patch, ~/rpmbuild/fedora-scm/python3-keep/00131-disable-tests-in-test_io.patch, ~/rpmbuild/fedora-scm/python3-keep/00132-add-rpmbuild-hooks-to-unittest.patch, ~/rpmbuild/fedora-scm/python3-keep/00134-fix-COUNT_ALLOCS-failure-in-test_sys.patch, ~/rpmbuild/fedora-scm/python3-keep/00135-fix-test-within-test_weakref-in-debug-build.patch, ~/rpmbuild/fedora-scm/python3-keep/00137-skip-distutils-tests-that-fail-in-rpmbuild.patch, ~/rpmbuild/fedora-scm/python3-keep/00139-skip-test_float-known-failure-on-arm.patch, ~/rpmbuild/fedora-scm/python3-keep/00141-fix-tests_with_COUNT_ALLOCS.patch, ~/rpmbuild/fedora-scm/python3-keep/00143-tsc-on-ppc.patch, ~/rpmbuild/fedora-scm/python3-keep/00146-hashlib-fips.patch, ~/rpmbuild/fedora-scm/python3-keep/00150-disable-rAssertAlmostEqual-cmath-on-ppc.patch, ~/rpmbuild/fedora-scm/python3-keep/00153-fix-test_gdb-noise.patch, ~/rpmbuild/fedora-scm/python3-keep/00155-avoid-ctypes-thunks.patch, ~/rpmbuild/fedora-scm/python3-keep/00156-gdb-autoload-safepath.patch, ~/rpmbuild/fedora-scm/python3-keep/00157-uid-gid-overflows.patch, ~/rpmbuild/fedora-scm/python3-keep/00160-disable-test_fs_holes-in-rpm-build.patch, ~/rpmbuild/fedora-scm/python3-keep/00163-disable-parts-of-test_socket-in-rpm-build.patch, ~/rpmbuild/fedora-scm/python3-keep/00164-disable-interrupted_write-tests-on-ppc.patch, ~/rpmbuild/fedora-scm/python3-keep/00173-workaround-ENOPROTOOPT-in-bind_port.patch, ~/rpmbuild/fedora-scm/python3-keep/00175-fix-configure-Wformat.patch, ~/rpmbuild/fedora-scm/python3-keep/00176-upstream-issue16754-so-extension.patch, ~/rpmbuild/fedora-scm/python3-keep/00178-dont-duplicate-flags-in-sysconfig.patch, ~/rpmbuild/fedora-scm/python3-keep/00179-dont-raise-error-on-gdb-corrupted-frames-in-backtrace.patch, ~/rpmbuild/fedora-scm/python3-keep/00180-python-add-support-for-ppc64p7.patch, ~/rpmbuild/fedora-scm/python3-keep/00184-ctypes-should-build-with-libffi-multilib-wrapper.patch, ~/rpmbuild/fedora-scm/python3-keep/00186-dont-raise-from-py_compile.patch, ~/rpmbuild/fedora-scm/python3-keep/00187-change-match_hostname-to-follow-RFC-6125.patch, ~/rpmbuild/fedora-scm/python3-keep/00188-fix-lib2to3-tests-when-hashlib-doesnt-compile-properly.patch, ~/rpmbuild/fedora-scm/python3-keep/00189-add-rewheel-module.patch, ~/rpmbuild/fedora-scm/python3-keep/00192-buffer-overflow.patch, ~/rpmbuild/fedora-scm/python3-keep/00193-skip-correct-num-of-pycfile-bytes-in-modulefinder.patch, ~/rpmbuild/fedora-scm/python3-keep/00195-dont-add-Werror-declaration-after-statement.patch, ~/rpmbuild/fedora-scm/python3-keep/00196-test-gdb-match-addr-before-builtin.patch, ~/rpmbuild/fedora-scm/python3-keep/05000-autotool-intermediates.patch, ~/rpmbuild/fedora-scm/python3-keep/pyfuntop.stp, ~/rpmbuild/fedora-scm/python3-keep/systemtap-example.stp, ~/rpmbuild/fedora-scm/python3-keep/libpython.stp, ~/rpmbuild/fedora-scm/python3-keep/macros.pybytecompile, ~/rpmbuild/fedora-scm/python3-keep/macros.python3, ~/rpmbuild/fedora-scm/python3-keep/find-provides-without-python-sonames.sh, ~/rpmbuild/fedora-scm/python3-keep/check-pyc-and-pyo-timestamps.py

While this would be much more useful:

[python3-nightly]
scm = hg
hg_url = http://hg.python.org/cpython
hg_folder = /tmp/python-hg/
spec_file = ~/rpmbuild/fedora-scm/python3-keep/python3.spec
patch_files = ~/rpmbuild/fedora-scm/python3-keep/*.patch, ~/rpmbuild/fedora-scm/python3-keep/*.stp, ~/rpmbuild/fedora-scm/python3-keep/macros.*, ~/rpmbuild/fedora-scm/python3-keep/*.sh, ~/rpmbuild/fedora-scm/python3-keep/*.py

The only thing needed should be glob in Copy patches section. I'll have a look at it.

Doesn't work on first run, fails to create archive

(dgroc) [dgroc@icarus ~]$ python dgroc/dgroc.py --debug
Processing project: rocketchat-desktop
Generating new source rpm for project: rocketchat-desktop
Cloning https://github.com/RocketChat/Rocket.Chat.Electron.git
Last commit: ac8d04ec
Command to generate archive: git archive --format=tar --prefix=rocketchat-desktop/ -o/home/dgroc/rpmbuild/SOURCES/rocketchat-desktop-ac8d04ec.tar HEAD
Update spec file: /home/dgroc/fedora-specs/copr/rocketchat-dev/rocketchat.spec
Release line before: Release:   102.git%{_shortcommit}%{?dist}
Release number: 103
Release line after: Release:        103.20170506gitac8d04ec%{?dist}
Source0 line after: Source0:        rocketchat-desktop-ac8d04ec.tar
Spec file updated: /home/dgroc/fedora-specs/copr/rocketchat-dev/rocketchat.spec
Strange result of the rpmbuild -bs:
  stdout:
  stderr:error: File /home/dgroc/rpmbuild/SOURCES/rocketchat-desktop-ac8d04ec.tar: No such file or directory

0 srpms generated
(dgroc) [dgroc@icarus ~]$ ls -l /home/dgroc/rpmbuild/SOURCES/rocketchat-desktop-ac8d04ec.tar
ls: cannot access '/home/dgroc/rpmbuild/SOURCES/rocketchat-desktop-ac8d04ec.tar': No such file or directory
(dgroc) [dgroc@icarus ~]$
(dgroc) [dgroc@icarus ~]$ python dgroc/dgroc.py --debug
Processing project: rocketchat-desktop
Generating new source rpm for project: rocketchat-desktop
Last commit: ac8d04ec
Command to generate archive: git archive --format=tar --prefix=rocketchat-desktop/ -o/home/dgroc/rpmbuild/SOURCES/rocketchat-desktop-ac8d04ec.tar HEAD
Update spec file: /home/dgroc/fedora-specs/copr/rocketchat-dev/rocketchat.spec
rocketchat-desktop: Spec already up to date
0 srpms generated
(dgroc) [dgroc@icarus ~]$ ls -l /home/dgroc/rpmbuild/SOURCES/rocketchat-desktop-ac8d04ec.tar
-rw-rw-r--. 1 dgroc dgroc 7976960 May  6 14:54 /home/dgroc/rpmbuild/SOURCES/rocketchat-desktop-ac8d04ec.tar
(dgroc) [dgroc@icarus ~]$ python dgroc/dgroc.py --debug
Processing project: rocketchat-desktop
Generating new source rpm for project: rocketchat-desktop
Last commit: ac8d04ec
Command to generate archive: git archive --format=tar --prefix=rocketchat-desktop/ -o/home/dgroc/rpmbuild/SOURCES/rocketchat-desktop-ac8d04ec.tar HEAD
Update spec file: /home/dgroc/fedora-specs/copr/rocketchat-dev/rocketchat.spec
rocketchat-desktop: Spec already up to date
0 srpms generated

Config is:

[main]
username = xenithorb
email = [email protected]
copr_url = https://copr.fedoraproject.org/

[rocketchat-desktop]
git_url = https://github.com/RocketChat/Rocket.Chat.Electron.git
git_folder = /tmp/rocketchat_source/
spec_file = ~/fedora-specs/copr/rocketchat-dev/rocketchat.spec

Strangely, running it twice produces the archive - for whatever reason. I tried to demonstrate that in the above output.

Current requirements pull in too-new pygit2

On Fedora 25 using your virutalenv instructions, I ran into:

  In file included from src/blob.h:34:0,
                   from src/blob.c:30:
  src/types.h:36:2: error: #error You need a compatible libgit2 version (v0.25.x)
   #error You need a compatible libgit2 version (v0.25.x)
    ^~~~~
  error: command 'gcc' failed with exit status 1

The solution was to clamp pygit2 <= 0.24.2 which was the last version that supports what libgit2 is currently in fedora at the time of my writing this: libgit2-devel-0.24.6-1.fc25.x86_64 it might resolve itself with time, but perhaps you should consider version locking to something known to be compatible.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.