tarantool / mkrepo

Maintain DEB and RPM repositories on S3

Create RPM and DEB repositories in S3

mkrepo is a repository generator with pluggable backends, which allows you to maintain an RPM or DEB repository on various storages, like local filesystem or S3, and periodically regenerate metadata.

Use it in tandem with your favourite CI system to produce a better pipeline. mkrepo helps you to get rid of ad-hoc cron jobs.

As a bonus, mkrepo supports on-premises S3 servers like Minio.

Works on Linux and OS X. Should also work on BSD and Windows, but I haven't checked.

Quickstart

Create an s3 bucket named e.g. builds and put a sample package package.rpm to s3://builds/rpmrepo/Packages. Then do the following:

./mkrepo.py s3://builds/rpmrepo

After this, you will find all metadata generated in s3://builds/rpmrepo/repodata

Run tests

To run the tests, use the following command::

make test

Dependencies

Python libraries:

• boto3

Command-line reference

mkrepo parses your ~/.aws/config and reads secret key and region settings. So you may skip them in command line invocation in case you have aws config.

  mkrepo.py [-h] 
            [--temp-dir TEMP_DIR]
            [--s3-access-key-id S3_ACCESS_KEY_ID]
            [--s3-secret-access-key S3_SECRET_ACCESS_KEY]
            [--s3-endpoint S3_ENDPOINT]
            [--s3-region S3_REGION]
            [--s3-public-read]
            [--sign]
            [--force]
            path [path ...]

• --temp-dir - /(optional)/directory used to store temporary artifacts (default is .mkrepo)
• --s3-access-key-id - /(optional)/ specify S3 access key ID
• --s3-secret-access-key - /(optional)/ specify S3 secret key
• --s3-endpoint - /(optional)/ specify S3 server URI
• --s3-region - /(optional)/ specify S3 region (default is us-east-1)
• --s3-public-read - /(optional)/ set read-only permission on files uploaded to S3 for anonymous users
• --sign - /(optional) sign package metadata
• --force - /(optional) when adding packages to the index, the malformed one will be skipped. By default, a malformed package will cause the utility to stop working. The malformed_list.txt file will also be added to the repository
• path - specify list of path to scan for repositories

Environment variables reference

• GPG_SIGN_KEY - the name of the key that will be used to sign package metadata.

Tips for working with GPG keys

• Create a new key:

gpg --full-generate-key

• To view all your keys, you can use:

gpg --list-secret-keys --keyid-format LONG

• Scripts can use something like this to get the Key ID:

export GPG_SIGN_KEY="$(gpg --list-secret-keys --with-colons | grep ^sec: | cut -d: -f5)"

• Export the key in ASCII armored format:

gpg --armor --export-secret-keys MYKEYID > mykeys.asc

• Import the key:

cat mykeys.asc | gpg --batch --import

• MKREPO_DEB_ORIGIN - the value of the "Origin" field of the "Release" file.
• MKREPO_DEB_LABEL - the value of the "Label" field of the "Release" file.
• MKREPO_DEB_DESCRIPTION - the value of the "Description" field of the "Release" file.

How it works

mkrepo searches the supplied path for either Packages or pool subdir. If it finds Packages, it assumes an rpm repo. If it finds pool, it assumes a deb repo.

Then it parses existing metadata files (if any) and compares timestamps recorded there with timestamps of all package files in the repo. Any packages that have different timestamps or that don't exist in metadata, are parsed and added to metadata.

Then new metadata is uploaded to S3, replacing previous one.

Credits

Thanks to Cyril Rohr and Ken Robertson, authors of the following awesome tools:

• rpm-s3
• deb-s3

Unfortunately, we needed a solution that is completely decoupled from CI pipeline, and the mentioned tools only support package push mode, when you have to use a tool to actually push packages to s3, insted of native s3 clients.

mkrepo's Issues

Add python3 support

Python2 sunset - 2020. So, the time is come to add python3 support.

Remove information about deleted packages from metafiles

AFAIU, now if package has been deleted from repository the metainformation must be deleted and generated again. If so, it will be nice to add checks that the packages from the metafiles exist, and remove information from the metafiles if the packages is absent.

setup.py is missing dependencies

I noticed when trying to try out this project via pip that the setup.py doesn't have any install dependencies - based on the requirements.txt on master right now, I think just adding the below to the setup call in setup.py should do the trick

install_requirements=[
"boto3==1.4.1"
"univers==30.9.0"
]

also happy to open a PR for this if it's easier for you, but for such a small/trivial change I wanted to raise the issue first

Generate other.xml.gz for the rpm repositories

Now the mkrepo generate following files with metainformation:

• primary.xml.gz
• filelists.xml.gz
• repomd.xml

Yet one other standard file with metainformation is other.xml.gz (contains an XML file describing miscellaneous information regarding each RPM archive)
AFAIU from https://www.slashroot.in/yum-repository-and-package-management-complete-tutorial - this file is used when a user queries information about a package with the help of "repoquery" command. Also without this file, when I tried to add the repository on CentOS 7, an error occured.
Need to add the generation of this file to mkrepo.

Useful links:
https://www.slashroot.in/yum-repository-and-package-management-complete-tutorial
http://blog.sweetxml.org/2008/03/anatomy-of-yum-repository-look-under.html
https://en.opensuse.org/openSUSE:Standards_Rpm_Metadata#How_to_extend_rpm-md_metadata_with_non_standard_data
https://www.jfrog.com/confluence/display/JFROG/RPM+Repositories

Use timezone independent timestamps for S3

Now, when comparing the file modification time (in the case of S3), timezone dependent timestamps are used, which can lead to an update of the meta information about all packages in the case of sequential launch of mkrepo on hosts with different timezones.
Proposed to use a timestamp independent of the timezone to check the modification time of a file.

Missing `primary.xml.gz` and `filelists.xml.gz` as the cause of the fall

If primary.xml.gz and filelists.xml.gz are missing (were removed), but repomd.xml exists, no new metadata will be generated.
This can be fixed by deleting the repomd.xml.

Updating rpm repository: tarantool-dev/restored/release/2.7/el/7/x86_64
Traceback (most recent call last):
  File "/home/leonid/work/mail/mkrepo/./mkrepo.py", line 113, in <module>
    main()
  File "/home/leonid/work/mail/mkrepo/./mkrepo.py", line 109, in main
    update_repo(path, args)
  File "/home/leonid/work/mail/mkrepo/./mkrepo.py", line 57, in update_repo
    rpmrepo.update_repo(stor, args.sign, args.temp_dir, args.force)
  File "/home/leonid/work/mail/mkrepo/rpmrepo.py", line 797, in update_repo
    data = storage.read_file(initial_filelists)
  File "/home/leonid/work/mail/mkrepo/storage.py", line 151, in read_file
    s3obj.download_fileobj(buf)
  File "/home/leonid/work/mail/mkrepo/lib/python3.9/site-packages/boto3/s3/inject.py", line 758, in object_download_fileobj
    return self.meta.client.download_fileobj(
  File "/home/leonid/work/mail/mkrepo/lib/python3.9/site-packages/boto3/s3/inject.py", line 678, in download_fileobj
    return future.result()
  File "/home/leonid/work/mail/mkrepo/lib/python3.9/site-packages/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/home/leonid/work/mail/mkrepo/lib/python3.9/site-packages/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/home/leonid/work/mail/mkrepo/lib/python3.9/site-packages/s3transfer/tasks.py", line 255, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "/home/leonid/work/mail/mkrepo/lib/python3.9/site-packages/s3transfer/download.py", line 340, in _submit
    response = client.head_object(
  File "/home/leonid/work/mail/mkrepo/lib/python3.9/site-packages/botocore/client.py", line 386, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/home/leonid/work/mail/mkrepo/lib/python3.9/site-packages/botocore/client.py", line 705, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

Consider using PyFilesystem

In order to prevent reinventing the wheel, consider using https://www.pyfilesystem.org/.

Add the ability to specify the values of several fields from the "Release" file

The values for the "Description", "Label" and "Origin" fields of the "Release" file are now hardcoded in https://github.com/knazarov/mkrepo/blob/master/debrepo.py . It would be nice to add the ability to set them (for example, from env. variables).

No "Release" file created at the architecture level

As far as I can see, at the architecture level, a "Release" file typically also created.
Example: https://nginx.org/packages/ubuntu/dists/trusty/nginx/binary-amd64/

Archive: stable
Component: nginx
Origin: nginx
Label: nginx
Architecture: amd64
Description: nginx apt repository

I don't know if this is critical or not.

Feature Request: Signing RPMs

It looks like mkrepo can only sign an RPM repo's metadata, not the actual RPM file itself.

It would be a really cool feature if mkrepo could:

1. Pull down the RPMs from the s3 bucket's Packages directory
2. Check if the RPMs are signed using the key specified in ~/. rpmmacros or the default key in GPG
3. If and only if the RPMs are not signed with the key specified in ~/.rpmmacros or the default key in GPG, sign the RPMs
4. Upload the signed RPMs to the Packages directory in the s3 bucket
5. Sign the repo metadata (which it already does)

Save a history of metafiles

When creating a metafile a hash in the name is used, so a new metafile is created instead of overwriting the old one. Proposed to implement some depth of storage of such metafiles (2 for example) to give coherent information to the client at any time

Pass arbitrary GPG key to sign a repository

I have a GPG key, which is stored in an environment variable. I want to instruct mkrepo to use it to sign a repository (as for APT as well as for YUM repository).

Now it is not quite obvious, how to do so: I should add the key manually using some gpg command and somehow ensure that it is a default key.

That would be nice to access a key from an environment variable.

This is NOT about signing a package.

Make boto3 a soft requirement

Right now boto3 is a hard requirement, because it's included on top of storage.py.

It may not be needed in every case, especially when generating repositories locally.
So the S3Storage class should load boto3 library on first instantiation.

Ubuntu Impish (and higher) packages: no entry control.tar.xz in archive

When I am trying to create a repo of Tarantool packages for Ubuntu Impish, I am getting the following error:

Adding: 'pool/impish/main/t/tarantool/tarantool-common_2.10.0.g0a5ce0b9c-1_all.deb'
no entry control.tar.xz in archive
xz: (stdin): File format not recognized
tar: Child returned status 1
tar: Error is not recoverable: exiting now
Can't parse 'pool/impish/main/t/tarantool/tarantool-common_2.10.0.g0a5ce0b9c-1_all.deb':
Command 'ar -p ./.rws_32721p7q/tmpfrzfaeyn/package.deb control.tar.xz |tar -xJf - --to-stdout ./control' returned non-zero exit status 2.

It looks like control TAR archive for Ubuntu Impish (and higher) has the zst compression instead of xz.

See for details: https://balintreczey.hu/blog/hello-zstd-compressed-debs-in-ubuntu

Harcoded component of 'main' when reading DEB packages from pool

This line: https://github.com/tarantool/mkrepo/blob/master/debrepo.py#L439

as well as this line: https://github.com/tarantool/mkrepo/blob/master/debrepo.py#L495

when processing index units mean that all index units would be associated with a 'main' component.

A better approach would be updating https://github.com/tarantool/mkrepo/blob/master/debrepo.py#L495 to instead extract the component from the path (like dist) and/or update https://github.com/tarantool/mkrepo/blob/master/debrepo.py#L439 to omit the /main suffix or make it a capture group for the component name

Don't always add version epoch

Package: https://mirror.yandex.ru/centos/7/os/x86_64/Packages/antlr-C%2B%2B-2.7.7-30.el7.x86_64.rpm

Data from https://mirror.yandex.ru/centos/7/os/x86_64/repodata/2b479c0f3efa73f75b7fb76c82687744275fff78e4a138b5b3efba95f91e099e-primary.xml.gz :

<package type="rpm">
  <name>antlr-C++</name>
  <arch>x86_64</arch>
  <version epoch="0" ver="2.7.7" rel="30.el7"/>
...

Data from mkrepo:

<package type="rpm">
  <name>antlr-C++</name>
  <arch>x86_64</arch>
  <version ver="2.7.7" rel="30.el7"/>

rpm info:

# rpm -qpi antlr-C++-2.7.7-30.el7.x86_64.rpm
Name        : antlr-C++
Epoch       : 0
Version     : 2.7.7
Release     : 30.el7
Architecture: x86_64
...

Error in yum:

File "/usr/lib/python2.7/site-packages/yum/sqlitesack.py", line 642, in _pkgExcludedRKNEVRA
    e = e.lower()
AttributeError: 'NoneType' object has no attribute 'lower'

Debug in yum:

 (n,a,e,v,r) = (data['name'], data['arch'], data['epoch'], data['version'], data['release'])
...
> /usr/lib/python2.7/site-packages/yum/sqlitesack.py(642)_pkgExcludedRKNEVRA()
-> e = e.lower()
(Pdb) l
637
638             if not self._pkgExcluder:
639                 return False
640
641             data = {'n' : n.lower(), 'pkgtup' : (n, a, e, v, r), 'marked' : False}
642  ->         e = e.lower()
643             v = v.lower()
644             r = r.lower()
645             a = a.lower()

Fix "other" processing

• While the "other" is parsed, optional attributes like "rel" can absent and unfortunately this will crash.
• AFAIU, in "header_to_other" function if the CHANGELOGNAME is absent in the rpm header (that is valid, because it is optional) the program will crash in 'author': escape(author.decode('utf-8')),

Perhaps there are a couple of similar moments in "other" processing. While working on this task, it is desirable to identify and eliminate them.

Fail to sync `live/1.10/el/7/x86_64`

The tarantool repositories specific bug. It is assumed that the problem will be fixed by the tarantool team with access to the repository.

2022-06-01T15:43:52.167754425Z app[web.1]: Traceback (most recent call last):
2022-06-01T15:43:52.167954314Z app[web.1]:   File "/app/.heroku/python/bin/mkrepo", line 5, in <module>
2022-06-01T15:43:52.167969891Z app[web.1]:     mkrepo.main()
2022-06-01T15:43:52.167979377Z app[web.1]:   File "/app/.heroku/python/lib/python3.9/site-packages/mkrepo.py", line 117, in main
2022-06-01T15:43:52.168122234Z app[web.1]:     update_repo(path, args)
2022-06-01T15:43:52.168157987Z app[web.1]:   File "/app/.heroku/python/lib/python3.9/site-packages/mkrepo.py", line 59, in update_repo
2022-06-01T15:43:52.168168105Z app[web.1]:     rpmrepo.update_repo(stor, args.sign, args.temp_dir, args.force)
2022-06-01T15:43:52.168177668Z app[web.1]:   File "/app/.heroku/python/lib/python3.9/site-packages/rpmrepo.py", line 1097, in update_repo
2022-06-01T15:43:52.168187173Z app[web.1]:     primary_str = dump_primary(primary)
2022-06-01T15:43:52.168196066Z app[web.1]:   File "/app/.heroku/python/lib/python3.9/site-packages/rpmrepo.py", line 495, in dump_primary
2022-06-01T15:43:52.168413070Z app[web.1]:     for key in sorted(fmt['provides']):
2022-06-01T15:43:52.168456583Z app[web.1]: TypeError: '<' not supported between instances of 'str' and 'NoneType'
2022-06-01T15:43:52.253078579Z app[web.1]: 2022-06-01 15:43:52,252 (WARNING) Synchronization failed: live/1.10/el/7/x86_64

Pass S3 credentials via environment variables

Now we can use --s3-* options, but this way the secrets appear in the cmdline and so in the ps aux output.

It is usual and convenient to pass secrets via environment.

cElementTree.ParseError: not well-formed (invalid token)

/var/www/repo.tarantool.io/tarantool/1.7/el/7/x86_64/
Traceback (most recent call last):
  File "/usr/local/mkrepo/mkrepo.py", line 102, in <module>
    main()
  File "/usr/local/mkrepo/mkrepo.py", line 98, in main
    update_repo(path, args)
  File "/usr/local/mkrepo/mkrepo.py", line 57, in update_repo
    rpmrepo.update_repo(stor, args.sign, args.temp_dir)
  File "/usr/local/mkrepo/rpmrepo.py", line 758, in update_repo
    primary = parse_primary(gunzip_string(data))
  File "/usr/local/mkrepo/rpmrepo.py", line 203, in parse_primary
    root = ET.fromstring(data)
  File "<string>", line 124, in XML
cElementTree.ParseError: not well-formed (invalid token): line 931, column 51

Set ACL for uploaded files

We need to be able to set the ACL (set read only for all) for the files (generated metainformation) uploaded to S3. Seems like this option can be set over arguments or / and environment variable.

Add work with signed `.dsc` files

Now, if we use a signed .dsc file, we get an error like:

Traceback (most recent call last):
  File "/Workspace/mkrepo/./mkrepo", line 5, in <module>
    mkrepo.main()
  File "/Workspace/mkrepo/mkrepo.py", line 117, in main
    update_repo(path, args)
  File "/Workspace/mkrepo/mkrepo.py", line 56, in update_repo
    debrepo.update_repo(stor, args.sign, args.temp_dir, args.force)
  File "/Workspace/mkrepo/debrepo.py", line 851, in update_repo
    process_index_units(repo_info, tempdir, 'sources')
  File "/Workspace/mkrepo/debrepo.py", line 707, in process_index_units
    unit.parse_dsc(local_file, file_path, mtime)
  File "/Workspace/mkrepo/debrepo.py", line 194, in parse_dsc
    self.parse_string(file.read())
  File "/Workspace/mkrepo/debrepo.py", line 235, in parse_string
    key, value = line.split(':', 1)
ValueError: not enough values to unpack (expected 2, got 1)

Example of the signed .dsc file: https://ftp.debian.org/debian/pool/main/3/389-admin/389-admin_1.1.35-2.dsc
Documentation: https://debian-handbook.info/browse/da-DK/stable/sect.source-package-structure.html

"filelists.xml" is not recoverable

The information about which packets metainformation should be updated is generated according to the primary.xml, consequently, if the filelists.xml was removed, the information about the previously existing packages will not be added to it.

Support Debian packages with a three number version

Tarantool Debian package files are named like tarantool_2.2.1.47.gd50b9d6-1_amd64.deb. upstream_version component is 2.2.1.47.gd50b9d6 and consists of three numbers from a tag, one number from git describe and a git commit hash. We however have repositories with two number tags and so a package will be named like tarantool-gis_0.1.22-1_amd64.deb. In this case debian_revision component (named dist in mkrepo code) will be determined as 'all', while it is expected to be '1'. Not sure about consequenses, but it looks incorrect.

The problem appears due to this regexp (there is also 2.2.1.47 vs 2.2.1-47 problem, see #14, but I'm not about that here):

mkrepo/debrepo.py

Line 248 in 4153b32

expr = r'^(?P<package>[^_]+)_(?P<version>[0-9]+\.[0-9]+\.[0-9]+\-[0-9]+)(\.(?P<revision>[^\-]+))?([\-]?(?P<dist>[^_]+))?_(?P<arch>[^\.]+)\.deb$'

To be honest I don't understood why we separate revision from a version here: it is not used anywhere and is not described in Debian policy as I see. Maybe we can simplify things and parse only debian_revision (dist) and architecture from a package file name considering everything before as a package name and an upstream_version. I think we can lean on a last hyphen to do so (NB: verify that there are no architecture names that contain a hyphen). The Debian policy specifies a version in the quite free form, so we should allow it to be in such form: [A-Za-z0-9.+~-]+.

We should also verify that things works well for RPM packages. I suggest to download all tarantool packagecloud repositories and check that all packages are parsed well. We can even create a unit test based on this. Maybe also worth to download a list of all package names in official Debian repositories and add them to the test.

@knazarov Are you have a vision on that?

Python version conflict

I tried to use mkrepo and it raised me next error:

Traceback (most recent call last):
  File "./mkrepo.py", line 111, in <module>
    main()
  File "./mkrepo.py", line 107, in main
    update_repo(path, args)
  File "./mkrepo.py", line 57, in update_repo
    rpmrepo.update_repo(stor, args.sign, args.temp_dir, args.force)
  File "/Users/v.ioffe/mkrepo/rpmrepo.py", line 888, in update_repo
    primary_str, primary_gz, revision, storage)
  File "/Users/v.ioffe/mkrepo/rpmrepo.py", line 758, in generate_repomd
    other = gzip.decompress(other_gz)
AttributeError: 'module' object has no attribute 'decompress'

I resolved it by thus diff:

v.ioffe@v-ioffe mkrepo % git diff
diff --git a/mkrepo.py b/mkrepo.py
index edfa745..d1ebcd2 100755
--- a/mkrepo.py
+++ b/mkrepo.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 import argparse
 import storage

It seems that python2 is not working with mkrepo.

Make mkrepo fault-tolerant

The utility should correctly handle cases of missing parts of the metafiles and cases when they can't be parsed. In such cases, we can delete the metafiles and regenerate them. This behavior should be enabled when using the "force" mode

Add an ability to specify "suite" and "version" for the Release file

It would be nice to add the ability to specify "suite" and "version" fields for the Release file via environment variables.

Add the ability to "recreate" a repository

I propose to add the ability to "recreate" a repository by setting some flag on start (--recreate, for example). In this case. mkrepo should remove the old repository metainformation without parsing it and create a new one.

Crashing while dump "primary" file

The problem is that in function "parse_ver_str" for "rel" by default the "None" is used.

Traceback (most recent call last):
  File "/home/leonid/work/mail/mkrepo/./mkrepo", line 4, in <module>
    mkrepo.main()
  File "/home/leonid/work/mail/mkrepo/mkrepo.py", line 116, in main
    update_repo(path, args)
  File "/home/leonid/work/mail/mkrepo/mkrepo.py", line 58, in update_repo
    rpmrepo.update_repo(stor, args.sign, args.temp_dir, args.force)
  File "/home/leonid/work/mail/mkrepo/rpmrepo.py", line 1071, in update_repo
    primary_str = dump_primary(primary)
  File "/home/leonid/work/mail/mkrepo/rpmrepo.py", line 496, in dump_primary
    for key in sorted(fmt['provides']):
TypeError: '<' not supported between instances of 'str' and 'NoneType'

fmt['provides']:

{
  ('tarantool-lrexlib-gnu', '0', None, '2.9.0.4'):{'name': 'tarantool-lrexlib-gnu', 'epoch': '0', 'rel': None, 'ver': '2.9.0.4', 'flags': 'EQ'},
  ('tarantool-lrexlib-gnu', '0', '1.el7.centos', '2.9.0.4'):{'name': 'tarantool-lrexlib-gnu', 'epoch': '0', 'rel': '1.el7.centos', 'ver': '2.9.0.4', 'flags': 'EQ'},
  ('tarantool-lrexlib-gnu(x86-64)', '0', '1.el7.centos', '2.9.0.4'):{'name': 'tarantool-lrexlib-gnu(x86-64)', 'epoch': '0', 'rel': '1.el7.centos', 'ver': '2.9.0.4', 'flags': 'EQ'}
}

https://rws.tarantool.org/live/1.10/el/7/x86_64

AFAIU the same problem can occur if "rel" is absent for a package in a previously generated "primary" file.

Using repodata subfolder rather than Packages subfolder to identify RPM repos

I have an existing RPM repo (in Artifactory) that didn't make use of Packages folders. That suggests to me that perhaps the repodata folder is a better indicator of an RPM repo here: https://github.com/tarantool/mkrepo/blob/master/mkrepo.py#L22

revision field from repomd.xml is not parsed

As far as I can see, the revision field is not parsed and in fact the default value is always used.

Add "sources" Indices for deb repositories

AFAIU now "mkrepo" creates only "packages" Indices. For more complete support of deb repositories necessary to add support of "sources" indexes.

Support *.tar.xz files in deb and current Tarantool core versioning

Since the latest ubuntu/bionic the new default pack format for debian packages is in use by dpkg tool:
tar.gz -> tar.xz
also Tarantool core packages uses versioning names like:
tarantool_2.3.0.62.g6c627af39-1_amd64.deb
but not as expected:
tarantool_2.3.0-62.g6c627af39-1_amd64.deb

Tests

It would be nice to have tests to detect regressions early.

• .deb and .rpm repo generation and parsing logic
• Integration with s3

Clean old files in repodata

mkrepo don't clean old files in repodata
https://gyazo.com/795a5ead6e5e5a6aefc8f048684bc10d

header for src rpm files contains wrong arch field

If we are using

kernel-rt-3.10.0-693.11.1.rt56.632.el7.src.rpm
kernel-rt-3.10.0-1127.rt56.1093.el7.src.rpm

See

The header contains the wrong arch for both.

v.ioffe@v-ioffe mkrepo % grep arch ./tmp_mkdir/repodata/filelists.xml
<package pkgid="fb7b23264c62705fe7174cad2b8325b7de3ef3b8bcaaf5705d47e3dac47bad4a" name="kernel-rt" arch="noarch">
<package pkgid="0ba4724f341b1b5581c31dde3294ef1f53e9f492fd382cc1415e33c6ef4a08b4" name="kernel-rt" arch="x86_64">

Expected arch="src"

Generating an invalid xml file with the & symbol

The utility generates a file that does not pass xml validation and which is not read by yum.

example:
package: https://mirror.yandex.ru/centos/7/os/x86_64/Packages/hunspell-el-0.8-7.el7.noarch.rpm

result primary.xml:

<?xml version="1.0" encoding="UTF-8"?>
<metadata xmlns="http://linux.duke.edu/metadata/common" xmlns:rpm="http://linux.duke.edu/metadata/rpm" packages="1">
<package type="rpm">
  <name>hunspell-el</name>
  <arch>noarch</arch>
  <version epoch="1" ver="0.8" rel="7.el7"/>
  <checksum type="sha256" pkgid="YES">b42a5d9db2bd1ca5fc8853ee81b90ac33c47c460cc85ff8d0b480e4ed89e8c98</checksum>
  <summary>Greek hunspell dictionaries</summary>
  <description>Greek hunspell dictionaries.</description>
  <packager>CentOS BuildSystem &lt;http://bugs.centos.org&gt;</packager>
  <url>http://ispell.math.upatras.gr/?section=oofficespell&subsection=howto</url>
  <time file="1694596934" build="1402344145"/>
  <size package="1414768" installed="7025304" archive="7026320"/>
  <location href="Packages/hunspell-el-0.8-7.el7.noarch.rpm"/>
  <format>
    <rpm:license>GPLv2+ or LGPLv2+ or MPLv1.1</rpm:license>
    <rpm:vendor>CentOS</rpm:vendor>
    <rpm:group>Applications/Text</rpm:group>
    <rpm:buildhost>worker1.bsys.centos.org</rpm:buildhost>
    <rpm:sourcerpm>hunspell-el-0.8-7.el7.src.rpm</rpm:sourcerpm>
    <rpm:header-range start="1384" end="5208"/>
    <rpm:provides>
      <rpm:entry name="hunspell-el" flags="EQ" epoch="1" ver="0.8" rel="7.el7"/>
    </rpm:provides>
    <rpm:requires>
      <rpm:entry name="hunspell"/>
    </rpm:requires>
    <rpm:obsoletes>
    </rpm:obsoletes>
  </format>
</package>
</metadata>

validation result:
Error : InvalidChar Line : 11 Message : char '&' is not expected.

yum error:
(process:30319): GLib-WARNING **: 09:31:48.638: GError set over the top of a previous GError or uninitialized memory. This indicates a bug in someone's code. You must ensure an error is NULL before it's set. The overwriting error message was: Parsing primary.xml error: EntityRef: expecting ';'