Coder Social home page Coder Social logo

hughsie / appstream-glib Goto Github PK

View Code? Open in Web Editor NEW
63.0 10.0 102.0 10.44 MB

This library provides objects and helper methods to help reading and writing AppStream metadata.

License: GNU Lesser General Public License v2.1

Shell 0.26% C 98.16% C++ 0.01% QMake 0.01% M4 0.39% Meson 1.17%

appstream-glib's Introduction

⚠️ WARNING: appstream-glib is heavy maintenance mode, use appstream instead

AppStream-Glib

This library provides GObjects and helper methods to make it easy to read and write AppStream metadata. It also provides a simple DOM implementation that makes it easy to edit nodes and convert to and from the standardized XML representation. It also supports reading of Debian-style DEP-11 metadata.

What this library allows you to do:

  • Read and write compressed AppStream XML files
  • Read compressed Debian YAML files
  • Add and search for applications in an application store
  • Get screenshot image data and release announcements
  • Easily retrieve the best application data for the current locale
  • Efficiently interface with more heavy-weight parsers like expat

For more information about what AppStream is, please see the wiki here: https://www.freedesktop.org/wiki/Distributions/AppStream/

Getting Started

To install the libappstream-glib library you either need to install the libappstream-glib package from your distributor, or you can build a local copy. To do the latter just do:

dnf install docbook-utils gettext-devel glib-devel \
            gobject-introspection-devel gperf gtk-doc gtk3-devel \
            json-glib-devel libarchive-devel libcurl-devel \
            libuuid-devel libyaml-devel \
            meson rpm-devel
mkdir build && cd build
meson .. --prefix=/opt -Dbuilder=false
ninja

Hacking

If you want a new feature, or have found a bug or a way to crash this library, please report as much information as you can to the issue tracker: https://github.com/hughsie/appstream-glib/issues -- patches very welcome.

New functionality or crash fixes should include a test in libappstream-builder/ as-self-test.c to ensure we don't regress in the future. New functionality should also be thread safe and also not leak any memory for success or failure cases.

Translations

Translations of the natural language strings are managed through a third party translation interface, transifex.com. Newly added strings will be periodically uploaded there for translation, and any new translations will be merged back to the project source code.

Please use https://www.transifex.com/projects/p/appstream-glib/ to contribute translations, rather than sending pull requests.

AppStream-Builder

appstream-builder is a tool that allows us to create AppStream metadata from a directory of packages. It is typically used when generating distribution metadata, usually at the same time as modifyrepo or createrepo.

What this tool does:

  • Searches a directory of packages and reads just the RPM header of each.
  • If a package contains an interesting file, just the relevant files are decompressed from the package archive.
  • A set of plugins are run on the extracted files, and if these match certain criteria AsbApplication objects are created.
  • Any screenshots referenced are downloaded to a local cache. This is optional and can be disabled with --nonet.
  • When all the packages are processed, some of the AsbApplication objects are merged into single applications. This is how fonts are collected.
  • The AsbApplication objects are serialized to XML and written to a compressed archive.
  • Any application icons or screenshots referenced are written to a .tar archive.

Getting Started

To run appstream-builder you either need to install the package containing the binary and data files, or you can build a local copy. To do the latter just do:

dnf install docbook-utils gettext-devel glib-devel \
            gobject-introspection-devel gperf gtk-doc gtk3-devel \
            libarchive-devel libsoup-devel \
            libuuid-devel libyaml-devel \
            meson rpm-devel rpm-devel
mkdir build && cd build
meson .. --prefix=/opt -Dbuilder=true
ninja

To actually run the extractor you can do:

./client/appstream-builder --verbose   \
                  --max-threads=8 \
                  --log-dir=/tmp/logs \
                  --packages-dir=/mnt/archive/Megarpms/21/Packages \
                  --temp-dir=/mnt/ssd/AppStream/tmp \
                  --output-dir=./repodata \
                  --screenshot-url=http://megarpms.org/screenshots/ \
                  --basename="megarpms-21"

Note: it is possible to use "globs" like /mnt/archive/Megarpms/21/Packages* to match multiple directories or packages.

This will output a lot of progress text. Now, go and make a cup of tea and wait patiently if you have a lot of packages to process. After this is complete you should finally see:

Writing ./repodata/megarpms-21.xml.gz
Writing ./repodata/megarpms-21-failed.xml.gz
Writing ./repodata/megarpms-21-ignore.xml.gz
Writing ./repodata/megarpms-21-icons.tar
Done!

You now have two choices what to do with these files. You can either upload them with the rest of the metadata you ship (e.g. in the same directory as repomd.xml and primary.sqlite.bz2) which will work with Fedora 22 and later:

modifyrepo_c \
    --no-compress \
    /tmp/asb-md/appstream.xml.gz \
    /path/to/repodata/
modifyrepo_c \
    --no-compress \
    /tmp/asb-md/appstream-icons.tar.gz \
    /path/to/repodata/

You can then do something like this in the megarpms-release.spec file:

Source1:   http://www.megarpms.org/temp/megarpms-20.xml.gz
Source2:   http://www.megarpms.org/temp/megarpms-20-icons.tar.gz

%install
DESTDIR=%{buildroot} appstream-util install %{SOURCE1} %{SOURCE2}

This ensures that gnome-software can access both data files when starting up.

What is an application

Applications are defined in the context of AppStream as such:

  • Installs a desktop file and would be visible in a desktop
  • Has an metadata extractor (e.g. libappstream-builder/plugins/asb-plugin-gstre amer.c) and includes an AppData file

Guidelines for applications

These guidelines explain how we filter applications from a package set.

First, some key words:

  • SHOULD: The application should do this if possible
  • MUST: The application or addon must do this to be included
  • CANNOT: the application or addon must not do this

The current rules of inclusion are thus:

  • Icons MUST be installed in /usr/share/pixmaps/*, /usr/share/icons/*, /usr/share/icons/hicolor/*/apps/*, or /usr/share/${app_name}/icons/*
  • Desktop files MUST be installed in /usr/share/applications/ or /usr/share/applications/kde4/
  • Desktop files MUST have Name, Comment and Icon entries
  • Valid applications with NoDisplay=true MUST have an AppData file.
  • Applications with Categories=Settings, Categories=ConsoleOnly or Categories=DesktopSettings MUST have an AppData file.
  • Applications MUST have had an upstream release in the last 5 years or have an AppData file.
  • Application icon MUST be available in 48x48 or larger
  • Applications must have at least one main or additional category listed in the desktop file or supply an AppData file. See https://specifications.freedesktop.org/menu-spec/latest/apa.html and https://specifications.freedesktop.org/menu-spec/latest/apas02.html for the full Categories list.
  • Codecs MUST have an AppData file
  • Input methods MUST have an AppData file
  • If included, AppData files MUST be valid XML
  • AppData files MUST be installed into /usr/share/metainfo
  • Application icons CANNOT use XPM or ICO format
  • Applications CANNOT use obsolete toolkits such as GTK+-1.2 or QT3
  • Applications that ship a desktop file SHOULD include an AppData file.
  • Screenshots SHOULD be in 16:9 aspect ratio
  • Application icons SHOULD have an alpha channel
  • Applications SHOULD ship a 256x256 PNG format icon or scalable SVG
  • Applications SHOULD ship a matching High Contrast icon
  • Applications SHOULD not depend on other applications
  • AppData files SHOULD include translations
  • Desktop files SHOULD include translations

Guidelines for fonts

  • Fonts MUST have a valid MetaInfo file installed to /usr/share/appdata
  • Fonts packaged in multiple packages SHOULD have multiple MetaInfo files
  • Fonts families SHOULD only have one description section
  • Fonts of different styles or weights of the same family SHOULD use <exte nds>
  • MetaInfo files SHOULD include translations where possible

License

LGPLv2+

appstream-glib's People

Contributors

alexlarsson avatar decathorpe avatar dimstar77 avatar dos1 avatar ebassi avatar fmuellner avatar hadess avatar heftig avatar hughsie avatar iainlane avatar ignatenkobrain avatar joaquimrocha avatar kalev avatar lantw44 avatar mavit avatar mcatanzaro avatar mingcongbai avatar norwayfun avatar oturpe avatar pinotree avatar piotrdrag avatar pohly avatar pwithnall avatar robert-ancell avatar samm-git avatar smcv avatar stefanbruens avatar suve avatar vogtinator avatar ximion avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

appstream-glib's Issues

appstream-util install installed files with wrong rights.

After running sudo appstream-util install ~/Téléchargements/arch.xml.gz, I get wrong rights in /usr/share/app-info/arch.xml.gz

Rights on the installed file :

ls -l /usr/share/app-info/xmls/
total 544
-rw-r----- 1 root root 548886 11 sept. 17:01 arch.xml.gz

Chmod 644 on /use/share/app-info/xmls/arch.xml.gz solved the problem.

Push appdata information to bodhi

Hi there,

I suggested the fedora infrastructure to add more information about packages in test by using your work to gather information.

Hopefully, this should help :

  • users to make easier search
  • better appdata visibility and improvements

This enhancement has be written here : fedora-infra/bodhi#749

You may have suggestions or knowledge about already existing prerequisites, don't hesitate to go comment.

Crashes on assertion in as_release_set_description()

Hi!
ASGlib crashes on non-null assertion when loading the following valid AppStream XML file:
http://people.freedesktop.org/~mak/stuff/neverball-lirepo/indices/amd64/Metadata.xml.gz

This is highly likely because the file doesn't contain a description for releases.
GDB backtrace:

as_release_node_parse (release=release@entry=0x7fffd000c430, node=node@entry=0x7fffd000a390, 
    ctx=ctx@entry=0x7fffd000abb0, error=error@entry=0x7fffe37fd900) at as-release.c:477
477                             as_release_set_description (release,
(gdb) bt full
#0  as_release_node_parse (release=release@entry=0x7fffd000c430, node=node@entry=0x7fffd000a390, 
    ctx=ctx@entry=0x7fffd000abb0, error=error@entry=0x7fffe37fd900) at as-release.c:477
        xml = 0x0
        n = 0x7fffd000a400
        tmp = <optimized out>
        taken = <optimized out>
        i = <optimized out>
#1  0x00007ffff7bb5ec4 in as_app_node_parse_child (error=0x7fffe37fd900, ctx=0x7fffd000abb0, 
    flags=AS_APP_PARSE_FLAG_NONE, n=0x7fffd000a360, app=0xb7eb40) at as-app.c:3709
        r = 0x7fffd000c430
        c = 0x7fffd000a390
        tmp = <optimized out>
        taken = <optimized out>
#2  as_app_node_parse_full (app=app@entry=0xb7eb40, node=node@entry=0x7fffd000a240, 
    flags=flags@entry=AS_APP_PARSE_FLAG_NONE, ctx=ctx@entry=0x7fffd000abb0, error=error@entry=0x7fffe37fd900)
    at as-app.c:3839
        n = 0x7fffd000a360
        tmp = <optimized out>
        prio = <optimized out>
#3  0x00007ffff7bb69ad in as_app_node_parse (app=app@entry=0xb7eb40, node=node@entry=0x7fffd000a240, 
    ctx=ctx@entry=0x7fffd000abb0, error=error@entry=0x7fffe37fd900) at as-app.c:3866
No locals.
#4  0x00007ffff7bc4788 in as_store_from_root (store=store@entry=0x7bb720, root=root@entry=0xbb8320, 
    icon_root=<optimized out>, icon_root@entry=0x7fffd0009df0 "/usr/share/app-info/icons", 
    error=error@entry=0x7fffe37fdba8) at as-store.c:827
        error_local = 0x0
        app = 0xb7eb40
        apps = <optimized out>
        n = 0x7fffd000a240
        tmp = <optimized out>
        ctx = 0x7fffd000abb0
        icon_path = 0x7fffd00169a0 "/usr/share/app-info/icons/6375db4bb7186c39a4860544884575f7"
        __FUNCTION__ = "as_store_from_root"
#5  0x00007ffff7bc4d96 in as_store_from_file (store=store@entry=0x7bb720, file=file@entry=0x8951a0, 
    icon_root=icon_root@entry=0x7fffd0009df0 "/usr/share/app-info/icons", cancellable=cancellable@entry=0x0, 
    error=error@entry=0x7fffe37fdba8) at as-store.c:961
        filename = 0x7fffd0009f70 "/usr/share/app-info/xmls/limba_6375db4bb7186c39a4860544884575f7.xml.gz"
        error_local = 0x0
        root = 0xbb8320
        __FUNCTION__ = "as_store_from_file"
#6  0x00007ffff7bc4f39 in as_store_load_app_info_file (error=0x7fffe37fdba8, cancellable=0x0, 
    icon_root=0x7fffd0009df0 "/usr/share/app-info/icons", 
    path_xml=0x7fffd0009e20 "/usr/share/app-info/xmls/limba_6375db4bb7186c39a4860544884575f7.xml.gz", 
    store=0x7bb720) at as-store.c:1427
        file = 0x8951a0
#7  as_store_load_app_info (store=store@entry=0x7bb720, path=path@entry=0x7fffd0001d50 "/usr/share/app-info", 
    format=format@entry=0x7ffff7bcc23d "xmls", cancellable=cancellable@entry=0x0, error=error@entry=0x7fffe37fdba8)
    at as-store.c:1514
        filename_md = 0x7fffd0009e20 "/usr/share/app-info/xmls/limba_6375db4bb7186c39a4860544884575f7.xml.gz"
        tmp = <optimized out>
        dir = 0x7fffd0001830
        error_local = 0x0
        icon_root = 0x7fffd0009df0 "/usr/share/app-info/icons"
        path_md = 0x7fffd0001d80 "/usr/share/app-info/xmls"

Please add more values to enum AsAppState

Please add these values to the enum AsAppState corresponding to some enum PkInfoEnum values:

Requested AsAppState values Existing PkInfoEnum values
AS_APP_STATE_UPDATING PK_INFO_ENUM_UPDATING
AS_APP_STATE_REINSTALLING PK_INFO_ENUM_REINSTALLING
AS_APP_STATE_DOWNGRADING PK_INFO_ENUM_DOWNGRADING
AS_APP_STATE_UNTRUSTED PK_INFO_ENUM_UNTRUSTED

I need them for https://bugzilla.gnome.org/show_bug.cgi?id=711171.

More details: I'm going to use PkTask in simulate mode. It provides a list of dependent packages which will also be changed during the current transaction. The action which would be taken on every dependent package is returned by pk_package_get_info () and it may be one of some PK_INFO_ENUM_... values, an example is pk_task_text_simulate_question_type_to_string (). (Actually, I can't imagine why REINSTALLING or UNTRUSTED should be returned but I can easily imagine examples when a package will be updated or downgraded.) Then I'm going to convert a list of PkPackage to a list of GsApp using gs_plugin_packagekit_add_results () to have the high quality and translated names of the packages as well as to be able to check whether a package is an application, an add-on, a core system component which cannot be removed, or a less important system component. This function is not perfect because it only recognizes PK_INFO_ENUM_INSTALLED and PK_INFO_ENUM_AVAILABLE. I can easily add PK_INFO_ENUM_INSTALLING and PK_INFO_ENUM_REMOVING but not the other values because I have no AS_APP_STATE_... counterpart for them. Unfortunately, for all other PK_INFO_ENUM... flags AS_APP_STATE_UNKNOWN is set as the GsApp state and it will make me unable to display an info like "the application Asdf will be updated and the application Ghjkl will be reinstalled."

[translations] Typo in German usage string

Currently, it responds to appstream-util --help in German with:

Anwendungsoptionen:
  --nonet           Verwenden Sie keinen Netzwerkzugriff

The second line says "You shall not use the network", which is the wrong person.

I already requested translation permission on Transifex, but without this issue I would forget about this issue.

A correct phrasing looks like this:

Anwendungsoptionen:
  --nonet           Keinen Netzwerkzugriff verwenden

Sources: I'm a native German speaker. I'm not sure whether my translation is good, but at least it's better than the current.

appstream-util install-origin no longer seems to work

As openSUSE newly has the appdata.xml.gz and appdata-icons.tar.gz published in the repository data (autogenerated on creating the published tree), I wanted to work further on using installing those in the end on machines.

For this, install-origin was introduced earler (as ALL repos would end up with an appdata.xml.gz, we cannot simply install the file; it needs to be renamed and updated, which was what install-origin would do).

with a 'valid' appdata.xml.gz and appdata-icons.tar.gz in $PWD (both files generated by appstream-builder), all I get is:

> ls /usr/share/app-info/xmls/

> appstream-util -v install-origin Tumbleweed appdata.xml.gz appdata-icons.tar.gz

> ls /usr/share/app-info/xmls/

>

recursive option

(appstream-builder:3571): As-WARNING **: No idea how to handle 20/release

there 20/release/*.rpm
I think we should add --recursive option. If it's activated - search rpms for all folders.
If not activated - skip and don't throw error.

dump works not correctly with using with .in files

$ cat Documents/git/upstream/evolution-rss/evolution-rss.metainfo.xml.in
<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright 2014 Igor Gnatenko <[email protected]> -->
<component type="addon">
<id>evolution-rss</id>
<extends>evolution.desktop</extends>
<_name>RSS Reader</_name>
<_summary>Read rss feeds</_summary>
<url type="homepage">http://gnome.eu.org/evo/index.php/Evolution_RSS_Reader_Plugin</url>
<metadata_license>CC0-1.0</metadata_license>
<project_license>GPL-2.0 and GPL-2.0+</project_license>
<updatecontact>[email protected]</updatecontact>
</component>
$ appstream-util dump Documents/git/upstream/evolution-rss/evolution-rss.metainfo.xml.in
<?xml version="1.0" encoding="UTF-8"?>
<components version="1.0">
 <component type="addon">
  <id>evolution-rss</id>
  <name>RSS Reader</name>
  <summary>Read rss feeds</summary>
  <project_license>GPL-2.0 and GPL-2.0+</project_license>
  <url type="homepage">http://gnome.eu.org/evo/index.php/Evolution_RSS_Reader_Plugin</url>
  <extends>evolution.desktop</extends>
 </component>
</components>

Some bugs:

  • It just dropping i18n tags
  • Adds \n at end of file. it shouldn't do this I think.
  • Do sorting of tags. Bug or feature ?

CAB files within directories fail to parse

Some CAB files need to be built with all the files contained in the DriverPackage directory in order to pass the signing process. It seems that when the CAB file is parsed the direction of the slashes get mixed up for how UNIX normally expected to parse them.

# G_MESSAGES_DEBUG=all appstream-util validate st.cab -v
st.cab: (appstream-util:27058): As-DEBUG: run AsStore:store-from-file
(appstream-util:27058): As-DEBUG: found file 0  DriverPackage\firmware.bin
(appstream-util:27058): As-DEBUG: found file 1  DriverPackage\firmware.inf
(appstream-util:27058): As-DEBUG: found file 2  DriverPackage\firmware.metainfo.xml
/tmp/appstream-glib-VEFE6X/DriverPackage\firmware.metainfo.xml could not be loaded: /tmp/appstream-glib-VEFE6X/DriverPackage\firmware.metainfo.xml could not be read: Failed to open file '/tmp/appstream-glib-VEFE6X/DriverPackage\firmware.metainfo.xml': No such file or directory

The DriverPackage* path comes out from what gcab provides via

gcab_cabinet_extract_simple

I18N

Some strings aren't marked for translation.

In client/as-utils.c (some strings appear more times):
"Old API version: %.2f\n"
"Conversion %s->%s not implemented"
"File format '%s' cannot be upgraded"
"Not enough arguments, expected .appdata.xml"
"Format not recognised" (and there is typo s->z)
"Unable to find version in: %s"
"Unable to find release in: %s"
"Unable to parse release in: %s"
"Unable to create release in: %s"
"Failed to detect section '%s'"
"Not enough arguments, expected app.desktop new.appdata.xml"
"Not enough arguments, expected data.xml"
"Not enough arguments, expected search terms"
"Not enough arguments, expected filename(s)"
"Not enough arguments, expected origin filename(s)"
"Failed to delete %s"
"Not enough arguments, expected appstream-id"
"No desktop applications found"
"Not enough arguments, expected filename.xml.gz status.html"
"Not enough arguments, expected matrix.xml filename.xml.gz"
"Not enough arguments, expected filename.xml.gz status.csv"
"Not enough arguments, expected filename.xml.gz applications-to-import.yaml"
"Not enough arguments, expected example.appdata.xml"
"%s has no Icon"
"%s missing icon %s: "
"%s invalid icon %s: "
"%s undersized icon (%ix%i)"
"%s is too small to be used: %ix%i"
"Could not parse '%s' as a URL"
"Downloading failed: %s"
"Not enough arguments, expected: file url [cachedir] [outputdir]"
"Not enough arguments, expected: file url"
"Not enough arguments, expected: file screenshot"
"Not enough arguments, expected STRING"
"Failed to create: %s"
"Not enough arguments, expected: old.xml new.xml"
"Not enough arguments, expected: old.xml helper.xml new.xml"
"No argument expected"
"No applications found"
"Failed to check root, %i problems detected" (require ngettext)

In client/as-builder.c:
"No ostree support enabled"
"Parsed %i/%i files..." (it si marked, but it requires ngettext)

And maybe other.

<description> has no translations

$ appstream-util validate-strict ../claws-mail.appdata.xml.in 
../claws-mail.appdata.xml.in: 
translations-required   <description> has no translations
Validation failed
<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright 2014 Colin Leroy <[email protected]> and the Claws Mail team -->
<application>
  <id type="desktop">claws-mail.desktop</id>
  <metadata_license>CC-BY-SA-3.0</metadata_license>
  <project_license>GPL-3.0+</project_license>
  <description>
    <_p>
      Claws Mail is a fast, powerful and very extensible email client.
    </_p>
    <_p>
      It is highly configurable and handles hundreds of thousands of emails
      easily. Messages are managed in an open format, and are easy to interact
      with.
    </_p>
    <_p>
      Lots of extra functionality is provided by plugins, such as PGP signatures
      and encryption, an RSS aggregator, a calendar, powerful spam filtering,
      Perl and Python interactions, HTML and PDF rendering, and more.
    </_p>
  </description>
  <screenshots>
    <screenshot type="default" width="928" height="522">http://www.claws-mail.org/appdata/claws-main.png</screenshot>
    <screenshot width="928" height="522">http://www.claws-mail.org/appdata/claws-setup.png</screenshot>
  </screenshots>
  <url type="homepage">http://www.claws-mail.org/</url>
  <updatecontact>[email protected]</updatecontact>
</application>

I've translated description per tags. I think appstream-glib should handle all tags inside desctiption and if they're translated - not throw error.

Does not read MediaBaseUrl field in DEP-11 files

Hi!
The MediaBaseUrl field in DEP-11 files is used as a "prefix" for media URLs, i.e. URLs of screenshots or icons of the remote type.
See the specification: http://www.freedesktop.org/software/appstream/docs/sect-AppStream-DEP11.html#spec-dep11-general

It fills in the same role Origin already has, and allows for easily switching the server which serves the media, which is a desired feature at Debian for derivatives as well as it will become useful in a setup where multiple mirrors exist for media content.

With the current implementation, asglib prevents GNOME-Software from showing any screenshots at all on Debian.
reference files for testing: ftp://ftp.debian.org/debian/dists/sid/main/dep11/

Cheers,
Matthias

Assumes cached icons in the wrong location

Hi!
I have the following AppStream distro XML file: http://people.freedesktop.org/%7Emak/stuff/neverball-lirepo/indices/amd64/Metadata.xml.gz

The file defines a neverball.desktop component, which has a cached icon set:

<icon type="cached">neverball-1.6.0.png</icon>

Icons of the size 128x128 are located in /usr/share/app-info/icons/6375db4bb7186c39a4860544884575f7/128x128/ (yes, "6375db4bb7186c39a4860544884575f7" unfortunately is the repository name in this case - fortunately, this doesn't need to be read by humans and is very convenient to be processed by machines). There are no 64x64 icons provided.

I would expect asglib to find the 128x128 icon in the respective directory, and just scale it down.
For some reason, asglib sets the icon to be /usr/share/app-info/icons/6375db4bb7186c39a4860544884575f7/neverball-1.6.0.png, which is completely wrong.

This is probably due to the <icon/> tag not having a size defined, which is totally fine spec-wise, as the AppStream client should in that case just test the icon cache directories to find the right icon (ideally in the 64x64 and 128x128 dir, legacy locations are not important anymore).

The icons should be searched for where the XML was located, so in /var/cache or /usr/share/, searching in both is not necessary (maybe that needs to be clarified in the spec...) Ref: #53

Cheers,
Matthias

asb-self-test fails

The sha1sums of the generated files data/tests/colorhug-als-2.0.*.cab don't match the hardcoded values in the test case (asb_test_firmware_func())

These are the sha1sums for the generated files on my system:
d7b9ae5b36029de606c9b047c229d9c8f5e3807c colorhug-als-2.0.0.cab
7d4cab8775213cbd223a4a3b236618ced0af9447 colorhug-als-2.0.1.cab
6301e24ae5fb2aa73c336fd40a5d2f1835afe088 colorhug-als-2.0.2.cab

appstream-util validate complains that there's no <name> when an <id> is present

Validating the appdata file for sound-juicer with appstream-util 0.34 fails as it wants a <name> tag even though there is an <id> tag in the file. With appstream-util 0.30 it passes. The documentation for <name> says

While this tag is requited for a generic component, for an application metainfo file it is not necessary, but only recommended. You can omit this tag if you want the software center to have the same strings as defined in the XDG desktop file. In some cases it might be required to have a different name in the app-store, but most appdata.xml files will not need this.

I don't know if the documentation is out of date and I should add a tag or if appstream-glib validate is being overzealous.

Failed to read package file list

(appstream-builder:13601): As-WARNING **: Failed to read package file list ./nonfree/java-1.6.0-sun-demo-1.6.0.41-1.R.x86_64.rpm

Make check fails on ppc64

Running make check on a PPC64 build host results in

[ 96s] make check-TESTS
[ 96s] make[3]: Entering directory '/home/abuild/rpmbuild/BUILD/appstream-glib-0.3.5/libappstream-builder'
[ 96s] make[4]: Entering directory '/home/abuild/rpmbuild/BUILD/appstream-glib-0.3.5/libappstream-builder'
[ 96s] ../test-driver: line 107: 11605 Aborted "$@" > $log_file 2>&1
[ 96s] FAIL: asb-self-test
[ 96s] ============================================================================
[ 96s] Testsuite summary for appstream-glib 0.3.5
[ 96s] ============================================================================
[ 96s] # TOTAL: 1
[ 96s] # PASS: 0
[ 96s] # SKIP: 0
[ 96s] # XFAIL: 0
[ 96s] # FAIL: 1
[ 96s] # XPASS: 0
[ 96s] # ERROR: 0
[ 96s] ============================================================================
[ 96s] See libappstream-builder/test-suite.log
[ 96s] ============================================================================
[ 96s] Makefile:1253: recipe for target 'test-suite.log' failed
[ 96s] make[4]: *** [test-suite.log] Error 1
[ 96s] make[4]: Leaving directory '/home/abuild/rpmbuild/BUILD/appstream-glib-0.3.5/libappstream-builder'
[ 96s] Makefile:1359: recipe for target 'check-TESTS' failed
[ 96s] make[3]: *** [check-TESTS] Error 2
[ 96s] make[3]: Leaving directory '/home/abuild/rpmbuild/BUILD/appstream-glib-0.3.5/libappstream-builder'
[ 96s] Makefile:1457: recipe for target 'check-am' failed
[ 96s] make[2]: *** [check-am] Error 2
[ 96s] make[2]: Leaving directory '/home/abuild/rpmbuild/BUILD/appstream-glib-0.3.5/libappstream-builder'
[ 96s] Makefile:1146: recipe for target 'check-recursive' failed
[ 96s] make[1]: *** [check-recursive] Error 1
[ 96s] make[1]: Leaving directory '/home/abuild/rpmbuild/BUILD/appstream-glib-0.3.5/libappstream-builder'
[ 96s] Makefile:538: recipe for target 'check-recursive' failed
[ 96s] make: *** [check-recursive] Error 1
[ 96s] error: Bad exit status from /var/tmp/rpm-tmp.MLBRLV (%check)

I will try to change the build bot to spit out the log mentioned in the error

appstream-util validate-relax markes files invalid for style issues

Runnin appstream-util validate-relax marks all files with style issues as invalid (exit code 1)

the current generated ones on openSUSE (Tumbleweed >= 20150312) have, for example:

• style-invalid : xfce4-sensors.desktop:

cannot end in '.' [Show sensor values.]
• style-invalid : yakuake.desktop:

cannot end in '.' [A drop-down terminal emulator based on KDE Konsole technology.]
Validation of files failed

Would be great if the style things were treated as warnings/info only (not really invalidating the file)

appstream-builder could not gather information when multiple versions of package exists

It is been observed that when a directory (that appstream-builder scans) contains multiple versions of, say, firefox packages, appstream-builder will not be able to pull all metadata/appdata from such pacakages.

Here is the command I used.

appstream-builder --api-version=0.8 \
                  --add-cache-id \
                  --cache-dir=/repodata/cache \
                  --log-dir=/tmp/logs \
                  --packages-dir=/appstream-worker/test \
                  --temp-dir=/tmp \
                  --output-dir=/repodata \
                  --basename=appstream \
                  --origin=aosc \
                  --enable-hidpi \
                  --verbose \
                  --include-failed \
                  --max-threads=16

As when test contains multiple versions of the firefox packages, as shown

firefox_37.0-0_amd64.deb    firefox_39.0-0_amd64.deb    firefox_40.0.3-0_amd64.deb  firefox_41.0.1-0_amd64.deb  firefox_43.0-0_amd64.deb    firefox_43.0-1_amd64.deb
firefox_38.0-0_amd64.deb    firefox_39.0.3-0_amd64.deb  firefox_40.0.3-1_amd64.deb  firefox_41.0.2-0_amd64.deb  firefox_43.0.1-0_amd64.deb
firefox_38.0.5-0_amd64.deb  firefox_40.0.2-0_amd64.deb  firefox_41.0-0_amd64.deb    firefox_42.0-0_amd64.deb    firefox_43.0.1-1_amd64.deb

Running the command above shows.

(appstream-builder:13580): Asb-DEBUG: searching for plugins in /usr/lib/asb-plugins-4
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_nm.so: nm
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_dbus.so: dbus
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_kde_services.so: kde-services
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_absorb.so: absorb
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_gir.so: gir
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_gstreamer.so: gstreamer
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_font.so: font
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_appdata.so: appdata
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_gettext.so: gettext
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_hardcoded.so: hardcoded
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_desktop.so: desktop
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_gresource.so: gresource
(appstream-builder:13580): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_kde_notifyrc.so: kde-notifyrc
Scanning packages...
Parsed 1/16 files...
Parsed 3/16 files...
Parsed 5/16 files...
Parsed 7/16 files...
Parsed 9/16 files...
Parsed 11/16 files...
Parsed 13/16 files...
Parsed 15/16 files...
Processing packages...
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-43.0-(null) is not enabled
(appstream-builder:13580): Asb-DEBUG: starting: firefox
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-43.0-1 is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   Getting filename match for firefox_40.0.3-1_amd64.deb
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-43.0.1-1 is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-39.0.3-0 is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-37.0-0 is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-40.0.3-(null) is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-41.0.1-(null) is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-38.0.5-0 is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-41.0.2-(null) is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-42.0-(null) is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-39.0-0 is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-38.0-0 is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-43.0.1-(null) is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-41.0-(null) is not enabled
(appstream-builder:13580): Asb-DEBUG: DEBUG:   firefox-40.0.2-(null) is not enabled
Merging applications...
Writing /repodata/appstream.xml.gz...
Writing /repodata/appstream-failed.xml.gz...
Writing /repodata/appstream-ignore.xml.gz...
Writing /repodata/appstream-icons.tar.gz...
Writing /repodata/appstream-screenshots.tar...
Done!

And the resulting appstream.xml contains only

<?xml version="1.0" encoding="UTF-8"?>
<components version="0.8" builder_id="appstream-glib:4" origin="aosc"/>

While if I only keep the latest version of firefox,

firefox_43.0.1-1_amd64.deb

In the /appstream-worker/test directory, the same command shows the following output.

(appstream-builder:13827): Asb-DEBUG: searching for plugins in /usr/lib/asb-plugins-4
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_nm.so: nm
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_dbus.so: dbus
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_kde_services.so: kde-services
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_absorb.so: absorb
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_gir.so: gir
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_gstreamer.so: gstreamer
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_font.so: font
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_appdata.so: appdata
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_gettext.so: gettext
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_hardcoded.so: hardcoded
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_desktop.so: desktop
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_gresource.so: gresource
(appstream-builder:13827): Asb-DEBUG: opened plugin /usr/lib/asb-plugins-4/libasb_plugin_kde_notifyrc.so: kde-notifyrc
Scanning packages...
Processing packages...
(appstream-builder:13827): Asb-DEBUG: starting: firefox
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Getting filename match for firefox_43.0.1-1_amd64.deb
(appstream-builder:13827): Asb-DEBUG: decompressing files: firefox
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Exploding tree for firefox
(appstream-builder:13827): Asb-DEBUG: examining: firefox
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Processing firefox_43.0.1-1_amd64.deb with appdata
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process() from appdata
(appstream-builder:13827): Asb-DEBUG: processing: firefox
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process_app() from nm
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process_app() from kde-services
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process_app() from kde-notifyrc
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process_app() from hardcoded
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process_app() from gstreamer
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process_app() from gresource
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process_app() from gir
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process_app() from gettext
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process_app() from font
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process_app() from desktop
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Running asb_plugin_process_app() from dbus
(appstream-builder:13827): Asb-DEBUG: deleting temp files: firefox
(appstream-builder:13827): Asb-DEBUG: writing log: firefox
Merging applications...
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Saved icon /tmp/icons/64x64/firefox.png
(appstream-builder:13827): Asb-DEBUG: DEBUG:   Saved icon /tmp/icons/128x128/firefox.png
(appstream-builder:13827): Asb-DEBUG: <components version="1.0">
  <component type="desktop">
    <id>firefox.desktop</id>
    <pkgname>firefox</pkgname>
    <name>Firefox</name>
    <summary>A standalone Web browser from Mozilla.org</summary>
    <summary xml:lang="ar">تصفح الشبكة العنكبوتية العالمية</summary>
    <summary xml:lang="ast">Restola pela Rede</summary>
    <summary xml:lang="bn">ইন্টারনেট ব্রাউজ করুন</summary>
    <summary xml:lang="ca">Navegueu per la web</summary>
    <summary xml:lang="cs">Prohlížení stránek World Wide Webu</summary>
    <summary xml:lang="da">Surf på internettet</summary>
    <summary xml:lang="de">Im Internet surfen</summary>
    <summary xml:lang="el">Μπορείτε να περιηγηθείτε στο διαδίκτυο (Web)</summary>
    <summary xml:lang="es">Navegue por la web</summary>
    <summary xml:lang="et">Lehitse veebi</summary>
    <summary xml:lang="fa">صفحات شبکه جهانی اینترنت را مرور نمایید</summary>
    <summary xml:lang="fi">Selaa Internetin WWW-sivuja</summary>
    <summary xml:lang="fr">Naviguer sur le Web</summary>
    <summary xml:lang="gl">Navegar pola rede</summary>
    <summary xml:lang="he">גלישה ברחבי האינטרנט</summary>
    <summary xml:lang="hr">Pretražite web</summary>
    <summary xml:lang="hu">A világháló böngészése</summary>
    <summary xml:lang="it">Esplora il web</summary>
    <summary xml:lang="ja">ウェブを閲覧します</summary>
    <summary xml:lang="ko">웹을 돌아 다닙니다</summary>
    <summary xml:lang="ku">Li torê bigere</summary>
    <summary xml:lang="lt">Naršykite internete</summary>
    <summary xml:lang="nb">Surf på nettet</summary>
    <summary xml:lang="nl">Verken het internet</summary>
    <summary xml:lang="nn">Surf på nettet</summary>
    <summary xml:lang="no">Surf på nettet</summary>
    <summary xml:lang="pl">Przeglądanie stron WWW</summary>
    <summary xml:lang="pt">Navegue na Internet</summary>
    <summary xml:lang="pt_BR">Navegue na Internet</summary>
    <summary xml:lang="ro">Navigați pe Internet</summary>
    <summary xml:lang="ru">Доступ в Интернет</summary>
    <summary xml:lang="sk">Prehliadanie internetu</summary>
    <summary xml:lang="sl">Brskajte po spletu</summary>
    <summary xml:lang="sv">Surfa på webben</summary>
    <summary xml:lang="tr">İnternet'te Gezinin</summary>
    <summary xml:lang="ug">دۇنيادىكى توربەتلەرنى كۆرگىلى بولىدۇ</summary>
    <summary xml:lang="uk">Перегляд сторінок Інтернету</summary>
    <summary xml:lang="vi">Để duyệt các trang web</summary>
    <summary xml:lang="zh_CN">浏览互联网</summary>
    <summary xml:lang="zh_TW">瀏覽網際網路</summary>
    <description><p>Firefox is an open source Web browser developed by the Mozilla.org and supported by the community. Firefox is extensible with support for thousands of plugins and extensions.</p></description>
    <icon height="128" width="128" type="cached">128x128/firefox.png</icon>
    <icon height="64" width="64" type="cached">64x64/firefox.png</icon>
    <categories>
      <category>Network</category>
      <category>WebBrowser</category>
    </categories>
    <kudos>
      <kudo>HiDpiIcon</kudo>
    </kudos>
    <mimetypes>
      <mimetype>application/vnd.mozilla.xul+xml</mimetype>
      <mimetype>application/xhtml+xml</mimetype>
      <mimetype>text/html</mimetype>
      <mimetype>text/mml</mimetype>
      <mimetype>text/xml</mimetype>
      <mimetype>x-scheme-handler/ftp</mimetype>
      <mimetype>x-scheme-handler/http</mimetype>
      <mimetype>x-scheme-handler/https</mimetype>
    </mimetypes>
    <project_license>MPL-1.1 or GPL-2.0+ or LGPL-2.0+</project_license>
    <url type="bugtracker">https://bugzilla.mozilla.org/</url>
    <url type="help">https://support.mozilla.org/products/firefox</url>
    <url type="homepage">https://www.mozilla.org/firefox/</url>
    <screenshots>
      <screenshot type="default">
        <image type="source">https://repo.aosc.io/appdata/screenshots/Firefox-1.png</image>
      </screenshot>
      <screenshot>
        <image type="source">https://repo.aosc.io/appdata/screenshots/Firefox-2.png</image>
      </screenshot>
      <screenshot>
        <image type="source">https://repo.aosc.io/appdata/screenshots/Firefox-3.png</image>
      </screenshot>
      <screenshot>
        <image type="source">https://repo.aosc.io/appdata/screenshots/Firefox-4.png</image>
      </screenshot>
    </screenshots>
    <provides>
      <binary>firefox</binary>
    </provides>
    <metadata>
      <value key="X-CacheID">firefox_43.0.1-1_amd64.deb</value>
    </metadata>
  </component>
</components>

Writing /repodata/appstream.xml.gz...
Writing /repodata/appstream-failed.xml.gz...
Writing /repodata/appstream-ignore.xml.gz...
Writing /repodata/appstream-icons.tar.gz...
Writing /repodata/appstream-screenshots.tar...
Done!

And the resulting xml file contains valid information.

While I would need to point out that only the newest version of firefox package contains an appdata xml.

appstream-glib-0.3.3: test fails

We get:
bin/sh ../libtool --tag=CC --mode=link x86_64-pc-linux-gnu-gcc -Wall -Wcast-align -Wno-uninitialized -Wmissing-declarations -Wredundant-decls -Wpointer-arith -Wcast-align -Wwrite-strings -Winit-self -Wreturn-type -Wformat-nonliteral -Wformat-security -Wmissing-include-dirs -Wmissing-format-attribute -Wsign-compare -Wtype-limits -Wuninitialized -Waggregate-return -Wdeclaration-after-statement -Wshadow -Wno-strict-aliasing -Winline -O2 -pipe -march=native -fno-stack-protector -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -o asb-self-test asb_self_test-asb-self-test.o -lgthread-2.0 -pthread -lgio-2.0 -lgobject-2.0 -Wl,--export-dynamic -lgmodule-2.0 -pthread -lglib-2.0 -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lsoup-2.4 -lgio-2.0 -lgobject-2.0 -lglib-2.0 libappstream-builder.la
libtool: link: x86_64-pc-linux-gnu-gcc -Wall -Wcast-align -Wno-uninitialized -Wmissing-declarations -Wredundant-decls -Wpointer-arith -Wcast-align -Wwrite-strings -Winit-self -Wreturn-type -Wformat-nonliteral -Wformat-security -Wmissing-include-dirs -Wmissing-format-attribute -Wsign-compare -Wtype-limits -Wuninitialized -Waggregate-return -Wdeclaration-after-statement -Wshadow -Wno-strict-aliasing -Winline -O2 -pipe -march=native -fno-stack-protector -Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu -o .libs/asb-self-test asb_self_test-asb-self-test.o -pthread -Wl,--export-dynamic -pthread ./.libs/libappstream-builder.so /var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-glib/.libs/libappstream-glib.so -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lsoup-2.4 -lyaml -lgthread-2.0 -lgio-2.0 -lgobject-2.0 -lgmodule-2.0 -lglib-2.0 -larchive -pthread
make[3]: Leaving directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder'
make check-TESTS
make[3]: Entering directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder'
make[4]: Entering directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder'
../test-driver: line 95: 5047 Trace/breakpoint trap (core dumped) "$@" > $log_file 2>&1
FAIL: asb-self-test
make[5]: Entering directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder'
Making all in plugins
make[6]: Entering directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder/plugins'
make[6]: Leaving directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder/plugins'
make[6]: Entering directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder'
make[6]: Nothing to be done for 'all-am'.
make[6]: Leaving directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder'

make[5]: Leaving directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder'

Testsuite summary for appstream-glib 0.3.3

TOTAL: 1

PASS: 0

SKIP: 0

XFAIL: 0

FAIL: 1

XPASS: 0

ERROR: 0

See libappstream-builder/test-suite.log

Makefile:1210: recipe for target 'test-suite.log' failed
make[4]: *** [test-suite.log] Error 1
make[4]: Leaving directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder'
Makefile:1316: recipe for target 'check-TESTS' failed
make[3]: *** [check-TESTS] Error 2
make[3]: Leaving directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder'
Makefile:1414: recipe for target 'check-am' failed
make[2]: *** [check-am] Error 2
make[2]: Leaving directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder'
Makefile:1103: recipe for target 'check-recursive' failed
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory '/var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3/libappstream-builder'
Makefile:526: recipe for target 'check-recursive' failed
make: *** [check-recursive] Error 1

[RFE] add to status.html links to VCS (rpm tag)

This will allow us to easy clone repo and check when was last commit and check if this project is dead and print warning in console. Following python code says that rpm contains that tag

#!/usr/bin/env python
import rpm
import os


ts = rpm.TransactionSet()
fd = os.open("erlang-17.4-1.fc22.x86_64.rpm", os.O_RDONLY)
h = ts.hdrFromFdno(fd)
print(h[rpm.RPMTAG_VCS])
$ ./test.py 
scm:git:https://github.com/erlang/otp

Allow tagging screenshots by desktop environment

It might be nice to have a way to tag screenshots for a particular desktop environment. For example, I might take screenshots of an application in both GNOME and KDE and want to display the GNOME screenshots to GNOME users and the KDE screenshots to KDE users.

Or maybe this would not be nice, since it implies much more work creating the screenshots, and there's unlikely to be much consistency in which desktops the screenshots are taken for.

feature: create table to perform screenshot sync post appdata building

Due to the current way openSUSE interated appstream-builder directly into the repo publishing process (same step createrepo is being started), the bot is generally isolated from the network, and appstream-builder is being executed without network access (--no-net)

due to this, it creates an appdata.xml.gz without any screenshot tags (sad)

I'd like to see an option that appstream-builder produces an additional download-map:remote URL file (remote-URL based on the screenshot-url

so that this map file could be taken to the 'screenshot hosting server' and the screenshots be sync'ed there directly (with information of source and expected destination in the hosted tree)

The result should be:

  • appdata.xml.gz contains screenshot URLs linking to the 'openSUSE screenshot host'
  • a map file put next to the appdata.xml.gz, which we can move over to the openSUSE screenshot host' and sync the screenshots based on this (drop 'no longer listed ones, download new ones)

appstream-builder is already now started with:
--no-net
--screenshot-uri=http://static.opensuse.org/screenshots

so the URL to put into the .xml file should be available.

No idea how to handle *.drpm

(appstream-builder:3750): As-WARNING **: No idea how to handle 20/libmediainfo-0.7.63-1.fc20.R_0.7.64-1.fc20.R.x86_64.drpm

I think if we have non-*.rpm we should skip them

Stops processing any metadata if only one file with an error was found

Hi!
I have an empty (as in blank, no <components/> node) metadata file in /var/cache/app-info/xmls, which causes asglib to silently fail to load the file.
This results in no further metadata being loaded, including the perfectly fine data in /var/cache/app-info/yaml or potential other XML documents.
It would be great if the loader code would be a bit more graceful on errors and doesn't stop loading any data.

build error: undefined reference to `archive_write_new'

While building appstream-glib 0.2.1 for Mageia

Full build log:
http://pkgsubmit.mageia.org/uploads/failure/cauldron/core/release/20140624184739.ovitters.valstar.21759/log/appstream-glib-0.2.1-1.mga5/build.0.20140624184810.log

This is very likely due to --as-needed

libtool: link:  gcc -shared -Wl,--as-needed  -fPIC -DPIC  .libs/libappstream_builder_la-asb-app.o .libs/libappstream_builder_la-asb-context.o .libs/libappstream_builder_la-asb-package.o .libs/libappstream_builder_la-asb-package-deb.o .libs/libappstream_builder_la-asb-task.o .libs/libappstream_builder_la-asb-utils.o .libs/libappstream_builder_la-asb-plugin.o .libs/libappstream_builder_la-asb-plugin-loader.o   -O2 -march=i586 -mtune=generic -Wl,--as-needed -Wl,--no-undefined -Wl,-z -Wl,relro -Wl,-O1 -Wl,--build-id -Wl,--enable-new-dtags -pthread -Wl,--export-dynamic -pthread   -pthread  -Wl,-rpath -Wl,/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-glib/.libs ../libappstream-glib/.libs/libappstream-glib.so -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lsoup-2.4 -lgthread-2.0 -lgio-2.0 -lgobject-2.0 -lgmodule-2.0 -lglib-2.0 -Wl,-soname -Wl,libappstream-builder.so.1 -Wl,-version-script -Wl,.libs/libappstream-builder.ver -o .libs/libappstream-builder.so.1.5.3
.libs/libappstream_builder_la-asb-utils.o: In function `asb_utils_write_archive':
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:300: undefined reference to `archive_write_new'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:301: undefined reference to `archive_write_add_filter_gzip'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:302: undefined reference to `archive_write_set_format_pax_restricted'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:303: undefined reference to `archive_write_open_filename'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:319: undefined reference to `archive_write_data'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:320: undefined reference to `archive_entry_free'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:309: undefined reference to `archive_entry_new'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:311: undefined reference to `archive_entry_set_pathname'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:312: undefined reference to `archive_entry_set_size'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:313: undefined reference to `archive_entry_set_filetype'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:314: undefined reference to `archive_entry_set_perm'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:315: undefined reference to `archive_write_header'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:323: undefined reference to `archive_write_close'
/home/iurt/rpmbuild/BUILD/appstream-glib-0.2.1/libappstream-builder/asb-utils.c:324: undefined reference to `archive_write_free'
.libs/libappstream_builder_la-asb-utils.o: In function `asb_utils_explode_file':

appstream-util install(-origin) should only install valid appdata.xml

I just ran into the 'issue' that GNOME Software showed me weird descriptions of apps, in all kind of languages.

Turned out, one repository published invalid metadata (not generated using appstream-builder), thus spoiling the reputation of GNOME-Software as a reliable front end.

I since implemented in the libzypp plugin handlng appstream metadata, that only files passing appstream-util validate-relax are being considered to be installed. The others have sufficiently severe issues that they should be ignored (style issues have been removed from being fatal for validate-relax just recently, for the same use case)

it would be great if install(-origin) would handle that directly, instead of passing the same .xml file to two parsers in sequence.

Improve appdata-xml.m4 macro deprecation

Deprecating the old APPDATA_XML macro should come with a corresponding bump in the serial, and would be better with AU_DEFUN rather than AC_DEFUN (so that a standard deprecation notice is emitted).

Non-void function should return a value

In file libappstream-glib/as-profile.c, function as_profile_start_literal has a return value, so it cannot use g_return_if_fail.

as-profile.c:125:2: error: non-void function 'as_profile_start_literal' should return a value [-Wreturn-type]
as-profile.c:126:2: error: non-void function 'as_profile_start_literal' should return a value [-Wreturn-type]

This is easy to fix:

diff --git a/libappstream-glib/as-profile.c b/libappstream-glib/as-profile.c
index a872573..c0594b4 100644
--- a/libappstream-glib/as-profile.c
+++ b/libappstream-glib/as-profile.c
@@ -122,8 +122,8 @@ as_profile_start_literal (AsProfile *profile, const gchar *id)
        g_autofree gchar *id_thr = NULL;
        g_autoptr(GMutexLocker) locker = g_mutex_locker_new (&profile->mutex);

-       g_return_if_fail (AS_IS_PROFILE (profile));
-       g_return_if_fail (id != NULL);
+       g_return_val_if_fail (AS_IS_PROFILE (profile), NULL);
+       g_return_val_if_fail (id != NULL, NULL);

        /* only use the thread ID when not using the main thread */
        self = g_thread_self ();

appstream builder: Don't rewrite the screenshot urls with --nonet

Currently, appstream-builder always rewrites the screenshot URLs. This is nice if the distributor is prepared to host the screenshots, but makes it very difficult if they aren't.

If we had a way to directly use upstream screenshots, it would become much easier to ship appstream metadata in COPR repos.

Feature: add --batch (or similar) to appstream-builder

As we're using appstream-builder in an automated process, the current mode of 'scanning the packages' leaves a big mess in the log file (escape sequences).

It would be great to get a --batch or similar command line option, that suppresses all this and limits itself to the nescessary output, that makes sense in a log file to be read.

url-not-found for existing links

? url-not-found         : <screenshot> url 'https://wiki.gnome.org/Projects/Tracker?action=AttachFile&do=get&target=tracker-needle.png' not found
/builddir/build/BUILDROOT/tracker-1.1.1-1.fc22.x86_64/usr/share/appdata/tracker-preferences.appdata.xml: FAILED:
? url-not-found         : <screenshot> url 'https://wiki.gnome.org/Projects/Tracker?action=AttachFile&do=get&target=tracker-preferences.png' not found
Validation of files failed

appstream-util: deal with duplicate data

I seem to have an issue with the 'inhouse generator' (nor asb) putting apps duplictae into the appstream.xml.gz file...

certainly not nice, but it should also not really be an issue, should it?

If it were, I would expect appstream-util validate-relax to complain about more things than just:
• duplicate-data : pidgin.desktop: has duplicated data
• duplicate-data : pitivi.desktop: has duplicated data
• duplicate-data : rhythmbox.desktop: has duplicated data
• duplicate-data : shotwell.desktop: has duplicated data
• duplicate-data : transmageddon.desktop: has duplicated data
• duplicate-data : zim.desktop: has duplicated data

Yet for the screenshots it does complain, which renders the entire validation process 'failed'.
Is this intentional?

Unclear meaning of "Not enough <p> content before <ul>"

appstream-util validate was showing this after adding second/third p section and sentence. The real error was that there were not enough characters. It would help a lot in fixing such issues if it states clearly that there are not enough characters preferably even with the number of missing characters.

touch -d "YYYY-MM-DD" is not portable

The command touch -c -m -d"2000-01-01" causes illegal time specification error on some systems.

The POSIX standard says the format of the date should be YYYY-MM-DDThh:mm:SS[.frac][tz] or YYYY-MM-DDThh:mm:SS[,frac][tz]. Changing "2000-01-01" to "2000-01-01T00:00:00" should fix the problem.

Asb:ERROR: asb_test_firmware_func: assertion failed (error == NULL)

Cloning the code from master and running

$ ./autogen.sh && make -k check

Shows that error.

Here is the log file content. If there are more useful information that I omitted, just let me now.

===============================================================
   appstream-glib 0.4.1: libappstream-builder/test-suite.log
===============================================================

# TOTAL: 1
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: asb-self-test
===================

/AppStreamBuilder/package: OK
/AppStreamBuilder/utils{replace}: OK
/AppStreamBuilder/utils{glob}: OK
/AppStreamBuilder/plugin-loader: OK
/AppStreamBuilder/firmware: Processing packages...
Merging applications...
Writing /tmp/asbuilder/output/appstream.xml.gz...
Writing /tmp/asbuilder/output/appstream-icons.tar.gz...
Writing /tmp/asbuilder/output/appstream-screenshots.tar...
**
Asb:ERROR:asb-self-test.c:945:asb_test_firmware_func: assertion failed (error == NULL): --- /tmp/b  2015-06-01 00:16:47.066893602 -0300
+++ /tmp/a  2015-06-01 00:16:47.066893602 -0300
@@ -13,17 +13,17 @@
 <releases>
 <release version="2.0.2" timestamp="1424116753">
 <location>http://www.hughski.com/downloads/colorhug2/firmware/colorhug-2.0.2.cab</location>
-<checksum type="sha1">ab2d0294e0d23c1718af79c6a17216c8322ae6da</checksum>
+<checksum type="sha1">9de398ebe3bbf477ad1fa344955dcba07e535dc0</checksum>
 <description><p>This unstable release adds the following features:</p><ul><li>Add TakeReadingArray to enable panel latency measurements</li><li>Speed up the auto-scaled measurements considerably, using 256ms as the smallest sample duration</li></ul></description>
 </release>
 <release version="2.0.1" timestamp="1424116753">
 <location>http://www.hughski.com/downloads/colorhug2/firmware/colorhug-2.0.1.cab</location>
-<checksum type="sha1">cc07f12dfea7103774decbcf48c47cc0604fbc4e</checksum>
+<checksum type="sha1">6a8e77f590fe8e0de85dc5c85d17819b903e0b87</checksum>
 <description><p>This unstable release adds the following features:</p><ul><li>Use TakeReadings() to do a quick non-adaptive measurement</li><li>Scale XYZ measurement with a constant factor to make the CCMX more sane</li></ul></description>
 </release>
 <release version="2.0.0" timestamp="1425168000">
 <location>http://www.hughski.com/downloads/colorhug2/firmware/colorhug-2.0.0.cab</location>
-<checksum type="sha1">81431b6d3c9f90c3137a8a132d4fb5e3681e6791</checksum>
+<checksum type="sha1">796c32daaaba5e91a3afce6afa8e31b7996bdb59</checksum>
 </release>
 </releases>
 </component>
 (-g-type-private--GTypeFlags, 0)
FAIL asb-self-test (exit status: 134)

appstream-glib-0.3.3 needs gtk-doc to build

We get this error when running autoreconf:
***** automake *****
***** PWD: /var/tmp/portage/dev-libs/appstream-glib-0.3.3/work/appstream-glib-appstream_glib_0_3_3
***** automake --add-missing --copy --foreign --force-missing

configure.ac:65: warning: The 'AM_PROG_MKDIR_P' macro is deprecated, and its use is discouraged.
configure.ac:65: You should use the Autoconf-provided 'AC_PROG_MKDIR_P' macro instead,
configure.ac:65: and use '$(MKDIR_P)' instead of '$(mkdir_p)'in your Makefile.am files.
configure.ac:59: installing './compile'
configure.ac:11: installing './missing'
client/Makefile.am: installing './depcomp'
automake-1.13: error: cannot open < gtk-doc.make: No such file or directory

Because gtk-doc.make is not supplied... this is not common as usually projects release tarballs with that files (or without needing to run autoreconf)

Thanks a lot

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.