earnestly / webfeeds Goto Github PK

webfeeds <https://github.com/Earnestly/webfeeds>

INTRODUCTION

    webfeeds is a tolerable aggregator of web feed links and titles.

    It passes the responsibility of reading the links to web browsers which are
    better suited to handle such content.  As a result it need not store
    anything more than the timestamp, URL and title for each entry.

WHY

    webfeeds came to exist when newsbueter (now newsboat) was suffering from
    tremendous memory leaks resulting in 1G of RES for roughly fifty feeds.

    It was a temporary measure that I wrote a script to download the XML
    directly, using XSLT to transform the input into the form I wanted.
    After several years I had noticed little need for more and went about
    slowly shaping webfeeds into what it is now.

    Performance improvements were gained from curl adding support for parallel
    downloads and due to the simple text database I have ended up with roughly
    3M of storage use covering roughly six years from sixty web feeds.

HOW

    The parse.xslt XSLT transforms discrete web feeds into a normalised XML
    format comprising a percent-encoded filepath in the form of domain/title
    and its associated entries containing the dates, links and titles.

        <c><f><p>percent-encoded filepath</p><e><i>
        <d>date</d>
        <l>link</l><t>title</t></i>...</e></f><f>...</f></c>

    N.B. Each date element is arranged on its own line so that dateconv matches
         them exclusively.

    The format is then used by write.xslt via <xsl:document> to generate file
    outputs in the desired format.

REQUIRES

    * curl[>=7.83.0]
    * xsltproc <http://xmlsoft.org/XSLT>
    * dateutils <https://github.com/hroptatyr/dateutils>
    * gzip

BUILD

    make

INSTALL

    make PREFIX=/usr install