Coder Social home page Coder Social logo

greg's Introduction

greg

A command-line podcast aggregator, written in python. It basically exposes some of the functionality of the excellent feedparser.

Installation

Use pip:

pip install --user greg

If you run Arch Linux, there is also an AUR package.

Configuration

To edit the configuration for greg, copy the system-wide greg.conf file to your local config folder:

mkdir -p ~/.config/greg && cp `greg retrieveglobalconf` ~/.config/greg/greg.conf

Then open and edit ~/.config/greg/greg.conf in a text editor. The configuration file is self-explanatory.

Usage

Let's start by adding a feed (RSS or Atom versions will do):

greg add PhilosophyBites http://philosophybites.com/atom.xml

The add command expects a name and a url of an RSS or Atom feed. You will use this name to refer to the feed whenever you interact with it.

If you were to run greg sync now, it would download the latest episode of the podcast to the default directory (which is ~/Podcasts; you can change how many episodes are dowloaded in the first sync, and the download directory, in the config file; see below). But maybe we just want to check out what this podcast is all about, so we download a list of available entries:

greg check -f PhilosophyBites

(the -f flag means that "PhilosophyBites" is the name of a feed. greg check also accepts urls directly, using the -u flag.)

This will give you the following kind of info:

0: Tom Sorell on Surveillance (2013-01-25T13:43:46+00:00)
1: John Campbell on Schizophrenia (2013-01-08T12:41:27+00:00)
2: Kendall Walton on Photography (2012-12-23T12:33:09+00:00)
3: Twitter Competition: Who's Your Favourite Philosopher? (2012-12-11T07:24:51+00:00)
4: Alan Ryan on Freedom and Its History (2012-12-08T11:16:45+00:00)
5: Nigel Warburton at Blackwell's Bookshop, Oxford 7pm Wed. Dec. 5th (2012-12-01T11:19:09+00:00)
6: Who's Your Favourite Philosopher? (2012-11-30T18:33:56+00:00)
7: Peter Adamson on Avicenna's Flying Man Thought Experiment (2012-11-26T15:57:18+00:00)
8: Links to Past Episodes (2012-12-01T11:53:03+00:00)
9: Tim Bayne on the Unity of  Consciousness (2012-11-11T22:20:17+00:00)
10: Galen Strawson on the Sense of Self (2012-05-05T12:56:05+01:00)
11: Liane Young on Mind and Morality (2012-10-27T12:39:22+01:00)
12: Gary L. Francione on Animal Abolitionism (2012-10-13T13:48:32+01:00)
13: Richard Sorabji on Mahatma Gandhi as Philosopher (2012-09-28T13:18:08+01:00)
14: Tim Crane on Non-Existence (2012-09-15T18:50:32+01:00)
15: Michael Tye on Pain (2012-08-31T20:51:01+01:00)
16: Daniel Dennett on Free Will Worth Wanting (2012-08-18T08:58:24+01:00)
17: Pat Churchland on What Neuroscience Can Teach Us About Morality (2012-08-03T22:52:12+01:00)
18: Rae Langton on Hate Speech (2012-07-28T20:14:27+01:00)
19: Molly Crockett on Brain Chemistry and Moral-Decision Making (originally on Bioethics Bites) (2012-07-22T21:14:35+01:00)

Interesting stuff. We'll download a couple of episodes, just to make sure that it's really worth it:

greg download 1, 5-7

and Greg says

Downloading John Campbell on Schizophrenia -- John_Campbell_on_Schizophrenia.mp3
Done
Downloading John Campbell on Schizophrenia -- John_Campbell_on_Berkeleys_Puzzle_1.mp3
Done
Downloading Who's Your Favourite Philosopher? -- Whos_Your_Favourite_Philosopher_.mp3
Done
Downloading Peter Adamson on Avicenna's Flying Man Thought Experiment -- Peter_Adamson_on_Avicennas_Flying_Man.mp3
Done
Downloading Peter Adamson on Avicenna's Flying Man Thought Experiment -- AdamsonMixSes.MP3
Done
Downloading Peter Adamson on Avicenna's Flying Man Thought Experiment -- Peter_Adamson_on_Plotinus_on_Evil.mp3
Done

As you can see, greg download accepts a range of episodes of the kind a, b, c-f, h, .... The numbers make reference to the numbers at the beginning of each entry provided by greg check. check creates a persistent file (feeddump in the data directory, ~/.local/share/greg/data by default, but you can change that in the config file, or passing a different path with the --datadirectory flag), so download will keep on working, and referring to the last check ever done.

All of these podcasts will be downloaded to the default download directory for the feed (if you used the -f flag) or the general default download directory (again, ~/Podcasts if you don't tell Greg otherwise. We'll learn how to change that soon), inside a subdirectory named after the podcast (we can change that default too.) After listening to them we decide that this podcast is well worth our time, and keep it, or we decide that it's not, and

greg remove PhilosophyBites

If we keep it, we might want to start syncing from, say, the 30th of April, 2013, on. So we edit the feed information

greg edit PhilosophyBites -d 2013-4-30

We may also use the now keyword to instruct greg to start syncing from now on:

greg edit PhilosophyBites -d now

-d or --downloadfrom change the date after which Greg should start downloading episodes when it syncs. Currently, the only two things one can edit in a feed are the download-from date and --url -- but many more things can be changed by editing the config file. greg edit -h will give help you with the edit options and syntax -- likewise for the rest of Greg subcommands.

All right. Let's add a second feed:

greg add History http://podcast.ulcc.ac.uk/accounts/kings/Philosophy_podcasts.xml

If you want to keep track of the feeds you have added, you can ask Greg:

greg info

which returns

PhilosophyBites
---------------
    url: http://philosophybites.com/atom.xml
    Next sync will download from: 30 Apr 2013 00:00:00.

History
-------
    url: http://podcast.ulcc.ac.uk/accounts/kings/Philosophy_podcasts.xml

Let us add another feed:

greg add MusicaAntigua http://www.rtve.es/api/programas/23353/audios.rss

This is a great program on ancient music at the Spanish public radio. The thing is, these guys do not tag their episodes, which is bad for most portable media players. Greg uses EyeD3 (as an optional dependency) to tag podcasts, if one so wishes. By default, it uses the podcast name for the artist tag, and the entry title for the title tag. To enable tagging for MusicaAntigua, copy the system-wide config file locally. (see Configuration above)

Then, add a section for MusicaAntigua:

[MusicaAntigua]

Tag = yes

In fact, you can fill out any tag however you see fit. For example,

tag_genre = Ancient Music
tag_comment = {date}

will fill the genre tag with the string "Ancient Music", and the comment tag with the download date.

Let's add a video podcast

greg add TEDTalks http://feeds.feedburner.com/TEDTalks_video

By default, Greg only donwloads audio files (in fact, files that have "audio" as part of their type). In order to download the right file in TEDTalks, then, you need to change that in the config file. Again, add a section:

[TEDTalks]

mime = video

You could also have a couple of types there, as in mime = audio, video; or any other type, mime = torrent, or whatever.

Another useful thing that you can change in the config file is the download handler; Greg by default uses requests, but you can use whatever you want.

I, for example, have

downloadhandler = wget {link} -P {directory}

in my local greg.conf. You can do all sorts of nice things with this. For example, when checking a podcast, you don't need to download it, but maybe just stream it, like this:

greg download 0 --downloadhandler "mplayer {link}"

If you want to ensure that the downloaded files are in chronological order, you can use placeholders to add the date at the beginning, like this:

downloadhandler = wget {link} -O {directory}/{date}_{filename}

One last thing: if you subscribe to a very active feed, and you are only interested in some of the entries, you can filter the feed. For example, if you only want to watch TED talks about Google, say, you can add the following line to the [TEDTalks] section:

filter = "Google" in "{title}"

(You need the quotes around {title} if the string you are filtering by has spaces, for example; they are strictly unnecessary here.)

For information about the {placeholders}, take a look at greg.conf. In greg.conf you can also change the download directory, and some other things. It should be self-explanatory.

greg's People

Contributors

barnabywalters avatar bdausses avatar jamesscottbrown avatar manolomartinez avatar migueldvb avatar narrat avatar pepie34 avatar rolfmorel avatar sabberworm avatar timwhite avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

greg's Issues

Existing podcasts

Maybe it could be checked before the download starts, if a podcast-file is already existing in the download folder?

New structure -- review requested

Hello, @nick, @Narrat, @FilipBB: I've pushed a new_structure branch with what I think is a saner structure for the repo. I was wondering if you could take a look, let me know whether you think it makes sense, or some other thing would make more sense.

Thanks!
M

Error "split() takes no keyword arguments" on new install

I have a new install of greg and am seeing the following messages when trying to do a download:

mal@stn6714:~$ greg download 1
Traceback (most recent call last):
  File "/home/mal/.local/bin/greg", line 9, in <module>
    load_entry_point('Greg==0.4.6', 'console_scripts', 'greg')()
  File "/home/mal/.local/lib/python3.2/site-packages/greg/gregparser.py", line 118, in main
    function(vars(args))
  File "/home/mal/.local/lib/python3.2/site-packages/greg/greg.py", line 825, in download
    issues = parse_for_download(args)
  File "/home/mal/.local/lib/python3.2/site-packages/greg/greg.py", line 497, in parse_for_download
    for group in single_arg.split(sep=","):
TypeError: split() takes no keyword arguments

The system is running on is Debian 7.11
Python is 3.2.3

Do you have any tips on what I might look at to identify and correct the issue?

Mal.

Tags are not applied to the downloaded file

Let's say I already have a podcast "A.mp3"
I download it again through greg, which will names the new file "A.mp3_". At this point everything is fine.
But if tags are activated, then it is the "A.mp3" which is altered and not the "A.mp3_" downloaded file.

For example, i had that file (0 size):

-rw-r--r-- 1 root   root    0 2015-10-20 17:28 229214750-podcastscience-233-la-these-de-micmaths.mp3

After redownloading it through greg, i got :

-rw-r--r-- 1 root   root    92M 2016-09-18 09:58 229214750-podcastscience-233-la-these-de-micmaths.mp3_
-rw-r--r-- 1 root   root    176 2016-09-18 09:58 229214750-podcastscience-233-la-these-de-micmaths.mp3

The 176 bytes are just the ID3 tags written by greg in the first "0 size" file. It is the newly downloaded file (with the _ suffix) that should have received the new tags.

Feature request: play command

Hi,
Some times I really just want to listen to the newest podcast without too much fuzz. For instance it would be great just to be able to say greg play PhilosophyBites latest or greg play PhilosophyBites 0 or perhaps I want to play the episode 8 conditional on the list of episodes (greg check).

This would give Greg more a feel of an on-demand radio, which I at least often need.

Customization could be player program (defaulting to mplayer of ffplay or something else) and temporary download directory (probably /tmp).

Crash bug when quotes are in episode name:

greg sync tnyrh
Checking tnyrh...
Downloading Episode 13: El Chapo v. Flores Brothers, and Jack Handeys Santa Fe -- tnyradiohour011516_cms566193_pod.mp3
Traceback (most recent call last):
  File "/home/mjf/.local/bin/greg", line 9, in <module>
    load_entry_point('Greg==0.4.5', 'console_scripts', 'greg')()
  File "/home/mjf/.local/lib/python3.4/site-packages/greg/gregparser.py", line 113, in main
    args.func(vars(args))
  File "/home/mjf/.local/lib/python3.4/site-packages/greg/greg.py", line 764, in sync
    downloaded = feed.download_entry(entry)
  File "/home/mjf/.local/lib/python3.4/site-packages/greg/greg.py", line 325, in download_entry
    condition = filtercond(placeholders)
  File "/home/mjf/.local/lib/python3.4/site-packages/greg/greg.py", line 529, in filtercond
    return eval(condition)
  File "<string>", line 1
    "Episode" in "Episode 12: Sarah Koenig on "Serial," and a Resilient Poet"
                                                    ^
SyntaxError: invalid syntax

Seems like quote character needs to escaped?

Previously working config fails with file not found

Hi there,

since my last update of greg a few days ago my previously working configuration fails. Specifically, on sync greg prints this message:

% greg sync Jurafunk
Checking Jurafunk...
Downloading Jurafunk Nr. 115: Kachelmann vs Schwarzer, Impressum bei Xing, Speicherung IP-Adressen und Hasseröder Männer-Camp -- jurafunk20140728.mp3
'/srv/cifs/multimedia/Podcasts/Jurafunk/Jurafunk Nr. 115: Kachelmann vs Schwarzer, Impressum bei Xing, Speicherung IP-Adressen und Hasseröder Männer-Camp: Datei oder Verzeichnis nicht gefunden
Done

"Datei oder Verzeichnis nicht gefunden" is German for "file or directory not found".

Checking the directory /srv/cifs/multimedia/Podcasts/Jurafunk I found the file is not there. Nevertheless greg marked it as downloaded so it never retries the failed download; each time I manually delete the faulty entry from ~/.local/share/greg/data/Jurafunk the syncing operation fails again.

The relevant part of my ~/.config/greg/greg.conf looks like this:

[DEFAULT] 

# Double % needed, bug in Greg?
date_format = %%Y

Data directory = ~/.local/share/greg/data
Download directory = /srv/cifs/multimedia/Podcasts/
Create subdirectory = yes
subdirectory_name = {podcasttitle}
Tag = no
tag_artist = {podcasttitle}
tag_title = {title}
tag_genre = Podcast
firstsync = 2
mime = audio
downloadhandler = wget {link} -P '{directory}'

[Jurafunk]

downloadhandler = wget '{link}' -q -O '{directory}/{title} ({date}).mp3'

Greg version is 0.4.2.

Vale,
Quintus

NoneType with a feed

Hey,

first of all thanks for the great tool.

With this feed:
[Latzel]
url = http://www.st-martini.net/rss/podcast/stm-predigten.xml
date_info = available

download with "greg download 1-315" works fine. However when trying "greg sync Latzel" greg returns:

Checking Latzel...
Traceback (most recent call last):
File "/sw/bin/greg", line 9, in
load_entry_point('Greg==0.4.6', 'console_scripts', 'greg')()
File "/sw/lib/python3.5/site-packages/greg/gregparser.py", line 118, in main
function(vars(args))
File "/sw/lib/python3.5/site-packages/greg/greg.py", line 768, in sync
feed.fix_linkdate(entry)
File "/sw/lib/python3.5/site-packages/greg/greg.py", line 261, in fix_linkdate
entry.linkdate = list(entry.published_parsed)
TypeError: 'NoneType' object is not iterable

I use a Mac with python-pip 8.10 .

Any ideas?

Thanks in advance
Martin

greg.conf.txt

Tagging doesn't seem to work when there is no specific section for a podcast

Hi,

I activated tagging in the config file (DEFAULT section, Tag = yes) but after downloading a podcast called "test", I get the following stacktrace :

[edouard@pioneer ~]$ greg sync
Checking test...
Downloading L'intégrale des Grosses têtes du 18 novembre -- 7780557844_l-integrale-des-grosses-tetes-du-18-novembre.mp3
Traceback (most recent call last):
  File "/usr/local/bin/greg", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.5/site-packages/greg/gregparser.py", line 112, in main
    args.func(vars(args))
  File "/usr/local/lib/python3.5/site-packages/greg/greg.py", line 760, in sync
    feed.download_entry(entry)
  File "/usr/local/lib/python3.5/site-packages/greg/greg.py", line 330, in download_entry
    tag(placeholders)
  File "/usr/local/lib/python3.5/site-packages/greg/greg.py", line 506, in tag
    feedoptions = placeholders.feed.config.options(placeholders.name)
  File "/usr/local/Cellar/python3/3.5.0/Frameworks/Python.framework/Versions/3.5/lib/python3.5/configparser.py", line 675, in options
    raise NoSectionError(section) from None
configparser.NoSectionError: No section: 'test'

I was hoping that my default tagging options would be used for every podcast without having to add a specific section, should that be the case ?

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 33: ordinal not in range(128)

[jdb@alarmpi ~]$ greg add TEDTalks http://feeds.feedburner.com/TEDTalks_audio
Traceback (most recent call last):
  File "/home/jdb/.local/bin/greg", line 11, in <module>
    sys.exit(main())
  File "/home/jdb/.local/lib/python3.5/site-packages/greg/parser.py", line 138, in main
    function(vars(args))
  File "/home/jdb/.local/lib/python3.5/site-packages/greg/commands.py", line 39, in add
    session = c.Session(args)
  File "/home/jdb/.local/lib/python3.5/site-packages/greg/classes.py", line 45, in __init__
    self.data_dir = self.retrieve_data_directory()
  File "/home/jdb/.local/lib/python3.5/site-packages/greg/classes.py", line 85, in retrieve_data_directory
    config.read([config_filename_global, self.config_filename_user])
  File "/usr/lib/python3.5/configparser.py", line 696, in read
    self._read(fp, filename)
  File "/usr/lib/python3.5/configparser.py", line 1012, in _read
    for lineno, line in enumerate(fp, start=1):
  File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 33: ordinal not in range(128)
[jdb@alarmpi ~]$ 

ConnectionResetError: [Errno 104] Connection reset by peer

Hello,

Quite often while downloading an episode i encounter the following error

Checking lineardigressions... 
Downloading Kalman Runners -- 230576907-linear-digressions-kalman-runners.mp3 
Traceback (most recent call last): 
 File "/home/username/.local/bin/greg", line 11, in <module> 
   sys.exit(main()) 
 File "/home/username/.local/lib/python3.5/site-packages/greg/parser.py", line 138, in main 
   function(vars(args)) 
 File "/home/username/.local/lib/python3.5/site-packages/greg/commands.py", line 176, in sync 
   downloaded = feed.download_entry(entry) 
 File "/home/username/.local/lib/python3.5/site-packages/greg/classes.py", line 329, in download_entry 
   aux.download_handler(self, placeholders) 
 File "/home/username/.local/lib/python3.5/site-packages/greg/aux_functions.py", line 223, in download_handler 
   urlretrieve(placeholders.link, placeholders.fullpath) 
 File "/usr/lib/python3.5/urllib/request.py", line 217, in urlretrieve 
   block = fp.read(bs) 
 File "/usr/lib/python3.5/http/client.py", line 448, in read 
   n = self.readinto(b) 
 File "/usr/lib/python3.5/http/client.py", line 488, in readinto 
   n = self.fp.readinto(b) 
 File "/usr/lib/python3.5/socket.py", line 575, in readinto 
   return self._sock.recv_into(b) 
ConnectionResetError: [Errno 104] Connection reset by peer

Is this an error with greg or the podcast website?

Configuration of downloaded files names for each podcast

I may be wrong, but i really don't see/understand how to rename downloaded files with greg ?
I would need that feature for some podcasts and not all of them.

Indeed, i have some podcasts where i want to keep the (clean) original filename, but i have a very few ones with quite messy non-standardized filenames. For them i would wish to rename them let's say with just the date.

So how to configure the podcast sections in greg.conf to define the mask of the downloaded files for a given podcast ?

YouTube downloader

Hey, @FilipBB, can you help me understand why something like this is needed? What it brings that greg currently cannot do?

I was thinking that perhaps with a bit extra work I could make something more general than a mere add-on for YT. Once I understand what it is doing.

Not downloading all episodes from feed (revolutionspodcast.com)

Just trying out greg for the first time here, this url: http://www.revolutionspodcast.com/rss.xml
greg only gets downloads "4.10- The Third Commission", but 10 episodes are listed with greg check.
Also, greg download doesn't manage to get them either. (by the way, how does greg download know which feed to select from?)

Post-download action?

This is probably possible by playing with the download handler, but it would be useful to have a post-download action for files. I can imagine all sorts of uses for this, but my planned use was so that new episodes could automatically be added to the playlist of an external player.

running 'greg' alone crashes:

~$ greg
Traceback (most recent call last):
  File "/home/mjf/.local/bin/greg", line 9, in <module>
    load_entry_point('Greg==0.4.5', 'console_scripts', 'greg')()
  File "/home/mjf/.local/lib/python3.4/site-packages/greg/gregparser.py", line 113, in main
    args.func(vars(args))
AttributeError: 'Namespace' object has no attribute 'func'

Problems with installation

Hi,
I'm on Debian and Python 2.7. After doing sudo python setup.py install and trying to run greg, I got this error:

File "/usr/local/bin/greg", line 19, in <module>
from greg.gregparser import main 
ImportError: No module named greg.gregparser

I tried running greg.py from the /greg/greg folder, but that gave another error:

File "greg.py", line 185
    print (feed, end=" ")
                    ^
SyntaxError: invalid syntax

Also, I had to add # coding=utf-8 to both setup.py and then to /usr/local/bin/greg. Without this, I got the following error:

SyntaxError: Non-ASCII character '\xc3' in file setup.py on line 8, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details

Here's my entire log:

mart@mart:~/downloads/greg$ sudo python setup.py install
running install
running build
running build_py
package init file 'greg/__init__.py' not found (or not a regular file)
creating build
creating build/lib.linux-i686-2.7
creating build/lib.linux-i686-2.7/greg
copying greg/gregparser.py -> build/lib.linux-i686-2.7/greg
copying greg/greg.py -> build/lib.linux-i686-2.7/greg
package init file 'greg/__init__.py' not found (or not a regular file)
running build_scripts
creating build/scripts-2.7
copying and adjusting bin/greg -> build/scripts-2.7
changing mode of build/scripts-2.7/greg from 644 to 755
running install_lib
creating /usr/local/lib/python2.7/dist-packages/greg
copying build/lib.linux-i686-2.7/greg/gregparser.py -> /usr/local/lib/python2.7/dist-packages/greg
copying build/lib.linux-i686-2.7/greg/greg.py -> /usr/local/lib/python2.7/dist-packages/greg
byte-compiling /usr/local/lib/python2.7/dist-packages/greg/gregparser.py to gregparser.pyc
byte-compiling /usr/local/lib/python2.7/dist-packages/greg/greg.py to greg.pyc
SyntaxError: ('invalid syntax', ('/usr/local/lib/python2.7/dist-packages/greg/greg.py', 183, 25, '        print (feed, end=" ")\n'))

running install_scripts
copying build/scripts-2.7/greg -> /usr/local/bin
changing mode of /usr/local/bin/greg to 755
running install_data
copying data/greg.conf -> /etc
running install_egg_info
Writing /usr/local/lib/python2.7/dist-packages/Greg-0.1.0.egg-info

mart@mart:~/downloads/greg$ cd ~
mart@mart:~$ greg
Traceback (most recent call last):
  File "/usr/local/bin/greg", line 19, in <module>
    from greg.gregparser import main 
ImportError: No module named greg.gregparser

Thanks for any help!

Based on the readme, it appears to be a perfect tool for me, though. So another thanks for your effort! :)

podcast url causes KeyError: 'type'

Sorry for posting so many issues, 😄

I just added this podcast URL: http://www.hellointernet.fm/podcast?format=rss

On greg sync I see the following error:

Checking Hello Internet...
Traceback (most recent call last):
  File "/usr/local/bin/greg", line 11, in <module>
    sys.exit(main())
 File "/usr/local/lib/python3.4/dist-packages/greg/gregparser.py", line 112, in main
    args.func(vars(args))
  File "/usr/local/lib/python3.4/dist-packages/greg/greg.py", line 793, in sync
    download_entry(feed, entry)
  File "/usr/local/lib/python3.4/dist-packages/greg/greg.py", line 557, in download_entry
    if any([mimetype in enclosure["type"] for mimetype in feed.mime]):
  File "/usr/local/lib/python3.4/dist-packages/greg/greg.py", line 557, in <listcomp>
    if any([mimetype in enclosure["type"] for mimetype in feed.mime]):
  File "/usr/local/lib/python3.4/dist-packages/feedparser.py", line 357, in __getitem__
    return dict.__getitem__(self, key)
KeyError: 'type'

I think maybe is the question mark in the podcast url causing an issue? Or maybe it's a malformed feed... I'll check it inside a feed validator.

Import from/Export to opml

Since this format is largely used to exchange lists of feeds, a command line option in greg would be useful to migrate from/to it.

First sync zero

It seem that "firstsync = 0" doesn't work, it would be nice to have the first sync download nothing.

As I am up to date with my podcasts I don't see the need to download them once more.

Thanks for this nice piece of software.

Plugin system

I suggest in this comment, you may bee add plugin system.

You seem to be interested in it.

I create this issue just for note forget.

Allow installation as user

Installing as a user is not possible at the moment because /etc/greg.conf is required and the setup.py tries to install it at this fixed place. Please consider a plain user install (pip install --user ...)

TypeError: a bytes-like object is required, not 'str'

(I will be very embarrassed if I've reported two non-issue issues in two days, but still...)

Greg was working fine for me yesterday. However now I'm suddenly getting the following when running 'greg sync':

Traceback (most recent call last):
File "/usr/bin/greg", line 9, in
load_entry_point('Greg==0.4.6', 'console_scripts', 'greg')()
File "/usr/lib/python3.5/site-packages/greg/gregparser.py", line 118, in main
function(vars(args))
File "/usr/lib/python3.5/site-packages/greg/greg.py", line 757, in sync
feed = Feed(session, target, None)
File "/usr/lib/python3.5/site-packages/greg/greg.py", line 114, in init
self.podcast = parse_podcast(session.feeds[feed]["url"])
File "/usr/lib/python3.5/site-packages/greg/greg.py", line 433, in parse_podcast
podcast = feedparser.parse(url)
File "/usr/lib/python3.5/site-packages/feedparser.py", line 3957, in parse
saxparser.parse(source)
File "/usr/lib/python3.5/site-packages/drv_libxml2.py", line 190, in parse
_d(reader.LocalName()))
File "/usr/lib/python3.5/site-packages/drv_libxml2.py", line 70, in _d
return _decoder(s)[0]
File "/usr/lib/python3.5/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
TypeError: a bytes-like object is required, not 'str'

Just to check, I created a fresh set of configuration files and added a single previously-working podcast (http://theblacktapes.libsyn.com/rss), but get the same error. This is under Arch Linux, and I may have done an update yesterday, but don't remember specifically which python packages (if any) might have been updated.

Can't sync with SpyCast

I've encountered an issue with the "SpyCast" feed (url: http://feeds.spymuseum.org/spycast/ ) that prevents greg from 'sync'ing. It throws this error:

Checking SpyCast...
Traceback (most recent call last):
File "/usr/local/bin/greg", line 11, in
sys.exit(main())
File "/usr/local/lib/python3.4/dist-packages/greg/gregparser.py", line 112, in main
args.func(vars(args))
File "/usr/local/lib/python3.4/dist-packages/greg/greg.py", line 752, in sync
feed.fix_linkdate(entry)
File "/usr/local/lib/python3.4/dist-packages/greg/greg.py", line 252, in fix_linkdate
entry.linkdate = list(entry.published_parsed)
TypeError: 'NoneType' object is not iterable

I have been able to download individual episodes from this podcast successfully with greg. I also have a -d date set on it, though I have the same date set on several other feeds and they don't exhibit this problem. It looks like greg is having trouble parsing a date -- is this yet-another-quirky-feed?

Support quiet mode

Hi there,

I just thought it might be useful to implement a -q or --quiet switch that turns off all output except for errors. I intend to run greg from cron, so while I could just redirect the standard output to /dev/null I think a specific switch for this seems more "clean". Most other downloading programs like wget (-q) or curl (-s) have such a switch, too.

Vale,
Quintus

sync with filter doesn't download anything...

I'm assuming that "sync" is supposed to download new entries, and that manual downloading isn't necessary (but the readme isn't clear to me).

I want to grab an audio podcast that publishes episodes and then all the components of those episodes as separate entries (unfortunately). But fortunately, the titles of the latter always include "Episode". So I've been able to filter on that.

But doing a greg sync tnyrh doesn't do anything, even though there is new material shown by greg list, and greg download 0-9 does grab new stuff named "Episode".

What am I doing wrong, or is there a bug maybe between sync and filter?

greg sync crashes when run from (f)cron

I've got a simple greg sync running as a cron job (via fcron under Arch Linux). It works fine when I run it in the terminal, but when running as a cron job I get the following output:

Traceback (most recent call last):
File "/usr/bin/greg", line 9, in
load_entry_point('Greg==0.4.6', 'console_scripts', 'greg')()
File "/usr/lib/python3.5/site-packages/greg/gregparser.py", line 118, in main
function(vars(args))
File "/usr/lib/python3.5/site-packages/greg/greg.py", line 745, in sync
session = Session(args)
File "/usr/lib/python3.5/site-packages/greg/greg.py", line 55, in init
self.data_dir = self.retrieve_data_directory()
File "/usr/lib/python3.5/site-packages/greg/greg.py", line 95, in retrieve_data_directory
config.read([config_filename_global, self.config_filename_user])
File "/usr/lib/python3.5/configparser.py", line 696, in read
self._read(fp, filename)
File "/usr/lib/python3.5/configparser.py", line 1012, in _read
for lineno, line in enumerate(fp, start=1):
File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 33: ordinal not in range(128)

Tagging with date_format

I am attempting to download a file using a custom name of the format: YYYYMMDD-filename.mp3

Here are the relevant settings:

date_format = %Y%m%d
file_to_tag = {date}-{filename_title}.mp3
downloadhandler = wget {link} -O {directory}/{date}-{filename_title}.mp3

When I run this, I receive the following error:

Traceback (most recent call last):
File "/usr/bin/greg", line 11, in
load_entry_point('Greg==0.4.7', 'console_scripts', 'greg')()
File "/usr/lib/python3.5/site-packages/greg/parser.py", line 138, in main
function(vars(args))
File "/usr/lib/python3.5/site-packages/greg/commands.py", line 176, in sync
downloaded = feed.download_entry(entry)
File "/usr/lib/python3.5/site-packages/greg/classes.py", line 325, in download_entry
placeholders = aux.check_directory(placeholders)
File "/usr/lib/python3.5/site-packages/greg/aux_functions.py", line 142, in check_directory
subdnametemplate, placeholders)
File "/usr/lib/python3.5/site-packages/greg/aux_functions.py", line 284, in substitute_placeholders
date=placeholders.date_string(),
File "/usr/lib/python3.5/site-packages/greg/classes.py", line 373, in date_string
date_format = self.feed.retrieve_config("date_format", "%Y-%m-%d")
File "/usr/lib/python3.5/site-packages/greg/classes.py", line 135, in retrieve_config
answer = self.config.get(section, value, fallback=default)
File "/usr/lib/python3.5/configparser.py", line 797, in get
d)
File "/usr/lib/python3.5/configparser.py", line 393, in before_get
self._interpolate_some(parser, option, L, value, section, defaults, 1)
File "/usr/lib/python3.5/configparser.py", line 443, in _interpolate_some
"found: %r" % (rest,))
configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%Y%m%d'

Custom download handler

Hi, I would like to be able to set a custom download handler based on mime-type (and/or per feed). Reason:

  • Would be useful to run greg in my machine, but make the downloads in the server that runs aria2.
  • People would be able to set download limit, number of connections...
  • Mime-type support would allow the download of podcasts avaliable as torrents (like the ones from: http://bitlove.org/) and video podcasts.

Ignore Defective Entries [Suggection]

Hi,
So, the latest entries in these feeds appear to be defective:
http://podcast.c-span.org/xml/radio_feed.xml
https://thisishell.com/broadcast.xml
http://www.bbc.co.uk/programmes/p02nrsmh/episodes/downloads.rss

When greg encounters them it outputs this error and exits:
... something went wrong. Are you connected to the internet?

As a result, greg is effectively broken when it is run as a cron job and 'greg sync' is limited. It would be extremely helpdul if there was an argument which would force greg to move on to the next feed even when an entry is defective or to ignore a given feed during 'greg sync'.

Make download() behave like sync() when it comes to entry dates.

As @FilipBB wrote here:

when using 'greg download' after 'greg check' it was saving all the files using localtime. I noticed because I use the {date} placeholder to save filenames. sync() uses the published date, so I looked into the download() function and it looks like it just sets entry.linkdate to the localtime without checking for entry.published_parsed or entry.updated_parsed. sync() does check for those, so I basically just copied the snippet from sync() to download() and it seems to be setting {date} correctly now. If there was another reason for setting entry.linkdate to localtime that I'm overlooking, please let me know,

Wipe unwanted ID3 tags

I'm finding a few podcasts that I download that have track numbers in them, for some reason, that I don't want. I tried setting the tag_track to an empty string, but that causes an error. Is there a way to wipe unwanted id3 tags in downloaded files?

I could see two ways of doing this:

  1. Have a {none} placeholder that caused that tag to be wiped.
  2. Have an option to wipe all tags from a file before writing only the specified ones.

Greg always tries to download everything

Hi there,

I’m trying to use Greg for synchronising this feed: http://www.jurafunk.de/assets/podcasts/jurafunk.xml

Feed information:

% greg info

Jurafunk
--------
    url: http://www.jurafunk.de/assets/podcasts/jurafunk.xml
    Next sync will download from: 01 Mar 2014 16:45:23.

Binärgewitter
-------------
    url: http://feeds.feedburner.com/binaergewitter-podcast-opus?format=xml
    Next sync will download from: 28 Feb 2014 00:00:00.

When I try to sync that feed, I always get this:

% greg sync
I cannot parse the time information of this feed.                    I'll use your current local time instead.
Checking Jurafunk...

And Greg starts to download ALL entries and the MP3 files (which are over 100 and which is definitely not what I want).

Greg version is 0.3.0.

Vale,
Quintus

greg add IsADirectoryError - missing global conf file

Traceback (most recent call last):
  File "/home/nick/.local/bin/greg", line 9, in <module>
    load_entry_point('Greg==0.4.5', 'console_scripts', 'greg')()
  File "/home/nick/.local/lib/python3.4/site-packages/greg/gregparser.py", line 117, in main
    args.func(vars(args))
  File "/home/nick/.local/lib/python3.4/site-packages/greg/greg.py", line 629, in add
    with open(session.data_filename, 'w') as configfile:
IsADirectoryError: [Errno 21] Is a directory: '/home/nick/.local/share/greg/data'

Issue when i try to sync

When i try to sync my feeds with 'greg sync', i get this error :

Traceback (most recent call last):
File "/usr/bin/greg", line 18, in
main()
File "/usr/lib/python3.3/site-packages/greg/gregparser.py", line 103, in main
args.func(vars(args))
File "/usr/lib/python3.3/site-packages/greg/greg.py", line 698, in sync
feed = Feed(session, target, None)
File "/usr/lib/python3.3/site-packages/greg/greg.py", line 117, in init
self.sync_by_date = self.has_date()
File "/usr/lib/python3.3/site-packages/greg/greg.py", line 197, in has_date
if session.feeds[name]["date_info"] == "not available":
File "/usr/lib/python3.3/configparser.py", line 1220, in getitem
raise KeyError(key)
KeyError: 'date_info'

And if i download with 'greg download 1', i get this :
... something went wrong.Are you sure your last greg check went well?

Thank for your help
Rwangum

greg does not list podcasts

`
~ % cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu Xenial Xerus (development branch)"

~ % which greg
.local/bin/greg
`

~ % greg list Traceback (most recent call last): File ".local/bin/greg", line 9, in <module> load_entry_point('Greg==0.4.6', 'console_scripts', 'greg')() File "/home/swadnerkar/.local/lib/python3.5/site-packages/greg/gregparser.py", line 118, in main function(vars(args)) File "/home/swadnerkar/.local/lib/python3.5/site-packages/greg/greg.py", line 734, in list_for_user session = Session(args) File "/home/swadnerkar/.local/lib/python3.5/site-packages/greg/greg.py", line 55, in __init__ self.data_dir = self.retrieve_data_directory() File "/home/swadnerkar/.local/lib/python3.5/site-packages/greg/greg.py", line 95, in retrieve_data_directory config.read([config_filename_global, self.config_filename_user]) File "/usr/lib/python3.5/configparser.py", line 696, in read self._read(fp, filename) File "/usr/lib/python3.5/configparser.py", line 1012, in _read for lineno, line in enumerate(fp, start=1): File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 33: ordinal not in range(128)

Output podcast names on separate lines (for greg list)

Currently if I use greg list the output looks like the following:

$ greg list
The History Of Rome Hardcore History Revolutions Radiolab History Extra BBC World War One Rex Factor The History of WWII Podcast Stuff You Missed in History Class The History of England Norman Centuries Hoor! Geschiedenis

It is not possible to distinguish the names from each other as I use spaces in podcast names.

It would be great if greg list output could be changed to output names on separate lines, which would make the above readable and easily parse-able by a program, if anybody would be so inclined.

I looked through the code: it would only require removing the end=" " argument to print() here.

Won't download anything

Greg seems to be just the program I am looking for but I can't get it to download anything.

I've installed both the "greg-git" and "greg" packages from the Arch User Repository and everything works fine until I type "greg download 1" (or any number). This is the output I receive:

Traceback (most recent call last):
File "/usr/bin/greg", line 18, in
main()
File "/usr/lib/python3.4/site-packages/greg/gregparser.py", line 103, in main
args.func(vars(args))
File "/usr/lib/python3.4/site-packages/greg/greg.py", line 855, in download
feed.download_entry(entry)
File "/usr/lib/python3.4/site-packages/greg/greg.py", line 287, in download_entry
for mimetype in self.mime]):
File "/usr/lib/python3.4/site-packages/greg/greg.py", line 287, in
for mimetype in self.mime]):
File "/usr/lib/python3.4/site-packages/feedparser.py", line 375, in getitem
return dict.getitem(self, key)
KeyError: 'type'

The only thing that means anything to me is "mime". I left the mime option as the default audio setting in the configuration file. Specifically: mime = audio

Thanks for taking the time to make this application for me to use.

Automating downloads?

I'm new to greg and my apologies if I'm missing something in the documentation or the configuration file, but does greg have the ability to keep a number of podcasts and clean up previous downloads? For example if I wanted to always have on hand the last 5 days of a subscribed show, I assume I would run 'greg download [0-4|-d YYYY-MM-DD]' via cron or something but would previous downloads be cleaned up or would I have to have another script or something do that for me?

I am planning on downloading the podcasts to a shared location that I'd like to expose to my home theater media manager and don't always listen everyday but it would be nice to have the last few days available.

I appreciate the work on this, it looks pretty awesome!

Failure to download episodes from "Hello Internet" podcast

I have been trying to download episodes from the "Hello Internet" podcast, but greg fails with an error message:

$ greg add HelloInternet http://www.hellointernet.fm/podcast?format=rss
$ greg sync HelloInternet
Checking HelloInternet...
Traceback (most recent call last):
  File "/usr/bin/greg", line 9, in <module>
    load_entry_point('Greg==0.4.4.3', 'console_scripts', 'greg')()
  File "/usr/lib/python3/dist-packages/greg/gregparser.py", line 112, in main
    args.func(vars(args))
  File "/usr/lib/python3/dist-packages/greg/greg.py", line 746, in sync
    feed.download_entry(entry)
  File "/usr/lib/python3/dist-packages/greg/greg.py", line 286, in download_entry
    for mimetype in self.mime]):
  File "/usr/lib/python3/dist-packages/greg/greg.py", line 286, in <listcomp>
    for mimetype in self.mime]):
  File "/usr/lib/python3/dist-packages/feedparser.py", line 383, in __getitem__
    return dict.__getitem__(self, key)
KeyError: 'type'

Other podcasts work fine. I'm running Debian Jessie with stock packages (python 3.4.2 and python3-feedparser 5.1.3).

Download in the right order

When downloading, we currently download every available issue in no particular order. That's wrong: if the connection is lost while we are doing this, we will leave a later date in the data file, and this will instruct greg not to subsequently download the earlier issues.

Greg should download issues from earlier to later.

installation

Hi Manolo

sorry to make an "issue" out of this, but could you provide a basic how-to install greg?

I'm hoping to install on a debian server, just need a few pointers...

thanks!

Lee

Password Protected Feeds

I have a password protected feed that I can't seem to get to work. I've tried putting user = xxxx and password = yyyy in greg.conf and also passing the username and password into the url, i.e. https://user:[email protected]/ (there is a trailing slash in feed url). Neither approach has worked so far. Is there a simple approach that I've missed? I can provide more specific information if needed.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.