korymath / talk-generator Goto Github PK

View Code? Open in Web Editor NEW

121.0 8.0 9.0 10.99 MB

talk-generator is capable of generating coherent slide decks based on a single topic suggestion.

License: MIT License

Python 100.00%

python python3 funny-experiments evolutionary-computation art

talk-generator's Introduction

Hello, I'm Kory!

I am a Research Scientist with DeepMind and a Lab Scientist with the Creative Destruction Lab.

I have a Ph.D. in Computing Science from the University of Alberta with the Alberta Machine Intelligence Institute.

My research focuses on understanding interaction between intelligent systems.

I am currently focusing on 🧑‍🦳-🤖 interfaces.

You might also know me as an improvisational theatre performance artist with Rapid Fire Theatre.

I like to fuse my interests by developing artificial intelligences to perform theatre alongside.

For more, https://korymathewson.com/ and Improbotics

talk-generator's People

Contributors

Stargazers

Watchers

Forkers

ml-lab stjordanis louisgv alulujasmine zhongxingpeng spencerx zorrock hanspeterstalder abdeslemsmahi

talk-generator's Issues

setup.py package installation

To add

setup.py should install some command line tools when someone does a pip install of the talk generator
** this will allow the user to execute the talkgenerator by typing talkgenerator instead of python etc etc etc
download the NLTK libraries on startup and print error message if they cannot be downloaded automatically.

Cleaning Out Old Web Front End

remove the flask portion of the app from the talk-generator since it has nothing to do with generating talks

Using a Generative Model for Image Generation

Wouldn't it be great if you could just input a word and have a bunch of images generated using a deep neural network for pixel-based image generation?

It is possible.

Let's make it happen?!

As a reviewer suggested, and as we have discussed many times in the past, there might be some use for a filter of "risky" content. Although it will be hard to eliminate all such violations (due to the fact that it is hard to know where to draw the line, and to recognise this in images), we can probably add a variable that turns certain (parts) of content generators off, in order to decrease the probability of weird images.

What should we name such a variable? We might take something like sfw or childfriendly, but I don't know if these names would be appropriate since it will be hard to absolutely guarantee that.

unsplash required? might need credentials or feature flag to shut off

Feel free to close if not an issue.

It looks like unsplash credentials are required at this point? If so, we can add some validation code to settings.py to ensure that the generator doesn't start if the credentials are not set in .env. Or we can just shut off unsplash if the credentials are not set.

WEBP image format throws exception

 * Finished generating slide 2 about bag of hamburger using Conclusion in 7.49 seconds *
Traceback (most recent call last):
  File "/Users/korymathewson/work/talk-generator/venv/lib/python3.6/site-packages/pptx/util.py", line 133, in get_prop_value
    return getattr(obj, cache_attr_name)
AttributeError: 'Image' object has no attribute '_ext'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/korymathewson/work/talk-generator/talkgenerator/slide/powerpoint_slide_creator.py", line 91, in _add_image
    placeholder = placeholder.insert_picture(image_url)
  File "/Users/korymathewson/work/talk-generator/venv/lib/python3.6/site-packages/pptx/shapes/placeholder.py", line 323, in insert_picture
    pic = self._new_placeholder_pic(image_file)
  File "/Users/korymathewson/work/talk-generator/venv/lib/python3.6/site-packages/pptx/shapes/placeholder.py", line 334, in _new_placeholder_pic
    rId, desc, image_size = self._get_or_add_image(image_file)
  File "/Users/korymathewson/work/talk-generator/venv/lib/python3.6/site-packages/pptx/shapes/placeholder.py", line 345, in _get_or_add_image
    image_part, rId = self.part.get_or_add_image_part(image_file)
  File "/Users/korymathewson/work/talk-generator/venv/lib/python3.6/site-packages/pptx/parts/slide.py", line 42, in get_or_add_image_part
    image_part = self._package.get_or_add_image_part(image_file)
  File "/Users/korymathewson/work/talk-generator/venv/lib/python3.6/site-packages/pptx/package.py", line 50, in get_or_add_image_part
    return self._image_parts.get_or_add_image_part(image_file)
  File "/Users/korymathewson/work/talk-generator/venv/lib/python3.6/site-packages/pptx/package.py", line 161, in get_or_add_image_part
    image_part = ImagePart.new(self._package, image)
  File "/Users/korymathewson/work/talk-generator/venv/lib/python3.6/site-packages/pptx/parts/image.py", line 44, in new
    partname = package.next_image_partname(image.ext)
  File "/Users/korymathewson/work/talk-generator/venv/lib/python3.6/site-packages/pptx/util.py", line 135, in get_prop_value
    value = f(obj)
  File "/Users/korymathewson/work/talk-generator/venv/lib/python3.6/site-packages/pptx/parts/image.py", line 244, in ext
    raise ValueError(tmpl % (ext_map.keys(), format))
ValueError: unsupported image format, expected one of: dict_keys(['BMP', 'GIF', 'JPEG', 'PNG', 'TIFF', 'WMF']), got 'WEBP'
_add_image error: unsupported image format, expected one of: dict_keys(['BMP', 'GIF', 'JPEG', 'PNG', 'TIFF', 'WMF']), got 'WEBP'

Breaking Issue: KeyError: 'language'

talkgenerator --topic 'garden snake' --num_slides 10
2019-08-16 11:32:19,457 - talkgenerator - INFO - ******************************************
2019-08-16 11:32:19,457 - talkgenerator - INFO - Making 10 slide talk on: garden snake
2019-08-16 11:32:19,678 - talkgenerator.conceptnet - INFO - Took 0.19663171700085513 seconds to poll Conceptnet for 'garden snake'
2019-08-16 11:32:19,678 - talkgenerator - INFO - Conceptnet related words failing: 'language'
2019-08-16 11:32:19,679 - talkgenerator - INFO - Conceptnet related words failing: 'language'
2019-08-16 11:32:19,679 - talkgenerator - INFO - SideTrackingTopicGenerator concept seeds: ['garden snake', None, None, None, None, 'garden snake', None, None, None, 'garden snake']
2019-08-16 11:32:19,679 - talkgenerator - INFO - Generating the slide deck in parallel
2019-08-16 11:32:19,688 - talkgenerator - INFO - * Generating slide 1 about garden snake using Title slide *
2019-08-16 11:32:19,688 - talkgenerator - INFO - * Generating slide 2 about garden snake using Two Captions Weird Reddit *
2019-08-16 11:32:19,688 - talkgenerator - INFO - * Generating slide 3 about garden snake using Two Captions Gifs *
2019-08-16 11:32:19,688 - talkgenerator - INFO - * Generating slide 4 about garden snake using Two History Pictures *
2019-08-16 11:32:19,688 - talkgenerator - INFO - * Generating slide 5 about garden snake using Yes/No/Funny Chart *
2019-08-16 11:32:19,688 - talkgenerator - INFO - * Generating slide 6 about garden snake using Inspirobot *
2019-08-16 11:32:19,688 - talkgenerator - INFO - * Generating slide 7 about garden snake using Correlation Curve *
2019-08-16 11:32:19,689 - talkgenerator - INFO - * Generating slide 8 about garden snake using Reddit Chart *
2019-08-16 11:32:19,689 - talkgenerator - INFO - * Generating slide 9 about garden snake using Yes/No/Funny Chart *
2019-08-16 11:32:19,689 - talkgenerator - INFO - * Generating slide 10 about garden snake using 2 Conclusions *
2019-08-16 11:32:19,689 - talkgenerator - INFO - * Finished generating slide 1 about garden snake using Title slide in 0.0 seconds *
2019-08-16 11:32:19,982 - talkgenerator.conceptnet - INFO - Took 0.2712950009954511 seconds to poll Conceptnet for 'garden snake'
2019-08-16 11:32:19,983 - talkgenerator - WARNING - WARNING: Problem logging in on Wikihow: Advanced Search disabled
2019-08-16 11:32:20,245 - talkgenerator - INFO - * Finished generating slide 6 about garden snake using Inspirobot in 0.56 seconds *
2019-08-16 11:32:20,275 - talkgenerator - WARNING - WARNING: Problem logging in on Wikihow: Advanced Search disabled
2019-08-16 11:32:20,318 - talkgenerator.conceptnet - INFO - Took 0.3359065800032113 seconds to poll Conceptnet for 'garden snake'
2019-08-16 11:32:21,494 - talkgenerator - INFO - * Finished generating slide 5 about garden snake using Yes/No/Funny Chart in 1.81 seconds *
2019-08-16 11:32:21,695 - talkgenerator - INFO - * Finished generating slide 9 about garden snake using Yes/No/Funny Chart in 2.01 seconds *
2019-08-16 11:32:22,494 - talkgenerator - INFO - * Finished generating slide 10 about garden snake using 2 Conclusions in 2.81 seconds *
2019-08-16 11:32:25,229 - talkgenerator - INFO - * Finished generating slide 8 about garden snake using Reddit Chart in 5.54 seconds *
2019-08-16 11:32:28,128 - talkgenerator - INFO - * Finished generating slide 4 about garden snake using Two History Pictures in 8.44 seconds *
2019-08-16 11:32:38,805 - talkgenerator - INFO - * Finished generating slide 2 about garden snake using Two Captions Weird Reddit in 19.12 seconds *
2019-08-16 11:32:42,938 - talkgenerator - INFO - * Finished generating slide 3 about garden snake using Two Captions Gifs in 23.25 seconds *
Traceback (most recent call last):
  File "/Users/korymathewson/work/talk-generator/venv/bin/talkgenerator", line 11, in <module>
    load_entry_point('talkgenerator', 'console_scripts', 'talkgenerator')()
  File "/Users/korymathewson/work/talk-generator/talkgenerator/run.py", line 11, in main_cli
    main(args)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/run.py", line 6, in main
    presentations, slide_deck = utils.generate_talk(args)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/utils.py", line 63, in generate_talk
    parallel=args.parallel,
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/presentation_schema.py", line 68, in generate_presentation
    used_tags,
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/presentation_schema.py", line 115, in _generate_slide_deck_parallel
    slide_nrs_to_generate,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 608, in get
    raise self._value
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/presentation_schema.py", line 347, in __call__
    prohibited_generators=self.prohibited_generators,
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/presentation_schema.py", line 228, in generate_slide
    slide_result = generator.generate(presentation_context, used_elements)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/slide_generator_data.py", line 70, in generate
    presentation_context, (used_elements, self._allowed_repeated_elements)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/slide/slide_generators.py", line 27, in generate_slide
    generated = self._slide_content_generator(presentation_context)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/sources/chart.py", line 309, in generate_correlation_curve
    x_label = _CORRELATION_WORD_GENERATOR(y_label)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/util/generator_util.py", line 230, in __call__
    generated = self._inner_generator(current)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/util/generator_util.py", line 24, in __call__
    generated = generator(seed)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/util/generator_util.py", line 211, in __call__
    weighted_list = self._weighted_list_creator(argument)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/sources/conceptnet.py", line 151, in get_weighted_related_words
    for edge in edges
  File "/Users/korymathewson/work/talk-generator/talkgenerator/sources/conceptnet.py", line 152, in <listcomp>
    if is_different_enough_label(edge["end"], word) and is_english(edge["end"])
  File "/Users/korymathewson/work/talk-generator/talkgenerator/sources/conceptnet.py", line 133, in is_english
    return not node["language"] or node["language"] == "en"
KeyError: 'language'

Docker image is 1 Gig, make it smaller

Large docker images are a pain to store and move around.

Using Multiple Input Keywords

Imagine a situation where a user does not only want to input a single word for the topic of a presentation, but rather, they want to input a sequence of words which together serve as a set of topics for the presentation.

Rather than input: cat -> output: slides related to cat and related things, the input might be cat, animal fight, new house, new cat -> output: slides which follow the story that the user has input.

How might we handle handling a sequence of multiple keywords?

Include hooks for additional corpus / data sources

Including:

Test failing on generate talk from multiple topics

Duplicate with standard test runners:

coverage run -m pytest; coverage html

_____________________________________________________________________ TestTalkGenerator.test_multiple_topics ______________________________________________________________________

self = <tests.test_talkgenerator.TestTalkGenerator testMethod=test_multiple_topics>

    def test_multiple_topics(self):
        self.default_args.configure_mock(topic="cat, dog, bread, house")
        self.default_args.configure_mock(num_slides=6)
        ppt, slide_deck, location = generator.generate_presentation_using_cli_arguments(
>           self.default_args
        )

tests/test_talkgenerator.py:63:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
talkgenerator/generator.py:44: in generate_presentation_using_cli_arguments
    open_ppt=args.open_ppt,
talkgenerator/generator.py:94: in generate_presentation
    save_ppt=save_ppt,
talkgenerator/schema/presentation_schema.py:109: in generate_presentation
    slide_deck.save_to_powerpoint(presentation)
talkgenerator/slide/slide_deck.py:29: in save_to_powerpoint
    return [slide.create_powerpoint_slide(prs_template) for slide in self._slides]
talkgenerator/slide/slide_deck.py:29: in <listcomp>
    return [slide.create_powerpoint_slide(prs_template) for slide in self._slides]
talkgenerator/slide/slides.py:30: in create_powerpoint_slide
    ppt_slide = self._ppt_slide_creator(prs, **self._arguments)
talkgenerator/slide/powerpoint_slide_creator.py:257: in create_two_column_images_slide
    _add_image_or_text(slide, 14, image_or_text_2, original_image_size)
talkgenerator/slide/powerpoint_slide_creator.py:179: in _add_image_or_text
    return _add_image(slide, placeholder_id, image_url_or_text, original_image_size)
talkgenerator/slide/powerpoint_slide_creator.py:129: in _add_image
    width, height = image_ref.image().size
talkgenerator/slide/powerpoint_slide_creator.py:71: in image
    return Image.open(self.get_bytes_io())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

fp = <_io.BytesIO object at 0x125b5d308>, mode = 'r'

    def open(fp, mode="r"):
        """
        Opens and identifies the given image file.

        This is a lazy operation; this function identifies the file, but
        the file remains open and the actual image data is not read from
        the file until you try to process the data (or call the
        :py:meth:`~PIL.Image.Image.load` method).  See
        :py:func:`~PIL.Image.new`. See :ref:`file-handling`.

        :param fp: A filename (string), pathlib.Path object or a file object.
           The file object must implement :py:meth:`~file.read`,
           :py:meth:`~file.seek`, and :py:meth:`~file.tell` methods,
           and be opened in binary mode.
        :param mode: The mode.  If given, this argument must be "r".
        :returns: An :py:class:`~PIL.Image.Image` object.
        :exception FileNotFoundError: If the file cannot be found.
        :exception PIL.UnidentifiedImageError: If the image cannot be opened and
           identified.
        :exception ValueError: If the ``mode`` is not "r", or if a ``StringIO``
           instance is used for ``fp``.
        """

        if mode != "r":
            raise ValueError("bad mode %r" % mode)
        elif isinstance(fp, io.StringIO):
            raise ValueError(
                "StringIO cannot be used to open an image. "
                "Binary data must be used instead."
            )

        exclusive_fp = False
        filename = ""
        if isinstance(fp, Path):
            filename = str(fp.resolve())
        elif isPath(fp):
            filename = fp

        if filename:
            fp = builtins.open(filename, "rb")
            exclusive_fp = True

        try:
            fp.seek(0)
        except (AttributeError, io.UnsupportedOperation):
            fp = io.BytesIO(fp.read())
            exclusive_fp = True

        prefix = fp.read(16)

        preinit()

        accept_warnings = []

        def _open_core(fp, filename, prefix):
            for i in ID:
                try:
                    factory, accept = OPEN[i]
                    result = not accept or accept(prefix)
                    if type(result) in [str, bytes]:
                        accept_warnings.append(result)
                    elif result:
                        fp.seek(0)
                        im = factory(fp, filename)
                        _decompression_bomb_check(im.size)
                        return im
                except (SyntaxError, IndexError, TypeError, struct.error):
                    # Leave disabled by default, spams the logs with image
                    # opening failures that are entirely expected.
                    # logger.debug("", exc_info=True)
                    continue
                except BaseException:
                    if exclusive_fp:
                        fp.close()
                    raise
            return None

        im = _open_core(fp, filename, prefix)

        if im is None:
            if init():
                im = _open_core(fp, filename, prefix)

        if im:
            im._exclusive_fp = exclusive_fp
            return im

        if exclusive_fp:
            fp.close()
        for message in accept_warnings:
            warnings.warn(message)
        raise UnidentifiedImageError(
>           "cannot identify image file %r" % (filename if filename else fp)
        )
E       PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x125b5d308>

venv/lib/python3.6/site-packages/PIL/Image.py:2931: UnidentifiedImageError

add image filter for weird and NSFW stuff

type error when no images are returned

Should handle this no response nicely...

run code: python3 run.py --topic 'edmonton' --num_slides 5 --num_images 1

(venv)  ✘  ~/work/talk-generator   master  python3 run.py --topic 'edmonton' --num_slides 5 --num_images 1
******************************************
Making 5 slide talk on: edmonton
Pickle saved to output/edmonton.pkl
******************************************
Generating slide 1 about edmonton using SlideGenerator[Title slide]
Generating slide 2 about edmonton using SlideGenerator[Giphy]
Generating slide 3 about edmonton using SlideGenerator[Google Images]
Traceback (most recent call last):
  File "run.py", line 436, in <module>
    main(args)
  File "run.py", line 345, in main
    presentation = schema.generate_presentation(arguments.topic, arguments.num_slides)
  File "/Users/korymathewson/work/talk-generator/presentation_schema.py", line 59, in generate_presentation
    self._generate_slide(presentation, seed_generator, slide_nr, num_slides, set())
  File "/Users/korymathewson/work/talk-generator/presentation_schema.py", line 72, in _generate_slide
    slide = generator.generate(presentation, seed)
  File "/Users/korymathewson/work/talk-generator/presentation_schema.py", line 21, in generate
    slide = self._generator(presentation, seed)
  File "/Users/korymathewson/work/talk-generator/slide_templates.py", line 128, in <lambda>
    return lambda prs, seed: create_full_image_slide(prs, title_generator(seed), image_generator(seed))
  File "run.py", line 272, in get_related_google_image
    img_paths = get_google_images(seed_word, 1)
  File "run.py", line 166, in get_google_images
    if len(paths) == 0:
TypeError: object of type 'NoneType' has no len()

downloaded files default to a temp directory instead of current working

I'd like to shift the downloaded file manager to download files to a temp directory that can be removed by the OS during it's normal times. python has a facility for this called tempfiles and it's built in. Thanks python.

Thoughts?

KeyError Prohibited Image Not Found?

Traceback (most recent call last):
  File "/Users/korymathewson/work/talk-generator/venv/bin/talkgenerator", line 11, in <module>
    load_entry_point('talkgenerator', 'console_scripts', 'talkgenerator')()
  File "/Users/korymathewson/work/talk-generator/talkgenerator/run.py", line 11, in main_cli
    main(args)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/run.py", line 6, in main
    presentations, slide_deck = utils.generate_talk(args)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/utils.py", line 63, in generate_talk
    parallel=args.parallel,
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/presentation_schema.py", line 68, in generate_presentation
    used_tags,
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/presentation_schema.py", line 115, in _generate_slide_deck_parallel
    slide_nrs_to_generate,
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 608, in get
    raise self._value
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/presentation_schema.py", line 347, in __call__
    prohibited_generators=self.prohibited_generators,
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/presentation_schema.py", line 228, in generate_slide
    slide_result = generator.generate(presentation_context, used_elements)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/slide_generator_data.py", line 70, in generate
    presentation_context, (used_elements, self._allowed_repeated_elements)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/slide/slide_generators.py", line 27, in generate_slide
    generated = self._slide_content_generator(presentation_context)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/slide/slide_generators.py", line 236, in __call__
    self._image_2_generator(presentation_context),
  File "/Users/korymathewson/work/talk-generator/talkgenerator/util/generator_util.py", line 24, in __call__
    generated = generator(seed)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/util/generator_util.py", line 24, in __call__
    generated = generator(seed)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/util/generator_util.py", line 193, in __call__
    generated = generator(context)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/content_generator_structures.py", line 88, in generate
    return self._generate(presentation_context)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/util/generator_util.py", line 171, in __call__
    if os_util.is_valid_image(downloaded_url):
  File "/Users/korymathewson/work/talk-generator/talkgenerator/util/os_util.py", line 84, in is_valid_image
    im = open_image(os.path.normpath(image_url))
KeyError: ('/Users/korymathewson/work/talk-generator/talkgenerator/data/prohibited_images/imgur_removed.jpg',)

Spelling mistake in README.md

Go to your App prefenrences

Raw text templates should be stored in a folder, and pulled into python code

I added the templates to a template folder. From there we can add to separate lists and then pull into a data structure which holds our templates when we start the generator. This is a way of separating code and data.

Failed to generated using ...

When the system fails to generate a slide, it moves on to another generator.
That said, it does not generate an error message:

 * Generating slide 43 about bedroom using Goodreads Quote *
Failed to generated using: Goodreads Quote

 * Generating slide 43 about bedroom using Full Screen Giphy *

For this given error, it is unclear why the system failed, and how to fix it?

Support German

Hey :)

I love your website and tool. Would it be easily possible to support German? So I could use the slides with my friends.

best regards
Marco

Incorporating Stock Images from Unsplash

Starting with building it as a source (#29), then add it to some of the generators

Web interface

The codebase is too diffucult to easily be used by (often non-tech-savvy) improvisators. As such, we should probably provide a web interface for our system at this point. We will be able to hide required credentials (e.g. for reddit and wikihow) in the server variables. A good place to host it for free is Heroku.

The web interface itself should probably just present a text box for the topic and a generate button, and possibly also the options to determine the number of slides and possibly the name of the presenter.

pixabay image source is broken

talkgenerator --topic "peanuts" --num_slides 10

Traceback (most recent call last):
  File "/Users/korymath/Documents/code/talk-generator/venv/bin/talkgenerator", line 33, in <module>
    sys.exit(load_entry_point('talkgenerator', 'console_scripts', 'talkgenerator')())
  File "/Users/korymath/Documents/code/talk-generator/venv/bin/talkgenerator", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/run.py", line 1, in <module>
    from talkgenerator import generator
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/generator.py", line 13, in <module>
    from talkgenerator.schema.content_generators import full_name_generator
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/schema/content_generators.py", line 3, in <module>
    from talkgenerator.sources import pixabay, pexels
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/sources/pixabay.py", line 5, in <module>
    from pixabay import Image
ModuleNotFoundError: No module named 'pixabay'

Compile and save to raw data.file

Server Environment Variables

Currently, CircleCI is not able to test all the generators due to missing Reddit authentication.
This is due to the fact that I don't want these secrets exposed in the repository in case we ever make it public.
We can get around this problem by adding an alternative way of entering these three keys necessary to create the access to Reddit by having server environment variables. This will also allow us to more easily set up a server (e.g. on Heroku) with these server variables.
We should probably look into how to do this properly

Add good example input/output pair to README.

Would be great to include a solid 15 slide generated talk to the repo.
Can we include

the run code to create it
the output file
a small gif of the slides rendered from keynote to a gif.

This is a nice visual and a solid milestone for the project.

Google images not downloading anymore

The Google image search function doesn't download any new images anymore now that it checks local cache. It only looks for local images

This is probably due to the "search_google_images=False" in the header of "get_images", as this will by default falsify line 182 (if len(all_paths[word]) == 0 and search_google_images:).
I'm not sure whether search_google_images should be true, or the "and" should be an "or" in search_google_images, or whether this variable should be removed (as it might be a leftover from testing) as this all depends on the intended semantics of that variable.

Removing the variable from the conjunction seems to fix the problem, as it still downloads if the images wasn't downloaded before, and uses the cache otherwise.
Could you take a look at it to see what was intended?

Breaking Issue: JSONDecodeError

Run code:

talkgenerator --topic 'gardens' --num_slides 10

Output:

2019-08-16 11:30:10,339 - talkgenerator - INFO - ******************************************
2019-08-16 11:30:10,339 - talkgenerator - INFO - Making 10 slide talk on: gardens
2019-08-16 11:30:10,733 - talkgenerator.conceptnet - INFO - Took 0.3449703459991724 seconds to poll Conceptnet for 'gardens'
Traceback (most recent call last):
  File "/Users/korymathewson/work/talk-generator/venv/bin/talkgenerator", line 11, in <module>
    load_entry_point('talkgenerator', 'console_scripts', 'talkgenerator')()
  File "/Users/korymathewson/work/talk-generator/talkgenerator/run.py", line 11, in main_cli
    main(args)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/run.py", line 6, in main
    presentations, slide_deck = utils.generate_talk(args)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/utils.py", line 63, in generate_talk
    parallel=args.parallel,
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/presentation_schema.py", line 47, in generate_presentation
    seed_generator = self._seed_generator(topics, num_slides)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/slide_topic_generators.py", line 40, in __init__
    fill_in_blank_topics_with_related(seeds)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/slide_topic_generators.py", line 69, in fill_in_blank_topics_with_related
    _fill_in(seeds, i)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/slide_topic_generators.py", line 98, in _fill_in
    for word in related
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/slide_topic_generators.py", line 99, in <listcomp>
    if not normalise_seed(word[1]) in seeds
  File "/Users/korymathewson/work/talk-generator/talkgenerator/schema/slide_topic_generators.py", line 117, in normalise_seed
    normalised = phrasefinder.get_rarest_word(normalised)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/sources/phrasefinder.py", line 46, in get_rarest_word
    return min(words, key=lambda word: get_absolute_frequency_any_casing(word))
  File "/Users/korymathewson/work/talk-generator/talkgenerator/sources/phrasefinder.py", line 46, in <lambda>
    return min(words, key=lambda word: get_absolute_frequency_any_casing(word))
  File "/Users/korymathewson/work/talk-generator/talkgenerator/sources/phrasefinder.py", line 36, in get_absolute_frequency_any_casing
    absolute_frequencies = _get_absolute_frequencies(word)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/sources/phrasefinder.py", line 17, in _get_absolute_frequencies
    pf_results = _search(word)
  File "/Users/korymathewson/work/talk-generator/talkgenerator/sources/phrasefinder.py", line 11, in _search
    result = requests.get(url).json()
  File "/Users/korymathewson/work/talk-generator/venv/lib/python3.6/site-packages/requests/models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Conclusions text should be dynamic.

Currently the conclusions text is always the same.

Conclusion 1 Conclusion 2

Perhaps we can use other templates to update the way that the last slide is presented?

Testing reddit and wikihow credentials

Is there a client side test for credentials?

Not creating the correct number of slides

python3 run.py --topic 'the secret to making a bagel' --num_slides 20

Should make a slide deck with 20 slides, this is currently not working.

Keeping the slide topic seeds simple

Currently, our system uses a generator that uses related conceptnet words to generate the next topic seed for a slide (where it comes back to the presentatation topic every couple of slides).
These seeds are sometimes uncommon words or not even words at all. They've also got different part of speeches often.
It might be interesting to look into n-grams to make the slide topic seed generator "prefer" more common words.
It might also be interesting to constrain the seeds to only be a noun, as that seems to work best with the current slide generators.
It might also be interesting to look at different graphs for generating related words, such as Wikipedia and Wiktionary links. The slide topic seed generator can then switch between these for more interesting and different types of results.

Update Install / Run code in the README

Currently, the installation and run scripts in the README do not work. Update these to reflect the current configuration of the software.

Setup as a Python Package so it can be installed and tested

At some point, we can change the package structure so that it could be installed and tested using python tools.

Making this a python package does not mean it will be public or in Pypi. Making this a package will allow talk-generator libraries and generators to be imported and used by other python applications in a more versatile way.

The conceptnet bug

Traceback (most recent call last):
  File "/Users/korymath/Documents/code/talk-generator/venv/bin/talkgenerator", line 33, in <module>
    sys.exit(load_entry_point('talkgenerator', 'console_scripts', 'talkgenerator')())
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/run.py", line 13, in main_cli
    main(args)
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/run.py", line 6, in main
    presentations, slide_deck, output_file = generator.generate_presentation_using_cli_arguments(
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/generator.py", line 34, in generate_presentation_using_cli_arguments
    return generate_presentation(
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/generator.py", line 99, in generate_presentation
    presentation, slide_deck = presentation_schema.generate_presentation(
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/schema/presentation_schema.py", line 92, in generate_presentation
    self._generate_slide_deck_parallel(
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/schema/presentation_schema.py", line 141, in _generate_slide_deck_parallel
    all_slide_results = pool.map(
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/schema/presentation_schema.py", line 398, in __call__
    return self.presentation_schema.generate_slide(
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/schema/presentation_schema.py", line 278, in generate_slide
    slide_result = generator.generate(presentation_context, used_elements)
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/datastructures/slide_generator_data.py", line 74, in generate
    slide_results = self._generator.generate_slide(
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/slide/slide_generator_types.py", line 34, in generate_slide
    generated = self._slide_content_generator(presentation_context)
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/sources/chart.py", line 280, in generate_location_pie
    result = generate_location_data(presentation_context)
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/sources/chart.py", line 253, in generate_location_data
    return _generate_conceptnet_data(
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/sources/chart.py", line 231, in _generate_conceptnet_data
    conceptnet_relations = conceptnet_function(seed)
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/sources/conceptnet.py", line 162, in get_weighted_related_locations
    edges = _get_edges(word, _LOCATION_ARGUMENTS)
  File "/Users/korymath/Documents/code/talk-generator/talkgenerator/sources/conceptnet.py", line 109, in _get_edges
    return data["edges"]
KeyError: 'edges'

Talkgenerator.submodules can use loggers instead of print

We can configure the talkgenerator submodules to use python logging module instead of printing out on standard out. This will allow consumers of the modules (those using the generators raw for other purposes) to configure the output as they see fit.

The talkgenerator.run module would then configure the loggers to continue to print on standard out.

Not a major thing needed for now since most people will run the talkgenerator from the command line (and thus probably want the output in standard out. This is more of a pythonic best practice that we can have the codebase follow.

Parallellisation of slide generators

Currently, generating the powerpoint takes quite some time. This could be sped up by generating the slides in parallel. This does have some implications for the code structure, and some dependencies that have to be taken into account:

python-pptx is not able to change the positions of slides, and slides have to be added incrementally. As such, instead of the slide_templates module generating the slide, it should return a function to add the slide, and the required slide generation arguments (=the generated content). The program can then add all of these one after another after all these objects are generated
The slide creation methods have boolean conditions checking if certain required content is present, before generating. This should be extracted out, such that this can be checked beforehand. An alternative is replacing this system with another way of checking if the required content got generated (or possibly always providing an alternative backup option)
There is a variable saying how much repeating elements between generated slides are allowed. This should still be respected, as it makes sure the slides feel original when they need to. The new interface for generating slides should thus also return a function that can be called in case the generated content is not satisfyin these originality constraints

Getting Pattern to work

Pattern is a very powerful Python library that would allow us to do more linguistic operations in our text generator, and thus offer more extensive possibilities in the text generator.
However, this used to be only available in Python 2, and only very recently started to be available in Python 3. I tried getting it to work with our code, but it didn't work out. We should probably look into this again in the near future when better support for Python 3 is provided, as they seem quite actively pushing that version out soon.

Unused files?

There are quite some files that are currently completely not covered by any test case, and not used anywhere in the project. I was wondering if we still need the following files, or if we can delete them, or if we need them if they should be in different repositories:

util/parallel_util.py
util/random_word_util.py
util/aws_s3.py
server/*
stress.py

What's your opinion about these files, @korymath @h0h0h0 ? Should any of them stay? (I presume the aws_s3.py is probably the only one being used by something externally?)

Conceptnet API being down stops slide generation

There was a small outage on Conceptnet Monday around noon ET. Slides did not generate. Creating issue for later fix. Worked around by waiting.

Stacktrace below:

~/projects/talk-generator/ $ python run.py --topic detroit --num_slides 10 --output_folder=./detroit/ --open_ppt=false
******************************************
Making 10 slide talk on: detroit
Traceback (most recent call last):
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1321, in getresponse
    response.begin()
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 296, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 257, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 54] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1321, in getresponse
    response.begin()
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 296, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 257, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run.py", line 821, in <module>
    main(args)
  File "run.py", line 85, in main
    presenter=arguments.presenter)
  File "/Users/shaun/projects/talk-generator/presentation_schema.py", line 112, in generate_presentation
    seed_generator = self._seed_generator(topic, num_slides)
  File "/Users/shaun/projects/talk-generator/slide_topic_generators.py", line 38, in __init__
    fill_in_blank_topics_with_related(seeds)
  File "/Users/shaun/projects/talk-generator/slide_topic_generators.py", line 56, in fill_in_blank_topics_with_related
    _fill_in(seeds, i)
  File "/Users/shaun/projects/talk-generator/slide_topic_generators.py", line 72, in _fill_in
    related = conceptnet.get_weighted_related_words(neighbour, 200)
  File "/Users/shaun/projects/talk-generator/conceptnet.py", line 111, in get_weighted_related_words
    edges = _get_edges(word, cache_util.HashableDict(limit=limit))
  File "/Users/shaun/projects/talk-generator/conceptnet.py", line 90, in _get_edges
    return _get_data(word, arguments)["edges"]
  File "/Users/shaun/projects/talk-generator/conceptnet.py", line 83, in _get_data
    result = requests.get(url).json()
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/requests/api.py", line 75, in get
    return request('get', url, params=params, **kwargs)
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/requests/sessions.py", line 524, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/requests/sessions.py", line 637, in send
    r = adapter.send(request, **kwargs)
  File "/Users/shaun/projects/talk-generator/venv/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))
(venv) :(  Mon Dec 03 12:24:25
~/projects/talk-generator/ $

Migrate TODO's incode to Issues in Github Repo

There are currently 9 outstanding TODOs in code.

These should be migrated to Github issues so that they can be appropriately tracked and addressed.

Click Here to Add Text in empty slide title boxes.

There are times when the slide Title box shows "Click here to add text". Not sure if this is an issue with the generator or with Powerpoint for Mac.

Example Slide. Tested on Powerpoint for Mac. Do not have windows so could not confirm if it was a platform specific issue.

Will defer to @korymath @twinters on proper handling.

WikiHow Advanced Search Login Problem

For several months, our WikiHow searcher seems to only be able to use the basic search engine, and not the advanced one any more. Not sure if this is fixable, or if they blocked access to it. The scraper just sees a message as if the user wasn't logged in, even though a logged-in session is given.

giphy image source is broken

Proposed solution is remove the giphy source.

Failing test below:

 $ coverage run -m pytest tests/test_giphy.py; coverage html
=============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.6.8, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /Users/korymath/Documents/code/talk-generator, configfile: pytest.ini
plugins: cov-2.10.1
collected 1 item

tests/test_giphy.py::GiphyTest::test_giphy_simple
---------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------
2020-08-30 16:34:31 DEBUG Starting new HTTP connection (1): api.giphy.com:80
2020-08-30 16:34:32 DEBUG http://api.giphy.com:80 "GET /v1/gifs/random?api_key=dc6zaTOxFJmzC&tag=cat HTTP/1.1" 429 None
2020-08-30 16:34:32 INFO Image: None
FAILED                                                                                                                                                                      [100%]

==================================================================================== FAILURES =====================================================================================
___________________________________________________________________________ GiphyTest.test_giphy_simple ___________________________________________________________________________

self = <tests.test_giphy.GiphyTest testMethod=test_giphy_simple>

    def test_giphy_simple(self):
        image = giphy.get_related_giphy("cat")
        logging.info('Image: {}'.format(image))
>       self.assertTrue(len(image.get_image_url()) > 0)
E       AttributeError: 'NoneType' object has no attribute 'get_image_url'

tests/test_giphy.py:11: AttributeError
-------------------------------------------------------------------------------- Captured log call --------------------------------------------------------------------------------
2020-08-30 16:34:31 DEBUG Starting new HTTP connection (1): api.giphy.com:80
2020-08-30 16:34:32 DEBUG http://api.giphy.com:80 "GET /v1/gifs/random?api_key=dc6zaTOxFJmzC&tag=cat HTTP/1.1" 429 None
2020-08-30 16:34:32 INFO Image: None
============================================================================= short test summary info =============================================================================
FAILED tests/test_giphy.py::GiphyTest::test_giphy_simple - AttributeError: 'NoneType' object has no attribute 'get_image_url'

PEP8 and YAPF Google Python Code Formatting

Work to format the code according to PEP8 and YAPF 'Google' format.

Add support for in-browser slide creation

For instance, building a presentation in markup and then building the presentation using a javascript library like remark.js (https://github.com/gnab/remark) would allow for generation and presentation in the browser.