bengreenier / overlayed

stream overlays that broadcasters can see 🎬🔴

overlayed 🎬🔴

Overlayed is a tool that enables streamers to see their overlays atop their desktop. I built it to interact more seamlessly with my viewers. If you want to interact in faster, more interesting ways it may be built for you too. ❤️

Table of contents

• Installation
• Configuration
  • Screen Layout
  • Built-in Overlays
  • Custom Overlays
• Contributing
  • Core
  • Building an Overlay
• License

Installation

Currently, This project is in Alpha status. As such, the only way to install is to clone this repository, and Build it.

Configuration

Note: Most folks will use our shiny new editor ✨

To open the settings editor, click the Overlayed icon in your taskbar, and choose "Settings".

Technical details

Overlayed uses electron-settings under the hood, which saves settings to different locations based on your operating system.

Windows	Mac	Linux
%APPDATA%/overlayed/Settings	~/Library/Application\ Support/Overlayed/Settings	$XDG_CONFIG_HOME/Overlayed/Settings
		or
		~/.config/Overlayed/Settings

Learn more in the electron-settings FAQ.

Screen Layout

Overlayed can be configured to overlay above a portion of your screen, which can be great if you don't share your entire display when streaming.

To configure the screen layout, modify the following settings:

{
  "overlayed": {
    "window": {
      "x": 780,
      "y": 0,
      "width": 1920,
      "height": 1080
    }
  }
}

• x - the left-most side of the overlay region
• y - the top-most side of the overlay region
• width - the total width of the overlay region
• height - the total height of the overlay region

Built-In Overlays

Overlayed ships with some built-in overlays. These live in this directory and are now configurable! You can find more information below.

Custom Overlays

Overlayed supports loading custom plugins from your home directory. To add custom overlays, create a folder in your home directory named .overlayed and a folder for each plugin inside that parent folder. For example:

home/
  .overlayed/
    TestPlugin/
      package.json
      TestPlugin.js
    OtherPlugin/
      package.json
      OtherPlugin.js

All plugins, including custom plugins, are now configurable by adding an object to the configuration file containing the plugin config, indexed by the plugin name. For instance: { "clock": { "locale": "en-US" } }

Contributing

This project loves new contributors, but it is maintained as a side project. If you feel you aren't getting traction, please be patient!

Core

To contribute to the core of the project (namely, this repository) you'll need the following things:

• Git
• Node

Then clone this repository and execute the following commands from the directory in which you cloned into (likely overlayed):

npm install
npm run build
npm start

These will (in order): install dependencies, build the project, start the application. Please note that this project uses typescript to build files into the dist/ folder. As such, the dist/ folder should not be modified by engineers, nor should it be included in PRs.

Building an Overlay

A more detailed walkthrough is coming soon! ✨

• Create a new directory .overlayed in your home directory
• Create a new directory under .overlayed for your overlay, like MyOverlay
• Run npm init or manually create a package.json file
• Ensure package.json contains name, version, and main
• Ensure main points to a .js file that has a default export (or module.exports.default = YourClass)
• Profit! Overlayed will attempt to load your plugin on start

For example:

TestOverlay.js:

export default class TestOverlay extends React.Component {
  render() {
    return <h1>Hello World</h1>
  }
}

package.json:

{
  "name": "test-overlay",
  "version": "0.0.1",
  "main": "TestOverlay.js"
}

License

MIT

overlayed's Issues

label plugins that are on screen

We should have labels for the visible plugins so that the broadcaster can always easily see what they look like

pixel perfect layout

Today, all plugins are constrained to a grid. It would be good to have a pixel perfect mode that could enable folks to absolutely position plugins. If this is a mode that can be toggled on for individual plugins (which would probably be ideal) then we could give it a hotkey so that when holding the key and moving a plugin you simply get a pixel grid instead of the default grid.

z indexing plugins

We need to design an explicit solution for broadcasters to chose how their loaded plugins render (by z index) so that you can stack plugins as needed. Perhaps this is advocating for layers.

For instance, imagine a webcam plugin, and some webcam chrome. Broadcasters would expect to be able to layer this content, as they can in OBS today.

broadcaster interactive plugins

Note: requires design proposal and review before implementation

This tracks conversation about how we should support plugins that broadcasters can interact with. This poses an interesting interaction that is not consistent with the primary goals of overlayed - that is, for interactive plugins the broadcaster should be able to click on them. That means that we need a window host that can receive focus and therefore key and mouse events.

I think the implication here is that we must switch window modes often, or maintain yet another electron window.

plugins should be able to update their own settings

Fairly self-explanatory - plugins should be able to write data to their own settings in the event that the developer for an overlay wants to do some runtime modification or potentially even an OOBE type thing (though I want to provide a more centralized way of doing OOBE for plugins eventually).

support multi-machine in single configuration

When we're designing the serialization system (see #3) we should provide a way to index settings by machine name, or similar. A preliminary design that comes to mind is something like:

"benMachine1": { /* per-machine settings, existing schema */ },
"benMachine2": { /* ... */},
...

detailed walkthrough for building an overlay

The readme has a brief walkthrough for creating an overlay, but a more detailed and developer friendly version should be created.

Please upvote if this is a priority for you!

readme out of date

Oops! Our main readme is missing some important updates! 🙈

We should document that built-in plugins are, in fact, configurable! All plugins are now configurable by adding an object to the configuration file containing the plugin config, indexed by the plugin name.

For instance:

{
    "clock": {
       "locale": "en-US"
    }
}

We should also clarify that these plugins can be hidden if they have a key hidden set to true. For instance:

{
    "clock": {
       "hidden": true
    }
}

That addresses the issue for built-in plugins, but this approach also works for custom plugins too. We should clarify that as well. 🚀 🎸

plugin permissioning

Today custom overlays (internally called plugins) are not sandboxed nor is there an permission model whatsoever to prevent loading malicious plugins. This is not ideal.

support consistent theming

Note: requires design proposal and review before implementation

we should support theming via some combination of top level css and React Context. The idea is to allow a broadcaster to "skin" all their overlay components, and consume themes created by others.

safely unmount of components on electron shutdown

When electron shuts down, today we don't unmount all the react components - this is a problem when we're trying to do things like smart de-serialization of settings when plugins unmount (because the settings won't get written to disk).

We need to create a solution for that.

hotkey to disable overlay(s)

We should be able to hide individual overlays, as well as the entire overlayed app (everything) via hotkeys!

style configuration editor

The configuration editor needs to have better styling applied - per https://github.com/mozilla-services/react-jsonschema-form it is designed to be used with bootstrap, and we aren't currently pulling that in. The min-bar for completing this is pulling in bootstrap, but we also have an opportunity to style it ourselves for a lighter footprint.

configuration editor

A user should be able to visually edit config

layout serialization

Today, the layout isn't persisted when overlayed shuts down. It should be!

support conditional plugin reload

Today when we update configuration the entire compositor reloads. It would be good to support conditional reloading of the changed plugins only.

Build doesn't work on mac

As per @noopkat's test, the build script for static things (build-static) doesn't work on Mac :( - we should repro and fix that!

toggling dev tools should allow mouse clicks

When one toggles on dev tools via the tray, it should also enable mouse clicks. This could be implemented by simply calling toggleWindowInteractivity along with the dev tools toggle.

how to do electron bundling

How do we want to package and ship the app?

refactor react+es6 method syntax

Per this review comment we can refactor our es6 methods to be "autobound" to the React this instance, cleaning up code by removing the need for this.callback = this.callback.bind(this).

For instance:

class Ex {
    constructor() {
        this.method = this.method.bind(this)
    }

    public method() {
        // do work
    }
}

to

class Ex {
    public method = () => {
        // do work
    }
}

Note: need to see how the linter + typescript feel about this, if they complain it may not be a fit.

webcam capture plugin

It would be great to have a plugin that uses webrtc camera capabilities to render a webcam into a plugin. Our clock plugin can likely be used as a base plugin example. If we're able to get things working relatively quickly, it would be good to also consider device enumeration and selection (likely via a config value) so that if a broadcaster has multiple webcams they can be chosen.

Fails to shutdown correctly on Mac

To reproduce:

• Clone repo at master branch (latest commit: 2a60664)
• npm install npm run build npm start
• Click close in the toolbar icon

installing plugins - async state race condition

This can occur when we are installing plugins but we receive a user request to shutdown - if we unmount the plugingrid before the install finishes, we'll end up trying to call setState on a component (plugingrid) that isn't mounted

direct x injection

Note: this is a second priority feature
Note: requires design proposal and review before implementation

It would be good to inject the chromium render (or a texture of the final composite) into a full screen dx application so that a broadcaster that intends to play fullscreen games (like fortnite, forza, etc) can still leverage overlayed. Likely using some tech like https://github.com/nefarius/Indicium-Supra

window dimensions should start full screen

See https://github.com/bengreenier/overlayed/blob/master/src/index.ts#L17 - today we have hardcoded default values, we should instead use the screen apis to determine the primary screens dimensions, and use those.

configure "edit mode" styling

"Edit Mode" refers to the tray-launched mode that enables to movement of overlays within the overlayed main window. We should provide a way to define the styling that is applied when that mode is active.

1. Plugins should be able to customize their own style in such a case
2. Global/Default styling should be applied
3. Plugin customized styling should be able to be toggled off by the user

how to do electron updates

How do we want to allow the app to update?

built-in extension for loading web content

We should have a plugin that is designed to load web content and display it. Most probably this should look like Muxy and Muxy should be a more specific version of it.

The general idea here is to author a plugin that surfaces some config to drive a https://electronjs.org/docs/api/webview-tag element.

clicking empty overlay space while in edit mode?

If you enter edit mode, and then click some empty window space that you'd expect to let a click through to the underlying window (as it does in normal mode) we should display a warning or message or similar content to "exit edit mode".

document example typescript custom overlay

Today our docs cover how to build a custom overlay using js - but since this project uses typescript it is likely that a contributor may wish to build a custom overlay using ts. We should describe how to do so.

Humongous tray icon on macOS 🐛

When running Overlayed on macOS the icon in the system tray doesn't get scaled properly.

plugin dep install rate should be more configurable

Today custom overlays (plugins by convention) run npm install at a rate of once a day - this means if you're developing an overlay, or have an overlay with loose semver requirements (and a downstream package updates) the only way to get more frequent updates is to delete the lockfile manually.

react component testing

we should have tests for our core react components - likely built using jest.

Allow react in dependencies

Today we explicitly request overlay devs don't include react as a dependency "Ensure you do not include react in your dependencies (it's implicit) - for development, you may choose to include it in devDependencies" which is a bit disingenuous.

We say this so that the overlayed loader doesn't flag your module as install required - which slows down it's load time. However, we should probably just parse deps and if the only one is react ignore it.

per-plugin settings

Today we don't support per-plugin settings! We'll definitely need that.

error handling when custom overlays fail

When an overlay is loaded, and throws some exception or fails to load cleanly we should better handle that error.

support hiding plugins

Currently all plugins that exist are loaded and must exist somewhere on the grid. This is clearly undesirable. 🛩

refactor object spreading logic

per this review comment there is a better way to do object spreading that is being used to extend an existing object with additional properties.

For instance:

const existing = { a: 'ayy' }
const modified = {...existing, b: 'bee' }

as opposed to

const existing = { a: 'ayy' }
const modified = {...existing, ...{ b: 'bee' } }

this should be done across the codebase at once, for consistency.