grayedfox / multidict

Because spell checking should be smart, not difficult.

NOT ACTIVELY MAINTAINED

This project is on hold until web-extensions exposes the native Hunspell implementation (and I have a bit of time an energy!), which will fix the memory consumption problems and make the whole thing a much simpler project. It will continue to live here for now but use with care.

Multidict

Welcome to Multidict, your friendly neighbourhood open source language detecting spellchecker.

Looking for the User Docs? Looking for the JSDOCS?

Downloading

You can download me directly from the Firefox add-ons store here.

Installing

1. Clone the repository using your preferred cloning mechanism (ssh or https)
2. Run npm install from the project base directory

Building

npm run build

This will bundle all required files into the /dist directory.

Serving

npm run serve

This will at first build all the files and then open a Firefox browser with some specific pages that can be used for testing.

Publishing

npm run publish

This will build files using web-ext build and package all source files into dist/sourcecode.zip

Make sure to bump version numbers inside package.json and manifest.json so that previous release zip file is not overwritten.

Updating dictionaries

npm run update-dicts

This will download all the latest available dictionaries from wooorm dictionaries.

Dependencies

The app depends on nspell, a JavaScript implementation of Hunspell, to provide spell checking.

The app depends on dictionaries downloaded from wooorm dictionaries which are themselves a compilation of different dictionaries from different sources. They each have their own license.

multidict's Issues

Refactor the spaghetti code that is the highlight.js file

The highlight.js file should be refactored in such a way that the a highlighter is a class which is instantiated by passing in all necessary properties (i.e. CSS and other style values). The necessary CSS and other style values should be generated with helper functions that do not form part of the core highlighter class.

• highlighter uses ES6 class syntax (init function removed)
• helper functions rewritten and moved into helpers file
• keyup event handler should be split into smaller chunks
• directory structure updated to make the sense

Spell checking only applies in a single tab

Switching tabs after a textarea has been spell checked on a different tab prevents the textarea on the newly switched to tab from being spell checked.

See GIF.

highlights maligned when scrollbar causes autowrap

Can reproduce this on GitHub. Cheek the peek:

In the peek I am pressing enter and then backspace to remove the newline character. Pressing enter aligns the content, deleting with backspace maligns it.

Fix suggestions cycle breaking after first rotation when user sets suggestions to 10+

Update both the developer and editor (user) docs

Documentation should be up to date:

• rewrite GitHub readme to be focused on developers/contributing
• host user friendly docs on github pages (or wiki?)

Add colour picker for custom highlights

Multidict should use wormer dictionaries

Multidict should either bundle or fetch the wormer dictionaries, they can be found here:

https://github.com/wooorm/dictionaries/tree/main/dictionaries

• update credits section in readme
• include licenses for each dictionary

Add Gmail support

As an editor I want spell checking to work inside Gmail since that is where I spend most of my time writing emails.

Background: gmail uses editable divs (instead of a text area) which is technically adding a whole new layer of selection and manipulation logic for handling.

Add custom colour picker for highlights

As an editor I want to be able to choose a custom colour for highlighting so that I have control over how multidict displays misspelt words.

• the editor can choose any RGB colour using a colour picker opened from the browser action/plugin icon settings panel
• the editor gets a preview of what misspelt words will look like when changing/choosing the colour

Add options to "IgnoreTitleCase", "IGNORECAPSWORDS", and "ignoreVariableNames"

As an editor I want to be able to ignore certain word formats (like CAPITLISEDWORDS, TitleCaseWords, and variableNameWords) so that these are not spell checked.

textareas sometimes not Multi spell checked

1. Go to product page for Multidict: https://addons.mozilla.org/en-US/developers/addon/multidict/edit
2. click edit
3. note the textarea and how it still has built in spell checking enabled and no multi dict highlights

Remove all firefox add-on store validation warnings

Not nice seeing the 5 validation warnings, they should be easy enough to remove and will up the chances of getting a feature.

• all unsafe innerHTML assignments inside code replaced with appropriate node.append() (or similar) methods
• new icon sizes added to match those specified in manifest.json

Highlight misaligned when scrolling to bottom of text area and last character is a newline char

Fixed: when highlighted misspelt words remain aligned even when scrolling to bottom of textarea (consistent when last charac is a newline char)
Start: where we add padding to highlight div based on textarea border and padding values

Sidebar doesn't close/open consistently when multiple present

This is partly due to a limitation with the extensions API (in that you can only close the active sidebar and only in response to a user action).

May be fixable in the future.

Remove browserify dependency/bundler

Project currently relies on browserify but it doesn't have to - the native extension way to "require" files is to break them up into content and background isolated logic and specify them in the manifest.json

• would likely need to split the helpers file into smaller parts with dedicated content and background methods
• would eliminate need to build/bundle files and make it easier to develop (can just watch and rebuild extension as normal)
• good time to also restructure and refactor existing code base into distinct, modular parts

Loading Italian dictionary causes OOM error

See: wooorm/nspell#11

Originally thought this only happened when loading 6 or more dictionaries but it's just the Italian one.

Hopefully updating the dictionaries in use will fix this but there is a chance we will need to disable Italian completely for now (if we can't find a dictionary that works)

Fix textarea being resized

The text area is resized when highlight container/divs are added. Noticeable when switching between textareas.

Removing a custom word also removes it from in use dictionaries

Adding an already valid word (i.e. "dark", in English) to custom words and then removing it will create a false positive and highlight the removed word even when using English to spellcheck.

1. Ensure English dictionary being used spell check
2. Add the word "dark" to custom words
3. Remove the word "dark" from custom words

The word "dark" will now be highlighted despite being correct and no longer be present in any dictionary that is in use.

Allow undo after choosing a word suggestion (restoring misspelt word)

As an editor I want to be able to undo changes and restore a misspelt word after choosing a suggestion.

Add unit tests

Getting to the point now where unit tests are sorely needed - code is getting complex enough and small changes that break things are getting harder to pinpoint. Need good unit test coverage, bare minimum all core functions.

• 33% coverage gained
• appropriate unit test framework chosen
• developer readme updated with test running instructions
• package.json updated to run tests with npm run test

Correctly spelt hyphenated words incorrectly marked as misspelt

A hyphenated word that has two correct words on either side of the hyphen should not be marked as misspelt.

Decouple dictionaries and languages from Firefox settings

At the moment adding and managing multiple languages is finicky and annoying and based on the Firefox display language settings.

While use getAcceptLanguages and getUILanguage makes sense for setting default spell checking languages, since we bundle all the dictionaries anyway we should absolutely allow the user to specify and choose the languages they prefer.

• only use getAcceptLanguages and getUILanguage if no saved languages settings exist in browser storage
• use browser storage to save and set enabled spell check languages
• add side panel Languages list which lists all currently in use languages (check boxes) that user can use
• readme updated

Investigate why multidict fails to show any highlights sometimes

Multidict sometimes fails to show any highlights at all, despite being able to communicate with the background script (content script still injected).

It may due to some race condition somewhere, errors include "spelling" (nspell instances) being undefined and sometimes no errors at all.

• log messages inside content script (i.e. after get spelling)
• set DEBUG = true and check timing of all messages received and responded by background script

URLs not properly ignored

Multidict should ignore URLs, which is expected behaviour and mirrors that of Firefox and other browsers.

Reduce memory impact of loading entire dictionaries into memory

Right now dictionaries are not buffered, but are rather read in their entirety before being used to instantiate a User:

Offending helper method that reads dictionaries.

Here the result is used to instantiate a User.

This is a massive resource hit and could likely be done better but I'm not quite sure how. Hopefully we don't need to wait until the web-extensions API exposes the actual browser spell checker (which would remove the need for nspell entirely) but that may be the case.

Fix highlight not clearing selected text on click/arrow press

Focusing a text area should trigger a spell check.

Clicking anywhere inside a textarea should clear the data-multdicit-selected-text attribute.

Pressing space or any arrow key (up, down, left, right) should update the data-multdict-selected-text attribute.

Add CI status/coverage/minzipped/etc buttons to readme

After adding all the CI tasks we should add some pretty buttons to the readme to show off a bit.

Add config option to replace words globally

As an editor I would like to be able to replace all instances of a misspelt word instead of just the instance I am getting suggestions for.

• editor can check a box inside browser action to "replace all instances of misspelt word"
• multidict should only replace words inside single textbox (i.e. different/separate text areas on the same page should be left alone)

WordCarousel occasionally appears in wrong place inside text area

See GIF.

Add customised design/image to Multi Dict

Need to either make or find something in the public domain to use for the template/icon. Consider using same icon currently bundled with repo. Should do this PR as the last one before pushing the MVP since we want to also bundle a good demo and ensure all docs are updated.

• ensure icon looks nice in all locations it is displayed
• add new annotated GIF demo that demonstrates all features

Decouple dictionaries from firefox settings

As a user I would like to be able to select which languages are used for spell checking instead of having to rely on the display accepted display languages set inside the Firefox settings.

• languages can be chosen from side panel
• initial supported languages based on getAcceptLanguages and getUILanguage
• readme updated

Add ability to clear personal dictionary

As an editor I want to be able to view and clear all words from my personal dictionary so that I don't have to remember what words are saved and can easily view all personal words.

• remove the hidden functionatlity that uses 'ClearAll' to clear all words
• add browser action item to clear a word from personal dictionary
• add browser action item to display all saved personal words

Sort dictionary alphabetically and add notifications

Dictionary should be sorted alphabetically and a notification should be shown when a user adds or removes a word from their custom dictionary.

• notification shown on add/remove
• dictionary sorted and saved in alphabetical order
• don't open personal dictionary after adding or removing a word while side panel open

Add personalised, persistent custom dictionary

As an editor, I want to be able to add and remove words from a personal dictionary so words I consider valid are not marked as incorrect.

• needs to persist across browser sessions
• should use sync storage (if available) so that custom words are shared across devices
• fallback to local storage when sync not available
• shortcut should be available for adding and removing words
• context menu should be available for adding and removing words

Update nspell depdendency and automate this task

This pull request...

• updates nspell
• adds nspell as an npm dependency (so it will be updated with npm update)
• refactors existing code to require nspell npm package instead of local nspell instance

Add task runner (github workflows)

Continuous integration task runner should:

• run all unit tests
• run the linter
• genereate JSDOC documentation and update static files on successful merge to master
• run linter/unit tests for each change made inside a pull requests (branches should not run tasks)
• failed linting/unit tests should block merging to master
• ensure unit test coverage goes up or remains the same

Add "report and issue" link to side panel

This pull request adds a report an issue link to the side panel.

Make languages orderable

The language list should be draggable so that preferred spelling is respected (i.e. Australian before American English)

Show word suggestions

As an editor I want to see and be easily able to choose from a list of word suggestions for misspelt words so that I can quickly replace the misspelt word.

Acceptance Criteria

• word suggestions should appear as soon as the caret is at placed at the beginning, end, or anywhere within a word i.e. |banan|anas| (here, caret is shown at beginning, end, and within the word) and user activates suggestion shortcut
• if there are multiple word suggestions, they should appear on a vertical carousel just above the misspelt word
• user should be able to navigate up and down (next and previous) words of the carousel by using designated short cut keys (defaults: next, alt+shift+down; previous, alt+shift+up)
• carousel should contain all word suggestions but show only current, previous, and next words
• carousel should wrap around (scroll inifinite)
• only the top 4 suggestions should be shown in the the carousel (context menu can have more than 4 suggestions)
• 3 or less words should still create a carousel effect, which can also scroll infinite
• user should be able to escape out of word suggestions without choosing one using left or right arrow keys

Remove all logging

Remove all that hideous logging unless in debug mode

Add JSDOC generation task and JSDOC comments to all methods/classes/functions

All methods and helpers should be renamed for clarity (if needed) and have up to date JSDOC comments

• JSDOC comments added to all text-methods
• JSDOC comments added to all helpers
• JSDOC comments added to all classes
• JSDOC documentation should be able to be generated with npm run gen-docs (this should also be automatically done upon merging to master)

highlights maligned when scrolling down in some textarea

Can reproduce on GitHub. Cheek the peek:

I am scrolling up and then down after inserting enough newline characters at the end of the text area to get a vertical scroll bar.

GitHub textarea size too small and comment field has no spell checking applied

Multiple bugs when using MultiDict for GitHub at the moment:

1. comment boxes, despite being textareas, don't have any spellchecking (checked in console and hwt container is never injected)

2. the highlights break CTR+Z shortcut inside text fields (and probably others?)

3. comment box resizes inside main body text area when writing a new issue: