Coder Social home page Coder Social logo

wooorm / refractor Goto Github PK

View Code? Open in Web Editor NEW
668.0 3.0 32.0 1.14 MB

Lightweight, robust, elegant virtual syntax highlighting using Prism

License: MIT License

JavaScript 100.00%
syntax-highlighting virtual-dom vdom react hast virtual syntax highlight html

refractor's Introduction

refractor

Build Coverage Downloads Size

Lightweight, robust, elegant virtual syntax highlighting using Prism.

Contents

What is this?

This package wraps Prism to output objects (ASTs) instead of a string of HTML.

Prism, through refractor, supports 270+ programming languages. Supporting all of them requires a lot of code. That’s why there are three entry points for refractor:

  • lib/core.js — 0 languages
  • lib/common.js (default) — 36 languages
  • lib/all.js — 297 languages

Bundled, minified, and gzipped, those are roughly 12.7 kB, 40 kB, and 211 kB.

When should I use this?

This package is useful when you want to perform syntax highlighting in a place where serialized HTML wouldn’t work or wouldn’t work well. For example, you can use refractor when you want to show code in a CLI by rendering to ANSI sequences, when you’re using virtual DOM frameworks (such as React or Preact) so that diffing can be performant, or when you’re working with ASTs (rehype).

A different package, lowlight, does the same as refractor but uses highlight.js instead. If you’re looking for a really good (but rather heavy) highlighter, try starry-night.

Playground

You can play with refractor on the interactive demo (Replit).

Install

This package is ESM only. In Node.js (version 14.14+, 16.0+), install with npm:

npm install refractor

In Deno with esm.sh:

import {refractor} from 'https://esm.sh/refractor@4'

In browsers with esm.sh:

<script type="module">
  import {refractor} from 'https://esm.sh/refractor@4?bundle'
</script>

Use

import {refractor} from 'refractor'

const tree = refractor.highlight('"use strict";', 'js')

console.log(tree)

Yields:

{
  type: 'root',
  children: [
    {
      type: 'element',
      tagName: 'span',
      properties: {className: ['token', 'string']},
      children: [{type: 'text', value: '"use strict"'}]
    },
    {
      type: 'element',
      tagName: 'span',
      properties: {className: ['token', 'punctuation']},
      children: [{type: 'text', value: ';'}]
    }
  ]
}

API

This package exports the identifier refractor. There is no default export.

refractor.highlight(value, language)

Highlight value (code) as language (programming language).

Parameters
  • value (string) — code to highlight
  • language (string or Grammar) — programming language name, alias, or grammar.
Returns

Node representing highlighted code (Root).

Example
import {refractor} from 'refractor/lib/core.js'
import css from 'refractor/lang/css.js'

refractor.register(css)
console.log(refractor.highlight('em { color: red }', 'css'))

Yields:

{
  type: 'root',
  children: [
    {type: 'element', tagName: 'span', properties: [Object], children: [Array]},
    {type: 'text', value: ' '},
    // …
    {type: 'text', value: ' red '},
    {type: 'element', tagName: 'span', properties: [Object], children: [Array]}
  ]
}

refractor.register(syntax)

Register a syntax.

Parameters
  • syntax (Function) — language function custom made for refractor, as in, the files in refractor/lang/*.js
Example
import {refractor} from 'refractor/lib/core.js'
import markdown from 'refractor/lang/markdown.js'

refractor.register(markdown)

console.log(refractor.highlight('*Emphasis*', 'markdown'))

Yields:

{
  type: 'root',
  children: [
    {type: 'element', tagName: 'span', properties: [Object], children: [Array]}
  ]
}

refractor.alias(name[, alias])

Register aliases for already registered languages.

Signatures
  • alias(name, alias|list)
  • alias(aliases)
Parameters
  • language (string) — programming language name
  • alias (string) — new aliases for the programming language
  • list (Array<string>) — list of aliases
  • aliases (Record<language, alias|list>) — map of languages to aliases or lists
Example
import {refractor} from 'refractor/lib/core.js'
import markdown from 'refractor/lang/markdown.js'

refractor.register(markdown)

// refractor.highlight('*Emphasis*', 'mdown')
// ^ would throw: Error: Unknown language: `mdown` is not registered

refractor.alias({markdown: ['mdown', 'mkdn', 'mdwn', 'ron']})
refractor.highlight('*Emphasis*', 'mdown')
// ^ Works!

refractor.registered(aliasOrlanguage)

Check whether an alias or language is registered.

Parameters
  • aliasOrlanguage (string) — programming language name or alias
Example
import {refractor} from 'refractor/lib/core.js'
import markdown from 'refractor/lang/markdown.js'

console.log(refractor.registered('markdown')) //=> false

refractor.register(markdown)

console.log(refractor.registered('markdown')) //=> true

refractor.listLanguages()

List all registered languages (names and aliases).

Returns

Array<string>.

Example
import {refractor} from 'refractor/lib/core.js'
import markdown from 'refractor/lang/markdown.js'

console.log(refractor.listLanguages()) //=> []

refractor.register(markdown)

console.log(refractor.listLanguages())

Yields:

[
  'markup', // Note that `markup` (a lot of xml based languages) is a dep of markdown.
  'html',
  // …
  'markdown',
  'md'
]

Examples

Example: serializing hast as html

hast trees as returned by refractor can be serialized with hast-util-to-html:

import {refractor} from 'refractor'
import {toHtml} from 'hast-util-to-html'

const tree = refractor.highlight('"use strict";', 'js')

console.log(toHtml(tree))

Yields:

<span class="token string">"use strict"</span><span class="token punctuation">;</span>

Example: turning hast into react nodes

hast trees as returned by refractor can be turned into React (or Preact) with hast-to-hyperscript:

import {refractor} from 'refractor'
import {toH} from 'hast-to-hyperscript'
import React from 'react'

const tree = refractor.highlight('"use strict";', 'js')
const react = toH(React.createElement, tree)

console.log(react)

Yields:

{
  '$$typeof': Symbol(react.element),
  type: 'div',
  key: 'h-1',
  ref: null,
  props: { children: [ [Object], [Object] ] },
  _owner: null,
  _store: {}
}

Types

This package is fully typed with TypeScript. It exports the additional types Root, Grammar, and Syntax.

Data

If you’re using refractor/lib/core.js, no syntaxes are included. Checked syntaxes are included if you import refractor (or explicitly refractor/lib/common.js). Unchecked syntaxes are available through refractor/lib/all.js. You can import core or common and manually add more languages as you please.

Prism operates as a singleton: once you register a language in one place, it’ll be available everywhere.

Only these custom built syntaxes will work with refractor because Prism’s own syntaxes are made to work with global variables and are not importable.

CSS

refractor does not inject CSS for the syntax highlighted code (because well, refractor doesn’t have to be turned into HTML and might not run in a browser!). If you are in a browser, you can use any Prism theme. For example, to get Prism Dark from cdnjs:

<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/prism/1.27.0/themes/prism-dark.min.css">

Compatibility

This package is at least compatible with all maintained versions of Node.js. As of now, that is Node.js 14.14+ and 16.0+. It also works in Deno and modern browsers.

Only the custom built syntaxes in refractor/lang/*.js will work with refractor as Prism’s own syntaxes are made to work with global variables and are not importable.

refractor also does not support Prism plugins, due to the same limitations, and that they almost exclusively deal with the DOM.

Security

This package is safe.

Related

Projects

Contribute

Yes please! See How to Contribute to Open Source.

License

MIT © Titus Wormer

refractor's People

Contributors

chenxsan avatar conorhastings avatar danistefanovic avatar karlhorky avatar mearns avatar mtt87 avatar otakustay avatar peterwilliams avatar rmadsen avatar tamagokun avatar thompsongl avatar wooorm avatar xiaoxiangmoe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

refractor's Issues

sideEffects: false incorrect

all.js and common.js have side effects that get deleted when tree-shaking because of this option in package.json

Regex delimiters from `js` not included in `jsx`?

Hi @wooorm , sorry for all the issues lately!

Working on link and highlighting tooling a bit right now...

Here's something that I found interesting: the js language returns some regex delimiter tokens, but the jsx language does not... should it be like that? I thought those extra tokens came from js-extras, maybe this is not included with jsx or typescript or tsx?

Ran the following on Runkit:

var refractor = require("refractor")
console.log(JSON.stringify(refractor.highlight('/\bno\-highlight\b/', 'js'), null, 2))
console.log(JSON.stringify(refractor.highlight('/\bno\-highlight\b/', 'jsx'), null, 2))

...and received the following:

JS

[
  {
    "type": "element",
    "tagName": "span",
    "properties": {
      "className": [
        "token",
        "regex"
      ]
    },
    "children": [
      {
        "type": "element",
        "tagName": "span",
        "properties": {
          "className": [
            "token",
            "regex-delimiter"
          ]
        },
        "children": [
          {
            "type": "text",
            "value": "/"
          }
        ]
      },
      {
        "type": "element",
        "tagName": "span",
        "properties": {
          "className": [
            "token",
            "language-regex"
          ]
        },
        "children": [
          {
            "type": "text",
            "value": "\bno-highlight\b"
          }
        ]
      },
      {
        "type": "element",
        "tagName": "span",
        "properties": {
          "className": [
            "token",
            "regex-delimiter"
          ]
        },
        "children": [
          {
            "type": "text",
            "value": "/"
          }
        ]
      }
    ]
  }
]

JSX

[
  {
    "type": "element",
    "tagName": "span",
    "properties": {
      "className": [
        "token",
        "regex"
      ]
    },
    "children": [
      {
        "type": "text",
        "value": "/\bno-highlight\b/"
      }
    ]
  }
]

TypeError: Cannot read property 'buildPlaceholders' of undefined

Just started using Refractor in a new project and it's been working really well. We have implemented it as follows to load languages dynamically:

const matches = codeBlockElement.match('language-([a-z]*)');
const language = matches && matches[1];
const code = codeElement.innerText;

let refractorNode;

/**
 * Try to load the lang, register it, and render the code block with Refractor.
 * If the lang is not recognized, just do nothing and we'll render it manually below.
 * */
try {
  // eslint-disable-next-line import/no-dynamic-require
  const requiredLanguage = require(`refractor/lang/${language}`);

  if (requiredLanguage && !Refractor.hasLanguage(language)) {
    Refractor.registerLanguage(requiredLanguage);
  }

  refractorNode = <Refractor language={language} value={code} />;
} catch {
  // Do nothing
}

return (
  {refractorNode || (
    <pre>
      <code className={`${language} language-${language || 'unknown'}`}>{code}</code>
    </pre>
  )}
);

Assume here that codeBlockElement is a DOM <code> element. This code executes after rendering markdown to HTML, in between when the HTML is injected into the virtual DOM (using Interweave)

So far so good... until we try and render an ERB block (indenting so you can see the backticks):

    ```erb
    <%= @foo %>
    ```

With that markdown, the language variable above gets set to 'erb' and the code variable gets set to '<%= @foo %>'. The whole block above actually executes without error but then this error is thrown after:

  15 | })
  16 | Prism.hooks.add('before-tokenize', function(env) {
  17 |   var erbPattern = /<%=?[\s\S]+?%>/g
> 18 |   Prism.languages['markup-templating'].buildPlaceholders(
  19 | ^    env,
  20 |     'erb',
  21 |     erbPattern

I looked at #9, which seems to be related. I will try to add a failing spec in the meantime.

Thanks for your help in advance.

[email protected]

Refractor locks prismjs to patch versions only

Prismjs just released 1.16.0, To avoid pulling in duplicate versions of prismjs into applications could we either upgrade to the latest minor version or unlock the dependency?

For now I'll lock my other prismjs dependency to patch only.

Thanks :)

Error: Unexpected token in `character-entities-legacy` when using this module

I'm building a React library using create-react-library, and I'm getting an error that tracks back to refractor when trying to install a package (react-syntax-highlighter) that depends on refractor.

My error looks like this:

rollup v0.64.1
bundles src/index.js → dist/index.js, dist/index.es.js...
[!] Error: Unexpected token
node_modules/character-entities-legacy/index.json (2:9)
1: {
2:   "AElig": "Æ",
            ^
3:   "AMP": "&",
4:   "Aacute": "Á",

I've tried uninstalling, deleting node_modules and reinstalling the package twice with no success.

I've consulted all of my usual resources to resolve this without any success. Am I missing something here?

Dependencies between languages

Prism's languages may depend on each other, for example, the vbnet requires basic to register before it, smarty, erb, php and soy require markup-templating as their base language

Currently, refractor doesn't handle the dependencies for each language, so code below will fail:

const refractor = require('refractor/core');
refractor.register(require('refractor/lang/vbnet'));

Will it be solved if vbnet's source code requries basic and register it itself?

[email protected]

Hi,

Prism just released a new version that contains some important fixes with regards to exponentially backtracking patterns.

Is it possible to bump to this minor version in refractor so packages like react-syntax-highlighter aren't affected by these issues?

Thanks in advance!

Weird behavior on 3.x, and not on 4.x

On trying to use @mapbox/rehype-prism with svelte, I came along with the issue of rendering bunch of [object Object] literals printed.

renders-strangely
(The @html directive let's your render html without escaping any angle brackets)

It even happens without applying imported rehypePrism plugin to the processor. Just import rehypePrism from '@mapbox/rehype-prism' somehow "stringifies" the objects. Commenting out the import statement as well shows the original content as expected.

renders-correctly
(Nothing special, just some C++ code working with OpenCV)

Digging deeper, I found that

  • @mapbox/rehype-prism depends on refractor^3.4.0 (currently 3.6.0 installed)
  • It happens during the register phase in refractor/index.js, thus happens by "just importing it".
    • By binary search, I have nailed down to three languages that cause the problem:
      refractor.register(require('./lang/arduino.js'))
      refractor.register(require('./lang/chaiscript.js'))
      refractor.register(require('./lang/cpp.js'))
      Commenting out all of them shows the correct result. Leaving any of them uncommented shows the stringified result.
  • refractor^4 seems to work fine (tested with 4.5.0)

The languages arduino and chaiscript registers and extends refractorCpp, so my bet is on that one (these are the only two langauges that requires cpp.js). However the file contains mostly parameters, so I'm stuck.

I would've start looking into the prism code or svelte, but found out for some reason it works okay with refractor^4. I wonder if there is an explanation here -- and hopefully don't have to start digging again.
I can't find much information about the difference between 3.x and 4.x, except for the 4.0.0 release note. My guess is that using ESM imports and not somehow interferes with the global scope, but cannot figure out what the actual difference is.

The project seems to keep both major versions, so I'm confused.

  • Are there explicit difference policies between the two versions?
  • Would it be rehype-prism library that should be fixed (tell them to use 4.x instead), or the 3.x branch?
  • Or maybe there both fine, and somethings wrong with the framework (svelte)?

Allow to pass Prism instance

Hi

We use this library to add language support to a custom version of Prism.

Right now we're using refractor/lang/<lang> methods directly but that does't work for languages that use .register method defined on Refractor instance.

It'd be great if there was a way to pass Prism instance so .register works.

Unless this is already possible, I can work on this, if you are open to merging in something like this.

Incorrect global in jsdom environment

After #37 was merged I appear to have some issues where prismjs thinks the global is one thing, while refractor thinks it's a different thing. This results in Prism.manual not being set, which makes prism attach events to the DOMReady events and such.

I'm not quite sure what the right solution is - I'll see if I can come up with a reproducible test case.

Release a `3.x` backport of updating to `prismjs@~1.25.0`?

Would you be willing to release a 3.x backport updating to prismjs@~1.25.0? 🙏🏻
(or better yet prismjs@^1.25.0 so this won't be required in case of future vulnerabilities ✨ )

We:

  • depend on [email protected] (@latest)
    • which depends on refractor@^3.2.0
      • which depends on prismjs@~1.24.0

Normally, I would have asked the maintainers of react-syntax-highlighter to update their version of refractor instead, but since your 4.x line introduced the breaking change of using ESM, I'm not so sure that they can do so without some major effort involved. 😬

Thanks for your consideration! 🙇🏻‍♂️

Allow Prism token name mappings to allow custom classNames

Hi there!

Wondering what you think about this: react-syntax-highlighter/react-syntax-highlighter#372

Essentially we're struggling the fact react-syntax-highlighter when using classnames can suffer from style leakage into code blocks. For example.

<style>.punctuation { color: 'emerald' }</style>
<span class="token punctuation">.</span>

We're trying to make this less unsafe to use in production, and noted a Prism plugin that does support remapping classnames. https://prismjs.com/plugins/custom-class/. This would nice to use, although there may be other approaches.

Do you have any thought on slightly opening/tweaking the refractor API to support className mapping?

Highlighting markup code (XML, HTML, etc) containing character entities throws an Error

Trying to highlight the raw code string <foo>&lt;</foo> will cause an error:

TypeError: env.content.replace is not a function

Here is a sandbox example showing the error does not occur with Prism itself but does with refractor:

https://codesandbox.io/s/wnx2rlpyww

It appears the culprit is the "wrap" hook called here. Prism's language definition files expect the content attribute of the env value passed to the hook to be a string, but refractor builds and passes a HAST object instead.

I think this could be solved by patching the language files shipped with refractor to work with a HAST object if possible (and skip the hook if it can't). For this particular bug, I think this should suffice, because the hook only acts when env.type === 'entity', and I believe the content objects for those are always of the form: {type: 'text', value: value} where value is a string.

However, if other plug-ins or future language definitions use the "wrap" hook differently, they'd have to be individually updated as well, and might need to support operations on larger or higher-level structures than text nodes (which refractor probably would not be able to convert from HAST to a string and back to HAST).

I can open a pull request to patch the groovy, markup and asciidoc language files if that sounds like a reasonable solution to you?

Can't resolve 'hastscript'

./node_modules/refractor/core.js
Module not found: Can't resolve 'hastscript' in ...

The hastscript package needs to be moved from devDependencies to dependencies in the package.json file

Markdown rendering strips tags from HTML code blocks

I've run into a problem using refractor 4.8.0 where when a Markdown code block is specified to be html, any tags inside that block get stripped out completely.

A minimal example that reproduces the issue is this snippet of Markdown:

```html
<h1>Some HTML</h1>
```

I set up a Replit example that shows the result of rendering this looks like:

<span class="token code"><span class="token punctuation">```</span><span class="token code-language">html</span>
<span class="token code-block language-html">Some HTML</span>
<span class="token punctuation">```</span></span>

As you can see, the H1 tags are completely gone.

It also only does this if you put the html language on the code block. Without it, the tags aren't stripped out.

Am I using the library wrong? It seems that if I replace the < and > with &lt; and &gt;, that fixes the issue, but if I do that on a full Markdown file, any inlined HTML tags will not be highlighted. It also doesn't seem like that sort of escaping is needed for other file types (e.g. JavaScript).

It looks like this is being caused because of textContent in markdown.js, but I don't know that changing that is the right solution here (or I'd open a PR 😀).

Let me know if you need any more information, or if I can help. Thanks!

Grouping by line

I've been considering how to add line highlights similar to https://github.com/gatsbyjs/gatsby/tree/master/packages/gatsby-remark-prismjs . This is what I came up with:

const refractor = require("refractor")
const rehype = require('rehype')

const code = `
(function(win) {
    console.log(win);
))(window || global);
`;

const highlightLines = [3];

const nodes = code.split('\n',)
  .map((line, index) => {
    return {
      type: 'element',
      tagName: 'span',
      properties: { className: ['line', highlightLines.includes(index) ? 'highlight' : ''] },
      children: refractor.highlight(line, 'js')
    };
  });

const html = rehype()
  .stringify({type: 'root', children: nodes})
  .toString();

Do you think this is something we could add to refractor itself?

Switching prismjs dependency from ~ to ^

First, thanks so much for refractor.

Is there a reason why prismjs is the only dependency that uses ~ vs ^?

As a result, I believe one cannot update to 1.25.0 on legacy version.

Thanks for helping out!

JSX or TSX indent

Problem

If I use it, I try to highlight js or ts, the components use a correct indentation, but when inserting jsx or tsx as a language, it happens that it is not indented correctly.

Is there a way to get the javascript/typescript indent and the jsx/tsx highlight?

jsx/tsx

image

javascript/typescript

image

code

import { refractor } from 'refractor';
import { toH } from 'hast-to-hyperscript';
import ts from 'refractor/lang/typescript';
import tsx from 'refractor/lang/tsx';
import js from 'refractor/lang/javascript';
import jsx from 'refractor/lang/jsx';
import bash from 'refractor/lang/bash';
import css from 'refractor/lang/css';
import diff from 'refractor/lang/diff';

refractor.register(ts);
refractor.register(tsx);
refractor.register(js);
refractor.register(jsx);
refractor.register(bash);
refractor.register(css);
refractor.register(diff);

const BlockCode = ({ codeText, language = 'jsx'}) => {

  const codeOut = refractor.highlight(codeText ?? '', language);

  return (
    <pre className={`language-${language}`}>
      <code className={`language-${language}`}>
        {toH(React.createElement, codeOut)}
      </code>
    </pre>
  );
};

css prims theme

pre[class*='language-'],
  code[class*='language-'] {
    background: rgb(54, 52, 73);
    color: #fcfcfa;
    text-shadow: 0 1px rgb(54, 52, 73);
    font-family: 'Fira Mono', Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono','Courier New', monospace;
    direction: ltr;
    text-align: left;
    white-space: pre;
    word-spacing: normal;
    word-break: normal;
    line-height: 1.5;
    -moz-tab-size: 4;
    -o-tab-size: 4;
    tab-size: 4;
    -webkit-hyphens: none;
    -moz-hyphens: none;
    -ms-hyphens: none;
    hyphens: none;
  }

  /* Selection */

  code[class*='language-']::-moz-selection,
  code[class*='language-'] *::-moz-selection,
  pre[class*='language-'] *::-moz-selection {
    background: hsl(220, 13%, 28%);
    color: inherit;
    text-shadow: none;
  }

  code[class*='language-']::selection,
  code[class*='language-'] *::selection,
  pre[class*='language-'] *::selection {
    background: hsl(221, 9%, 35%);
    color: inherit;
    text-shadow: none;
  }

  /* Code blocks */

  pre[class*='language-'] {
    padding: 1em;
    margin: 0.5em 0;
    overflow: auto;
    border-radius: 14px;

    code div {
      font-size: 16px;
      line-height: 24px;
      font-weight: 500;
      font-family: 'Fira Mono', Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono','Courier New', monospace;
    }
  }

  /* Inline code */

  :not(pre) > code[class*='language-'] {
    padding: 0.2em 0.3em;
    border-radius: 0.3em;
    white-space: normal;
  }

  /*********************************************************
  * Tokens
  */

  .namespace {
    opacity: 0.7;
  }

  .token.comment,
  .token.prolog,
  .token.doctype,
  .token.cdata {
    color: #6a9955;
  }

  .token.punctuation {
    color: #d4d4d4;
  }

  .token.property,
  .token.tag,
  .token.boolean,
  .token.number,
  .token.constant,
  .token.symbol,
  .token.deleted {
    color: #b5cea8;
  }

  .token.selector,
  .token.attr-name,
  .token.string,
  .token.char,
  .token.inserted {
    color: #ce9178;
  }

  .token.operator,
  .token.entity,
  .token.url,
  .language-css .token.string,
  .style .token.string {
    color: #d4d4d4;
    background: rgb(54, 52, 73);
  }

  .token.atrule,
  .token.attr-value,
  .token.keyword {
    color: #ff6188 !important;
  }

  .token.function {
    color: #dcdcaa;
  }

  .token.regex,
  .token.important,
  .token.variable {
    color: #d16969;
  }

  .token.important,
  .token.bold {
    font-weight: bold;
  }

  .token.italic {
    font-style: italic;
  }

  .token.constant {
    color: #9cdcfe;
  }

  .token.class-name,
  .token.builtin {
    color: #4ec9b0;
  }

  .token.parameter {
    color: #9cdcfe;
  }

  .token.interpolation {
    color: #9cdcfe;
  }

  .token.punctuation.interpolation-punctuation {
    color: #569cd6;
  }

  .token.boolean {
    color: #569cd6;
  }

  .token.property {
    color: #9cdcfe;
  }

  .token.selector {
    color: #d7ba7d;
  }

  .token.tag {
    color: #569cd6;
  }

  .token.attr-name {
    color: #9cdcfe;
  }

  .token.attr-value {
    color: #ce9178;
  }

  .token.entity {
    color: #4ec9b0;
    cursor: unset;
  }

  .token.namespace {
    color: #4ec9b0;
  }

  /*********************************************************
  * Language Specific
  */

  pre[class*='language-javascript'],
  code[class*='language-javascript'] {
    color: #4ec9b0;
  }

  pre[class*='language-css'],
  code[class*='language-css'] {
    color: #ce9178;
  }

  pre[class*='language-html'],
  code[class*='language-html'] {
    color: #d4d4d4;
  }

  .language-html .token.punctuation {
    color: #808080;
  }

  /*********************************************************
  * Line highlighting
  */

  pre[data-line] {
    position: relative;
  }

  pre[class*='language-'] > code[class*='language-'] {
    position: relative;
    z-index: 1;
  }

🐛 Refractor doesn't seem to be imported in Production

Node -v 14.17.4
React -v 17

I am using @craco/craco to configure webpack, so after I have generated an app with npm run build and visiting the build app with serve -s build shows an empty page. On the console it says Error: Unknown language: html is not registered, however I have imported the refractor as import { refractor } from "refractor/lib/common.js" and also tried in this way import { refractor } from "refractor";, both ways does not seem to work.

import { useEffect, useState } from "react";
import { refractor } from "refractor";
import { toHtml } from "hast-util-to-html";

export default function Code() {
  const [code, setCode] = useState(``);

  useEffect(() => {
    try {
      setCode(
        toHtml(refractor.highlight("<h1>Assalamu Alaykum!</h1>", "html"))
      );
    } catch (error) {
      console.log(error);
    }
  }, []);

  return (
    <div className="w-full relative group mb-7">
      <pre className="language-css text-xl | relative">
        <div
          dangerouslySetInnerHTML={{
            __html: code,
          }}
        ></div>
      </pre>
    </div>
  );
}

Scripts in package.json:

"scripts": {
    "start": "TAILWIND_MODE=watch craco start",
    "build": "CI=false craco build",
    "test": "craco test",
    "eject": "react-scripts eject"
}

How to use Prism’s `diff` plugin?

Thank you for implement great library!

I use this library within @mapbox/rehype-prism.

I want to multiple syntax highlight like as diff-typescript or typescript-diff, but it is unavailable. the log is following message.

Error: Unknown language: `diff-typescript` is not registered

In @mapbox/rehype-prism, run following line with diff-typescript.

      result = refractor.highlight(nodeToString(node), lang);

Is there a way to solve this issue like piping multiple syntax (difftypescript)?

Thanks!

Incompatibility with Prism.js

Hey @wooorm ! 👋 Long time no speak! Hope you are well.

Input:

export default function Home() {
  return (
    <div>
      <FilterInput
        options={genreList} // genreList from assets directory
        value={genreFilter} // genreFilter from useState()
        filterSetter={setGenreFilter} // setGenreFilter from useState()
        name="genre"
      />
    </div>
  );
}

Output (refractor):

Demo (try the Run button at the top): https://replit.com/@karlhorky/FrequentWickedDemos#index.mjs

{
  "type": "root",
  "children": [
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "keyword"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "export"
        }
      ]
    },
    {
      "type": "text",
      "value": " "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "keyword"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "default"
        }
      ]
    },
    {
      "type": "text",
      "value": " "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "keyword"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "function"
        }
      ]
    },
    {
      "type": "text",
      "value": " "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "function"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "Home"
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "("
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": ")"
        }
      ]
    },
    {
      "type": "text",
      "value": " "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "{"
        }
      ]
    },
    {
      "type": "text",
      "value": "\n  "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "keyword"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "return"
        }
      ]
    },
    {
      "type": "text",
      "value": " "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "("
        }
      ]
    },
    {
      "type": "text",
      "value": "\n    "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "tag"
        ]
      },
      "children": [
        {
          "type": "element",
          "tagName": "span",
          "properties": {
            "className": [
              "token",
              "tag"
            ]
          },
          "children": [
            {
              "type": "element",
              "tagName": "span",
              "properties": {
                "className": [
                  "token",
                  "punctuation"
                ]
              },
              "children": [
                {
                  "type": "text",
                  "value": "<"
                }
              ]
            },
            {
              "type": "text",
              "value": "div"
            }
          ]
        },
        {
          "type": "element",
          "tagName": "span",
          "properties": {
            "className": [
              "token",
              "punctuation"
            ]
          },
          "children": [
            {
              "type": "text",
              "value": ">"
            }
          ]
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "plain-text"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "\n      <FilterInput\n        options="
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "{"
        }
      ]
    },
    {
      "type": "text",
      "value": "genreList"
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "}"
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "plain-text"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": " // genreList from assets directory\n        value="
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "{"
        }
      ]
    },
    {
      "type": "text",
      "value": "genreFilter"
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "}"
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "plain-text"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": " // genreFilter from useState()\n        filterSetter="
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "{"
        }
      ]
    },
    {
      "type": "text",
      "value": "setGenreFilter"
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "}"
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "plain-text"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": " // setGenreFilter from useState()\n        name=\"genre\"\n      />\n    "
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "tag"
        ]
      },
      "children": [
        {
          "type": "element",
          "tagName": "span",
          "properties": {
            "className": [
              "token",
              "tag"
            ]
          },
          "children": [
            {
              "type": "element",
              "tagName": "span",
              "properties": {
                "className": [
                  "token",
                  "punctuation"
                ]
              },
              "children": [
                {
                  "type": "text",
                  "value": "</"
                }
              ]
            },
            {
              "type": "text",
              "value": "div"
            }
          ]
        },
        {
          "type": "element",
          "tagName": "span",
          "properties": {
            "className": [
              "token",
              "punctuation"
            ]
          },
          "children": [
            {
              "type": "text",
              "value": ">"
            }
          ]
        }
      ]
    },
    {
      "type": "text",
      "value": "\n  "
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": ")"
        }
      ]
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": ";"
        }
      ]
    },
    {
      "type": "text",
      "value": "\n"
    },
    {
      "type": "element",
      "tagName": "span",
      "properties": {
        "className": [
          "token",
          "punctuation"
        ]
      },
      "children": [
        {
          "type": "text",
          "value": "}"
        }
      ]
    }
  ]
}

I'm assuming this leads to this output, which I can observe with react-refractor:

Screen Shot 2021-06-22 at 19 14 28


Testing on https://prismjs.com/test.html#language=jsx, it looks like:

Screen Shot 2021-06-22 at 19 15 07

Can I remove `"type": "module"` from package.json ?

to fix:
Error [ERR_REQUIRE_ESM]: Must use import to load ES Module

Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/XXX/node_modules/refractor/package.json.

Refractor usage in browser

Thanks a lot for making this plugin, I previously used rehype-prism and it was working like a charm.

For my current project, I'm implementing syntax highlighting which has to work client side, so I was following this documentation sections, though I have some difficulties injecting it in unified pipeline:

Code:

... other imports
import refractor from 'refractor/core.js'; 
import js from 'refractor/lang/javascript.js';
refractor.register(js);

const Comment = (props) => {
	const nodes = refractor.highlight(props.body, 'js');
	const prismjs = rehype()
		.stringify({type: 'root', children: nodes});

	const body = unified()
		.use([
			markdown,
			emoji,
			breaks,
			[remark2rehype, {allowDangerousHTML: true}],
			rehypeRaw,
			[github, {repository: 'user/repo'}],
			prismjs
			format,
			html
		])
		.processSync(props.body)
		.toString();

	return (
		<div className="comment">
			<img src={props.avatar} className="comment__avatar" />
			<div className="comment__author">{props.author}</div>
			<div className="comment__date">{props.date}</div>
			<div dangerouslySetInnerHTML={{__html: body}} />
		</div>
	)
}

export default Comment;

String:

```js
alert('xxx')
```

Error:

Error: Expected usable value, not `<span class="token template-string"><span class="token string">``</span></span><span class="token template-string"><span class="token string">`js
alert('xxx')
`</span></span><span class="token template-string"><span class="token string">``</span></span>`

▼ 3 stack frames were expanded.
add
node_modules/unified/index.js:209
addList
node_modules/unified/index.js:224
Function.use
node_modules/unified/index.js:177

Can refractor automatically determine language syntax or it has to be specifically defined in refractor.highlight(data, 'language-syntax')?

Any help will be greatly appreciated :octocat:

Vulnerabilities introduced in your package?

Hi ,@wooorm @xiaoxiangmoe , there are 3 vulnerabilities issue in your package:

Issue Description

3 vulnerabilities (high severity) CVE-2021-23341, CVE-2021-32723 and CVE-2020-15138 detected in package prismjs (>=1.1.0 <1.21.0) is directly referenced by refractor 2.10.1. We noticed that such a vulnerability has been removed since refractor 3.3.0.

However, refractor's popular previous version refractor
2.10.1
(790,634 downloads per week) is still transitively referenced by a large amount of latest versions of active and popular downstream projects (about 3,846 downstream projects, e.g., @storybook/addon-info 5.3.21, @storybook/addon-notes 5.3.21, storybook-readme 5.0.9, @storybook/react-native-server 5.3.23, @types/storybook__addon-info 5.2.4, @2fn/[email protected], @a8k/[email protected], etc.). As such, issue CVE-2021-23341, CVE-2021-32723 and CVE-2020-15138 can be propagated into these downstream projects and expose security threats to them.

These projects cannot easily upgrade refractor from version 2.10.1 to 3.3.* . For instance, refractor 2.10.1 is introduced into the above projects via the following package dependency paths:
(1)@2fn/[email protected] ➔ @2fn/[email protected][email protected][email protected][email protected]
(2)a8k/[email protected][email protected] ➔ @storybook/[email protected] ➔ @storybook/[email protected] ➔ @storybook/[email protected][email protected][email protected][email protected]
(3) @axa-ch/[email protected][email protected] ➔ @storybook/[email protected][email protected][email protected][email protected]
(4) @bbc/[email protected] ➔ @bbc/[email protected] ➔ @storybook/[email protected] ➔ @storybook/[email protected][email protected][email protected][email protected]
(5)@betty-blocks/[email protected] ➔ @betty-blocks/[email protected][email protected][email protected][email protected]
........

The projects such as @2fn/ui, storybook-react-tmp, storybook-addon-i18next, @bbc/digital-paper-edit-react-components and @betty-blocks/preview which introduced [email protected] are not maintained anymore. These unmaintained packages can neither upgrade refractor nor be easily migrated by the large amount of affected downstream projects.

On behalf the downstream users, could you help us remove the vulnerabilities from package [email protected]?

Suggested Solution

Since these unactive projects set a version constaint ~2.10.* for refractor on the above vulnerable dependency paths, if refractor removes the vulnerabilities from 2.10.1 and releases a new patched version [email protected],
such a vulnerability patch can be automatically propagated into the 3,846 affected downstream projects.

In [email protected], you can kindly try to perform the following upgrade:
prismjs ~1.17.0 ➔ ~1.23.0;
Note:
[email protected] (>=1.23.0) has fixed the vulnerabilities (CVE-2021-23341, CVE-2021-32723 and CVE-2020-15138)

Thanks you for your contributions.

Sincerely yours,
Paimon

Backport the prismjs 1.27.0 upgrade to 3.x

react-syntax-highlighter is still using refractor 3 for now. As prismjs 1.27.0 fixes an XSS vulnerability, would you agree to backport that upgrade to a 3.x version of refractor ?

Highlighting string interpolations in groovy throws an Error

Trying to highlight groovy code that contains string interpolations, ex:

def sum = "The sum of 2 and 3 equals ${2 + 3}"

throws the following error:

Error: Expected `string` for `name`, got `[object Object]`

I'll link a PR to this issue that creates a failing test case and partially solves the bug. However, I think another change needs to be made to the language file in order to get the test to pass, and I'm not quite sure how to modify the babel transform you introduced in #9 in order to accomplish this.

This line would need to be changed to:

env.content = Prism.highlight(env.content.value, {

Demo?

Hi @wooorm 👋

What are your thoughts on adding a link to an interactive demo to the readme?

I've made so many demo repls on Replit for refractor when I needed to debug something - would be cool to be able to just click on the existing demo and modify some things!

Have a chance to lazy initialize refractor

PrismJS itself contains a bunch of predefined behaviors, for example it automatically listen the message event when in a web worker, these behaviors can be disabled via global flags such as self.disableWorkerMessageHandler = true;

With refractor it initialize a PrismJS instance when module is imported, we don't have a good chance to set these flags when use es modules, so some lazy initialization would be very helpful:

import createRefractor from 'refractor/lazy';

self.disableWorkerMessageHandler = true;

const refractor = createRefractor();
refractor.highlight(code, language);

Incorrect Prism global in browser environment

Related to #38, but in this case focused on browser environments. The global Prism instance is not correctly setting manual or disableWorkerMessageHandler as intended in core.js:

ctx.Prism = {manual: true, disableWorkerMessageHandler: true}

Screen Shot 2021-09-21 at 10 06 04 AM

The result is that code elements are automatically processed by Prism, and in my case altering the DOM in an unwanted manner.

Screen Shot 2021-09-21 at 10 05 19 AM

Minimal reproduction in codesandbox: https://codesandbox.io/s/stoic-mccarthy-9fjzr?file=/src/App.js

N.B.: I think this only affects v4.x; window.Prism is not defined in v3.4.0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.