kylebarron / language-stata

Syntax highlighting for Stata in Atom

Stata language support in Atom

• News
• Features
• Installation
• Configuration
• Running Code
• Dynamic Documents

News

Use the Stata Jupyter kernel with Atom's Hydrogen package to show Stata results inline. It works with Windows, macOS, and Linux.

Example uses the Atom Material Syntax theme and Fira Code font.

Features

This package highlights:

• System commands, functions, and function arguments
• Macros, both global and local
  • Accurately colors nested macros and escaped macros in strings when you want the inner macro to evaluate at runtime
  • Colors macro extended functions inside `: ... ' as well as after local lname:
• Comments, more accurately than Stata's Do-file Editor.
• Regular expressions
  • Colors both the limited syntax provided through the regexr() and regexm() functions, as well as the vastly expanded regex syntax provided in Stata 14 and 15 through the ustrregexm(), ustrregexrf(), and ustrregexra() functions.
• Dynamic Markdown and LaTeX documents. Instructions below.

Other nice features:

• Works with unicode identifiers. Use unicode anywhere it's legal Stata syntax.
• Autocomplete for functions with a drop-down help menu. (This can be turned off in the settings).
• Autocomplete for commands and macros.
• Alerts you if your variable name is illegal, i.e. if your variable name is more than 32 chars, starts with a number, or is a reserved name.
• Alerts you if you have any text other than } on a line ending a foreach/forvalues/if/else command
• Local macro back tick autocompletion. When you write a `, Atom automatically fills in a ' after your cursor
• Makes it easy to spot incorrect nesting of compound quotes
• Support for programming ligatures for all valid Stata syntax for fonts that support them, like the Fira Code font.
• Highlights SQL queries used in the odbc command. (The language-sql base package must be active.)
• Highlights Docblockr-style keywords inside comments (anything like @Note)

Installation

To install, do one of the following:

• Go to Preferences/Settings > Install > Packages; and then search for language-stata
• At the command line, type apm install language-stata

The local macro back tick autocompletion won't function until you fully restart Atom. Do ctrl-shift-P or cmd-shift-P to bring up the command palette, type Window: Reload, and click enter.

Configuration

Atom allows you to toggle whether a line is commented using ctrl+/. As of version 1.6.5, the comment character this uses is // by default. You can change this to use /* */ or * characters to comment lines.

To change to /* */ comments, you can put the following in your config.cson file.

'.source.stata':
  editor:
    commentStart: '/* '
    commentEnd: ' */'

To change to * comments, use the following. However I don't recommend using this character¹.

'.source.stata':
  editor:
    commentStart: '* '

Note that in your config.cson file there can only be a single '.source.stata' top-level key, and only a single editor key under '.source.stata'. If you customize some other settings within the Stata grammar, you might already have a '.source.stata' key, and thus you would add the commentStart key to it.

Running Code

There are three ways to run code in Stata from Atom

1. The Hydrogen package in conjunction with the Stata Jupyter kernel shows Stata results inside Atom next to your code. The gif at the top of the page is an example of this setup. A few features:

   • Works with Windows, macOS, and Linux. Has an easier install on Windows than stata-exec.
   • Use a different session of Stata for each file, or connect them all to the same session.
   • Autocompletions as you type based on the variables and macros in memory
   • Use any type of comments in your code
   • Low-latency connections with remote sessions of Stata. Possible to reconnect to a running remote session if you get disconnected.
   • Use #delimit ; interactively with your code
   • Run code blocks within a Stata dynamic document

   You'll need to have both Hydrogen and stata_kernel installed. You can install Hydrogen from the Atom settings pane; see the stata_kernel documentation for more information on how to install stata_kernel.

2. The stata-exec package sends selected Stata code to an open Stata GUI window on Windows, macOS, and Linux. This differs from Hydrogen because it allows you to still interact with the Stata GUI. This might be easier for users who are new to Stata. However, it can be difficult to successfully install this on Windows.

3. The script package will run code in the Stata console, but has the limitations 1) each command is run in a separate session of Stata, 2) it currently doesn't work with selections; you have to run the entire file, 3) it doesn't work on Windows.

Dynamic Documents

Stata 15 brought new features for working with dynamic documents. The dyndoc command lets you write in Markdown and converts your file and code to HTML for viewing in a web browser.

It also added the dyntext command, which fills in Stata output for any text file, without touching the text itself. This lets you then use third-party document generators like Pandoc and LaTeX to generate documents.

Syntax Highlighting

This package provides syntax highlighting for Stata code written inside Stata's dynamic tags for Markdown and LaTeX documents.

By default, this package's Markdown and LaTeX syntax highlighting will be applied for files ending in .domd and .dotex respectively. The language-markdown and language-latex packages must be installed for the highlighting to work.

If you name your file with a different extension, you can manually set the highlighting by clicking on the "Plain Text" button on the bottom right of the screen (or by pressing CTRL+SHIFT+L). Then type stata and you'll see a list of choices: For a .do file, choose source.stata; for a Markdown dynamic document choose source.dyndoc.md.stata; and for a LaTeX dynamic document choose source.dyndoc.latex.stata.

Both Hydrogen and stata-exec should work for running code interactively, even within a dynamic document.

Examples

An example of the PDF output of using dyntext and Pandoc is in the examples folder: dyntext.pdf.

That file was created by running

dyntext dyntext.domd, saving(dyntext.md) replace

from inside Stata 15, and then with

pandoc dyntext.md -o dyntext.pdf

on the command line using Pandoc.

The file dyntext.dotex is a proof-of-concept and should compile with LaTeX but the output is not shown here.

Webdoc support

If you use the user-created command webdoc, you can add highlighting by using a .dowd file extension or by manually selecting the language of the current file to be "Stata Webdoc".

Footnotes

1: The following code is legal Stata code, but Atom will confuse the * used as multiplication with the * used for a comment. So if your cursor is on the second line and you press ctrl+/, Atom will remove the * symbol and the semantic meaning of the multiplication will be lost. Thus using // as the comment symbol is safer.

display 2 ///
  * 2

language-stata's Issues

Fix `merge` to not discolor everything after it when typing

regex bugs

Color optional / in syntax arguments

e.g.

description_of_if

        type                nothing
     or
        optionally type     [
        type                if
        optionally type     /
        type                ]      (if you typed [ at the start)

    Examples:
        syntax ... if ...
        syntax ... [if] ...
        syntax ... [if/] ...
        syntax ... if/ ...

    If you type nothing, the command does not allow an if exp.

    Typing [ and ] means that the if exp varlist is optional.

    After the syntax command, the resulting if exp is returned in `if'.  This macro contains if followed by the expression, unless you specified /, in which case
    the macro contains just the expression.

Fix handling of escape character

Escaping local macro inside quotation marks breaks highlighter.

Merge with `stata-language`?

Hey @kylebarron seems like this package has gotten well ahead of our iffy support of stata-language. Happy to merge our repo into yours so there are no longer competing syntax packages (if that's possible? Not quite clear on how atom package manager handles that) or close and redirect it.

cc: @benwhalley

Add example.do to check theme colors all tag names

Color macro assignment as string

Consider the code:

local effects abseffect(
foreach eff in `effect' {
    local effects `effects' `=`eff' * `mean''
}
local effects `effects' )

In my theme that shows as:

However, unless constructed as local macroname: or local macroname = , the text after the macro name is stored as a string exactly as typed. Therefore, it might reinforce the difference between those macro constructs by coloring the text after local macroname as a string.

Triple left tick/quote highlight

Originally discussed

https://www.reddit.com/r/stata/comments/79l8ge/what_color_scheme_do_you_use_in_stata/

1 // Make data data dictionary in markdown file.
2 log using asca_data_dictionary.md, replace text name(data_dictionary)
3 noi di "```"
4  - - -  do some other output - - - 
5 noi di "```"
6 log close data_dictionary

Line 3, which is intended to place github flavor codeblock markdown, interrupts the syntax highlighting. Stata's editor highlights this just fine.

Workaround

// Establish a macro to contain the triple quote mark.
local opclmd char(96) char(96) char(96)
// Make data data dictionary in markdown file.
log using asca_data_dictionary.md, replace text name(data_dictionary)
di `opclmd'
 - - - or do other code - - - 
codebook
 - - - finish other code - - -
di `opclmd'
log close data_dictionary

More on this use case

Demonstration of using stata log file feature for purposes of automated generation of markdown documents:

https://github.com/adamrossnelson/StataQuickReference/blob/master/md_demo.do (do file)
https://github.com/adamrossnelson/StataQuickReference/blob/master/wkspc/md_demo.md (output)

Make manual coloring for display command

Add basic smcl highlighting

Error detection for macros is strict about spaces

language-stata flags it as an error to have macro assignment without spaces, even though it's valid stata code. Would you consider changing the parser to not flag this as an error?

Here's a demo:

sysuse auto
sum price, detail

local x=r(p90)-r(p10)
local x= r(p90)-r(p10)

regex highlighting error

In the following code, the . doesn't show up correctly

regexm(find_value, "([^0-9\.])")

figure out escaped macro marks in strings

e.g. this is valid code but looks weird:

global dmel   "$datadir/raw/\`pct\'pct/dme/\`year\'/dmel\`year\'"

the \ are sometimes colored as a macro

Color as illegal the perl syntax that isn't accepted in regexm?

try to be smarter about prefixes

ideally, only color a command name when it's at the beginning of a line or after a prefix

Bugs with extended macros

local line : subinstr local line "//NULL " ""
local line : subinstr local line "." "_", all //avoid "." in name
local name : di `"`macval(line)'"'

Red highlighting when global macro with braces inside loop

Treat factor variable prefixes as system constants

i.e. i., c., and o.

Check that there's autocomplete for "*"

So that words in general can be autocompleted instead of just functions, macros.

What's the recommended way to add custom commands?

If I install some program, say reghdfe (https://github.com/sergiocorreia/reghdfe) what is the best way to make the new functions get recognized? Shall I just manually update the code and add the desired command?

Bug with multiple expressions in parentheses

No Syntax Highlighting After -Duplicates Drop-

just make a vscode branch in this repository

Refactor built in commands and functions

See if you can use + to split into ~80 character lines of matches

color syntax command specifically

don't forget to include (int) and (str) as types

refactor code for expressions

Many aspects of Stata code take arbitrary expressions. Create a type in the repository for expressions that aligns with Stata's syntax for it.

Compound double quotes doesn't work on multiple lines

Could be related to letting \n end a comment.

subsetting should color macros

Maybe related to #44

Rewrite readme to show off each feature

Add tests like language-python

https://github.com/atom/language-python/blob/master/spec/python-spec.coffee
https://github.com/atom/language-python/blob/master/spec/language-python-spec.coffee

color macros inside strings as macros

color macros inside strings as macros

Add `#delimit ;` support

Multi-line comment not gray when contains another multi-line comment

This code should all be gray because it is within /* */. It seems that when a block within /* */ contains another /* */, this doesn't work.

syntax command bug

Regex highlighting does not work if regexm after another function

I first noticed this with assert(mi(code14) | regexm(code14, "^(SC|[0-9])C$")), an actual use case where there was no highligting.

In general, though, the regex highlighting is awesome! You did a great job!

Include R after Rcall

gen can be called with just `g`

Think about different set of colorings for options

All options must start with a , but then you'd have to deal with ///.
So maybe you could say begin(,) end(\n that isn't preceded by /// anywhere on that line)

And then the insides could have usual options like replace, clear, append, and also some more unusual ones like nogen

Path characters can be \!

add `ustrregexr*` to autocomplete functions

include ustrregexra() and ustrregexrf()

syntax issue

hello(integer 5) doesn't color past inte

Add @ERROR in comment as invalid.illegal...stata

Include sql for use with odbc command

https://www.stata.com/help.cgi?odbc

Red highlights after brace not right

This is a pretty particular context, but for my -outreg- command I need to use braces and I'd like the following text not to be highlighted yellow. Is there a way to fix this?

"/**/" incorrectly recognized as comment (on Atom)

In Stata syntax, "/**/" can be used to add a comment or break a long line. "/" can be put at the end of a line and then "/" can be put in the begin of the second line. In the way, Stata will read it as one line when you running the .do file. For example, the following two lines of command should perform the same.

keep var1 var2 var3 var4/*
*/ var5 var6

keep var1 var2 var3 var4 var5 var6

Right now, "*/ var5 var6" is recognized as a comment on Atom. The funny thing is that this issue only occurs on Atom, not Visual Studio Code.
Btw, I'm using Stata 15.

Bug with gen with no spaces before =

Program arguments not yellow after ///

Sometimes people list arguments for a program after ///, which allows you to quickly explain the argument after the ///. The package does not highlight these arguments yellow.

Write up changelog from Git commit history

Autocomplete single-quotes for local macros

A long-standing request we never figured out how to implement...

benwhalley/atom-language-stata#15

Fix gen to work with local macros (i.e. from tempvar command)