seedv / seedlang Goto Github PK

View Code? Open in Web Editor NEW

9.0 9.0 1.0 7 MB

An embeddable and visualizable scripting engine for .Net and Unity.

Home Page: https://seedv.github.io/SeedLang/

License: Apache License 2.0

Shell 0.29% JavaScript 4.03% ANTLR 0.98% C# 93.74% Lua 0.07% Python 0.89%

animation dot-net education gaming scripting-engine scripting-language unity visualization

seedlang's People

Contributors

Stargazers

Watchers

Forkers

594270461

seedlang's Issues

null value from ANTLR is not handled, for expressions like "eval x" or "eval x*2"

Start SeedLang.Shell, enter "eval x\n":

The returned node of parser.single_stmt() contains a null field: node.Expr = null. Meanwhile localCollection.Diagnostics.Count is still 0. The outer logic treated this as a normal case and continued the execution. The output from the shell is:

eval

Start SeedLang.Shell, enter "eval x+2\n":

Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
at SeedLang.Ast.BaseValue.op_Addition(BaseValue lhs, BaseValue rhs) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Ast/BaseValue.cs:line 25

Distinguish division-by-zero from infinite and NaN, and report separate diagnostic messages for the three cases

The current executor does not distinguish from divide-by-zero and infinite-result:

> eval 1 / 0
1 / 0 = Infinity
eval Infinity

> eval 1e100 * 1e100 * 1e100 * 1e100
1E+100 * 1E+100 = 1E+200
1E+200 * 1E+100 = 1E+300
1E+300 * 1E+100 = Infinity
eval Infinity

As per the requirement of a calculator, the client needs the info to show different messages, e.g.,

For divide-by-zero, show the "DIVBYZERO" error.
For infinite-result, show the "OVERFLOW" error.

And, the current SeedLang/Common/Diagnostic.cs only contains the localized message. How can a client code tell which error is the case? I think we need to also include the message ID in class Diagnostic.

Implement break statement

Implement continue statement

Implement SeedCalc language

Detect reference cycle

>>> a = [1, 2]
>>> b = [a]
>>> a[1] = b
>>> a

Replace Eval statement with Expression statement

Remove Eval statement.
Implement Expression statement.
Support visualization for Expression statement as well.
Use consistent interfaces for SeedBlock and SeedPython.

Pass out more semantic info via visualization events

When implementing the first SeedLang+Unity demo I found that the client code would require more semantic info for every visualization event. Here are some examples:

For the AssignmentEvent, the only thing client can get is the name of the variable. How can the client code know the variable counter is in the global scope or the local function scope? How can the client code tell if the variable counter is a function parameter?
For the BinaryEvent, the client code can get the left value and the right value, but how can the client code know if the left value is retrieved from a local variable counter?

... ...

This issue is kind of related to the topic of the semantic tokens that are to be highlighted in the code editor. Probably for every visualizer event, we need to pass out an EventContext object to the visualizer. The EventContext may contain:

The corresponding source code.
The semantic parse result of the corresponding source code.
The relationship (if SeedLang Runtime can figure out) between the semantic tokens in the source code and the event parameters such as left value, right value, variable name, etc.

Just some random thoughs.

SeedCalc parser returns wrong number of tokens

Input: 5+(
SeedCalc parsing result: 5, +, (, (

Input: 5+(3*2
SeedCalc parsing result: 5, +, (, 3, *, 2, 2

Similar cases result in exceptions in SeedLang.Shell:

dotnet run --project src/SeedLang.Shell -- -l SeedCalc 
> 5+(
---------- Source ----------
5+(Unhandled exception. System.ArgumentOutOfRangeException: Length cannot be less than zero. (Parameter 'length')
   at System.String.Substring(Int32 startIndex, Int32 length)
   at SeedLang.Shell.Repl.WriteSource(String source, IReadOnlyList`1 syntaxTokens) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Repl.cs:line 145
   at SeedLang.Shell.Repl.Execute() in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Repl.cs:line 123
   at SeedLang.Shell.Program.Run(Options options) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 50
   at SeedLang.Shell.Program.<>c__DisplayClass1_0.<Main>b__3(Options options) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 42
   at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult`1 result, Action`1 action)
   at SeedLang.Shell.Program.Main(String[] args) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 38

Default value is changed when a variable is assigned with a new value

This can be reproduced in the main branch, not related to the pull request being reviewed.

When there is no variable being assigned with values:

SeedLang.Shell 0.2.0-preview20220221060402
Copyright 2021-2022 The SeedV Lab.

>>> a
---------- Source ----------
a

---------- Run ----------
a
Eval result: None
>>> b
---------- Source ----------
b

---------- Run ----------
b
Eval result: None

When a variable is assigned with a new value:

SeedLang.Shell 0.2.0-preview20220221060439
Copyright 2021-2022 The SeedV Lab.

>>> a = 3
---------- Source ----------
a = 3

---------- Run ----------
>>> b
---------- Source ----------
b

---------- Run ----------
b
Eval result: 3
>>> c
---------- Source ----------
c

---------- Run ----------
c
Eval result: 3

>>> d = 5
---------- Source ----------
d = 5

---------- Run ----------
>>> e
---------- Source ----------
e

---------- Run ----------
e
Eval result: 5
>>> f
---------- Source ----------
f

---------- Run ----------
f
Eval result: 5

Every undefined variable will share the new value that is just assigned to a particular variable.

Secure SeedLang when Unity Managed Code Stripping is on

See wixette/isb#7

How to automate a process to ensure that SeedLang has no such issue?

SeedLang Calculator implement

Implement eval statement

eval 1 + 2

SeedCalc does not report eval result for a single number

When the expression has only one number operand, SeedCalc doesn't report any eval result to the visualizer:

dotnet run --project src/SeedLang.Shell -- -l SeedCalc
> 3343
---------- Source ----------
3343

---------- Run ----------

Compare to a similar case in SeedPython:

dotnet run --project src/SeedLang.Shell -- -l SeedPython

> 343
---------- Source ----------
343

---------- Run ----------
343: Eval result: 343

Inconsistencies between SeedPython and Python

Let's use this issue to track and discuss the inconsistent cases between the two languages: SeedPython and Python.

Expression statements with comparison operator results in Unhandled exception

dotnet run --project src/SeedLang.Shell 
SeedLang.Shell 0.2.0-preview20220222133854
Copyright 2021-2022 The SeedV Lab.

>>> 1 == 1
---------- Source ----------
1 == 1

---------- Run ----------
Unhandled exception. System.InvalidOperationException: Stack empty.
   at System.Collections.Generic.Stack`1.ThrowForEmptyStack()
   at System.Collections.Generic.Stack`1.Peek()
   at SeedLang.Interpreter.NestedJumpStack.get_FalseJumps() in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Interpreter/NestedJumpStack.cs:line 30
   at SeedLang.Interpreter.Compiler.VisitSingleComparison(Expression left, ComparisonOperator op, Expression right, Range range) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Interpreter/Compiler.cs:line 363
   at SeedLang.Interpreter.Compiler.Visit(ComparisonExpression comparison) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Interpreter/Compiler.cs:line 93
   at SeedLang.Ast.AstWalker.Visit(Expression expression) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Ast/AstWalker.cs:line 44
   at SeedLang.Interpreter.Compiler.Visit(ExpressionStatement expr) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Interpreter/Compiler.cs:line 209
   at SeedLang.Ast.AstWalker.Visit(Statement statement) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Ast/AstWalker.cs:line 92
   at SeedLang.Ast.AstWalker.Visit(AstNode node) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Ast/AstWalker.cs:line 27
   at SeedLang.Interpreter.Compiler.Compile(AstNode node, GlobalEnvironment env) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Interpreter/Compiler.cs:line 46
   at SeedLang.Runtime.Executor.Run(String source, String module, SeedXLanguage language, RunType runType, DiagnosticCollection collection) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Runtime/Executor.cs:line 88
   at SeedLang.Shell.Repl.Execute() in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Repl.cs:line 49
   at SeedLang.Shell.Program.Run(Options options) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 65
   at SeedLang.Shell.Program.<>c__DisplayClass1_0.<Main>b__3(Options options) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 54
   at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult`1 result, Action`1 action)
   at SeedLang.Shell.Program.Main(String[] args) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 50

Similar code that generates the same exception:

5 > 3
a = 5 > 3
a <= b

Whether or how to deal with floating number losing-precision issue?

See

SeedLang uses double to represent number values internally as Lua does.

Question 1: Do we want to represent numbers as accurately as possible?

Question 2: Do we need LargeDecimalNumber to represent numbers that are larger than Double.MaxValue?

Define project structure for C# libraries

Add Engine class as the interface for SeedLang

Single value parsing and evaluating doesn't report RuntimeOverflow

In both SeedPython and SeedCalc, an expression like a+b reports RuntimeOverflow as expected:

dotnet run --project src/SeedLang.Shell -- -l SeedPython
>>> 2e300 * 2e300
---------- Source ----------
2e300 * 2e300

---------- Run ----------
2e300 * 2e300
: 20211228145042791 Error (SeedLang.Ast) <> [Ln 1, Col 0 - Ln 1, Col 12] 27: RuntimeErrorOverflow

But when parsing and evaluating a single value, SeedLang doesn't report any error:

>>> 1e9999
---------- Source ----------
1e9999

---------- Run ----------
1e9999
Eval result: 0

>>> a = 1e9999
---------- Source ----------
a = 1e9999

---------- Run ----------
>>> a
---------- Source ----------
a

---------- Run ----------
a
Eval result: 0

How to access list elements in the handler of Event.Assignment and Event.VTagExited?

Both Event.Assignment and Event.VTagExited use IValue to pass out variables' values. E.g.:

  public class Assignment : AbstractEvent {
      public string Name { get; }
      public VariableType Type { get; }
      public IValue Value { get; }
  }

Given the current definition of IValue:

  public interface IValue {
    bool IsNil { get; }
    bool IsBoolean { get; }
    bool IsNumber { get; }
    bool IsString { get; }
    bool IsList { get; }
    bool IsFunction { get; }

    bool Boolean { get; }
    double Number { get; }
    string String { get; }
    int Length { get; }
  }

How does the client code get the element values of a list value? There is neither ToList conversion nor GetAt method when IValue is a list...

This is pretty similar to the requirement of a typical debugger's inspector: during a breakpoint of a program, the inspector needs to read the element values of a list value that is hold by a variable in the current execution scope.

Eval with parenthesis fails

In SeedLang.Shell, Input:

6(5*6)

Expected result: reporting syntax errors.

Actual result (no error):

---------- Source ----------
6(5*6)

---------- Run ----------
6(5*6): 5 * 6 = 30
6(5*6): Eval result: 30

Similar with the cases like:

(234+3432)3

No error is reported:

---------- Source ----------
(234+3432)3

---------- Run ----------
(234+3432)3: Eval result: 3

Wrong parsing/running behavior of floor division "//"

Parsing

IReadOnlyList<SyntaxToken> syntaxTokens =
          Executor.ParseSyntaxTokens("9//9", "main", SeedXLanguage.SeedPython, null);

Expected parsed tokens: 4

Actual parsed tokens: 2

Running

As in SeedLang.Shell:

9//9

Expected result (as Python floor division): 1
Actual result: 9

BTW

Calculators only have the "÷" key. And the key is mapped to "/" in SeedCalc code. It doesn't make sense to allow users to input two "÷"s and explain it as a "//" operator.

A similar thing could happen when we support "**“ as the power operator later. Since it doesn't make sense to have two continuous "×" as a power operator.

It seems I need to forbid repeating "/" and "*" in SeedCalc.

On the other hand, continuous "+" and "-" could be good things in SeedLang. Because SeedLang already supports things like "3++++4" and "3+-+-+--4", etc.

But is it friendly to calculator users to support such weird expressions like "3+-+-+--4"? I guess it'd be better to only allow at most two of them, such as "3 + +4", "3 + -5", etc. When the operator is the leftmost token, only allow one of them, such as "+3 +4", "-3 + 4", etc.

Option 1) Implement these calculator-specific logic in SeedCalc.

Option 2) SeedLang provides a language named SeedXLanguage.SeedCalc. In this language, turn off operators like "//" and "**", and limit the continuous "+" and "-" operators as in the above way.

Which one do you prefer? I am okay with either of them.

The generated C# code of Antlr reports CS3021 warnings

A clean dotnet build results in a number of similar CS3021 warnings. E.g.,

~/src/SeedLang/csharp/src/SeedLang.X/antlr/SeedPythonBaseVisitor.cs(36,22): warning CS3021: 'SeedPythonBaseVisitor<Result>' does not need a CLSCompliant attribute because the assembly does not have a CLSCompliant attribute [~/src/SeedLang/csharp/src/SeedLang.X/SeedLang.X.csproj]

According to the doc at https://docs.microsoft.com/en-us/dotnet/standard/language-independence we may not want to enforce our assembly to be [assembly: CLSCompliant(true)]. Can we just simply suppress the warnings?

Dll dependencies when using SeedLang in Unity

When importing SeedLang.dll into Unity as platform plugins, there are a couple of dependencies. We can use this issue to track and record relevant info, and try to minimize the dependencies when using SeedLang in Unity.

netstandard2.0/SeedLang.dll

Depends on the following DLLs (which can be got from NuGet packages):

System.Collections.Immutable.dll (netstandard2.0, ver 1.5.0)
Antlr4.Runtime.Standard.dll (netstandard2.0, ver 4.9.2)

Support source code folding

See https://en.wikipedia.org/wiki/Code_folding

To support code folding in IDEs, the SeedLang AST needs to be associated with source code reference info. With source code reference info, IDEs can let use to collapse or expanse a syntax block in the parsed AST.

Include Message ID in Diagnostic info

Client code may need to distinguish one message from another, to show the error message in different ways.

E.g., a calculator may show the division-by-zero error as an abbreviation string "ERR:DIVBY0", and group a number of syntax errors and show the same "ERR:SYNTAX" for them all.

SeedCalc running in SeedLang.Shell throws SyntaxErrorUnwantedToken '\n' <EOF>

dotnet run --project src/SeedLang.Shell -- -l SeedCalc
>>> 3+2
---------- Source ----------
3+2

---------- Run ----------
3+2
: 20211228143917347 Fatal (SeedLang.X) <> [Ln 1, Col 3 - Ln 1, Col 3] 33: SyntaxErrorUnwantedToken '\n' <EOF>
>>>

Remove AST executor

We can remove this feature to reduce maintenance efforts safely. The AST tree can be compiled to bytecode to execute if the client would like to run an AST tree.

The design doc of SeedBlock

It's time to clarify the design considerations of SeedBlock, since the implementation code has started to be checked in. See the below notes for a code review discussion:

The term "input" comes from Google Blockly's terminology, with a slight modification.

See https://developers.google.com/blockly/guides/create-custom-blocks/define-blocks#block_inputs for Google Blockly's definition.

In Google Blockly, "input" means all types of blocks that can be connected to the target block.

Compared to Google Blockly, SeedLang Block has some significant design decisions:

There is no "output" connection in blocks. I.e., all blocks are divided into two categories: statements (without result value) and expressions (with result values). With this design, there is no need to specify how it can be connected to another block's input slot. Thus in SeedLang Block, every input slot can be docked with an expression block, taking its result value as the input value.

Based on the above consideration, it's required to distinguish the Input slots from the statement slots.

I will describe this in detail in the design doc of SeedBlock, at /design/seed_block.md shortly. Will open an issue for me on this task.

Grammar allows empty return but parser and compiler throw exceptions

How to repro:

SeedLang.Shell 0.2.0-preview20220306085334
Copyright 2021-2022 The SeedV Lab.

>>> def foo():
...   return
... 
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at SeedLang.X.SeedPythonVisitor.VisitReturn_stmt(Return_stmtContext context) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/X/SeedPythonVisitor.cs:line 171
   at SeedLang.X.SeedPythonParser.Return_stmtContext.Accept[TResult](IParseTreeVisitor`1 visitor) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/obj/Debug/net6.0/SeedPythonParser.cs:line 1505
...

The following code change fixes the issue only at the parser level:

public override AstNode VisitReturn_stmt(
        [NotNull] SeedPythonParser.Return_stmtContext context) {
      return _helper.BuildReturn(context.RETURN().Symbol, context.expressions()?.expression(),
                                 context.expressions()?.COMMA(), this);
    }

With this fix, the same code results in an exception at the compiler level:

SeedLang.Shell 0.2.0-preview20220306085050
Copyright 2021-2022 The SeedV Lab.

>>> def foo():
...   return
... 
---------- Source ----------
1     def foo():
2       return
---------- Run ----------
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at SeedLang.Interpreter.Compiler.Visit(ReturnStatement return) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Interpreter/Compiler.cs:line 307
   at SeedLang.Ast.AstWalker.Visit(Statement statement) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Ast/AstWalker.cs:line 107
   at SeedLang.Interpreter.Compiler.Visit(FuncDefStatement funcDef) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Interpreter/Compiler.cs:line 264
   at SeedLang.Ast.AstWalker.Visit(Statement statement) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Ast/AstWalker.cs:line 98
   at SeedLang.Ast.AstWalker.Visit(AstNode node) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Ast/AstWalker.cs:line 27
   at SeedLang.Interpreter.Compiler.Compile(AstNode node, GlobalEnvironment env, RunMode runMode) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Interpreter/Compiler.cs:line 49
   at SeedLang.Runtime.Executor.Run(String source, String module, SeedXLanguage language, RunType runType, RunMode runMode, DiagnosticCollection collection) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Runtime/Executor.cs:line 94
   at SeedLang.Shell.Repl.Execute() in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Repl.cs:line 49
   at SeedLang.Shell.Program.Run(Options options) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 68
   at SeedLang.Shell.Program.<>c__DisplayClass1_0.<Main>b__3(Options options) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 57
   at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult`1 result, Action`1 action)
   at SeedLang.Shell.Program.Main(String[] args) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 53

BlockProto - a quick prototype maker for SeedBlock

This will be a utility under the /utils dir. The utility reads a simple definition of one or more blocks from the command line and outputs the prototype image in the SVG and PNG format.

The output images of the utility are used for design documentation, as well as quick prototypes. That is, the rendering ability of this utility is way weaker than the production renderers.

Based on the same consideration, the utility's input is way simpler than the formal block exchange format, too.

Automate the release process

With the commit 87154f3, A new SeedLang release can be manually published to NuGet. It's good enough for the current dev stage.

Going forward, we will need to automate the release process, for example:

Once a new version number is given with a release PR, and the PR has been approved by the core team:
- Auto build and verify the source code.
- Auto create the release branch and add the git tag.
- Auto publish to NuGet.
- Auto add release info and links on the git releases page.
- ...

Wrongly formatted source code in error info

SeedLang.Shell 0.2.0-preview20220410023855
Copyright 2021-2022 The SeedV Lab.
>>> a = [1, 2, 3] 
>>> if len(a) > 1:
...   print(****)
...
---------- Source ----------
1     if len(a) > 1:
2       print(****)
---------- Run ----------
1     if len(a) > 1:
2       print(****)
20220410103942998 Fatal (SeedLang.X) <> [Ln 1, Col 0 - Ln 2, Col 9] 37: SyntaxErrorNoViableAlternative 'iflen(a)>1:\n  print(**'
1     if len(a) > 1:
20220410103942998 Fatal (SeedLang.X) <> [Ln 1, Col 13 - Ln 1, Col 13] 38: SyntaxErrorUnwantedToken ':' {';', NEWLINE}
2       print(****)
20220410103942998 Fatal (SeedLang.X) <> [Ln 2, Col 0 - Ln 2, Col 1] 35: SyntaxErrorInputMismatch '  ' <EOF>

Notice that in the first error info, the source code is formatted as 'iflen(a)>1:\n print(**'. There should be at least one space between if and len.

And, should we just show the original source code as it is in the error message, without removing any of its space characters?

V-tag with arguments defined for a function call

For a quick sort algorithm:

# [[ Index(start, end) ]]
def partition(start, end, a):
    # [[ Index ]]
    pivot_index = start
    ... ...

The current V-tag implementation reports the following error:

---------- Run ----------
4     # [[ Index(start, end) ]]
: 20220410105627796 Fatal (SeedLang.Interpreter) <> [Ln 4, Col 11 - Ln 4, Col 15] 33: RuntimeErrorVariableNotDefined

start and end are parameters of the observed function. They do have values when the function is called in runtime.

Design of source code lines for bytecode

We have added VISNOTIFY opcode for visualizers notification. The code range or semantic info can be included in the corresponding event classes. It seems the code range is not needed for each instructor in the chunk class. Only a line number for each instructor is enough for indicate the line of source code that this instructor belongs to.
It helps to implement the pause and resume feature. And it can also reduce the memory needed.

private readonly List<Instruction> _bytecode = new List<Instruction>();
# Remove ranges from the chunk class
# private readonly List<Range> _ranges = new List<Range>();
# Add line number for each instructor 
private readonly List<int> _lines = new List<int>();

But there are two problems for this design. First is sometime it's hard to match instructor to source code lines, for example:

for i in (1, 2, 3):
  print(i)

The compiled bytecode are as following. Which source code line should the FORLOOP instructor belongs to? 1 or 2? I think maybe line 1 shall be highlight when the execution stops in instructor 16. How about instructor 17?

(Line 1)  1    LOADK     1 -1             ; 1                 [Ln 1, Col 10 - Ln 1, Col 10]
(Line 1)  2    LOADK     2 -2             ; 2                 [Ln 1, Col 13 - Ln 1, Col 13]
(Line 1)  3    LOADK     3 -3             ; 3                 [Ln 1, Col 16 - Ln 1, Col 16]
(Line 1)  4    NEWTUPLE  0 1 3                                [Ln 1, Col 9 - Ln 1, Col 17]
(Line 1)  5    LOADK     1 -4             ; 0                 [Ln 1, Col 0 - Ln 2, Col 9]
(Line 1)  6    LEN       2 0 0                                [Ln 1, Col 0 - Ln 2, Col 9]
(Line 1)  7    LOADK     3 -1             ; 1                 [Ln 1, Col 0 - Ln 2, Col 9]
(Line 1)  8    FORPREP   1 7              ; to 16             [Ln 1, Col 0 - Ln 2, Col 9]
(Line 1)  9    GETELEM   4 0 1                                [Ln 1, Col 0 - Ln 2, Col 9]
(Line 1)  10   SETGLOB   4 6                                  [Ln 1, Col 0 - Ln 2, Col 9]
(Line 2)  11   GETGLOB   4 0                                  [Ln 2, Col 2 - Ln 2, Col 9]
(Line 2)  12   GETGLOB   5 4                                  [Ln 2, Col 2 - Ln 2, Col 6]
(Line 2)  13   GETGLOB   6 6                                  [Ln 2, Col 8 - Ln 2, Col 8]
(Line 2)  14   CALL      5 1 0                                [Ln 2, Col 2 - Ln 2, Col 9]
(Line 2)  15   CALL      4 1 0                                [Ln 2, Col 2 - Ln 2, Col 9]
(Line 1 or 2?)  16   FORLOOP   1 -8             ; to 9              [Ln 1, Col 0 - Ln 2, Col 9]
(Line 3?)  17   RETURN    0 0

Second problem is there isn't line number for block programs. Should we use following classes to express source code line for block and text programs?

class abstract SourceCodeLine {
}

class BlockLine: SourceCodeLine {
  public string BlockId;
}

class TextLine: SourceCodeLine {
  public int Line;
}

Boolean expression with comparison and boolean constant crash issue

SeedLang.Shell crashes when running following code:

if 1 < 2 and True:
  print(1)

Implement v-tag parser

Implement v-tag parser for following code:

# [[ Swap, Inline ]]

# [[ Swap(a, i, j) ]]

# [[ Swap(a, b)
a, b = b, a
# ]]

Support syntax highlighting

"[[" "[[[" "]]" "]]]", ... cannot be parsed correctly

SeedLang.Shell 0.2.0-preview20220410022950
Copyright 2021-2022 The SeedV Lab.

>>> a = [[[1]]]
---------- Source ----------
1     a = [[[1]]]
---------- Run ----------
1     a = [[[1]]]
20220410103000855 Fatal (SeedLang.X) <> [Ln 1, Col 4 - Ln 1, Col 5] 38: SyntaxErrorUnwantedToken '[[' {'True', 'False', 'None', 'not', '+', '-', '(', '[', '{', NAME, NUMBER, STRING}
1     a = [[[1]]]
20220410103000855 Fatal (SeedLang.X) <> [Ln 1, Col 6 - Ln 1, Col 9] 37: SyntaxErrorNoViableAlternative '[1]]'
1     a = [[[1]]]
20220410103000856 Fatal (SeedLang.X) <> [Ln 1, Col 8 - Ln 1, Col 9] 35: SyntaxErrorInputMismatch ']]' {';', NEWLINE}
>>> a = [[1]]  
---------- Source ----------
1     a = [[1]]
---------- Run ----------
1     a = [[1]]
20220410103007671 Fatal (SeedLang.X) <> [Ln 1, Col 4 - Ln 1, Col 5] 38: SyntaxErrorUnwantedToken '[[' {'True', 'False', 'None', 'not', '+', '-', '(', '[', '{', NAME, NUMBER, STRING}
1     a = [[1]]
20220410103007671 Fatal (SeedLang.X) <> [Ln 1, Col 7 - Ln 1, Col 8] 38: SyntaxErrorUnwantedToken ']]' {';', NEWLINE}

As a comparison:

>>> a = [ [1] ]
---------- Source ----------
1     a = [ [1] ]
---------- Run ----------
>>> a = [ [ [1] ] ]
---------- Source ----------
1     a = [ [ [1] ] ]
---------- Run ----------

A script to release SeedLang to NuGet

As a requirement of SeedCalc SeedV/SeedCalc#6, it's okay for the first stage to manually publish SeedLang to NuGet then manually fetch the DLLs for SeedCalc.

The script does:

Setup a new release branch based on the latest commit of the main branch.
On top of the release branch:
- Auto increase the build number (as in the version number of the SeedLang NuGet package).
  - The build number is configured by the release branch automatically, while the major number and the minor number are configured by the main branch manually.
- Auto publish the NuGet package.
- Auto release DLLs to a git repo's release entry.

TODO: check on existing lightweight release tools to see if there are any suitable candidates.

Assignment and Comparison visualizers are not triggered in SeedLang.Shell

In SeedLang.Shell, it seems there is no visualizer output during an assignment operation or a comparison operation:

dotnet run --project src/SeedLang.Shell -- --visualizers=All        
SeedLang.Shell 0.2.0-preview20220222142217
Copyright 2021-2022 The SeedV Lab.

>>> a = 3 + 4
---------- Source ----------
a = 3 + 4

---------- Run ----------
a = 3 + 4
Binary: 3 + 4 = 7
>>> a
---------- Source ----------
a

---------- Run ----------
a
Eval result: 7
>>> if 5 > 3:
...   b = 3
... 
---------- Source ----------
if 5 > 3:
  b = 3

---------- Run ----------
>>> if a > b:
...   a = 2
... else:
...   a = 3
... 
---------- Source ----------
if a > b:
  a = 2
else:
  a = 3

---------- Run ----------
>>>

As in the above example, only "Binary" and "Eval" visualizers are triggered.

Add original copyright and license info of ANTLR v4 grammar into our drivers code

Unhandled exception. System.NullReferenceException at PythonVisitor.cs:line 88

How to repro

At commit 5da3fe6

Open SeedLang.Shell, enter

eval 8+((
eval 8+(((
eval 8*(((
eval 8/(((

Exception

Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at SeedLang.X.PythonVisitor.VisitGrouping(GroupingContext context) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/X/PythonVisitor.cs:line 88
   at SeedLang.X.SeedPythonParser.GroupingContext.Accept[TResult](IParseTreeVisitor`1 visitor) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/obj/Debug/net5.0/SeedPythonParser.cs:line 1108
   at Antlr4.Runtime.Tree.AbstractParseTreeVisitor`1.Visit(IParseTree tree)
   at SeedLang.X.VisitorHelper.BuildGrouping(IToken openParen, ParserRuleContext exprContext, IToken closeParen, AbstractParseTreeVisitor`1 visitor) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/X/VisitorHelper.cs:line 72
   at SeedLang.X.PythonVisitor.VisitGrouping(GroupingContext context) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/X/PythonVisitor.cs:line 88
   at SeedLang.X.SeedPythonParser.GroupingContext.Accept[TResult](IParseTreeVisitor`1 visitor) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/obj/Debug/net5.0/SeedPythonParser.cs:line 1108
   at Antlr4.Runtime.Tree.AbstractParseTreeVisitor`1.Visit(IParseTree tree)
   at SeedLang.X.VisitorHelper.BuildBinary(IToken opToken, BinaryOperator op, ParserRuleContext[] exprContexts, AbstractParseTreeVisitor`1 visitor) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/X/VisitorHelper.cs:line 46
   at SeedLang.X.PythonVisitor.VisitAdd_sub(Add_subContext context) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/X/PythonVisitor.cs:line 49
   at SeedLang.X.SeedPythonParser.Add_subContext.Accept[TResult](IParseTreeVisitor`1 visitor) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/obj/Debug/net5.0/SeedPythonParser.cs:line 1126
   at Antlr4.Runtime.Tree.AbstractParseTreeVisitor`1.Visit(IParseTree tree)
   at SeedLang.X.VisitorHelper.BuildEval(IToken evalToken, ParserRuleContext exprContext, AbstractParseTreeVisitor`1 visitor) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/X/VisitorHelper.cs:line 138
   at SeedLang.X.PythonVisitor.VisitEval_stmt(Eval_stmtContext context) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/X/PythonVisitor.cs:line 118
   at SeedLang.X.SeedPythonParser.Eval_stmtContext.Accept[TResult](IParseTreeVisitor`1 visitor) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/obj/Debug/net5.0/SeedPythonParser.cs:line 488
   at Antlr4.Runtime.Tree.AbstractParseTreeVisitor`1.VisitChildren(IRuleNode node)
   at SeedLang.X.SeedPythonBaseVisitor`1.VisitSmall_stmt(Small_stmtContext context) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/obj/Debug/net5.0/SeedPythonBaseVisitor.cs:line 90
   at SeedLang.X.SeedPythonParser.Small_stmtContext.Accept[TResult](IParseTreeVisitor`1 visitor) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/obj/Debug/net5.0/SeedPythonParser.cs:line 380
   at Antlr4.Runtime.Tree.AbstractParseTreeVisitor`1.Visit(IParseTree tree)
   at SeedLang.X.PythonVisitor.VisitSingle_stmt(Single_stmtContext context) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/X/PythonVisitor.cs:line 41
   at SeedLang.X.SeedPythonParser.Single_stmtContext.Accept[TResult](IParseTreeVisitor`1 visitor) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/obj/Debug/net5.0/SeedPythonParser.cs:line 119
   at Antlr4.Runtime.Tree.AbstractParseTreeVisitor`1.Visit(IParseTree tree)
   at SeedLang.X.BaseParser.Parse(String source, String module, ParseRule rule, DiagnosticCollection collection, AstNode& node, IReadOnlyList`1& tokens) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/X/BaseParser.cs:line 55
   at SeedLang.Runtime.Executor.Parse(String source, String module, SeedXLanguage language, DiagnosticCollection collection) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang/Runtime/Executor.cs:line 67
   at SeedLang.Shell.Repl.Execute() in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Repl.cs:line 95
   at SeedLang.Shell.Program.Run(Options options) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 50
   at SeedLang.Shell.Program.<>c__DisplayClass1_0.<Main>b__3(Options options) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 42
   at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult`1 result, Action`1 action)
   at SeedLang.Shell.Program.Main(String[] args) in /Users/ygwang/src/SeedLang/csharp/src/SeedLang.Shell/Program.cs:line 38

Notify chained name for variables

@wixette Just want to confirm that we really need chained name for variables, for example, Global.var1, Global.func1.var1. They shall be static scoping name chains. We need implement some mechanism to collect the chained name when variables are defined.

How about class and instance variable after we implement Class? Shall we use Global.class1.var1 and Global.func1.object1.var1 as the chained name? Is there any confusion here?

Include source code reference info in visualization events

Example case:

A calculator plans to show which part of a text expression is under calculation. The current visualization events only contain the info of the operator, the operants, and the calculation result. Without knowing the range of each token in the source code, the calculator is not able to highlight the text part that is under calculation.

Technically, we need a well-designed source code reference feature in the following components:

A SeedBlock block needs a source code reference if it is parsed from an inline text - to support syntax highlight of the inline text.
A SeedAst node needs a source code reference - to support visualization during Ast-mode execution.
The bytecode compiled from SeedAst needs source code reference to support both debugging and visualization.

In SeedLang, a source code reference info may refer to:

The ID or ID range of a SeedBlock block.
The text range of an inline text of SeedBlock.
The text range of a SeedX source code.

Where and how to store the reference info?

For SeedBlock blocks, the source code reference info will be stored in each block, in-memory only (the info won't be serialized when saving a block program).
For SeedAst nodes, the source code reference info can be stored in each node, or can be stored in a separate in-memory structure. The info is in-memory only and won't be serialized.
For compiled bytecode, the source code reference info can be stored in a separate ref chunk, without affecting the execution of the bytecode. When the bytecode is serialized to disk, the reference info can either be stored in a separate file, or in the reference chunk of the same file.

dotnet run --project src/SeedLang.Shell -- -v Assignment
SeedLang.Shell 0.2.1-preview20220502124751
Copyright 2021-2022 The SeedV Lab.

Enabled Visualizers: Assignment

>>> a = [1, 2, 3]
---------- Source ----------
1     a = [1, 2, 3]

---------- Run ----------
1     a = [1, 2, 3]
Assign global a = [1, 2, 3]
>>> a[0] = 3
---------- Source ----------
1     a[0] = 3

---------- Run ----------

There is no notification for the code line a[0] = 3.

Support list slicing expression

Sorting algorithms such as merge sort depend on slicing expressions to partition the data into left list and right list, e.g.:

def merge_sort(a):
    if len(a) > 1:
        mid = len(a) // 2
        left = a[:mid]
        right = a[mid:]
        merge_sort(left)
        merge_sort(right)

The slicing expression:

L[ start : stop : step ]

The slice function may be considered later.