[suggestion] Allow bracketless version of if/while/for/foreach about expressionevaluator

[codingseb]

OK I am currently working on this.
And your solution is good but I can not implement it in this strict manner.
Here a few reasons :

1. I will need a notion of brackets and a notion separator for head and body separation. This because brackets can be nested ((something) - (otherThing)) and a separator not so the parsing process is different.
2. We need at least one of these 2 notions to be mandatory otherwise it is very hard to know what is the head and what is the body in some cases.
3. The parsing process for the different kind of syntaxes of the head part is different of the parsing process of the different kind of syntaxes for the body (especially if we want to be able to use indentation for body blocks like in python).

Also take note that this is include in a refactoring of the script evaluation part of EE to be more flexible.
I also try to add the following features :

1. An option to chose how to separate expressions ; or something else like end of line.
2. A way to redefine scripts keywords or create a custom one

So here is what I came up with so far :
(I try to be language agnostic but also to prevent syntax evaluation conflicts or errors so it's a little bit different of your proposition)

public enum SyntaxForHeadExpressionInScriptBlocksKeywords
{
    HeadBrackets,
    SeparatorBetweenHeadAndBlock,
    Both,
    Any
}

public enum SyntaxForScriptBlocksIdentifier
{
    OptionalBracketsForStartAndEndWhenSingleStatement,
    MandatoryBracketsForStartAndEnd,
    Indentation
}

// ...

/// <summary>
/// To specify the character or string that begin the head statements of script blocks keywords (if, else if, for, foreach while, do.. while)
/// Default value : <c>"("</c>
/// </summary>
public string OptionScriptBlocksKeywordsHeadStatementsStartBracket { get; set; } = "(";

/// <summary>
/// To specify the character or string that end the head statements of script blocks keywords (if, else if, for, foreach while, do.. while)
/// Default value : <c>")"</c>
/// </summary>
public string OptionScriptBlocksKeywordsHeadExpressionEndBracket { get; set; } = ")";

/// <summary>
/// To specify the character or string that separate the head statements and the block statements of script blocks keywords (if, else if, for, foreach while, do.. while)
/// Default value : <c>":"</c>
/// </summary>
public string OptionScriptBlockKeywordsHeadExpressionAndBlockSeparator { get; set; } = ":";

/// <summary>
/// Specify how to detect the separation between head expression and the block of code is made in script block keyword (if, else if, for, foreach while, do.. while)
/// Default value : <c>HeadBrackets</c>
/// </summary>
public SyntaxForHeadExpressionInScriptBlocksKeywords OptionSyntaxForHeadExpressionInScriptBlocksKeywords { get; set; } = SyntaxForHeadExpressionInScriptBlocksKeywords.HeadBrackets;

/// <summary>
/// To specify the character or string that start a block of code used in script blocks keywords (if, else if, for, foreach while, do.. while) and multiline lambda.
/// Default value : <c>"{"</c>
/// </summary>
public string OptionScriptBlockStartBrackets { get; set; } = "{";

/// <summary>
/// To specify the character or string that end a block of code used in script blocks keywords (if, else if, for, foreach while, do.. while) and multiline lambda.
/// Default value : <c>"}"</c>
/// </summary>
public string OptionScriptBlockEndBrackets { get; set; } = "}";

/// <summary>
/// Specify the syntax to use to detect a block of code in script blocks keywords  (if, else if, for, foreach while, do.. while) and multiline lambda
/// Default value
/// </summary>
public SyntaxForScriptBlocksIdentifier OptionSyntaxForScriptBlocksIdentifier { get; set; } = SyntaxForScriptBlocksIdentifier.OptionalBracketsForStartAndEndWhenSingleStatement;

It should already allow a few different syntaxes :

C# like syntax

if(condition) DoSomeThingElse();

if(condition)
{
    // Do a lot of stuff or only one
}

Python like syntax

if condition:
    DoSomething

Pascal like syntax

if condition then
begin
    DoSomething();
end

Some exotic syntax

if |condition|
    DoSomething()

[codingseb]

Some small breaking changes are linked to this issue. So I think the next version of EE that will come with all of these scripts customization stuff will be a 1.5.x.x major version.

Also, the possibility to set arrays for block keywords syntax options seem harder than I thought especially for all brackets stuff that support nesting (imbrication) and for what we need to now which end bracket close which start bracket.

[codingseb]

Yes it's a good idea. I will look how to integrate this kind of option.

Take note that "EE" also support one expression blocks for these keywords :

for(x = 0; x < 5;x++)
     result += $"{x},";
//...
while(somethingIsTrue)
    DoSomethingElse();
//...
foreach(stuff in collectionOfThings)
    DoSomethingWith(stuff);
//...
if(y != 0)
    result = 1;
else if(x == 0)
    result = 2;
else
    result = 4;

So in in these cases, if we want to remove brackets we need a at least a character to separate the keyword condition expressions of it's "action" expression.

I could make it an enum option in combination with a "separator" property to specify how to separate these two parts.
It could be ':' by default to get a little closer to python like scripting.

The idea :

public enum OptionSyntaxForScriptKeywords
{
    CScharpSyntax, // ScriptKeywordsSeparator not used here
    OptionalParenthesesAndMandatorySeparatorWithoutCurlyBrackets,
    OptionalParenthesesAndAlwaysMandatorySeparator
}

// ... in ExpressionEvaluator ...
public OptionSyntaxForScriptKeywords OptionSyntaxForScriptKeywords { get; set; } = OptionSyntaxForScriptKeywords.CScharpSyntax;

public string ScriptKeywordsSeparator { get; set; } = ":";

So stuff like this could be possible :

if x > 2:
    DoSomething();

// or
if x > 2:
{
    // Do a lot of stuff
}

(Just keep in mind that the first goal of EE is to be as near as possible of C# by default)

We could also reflect later for an other option to do blocks with indentation in place of curly brackets. But I don't know what it would imply in EE code for now.

[lofcz]

I've took a quick peek at the regexes currently being used to describe bracketless keywords but I'm not sure how hard would it be to implement the proposed change. Being C# first is definitely great but allowing for deriving hyperspecific languages from that with a few options is another great feature of EE and this would further improve the situation.

I think that setting default delimiter to : instead of whitespace is wise as it would probably be more commonly used.

Examples at the bottom look really great!

[lofcz]

Just a note, instead of:

public enum OptionSyntaxForScriptKeywords
{
    CScharpSyntax, // ScriptKeywordsSeparator not used here
    OptionalParenthesesAndMandatorySeparatorWithoutCurlyBrackets,
    OptionalParenthesesAndAlwaysMandatorySeparator
}

consider:

string keywordHeadParenthesesStart = "(";
string keywordHeadParenthesesEnd = ")";

string keywordBodyParenthesesStart = "{";
string keywordBodyParenthesesEnd = "}";


public enum OptionSyntaxKeywordExpression
{
    MandatoryParentheses,
    OptionalParentheses
}

OptionSyntaxKeywordExpression keywordHeadParenthesesStyle = OptionSyntaxKeywordExpression.MandatoryParentheses;
OptionSyntaxKeywordExpression keywordBodyParenthesesStyle = OptionSyntaxKeywordExpression.MandatoryParentheses;

Another possibility here would be expression/statement instead of head/body.

Main point of this is to keep naming simple and language agnostic and allow for all possible permutations of these two styles.
Hence:

// 1. headStyle is mandatory and body style is mandatory
if (expr) {
  statement
}

// 2. headStyle is optional and body style is mandatory
if expr : {
  statement
}

// 3. headStyle is optional and body style is optional
if expr : statement

// 4. headStyle is mandatory and body style is optional
if (expr) statement

At style 1 keywordHeadParenthesesStart, keywordHeadParenthesesEnd, keywordBodyParenthesesStart and keywordBodyParenthesesEnd are at their default (proposed) values.
At style 2 keywordHeadParenthesesStart = " " and keywordHeadParenthesesEnd = ":"
At style 3 all four values are " "

By " " I mean a token consisting of 1..N consecutive whitespace / new line characters (\n, \r\n, \r) - any combination of these.

Do you think this would be possible?

[codingseb]

Yes your solution is more flexible and the language agnostic style is good.
I think it is possible but it will take some refactoring of the script part first to allow this.
For now the script part is more rigid than the expression part.

I will take this as the next improvement of the lib.

[lofcz]

This looks really great! After reading this the only thing I'm wondering about it whether it would be possible to have

public string[] OptionScriptBlocksKeywordsHeadStatementsStartBracket = new {"("};

Then internally detect which arrays are only one token - and treat these as simple strings for increased performance and iterate others. Hence:

public string[] OptionScriptBlockKeywordsHeadExpressionAndBlockSeparator { get; set; } = new {":", "|"};

Some exotic syntax:

 if condition: // this is valid
    MyFn();

 if condition| // this is also valid
    MyFn();

Do you think this would be possible, worth the effort and without significant perf impact? My idea here is to allow for multiple styles to be intermixed, as we can do in c#:

if (expr) {
  statement();
}

if (expr)  
   statement();

[codingseb]

Yes I already thought about this too.
In place of an array I would add a way to use the string as a Regex with some option like `OptionScriptBlocksKeywordsHeadStatementsAsRegex = true;"
Or directly make these Properties as Regex What do you think ?

public Regex OptionScriptBlockKeywordsHeadExpressionAndBlockSeparator { get; set; } = new Regex(@"^[:|]|then(?=\s)", RegexOptions.IgnoreCase);

[lofcz]

I'm concerned about performance impact here. If possible, could you please run a test so we could see how much cpu power are we using here? Regex vs plain string (and possibly vs array).

[codingseb]

Here is an Idea to allow block keywords customization
(as asked in OptionKeywordForInstanceCreation => OptionNewKeywordAliases commit)

IDictionary<string, Func<...>> blockKeywords { get; set; } = new Dictionary<string, Func<...>>()
{
    {"while", (keywordAttributes, subScript, ref isReturn, ref isBreak, ref isContinue, ref lastResult ...) => 
        {
            while (!isReturn && (bool)ManageJumpStatementsOrExpressionEval(keywordAttributes[0]))
            {
                lastResult = ScriptEvaluate(subScript, ref isReturn, ref isBreak, ref isContinue);

                if (isBreak)
                {
                    isBreak = false;
                    break;
                }
                if (isContinue)
                {
                    isContinue = false;
                }
            }
        }},
//...
};

It's always a reflection in progress and of course it will need some adaptations for things like if-else if -else and try-catch-finally famillies of keywords.

The idea is that for creating aliases we could just do :

blockKeywords["whileAlias"] = blockKeywords["while"];

But it would also allow creating quite easily custom block keyword :

blockKeywords["myNewKeyword"] = (keywordAttributes, subScript, ref isReturn, ref isBreak, ref isContinue, ref lastResult ...) => 
{
    // My custom interpretation of the custom keyword
};

[codingseb]

And for more python like forloops :

blockKeywords["for"] = blockKeywords["foreach"];
blockKeywords.Remove("foreach");