Without Y_COMMANDS_NO_IPC, the command doesn't get call correctly about ysi-includes

[Crayder]

I think you mean @Y-Less (dash, not underscore)

[Misiur]

Recently I added punycode support for all commands, so this might be slowing down stuff a bit as well.

[YashasSamaga]

Did some benchmarks
10000000 Iterations
YSI-old takes around 30000ms on average
YSI-new takes around 38000ms on average

[YashasSamaga]

#if !defined Y_COMMANDS_NO_IPC
    if (!IsPlayerConnected(p)) return Command_OnReceived(NO_PLAYER, p, c);
#endif

It obviously won't work with CallLocalFunction because I am passing an invalid playerid.

Not a bug!

[Y-Less]

Glad I could help :D

As for the speed tests, y_commands is designed for huge numbers - hundreds of commands as would be found in RPGs etc. Everyone knows it is slow with 2 or 3. For my benchmarks I always did several things:

1. Had hundreds of commands (easy to just generate a file with names like "/aaa", "/aab" etc).

2. Called the first one in the list ("/aaa"), the middle on ("/mmm") and the last one ("/zzz") to account for search methods.

3. Called a command that doesn't exist ("/nope"), because some people enter invalid commands.

Despite all that, I have no doubt that it is possible to write a faster command processor, but it doesn't really matter and shouldn't be the primary aim! Most people try to write a faster one by stripping out features just so they can get the "fastest" label. Speaking of which, check out mcmd as well:

http://forum.sa-mp.com/showthread.php?t=172186
http://forum.sa-mp.com/showthread.php?t=173292

Here is my entry in to the "fastest" category:

public OnPlayerCommandText(playerid, cmdtext[]) return 1;

I simply removed the "names", "commands", and "return" features to get some extra speed...

If you want more sensible suggestions:

You should also account for permissions, since y_commands is checking them and no other processors are. Thus, the first thing people do in almost every command is check if the player can use the command - that's additional overhead that y_commands doesn't need. I.e. the time of interest should be calling and running the command together, not just calling the command if running it is slow.

Other features (if you can support more than just "/cmd" while still being faster, more power to you):

1. Alt commands, in (I)ZCMD these are hacks, call one command to call a second - have you benchmarked any of those?

2. What happens when one command is defined in multiple scripts? Or if there aren't multiple scripts did you define YSI_NO_MASTER?

3. I18N - what happens when you type "/команда", and can you define a command with that name? In y_commands, that's (punycode):

YCMD:@80aalwkhe(playerid, params[], help)
{
}

Or you can use alt commands for the same effect but clearer.

4. Shortcuts - players can define their own set of single letter commands to alias longer commands, instead of relying on the server settings them to be the ones they use the most.

5. Other command prefixes: Supports any other leading character, such as !ban instead of /ban.

6. Delete commands.

7. Iterators to list all commands and player's accessible commands.

8. Help.

In short, you can claim that a command processor is faster than y_commands, and if you believe it you can claim that one is better. However, I believe that claiming that one is better purely because it is faster is misguided at best and downright dishonest at worst.

If you say "this is faster than y_commands, but doesn't have X, Y, and Z", then that's fine - if people don't want/need those features they can use it. But if people use yours, then add on additional code to check if a player can use it, then complain anyway because it doesn't work on their Portuguese server, then they have totally destroyed all the advantages the slight speed advantage gave.

If you write one that is faster than y_commands and has all its features, or at least a significant number of its features (and believe me, if you drop support for multiple scripts I can tell you EXACTLY where to get WAY more speed), then that would be amazing! I used to hope someone would because then I could learn from what they did, now I don't care so much because I don't do SA:MP coding, but it is still good for others.

[YashasSamaga]

Thank You! ;)

I made a native version which uses get_property. This isn't good because get_property anyway has to search!! Hence, doesn't make a significant improvement in performance.

https://github.com/YashasSamaga/I-ZCMD/blob/master/single-use-version%28izcmd%20beta%29.inc

I was recently working on a new concept, using a trie and a little of suffix tree to store commands.Search the public native list and get all the command functions and construct a trie. When the command is called, the script searches for the index(assigned by the script) from the trie. From this index (I have enum arrays) I can check for other details and finally get the address and call the function.

I made a small test trie which stores addresses of local functions and compared it with CallLocalFunction and it turns out, surprising, to be way much faster than using CallLocalFunction. I wonder what algorithm CallLocalFunction uses to find a function.

The problems I have now are:
Cross-Script Communication:The only way I can share information b/w scripts is using properties and CallRemoteFunction which isn't that efficient.

I am actually struggling to get all the command functions from the public function list. I want to avoid using amx_assembly because just for the sake of one include I need to make other download another script(s).

[Y-Less]

I would strongly recommend using amx_assembly for that, as you can also use it for calling the commands instead of using raw #emit (which will save hacks like switch (false)).

You are now hitting the problems I had - I didn't use directly calling functions by address because of cross-script communications, resulting in using CallRemoteFunction which is inefficient as you rightly point out. CallLocalFunction uses a binary search of strings, which can be seen in amx_FindNative:

int AMXAPI amx_FindNative(AMX *amx, const char *name, int *index)
{
  int first,last,mid,result;
  char pname[sNAMEMAX+1];

  amx_NumNatives(amx, &last);
  last--;       /* last valid index is 1 less than the number of functions */
  first=0;
  /* binary search */
  while (first<=last) {
    mid=(first+last)/2;
    amx_GetNative(amx, mid, pname);
    result=strcmp(pname,name);
    if (result>0) {
      last=mid-1;
    } else if (result<0) {
      first=mid+1;
    } else {
      *index=mid;
      return AMX_ERR_NONE;
    } /* if */
  } /* while */
  /* not found, set to an invalid index, so amx_Exec() will fail */
  *index=INT_MAX;
  return AMX_ERR_NOTFOUND;
}

That's directly from the PAWN VM, not SA:MP code, which is sometimes a useful go-to for more advanced questions. This search dictates that the header must be in alphabetical order, which affected how y_hooks was written (but not much):

https://github.com/Zeex/pawn/blob/b261bacfa490b4e381ec9f76e1046f709d6ba6f7/source/amx/amx.c#L1467-L1492

It isn't actually THAT hard to beat it - it does a full text comparison every time it is called. A cleverer search with less text comparisons and more result caching over a smaller set of functions could beat this even in PAWN over C. And calling the function with assembly instead is VASTLY faster! I never considered a trie, I was moving towards a hashmap with a binary search as the second level instead of a list.

[YashasSamaga]

Did some new speed tests, any suggestions?

http://pastebin.com/2KBBzcdz

Test Results are here @ Speed Test Section
forum.sa-mp.com/showthread.php?p=3475795

@Y-Less If you don't mind, rate this
http://forum.sa-mp.com/showthread.php?t=580289

Where can I find your Code Optimizations thread (its deleted)? I see some of them re-posting some of your tutorials and includes. I don't know where they find them.

[Crayder]

About your speed tests, YCMD is compatible with ZCMD syntax (CMD:), so there was no need for these two lines:

    #undef CMD
    #define CMD YCMD

For the same reason, you don't need the help parameter at all on the commands.

Just write all of the commands as you would with ZCMD. This will work with both iZCMD and YCMD. CMD:blah(playerid, params[])

[Y-Less]

True, but those won't affect the timings. I like the way you account for the speed of CallLocalFunction, although it isn't really needed as it will be a constant overhead for both versions. Otherwise, it looks good.

[Crayder]

True, but those won't affect the timings

I know, but a lot of people still don't know that YCMD is compatible with ZCMD syntax. I look at all the releases on the forums that say that they require ZCMD. All of those, unchanged, will also support YCMD and a couple other (not as great or well known) command processors. Yashas could've saved a bit of his work (he went a little overboard with those redefinitions) if he knew this.

[YashasSamaga]

I know that YCMD is compatible with ZCMD syntax, thats why I #undef CMD before defining it again. But I don't know why I had to do that lol

[Crayder]

You didn't have to redefine anything... :P

[YashasSamaga]

https://github.com/YashasSamaga/I-ZCMD/blob/master/beta-izcmd03.inc
Need to clean the code, looks messy!

I made a trie version,hashtable (FNV), and a sorted array version and the winner was sorted array of command names with a binary search.

I was also thinking about re-sorting the array such that all 1 letter commands come first,2 letter commands come next, 3 .. 4.. 5....

I observed something weird with packed strings, if I pack all the strings of an unpacked sorted array, the sorting is preserved. My bubble sort code never swaps anything. Was this by fluke? or it can be proved mathematically?

[Y-Less]

1. Why would packing affect the order at all?

2. Why are you using bubble sort?

3. You don't need both a start index and an end index - the end of one letter is surely the start of the next one. See fixes.inc for a fixed example:

https://github.com/ziggi/sa-mp-fixes/blob/066ff1a91c6522fe9273cf1ff728e10a89e2c3ab/fixes.inc#L474-L479
https://github.com/ziggi/sa-mp-fixes/blob/066ff1a91c6522fe9273cf1ff728e10a89e2c3ab/fixes.inc#L1322-L1327
https://github.com/ziggi/sa-mp-fixes/blob/066ff1a91c6522fe9273cf1ff728e10a89e2c3ab/fixes.inc#L2322-L2369
https://github.com/ziggi/sa-mp-fixes/blob/066ff1a91c6522fe9273cf1ff728e10a89e2c3ab/fixes.inc#L7898-L7964

Note that the link there is not from the latest commit in that repository as I don't think the latest commit was very good...

4. Use strcmp! It is faster than your character check method...

[YashasSamaga]

1. I am using [] subscript for packed_array. Jagged arrays, so the characters will be messed up. And for performing the binary search I am using the integer value of the packed_array.
2. Just in case packing would change the order (will improve the sorting later)
3. The start of another array can be -1 (If I need the start & end of commands starting from F, I cannot use the start of G because I won't know if there is a command which starts with the letter G , that means the start index of G would be -1)
4. A function name can have a maximum of 28 (after removing the command prefix "cmd_") characters, that means 7 characters for a packed array. 7 if checks is slightly faster than strcmp but commands are usually just 5-10 characters long so I would need only 3 checks which is way faster than strcmp.

By packing I can reduce the number of checks I need to do to compare two strings. I can compare 4 characters of two packed arrays in one shot by comparing the integer value of the cells of the packed arrays.

By the way strcmp is slower while comparing packed strings.

[YashasSamaga]

Aren't these two code equivalent in functionality?

loop: //code here

#emit LOAD.pri start
#emit LOAD.alt end
#emit JSLESS loop  //Jump to loop if PRI < ALT

if(start > end) goto cmd_invalid;
goto loop;

The #emit version doesn't work for some reason :/

If I check the assembly generated by PAWNCC for if(start > end) , it uses JSLEQ. :|

http://prntscr.com/7rrmyx

Why doesn't it use JSLESS?

[Y-Less]

a < b is equivalent to b <= a with the true/false branches swapped. If start and end are in pri and alt not alt and pri, then the check needs to be inverted, And I've never seen working #emit code with jumps in.

For the rest of your points, I would love to see the timings that show that 5-7 if checks in a complex loop are still more efficient than strcmp! I can MAYBE see 5 in a row being better, but yours aren't in a row - there's a lot of code between them.

As for the -1 start index. The way fixes.inc gets around that is the while (start != end) check, so the letters "E" and "F" both have the same start index, which is the start index for all the "F"s. When a library beginning with "E" is requested, the start and end indexes are thus the same and the loop instantly ends.

[YashasSamaga]

1. Yea, but why does the compiler invert? Why can't it simply use JSLESS when it exists. It can keep the meaning :p anyway who cares :p
   2)Here you go:
   http://prntscr.com/7rrz8p - The Test Code
   http://prntscr.com/7rs0gc - Test Results for equal strings "abcdefghij"
   http://prntscr.com/7rs0s3 - Test results for equal strings "abcdefghijklmnopqstuvwxzy" - Almost 2x faster
   http://prntscr.com/7rs15v - Test Results for unequal strings (abcDefghijklmnopqstuvwxzy & abcdefghijklmnopqstuvwxzy) - almost 5x faster

And its very unlikely that many commands have the same set of first few characters and the rest different. Even if such commands exist, the number of commands which differ in the first 4 characters itself will outnumber them. That means most of the times I would need only one check.

Moreover, I need to do manual checking to find out whether to go left or right (in sorted array) or else I need to rely on what strcmp returns and should compare the two characters at that location to decide where to go.

[YashasSamaga]

Damn, JSLESS is bugged, the jump code worked fine with JSLEQ. PAWN Bug? :v :v

This works fine!

#emit LOAD.pri start
#emit LOAD.alt end
#emit JSLEQ loop

goto cmd_invalid;

Where can I learn how to use amx_assembly? I only know to use the functions provided by the library and @ emit.

EDIT: Ah, JSLESS is perfectly alright , I was just stupid, JSLESS doesn't check if they are equal and it turned out by coincidence that my test code they were becoming equal :v :v

[Y-Less]

That code has exactly the bug I thought it would. You aren't testing real code, you are testing a fake version. So yes, that code may be faster, but that's because there is nothing between the comparisons. Your real code has other bits between them, such as:

https://github.com/YashasSamaga/I-ZCMD/blob/master/beta-izcmd03.inc#L308-L312

That makes a difference to the timings as well, because now you are doing that extra code multiple times per command instead of just once as you would with strcmp. Again, look at the code from fixes.inc - people forget what strcmp actually returns. It is not "are these strings the same", it is "what is the difference between these strings", and when they are the same that difference is zero. As a result, when they aren't the same the function returns a numerical representation of the difference, which is positive or negative depending on which string was bigger - the sign of that number instantly gives you your new direction.

[YashasSamaga]

Made a strcmp version (command names are unpacked).
Benchmarks:
strcmp version:600ms on average
no-strcmp version:470ms on average

strcmp version code
http://pastebin.com/gGz0a5h9