How "to call expand_term/2 explicitly" (per manual 8.17.2) about gprolog

[didoudiaz]

gprolog is currently not able to deal with term expansion in its general form. Indeed, term expansion has been designed for all-in-one Prolog systems which mainly assumes a top-level, a (byte-code) compiler written in Prolog and executed in the same current environment (ie. knowing all user-defined predicates) which asserts clauses as soon as read and compiled. This is an historic (and unfortunate) choice, which is restrictive (beyond term expansion). On the other hand, gprolog is based on a compiler which is a separate tool (called gplc - similar to gcc for C). Even when you consult a file, consult/1 invokes gplc. It is thus not obvious for this external tool to invoke a predicate defined in the current database, since both environments are separated. In addition, the compiler is mainly designed to produce native code (even if a byte-code also exists), not to execute it while compiling it ! For all these reasons, the general term expansion mechanism is not well suited for gprolog. IMHO, this is not well designed. For instance, in your case (for wordnet) what we need is a macro system for Prolog (e.g. as the C preprocessor which does not need to execute an arbitrary C code). Unfortunately, such a macro system does not exists and would 100% gprolog specific...

I continue thinking to see if I can find a solution for your specific case... (any suggestion is welcome)

[pmoura]

Logtalk's term-expansion mechanism answers one of the main points raised by Daniel:

(...) executed in the same current environment (ie. knowing all user-defined predicates) which asserts clauses as soon as read and compiled.

The idea that the results of term-expansion are immediately available for use in compiling following clauses in the same source file is problematic as it makes predicates order in a file significant and restricts Prolog systems implementations to interpret source files as script-like. Moreover, mixing term-expansion rules with the terms to be expanded in the same source file makes reuse and combining expansions (in expansion workflows; see e.g. https://logtalk.org/manuals/libraries/hook_flows.html) more difficult and less transparent. These and other issues are discussed at:

• https://logtalk.org/manuals/userman/expansion.html
• https://logtalk.org/blog.html?tag=term-expansion

But back to the specific problem at hand. The solution I suggested allows you to expand the files in the repo you forked and then commit the resulting files in your fork. This way, there will be no need to perform any term-expansion at runtime when loading the files. P.S. You can also use Logtalk to solve other portability issues but let's not go out of the scope of this issue here.

[pmoura]

At https://github.com/LogtalkDotOrg/wordnet I'm currently only providing pack manifest files for the public WordNet repos without any attempt to make their code more portable. As you're aware, the core files are portable (being mostly Prolog facts) and only some utilities predicates on top of them require porting/updating for broader compatibility across Prolog systems. Part of the problem is that what's a built-in predicate in a system (e.g. format/2) can be a library predicate in another system. This can be solved e.g. (for most but not all systems) using de facto standard conditional compilation directives and the dialect flag. There's also the alternative you mentioned of using different adapter files.

But back to the issue topic. Some Prolog systems (e.g. SWI-Prolog) view source files as script-like and compile them in one go while others use multiple pass compilation (e.g. Ciao). Daniel already described GNU Prolog approach. Logtalk is also multi-pass. Regarding the term-expansion mechanism itself, there's no standard for it (official or de facto) and there are significant differences between systems (when they actually provide some version of it; not all do). Logtalk provides the only available portable implementation.

[didoudiaz]

Sorry for the delay ! In the following I explain the new support for term_expansion/2 and show how to use it for wordnet.

In the wordnet specific use-case, there is an alternative solution based on an efficient rewriting of the critical Prolog code avoiding thus the use of the term_expansion/2 trick. I will propose it in the initial discussion.

Introduction to term_expansion/2 support

gprolog 1.6.0 has a support for term_expansion/2 (a pre-release version is available in the last commit). However, recall that in gprolog, the top-level environment is different from the compiler environment (which is an external tool, designed to be used on the command line, and invoked by consult). As a result, the compiler must be aware about term_expansion/2 clauses and its environment (auxiliary predicates which can be called by term_expansion/2, associated directives,...). By default, the gprolog compiler recognizes term_expansion/2 clauses and embeds them in its compiler environment (by asserting them).

This is enough for wordnet (for more complex cases, a new directive compiler_mode/1 is provided to fine-control what should be embeded and/or compiled). In the following, all commands are executed in the root of the project directory (i.e. by default wordnet-prolog).

First create a file wn_term_expans.pl in the root directory with the solution proposed for scryer Prolog. Recall, it speeds up the access to sk/3 when only the third argument is instantiated.

:- discontiguous(sk_1/3).
:- discontiguous(sk_3/3).

term_expansion(sk(X,Y,Z),
               [sk_1(X,Y,Z),
                sk_3(Z,X,Y)]).

sk(X, Y, Z) :-
        (   nonvar(X) ->
            sk_1(X, Y, Z)
        ;   nonvar(Z) ->
            sk_3(Z, X, Y)
        ;   sk_1(X, Y, Z)
        ).

Solution 1

Then modify prolog/wn_sk.pl to insert at the beginning the line:

:- include(wn_term_expans).

This works

1. Native-code compilation:

$ gplc -o sk_test1 prolog/wn_sk.pl 

This creates the executable sk_test1 which can be used to test if sk/3 with the third argument instantiated is fast:

$ ./sk_test1
GNU Prolog 1.6.0 (64 bits)
Compiled Jun 22 2023, 15:46:04 with gcc
Copyright (C) 1999-2023 Daniel Diaz

| ?- sk(X, Y, 'insufferably%4:02:01::').

X = 400519841
Y = 1

(1 ms) yes

2. Consulting under the top-level

GNU Prolog 1.6.0 (64 bits)
Compiled Jun 22 2023, 15:46:04 with gcc
Copyright (C) 1999-2023 Daniel Diaz

| ?- consult('prolog/wn_sk').
 yes

| ?- sk(X, Y, 'insufferably%4:02:01::').

X = 400519841
Y = 1

yes
(1 ms)

Both work. However, this requires to modify a file in the prolog subdirectory which is not ideal.

Solution 2

Pass the prolog/wn_sk.pl file at compile-time.

1. Native-code compilation:

$ gplc -o sk_test2 --include wn_term_expans.pl prolog/wn_sk.pl 
$ ./sk_test2
GNU Prolog 1.6.0 (64 bits)
Compiled Jun 22 2023, 15:46:04 with gcc
Copyright (C) 1999-2023 Daniel Diaz

| ?- sk(X, Y, 'insufferably%4:02:01::').  

X = 400519841
Y = 1

2. Consulting under the top-level
   For the specific case of wn_sk.pl use the new consult/2 built-in predicate as follows:

GNU Prolog 1.6.0 (64 bits)
Compiled Jun 22 2023, 15:46:04 with gcc
Copyright (C) 1999-2023 Daniel Diaz

| ?- consult('prolog/wn_sk', [include(wn_term_expans)]).
yes

| ?- sk(X,Y,'insufferably%4:02:01::').  

X = 400519841
Y = 1

If this second method is adopted, wordnet wn_load.pl must be adapted to detect when loading prolog/wn_sk.pl. For instance, replace all call to consult(F) by loadfile(F) defined as follows:

:- if(current_prolog_flag(dialect, gprolog)).

loadfile(F) :-
     (   decompose_file_name(F,_,wn_sk,_) ->
         Options = [include('wn_term_expans.pl')]
     ;   Options = []
     ),
     consult(F, Options).

:- else.

loadfile(F) :-
     consult(F).

:- endif.

[didoudiaz]

@dcnorris did you see other solution (in scryer-prolog issue) ? As I said there, I had to modify some code of your fork (to be less SWI-dependent). I can share it with you if you are still interested in updating your fork (you can contact me by mail didou.diaz at gm...).

[dcnorris]

Paulo Moura offered this suggestion on the Scryer Prolog forum. How does this compare with the macro system you envision?

[dcnorris]

Thank you, Paulo, for this further explanation. I had been approaching this on the view that wordnet-prolog is a very 'vanilla' application of Prolog, and accordingly had hoped that I might find at least 1 other Prolog implementation besides Scryer to which it could be adapted readily, with minimal changes to the codebase — say, by loading an implementation-specific adapter file ('scryer.pl', 'gprolog.pl', etc.) first. This way, wherever all the adapter files share the same code, I would have a rationale for pushing that change down to the originally forked code. (For example, both scryer.pl and gprolog.pl contain the same snippet:

apply(P, L) :- G =.. [P|L], G.

This, it seems to me, would 'recommend' making this transformation in the original codebase. And what I contributed back via PR would constitute an adaptation (again, of supposedly 'vanilla' code) to multiple engines and not merely to my favorite one. ;)

What I am understanding from this conversation is that my (traditional?) view of loading Prolog scripts, interacting at a toplevel, etc., is not universal — and maybe not even ideal. FWIW, I have failed today with yet another Prolog engine and may well give up this attempt and just admit that all I've done is adapted wordnet-prolog to Scryer.