imatix / gsl Goto Github PK

Just a wish -- all docs are in markdown format.

This will not only look good on github, but most importantly, it allows generating manpages, so that users can refer to gsl functions as easy as like man strcpy.

To allows generating manpages, the markdown should be in ronn-format (http://rtomayko.github.io/ronn/ronn.7.html).

generating manpages will then be as easy as

ronn < ggxml.md | gzip > man7/ggxml.7.gz

Ref: http://rtomayko.github.io/ronn/ronn.1.html.

The first phase can be as simple as get the title right, and leave all fancy editing/sectioning to future.

Is there a simple way to have gsl read from stdin instead of a given script file? If so, could you please document it?

I understand that I can use the workaround below to open stdin device and read from it:

define in = file.open("/dev/stdin", "r")
while 1
    string = file.read(in)
    gsl string
endwhile

This doesn't seem to work either:

$ gsl <(echo 'echo "test"')
2015/05/08 19:09:03: GSL/4.1a/github Copyright (c) 1996-2011 iMatix Corporation
2015/05/08 19:09:03: gsl/4 I: Processing /dev/fd/63...
2015/05/08 19:09:03: gsl/4 E: Error processing /dev/fd/63...
2015/05/08 19:09:03: File not found

Here is where it's missing: https://github.com/oranjuice/monero-0mq/blob/master/src/wap_proto.c#L602

The entire project with the xml protocol files are in: https://github.com/oranjuice/monero-0mq/tree/master/src

It uses zproto, and I'm not sure what was responsible for this.

Hi, I have have two functions to do code generation recursively on a directory.

But it looks like "gsl from my.filepath" has something wrong with directory.open, please help.

When I use "gsl file.slurp(...)" instead, it seems working fine.

function GenCode (dirname, filename)
    my.filepath = my.dirname + '/' + my.filename
    if regexp.match ('\(.*\)\.gsl$', my.filename, my.name)
        echo "GenCode: $(my.filepath:)"

        ######################################
        #gsl from my.filepath
        gsl file.slurp (my.filepath)
        ######################################

    elsif regexp.match ('.*~$', my.filename)
        # ignore slently...
    else
        echo "Ignored '$(my.filepath:)'"
    endif
endfunction

function GenCodeRecursively (dirname)
    echo "GenCodeRecursively: " + my.dirname
    my.dir = directory.open (my.dirname)
    for my.dir.file as f
        GenCode (my.dirname, f.name)
    endfor
    for my.dir.directory as sub
        GenCodeRecursively (my.dirname + '/' + sub.name)
    endfor
endfunction

Currently I can see no feasible way to output a block of multiple lines of text in a way that does not add trailing whitespace to all lines to match the length of the longest line.

I understand that many usages rely on having this trailing whitespace added, but In the interest of not having useless trailing whitespace all over the generated code (think comment blocks), it would be nice to have a way to opt out of that behavior.

Is there some formatting option or workaround already that I'm not aware of?

Something that I am using quite a lot with my models is the ability to include XML files into others. I was usually doing it in hand-writing every time a GSL script to include a specific file (similarly to the example provided by the GSL guide) but honestly that sucks ;-) and I guess it's not in the MOP spirit. That's why I created a recursive function that includes every single node named include with an attribute file.

Here's an example of model in XML including other XML files:

script.gsl:

.template 0
process_inclusion()
.endtemplate

project.xml:

<project script="script.gsl">
  <include file="file1"/>
  <include file="file2"/>
</project>

file1.xml:

<file1_markup/>

file2.xml:

<file2_markup/>
<file2_markup/>
<file2_markup/>
<include file="file3/file3"/>

file3/file3.xml:

<file3_markup/>
<file3_markup/>

Here is the actual source code of the function:

.template 0

function global.process_inclusion(context)
  my.context ?= project

  my.found = 1
  while my.found
    my.found = 0
    for my.context.include as i
      # include file [i.file].xml
      xml to my.context from "$(i.file:).xml"
      delete i
      my.found = 1
    endfor
  endwhile

  for my.context. as ii where ii.name()?"include" <> "include"
    process_inclusion(ii)
  endfor
endfunction

.endtemplate

Another solution could be to use the XInclude XML inclusion mechanism, here is the w3c rfc: http://www.w3.org/TR/xinclude/

When I enter a CDATA section gsl throws the following error, e.g.

2014/04/25 10:40:07: telegram.xml 11: <![
'--' or DOCTYPE declaration expected

Suppose I have an XML block with enumerated values of different lengths:

$ cat enum.xml
<code script="enum.gsl">
    <enum name="fruit" script="enum.gsl">
        <value code="1" val="orange"/>
        <value code="200" val="kiwi"/>
    </enum>
</code>

And I want to generate aligned code:

enum class fruit {
    orange =   1,
    kiwi   = 200
};

Is there a way to enforce this proper alignment in the generated code without manually having to pad the values?

https://github.com/imatix/gsl/blob/master/src/ggpcre.c#L129

    rc = pcre_exec (re,
                    NULL,
                    subject-> value. s,
                    (int) strlen (subject-> value. s),
                    0,
                    0,  /* This is the options parameter in the pcre call. */
                    ovector,
                    oveccount);

Some other tools such as the Sphinx document generator implements a cache process that avoid rebuilding every time every file if some remain unchanged, since GSL doesnt have any caching process, that results in creating again and again every time the same files without checking if files were even been changed, that why I hacked a little caching system that only operates at GSL post-processing stage, once files are generated. It will be nice to see such a feature as another built-in GSL class.

Basically, the following functions can be used in-place of the usual .output .close GSL directives:

• cache_open(FILENAME [, FILEPATH [, BUILDDIR [, CACHEDIR ] ] ] ) -> FILE_CACHED
• cache_close(FILE_CACHED [, BUILDDIR [, CACHEDIR ] ] )

that use the samefiles(f1, f2) function in order to compare the content of two files thanks to the file.slurp and string.hash builtin functions.

Here is a basic usage of these functions:

.template 1
.
.cached = cache_open('helloworld.txt')
Hello \
World
.cache_close(cached)
.
.endtemplate

Calling this code twice will only create the build/helloworld.txt once while the build/.cache/helloworld.txt will be built twice.

And here is source code:

.template 0

global.builddir = 'build/'
global.cachedir = 'build/.cache'

directory.create(global.builddir)
directory.create(global.cachedir)

function global.check_arg_missing(ctx, key, value)
  if !defined(my.value)
    abort '[$(my.ctx)] arg "$(my.key:)" is missing'
  endif
endfunction

function global.samefiles(f1, f2)
  check_arg_missing('samefiles', 'f1', my.f1)
  check_arg_missing('samefiles', 'f2', my.f2)

  if !file.exists(my.f1)
    # my.f1 file doesnot exist
    return 0
  endif

  if !file.exists(my.f2)
    # my.f2 file doesnot exist
    return 0
  endif

  my.err = ''
  my.cnt1 = file.slurp(my.f1, my.err)?

  if !defined(my.cnt1)
    # Error: my.err
    return 0
  endif

  my.err = ''
  my.cnt2 = file.slurp(my.f2, my.err)?

  if !defined(my.cnt2)
    # Error: my.err
    return 0
  endif

  my.h1 = string.hash(my.cnt1)
  my.h2 = string.hash(my.cnt2)

  return my.h1 = my.h2
endfunction

function global.cache_open(fn, path, bdir, cdir)
  my.cdir ?= cachedir
  my.bdir ?= builddir
  my.path ?= ""

  if my.path <> ""
    directory.create("$(my.bdir:)/$(my.path:)")
    directory.create("$(my.cdir:)/$(my.path:)")
    my.fn = my.path + '/' + my.fn
  endif

  # create and cache file my.fn
  output "$(my.cdir:)/$(my.fn:)"

  return my.fn
endfunction

function global.cache_close(fn, bdir, cdir)
  my.cdir ?= cachedir
  my.bdir ?= builddir

  close

  my.src = my.cdir + '/' + my.fn
  my.dst = my.bdir + '/' + my.fn

  if !samefiles(my.src, my.dst)
    # create file my.fn
    file.delete(my.dst)
    file.copy(my.src, my.dst)
  else
    # file my.fn remains unchanged
  endif
endfunction

.endtemplate

I may be misinterpreting this example from README.md, but I despite attempting several variations I can't seem to make it do anything.

function assign (dest, source)
    $(my.dest) = my.source
endfunction

A similar variant succeeds in changing my.dest (only internal to the function):

function assign (dest, source)
    assign.dest = my.source
endfunction

It seems like the intent is to pass the value change back to the calling scope, which would be very useful.

Can gsl now handle UTF-8 gracefully?

I saw "GSLGen and UTF-8" @
http://exold.com/article/gslgen-and-utf-8

but wondering what's the latest status of gsl on UTF8.

Thanks

Hi,

First of all sorry for posting a question here, but I thought chances for an answer are better here than on Stackoverflow.

As I am learning Rust currently, I decided to implement a commandline youtube uploader. To do that, one will need access to the Youtube Web API. When looking at the google API implementation for Go, I realised that it is nearly entirely generated. The generator is written in Go, reading structured data in Json format.

Even though I am implementing the first sketch manually, I want to go ahead and generate all the rest, based on the experiences I made by hand.

Now the question: If you are looking at the API description, do you think (if converted to XML), GSL will be the tool to generate valid Rust ? I believe it should be possible from what I have read so far, but I want to be sure.

Generally, besides traversing the XML tree, I will have to map names (like 'boolean' to 'bool'), and possibly access XML structures by name, as parts of the XML tree softly reference each other. The name-mapping part should be doable using functions/macros, but I am not entirely sure about XML access by name (which is stored in a variable).

If you give me a thumbs up, I will be happy to dive into GSL, as I have been wanting to use it for quite a while now ! A code generator is missing in my tool-set, and I totally see the point when implementing protocols and basically anything that can be described by structured data.

Thank you !

GSL is crashing on OSX and linux x64 when trying to compile the example program:

.template 0
 amount = 1000
 year = 2006
 while year < 2026
     amount = amount * 1.05
     year = year + 1
 endwhile
 echo amount
.endtemplate

Steps to reproduce

You can reproduce this problem easily on OSX or linux AMD 64 provided you have

• /usr/bin/python
• make

If you are on another system, it will still work provided you have GSL in your path and are willing to adjust the makefile.

git clone https://github.com/Byron/youtube-rs
cd youtube-rs
git checkout f2ca8c3
make api-deps

Which yields

✗ make api-deps
./etc/bin/gsl_OSX-AMD64 -q -script:src/gsl/deps.gsl ./etc/api/shared.xml
Fatal error in application - aborted
Abort.  Compile with SMT_DETECT for more info.
make: *** [.api.deps] Abort trap: 6

Meta

✗ gsl -v
GSL/4.1a/github
Production release
Copyright (c) 1996-2011 iMatix Corporation
Compiler: gcc -c -O2 -D_REENTRANT -D_GNU_SOURCE -Wall -Wno-unused -fno-strict-aliasing -DBASE_THREADSAFE -I. gsl.c

Install GNU Make and GNU Compiler. CLANG at the moment will leave you bewildered and in tears as it fails to find libraries to link. I don't know enough about the ./c script to puzzle out how to make this work "naturally" on FreeBSD.

Edit Makefile and add "-lm" to src/Makefile where you see CCLIBS configured. That is, it should look like:

export CCLIBS = -lpcre -lm

Cd to src and run:

CCNAME=gcc47 gmake
gmake install

I am trying to use gsl_integration_qags in a cython code where I use gsl inside a class and it uses an instance of the class to compute the integration but apparently it is not very strightforward to use it in a class. In the following it is my written code:
import cython
cimport cython
import numpy as np
cimport numpy as np
from cython_gsl cimport *
from math import *
cdef extern from "math.h":
double log(double x) nogil

ctypedef double * double_ptr
ctypedef void * void_ptr
from math import *
cdef class Foo(object):
def init(self, double a=1.2, double b=0.6):
self.a = a
self.b = b
def integrand(self,double x) nogil:
cdef double self.a = (<double_ptr> params)[0]
cdef double self.b = (<double_ptr> params)[1]
return self.a_log(x)+self.b/x*3
def whole(self, double upper_limit=10,double lower_limit=0):
cdef gsl_integration_workspace * w
cdef double result, error, expected, alpha
w = gsl_integration_workspace_alloc (1000)
expected = -4.0
params[0] = self.a
params[1] = self.b
cdef gsl_function F
F.function = &self._integrand
F.params = params

    gsl_integration_qags (&F, lower_limit,upper_limit , 0, 1e-7, 1000, w, &result, &error)
    return result,error

the error message when I wanted to build it was as following :

Cannot assign type double (Foo, double, void ) nogil' to 'double ()(double, void *) nogil

Could anybody point out what is the problem and how could I solve it?

In my travels through the GSL code, I discovered that directory iteration using a file entry object stops when there's a problem with a file
in the directory, but gsl does not indicate that an abnormal event occurred. Actually, that's not entirely true. If the problematic file is chosen as the first file entry, the open call will fail. But even in that case error is not set. That can be fixed easily, however. My larger concern is when the file is not the first file entry. I see that the error occurs in the chain of calls:

file_entry_next_sibling -> read_dir -> populate_entry

When populate_entry calls stat(2), it discovers the problem and punts. As a result, a problem looks exactly the same as readdir returning NULL.

Initially, I thought of just calling readdir again, similar to what happens for '.' and '..' entries, but that's just masking the problem. I think it should be a fatal error instead.

Since most problems are system related, we could check errno in file_entry_next_sibling
but I don't see how to gracefully send the message back.

It seems to me that if the directory and file entry classes are presenting an XML interface,
they should actually read all files at the start and validate them, just like any other XML input. This would cause any problems to appear during directory.open, where there is a mechanism to report and handle errors. Obviously, this would also use more memory, but I don't think this would be problem.

Here's a script showing the problem: https://gist.github.com/gyepisam/4739863

On my system, gsl will report fewer files than actually exist.

Regards

-Gyepi

Hello,

We are getting a random crash on windows only, the crash never occurs in debug builds, only release builds. I have never been able to repro it on my machine, it happens on some of our build machines, randomly, the GSL file causing the crash is also random.
Here is the SMT_DETECT ouput : Abort at GGSCRP:1: (0, 13)

Max

Sometime since commit b260401 the following script breaks:

.#  lookup_code_generator.gsl
.#
.#  Generates code files for lookups from the messaging model
.#
.gsl from "/project/doc/common_functions.gsl"
.#
.#

The command passed to gsl is:

gsl -script:lookup_code_generator /project/doc/model/message_model.xml

The output is:

2015/03/03 13:20:58: GSL/4.1a/github Copyright (c) 1996-2011 iMatix Corporation
2015/03/03 13:20:58: gsl/4 I: Processing /project/doc/model/message_model.xml...
2015/03/03 13:20:59: (lookup_code_generator.gsl 5) File:  not found

Note that there are two spaces after File:. Looking at the commit log, I suspect the name of the file is being freed before it can be used, but I haven't tried to debug it.

Broken with: 91b9c6d
Last known good commit: b260401

Hi all,

All is in the subject :)

This is called after formulating the proper directory.

curpath = strip_file_name (pathname);

Given that the function is designed to accept a directory and not a file name, this always produces an incorrect result.

https://github.com/imatix/gsl/blob/master/src/ggfile.c#L1286

From the README

To change modes, set the value of the identifier ignorecase in the gsl scope to 0 or 1.
Eg: `[gsl].ignorecase = 0'

Attempting this with the gsl. prefix yields an "unexpected token" error (pointing after the zero).

gsl.ignorecase = 0

Attempting this without the gsl. prefix does not error but doesn't appear to produce the expected result.

ignorecase = 0

function test_variable_case_sensitivity()
    define foo = "Case-sensitive variables!"
    define Foo = "Case-INsensitive variables!"
    echo foo
endfunction

test_variable_case_sensitivity()

Case-INsensitive variables!

I've been trying to figure this out for some time. Poured over the doc, even tried the source and tests. But I haven't been able to figure out how to pass anything but a scalar to a function or macro. As a simple test case I'd like to be able to reimplement the count() function. If this is possible I'd gladly take the task to update the doc.

Hi all,

I have some problem with scope and internal count function.
I want to enumerate and copy to 'z' all distinct 'y'.
Here is my testcase of dernormalization :

xml to root <<.
<z>
<x n="x1"><y n="y1" /></x>
<x n="x2"><y n="y2" /></x>
<x n="x3"><y n="y1" /></x>
<x n="x4"><y n="y2" /></x>
</z>
.
for z
    for x
        echo "x=" + n
        for y
            echo "  y=" + n
            c = count (z.y, count.n = y.n)
            echo "  c=" + c
            if c = 0
                copy y to z
            endif
        endfor
    endfor
endfor
root.save("testcount.xml")

and the result which is not expected !

GSL/4.1a/github Copyright (c) 1996-2011 iMatix Corporation
gsl/4 I: Processing testcount.gsl...
gsl/4 M: x=x1
gsl/4 M:   y=y1
gsl/4 M:   c=0
gsl/4 M: x=x2
gsl/4 M:   y=y2
gsl/4 M:   c=1
gsl/4 M: x=x3
gsl/4 M:   y=y1
gsl/4 M:   c=1
gsl/4 M: x=x4
gsl/4 M:   y=y2
gsl/4 M:   c=1

I do not understand these results !
But when I change scope for 'y' with alias 'yy', all is fine :

for z
    for x
        echo "x=" + n
        for y as yy
            echo "  y=" + n
            c = count (z.y, count.n = yy.n)
            echo "  c=" + c
            if c = 0
                copy yy to z
            endif
        endfor
    endfor
endfor

I get the correct result :

gsl/4 M: x=x1
gsl/4 M:   y=y1
gsl/4 M:   c=0
gsl/4 M: x=x2
gsl/4 M:   y=y2
gsl/4 M:   c=0
gsl/4 M: x=x3
gsl/4 M:   y=y1
gsl/4 M:   c=1
gsl/4 M: x=x4
gsl/4 M:   y=y2
gsl/4 M:   c=1

I do not understand why there is such scopes problems?
Thanks for any explanation.

needs -lpcre

Can we use install instead of cp in the makefile for installation?

I'm planning to package gsl as a Debian package. Using install instead of cp will make it one-issue simpler in Debian packaging. I can volunteer to make the changes.

thanks

Consider:

# test.gsl
output "foo"
template 1
a
b\
.endtemplate
close
output "bar"
template 1
c
d
.endtemplate

My goal is to suppress the final line ending in the file foo. Per documentation:

if the last character of an template line is a backslash then the line is output with no line terminator

Alas, this doesn't work. The b is carried over to bar, its contents looking as follows:

bc
d

It gets worse: when the output is fed to just one file, we have a crash:

# crash.gsl
output "foo"
template 1
a
b\
.endtemplate
close

function recursive (N)
    recursive.localvar = my.N - 1
    recursive (localvar)
endfunction

The above example from the README.md produces error "Undefined expression: my.N". I tried a few variations but could not achieve a working example.

I would like to build rpm and distribute it on fedora 20, epel6 and epel7 repositories. I found out that name "gsl" is already reserved

gsl.x86_64 : The GNU Scientific Library for numerical analysis

What would be alternative name for imatix/gsl? gslcg (gsl code genetator)?

I don't know how to send in the pull request this time, because
https://help.github.com/articles/using-pull-requests
does not cover the case of what if I just want to send in a specific change. I.e., the
suntong/gsl@c2ff940...d81c798
one, without the preceding changes I made to the example directory.

The change is quite simple, as in
suntong@d81c798

I'll submit a pull request anyway, if you don't mind polluting your example directory with my test cases. Feel free to ignore it.

This has been making my code more complex than I would like. In order to demonstrate the issue I've set up the following test case:

# Simulate a case where a scope predicate may not find a node.
function defined_if_positive(value)
    if (my.value > 0)
        return my.value
    endif
endfunction

function test_undefined()
    define test1 = 1
    if (defined(test1))
        echo "test1: $(test1)"
    endif

    if (!defined(test2))
        echo "test2: undefined"
    endif

    if (defined(defined_if_positive(0)))
        echo "test3: $(defined_if_positive(0))"
    else
        echo "test3: undefined"
    endif

    if (defined(defined_if_positive(4)))
        echo "test4: $(defined_if_positive(4))"
    else
        echo "test4: undefined"
    endif

    # Test4 is the resolution to the break in this idiom,
    # but as a result the function must be executed twice.
    test5 = defined_if_positive(0)
    if (defined(test5))
        echo "test5: $(test5)"
    else
        echo "test5: undefined"
    endif
endfunction

test1: 1
test2: undefined
test3: undefined
test4: 4
(sandbox.gsl xx) Undefined expression: defined_if_positive(...)

Notice that the result of a function cannot be assigned to a variable in the case where the result is undefined. However variables are capable of being undefined and can be tested as to whether they are defined using defined(<variable>). Also the result of an expression that is undefined can be evaluated and passed to defined(<expression>).

But an undefined expression cannot be assigned to a variable. This results in the need to twice execute any function that may produce undefined. It is not always possible to avoid returning undefined in a case where the defined value is required. This is an example from actual use.

function get_archive_version(repository, name, system)
    define repo = get_archive_version.repository

    # GSL can't assign an undefined value to a variable.
    if (defined(repo->dependency(is_match(dependency, my.name, my.system))))
        return repo->dependency(is_match(dependency, my.name, my.system)).archive
    endif

    # GSL can't assign an undefined value to a variable.
    return ""
endfunction

Note that it's the predicated query that may or may not return defined, which is simply a function of the XML data. So the query needs to be executed twice. And then there's the seam at the second-to-last line where an empty string must be returned to avoid perpetuating the issue.

It would be really nice to learn I'm missing something, but if not it would be really nice for a variable to be assigned as undefined. It seems straightforward, since this is the initial state of any symbol in the language, as the test code above shows.

Something that I am quite familiar with other languages such as C++ or Python is the use of list or vector container. I saw on GSL one can use XML data structure instead of that, but sometimes it is even more efficient and reduces drastically code length using a list mechanism since XML can be much more seen as map or dictionary than a single list container.

Therefore, I created a small library of functions that implements functions for such a purpose here is the list of these functions:

• list_new: create a new list
• list_exists: check if the passed argument is a list
• list_empty: does the list empty ?
• list_append: append a new item into the list
• list_get: get the i^th item of the list
• list_delete: delete the i^th item of the list
• list_foreach: apply a callback function to every single item in the list
• list_print: standard output the whole list

It is also possible to add a list into another. Here is a GSL script showing a typical usage:

.template 0

l1 = list_new()
l2 = list_new()

list_append(l2, 'aaa')
list_append(l2, 'zzz')

list_append(l1, l2)
list_append(l1, 'bbb')
list_append(l1, 'ccc')

define l2.dummy = 12
list_append(l2, 'www')

list_append(l1, l2)
list_append(l1, 'ddd')
list_append(l1, 'eee')
list_append(l1, 'fff')

list_delete(l1, 0)

list_print(l1)

echo list_get(l1, 1)

.endtemplate

And the result:

gsl/4 M: [[aaa,zzz],ccc,[aaa,zzz,www],ddd,eee,fff]
gsl/4 M: ccc

I don't know if anyone see useful to add them as a builtin GSL feature.

Anyway, here is the source code:

.template 0

function global.check_arg_missing(ctx, key, value)
  if !defined(my.value)
    abort '[$(my.ctx:)] arg "$(my.key:)" is missing'
  endif
endfunction

function global.list_new
  return XML.new('list')
endfunction

function global.list_exists(list)
  check_arg_missing('list_exists', 'list', my.list)

  return name(my.list)?"" = "list"
endfunction

function global.list_empty(list)
  check_arg_missing('list_exists', 'list', my.list)

  return !defined(my.list->.)
endfunction

function global.list_append(list, item)
  check_arg_missing('list_append', 'list', my.list)
  check_arg_missing('list_append', 'item', my.item)

  if list_exists(my.item)
    copy my.item to my.list as list
  else
    new my.list.item as i
      define i. = my.item
    endnew
  endif
endfunction

function global.list_get(list, i)
  check_arg_missing('list_get', 'list', my.list)
  check_arg_missing('list_get', 'i', my.i)

  for my.list. as ii where item() = my.i+1
    return ii
  else
    abort '[list_get] no item found @ $(my.i:)'
  endfor
endfunction

function global.list_delete(list, i)
  check_arg_missing('list_delete', 'list', my.list)
  check_arg_missing('list_delete', 'i', my.i)

  my.item = list_get(my.list, my.i)
  delete my.item
endfunction

function global.list_foreach(list, callback, record)
  check_arg_missing('list_foreach', 'list', my.list)
  check_arg_missing('list_foreach', 'callback', my.callback)

  my.record ?= XML.new('record')

  for my.list. as i
    $(my.callback)(i, my.record)
  endfor

  return my.record
endfunction

function print_callback(item, record)
  check_arg_missing('print_callback', 'item', my.item)
  check_arg_missing('print_callback', 'record', my.record)

  if list_exists(my.item)
    my.sublist_record = list_foreach(my.item, 'print_callback')
    copy my.sublist_record to my.record
  else
    new my.record.field as f
      define f. = my.item.
    endnew
  endif
endfunction

function recursive_print(record)
  check_arg_missing('recursive_print', 'record', my.record)

  my.s = "["

  for my.record. as i
    if name(i)?"" = "record"
      my.s += recursive_print(i)
    else
      my.s += i.
    endif

    if !last(i)
      my.s += ','
    endif
  endfor

  my.s += "]"

  return my.s
endfunction

function global.list_print(list)
  check_arg_missing('list_print', 'list', my.list)

  my.record = list_foreach(my.list, 'print_callback')
  echo recursive_print(my.record)
endfunction

.endtemplate

Could someone please explain the behavior of class() so that I can test and document it? This is the current documentation:

class (item)
    To be explained.

Hi all,

In previous gsl version, there where a Zip class with, for example, methods like zip.new(), zip.add(), zip.close().

But it seems that this class is no longer there.
Can anyone confirm this?
Thanks.

I'm not a first class C citizen, but I think it's really hard to compile GSL. First I thought it's as easy as "just use a UNIX box and do make". Yes, I'm "the guy who loves his Windows box", but I used an ubuntu to try it; it is not that "just" easy. Also with MSVC (my oldest Version is 10.0 / 2010 express) I just don't get it compiled. A hell of batch files are looking for VC98 compilers, deprecated options etc. With the current compiler it doesn't get the prelude.h and if I'm inserting the include hard into gsl.c I get a ton of new problems.

I just wanted to try GSL. After reading ZeroMQ The Guide, having an eye on the RFCs, and actually being impressed how simple all that looks (even in C), and how simple all that works (bound to c#) - I'm now trying to "get into" GSL.
But now I ended up in wasting hours of studying batch files, C compilers, arguments, linking problems and incompatible makefiles/project files and whatever.
Can you please provide downloadable binaries (Win32/UNIX x86/amd64)?

Hi there,

I was wondering if there is a magic trick in GSL to create things like namespaces, I saw the built-in classes already benefit from some. I've tried to use scopes out of a function in order to emulate a namespace but the syntax does not match!

Awesome project btw!
Caner

I noticed that some docs have two versions, .txt and .md.
I presume that their content is the same, but for further maintenance purpose, and the DRY principle, there will be time that we drop one format and keep only one version (preferable .md).

Hi again,

This bug was hard to demonstrate. I've tried to scale down my script as much as possible, but it's still a bit big though. Please check out memory-bug.gsl & memory-bug.xml from https://github.com/suntong001/gsl/tree/master/doc/examples.

This is how it runs:

$ gsl4 -script:memory-bug.gsl memory-bug.xml > /dev/null 
2013/12/26 17:47:41: GSL/4.1a/github Copyright (c) 1996-2011 iMatix Corporation
2013/12/26 17:47:41: gsl/4 I: Processing memory-bug.xml...
Clean-memory assertion failed - <Unknown> (0)
Details are in memtrace.xml
Aborted

$ cat memtrace.xml
<?xml version="1.0"?>
<transaction>
<block index="0" size="16" file="<Unknown>" line="0" pointer="0x10fb7c0" />
</transaction>

This is the best I can do to scale down my script. I know that if I remove either the following part, the Clean-memory assertion failed problem will be gone:

• the for loop in gsl
• the if statement after the for loop in gsl
• the Internal getopt error line from .xml

Please investigate.
Thanks

Hi there,

Just to announce that I just released the project 4i entirely based on GSL. The later enables to model-oriented program entirely thanks to modules. You can build your new module from-scratch or extend an existing one. Some primitive modules have been released as well to ease its adoption.

You can find the main tool there: https://github.com/4geit-tool/4i
And the resources such as the modules there: core and there: module.

I am actually working on a documentation generator that way every single module can benefit from it.

Here is the common "hello world" example in action: https://github.com/4geit-project/hello
It's basically based on the module 4i.language.python.hello which extends the module 4i.language.python which extends the module 4i.language which extends the root module 4i.

And an asciicast demo to show you quickly how the hello world project is build with it: https://asciinema.org/a/17555

Every module created on 4i is based on the root module, that means they have to be compliant with a minimum rules of the root module (protocol).

It's is still in an early stage of development (sorry about the lack of the doc) however I am already using it for my own project, so feel free to ask me any questions, concerns or feedback you have. Thanks.

I'm encountering multiple problems attempting to use the latest build of GSL on Windows to enumerate files and/or directories.

Using the documentation as an example:

dir = directory.open(".", error_text) ?

if defined(dir)
  for dir.file as f
      echo "file:$(f.name) has size: $(f.size)"
  endfor
else
    abort "Error: " + error_text
endif

The run fails because f.name (and f.size) is undefined. But further trials also shows:

• Based on the size of the enumerations, it's clear that all files and directories are enumerated (despite the dir.file qualification). This gives the same length enumeration as dir. even when there is a mix of files and directories in the opened directory.
• When dir.directory is qualified, no directories (or files) are ever enumerated, the enumeration is always empty.
• The enumerated objects (file and path) do define a path attribute, but it is always empty.
• No documented attributes other than path are defined on returned files or directories.

This last item is not technically a break, but would make portable code hard, because presumably this is a behavior specific to Windows. But note that other path references in GSL on Windows don't require forward slashing:

• The value of dir.path is correct in the case where the opened directory ends with "" but not when it ends with "/" (or when it is not terminated).

../libbitcoin/include  => x:\xxxxx\libbitcoin/
../libbitcoin/include/ => x:\xxxxx\libbitcoin/
../libbitcoin/include\ => x:\xxxxx\libbitcoin\include/

I'm really stuck at this point. I looked through the GSL source, but it's going to be a slog for me. Any help would be much appreciated!

Getting this on Travis and locally (Ubuntu). I reverted the most recent commit locally and it's OK.

$ gsl -q generate.xml
Fatal error in application - aborted
Abort.  Compile with SMT_DETECT for more info.
Aborted (core dumped)

To make the XML parsing more efficient I suggest to add an optional XML schema validation process attached to a hand-written XSD file before GSL works on it. xmllint is doing it quite fine and this is actually what I am using before calling gsl.

<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
              xsi:noNamespaceSchemaLocation="project.xsd"/>

To accomplish that efficiently you can benefit from the XMLSchema parameters such as xsi:schemaLocation or xsi:noNamespaceSchemaLocation.

Another feature that I would like to see as a builtin feature in GSL is XML node referencing. I have started to create much more complex model structures and one should avoid redundancy in MOP methods though, there are still situations when it occurs. Let's provide an example to illustrate the basic idea:

<project script="script.gsl">
  <component name="componentA">
    <element name="AAA"/>
    <element name="BBB"/>
    <element name="CCC"/>
  </component>

  <component name="componentB">
    <element name="AAA"/>
    <element name="BBB"/>
    <element name="CCC"/>
  </component>
</project>

In such case, a naive optimization one can approach to avoid the redundant component node is to identify the node component once, let's say using an id attribute and each time we create a component node with the same structure just references to the identified one thanks to a ref attribute. Let's provide an example of the meaning:

<project script="script.gsl">
  <component id="mycomponent">
    <element name="AAA"/>
    <element name="BBB"/>
    <element name="CCC"/>
  </component>

  <component ref="mycomponent" name="componentA"/>
  <component ref="mycomponent" name="componentB"/>
</project>

In this example, I removed the duplicated component node as well as the name attribute and created two other nodes with the respective names and referencing both to the component identified as mycomponent. The attribute name of the new nodes are only defined at their own level.

More interestingly, I'd like to reference to single nodes that were created in another level or context in my XML model as illustrated in the following example:

<project script="script.gsl">
  <component id="common">
    <component id="mycomponent">
      <element name="AAA"/>
      <element name="BBB"/>
      <element name="CCC"/>
    </component>
    …
  </component>

  <component id="module1">
    <component ref="common.mycomponent" name="componentA"/>
    <component ref="common.mycomponent" name="componentB"/>
  </component>
</project>

In the previous example, I illustrated how to refine the granularity of component nodes using several encapsulated component nodes or just organize them in a way if there are many nodes, they can be identified easily. To solve such a problem and as you may notice, the ref attributes can handle paths in almost a similar way than XPath mechanism's doing. In the example, I am using common.mycomponent to reference to the component node located in the component with the common id and component with the id mycomponent.

One can argue that GSL scripting can fix such a purpose but that's not the point, the point is that I dont want to resort to GSL each time I want to avoid redundancy issue within my XML models. That's why I introduced a few new functions:

• get_node( PATH, NODENAME, INIT )
• get_node_attempt( PATH, NODENAME, CONTEXT )
• get_node_name( NODE, NODENAME )
• get_node_id( NODE, DELIM, NODENAME )
• get_node_attribute( NODE, ATTRIBUTE, DELIM, NODENAME )
• get_node_length( NODE, NODENAME )
• print_node( NODE, NODENAME )
• override_attribute( N1, NN, ATTRIBUTE )
• append_attribute( N1, NN, ATTRIBUTE )
• override_reference( N1, ORIGIN, NODENAME, CONTEXT )
• replace_by_reference( N1, N2, NODE_SRC, NODE_DST, CONTEXT )
• process_references( NODE_SRC [, NODE_DST [, CONTEXT] ] )

But the most important one is:

process_references( NODE_SRC [, NODE_DST [, CONTEXT] ] )

process_references will recursively walk across every nodes that matches to a reference (attribute ref) and a specified node name.

Using the previous XML example, here is a script that actually processes every component that references to other component nodes:

.template 0
process_references('component')
.endtemplate

That was an easy model but what about handling this one:

<project script="script.gsl">
  <component id="common">
    <component id="mycomponent">
      <element name="AAA"/>
      <element name="BBB"/>
      <element name="CCC"/>
    </component>
    …
  </component>

  <component id="module1">
    <division ref="common.mycomponent" name="componentA"/>
    <division ref="common.mycomponent" name="componentB"/>
  </component>
</project>

In such case, I am still referencing to the same component mycomponent but this time it's done from a division node. To accomplish that, here is the script code:

.template 0
process_references('component', 'division')
.endtemplate

Caution: you cannot reference to a node that was not previously defined yet, this could be an enhancement to do that way one doesn't need to focus on the node placement anymore.

The ref attribute can handle a relative path or an absolute path. The both previous examples illustrated these kind of path. The function will first try to find a relative path id and if it doesn't it looks for an absolute path.

In the relative case, one can use : in order to go up to the parent node, for instance:

    …
    <division ref=":common.mycomponent" name="componentA"/>
    …

Nota: It is also possible to use name attribute instead of id to reference to a node.

And here is the effective source code:

.template 0

function global.get_node(path, nodename, init)
  my.init_path = my.path
  my.init ?= project
  my.xml = my.init

  while my.xml <> project & string.substr(my.path, 0, 0) = ":"
    my.xml = my.xml.parent()
    my.path = string.substr(my.path, 1)
  endwhile

  while string.cntch(my.path, '.')
    my.prefix = string.prefix(my.path, '.')
    my.xml = my.xml->$(my.nodename:)((defined(id) & id = my.prefix) | (defined(name) & name = my.prefix))?
    my.path = string.substr(my.path, string.length(my.prefix)+1)
  endwhile

  my.xml = my.xml->$(my.nodename:)((defined(id) & id = my.path) | (defined(name) & name = my.path))?

  return my.xml
endfunction

function global.get_node_attempt(path, nodename, context)
  my.res = get_node(my.path, my.nodename, my.context)?

  if !defined(my.res)
    my.res = get_node(my.path, my.nodename)?
  endif

  if !defined(my.res)
    error('$(my.nodename:) "$(my.path:)" not found')
  endif

  return my.res?
endfunction

function global.get_node_name(node, nodename)
  my.nodename ?= name(my.node)

  my.s = "$(rename(my.node.name))"
  my.node = my.node.parent()
  while defined(my.node) & my.node.name() = my.nodename
    my.s = "$(rename(my.node.name)).$(my.s:)"
    my.node = my.node.parent()
  endwhile

  return my.s
endfunction

function global.get_node_id(node, delim, nodename)
  my.nodename ?= name(my.node)
  my.delim ?= '.'

  my.s = "$(my.node.id:)"
  my.node = my.node.parent()
  while defined(my.node) & my.node.name() = my.nodename
    my.s = "$(my.node.id:)$(my.delim:)$(my.s:)"
    my.node = my.node.parent()
  endwhile

  return my.s
endfunction

function global.get_node_attribute(node, attribute, delim, nodename)
  my.nodename ?= name(my.node)
  my.delim ?= '.'
  my.attribute ?= 'id'

  my.s = "$(my.node.$(my.attribute:):)"
  my.node = my.node.parent()
  while defined(my.node) & my.node.name() = my.nodename
    my.s = "$(my.node.$(my.attribute:):)$(my.delim:)$(my.s:)"
    my.node = my.node.parent()
  endwhile

  return my.s
endfunction

function global.get_node_length(node, nodename)
  my.nodename ?= name(my.node)

  my.count = 0
  my.node = my.node.parent()
  while defined(my.node) & my.node.name() = my.nodename
    my.count += 1
    my.node = my.node.parent()
  endwhile

  return my.count
endfunction

function global.print_node(node, nodename)
  my.nodename ?= name(my.node)
  my.n = get_node(my.node, my.nodename)?

  if !defined(my.n)
    error('$(my.nodename:) "$(my.node:)" not found')
  endif

  my.n = get_node_name(my.n, my.nodename)

  return my.n
endfunction

function global.override_attribute(n1, nn, attribute)
  if defined(my.n1.$(my.attribute))
    my.nn.$(my.attribute) = my.n1.$(my.attribute)
  endif
endfunction

function global.append_attribute(n1, nn, attribute)
  my.nn.$(my.attribute) ?= ""
  if defined(my.n1.$(my.attribute))
    my.nn.$(my.attribute) += " " + my.n1.$(my.attribute)
  endif
endfunction

function global.override_reference(n1, origin, nodename, context)
  my.nn = my.context->$(my.nodename)?(!defined(ref) & defined(name) & name = my.n1.ref)?

  if !defined(my.nn)
    return
  endif

  override_attribute(my.n1, my.nn, 'id')
  override_attribute(my.n1, my.nn, 'name')
  append_attribute(my.n1, my.nn, 'description')
  define my.nn.origin = my.origin
endfunction

function global.replace_by_reference(n1, n2, node_src, node_dst, context)
  my.nodename = "$(my.node_dst:)_$(item():)"
  copy my.n2 to my.context before my.n1 as $(my.nodename:)
  my.origin = ""
  if defined(my.n2.id)
    my.origin = get_node_attribute(my.n2, 'id')
  else
    my.origin = get_node_attribute(my.n2, 'name')
  endif
  override_reference(my.n1, my.origin, my.nodename, my.context)
  delete my.n1
  move my.context->$(my.nodename) to my.context as $(my.node_dst:)
endfunction

function global.process_references(node_src, node_dst, context)
  my.node_dst ?= my.node_src
  my.context ?= project

  for my.context. as ii where ii.name()?"" <> ""
    process_references(my.node_src, my.node_dst, ii)
  endfor

  for my.context.$(my.node_dst:) as n where defined(n.ref)
    my.nn = get_node_attempt(n.ref, my.node_src, my.context)?
    if !defined(my.nn)
      next
    endif
    if defined(my.nn) & my.nn <> n
      replace_by_reference(n, my.nn, my.node_src, my.node_dst, my.context)
    endif
  endfor
endfunction

.endtemplate

Found another regression between GSL/3 and GSL/4.

Please check out inherit.gsl & inherit.xml from https://github.com/suntong001/gsl/tree/master/doc/examples.

This is how they run:

$ gsl3 -script:inherit.gsl inherit.xml 
2013/12/26 15:53:58: GSL/3.1 Copyright (c) 1991-2003 iMatix Corporation
2013/12/26 15:53:58: gsl3 I: Processing inherit.xml...
2013/12/26 15:53:58: gsl3 M: defined

item level

2013/12/26 15:53:58: gsl3 M: undefined
2013/12/26 15:53:58: gsl3 M: undefined

$ gsl4 -script:inherit.gsl inherit.xml 
2013/12/26 15:53:40: GSL/4.1a/github Copyright (c) 1996-2011 iMatix Corporation
2013/12/26 15:53:40: gsl/4 I: Processing inherit.xml...
2013/12/26 15:53:40: gsl/4 M: defined

item level

2013/12/26 15:53:40: gsl/4 M: defined

options level

2013/12/26 15:53:40: gsl/4 M: defined

options level

I don't know if such inheriting is by design (I vaguely recall seeing it somewhere), but the gsl3 behavior is definitely more favorable to me, and especially the case 2 (the item name="2" one), it just make scripting with gsl3 much easier than gsl4.

Thanks

Consider following XML

<?xml version="1.0" ?>
<test:Schema xmlns="gsl"
             xmlns:test="test">
    <test:Hello test:name="World">!</test:Hello>
</test:Schema>

and GSL

.template 0

function xmlns ()
  echo name()
  for . as node
    echo node.name() + " " + node
  endfor
endfunction

xmlns()

echo [test:Schema]->[test:Hello].[test:name]

.endtemplate

Now the for . as node outputs just the XMLElement, but not XMLAttribute nodes:

2014/05/05 08:33:35: GSL/4.1a/github Copyright (c) 1996-2011 iMatix Corporation
2014/05/05 08:33:35: gsl/4 I: Processing test2.xml...
2014/05/05 08:33:35: gsl/4 M: test:Schema
2014/05/05 08:33:35: gsl/4 M: test:Hello !
2014/05/05 08:33:35: gsl/4 M: World

So the issue or question is, how to enumerate attributes when their name is unknown and not hard coded?

(What I want or expected)

2014/05/05 08:33:35: GSL/4.1a/github Copyright (c) 1996-2011 iMatix Corporation
2014/05/05 08:33:35: gsl/4 I: Processing test2.xml...
2014/05/05 08:33:35: gsl/4 M: test:Schema
2014/05/05 08:33:35: gsl/4 M: xmlns gsl
2014/05/05 08:33:35: gsl/4 M: xmlns:test test
2014/05/05 08:33:35: gsl/4 M: test:Hello !
// 2014/05/05 08:33:35: gsl/4 M: test:name World
2014/05/05 08:33:35: gsl/4 M: World

I've been using GSL in Xcode as a custom build step, but the output produced by the GSL command is inconsistent and hard to parse into something that Xcode likes as an error message. I've got the following build rule at the moment

errors=`/usr/local/bin/gsl -q -script:${INPUT_FILE_DIR}/specs.gsl -a ${INPUT_FILE_PATH} ${SCRIPT_OUTPUT_FILE_0} 2>&1 | sed -E -n 's#[0-9]+/[0-9]+/[0-9]+[ ][0-9]+:[0-9]+:[0-9]+:[ ]\(*([^g][^s][^l].*)[ ]+([0-9]+):*\)*(.*)#'${INPUT_FILE_DIR}'/\1:\2: \3#p'`
if [ "$errors" ]; then
    echo "$errors"
    exit 1
fi

This matches up the lines starting with date/time, extracts the filename, line number and error message and formats it suitable for Xcode to understand as a build error. You also need to return a non-zero exit code on failure for Xcode to be happy.

With this, you get errors in the Xcode build navigator as you would for regular source code, although XML errors currently won't jump to the source file/line when clicked as the path gets repeated.

<file>:<line>: <message>

As currently implemented, GSL is unhelpful, as formatting of errors in GSL and XML sources are different (braces in one, not the other, around the filename and line number) and the message is split across multiple lines.

Whilst I'm not asking for Xcode compatible output, it would at least be nice if the GSL output was consistent for different failure types. The source just appears to be using a coprintf routine rather than a dedicated logging function to help here.

If anyone can suggest a preferred output style (or if the existing one cannot be deprecated), I may well consider creating a pull request for a suitable implementation.

smtmsg.c missing

It seems to me that it would be very useful to switch between template/script mode automatically depending on the use of function/macro keyword.

Presently the syntax forces you to write:

.template 0
function test1
  echo "test1"
endfunction

template 1
.macro test2
.  echo "test2"
.endmacro
endtemplate
.endtemplate

I propose to make a change so that the use of "macro" keyword in the script mode would automatically switch to template mode in the body of that macro (if not already in the template mode), and vice versa (use of "function" keyword would automatically switch to script mode in the body of the function if not already in the script mode):

.template 0
function test1  # already in script mode - not switching template mode
  echo "test1"
endfunction

macro test2     # in script mode - automatically switching to template mode
.  echo "test2"
endmacro        # returning back to script mode

endtemplate

I would've submitted a pull request as I thought this was a simple change, but frankly after looking at the source code for 20min, couldn't find a place where the template mode is being set when evaluating the function/macro. :(

otherwise it passes "-I" with no argument

Hi,

I found out some regressions between GSL/3.1 and GSL/4.1, and have put my code under
https://github.com/suntong001/gsl/tree/master/doc/examples

Files:

1. regression-gsl.gsl
2. regression-case1.xml
3. regression-case2.xml

Here is how they behave under GSL/3.1:

$ gsl -script:regression-gsl.gsl regression-case1.xml 
2013/12/25 23:14:40: GSL/3.1 Copyright (c) 1991-2003 iMatix Corporation
2013/12/25 23:14:40: gsl3 I: Processing regression-case1.xml...
2013/12/25 23:14:40: gsl3 M: is empty

$ gsl -script:regression-gsl.gsl regression-case2.xml 
2013/12/25 23:14:44: GSL/3.1 Copyright (c) 1991-2003 iMatix Corporation
2013/12/25 23:14:44: gsl3 I: Processing regression-case2.xml...
2013/12/25 23:14:44: gsl3 M: not empty
Test

In GSL/4.1 (https://github.com/imatix/gsl), it just errors out.

Please verify.
Thanks