Coder Social home page Coder Social logo

datalog-parser's Introduction

datalog-parser

A Datalog parser. This parser is used by Datahike and follows the Datalog dialect of Datomic.

Note: This repository has been moved from the lambdaforge organization to replikativ. So, you will find older releases of the parser at the lambdaforge clojars page.

Usage

Add the current release of io.replikativ/datalog-parser to your project.clj. Start a repl and run:

(require '[datalog.parser :as parser])

(parser/parse '[:find ?x :in $ ?y :where [?x :z ?y]])

;;=> (namespaces omitted for brevity)
;; #Query{:qfind  #FindRel{:elements [#Variable{:symbol ?x}]}
;;        :qwith  nil
;;        :qin    [#BindScalar{:variable #SrcVar{:symbol $}}
;;                 #BindScalar{:variable #Variable{:symbol ?y}}]
;;        :qwhere [#Pattern{:source #DefaultSrc{}
;;                          :pattern [#Variable{:symbol ?x}
;;                                    #Constant{:value  :z}
;;                                    #Variable{:symbol ?y}]}]}

For more examples look at the tests.

Benchmarking and Profiling

To benchmark or profile the parser, change to the parser-perf namespace or require it:

(in-ns 'datalog.parser-perf')

Then run the parse benchmark or profiler:

(parse)
(profile)

To see the produced flame graph, you can start a web server at port:

(in-ns 'datalog.parser-perf')
(prof/server-files port)

TODO

0.2.0

Unparsing support, missing types:

  • PullSpec
  • PullAttrName
  • PullReverseAttrName
  • PullLimitExpr
  • PullDefaultExpr
  • PullWildcard
  • PullRecursionLimit
  • PullMapSpecEntry
  • PullAttrWithOpts

License

Copyright © 2020 lambdaforge UG (haftungsbeschränkt), Nikita Prokopov

This program and the accompanying materials are made available under the terms of the Eclipse Public License 1.0.

datalog-parser's People

Contributors

bsless avatar cldwalker avatar jonasseglare avatar jsmassa avatar kordano avatar timokramer avatar whilo avatar zoren avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

datalog-parser's Issues

parse-rules fails on any unknown variable

Hi,
Thanks for the handy library! I was looking to use datalog.parser.impl/parse-rules for linting rules. I noticed that the following throws an error Reference to the unknown variables: (?region ?account ?call) even though it's a valid rule and the unbound variables serve a purpose:

[(aws-vpc-attr ?id ?attr ?val)
       ["aws" ?account ?region "ec2" "describe-vpc-attribute" ?call :vpc-id ?id]
       ["aws" ?account ?region "ec2" "describe-vpc-attribute" ?call ?attr ?val]]

Is this the desired behavior? Also curious why this in impl as it'd probably be useful for others.

Cheers, Gabriel

Can't parse on ClojureScript

Hi, I'm currently using Shadow-CLJS with the following code:

(parser/parse '[:find ?title
                :where
                [?e :movie/title ?title]
                [?e :movie/year 1987]])

But this is giving me two errors - first, it's giving me an error:

"No protocol method Traversable.-traversable? defined for type number: 1987"

Then some compilation warnings:

------ WARNING #1 - :undeclared-var --------------------------------------------
 Resource: datalog/parser/impl/util.cljc:23:19
 Use of undeclared Var datalog.parser.impl.util/raise
--------------------------------------------------------------------------------

------ WARNING #2 - :redef -----------------------------------------------------
 Resource: datalog/parser/impl.cljc:36:1
 distinct? already refers to: cljs.core/distinct? being replaced by: datalog.parser.impl/distinct?
--------------------------------------------------------------------------------

------ WARNING #3 - :undeclared-var --------------------------------------------
 Resource: datalog/parser/impl.cljc:433:33
 Use of undeclared Var datalog.parser.impl/java
--------------------------------------------------------------------------------

------ WARNING #4 - :undeclared-var --------------------------------------------
 Resource: datalog/parser/impl.cljc:442:34
 Use of undeclared Var datalog.parser.impl/java
--------------------------------------------------------------------------------

New attribute specifications for pull are not supported

The current attribute expressions supported by the datalog parser are considered "legacy" by datomic.

The attribute specs define a number of currently unsupported cases.

Example:
Legacy:
[(limit :my-key 10)] [(default :my-key "something)]

"New":
[[:my-key :as :another-key :limit 10 :default "something" :xform str/lower-case]]

Satisfies? performance is terrible

Hello
I did some performance analysis on the datalog parser. It's true that it uses caching (which is good), but it can still be significantly improved by getting rid of satisfies?, which is probably hit pretty often by traversable?.

I think a possible hack/cheat to work around it can be adding a set behind an atom which will be updated with tagname every deftrecord.
Then satisfies? can be written as

(defn traversable?
  [e]
  (contains? @traversables
             (keyword (.getName ^Class (type e)))))

Another solution is to use a multimethod like:

(defmulti traversable? identity)
(defmethod traversable? :default false)
(defmethod traversable? (keyword tagname) true)

What do you think? It might be irrelevant due to caching.

Attached flame graph with profiling results from query parsing

image

or-join too strict?

I mistakenly opened an issue here, now that I looked more closely I see this error is coming from this library.

Changing validate-or to this

(defn validate-or [clause form]
  (let [{{required :required
          free     :free} :rule-vars} clause
        vars                          (concat required free)]
    (validate-join-vars vars (:clauses clause) form)
    clause))

solves my issue but blows up 3 tests:

FAIL in (or-clause) (impl_test.cljc:427)
expected: (thrown-with-msg? ExceptionInfo #"Join variable not declared inside clauses: \[\?y\]" (dp/parse-clause (quote (or [?x] [?x ?y]))))
  actual: nil

lein test :only datalog.parser.impl-test/or-clause

FAIL in (or-clause) (impl_test.cljc:430)
expected: (thrown-with-msg? ExceptionInfo #"Join variable not declared inside clauses: \[\?y\]" (dp/parse-clause (quote (or [?x] [?y]))))
  actual: nil

lein test :only datalog.parser.impl-test/or-clause

FAIL in (or-clause) (impl_test.cljc:433)
expected: (thrown-with-msg? ExceptionInfo #"Join variable not declared inside clauses: \[\?x\]" (dp/parse-clause (quote (or-join [?x ?y] [?x ?y] [?y]))))
  actual: nil

I'm not quite sure why the variables of the or-join would need to be used in each clause? As I said in the other issue, datomic doesn't seem to have this restriction.

unnecessary loop in validate-or-join

In namespace datalog.parser.impl something seems to have gone wrong as:

  • validate-or-join-vars doesn't use clauses parameter
  • validate-or-join loops over clauses onlx changing clauses parameter of validate-or-join-vars

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.