afs / mantis Goto Github PK
View Code? Open in Web Editor NEWDatabase Operating Environment
License: Apache License 2.0
Database Operating Environment
License: Apache License 2.0
Initial testing of TDB2 loading 9M triples fails after 1.4M with disk full - 16GB. This is with jena-fuseki 3.5.0-SNAPSHOT + TDB2 as described below.
Configuration with TDB and released Jena 3.4.0 loads to completion with total of 3.1GB in use.
I observe that TDB2 is allocating disk space very rapidly during loading. The loading is being done via the graph store protocol, org.apache.jena.query.DatasetAccessor.putModel
for each of 146K named graphs.
Am I missing a configuration piece or is this perhaps an interaction with something in Jena 3.5.0-SNAPSHOT?
I cloned mantis and did
mvn install
and then cloned jena (3.5.0-SNAPSHOT), updated the jena-fuseki-war pom.xml with the dependency and did a
mvn install
in jena and
mvn package
in jena-fuseki-war and produced the desired war file with tdb2-0.3.0.jar.
I then updated the configuration file with:
@prefix tdb2: <http://jena.apache.org/2016/tdb#> .
. . .
[] ja:loadClass "org.seaborne.tdb2.TDB2" .
tdb2:DatasetTDB2 rdfs:subClassOf ja:RDFDataset .
. . .
:dataset_test rdf:type tdb2:DatasetTDB2 ;
tdb2:location "/etc/fuseki/databases/test" ;
tdb2:unionDefaultGraph true ;
Java is 1.8.0_144 on Debian 4.9.30-2+deb9u3 (2017-08-06) x86_64 GNU/Linux and Tomcat 8.0.12.
Here's the complete fuseki config: bdrc.ttl.zip
TDB2 has a reorganized NodeId
which usually has space for 56 bit encoded values inline. It also has space for 62 bits for the one case of xsd:double but there is no implementation yet.
xsd:double
, a IEEE 754 binary64.To inline the value without increasing the size of NodeId
, 2 bits need to be saved. Values not meeting the restriction will be stored as nodes in the node table.
Proposal: keep the significand (mantissa) as 52 bits and reduce the exponent from 11 bits to 9 bits - making a size limit of inlined values 1.157921e+77, rather than the full 1.797693e+308.
Infinities, +/- zero and NaNs are inlined.
This is recoverable - inline values will be perfected recovered.
Reducing the precision instead risks looses accuracy.
TDB2 already added the inlining of xsd:floats and retaining the datatypes of integer derived datatypes.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.