Coder Social home page Coder Social logo

flintstone's Introduction

Flintstone

Total Downloads Build Status

A key/value database store using flat files for PHP.

Features include:

  • Memory efficient
  • File locking
  • Caching
  • Gzip compression
  • Easy to use

Installation

The easiest way to install Flintstone is via composer. Run the following command to install it.

composer require fire015/flintstone
<?php
require 'vendor/autoload.php';

use Flintstone\Flintstone;

$users = new Flintstone('users', ['dir' => '/path/to/database/dir/']);

Requirements

  • PHP 7.3+

Data types

Flintstone can store any data type that can be formatted into a string. By default this uses serialize(). See Changing the formatter for more details.

Options

Name Type Default Value Description
dir string the current working directory The directory where the database files are stored (this should be somewhere that is not web accessible) e.g. /path/to/database/
ext string .dat The database file extension to use
gzip boolean false Use gzip to compress the database
cache boolean or object true Whether to cache get() results for faster data retrieval
formatter null or object null The formatter class used to encode/decode data
swap_memory_limit integer 2097152 The amount of memory to use before writing to a temporary file

Usage examples

<?php

// Load a database
$users = new Flintstone('users', ['dir' => '/path/to/database/dir/']);

// Set a key
$users->set('bob', ['email' => '[email protected]', 'password' => '123456']);

// Get a key
$user = $users->get('bob');
echo 'Bob, your email is ' . $user['email'];

// Retrieve all key names
$keys = $users->getKeys(); // returns array('bob')

// Retrieve all data
$data = $users->getAll(); // returns array('bob' => array('email' => '[email protected]', 'password' => '123456'));

// Delete a key
$users->delete('bob');

// Flush the database
$users->flush();

Changing the formatter

By default Flintstone will encode/decode data using PHP's serialize functions, however you can override this with your own class if you prefer.

Just make sure it implements Flintstone\Formatter\FormatterInterface and then you can provide it as the formatter option.

If you wish to use JSON as the formatter, Flintstone already ships with this as per the example below:

<?php
require 'vendor/autoload.php';

use Flintstone\Flintstone;
use Flintstone\Formatter\JsonFormatter;

$users = new Flintstone('users', [
    'dir' => __DIR__,
    'formatter' => new JsonFormatter()
]);

Changing the cache

To speed up data retrieval Flintstone can store the results of get() in a cache store. By default this uses a simple array that only persist's for as long as the Flintstone object exists.

If you want to use your own cache store (such as Memcached) you can pass a class as the cache option. Just make sure it implements Flintstone\Cache\CacheInterface.

flintstone's People

Contributors

adammbalogh avatar fire015 avatar jmargolin-rmn avatar nikush avatar nyamsprod avatar sinky avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

flintstone's Issues

Unserialize Notice

Hello,

I don't know if I'm doing something anormal, but I have several notice with Flintstone :

unserialize(): Error at offset ** of ** bytes in /***/vendor/fire015/flintstone/src/Formatter/SerializeFormatter.php on line 29

Regards

JSON Format

I see the json writing format like this:

Blog={"source":"uri","url":"blog","target":"_self"}

This cause my json file cannot be read as json file directly with json_decode(file_get_contents('/path/to/filename.json')). Is it possible to change or modify writing format to proper json format like this?

{"Blog":{"source":"uri","url":"blog","target":"_self"}}

location of database?

My apologies if this is addressed but I could not find it in the issues or the code. I am wondering if it is possible to optionally pass the location of the database in the __construct. Because I run different scripts and operations on my "notes" folder, I would like to avoid having the notes.dat be located in the same folder. Thank you greatly

Type Hinting against the FormatterInterface

the private setFormatter function could be simplified using PHP 5+ type hinting.

use Flintstone\Formatter\FormatterInterface;
use Flintstone\Formatter\SerializeFormatter;
...
...

private function setFormatter(FormatterInterface $formatter = null)
{
    $this->formatter = $formatter ?: new SerializeFormatter;
}

Where is the Distribution version?

I don't see any distribution version of FLINTSTONE,, is there a single .php distribution file that i can just simply include in my php app, (using the include statement) that is the whole of flintstone, or i have to manually copy it all to a single file? If i have to then how?

Cannot add key/value to same ID

Im trying to create simple page views log system for my website which works on daily basis. Date (year,month,day) is used as key.

post-1={
"2015-10-18": 52
}

Incrementing value by 1, per each page view works (when key matches current date or is smaller than current date), but when I try to add new key/value (date, day views) to "post-1" or any other, key/value gets overwritten instead of being added to JSON array. For example whole array looks like that above, but when date is changed, that "2015-10-18" is changed to current date and value is reset to zero, instead of pushing new date into array. Does that happen because of Flintstone DB limitations or am I doing something wrong with JSON?

Idea: support CSV files

I have an idea for a Flintstone CSV databaseuse.

Why CSV?

  • Supported by php via fgetcsv, so very fast
  • Supports multi-line automatically, no conversion
  • Usable by other applications, csv is universal
  • smallest format, since keys are not repeated.

Obviously, this would require that the keys be defined ahead of time, since of at runtime. So it might not be the right solution for everyone.

I've written a class that returns an associative array from a CSV file, I use it as a generic way of loading CVS.

https://github.com/survos/survos/blob/main/packages/grid-group-bundle/src/Service/Reader.php

For example, I want to save the results of my API calls to a translation service, and want to store the source language, target languate, original text and translated text.

I can imagine it working like this

        $transCache = new CsvFlintstone($prefix, [
            'key' => '_id', // the lookup key, could be default
            'keys' => ['source','target','orginal', 'translated_text'],


        $translation = $translationService($source, $textToTranslate);
        $key = md5($textToTranslate . $source);

          $transCache->set($key, array_merge(['source' => $source, 'target' => $target, 
                   'original' => $textToTranslate,                  
                    'translated_text' => $translation]));

This would create a CSV file

_id,source,target,original,translated_text
294FD2412,en,es,hello,hola
4935535FD,en,fr,hello,bon jour

It's small, fast, and re-usable.

I think it's just a matter of extending the Flintstone database class. Thoughts?

Use Indexes

Why not add some btree indexes on the keys so you don't have to manually look over ever line of the file for the key?

The current caching helps a little.

Flinstone out of composer

How to use FlintStone out of composer ?
I made a require_once of flintstone.php file and then a new Flinstone but unfortunately I get an error, the class is already defined

require_once "core/vendor/flintstone/Flintstone.php";
$options = ['dir' => self::DATA_DIR];
$users = new Flintstone('users', $options);

Error :
Fatal error: Cannot declare class Flintstone\Flintstone, because the name is already in use in C:\laragon\www\ZwiiCMS\core\vendor\flintstone\Flintstone.php on line 5

Thanks for you help.

Performance considerations

Thanks for a great library.
I have implemented flintstone in one of my projects to store key/value pairs for URL rewrites. The datastore is growing fast, and I am wondering if this is going to become a performance hog with potentially hundreds of reads per page load and a bunch of simultaneous user sessions. Have you done any benchmarks? Was such a use case ever considered for flinstone? Should I just require memcached/redis instead?

Use magic methods

So, there are 6 main methods in Flintstone.

  • get
  • set
  • remove
  • flush
  • getKeys
  • getAll

Even 4 of them can be replaced with better, PHP's build-in magic methods. That would make use of Flintstone easier and more efficient. Few stuff to do:

  • Rename get to __get, set to __set

  • Rename remove to __unset

  • Rename flush to __destruct

  • Make get, set and remove synonymous to their new names.

    function get($k, $v){$this->__get($k, $v)}
    

    etp.

  • Add flush function to do nothing (as PHP will automatically flush the database when shutting down, that's how __destruct works)

  • You might also want to add __isset to check if key exists.

This will simplify the syntax:

$db->key = "value";
$db->{"key with special chars"} = "value";
echo $db->key;

unset($db->key); # Remove key
var_dump(isset($db->key)) # Check if it's set.

# Notice there is now no need to add $db->flush();

As a bonus, it will add a possibility to create DB references, which would be amazing feature, that doesn't exist in SQL, for example.

$ref =& $db->key; # Reference var $ref to key `key`
$ref = "Hello ";
$ref .= "World!";
echo $db->key; # Hello World!

That's not all yet. You would have possibility to use special assign operators like += etp.:

$db->key += 5;

Would be really nice to see it in the database. It's not hard to add prefixes and synonymous functions, isn't it? I'm already trying to do it on local version on Flintstone.php file.

Thanks.

set returns void instead of bool

Hi!

I'm currently writing a wrapper around flintstone/redis and noticed a little difference between your docs and implementation.

The docs state

Set data for the key name. Data can be a string, integer, float or array. Returns true or will throw an exception if fails to set.

But in reality, set isn't returning true

Object Storage

Noticed that Flintstone doesn't currently support object storage per key but will support an object loaded into an array.

Should there be support for object storage directly?

Sometimes gets "stuck" setting a key/value

Occasionally my program gets "stuck" while trying to set a key/value pair. To solve it, I rename the file and pipe it back out, then it works.

mv myKeyValues.dat x && cat x > myKeyValues.dat

I'm guessing that it's related to locking, but I guess it could be character encoding or something else. How do you suggest solving it?

I'm using PHP as the encoding, I guess I could try JSON as well. Of course, it doesn't happen frequently, so it's been hard to reproduce.

Update only one name in database

I have a base

1461223657645=a:3:{s:5:"notes";s:4:"test";s:8:"worktime";s:4:"3316";s:9:"workmoney";s:5:"13.81";}

I want to change only notes. How to do it?

Registration form and checking email and login

$user = mysql_query("SELECT * FROM `users` WHERE `login`='$post_login' AND `email`='$post_lemail'  LIMIT 1");
  if(mysql_num_rows($user) != 0) { echo 'OK'; }

Show me please how this code would look for flintstone.

user.dat:

bob=a:6:{s:5:"email";s:12:"[email protected]";s:8:"password";s:32:"e10adc3949ba59abbe56e057f20f883e";}

Can't use more than 1 storage file at once

//load the DB
$db = new Flintstone(array('dir' => $flintstone['database']));

//use two files
$stats = $db->load($flintstone['tables']['status']);
$packs = $db->load($flintstone['tables']['packages']);

//now set
$packs->set($key, array('xml' => 'xml'));
$stats->set($key, array('status' => 'RECEIVED'));

I would expect to get entries in both files but I get only in stats that was loaded last.

Incorrect handling of double quotes in strings

Flintstone escapes double quotes (") in strings, when saving a value and then loading it back (from file, not memory) the values don't match.

Simple test case:

$da= new Flintstone(array('dir' => '.'));
$a = 'We have "quotes" in here';
$da->load("test")->set("a",$a);

$db= new Flintstone(array('dir' => '.'));
$a = 'We have "quotes" in here';
$b = $db->load("test")->get("a");

echo $a."=".$b."? ".($a === $b ? "true" : "false");

I fixed it by commenting out line 534:
$string = preg_replace('!s:(\d+):"(.*?)";!se', "'s:'.strlen('$2').':\"$2\";'", $string);
but this might break something else.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.