Coder Social home page Coder Social logo

db-based-replicated-filesystem's Introduction

                      mysqlfs - MySQL FileSystem

$Id: README 55 2009-07-12 22:23:33Z chickenandporn $

MySQLfs is a FUSE filesystem driver which stores files in a MySQL database.

* Requirements

  - mysql-client libraries 5.0 or later
  - mysql-server 5.0 or later
  - fuse 2.5 or later
  - autotools 

* Build

  If compiling from version-control, use "autoreconf -vfi" first.

  For both version-control and tarball version run:
  $ ./configure
  $ make

* Usage

1. Create database and account
   mysql> CREATE DATABASE mysqlfs;
   mysql> GRANT SELECT, INSERT, UPDATE, DELETE ON mysqlfs.* TO mysqlfs@"%" IDENTIFIED BY 'password';
   mysql> FLUSH PRIVILEGES;

   (note FAQ: Errors #1 "Access Denied For User" below)

2. Create tables
   $ mysql -uroot -p mysqlfs < schema.sql

   (note FAQ: Errors #2 "Can't Create/Write to File" below)

3. Mount database as a filesystem
   $ mkdir fs
   $ ./mysqlfs -ohost=localhost -ouser=user -opassword=pass -odatabase=mysqlfs fs

4. Instead of setting connection options on the command line
   you may create a [mysqlfs] section in your ~/.my.cnf file and
   set the parameters there.

* Options

  -ohost=<hostname>
    MySQL server host

  -ouser=<username>
    MySQL username

  -opassword=<password>
    MySQL password

  -odatabase=<db>
    MySQL database name

* FAQ: ERRORS

1. Access Denied For User 'mysql'@'localhost'

   ie: (using the example GRANT above) 
   $ ./mysqlfs -ohost=localhost -ouser=mysqlfs -opassword=password -odatabase=mysqlfs fs

   2009-06-14 00:59:29 89956 ERROR: mysql_real_connect(): Access denied for user 'mysqlfs'@'localhost' (using password: YES)
   2009-06-14 00:59:29 89956 ERROR: mysql_real_connect(): Access denied for user 'mysqlfs'@'localhost' (using password: YES)
   2009-06-14 00:59:29 89956 Failed to connect MySQL server.
   2009-06-14 00:59:29 89956 Error: pool_init() failed

   MySQL is sticky sometimes with access; on MacOSX, I had to specifically allow localhost:

   mysql> GRANT SELECT, INSERT, UPDATE, DELETE ON mysqlfs.* TO mysqlfs@"localhost" IDENTIFIED BY 'password';

   $ sudo /usr/local/mysql/bin/mysqladmin reload

   Remember, also, that 'localhost' and '127.0.0.1' may appear differently.  Check also your /etc/hosts entry, some
   systems define 127.0.0.1 as their uname() which causes problems doing PTRs against it, expecting 'localhost'
   but getting 'myhostname'


2. Can't Create/Write to File (while building database)

   ie: (using the "Create tables" step #2 above)
   $ sudo /usr/local/mysql/bin/mysql mysqlfs < schema.sql
   ERROR 1 (HY000) at line 23: Can't create/write to file '/usr/local/mysql/data/mysqlfs/data_blocks.MYI' (Errcode: 13)

   On MacOSX with http://dev.mysql.com/get/Downloads/MySQL-5.1/mysql-5.1.35-osx10.5-x86.dmg ,
   rather than "CREATE DATABASE mysqlfs", I had to use:

   $ sudo /usr/local/mysql/bin/mysqladmin create mysqlfs && sudo chown _mysql /usr/local/mysql/data/mysqlfs

   The grant/flush commands worked fine after that.  In my usage, I'm "sudo" which can affect the output, so the
   error might have been mine alone.  I'm commenting here in case someone else sees it.


Copyright (C) 2006 Tsukasa Hamano <[email protected]>
Copyright (C) 2006 Michal Ludvig <[email protected]>

db-based-replicated-filesystem's People

Contributors

samzyy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

db-based-replicated-filesystem's Issues

Is this still active?

Is this project still active?
Haven't seen any activity in a while.
I'm looking into it and would like to work on it if it's active.

Consideration: large scalable fs

Hi,

Wonderfull project, this kind of projects with curousity the primairy focus are in my opinion the most worthfull research projects out there.

Having said that, i have some questions/considerations:

  • you didn’t disclose much about your testsetup. I think the problem you are trying to solve is related to large multiuser filesystems where you actually have the problem of the infra layering you described in your presentation. But in your presentation it seems (i’m assuming) you tested on a single system without the raid controllers, lots of systems/disks. So I would suggest testing against large systems. On small single systems the fs is already optimised i recon and obviously you would not run gelera on a single system anyway. I’m guessing the performance hit will be less at larger workloads, multiple user/systems where scale can benefit from the less layered infrastructure.
  • Although i admire the project, I think your goals have been superceded by Ceph, which has CephFS, which is a scalable multiuser system which can (or choose not to) know about the hardware underneath. The current implementation of Ceph does away with having a fs on the disks and has dedicated metadata servers. Ceph has at least a horizontal scalable posix fs and can be optimised for the hardware it uses. It’s (i think) a better way to test against as your idea, as it tries to solve the same issue at the same scale.

If you get great results, im sure the Ceph team is interested and they have much more manpower to make something of the project.

Some disclaimer here: Im no way an expert, so i can easilly be proven wrong, so comments welcome!

rename() doesn't check destination.

Currently I can do

mkdir foo
touch foo/bah
mkdir xxx
rename("xxx", "foo")

The result is that the rename succeeds and bah becomes an orphan in the inodes and tree tables,
What should happen is an EEXIST or ENOTEMPTY error return from rename().

Idea - taggable files

It sounds like this would be a trivial feature to implement. Just leaving a note in case someone wants to try it.

fsck doesn't fix orphans

Orpahned files/directories should be cleaned up somehow --- added to /lost+found perhaps --- on fsck.

An orphan is a tree entry where the parent is not in the tree.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.