slavikmipt / tgcloud

Opensource Telegram based cloud storage

tgcloud

UNDER DEVELOPMENT v1.1

• secret.py : rename secret.py.template, insert api_hash and api_id obtained with https://my.telegram.org

• Install Python2.7 and Python3.6

• Clone the repository

cd ~
git clone https://github.com/SlavikMIPT/tgcloud.git

• Install requirements for Python3.6

sudo pip3 install -r requirements.txt

• Create a session by running from the project folder

python3.6 telegram_create_session.py

• Install fuse bindings

sudo yum install python-fuse

• Create Mount Folder

mkdir storage

• Run VFS from the project folder:

debug:

python2.7 dedupfs/dedupfs.py -df --block-size 20971520 -o auto_unmount -o hard_remove storage/

as a service:

replace <username> in tgcloud.service

sudo cp tgcloud.service /ect/systemd/system/
sudo systemctl enable tgcloud.service
sudo systemctl daemon-reload
sudo systemctl start tgcloud.service
sudo systemctl status tgcloud.service -l

Version 1.1

It works more vigorously, but still a raw prototype - just POC

Can crash, it is unstable and slow in some cases .

For tests, it is better to use a separate telegram profile.

If banned - mail [email protected]

You can try to use web based filebrowser or SFTP

You are welcome to collaborate - contact Telegram: @SlavikMIPT Channel: @MediaTube_stream

tgcloud's Issues

Упаковать в Докер

Is your feature request related to a problem? Please describe.
Сейчас не тривиальный способ установки и есть возможность получить "бан" за неправильную авторизацию.
С точки зрения разработчика так же хочется иметь простой способ запуска проекта.
Так же это позволит разрабатывать на других ОС (н-р MacOS)

Describe the solution you'd like
Описать все шаги установки в Dockerfile, чтобы больше об этом не думать.
Тогда можно будет сделать монтируемую папочку (volume) в контейнер, который будет синхрить её содержимое в телеграм.

Describe alternatives you've considered
Пакет для ubuntu. Сложно и рано.

Увеличить скорость

Установил на европейский VPS с хорошим каналом, но средняя скорость ≈60 Mbit.
Как можно её увеличить?
Изменить размер блока?
Увеличить число потоков?
Что-то ещё?

Проблема больших файлов

Скопировав большой файл в папку, которую обрабатывает программа получил следующую ошибку:

Rolling back changes
--------------------------------------------------
Caught exception in release(): [Errno 12] Cannot allocate memory
Traceback (most recent call last):
  File "dedupfs/dedupfs.py", line 481, in release
    self.__write_blocks(inode, buf, apparent_size)
  File "dedupfs/dedupfs.py", line 824, in __write_blocks
    bufsize=-1)
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 938, in _execute_child
    self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory

Файл весил ~2Гб, система: Ubuntu 18.04 LTS, 4Gb RAM.

Двух этапная аунтефикация

Сегодня заклонил репозиторий на Ubuntu 18, обнаружил там tg_access.py :) ввел все необходимое, после чего все равно в терминале вводил телефон, далее код от Telegram, а потом ошибка: SessionPasswordNeededError. После чего я благополучно получить посылку на 24 часа ожидания логина :). И через минут 5 пришло от Telegram, что есть не завершенные попытки входа.

FS не производит запись файлов

download_service успешно создает сессию, но при попытке записи файлов записывается только мета-информация, которая теряется при размонитровании папки.
Лог dedupfs:

readdir[0] from 0
   unique: 122, success, outsize: 120
unique: 124, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 124, success, outsize: 16
unique: 126, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 126, success, outsize: 16
unique: 128, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 128, success, outsize: 16
unique: 130, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 130, success, outsize: 32
unique: 132, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 132, success, outsize: 120
unique: 134, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 134, success, outsize: 120
unique: 136, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 136, success, outsize: 16
unique: 138, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 138, success, outsize: 16
unique: 140, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 140, success, outsize: 16
unique: 142, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 142, success, outsize: 32
unique: 144, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 144, success, outsize: 120
unique: 146, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 146, success, outsize: 120
unique: 148, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 148, success, outsize: 16
unique: 150, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 150, success, outsize: 16
unique: 152, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 152, success, outsize: 16
unique: 154, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 154, success, outsize: 32
unique: 156, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 156, success, outsize: 120
unique: 158, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 158, success, outsize: 120
unique: 160, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 160, success, outsize: 16
unique: 162, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 162, success, outsize: 16
unique: 164, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 164, success, outsize: 16
unique: 166, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 166, success, outsize: 32
unique: 168, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 168, success, outsize: 120
unique: 170, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 170, success, outsize: 120
unique: 172, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 172, success, outsize: 16
unique: 174, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 174, success, outsize: 16
unique: 176, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 176, success, outsize: 16
unique: 178, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 178, success, outsize: 32
unique: 180, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 180, success, outsize: 120
unique: 182, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 182, success, outsize: 120
unique: 184, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 184, success, outsize: 16
unique: 186, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 186, success, outsize: 16
unique: 188, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 188, success, outsize: 16
unique: 190, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 190, success, outsize: 32
unique: 192, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 192, success, outsize: 120
unique: 194, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 194, success, outsize: 120
unique: 196, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 196, success, outsize: 16
unique: 198, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 198, success, outsize: 16
unique: 200, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 200, success, outsize: 16
unique: 202, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 202, success, outsize: 32
unique: 204, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 204, success, outsize: 120
unique: 206, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 206, success, outsize: 120
unique: 208, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 208, success, outsize: 16
unique: 210, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 210, success, outsize: 16
unique: 212, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 212, success, outsize: 16
unique: 214, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 214, success, outsize: 32
unique: 216, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 216, success, outsize: 120
unique: 218, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 218, success, outsize: 120
unique: 220, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 220, success, outsize: 16
unique: 222, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 222, success, outsize: 16
unique: 224, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 224, success, outsize: 16
unique: 226, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 226, success, outsize: 32
unique: 228, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 228, success, outsize: 120
unique: 230, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 230, success, outsize: 120
unique: 232, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 232, success, outsize: 16
unique: 234, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 234, success, outsize: 16
unique: 236, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 236, success, outsize: 16
unique: 238, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 238, success, outsize: 32
unique: 240, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 240, success, outsize: 120
unique: 242, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 242, success, outsize: 120
unique: 244, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 244, success, outsize: 16
unique: 246, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 246, success, outsize: 16
unique: 248, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 248, success, outsize: 16
unique: 250, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 250, success, outsize: 32
unique: 252, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 252, success, outsize: 120
unique: 254, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 254, success, outsize: 120
unique: 256, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 256, success, outsize: 16
unique: 258, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 258, success, outsize: 16
unique: 260, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 260, success, outsize: 16
unique: 262, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 262, success, outsize: 32
unique: 264, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 264, success, outsize: 120
unique: 266, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 266, success, outsize: 120
unique: 268, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 268, success, outsize: 16
unique: 270, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 270, success, outsize: 16
unique: 272, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 272, success, outsize: 16
unique: 274, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 1770
   unique: 274, success, outsize: 32
unique: 276, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1770
getattr /
   unique: 276, success, outsize: 120
unique: 278, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
readdir[0] from 0
   unique: 278, success, outsize: 120
unique: 280, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 280, success, outsize: 16
unique: 282, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 1770
   unique: 282, success, outsize: 16
unique: 284, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 284, success, outsize: 16
unique: 286, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 3899
   unique: 286, success, outsize: 32
unique: 288, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 3899
getattr /
   unique: 288, success, outsize: 120
unique: 290, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3900
readdir[0] from 0
   unique: 290, success, outsize: 120
unique: 292, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 3901
   unique: 292, success, outsize: 32
unique: 294, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3900
   unique: 294, success, outsize: 16
unique: 296, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 3901
getattr /
   unique: 296, success, outsize: 120
unique: 298, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3900
   unique: 298, success, outsize: 16
unique: 300, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 300, success, outsize: 16
unique: 302, opcode: LOOKUP (1), nodeid: 1, insize: 52, pid: 3900
LOOKUP /autorun.inf
getattr /autorun.inf
   unique: 302, error: -2 (No such file or directory), outsize: 16
unique: 304, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3901
readdir[0] from 0
   unique: 304, success, outsize: 120
unique: 306, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3901
   unique: 306, success, outsize: 16
unique: 308, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3901
   unique: 308, success, outsize: 16
unique: 310, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 310, success, outsize: 16
unique: 312, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 3901
getattr /
   unique: 312, success, outsize: 120
unique: 314, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1791
getattr /
   unique: 314, success, outsize: 120
unique: 316, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 3901
   unique: 316, success, outsize: 32
unique: 318, opcode: LOOKUP (1), nodeid: 1, insize: 47, pid: 3901
LOOKUP /.Trash
getattr /.Trash
   unique: 318, error: -2 (No such file or directory), outsize: 16
unique: 320, opcode: LOOKUP (1), nodeid: 1, insize: 52, pid: 3901
LOOKUP /.Trash-1000
getattr /.Trash-1000
   NODEID: 2
   unique: 320, success, outsize: 144
unique: 322, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3901
readdir[0] from 0
   unique: 322, success, outsize: 120
unique: 324, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3901
   unique: 324, success, outsize: 16
unique: 326, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 3901
getattr /
   unique: 326, success, outsize: 120
unique: 328, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3901
   unique: 328, success, outsize: 16
unique: 330, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 330, success, outsize: 16
unique: 332, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 3903
statfs /
   unique: 332, success, outsize: 96
unique: 334, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 3901
getattr /
   unique: 334, success, outsize: 120
unique: 336, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 3901
statfs /
   unique: 336, success, outsize: 96
unique: 338, opcode: LOOKUP (1), nodeid: 1, insize: 61, pid: 3901
LOOKUP /Untitled Diagram.png
getattr /Untitled Diagram.png
   unique: 338, error: -2 (No such file or directory), outsize: 16
unique: 340, opcode: CREATE (35), nodeid: 1, insize: 77, pid: 3901
create flags: 0x80c1 /Untitled Diagram.png 0100600 umask=0002
   create[0] flags: 0x80c1 /Untitled Diagram.png
getattr /Untitled Diagram.png
   NODEID: 4
   unique: 340, success, outsize: 160
unique: 342, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1884
getattr /
   unique: 342, success, outsize: 120
unique: 344, opcode: GETXATTR (22), nodeid: 4, insize: 68, pid: 3901
   unique: 344, error: -38 (Function not implemented), outsize: 16
unique: 346, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 0 flags: 0x8001
   write[0] 4096 bytes to 0
   unique: 346, success, outsize: 24
unique: 348, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 4096 flags: 0x8001
   write[0] 4096 bytes to 4096
   unique: 348, success, outsize: 24
unique: 350, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 8192 flags: 0x8001
   write[0] 4096 bytes to 8192
   unique: 350, success, outsize: 24
unique: 352, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 12288 flags: 0x8001
   write[0] 4096 bytes to 12288
   unique: 352, success, outsize: 24
unique: 354, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 16384 flags: 0x8001
   write[0] 4096 bytes to 16384
   unique: 354, success, outsize: 24
unique: 356, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 20480 flags: 0x8001
   write[0] 4096 bytes to 20480
   unique: 356, success, outsize: 24
unique: 358, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 24576 flags: 0x8001
   write[0] 4096 bytes to 24576
   unique: 358, success, outsize: 24
unique: 360, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 28672 flags: 0x8001
   write[0] 4096 bytes to 28672
   unique: 360, success, outsize: 24
unique: 362, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 32768 flags: 0x8001
   write[0] 4096 bytes to 32768
   unique: 362, success, outsize: 24
unique: 364, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 36864 flags: 0x8001
   write[0] 4096 bytes to 36864
   unique: 364, success, outsize: 24
unique: 366, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 40960 flags: 0x8001
   write[0] 4096 bytes to 40960
   unique: 366, success, outsize: 24
unique: 368, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 45056 flags: 0x8001
   write[0] 4096 bytes to 45056
   unique: 368, success, outsize: 24
unique: 370, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 49152 flags: 0x8001
   write[0] 4096 bytes to 49152
   unique: 370, success, outsize: 24
unique: 372, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 53248 flags: 0x8001
   write[0] 4096 bytes to 53248
   unique: 372, success, outsize: 24
unique: 374, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 57344 flags: 0x8001
   write[0] 4096 bytes to 57344
   unique: 374, success, outsize: 24
unique: 376, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 61440 flags: 0x8001
   write[0] 4096 bytes to 61440
   unique: 376, success, outsize: 24
unique: 378, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 65536 flags: 0x8001
   write[0] 4096 bytes to 65536
   unique: 378, success, outsize: 24
unique: 380, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 69632 flags: 0x8001
   write[0] 4096 bytes to 69632
   unique: 380, success, outsize: 24
unique: 382, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 73728 flags: 0x8001
   write[0] 4096 bytes to 73728
   unique: 382, success, outsize: 24
unique: 384, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 77824 flags: 0x8001
   write[0] 4096 bytes to 77824
   unique: 384, success, outsize: 24
unique: 386, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 81920 flags: 0x8001
   write[0] 4096 bytes to 81920
   unique: 386, success, outsize: 24
unique: 388, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 86016 flags: 0x8001
   write[0] 4096 bytes to 86016
   unique: 388, success, outsize: 24
unique: 390, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 90112 flags: 0x8001
   write[0] 4096 bytes to 90112
   unique: 390, success, outsize: 24
unique: 392, opcode: WRITE (16), nodeid: 4, insize: 4176, pid: 3901
write[0] 4096 bytes to 94208 flags: 0x8001
   write[0] 4096 bytes to 94208
   unique: 392, success, outsize: 24
unique: 394, opcode: WRITE (16), nodeid: 4, insize: 2407, pid: 3901
write[0] 2327 bytes to 98304 flags: 0x8001
   write[0] 2327 bytes to 98304
   unique: 394, success, outsize: 24
unique: 396, opcode: GETATTR (3), nodeid: 4, insize: 56, pid: 3901
getattr /Untitled Diagram.png
   unique: 396, success, outsize: 120
unique: 398, opcode: FLUSH (25), nodeid: 4, insize: 64, pid: 3901
   unique: 398, error: -38 (Function not implemented), outsize: 16
unique: 400, opcode: RELEASE (18), nodeid: 4, insize: 64, pid: 0
release[0] flags: 0x8001
Rolling back changes
--------------------------------------------------
Caught exception in release(): [Errno 2] No such file or directory
Traceback (most recent call last):
  File "dedupfs/dedupfs.py", line 482, in release
    self.__write_blocks(inode, buf, apparent_size)
  File "dedupfs/dedupfs.py", line 825, in __write_blocks
    bufsize=-1)
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
--------------------------------------------------
Returning -5
   unique: 400, success, outsize: 16
unique: 402, opcode: SETATTR (4), nodeid: 4, insize: 128, pid: 3901
getattr /Untitled Diagram.png
   unique: 402, success, outsize: 120
unique: 404, opcode: SETATTR (4), nodeid: 4, insize: 128, pid: 3901
utimens /Untitled Diagram.png 1560757447.000000000 1560708081.861436000
getattr /Untitled Diagram.png
   unique: 404, success, outsize: 120
unique: 406, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 3905
   unique: 406, success, outsize: 32
unique: 408, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3905
readdir[0] from 0
   unique: 408, success, outsize: 168
unique: 410, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3905
   unique: 410, success, outsize: 16
unique: 412, opcode: LOOKUP (1), nodeid: 1, insize: 48, pid: 3905
LOOKUP /.hidden
getattr /.hidden
   unique: 412, error: -2 (No such file or directory), outsize: 16
unique: 414, opcode: LOOKUP (1), nodeid: 1, insize: 47, pid: 3901
LOOKUP /.Trash
getattr /.Trash
   unique: 414, error: -2 (No such file or directory), outsize: 16
unique: 416, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3905
   unique: 416, success, outsize: 16
unique: 418, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 3908
getattr /
   unique: 418, success, outsize: 120
unique: 420, opcode: GETATTR (3), nodeid: 4, insize: 56, pid: 3901
getattr /Untitled Diagram.png
   unique: 420, success, outsize: 120
unique: 422, opcode: LOOKUP (1), nodeid: 1, insize: 47, pid: 3908
LOOKUP /.Trash
getattr /.Trash
   unique: 422, error: -2 (No such file or directory), outsize: 16
unique: 424, opcode: GETATTR (3), nodeid: 4, insize: 56, pid: 3908
getattr /Untitled Diagram.png
   unique: 424, success, outsize: 120
unique: 426, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 426, success, outsize: 16
unique: 428, opcode: LOOKUP (1), nodeid: 1, insize: 47, pid: 3901
LOOKUP /.Trash
getattr /.Trash
   unique: 428, error: -2 (No such file or directory), outsize: 16
unique: 430, opcode: LOOKUP (1), nodeid: 1, insize: 47, pid: 3906
LOOKUP /.Trash
getattr /.Trash
   unique: 430, error: -2 (No such file or directory), outsize: 16
unique: 432, opcode: GETATTR (3), nodeid: 4, insize: 56, pid: 3901
getattr /Untitled Diagram.png
   unique: 432, success, outsize: 120
unique: 434, opcode: GETATTR (3), nodeid: 4, insize: 56, pid: 3906
getattr /Untitled Diagram.png
   unique: 434, success, outsize: 120
unique: 436, opcode: OPENDIR (27), nodeid: 1, insize: 48, pid: 3907
   unique: 436, success, outsize: 32
unique: 438, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3905
readdir[0] from 0
   unique: 438, success, outsize: 168
unique: 440, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3905
   unique: 440, success, outsize: 16
unique: 442, opcode: READDIR (28), nodeid: 1, insize: 80, pid: 3905
   unique: 442, success, outsize: 16
unique: 444, opcode: LOOKUP (1), nodeid: 1, insize: 47, pid: 3908
LOOKUP /.Trash
getattr /.Trash
   unique: 444, error: -2 (No such file or directory), outsize: 16
unique: 446, opcode: GETATTR (3), nodeid: 4, insize: 56, pid: 3908
getattr /Untitled Diagram.png
   unique: 446, success, outsize: 120
unique: 448, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 3908
getattr /
   unique: 448, success, outsize: 120
unique: 450, opcode: RELEASEDIR (29), nodeid: 1, insize: 64, pid: 0
   unique: 450, success, outsize: 16
unique: 452, opcode: LOOKUP (1), nodeid: 1, insize: 63, pid: 1791
LOOKUP /Untitled%20Diagram.png
getattr /Untitled%20Diagram.png
   unique: 452, error: -2 (No such file or directory), outsize: 16
unique: 454, opcode: STATFS (17), nodeid: 1, insize: 40, pid: 3907
statfs /
   unique: 454, success, outsize: 96
unique: 456, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1791
getattr /
   unique: 456, success, outsize: 120
unique: 458, opcode: LOOKUP (1), nodeid: 1, insize: 63, pid: 1791
LOOKUP /Untitled%20Diagram.png
getattr /Untitled%20Diagram.png
   unique: 458, error: -2 (No such file or directory), outsize: 16
unique: 460, opcode: LOOKUP (1), nodeid: 1, insize: 61, pid: 1791
LOOKUP /Untitled Diagram.png
getattr /Untitled Diagram.png
   NODEID: 4
   unique: 460, success, outsize: 144
unique: 462, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1791
getattr /
   unique: 462, success, outsize: 120
unique: 464, opcode: LOOKUP (1), nodeid: 1, insize: 63, pid: 1791
LOOKUP /Untitled%20Diagram.png
getattr /Untitled%20Diagram.png
   unique: 464, error: -2 (No such file or directory), outsize: 16
unique: 466, opcode: LOOKUP (1), nodeid: 1, insize: 61, pid: 1791
LOOKUP /Untitled Diagram.png
getattr /Untitled Diagram.png
   NODEID: 4
   unique: 466, success, outsize: 144
unique: 468, opcode: GETATTR (3), nodeid: 1, insize: 56, pid: 1791
getattr /
   unique: 468, success, outsize: 120
unique: 470, opcode: LOOKUP (1), nodeid: 1, insize: 61, pid: 1791
LOOKUP /Untitled Diagram.png
getattr /Untitled Diagram.png
   NODEID: 4
   unique: 470, success, outsize: 144

В чем может быть проблема?

Ошибка в arh linux при запуске сессии

python telegram_create_session.py
Traceback (most recent call last):
File "telegram_create_session.py", line 12, in
client = TelegramClientX(entity, api_id, api_hash, update_workers=None, spawn_read_thread=True)
NameError: name 'entity' is not defined

Python-fuse cannot be installed in Ubuntu 20.04 or newer

Hi,

Is it possible to make this program fully compatible with python 3?
I cannot install python-fuse (python2) in Ubuntu 20.04 or 20.10, because python3-fuse now replaces python2-fuse and python-fuse, so python-fuse is not found.

Thanks

У каждого клиента будет веб интерфейс

У каждого клиента будет свой (локально запускаемый) веб интерфейс с доступом к общему хранилищу файлов?

1

Использование Windows сервера

Можно ли как-то сие чудо запустить на Windows 10? Попробовал подсистему Ubuntu - не работает FUSE, как и на Windows не могу найти необходимых софтин (Нашел winfsp и Docan). Дошел до ошибки именно с FUSE:

Error: The Python FUSE binding isn't installed!
If you're on Ubuntu try running `sudo apt-get install python-fuse'.

Что делать? Просто не очень очеться разворачивать виртулаку ради такого....

Docker and Docker hub Support

As of right now, it is complicated to host the software. It would be better if there was a container, that I could just pull and this would work.

Links for documentation and Container hosting service

Docker Docs: https://docs.docker.com/
Docker Hub: https://hub.docker.com/

Python 3 compatibility issues

Python 2 end of life is in less than 200 days...

flake8 testing of https://github.com/SlavikMIPT/tgcloud on Python 3.7.1

$ flake8 . --count --select=E9,F63,F72,F82 --show-source --statistics

./dedupfs/get_memory_usage.py:42:13: E999 SyntaxError: invalid syntax
    print msg % (format_size(counter), format_size(new_memory_usage), format_size(difference), format_size(deviation))
            ^
./dedupfs/fuse.py:59:34: E999 SyntaxError: invalid syntax
                           (var, `pattern`))
                                 ^
./dedupfs/dedupfs.py:49:19: E999 SyntaxError: invalid syntax
except ImportError, e:
                  ^
3     E999 SyntaxError: invalid syntax
3

E901,E999,F821,F822,F823 are the "showstopper" flake8 issues that can halt the runtime with a SyntaxError, NameError, etc. These 5 are different from most other flake8 issues which are merely "style violations" -- useful for readability but they do not effect runtime safety.

• F821: undefined name name
• F822: undefined name name in __all__
• F823: local variable name referenced before assignment
• E901: SyntaxError or IndentationError
• E999: SyntaxError -- failed to compile a file into an Abstract Syntax Tree

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Abandoned?

The last commit was 2 years 3 months ago, and also python2 is still being used.

Отсутсвует шаблон tg_access.py

В tg_access.py нужно вставить данные api_hash, api_id и номер телефона, но нет четкого шаблона (как и самого файла). Возможно, было бы логично добавить его?

Нулевой размер файлов

Здравствуйте!
Использовал tgcloud на 32-битном Linux Mint Tricia. Были довольно частые вылеты (те, которые number of entries reached 0), те же торренты качать не получалось (в общем-то, стабильнее всего работали gallery-dl и youtube-dl), однако облако работало и файлы upipe в телеге действительно появлялись. Когда перешёл на 64-битный Linux Mint Ulyana, стал тоже устанавливать tgcloud по инструкции. Однако здесь файлы upipe в телегу не отправляются (вместо этого они у меня скапливались в папке tgcloud), а любой файл, который помещаешь в папку storage (копированием или как-то ещё), становится там нулевого размера и ожидаемо не открывается. Пробовал создавать текстовые файлы и изменять их, сразу же после выхода из xed они становятся снова пустыми. Файлы, которые я на 32-битной системе успел сохранить в облаке, больше не доступны. Проверял на старом ПК и новом ноутбуке, а также на планшете с Linux Deploy (там установлена Debian, архитектура armhf).
Заранее спасибо!

Удаленный доступ

Кто как делает удаленный доступ к папки в которая смонтирована в dedupfs? Просто до сих пор не могу найти рабочих решений: пробовал использовать sftp но нет папки смонтированной, filebrowser , который предложен автором крайне долго читает папку, хотя там только ~2Gb информации (минут 5 наверно надо ждать), если с винды, то поставил sshfs и winfsp, но как дело доходит до работы со смонтированой папкой, то пиши пропало. Все чем можно закидывать файлы и читать это nautilus, и то sftp так же не видит, т.е. только прямой доступ, что не очень. Буду рад любым рабочим вариантам (желатьно с возможностью работы на windows).

ФС как сервис

Попробовал сделать запуск файловой системы серивсом, нашел не доработки:

• sudo cp tgcloud.service /ect/systemd/system надо поменять /ect на /etc. Просто опечатка.
• В самом tgcloud.serive: обычно python2.7 установлен в /usr/bin/python2.7, а не /bin/python2.7.