alpernebbi / git-annex-adapter Goto Github PK
View Code? Open in Web Editor NEWCall git-annex commands from Python
License: GNU General Public License v3.0
Call git-annex commands from Python
License: GNU General Public License v3.0
The tests included in the source do not seem to match the to-be-tested code. The use a completely different API ...Repo.make_annex()
etc. Are these meant to work?
Some fairly trivial errors occur when running the tests using pygit2 1.2.1:
ERROR: test_file_tree (tests.test_repo.TestGitAnnexRepoOnThreeFiles) (file='foo')
Flat AnnexedFileTree should be correct.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/build/source/tests/test_repo.py", line 40, in test_file_tree
self.assertEqual(tree[f].key, k)
AttributeError: '_pygit2.Blob' object has no attribute 'key'
======================================================================
ERROR: test_file_tree (tests.test_repo.TestGitAnnexRepoOnThreeFiles) (file='bar')
Flat AnnexedFileTree should be correct.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/build/source/tests/test_repo.py", line 40, in test_file_tree
self.assertEqual(tree[f].key, k)
AttributeError: '_pygit2.Blob' object has no attribute 'key'
======================================================================
ERROR: test_file_tree (tests.test_repo.TestGitAnnexRepoOnThreeFiles) (file='baz')
Flat AnnexedFileTree should be correct.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/build/source/tests/test_repo.py", line 40, in test_file_tree
self.assertEqual(tree[f].key, k)
AttributeError: '_pygit2.Blob' object has no attribute 'key'
due to some API deprecation I assume.
I want to use git-annex-adapter to add multiple files to git-annex and to also set their metadata.
However, the AnnexedFileTree class expects to work with a committed treeish. But git-annex can already run git-annex metadata -s
directly after git-annex add
but before git commit
.
So for now, I need to first commit a file before I can manipulate the metadata with git-annex-adapter.
Currently, git-annex 6.20160727 (or later)
is supported, but this isn't in most of the distro repositories:
6.20160719-1
6.20160719-1
5.20151208-1build1
6.20161118-1
6.20161118-1
6.20161012-1
5.20141125
5.20140717-12.fc25
5.20140717-12.fc25
The adapter should be made compatible with these versions until they are upgraded by the maintainers.
A short list of things that I rely on and aren't available in these versions:
metadata --batch --json
(6.20160727
)fields
object in JSON output. (6.20160726
)I'd need a function to get annexed files as in
git annex find --metadata my-key="*"
Are you still there to merge a patch and release a new version on PyPI?
So you know, https://github.com/datalad/datalad is a data management system that appears written in python and deeply interconnects between python and git-annex.
I'm looking into your tool because datalad seems kinda big and unwieldy for getting started small, but I was surprised not to see a reference to it.
Hello,
I'd like to package git-annex-metadata-gui for Debian. Could you tag a stable release, please?
Thanks.
Instead of relying on parsing the key from the blob, just use git-annex.
Add Dockerfile for easy verification of dependencies and consistent environment for test execution.
Python 3.7.0 (default, Sep 5 2018, 03:25:31)
[GCC 6.3.0 20170516] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import git_annex_adapter
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.7/site-packages/git_annex_adapter/__init__.py", line 50, in <module>
git_annex_version = float(version_str.split('-g', 1)[0])
ValueError: could not convert string to float: '6.20181011+git124'
>>> import os
>>> os.system('git-annex version')
git-annex version: 6.20181011+git124-g94aa0e2f6-1~ndall+1
build flags: Assistant Webapp Pairing S3(multipartupload)(storageclasses) WebDAV Inotify DBus DesktopNotify TorrentParser MagicMime Feeds Testsuite
dependency versions: aws-0.19 bloomfilter-2.0.1.0 cryptonite-0.25 DAV-1.3.2 feed-1.0.0.0 ghc-8.2.2 http-client-0.5.12 persistent-sqlite-2.8.1.2 torrent-10000.1.1 uuid-1.3.13 yesod-1.6.0
key/value backends: SHA256E SHA256 SHA512E SHA512 SHA224E SHA224 SHA384E SHA384 SHA3_256E SHA3_256 SHA3_512E SHA3_512 SHA3_224E SHA3_224 SHA3_384E SHA3_384 SKEIN256E SKEIN256 SKEIN512E SKEIN512 BLAKE2B256E BLAKE2B256 BLAKE2B512E BLAKE2B512 BLAKE2B160E BLAKE2B160 BLAKE2B224E BLAKE2B224 BLAKE2B384E BLAKE2B384 BLAKE2S256E BLAKE2S256 BLAKE2S160E BLAKE2S160 BLAKE2S224E BLAKE2S224 BLAKE2SP256E BLAKE2SP256 BLAKE2SP224E BLAKE2SP224 SHA1E SHA1 MD5E MD5 WORM URL
remote types: git gcrypt p2p S3 bup directory rsync web bittorrent webdav adb tahoe glacier ddar hook external
operating system: linux x86_64
supported repository versions: 3 5 6
upgrade supported from repository versions: 0 1 2 3 4 5
0
NeuroDebian have made a new release of git-annex-standalone, 8.20210223-1~ndall+1. This version string breaks the version detection in git_annex_adapter/__init.py
Traceback (most recent call last):
File ".../git_annex_adapter/__init__.py", line 56, in <module>
git_annex_version = float(version_str)
ValueError: could not convert string to float: '8.20210223-1~ndall+1'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./repo-mirror/git-annex-aware-repo-server.py", line 32, in <module>
from git_annex_adapter.exceptions import NotAGitAnnexRepoError
File ".../git_annex_adapter/__init__.py", line 59, in <module>
msg = fmt.format(git_annnex_version)
NameError: name 'git_annnex_version' is not defined
Pull request imminent
I'm trying to run the tests with python -m unittest
, but I'm getting the following error:
======================================================================
FAIL: test_process_git_status (tests.test_process.TestProcessOnEmptyAnnex)
Process should be able to read whole output
----------------------------------------------------------------------
Traceback (most recent call last):
File /tmp/nix-build-python3.6-git-annex-adapter-0.2.0.drv-0/git-annex-adapter-v0.2.0-src/tests/test_process.py, line 154, in test_process_git_status
+ 'and use git add to track)',
AssertionError: Lists differ: ['On [16 chars]'', 'No commits yet', '', 'nothing to commit ([42 chars]ck)'] != ['On [16 chars]'', 'Initial commit', '', 'nothing to commit ([42 chars]ck)']
First differing element 2:
'No commits yet'
'Initial commit'
['On branch master',
'',
- 'No commits yet',
+ 'Initial commit',
'',
'nothing to commit (create/copy files and use git add to track)']
----------------------------------------------------------------------
Could you please document somewhere how to run the tests? Maybe the cause is that the tests are run in a different order?
Some git-annex packages format the version in a way that is not expected by the check performed in git_annex_adapter/__init__.py
.
>>> import git_annex_adapter
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/src/app/git_annex_adapter/__init__.py", line 50, in <module>
git_annex_version = float(version_str.split('-g', 1)[0])
ValueError: could not convert string to float: '6.20180329+gitga5fe62bed-1~ndall+1'
Hi,
I'm looking for a way to automatically fetch/download annexed files on demand if they are not present locally. Is this possible with this package?
Many thanks
Is git-annex-adapter
intended to work against git-annex
8.x?
We get the following test failures
running install tests
.....F.F./nix/store/cpzs1hpwzs23c41haa4dap0zjfx6xych-python3-3.7.9/lib/python3.7/subprocess.py:883: ResourceWarning: subprocess 493 is still running
ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
./nix/store/cpzs1hpwzs23c41haa4dap0zjfx6xych-python3-3.7.9/lib/python3.7/subprocess.py:883: ResourceWarning: subprocess 670 is still running
ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
./nix/store/cpzs1hpwzs23c41haa4dap0zjfx6xych-python3-3.7.9/lib/python3.7/subprocess.py:883: ResourceWarning: subprocess 889 is still running
ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
..F............/nix/store/cpzs1hpwzs23c41haa4dap0zjfx6xych-python3-3.7.9/lib/python3.7/subprocess.py:883: ResourceWarning: subprocess 1646 is still running
ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
.
======================================================================
FAIL: test_init_annex_version_five (tests.test_init_annex.TestInitAnnexOnEmptyRepo)
Repository version 5 should be valid.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/build/source/tests/test_init_annex.py", line 58, in test_init_annex_version_five
self.assertEqual(annex_repo.config['annex.version'], '5')
AssertionError: '8' != '5'
- 8
+ 5
======================================================================
FAIL: test_init_annex_version_six (tests.test_init_annex.TestInitAnnexOnEmptyRepo)
Repository version 7 should be valid.
----------------------------------------------------------------------
Traceback (most recent call last):
File "/build/source/tests/test_init_annex.py", line 63, in test_init_annex_version_six
self.assertEqual(annex_repo.config['annex.version'], '7')
AssertionError: '8' != '7'
- 8
+ 7
======================================================================
FAIL: test_process_annex_info_batch (tests.test_process.TestProcessOnEmptyAnnex)
Process should be able to read multiple lines
----------------------------------------------------------------------
Traceback (most recent call last):
File "/build/source/tests/test_process.py", line 229, in test_process_annex_info_batch
'remote annex size: 0 bytes',
AssertionError: Lists differ: ['trust: semitrusted', 'remote annex keys: 0'] != ['remote annex keys: 0', 'remote annex size: 0 bytes']
First differing element 0:
'trust: semitrusted'
'remote annex keys: 0'
- ['trust: semitrusted', 'remote annex keys: 0']
+ ['remote annex keys: 0', 'remote annex size: 0 bytes']
----------------------------------------------------------------------
Ran 27 tests in 8.325s
FAILED (failures=3)
which worked with earlier versions of git-annex
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.