proycon / clam Goto Github PK

The metadata will be generated on project start so the information is there, but there is currently no good high-level function to get to it. This is needed to get the output filename directly, rather than deriving it again, prevent duplication of either filename, or code to derive it.

When uploading a file with a different filename than the one in InputTemplate/filename, it should automatically get renamed. The old name persists now.

CLAM may currently be too strict to be usable by AJAX clients.

Cross-Origin Request Blocked: The Same Origin Policy disallows reading                                                                                                                                                                                 
the remote resource at http://www.spraak.org/webservice/transcribe/.                                                                                                                                                                                    (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).       

Allow configurable Access-Control-Allow-Origin on all CLAM responses.

Difficult issue with urllib2 and poster.streaminghttp library. They don't play along nicely when large files are uploaded. Fairly complicated to fix.

(low priority, please comment if you would like this feature to be implementer sooner)

Screen input: (no files present yet)
When you forget to add a filename, and push START the system presents some minimal output,
but doesn't work and doesn't tell you what went wrong.
Why not a default filename. Or warning.

(by Stefan Fischer)
The alignment of the 'Show input files' button is weird. Why not center it?

See http://swagger.io/ , https://openapi.org

CLAM is currently English only. Please vote here if you would like it to be properly internationalized so it can support multiple languages.

Implement extra logging of user activity
Allow administrators to more easily manage projects (view logs, delete, pause)

(Requested by Daan Broeder, MPI)

doesn't work

Trying to install clam through pip fails:

$ pip install --install-option="--prefix=/scratch/lars/apps" clam

gives the error message:

Downloading/unpacking clam
  Downloading CLAM-0.7.8.3.tar.gz (86Kb): 86Kb downloaded
  In the tar file /tmp/pip-zqCe0E-unpack/CLAM-0.7.8.3.tar.gz the member CLAM-0.7.8.3/clam/config/oersetter.py is invalid: 'NoneType' object has no attribute 'isreg'
  Running setup.py egg_info for package clam
    Traceback (most recent call last):
      File "<string>", line 14, in <module>
      File "/scratch/lars/apps/src/build/clam/setup.py", line 8, in <module>
        os.chdir(os.path.dirname(sys.argv[0]))
    OSError: [Errno 2] No such file or directory: ''
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 14, in <module>

  File "/scratch/lars/apps/src/build/clam/setup.py", line 8, in <module>

    os.chdir(os.path.dirname(sys.argv[0]))

OSError: [Errno 2] No such file or directory: ''

----------------------------------------
Command python setup.py egg_info failed with error code 1
Storing complete log in /home/buitinck/.pip/pip.log

The pip.log file contains the same exception message, followed by the traceback:

Command python setup.py egg_info failed with error code 1
Exception information:
Traceback (most recent call last):
  File "/scratch/lars/apps/lib/python2.6/site-packages/pip-1.0.2-py2.6.egg/pip/basecommand.py", line 126, in main
    self.run(options, args)
  File "/scratch/lars/apps/lib/python2.6/site-packages/pip-1.0.2-py2.6.egg/pip/commands/install.py", line 223, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/scratch/lars/apps/lib/python2.6/site-packages/pip-1.0.2-py2.6.egg/pip/req.py", line 986, in prepare_files
    req_to_install.run_egg_info()
  File "/scratch/lars/apps/lib/python2.6/site-packages/pip-1.0.2-py2.6.egg/pip/req.py", line 222, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/scratch/lars/apps/lib/python2.6/site-packages/pip-1.0.2-py2.6.egg/pip/__init__.py", line 255, in call_subprocess
    % (command_desc, proc.returncode))
InstallationError: Command python setup.py egg_info failed with error code 1

I'm using CLAM in Python 2 and I think I found a bug; in clam/common/viewers.py there's a if-else-branch that will use BytesIO if the sys.version is < 3 and lines[0] is a str, but BytesIO is not imported. Changing this occurrence to StringIO seems to resolve the issue.

There's another occurrence of this a few lines later. If you want I can make a pull request.

Related to issue proycon/flat#50 , for FLAT >= v0.7.1

Would it be an idea to disable wrapping in the SimpleTableViewer? Or add a setting in the API?
An example use case is disabling sentences and values for that sentence in a tableviewer. Currently browsers will wrap then wrap the text.

Maybe this is already possible, but it would be nice to be able to use Python's dictionary method get() (with a default if the key is not found) for CLAMData, so that you can put e.g.

clamdata.get('test', 1.23)

instead of

test = 1.23
if 'test' in clamdata:
     test = clamdata['test']

or something equivalent. Maybe by subclassing Mapping?

Implement mechanism to pass certificate (to requests library) in CLAM Client API. Currently fails:

See also: http://stackoverflow.com/questions/10667960/python-requests-throwing-up-sslerror

   File "<ipython-input-3-6bd0a1e6835d>", line 1, in <module>
     runfile('C:/xampp/htdocs/bramvanroy/projects/tree-visualizer/get-alpino-parse.py', wdir='C:/xampp/htdocs/bramvanroy/projects/tree-visualizer')

   File "C:\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 685, in runfile
     execfile(filename, namespace)

   File "C:\Anaconda3\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 85, in execfile
     exec(compile(open(filename, 'rb').read(), filename, 'exec'), namespace)

   File "C:/xampp/htdocs/bramvanroy/projects/tree-visualizer/get-alpino-parse.py", line 22, in <module>
     clamclient.create(project)

   File "C:\Anaconda3\lib\site-packages\clam-2.1.0-py3.4.egg\clam\common\client.py", line 212, in create
     return self.request(project + '/', 'PUT')

   File "C:\Anaconda3\lib\site-packages\clam-2.1.0-py3.4.egg\clam\common\client.py", line 134, in request
     r = request(self.url + url,**requestparams)

   File "C:\Anaconda3\lib\site-packages\requests\api.py", line 122, in put
     return request('put', url, data=data, **kwargs)

   File "C:\Anaconda3\lib\site-packages\requests\api.py", line 50, in request
     response = session.request(method=method, url=url, **kwargs)

   File "C:\Anaconda3\lib\site-packages\requests\sessions.py", line 465, in request
     resp = self.send(prep, **send_kwargs)

   File "C:\Anaconda3\lib\site-packages\requests\sessions.py", line 573, in send
     r = adapter.send(request, **kwargs)

   File "C:\Anaconda3\lib\site-packages\requests\adapters.py", line 431, in send
     if isinstance(e, _SSLError):

 SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)

(submitted by Stefan Fischer, to be reproduced still)

Hi!

I wonder if it's possible to use a URL to a document as input for CLAM directly. It would be very interesting.

When working with optional inputtemplates, fetching the file seems a bit cumbersome (or I'm doing it wrong). If I would define an optional InputTemplate like so:

InputTemplate('nounclassification', PlainTextFormat, 'Noun Classification',
    StaticParameter(id='encoding',name='Encoding',
        description='The character encoding of the file',value='utf-8'),
    extension='.data',
    optional=True,
    unique=True,
)

Fetching the inputfile and arranging an alternative if not present seems to need a try-catch-all-block:

try:
    nc = clamdata.inputfile('nounclassification')
    # Do something useful here
except Exception:
    # When no inputfile is found, revert to the default

Is there a more clean way to work with optional inputfiles?

(by Stefan Fischer)
The log message container is rather small. I think twice as many lines would make scrolling through the list easier.

(by Stefan Fischer)
It would be nice to have a variable CUSTOMHTML_PROJECTFAILED. The user could be provided with hints about possible reasons for failure

When there is only one choice possible for the filetype, it is quite annoying to be forced to make a choice anyway (both upload and screen input)

Hi!
I would like to know if it's possible to make anonymous users (CLAM without user admin) have cookie based sessions. The idea is to have different projects, files and results based on cookies instead of users. Now annoym users share their project, files, etc.
Thanks

Only one choice available to data API. Occurs in frogwrapper.py ..

Clients may fail to cleanup projects for whatever reason, a facility to remove stale projects is desired.

It seems feasible to add CWL support to CLAM: a CWL workflow should convertible to a CLAM profile. The CLAM wrapper can in turn invoke cwltool (the CWL reference implementation) with the cwl workflow and an input yml file with the user input (derived on the fly from the CLAM xml file).

Please upvote if this feature is desired.

Relevant resources:

• https://github.com/NLeSC/scriptcwl
• https://github.com/common-workflow-language/cwltool

Support authentication using OAuth, delegated authentication & single sign-on.. To be researched and proposed still, will have to be devised in collaboration with Max Planck Institute and CLARIN needs. Please comment if you would like this feature.

Libraries to consider: https://github.com/litl/rauth