librecat / librecat

A publication management system

LibreCat - an institutional repository

LibreCat is a new institutional repository system developed by LibreCat Group which has as its key features:

• institutional repository
• publication list manager for reseachers
• institutional research data archive.

The development started in 2013 in Bielefeld and was made available on GitHub from the start. Since 2015 the code is in production at Bielefeld. In 2016 Ghent University started using the cataloging backend in production.

Features

• Cataloging of many record types: Book, Book (Editor), Book Chapter, Book Review, Conference Abstract, Conference (Editor), Conference Paper, Dissertation, Encyclopedia Article, Journal Article, Special Issue, Newspaper Article, Preprint, Report, Translation, Translation (Section), Working Paper, Thesis, Research Data, Project, Award, Research Group
• Drag and drop upload of full-text publications
• Copycat from DOI, PubMED, ArXiv and Web of Science
• Google Scholar indexation support
• Citation styles configurable from Zotero Style Repository
• Full MathJAX Latex support to add mathematical formulas in abstracts and titles
• Pluggable authentication modules
• Delegate input and management to user others
• Multilingual support
• ElasticSearch indexing
• Pluggable file store backend
• Command line support using 'Catmandu'
• OAI-PMH and SRU
• REST / content negotiation
• Signposting
• The LibreCat is open source and shipped with the same license as the Perl language: http://dev.perl.org/licenses/

Install

See our Wiki at: https://github.com/LibreCat/LibreCat/wiki

librecat's Issues

Thumbnail access and privacy issues

The thumbnail implementation needs to be improved:

• In Ghent we have a use case where thumbnails for closed access material should be hidden (the front page of an article can contain sensitive material)
• Thumbnails are now generated on an record level. This should maybe be done on a article document level?
• Thumbnails should maybe written in a FileStore separate from the fulltext content (faster access, easier to regenerate)
• Add command line tools to easy regenerate thumbnails

Documenting the schema

I have started writing a data schema documentation, see Data Schema. I try to figure out the meaning of the fields. Could you please check it whether the intented usage is in sync what I have recorded, and fill the gaps I was not able to figure out?

Button to interrupt file upload

Usability: once an upload has started in the dropzone, it can't be stopped.

We need a button for this and I know that dropzone.js supports it.

Single Sign On: howto

There is now a new branch "feature_sso", with support
for CAS authentication. It works in the following way:
(See LibreCat::Auth::SSO)

There are two controllers:

• one controller that handles all authentication with the external login service (CAS, ORCID). This is always a Plack Application. When done, it registers a session key called "auth_sso", that look like this:

{
   "type" => "CAS",
   "response" => "<xml response of cas ..>"
}

and redirects to the authorization controller.

Example: LibreCat::Auth::SSO::CAS

• one controller that uses the information in "auth_sso" to create a user session.
  This is called the authorization controller. This can be either a Plack app or a dancer route.
  This is typically done by application code. This controller must have access to the
  same session information as the authentication controller.

Control flow (example):

• go to "/auth/cas" ( just an example )
• LibreCat::Auth::SSO::CAS is mounted on "/auth/cas", and checks if the session_key "auth_sso" is present.
  • not present: redirects to CAS, but adds its own url as the service
    cas returns to this controller with a ticket, and it validates the received ticket
    and creates "auth_sso" that contains the ticket.
    redirects to authorization controller "/auth/cas/callback".
  • present: redirects to the authorization_url "/auth/cas/callback"
• authorization controller "/auth/cas/callback":
  • checks if session key "auth_sso" exists:
    • true: create user session
    • false: access denied

This flow is inspired by omniauth.

Does this suit your needs?

P.S. this branch has been created a long time ago, so it probably cannot be merged automatically..

Technical metadata is cleared when updating the file metadata

For instance, add an upload to a record. Submit. Update the upload (e.g. set other access rights). The content_type , file_size will be cleared and the date_created reset to the current date

Typo in catmandu.schemas.yml: abtract

Among the publication properties there is an "abtract" instead of "abstract".

Installation doesn't work as documented, dependency missing in CPAN

Running the cpanminus application to install dependencies fails for Catmandu::Importer::CrossRef

MacBook-Pro-5:LibreCat cm$ sudo cpanm-5.22 --notest --installdeps .
--> Working on .
Configuring /Users/cm/projects/LibreCat ... OK
==> Found dependencies: Catmandu::Importer::CrossRef
! Finding Catmandu::Importer::CrossRef on cpanmetadb failed.
! Finding Catmandu::Importer::CrossRef (0) on mirror http://www.cpan.org failed.
! Couldn't find module or a distribution Catmandu::Importer::CrossRef
! Installing the dependencies failed: Module 'Catmandu::Importer::CrossRef' is not installed
! Bailing out the installation for ..

Quick and Easy upload support for delegates and reviewers doesn't work

The QAE upload has support in code to set a reviewer or a delegate, but the fields are never set by any controller or route and are always left empty in the forms as presented in the webinterface.

Is this setting of delegates/reviewers a feature/bug that needs to be solved or a work in progress that needs to be implemented, or something else?

Language setting for abstract doesn't work

Setting a new language in the abstract will result in a correct storage of the language in the record. But, when editing the record again the language in the interface is switched back to english

Add some security enhancers

E.g.

https://metacpan.org/pod/Plack::Middleware::CSRFBlock

Add a white list of fields that are allowed to be updated via a web form

It is possible that services such as CSRF add extra fields to the web forms which end up in the database. Also, malicious users can change the content of the form and add extra fields and parameters that can overwrite stored fields.

What is the difference between publications' type and document_type?

In the schema document the type and document_type fields are very similar. The only difference is that type can have "original" as a value, while document_type has not. The rest of the values (bi_dissertation, book, book_chapter, book_editor, conference, conference_abstract, conference_editor, dissertation, encyclopedia_article, journal_article, journal_editor, newspaper_article, patent, preprint, report, research_data, review, translation, translation_chapter, working_paper) are identical.

• What is the real difference between these two fields?
• What does "original" type mean?
• Why the *_editor types (book_editor, journal_editor, conference_editor) on the list? For my perspective "editor" should be a type of contribution, and not a document type.

RSS feeds

In the author pages there is in the HTML header a reference to daily/weekly/monthly RSS feeds. These links don't seem to work, nor the http://localhost:5001/feed

Research Data form - Add a contact address for download

Data publications can point to the dataset by:

• Uploads in LibreCat
• Links to an external resource

We got a request to add a third option: the researcher can add contact details (email etc) to be displayed. "Please contact .... at ... for access to this dataset"

Implementing an if/elseif/else branch in fix file - how?

I am working on a mapping Web of Science metadata file into LibreCat publication file. I have a field called PT (publication type), which might have four different values: "J" (journal), "B" (book), "S" (series) and "P" (patent). I'd like to translate them to some LibreCat values, but since I haven't found construction for elsif (I tried else if, elseif, elif etc.). What I finally came up was this:

if any_match(PT,"^J$")
  set_field(_r.type, "journal_article")
else
  if any_match(PT,"^B$")
    set_field(_r.type, "book")
  else
    if any_match(PT,"^P$")
      set_field(_r.type, "patent")
    else
      if any_match(PT,"^S$")
        set_field(_r.type, "series")
      else
        copy_field('PT', '_r.type')
      end
    end
  end
end

Is there a construction like the following one?

if any_match(PT,"^J$")
  set_field(_r.type, "journal_article")
elsif any_match(PT,"^B$")
  set_field(_r.type, "book")
elsif any_match(PT,"^P$")
  set_field(_r.type, "patent")
elsif any_match(PT,"^S$")
  set_field(_r.type, "series")
else
  copy_field('PT', '_r.type')
end

OAI-PMH metadata format export problems

The rdf produced illegal XML:

 <fabio:hasPubMedId>1</fabio>

The mods export isn't working:

 {

"error": "Warning caught during route execution: Use of uninitialized value in concatenation (.) or string at /opt/librecat/lib/LibreCat/App/Search/Route/api.pm line 56.\n",
"exception": "Warning caught during route execution: Use of uninitialized value in concatenation (.) or string at /opt/librecat/lib/LibreCat/App/Search/Route/api.pm line 56.\n"
}

HTML tags should be escaped in all templates

Not all HTML templates do escape user input. All user values should have an 'html' filter in the template toolkit

[% foobar | html %]

Setting personal alias doesn't work

Login as normal user. Click on the text 'No Author IDs set' next to your avatar. Fill in a text in Alias. Press save.

Gives an 404 with the error

{
  "error": "Serializer (Dancer::Serializer::JSON) failed at serializing http://localhost:5001/librecat:\nencountered object 'http://localhost:5001/librecat', but neither allow_blessed, convert_blessed nor allow_tags settings are enabled (or TO_JSON/FREEZE method missing) at (eval 563) line 151.\n"
}

Librecat needs a layout and code stripped from Bielefeld specific logos names

Bielefeld specifics

• Logos, banners, text on the homepage
• HRZ specific fields in login screen
• Explicit connections to the Bielefeld Information System in the App controllers and routes
• PubTheses services, links to Research Data Management etc

Insight from user tracking

Related to the question of how to provide the best possible user experience:

Has anyone used user tracking on LibreCat sites in the past and can give me some insights like what typical and/or frequent user flows are and what features are most used?

Adding new routes, overriding existing routes

Do we need something like tihs? Tell us your use case!

Adding a new external user gives - "Person ID not found" after login

Add a new external user as super admin.
Login as the new user and you get a 'Person ID not found' message on screen.
Adding new records for this person will lead to:

"error": "Warning caught during route execution: Use of uninitialized value in string eq at /opt/librecat/bin/../lib/Catmandu/Fix/fix_too_many_departments.pm line 16.\n"

Infinite loop in MIME::Types?

t/LibreCat/FileStore/BagIt.t hangs on loading the mimetypes database file

$ perl t/LibreCat/FileStore/BagIt.t

Gets away with downgrading MIME::Types to 1.38

Updates of bibliographic metadata triggers (unneeded) regeneration of thumbnails

Thumbnail generation can be expensive process. Regeneration should only be done when needed

Commonmarkjs and Mathjax installation

There is in LibreCat a dependency in the templates on CommonMark and MathJax but this is not included in the github, nor part of the install documentation. Is there any preferred version to install or a preferred installation routine? Are there other Javascript dependencies not included in the when cloning the LibreCat git and need extra installation steps?

File-upload allows to upload a file more than one time

Go into a record. Upload a file, upload the file again and again. Multiple file sections pointing to the same file will be added. When at a later stage one of these file are deleted, all other copies loose their associated binary content.

Add technical metadata to the file download options

• File size
• Mimetype
• Last version date
• Checksum

The _version field of each record is counted globally but not on record level

Each record contains a _version field. This is a counter for the versions of the record. But, in the current 'dev' branch this counter works on a global level. The first record gets _version = 1 , the record record _version = 2 etc

Better handling of streaming data

Downloads are handled by streaming data to the browser. If the remote clients closes the connection the streaming on the server continues. Need a code implementation that fixes this.

Validate ISBN/ISSN etc.

use fixes from Catmandu::Identifier.

What we do in case on invalid values? Return the form with an error? Delete the field and save record to db?

Fields Links->Link to external source are not clear how they work

These fields are not stored in the 'dev' branch

Logout button not working

In the 'dev' branch.

After logging into LibreCat with success , pressing 'logout' will keep the user logged in.

Inconsistent use of whitespaces in index.sh file

The trailing whitespace characters (spaces and TABs) are inconsistent in ./index.sh file.

App::Catalogue::Route::upload isn't connected to a FileStore abstraction

There are still direct access to the file system left in the code that need to be switched to the configured FileStore

Add support (automatic) registration of users

Currently in LibreCat, a user needs to be registered in the researchers database to get a working login (via LDAP or any other method). This is not feasible in an institute like Ghent, where we can't have an administrative workflow to register users.

1. Create a page where authenticated LDAP users can register themselves.
2. Possible make this registration automatic (with as less administrative fuss as possible).

Fix deletion of files from the repository

When deleting a file from a record, they are not possible anymore in the public view but are still available in the backend database. Provide functionality to optionally clean the backend of deleted files (and delete the thumbnails if required).

ISBN field is not saved in web form

Create a new record. Fill in an ISBN/ISSN and click on save. The field is missing in the data

Record storage in ElasticSearch and Mongo/MySQL

In the current branched all publications are stored in ElasticSearch first and optionally backupped in Mongo/MySQL later. This workflow has the wrong order. Records should be first stored in the secure/permanent store (Mongo/MySQL) and indexed in an ElasticSearch as a second step.

Add documentation how to install/activate the citation style processor (CSL)

The configuration is available in:

 catmandu.citation.yml

But where to find the software, what is required,...

Provide Vagrantfile for dev environment

Still have problems to get gearman running or to correctly install the perl module (some C errors appear).

Maybe it is more convenient to have a Vagrantfile in this repository, where everything works...

Mechanism for customizing/overriding default views

Related to #6
By removing institution-specific layouts/logos/etc. we need a mechanism to override the default views and put the custom views in place (maybe it's useful to replace only some of them).

@nics Do you think we could use Template Toolkit's INCLUDE_PATH option (which can handle an array of paths)?

INCLUDE_PATH => '/my/path1:/my/path2'

And do we know if this usage is safe? (i.e. always searches /my/path1 first?)

Any other ideas?

Question: is there a way to list/display departments

I have just a question: in the data schema there is a department entity. Where it is displayed? I am working on importing the department list and connecting IDs to researchers, but I don't see where departmental info is displayed - other than in the list of authors?

Template Toolkit fails when then Perl app dies

When a Dancer application dies, the template toolkit becomes unstable. The output of the web forms only contain literal template toolkit code

Typing a return key in a input field triggers the submit of the form

Edit any record. Put the focus on a input field and press 'return'. The form will be automatically submitted (but any changes are cancelled).

What is the use of 'first_author' and 'add_autorenansetzung' in the codebase

I see the 'first_author' is a field defined in the EL configuration and is used in the templates, but it is never set? There is a 'add_autorenansetzung' fix but it is not used?

Support Datacite XML v4

see http://schema.labs.datacite.org/meta/kernel-4/

Create command line tools to sync file store information with stored record information

When updating/uploading files in the file store via the command line tools, the publication data should be updated too.

Maybe create a separate command to sync this data:

 $ bin/librecat store add 123 /tmp/myfile.pdf
 $ bin/librecat publication filesync 123

Missing field in the schema description (researcher/department.name)

From the automatically generated schema Wiki page the researcher's department name is missing.

Here is an extract from the devel/researcher.yml, which shows the missing field:

department:
- _id: 9996
  name: 'Relativistic Theory Group'

In the Wiki page only the department._id is declared, the department.name is missing.

Marked publications doesn't work

• Go to an author page.
• Mark one or more publications.
• Click "Marked Publication(s)"
• The page is empty

dev master merge

I have merged our current master branch into the dev branch and created a new branch called test_merge_dev (I know, I'm not very creative when it comes to branch names...).

There are still some issues with file upload (or the file store?): dropzone in the edit form doesn't work. And with request a copy: creation of rac link doesn't work. But maybe these are problems that already existed in the original dev branch?

Once these problems have been solved, you should be able to merge the test_merge_dev branch into dev.