islandora / documentation Goto Github PK
View Code? Open in Web Editor NEWContains islandora's documentation and main issue queue.
License: MIT License
Contains islandora's documentation and main issue queue.
License: MIT License
Pull together a set of Islandora upgration test fixtures.
Explain what is going on. That would probably be helpful.
New users who have never installed VirtualBox, will go for the prominently marketed 5.0 version on virtualbox.org. Existing VirtualBox users, may naturally assume the need to upgrade to the latest version (my story).
Requirements on the install/README page should be updated to reflect this pitfall, maybe with a specific recommendation for VirtualBox 4.3.
Error when "vagrant up" is ran against VirtualBox 5.0:
...and add install scripts to it.
It would be great to have a visual diagram of how of how the components fit together. There is an example here but I don't know how much use it will be as a model for how we should approach this one.
The README is off to a good start but it would be awesome if you could add a bit more documentation to bring greater clarity to the build and deployment process. For example, karaf is mentioned in the deployment section but isn't listed in the requirements, and having a sequence of steps that could be followed (dare I say copy-and-pasted) to get setup would be helpful.
Islandora Sync needs a test suite.
No attempt is made to handle different versions of datastreams. I haven't even checked to see what the result of the current work is w/r/t versions.
You should split on foxml:datastreamVersion and process each individually. You can check out an example of splitting on xpath here: https://github.com/daniel-dgi/migration-utils/blob/camel-service/migration-service/src/main/java/org/fcrepo/migration/service/routes/ContainerIngesterRoute.java#L35
Some basic end-user documentation that needs to be written, with screenshots. The 7.x-1.x documentation has an example: https://wiki.duraspace.org/display/ISLANDORA/How+to+Purge+an+Object
The url is embarassingly hardcoded at the moment. Use the spring config to setup the fcrepo-camel component using property-placeholders so you can load in the url for the rest endpoint, as well as auth creds, from an external config file.
This applies to both this repo and the camel-service branch of https://github.com/daniel-dgi/migration-utils
This ticket applies to https://github.com/daniel-dgi/migration-utils/tree/camel-service
Create a custom processor to pretty print FOXML on the way into the service, as the way Fedora 3 exports it is not parsible by Saxon! Feel free to use a third party library to help you pretty print. No need to re-code the wheel here.
The processing should occur at the point of entry, just before the call to ingestContainer here: https://github.com/daniel-dgi/migration-utils/blob/camel-service/migration-service/src/main/java/org/fcrepo/migration/service/routes/FoxmlMigratorRoute.java#L15
This ticket applies to https://github.com/daniel-dgi/migration-utils/tree/camel-service
Use the transactional functionality of the fcrepo-camel component when performing upgrations.
See the README at https://github.com/acoburn/fcrepo-camel for more. Should be a matter of spring config and and adding a .transacted() statement at the beginning of things.
While poking around, I changed the types of the dc.Creator and dc.description fields (in structure > Content Types > Basic Image > manage fields to allow filtered HTML.
When I then enter some allowed html tags (e.g. <em />
) and click save, at that instant the Drupal content will show up with the tags rendered. But if I reload the page, the <em>
tags are sticking out again.
This probably needs a conversation: Should HTML even be allowed in DC fields?
add fcrepo camel component to install
Looking at the existing documentation, and pulling on discussions from the last couple Fedora 4 IG calls, I'd like to clarify what exactly folks are looking for documentation-wise. Currently we have documentation for:
@sprklinginfo @rosiel @dmoses would you mind articulating what else we could focus on documentation-wise? I can then pull that in a task list for this ticket, and we can collectively work on it during the sprint.
See the following "sub-issues" for this ticket:
"Does Pathauto also filter and replace mentions of repository URLs in the body of responses (i.e. returned RDF)?"
Should Fuseki have an Upstart script? In development it isn't much of a concern, but it would be good to have one for production later on.
As asked here: https://groups.google.com/forum/m/#!topic/islandora-dev/QE8uUpuqNzA
How will data flow from Drupal to Fedora?
Pull down the xpath field module and give it a whirl. Let's see how well it works on our xml datastreams in drupal.
Source is here: https://github.com/alxp/xpath_field
If it works well enough, add it to the vagrant install!
Some basic end-user documentation that needs to be written, with screenshots. The 7.x-1.x documentation has an example: https://wiki.duraspace.org/display/ISLANDORA/How+to+Add+an+Item+to+a+Digital+Collection
For some reason I brought up a completely new copy of this vagrant box and it set.
DocumentRoot /var/www/html/drupal/drupal
in /etc/apache2/site-enabled/000-default.conf, possibly the Document Root already has drupal in it from somewhere else and this line sticks a second one in it?
We should be using semantic versioning for the overall version number of the software in addition to each 'sub-project' (e.g. sync, services, etc...). Drupal modules will remain in the Drupal versioning scheme.
Directly related to #15
This ticket applies to https://github.com/daniel-dgi/migration-utils/tree/camel-service
Add the applicable object properties from /foxml:digitalObject/foxml:objectProperties to the parent container when migrating from foxml. Do this before ingesting datastreams. See the TODO here: https://github.com/daniel-dgi/migration-utils/blob/camel-service/migration-service/src/main/java/org/fcrepo/migration/service/routes/ContainerIngesterRoute.java#L32
Some basic end-user documentation that needs to be written, with screenshots. The 7.x-1.x documentation has an example: https://wiki.duraspace.org/display/ISLANDORA/How+to+Create+a+New+Islandora+Collection
See this crazy conversation here: https://github.com/Islandora/Islandora-Fedora4-Interest-Group/issues/11
Long story short, check out jquery.xmleditor and doctored.js and see if they're any good.
Let us know if one of them is a clear winner.
Created a collection via the Drupal interface.
(It did have the title "Archives & Special Collections")
When I went to delete it, it is removed from Drupal. But this is what I saw in the catalina.out logs.
[2015-09-28 18:06:09] LogUpdateProcessor INFO [collection1] webapp=/solr path=/update params={wt=json} {} 0 1
[2015-09-28 18:06:09] SolrCore ERROR org.apache.solr.common.SolrException: undefined field sm_parent_document_id
at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1269)
at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:434)
at org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74)
at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175)
at org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:207)
at org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:374)
at org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742)
at org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:387)
at org.apache.solr.parser.SolrQueryParserBase.handleQuotedTerm(SolrQueryParserBase.java:555)
at org.apache.solr.parser.QueryParser.Term(QueryParser.java:403)
at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185)
at org.apache.solr.parser.QueryParser.Query(QueryParser.java:139)
at org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96)
at org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151)
at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50)
at org.apache.solr.search.QParser.getQuery(QParser.java:141)
at org.apache.solr.update.DirectUpdateHandler2.getQuery(DirectUpdateHandler2.java:354)
at org.apache.solr.update.DirectUpdateHandler2.deleteByQuery(DirectUpdateHandler2.java:384)
at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:80)
at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:55)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalDelete(DistributedUpdateProcessor.java:931)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doDeleteByQuery(DistributedUpdateProcessor.java:1433)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:1226)
at org.apache.solr.update.processor.LogUpdateProcessor.processDelete(LogUpdateProcessorFactory.java:121)
at org.apache.solr.handler.loader.XMLLoader.processDelete(XMLLoader.java:349)
at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:278)
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:174)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:99)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1976)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:777)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:313)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
It is not removed from Fedora.
This enhancement is for work that has already been produced. If the design is not desirable by the solution pack owners, we will redesign and reimplement as they see fit. The goal is to contribute code that solves the issue below.
Currently the audio solution pack requires that LAME is installed. The main issue faced by someone trying to use the solution pack without LAME is that the drupal hook_requirements method of the solution pack uses REQUIREMENT_WARNING which will not allow normal operation of drupal's update.php to update the database - requiring an override. The minor issue is that messages are sent to the watchdog log on every audio ingest.
In this case, someone is not able to install LAME on the Drupal server, but is able to generate the PROXY_MP3 as a derivative to store in Fedora. The desired result is that the Drupal administrator should be able to install the module, use update.php, and the user would be able to use the player that the solution pack uses.
This work has already been done in the following way, but my customer (Smithsonian Institution) is open to redesign and rework by the company I work for:
islandora_audio.module:
Some basic end-user documentation that needs to be written, with screenshots. The 7.x-1.x documentation has an example: https://wiki.duraspace.org/display/ISLANDORA/How+to+Edit+an+Object%27s+Metadata
Islandora Services needs a test suite.
Create a drupal module for basic image to hold the drupal content type for it. It should be a node. You should have certain basic fields, along with file fields for MEDIUM_SIZE and TN datastreams, as well as xml fields for MODS/FITS (etc) datastreams. The particulars haven't been set in stone, and this will be subject to change over time. So let's just get the process started and try to model content in Drupal!
We should have this exported in code so that it is available to end users. Do not make this a drupal feature, as it should be considered a starting point for customization and not holy law.
Don't forget to add any extra modules you need (xml fields, file, etc...) to the vagrant install!
This is a followup to a discussion started during the March 27 meeting of the Fedora 4 Interest Group.
During the call, we discussed the impact of not using content model objects (as Islandora currently uses) but instead expressing collections as RDF properties on member objects. Some of the implications we identified are:
Topic for discussion: is it preferable to model collections content models as full Fedora 4 objects or to express collection membership content models as RDF simple string properties. Examples of implementations to illustrate your arguments would be useful.
We need to get a handle on how we're gong to nid -> path map nodes in drupal to containers in Fedora. Does it live in the drupal db? Externally? Make sure it can handle multisites, as there may be node collisions. Figuring this out will let us work on implementing services defined in the services docs.
Utilizing the vagrant aws and digitalocean plugins, do what the title says.
You should make another route to handle this to try and keep things clean.
The parent container's path is accessible as a property on the exchange, although it has already been set as a header that will be utilized by the fcrepo camel component. See: https://github.com/daniel-dgi/migration-utils/blob/camel-service/migration-service/src/main/java/org/fcrepo/migration/service/routes/DatastreamIngesterRoute.java#L23
The compound object solution pack adds an 'isConstituentOf' relationship to child objects' RELS-EXT. This should be listed on the Islandora namespace RELS-EXT predicates page. I can open a PR - let me know.
This includes version bump, and updating the download url, since the url pattern changed slightly with the 4.3.0 release.
https://github.com/fcrepo4-exts/fcrepo-camel-toolbox/releases/tag/fcrepo-camel-toolbox-4.3.0.
Also, verify messages are being send. I did a preliminary investigates, and saw a lot of 61616 message being unable to be send in catalina.out
.
Feel free to completely ignore this if it doesn't align with your preferences and direction right now but I think it may already, even at this early stage, be advantageous to split the different projects from each other and have a git repository for each. Along the lines of:
As the future will inevitably move in this direction anyway it would address the only point in the process that looks unrealistic / artificial from a prototype deployment / devops perspective. I'm thinking of:
https://github.com/Islandora-Labs/islandora/blob/7.x-2.x/install/drupal.sh#L52
-- where it would be more natural to clone the islandora modules repository into drupal, and ...
https://github.com/Islandora-Labs/islandora/blob/7.x-2.x/install/sync.sh#L5
-- where the sync script knows about the download location of the "umbrella" repository and works with it directly, unlike how the other, similar scripts (fcrepo, fuseki, solr) are (for the most part) self contained and can be understood independently.
add apache camel to install
See #50 @daniel-dgi comment #50 (comment) & #50 (comment)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.