gsdenys / alfresco-bulk-export

Automatically exported from code.google.com/p/alfresco-bulk-export

Hi I'm Denys 👋

A senior IT professional, with more than 15 years experience, focused on information, integration and content management technologies with several successful projects implemented in different markets. I've been working with product and software development crossing the lines between technical and business, also supporting pre-sales.

With long time of knowledge in high performance and corporate software, I was lucky to work with many different software, like Alfresco, Nuxeo, Bizagi, Activiti, Flowable, Camunda, Zeebe, Kong, Nginx, RabbitMQ among others, as well some programing languages like C++, Java, Python, Javascript, Lua, typescript and now curious about Go and .net Core.

The last 3 year I've been work with digital transformation (internally and externally) through technology improving, applications architecture and software factory management. Supporting the development team, I'm acting as guide leader to software development and application architecture.

alfresco-bulk-export's Issues

Export get wrong timestamp for 24h countrys

The generated timestamp get truncated if hours > 12, e.g. "17 sep 2018 14:54:15 GMT+0200 (CEST)" becomes "17 sep 2018 02:54:15 GMT+0200 (CEST)".

The expression "hh:mm:ss.SSSZ" should have capital "HH"-letters to support 24 hours clock.

Change this:

alfresco-bulk-export/src/main/java/org/alfresco/extensions/bulkexport/dao/AlfrescoExportDaoImpl.java

Line 514 in f6f615d

SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd'T'hh:mm:ss.SSSZ");

into this:
SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ");

does not export versioning of nodes

Hello, I'm on alfresco community 5.2 and the export tool seems to export only last version of each node, although the exportVersion option has been set to "true". Is this a known issue, or am I missing something in the parameters ?

Here is an example of the url I use : http://localhost/alfresco/service/extensions/bulkexport/export?nodeRef=workspace://SpacesStore/{my_node_id}&base=c://tmp&exportVersions=true

Thanks

Jar file added successfully but URL showing 404.

What steps will reproduce the problem?
1.stop server
2.added jar file & start server
3.run the following URL from browser
http://{host}:{port}/alfresco/service/extensions/bulkexport/export?nodeRef={node
ref}&base={base}
What is the expected output? What do you see instead?
it should show finished successfully message.
but it showing following error:-
The Web Script /alfresco/service/extensions/bulkexport/export has responded 
with a status of 404 - Not Found.

404 Description:     Requested resource is not available.

Message:    00310001 Script url /extensions/bulkexport/export does not map to a 
Web Script.

Exception:  org.springframework.extensions.webscripts.WebScriptException - 
00310001 Script url /extensions/bulkexport/export does not map to a Web Script.

org.springframework.extensions.webscripts.AbstractRuntime.executeScript(Abstract
Runtime.java:173)
org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebSc
riptServlet.java:118)
javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilt
erChain.java:305)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.
java:210)
org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizatio
nFilter.java:61)
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilt
erChain.java:243)
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.
java:210)
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:2
22)
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:1
23)
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.jav
a:472)
..............................
What version of the product are you using? On what operating system?
alfrersco-4.2.c and on unix server.

Please provide any additional information below.
[root@csuite3 bin]# java -jar alfresco-mmt.jar list  
/opt/alfresco-4.2.c/tomcat/webapps/alfresco.war
Module 'org.alfresco.extencion.bulkexport' installed in 
'/opt/alfresco-4.2.c/tomcat/webapps/alfresco.war'
   -    Title:        Bulk Export
   -    Version:      0.0.5
   -    Install Date: Fri Jan 31 17:40:25 IST 2014
   -    Description:   Export data to Bulk Import format.
Module 'org.alfresco.integrations.google.docs' installed in 
'/opt/alfresco-4.2.c/tomcat/webapps/alfresco.war'
   -    Title:        Alfresco Google Docs Integration
   -    Version:      2.0.1
   -    Install Date: Fri Jan 31 16:56:05 IST 2014
   -    Description:   Alfresco Docs Integration
Module 'org.alfresco.module.vti' installed in 
'/opt/alfresco-4.2.c/tomcat/webapps/alfresco.war'
   -    Title:        Vti
   -    Version:      1.2
   -    Install Date: Fri Jan 31 16:56:03 IST 2014
   -    Description:   Alfresco Vti Extension
Module 'org_alfresco_module_wcmquickstart' installed in 
'/opt/alfresco-4.2.c/tomcat/webapps/alfresco.war'
   -    Title:        Web Quick Start
   -    Version:      1.0
   -    Install Date: Fri Jan 31 16:56:01 IST 2014
   -    Description:   Web Quick Start

Original issue reported on code.google.com by [email protected] on 31 Jan 2014 at 7:08

issues exporting multi tenant repository

Should it work?

When I try to export documentLibrary noderef from a tenant then the store cannot be found. Seems it looks for the files in the wrong place.

Used latest release, which at this point is org.alfresco.extencion.bulkexport version 0.0.6. And Alfresco CE 6.1.2.

java.lang.OutOfMemoryError: Java heap space

Hello!

I am trying to run the bulk exporter on an environment that has a large set of sub folders and documents. (something about 30GB, but we need export more data on another environment (about 1TB) is posible with that tool ?)

here is my export link for env with 30GB

http://host:8080/alfresco/service/extensions/bulkexport/export?nodeRef=workspace://SpacesStore/[XXXX]&base=D:/export&ignoreExported=false&exportVersions=true&revisionHead=false&useNodeCache=true&nbOfThreads=8

after some time ( typicaly 3 hours) im getting this exception

2021-11-18 14:14:52,052 ERROR [org.springframework.extensions.webscripts.AbstractRuntime] [http-nio-8080-exec-3] Exception from executeScript: 10180145 Wrapped Exception (with status template): 10181543 Exception in Transaction. org.springframework.extensions.webscripts.WebScriptException: 10180145 Wrapped Exception (with status template): 10181543 Exception in Transaction. at org.springframework.extensions.webscripts.AbstractWebScript.createStatusException(AbstractWebScript.java:1139) at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:171) at org.alfresco.repo.web.scripts.RepositoryContainer$3.execute(RepositoryContainer.java:519) at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:450) at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecute(RepositoryContainer.java:595) at org.alfresco.repo.web.scripts.RepositoryContainer.transactionedExecuteAs(RepositoryContainer.java:664) at org.alfresco.repo.web.scripts.RepositoryContainer.executeScriptInternal(RepositoryContainer.java:362) at org.alfresco.repo.web.scripts.RepositoryContainer.executeScript(RepositoryContainer.java:315) at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:399) at org.springframework.extensions.webscripts.AbstractRuntime.executeScript(AbstractRuntime.java:210) at org.springframework.extensions.webscripts.servlet.WebScriptServlet.service(WebScriptServlet.java:132) at org.alfresco.repo.web.scripts.AlfrescoWebScriptServlet.service(AlfrescoWebScriptServlet.java:43) at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.alfresco.web.app.servlet.ServletMetricsFilter.doFilter(ServletMetricsFilter.java:161) at org.alfresco.repo.web.filter.beans.BeanProxyFilter.doFilter(BeanProxyFilter.java:89) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.alfresco.web.scripts.servlet.X509ServletFilterBase.doFilter(X509ServletFilterBase.java:196) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.alfresco.web.app.servlet.GlobalLocalizationFilter.doFilter(GlobalLocalizationFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.alfresco.web.app.servlet.ClearSecurityContextFilter.doFilter(ClearSecurityContextFilter.java:53) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:201) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:544) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:698) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:698) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:311) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:764) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:568) at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:764) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:364) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:616) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1629) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: org.alfresco.error.AlfrescoRuntimeException: 10181543 Exception in Transaction. at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:536) at org.alfresco.repo.index.shard.ShardRegistryImpl.lambda$registerShardState$6(ShardRegistryImpl.java:297) at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:602) at org.alfresco.repo.index.shard.ShardRegistryImpl.registerShardState(ShardRegistryImpl.java:297) at org.alfresco.repo.solr.SOLRTrackingComponentImpl.registerShardState(SOLRTrackingComponentImpl.java:1370) at jdk.internal.reflect.GeneratedMethodAccessor727.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.alfresco.repo.management.subsystems.SubsystemProxyFactory$1.invoke(SubsystemProxyFactory.java:79) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) at com.sun.proxy.$Proxy121.registerShardState(Unknown Source) at org.alfresco.repo.web.scripts.solr.TransactionsGet.executeImpl(TransactionsGet.java:131) at org.springframework.extensions.webscripts.DeclarativeWebScript.executeImpl(DeclarativeWebScript.java:235) at org.springframework.extensions.webscripts.DeclarativeWebScript.execute(DeclarativeWebScript.java:64) ... 52 more Caused by: java.lang.OutOfMemoryError: Java heap space

Is there any idea what to do with that? Im new in alfresco soo be patient pls :-))

Our version:
Windows server 2016

Alfresco Share v6.2.1
(r90b39e43d479b3b3a1fccd89996a6c49e9140edc-b12, Aikau 1.0.101.19, Spring Surf 6.2.1, Spring WebScripts 7.14, Freemarker 2.3.28, Rhino 1.7.11, Yui 2.9.0-alfresco-20141223)

Alfresco Enterprise v6.2.1
(rffab6782-b2500) schema 13001

Ive just tried to incerase jvm like in this issue

https://hub.alfresco.com/t5/alfresco-content-services-forum/how-to-increase-jvm/td-p/48214/page/2

we have tomcat as windows service, but no properties tab "Java" on file tomcat8.exe :(

thank for your reply!

Correct a typo

I noticed that there is a typo in packages names and xml files.

The word extencion, with c, is incorrect... it would be extension, with s.

I've fixed this in my local repository, and I would want to add this fix to the 
project.

Original issue reported on code.google.com by [email protected] on 14 Sep 2012 at 1:40

Metadata with XML content

What steps will reproduce the problem?

1. Create new content in alfresco
2. Edit the content created posing on metadata (d: text) the contents of XML
3. Export this content
4. Import the result using bulk-Import.

This steps generate a node on destination but ignore XML.

What is the expected output? What do you see instead?

  I spected that during export execution all metadata special characters was converted to html entity. 

Original issue reported on code.google.com by [email protected] on 19 Mar 2012 at 8:32

Bulk export does not export nodes with no content

I have a node of type "cm:content" which does not have property "ContentModel.PROP_CONTENT" within a "cm:folder".
The bulk export does not export this node because of:

ContentReader reader = contentService.getReader(nodeRef, ContentModel.PROP_CONTENT);
        if (reader == null)
        {
            // no data for this node
            return false;
        }

within the following method:

org.alfresco.extensions.bulkexport.dao.AlfrescoExportDaoImpl.getContentAndStoreInFile(NodeRef, String)

Reporting showing status of export

What steps will reproduce the problem?

Enhancement: to show the status of the export either on screen or via a logfile 
- report could be a live copy of each file / folder in realtime or just a 
collated report at the end showing failures, errors, successful exports etc.

What version of the product are you using? On what operating system?

Currently using 0.0.5 version of the add-on.

Original issue reported on code.google.com by [email protected] on 30 May 2014 at 9:44

Releases...

Good morning,
I can find a release for 0.0.6.

Do you have a release for https://github.com/Alfresco/alfresco-bulk-export/releases/tag/bulkexport-1.0? or instructions on how to make this into an .amp?

Is 1.0 good code? Does it work?

Thanks.
Very neat module.

does not export custom metadata

hi,
the metadata.properties file is created correctly but the values for custom metadata entries are empty!

Bulk Exporter not exporting sub directories under the node ref specified

I am trying to run the bulk exporter on an environment that has a large set of sub folders and documents.

It only seems to import the content files, but doesn't recursively go to the subfolders. There is no error generated during the export and reports successful export.

This is the url I used:
http://localhost:8080/alfresco/s/extensions/bulkexport/export?base=/opt/alfresco/export&nodeRef=workspace://SpacesStore/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx&ignoreExported=true&useNodeCache=true

I tried this on another environment and it worked. Is there a limitation on the number sub folders the bulk exporter can traverse?

Any assistance is greatly appreciated.

Peter Choe

Doesn't work, but Module management tool show it installed

What steps will reproduce the problem?
1.Shutdown Alfresco instance
2.Used the Alfresco Module Management Tool
3. See Imagen "bulkexporte list alfresco.war"
4. Start Alfresco instance
5. Doesn't show problems
6. try with 
alfresco/service/extensions/bulkexport/export?nodeRef={noderef}&base={base}&igno
reExported={ignoreExported?}
7. shows anything just Web Script Status 404 - Not Found
8. I saw inside of alfresco/service, and desn't show anything
9. try againt all this, and nothing.
10.version community 4.2.c 

Thanks you and import bulk work great tool

Original issue reported on code.google.com by [email protected] on 5 Sep 2013 at 3:15

Attachments:

• [bulkexporte list alfresco.war.PNG](https://storage.googleapis.com/google-code-attachments/alfresco-bulk-export/issue-7/comment-0/bulkexporte list alfresco.war.PNG)

Command Line Run

This Tools not support command line tools. 

Original issue reported on code.google.com by [email protected] on 4 Mar 2013 at 3:05

Export ACLs option

Hi
I know the bulk importer doesn't import the acls. But since we are navigating through the entire structure why not give the option to also export the ACL at the same time?

What do you think?

Simon

Multi-Value Properties

Multi-value properties are being written to metadata.properties.xml file in format like:
[apples, oranges, bananas]
Alfresco Bulk Filesystem Import Tool (BFSIT) imports these as same ([apples, oranges, bananas])
I revised code such that Multi-value properties will be written to metadata.properties.xml file as:
apples,oranges,bananas
without [] and spaces which allows for Multi-value properties to be correctly imported using BFSIT
I can add this code for review if wanted

Export tool do not export aspects or associations

As the title says the export tool does not export metadata about aspects and associations.

I have a folder with photos that automatically gets the exif:exif secondarytype when uploaded in Alfresco.
If I export that directory this is an example of metadat, you can see that non info about exif property are exported, in the alfresco appliance (and even if I check with CMIS Workbench) there are all the exif property setted.

So when I export and then import that dir in another Alfresco the exif property are lost.
The same behaviour appens if you try to export document with associations or other aspects.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
	<entry key="type">mioFoto:foto</entry>
	<entry key="aspects">cm:versionable,cm:titled,cm:auditable,cm:geographic,sys:referenceable,sys:localized,cm:author,exif:exif</entry>
	<entry key="mio:uiVisibleProperties"></entry>
	<entry key="cm:creator">admin</entry>
	<entry key="cm:autoVersion">true</entry>
	<entry key="cm:title"></entry>
	<entry key="cm:modifier">admin</entry>
	<entry key="cm:versionType"></entry>
	<entry key="cm:versionLabel">1.0</entry>
	<entry key="cm:autoVersionOnUpdateProps">false</entry>
	<entry key="cm:created">2017-04-28T11:58:42.891+02:00</entry>
	<entry key="sys:store-protocol">workspace</entry>
	<entry key="sys:store-identifier">SpacesStore</entry>
	<entry key="mioFoto:autore"></entry>
	<entry key="cm:description"></entry>
	<entry key="cm:name">2017-04-04 13.36.15.jpg</entry>
	<entry key="cm:author"></entry>
	<entry key="cm:longitude">11.339167</entry>
	<entry key="cm:initialVersion">true</entry>
	<entry key="sys:locale">it</entry>
	<entry key="cm:modified">2017-04-28T11:58:42.891+02:00</entry>
	<entry key="cm:latitude">44.496667</entry>
</properties>

export including versions of each document

Create a way to export including versions of each document

Original issue reported on code.google.com by [email protected] on 4 Mar 2013 at 3:06

Option to disable metadata.properties.xml file creation

What steps will reproduce the problem?

Enhancement: Include additional checkbox that can enable/disable the creation 
of metadata properties files during the export.

What is the expected output? What do you see instead?

Only the files themselves should be exported, with no accompanying 
metadata.properties.xml files if the option has been enabled

What version of the product are you using? On what operating system?

0.0.5

Original issue reported on code.google.com by [email protected] on 30 May 2014 at 9:45

A few files missing from export

We're exporting documents and workflow information from alfresco using this package. We're running into an issue where we're getting almost all of the documents except in rare cases where we're missing one or two.

The notable things about the file we're missing:

1. It was locked (is now unlocked, but still didn't export)
2. It's extension is .PDF rather than a .pdf (not sure if the git package uses case insensitivity.

Can anyone think of anything else that might be different about this particular file for it to not be exported?

Thanks.

Alfresco 3.4 Community

Make usefull in the Alfresco 3.4 Community

Original issue reported on code.google.com by [email protected] on 4 Mar 2013 at 3:08

Can export be done so that sub folders are not exported?

Is it possible to only export files from a node and not any sub folders in the node?

Metadata in XML content 2

Hello, 

For the metadata with xml content, you may need to adjust the code in 
FileFolder class from this: 

private String formatProperty(String value){

    //format < and >
    value = value.replaceAll("<", "&lt;").replaceAll(">", "&gt;");
    //format &
    value = value.replaceAll("&", "&amp;");

    return value;
}

to this: 

private String formatProperty(String value){

    //format &
    value = value.replaceAll("&", "&amp;");
    //format < and >
    value = value.replaceAll("<", "&lt;").replaceAll(">", "&gt;");

    return value;
}

Because after you replace "<" to "&lt;", it will result in: "&amp;lt;" which is 
not desired. 

Original issue reported on code.google.com by erenbekce on 29 Mar 2012 at 11:07