Coder Social home page Coder Social logo

edisonpem / strapi-plugin-import-export-content Goto Github PK

View Code? Open in Web Editor NEW
147.0 147.0 68.0 194 KB

Csv and Json import / export content plugin to Strapi

Home Page: https://www.npmjs.com/package/strapi-plugin-import-export-content

License: MIT License

JavaScript 97.34% CSS 2.66%
csv json strapi strapi-cms strapi-plugin strapicms

strapi-plugin-import-export-content's People

Contributors

creazy231 avatar edisonpem avatar francois2metz avatar heyjaypray avatar nurielmeni avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

strapi-plugin-import-export-content's Issues

Export feature not working on Latest Verison

This error is still not resolved. Still getting the error.

error1

When we check console the we see the following error message.

error2

using MongoDB atlas as the database with Strapi latest version

Kindly check

Import to Users Collection

Hi,

does the plugin support importing CSV to users collection?

I've installed the plugin and created a sample csv file containing a user data to be added.

But I dont know how to use it, since the Import Destination only shows Single Types collection.

any advice?

Can't Export

When I try to export data, I get the following error:

error: "Not Acceptable"
message: "could not parse: TypeError: Cannot destructure property 'attributes' of '(intermediate value)(intermediate value)(intermediate value)' as it is undefined."
statusCode: 406

Expand the list of CSV mime types

Thanks for putting the effort in compiling some steps of the tutorials in a easy to use solution!

Unfortunately a couple of days back, after I had installed the plugin on a Strapi instance, I encountered an issue where uploading a CSV file is not recognized and gets rejected, after a little further investigation it seems that the file has mime type text/plain instead of text/csv .

I've did some investigation, but nothing I tried helped, and also stumbled upon some information that CSV file, might as well be with mime application/vnd.ms-excel because of some MS Excel reasons..

Will it be an interest to include additional types or if you could suggest any feasible workaround?

No Relations imported!

Hello, first thanks for that plugin.

I have one to many relations in my collection, for that field also data is not importing, how can I do that?

For export there is no problem, all relations are exported, but while importing the exported file all relations discarded.

Thanks in advance.
Please let me know if you need any further details.

Can't import components and core_store

Not sure if I'm missing something but when trying to import exports from a mongo csv/json export there wasn't a way to import my components or core_store files.

Thanks!

Not able to import CSV properly

Hey,

Thanks for the work on this plugin. Having a bit of an issue though....

When I try to import from CSV I'm getting an error about the file type. The CSV I'm using is actually using a modified export so not sure what the issue is. I can however paste that same information as raw data and it's working just fine

Also, is it possible to put a feature on the roadmap that will merge existing records based on the id etc? I suppose the id might also need to be exported for that. If I figure out how to do it I'll mention but I'm still a newbie developer so a bit above my head at the moment...

Empty CSV fields can't be imported until "null" is written there

If I export collection as CSV file which has empty fields (date, image, boolean fields), I might get such data in CSV:

title,image,date,some_boolean_field
Test title,,,

If I try to import this file back I see the message: "No items imported". It does not happen with JSON format.

Workaround: add "null" to an empty fields, then it works. Example:

title,image,date,some_boolean_field
Test title,null,null,null

Media Import not working

It seams that the media import doesn't work. When I upload a file using Strapi Media Library and use it in one collection entry, then export a json and import the json again the media file isn't uploaded to the new Strapi instance.

Plugin Title/Description mismatch

Hi!
This is a great plugin and gets the job done. Just a minor improvement will make it look more professional.
The plugin title in the left menu looks good but
Screen Shot 2021-08-07 at 10 31 05 am
When we go the plugin's landing page, it shows the following Title/Description
Import Content
Import CSV and JSON into your Content Types
Screen Shot 2021-08-07 at 10 28 44 am

Instead of

Import/Export Content

Import/Export CSV and JSON into your Content Types
Screen Shot 2021-08-07 at 10 29 09 am

Thanks!!

Missing strapi-utils and MaxListenersExceededWarning after running

After installation i got the error Error: Cannot find module 'strapi-utils' and I installed strapi-utils and now admin is working fine. But getting the MaxListenersExceededWarning: Possible EventEmitter memory leak detected warning everytime when running the server.

I have removed .cache,build, node modules and rebuild and tried again but same result. Any clues how to fix this?

Thanks

Cannot read property 'concat' of undefined

I am trying to import to a collection type and I face a error on
/node_modules/strapi-admin/services/permission/permissions-manager/index.js:52:42

at Object.queryFrom (/Users/wrongkitchen/Sites/strapi-demo/node_modules/strapi-admin/services/permission/permissions-manager/index.js:52:42)
at getAll (/Users/wrongkitchen/Sites/strapi-demo/plugins/import-export-content/services/exporter.js:28:36)
at Object.exportItems (/Users/wrongkitchen/Sites/strapi-demo/plugins/import-export-content/services/import-export-content.js:44:31)
at Object.exportItems (/Users/wrongkitchen/Sites/strapi-demo/plugins/import-export-content/controllers/import-export-content.js:93:34)
at dispatch (/Users/wrongkitchen/Sites/strapi-demo/node_modules/koa-router/node_modules/koa-compose/index.js:44:32)
at next (/Users/wrongkitchen/Sites/strapi-demo/node_modules/koa-router/node_modules/koa-compose/index.js:45:18)
at dispatch (/Users/wrongkitchen/Sites/strapi-demo/node_modules/koa-compose/index.js:42:32)
at /Users/wrongkitchen/Sites/strapi-demo/node_modules/strapi/lib/middlewares/router/utils/routerChecker.js:79:28
at dispatch (/Users/wrongkitchen/Sites/strapi-demo/node_modules/koa-compose/index.js:42:32)
at module.exports (/Users/wrongkitchen/Sites/strapi-demo/node_modules/strapi-plugin-users-permissions/config/policies/permissions.js:10:12)
at dispatch (/Users/wrongkitchen/Sites/strapi-demo/node_modules/koa-compose/index.js:42:32)
at /Users/wrongkitchen/Sites/strapi-demo/node_modules/strapi-utils/lib/policy.js:68:11
at dispatch (/Users/wrongkitchen/Sites/strapi-demo/node_modules/koa-compose/index.js:42:32)
at /Users/wrongkitchen/Sites/strapi-demo/node_modules/koa-compose/index.js:34:12
at dispatch (/Users/wrongkitchen/Sites/strapi-demo/node_modules/koa-router/node_modules/koa-compose/index.js:44:32)
at next (/Users/wrongkitchen/Sites/strapi-demo/node_modules/koa-router/node_modules/koa-compose/index.js:45:18)

anyone knows the solution?

Export is limited to 100 items

Hi,
The export is probably using the find service with the default limit 0f 100, and so only can handle collections up to 100 items. By changing/adding _limit: -1 you can get all the items.

How to use this plugin

I installed this plugin on my Strapi installation with npm i strapi-plugin-import-export-content and run after that npm run build. When I open up the Admin UI I can't find an option to use the plugin.

Did I do something wrong or what do I have to do to use the plugin?

Support plugin models

Currently the plugin only supports melds created directly on strapi.
Sometimes we have models created by third party plugins.

I can make o PR on this if I have some clues where to start.

Thanks a lot for the plugin btw

Uncaught ReferenceError: Prism is not defined

Great work on the plugin!

This is a replica of issue #1

main.88cc64a2.chunk.js:1 Uncaught ReferenceError: Prism is not defined
    at Object.<anonymous> (main.88cc64a2.chunk.js:1)
    at a (runtime~main.13b0b220.js:1)
    at Object.<anonymous> (main.88cc64a2.chunk.js:1)
    at a (runtime~main.13b0b220.js:1)
    at Object.<anonymous> (main.88cc64a2.chunk.js:1)
    at a (runtime~main.13b0b220.js:1)
    at Object.<anonymous> (main.88cc64a2.chunk.js:1)
    at a (runtime~main.13b0b220.js:1)
    at Object.<anonymous> (main.88cc64a2.chunk.js:1)
    at a (runtime~main.13b0b220.js:1)

Tested on Strapi versions: 6.4.0 and up
Node v14.15.4
Browser: Chrome, Edge, Safari and Firefox
Samme issue with v0.2.0, 0.3.2, 0.3.3 an 0.3.4

Tested by installing with both yarn and npm.

Also tried deleting node_modules, pacakge-lock and rebuilt with --clean.

SyntaxError: JSON.parse

I tried to export the content and got the following error - SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

No Items imported!

When I was trying to import one complex model with some json fields, It is saying me like No Items Imported! and also not showing any errors why its not importing or anything.

How to use?

Do I install this into a pre-existing Strapi installation?

Does it automatically register itself?

I'm not sure how to use this.

Collection data and media matching by id-field does not work for new Strapi instance

Sorry in advance if the topic already came up earlier!

Currently, we are using the plugin to export collection data from one Strapi instance and importing it on another Strapi instance. But in order to make this work, we had to patch the contentChecker.js to not only match media and relation references by id, but also by the 'name' property of entities. (See my fork if interested).

The question is if there is maybe a plan to mark a single field as the unique identifier for the importer to match the correct relation and media references? Or maybe there is already a different solution for the problem.

Thanks in advance!

P.S. If you want to skype and talk about this issue, just pm me.

Strapi panel not showing

Hi, this is a very interesting plugin that i want to try to use.
However, the simple action of installing and build (yarn build --clean) made the administration panel not to be show anymore...
I am removing for now, but i'd like to use it in future

Import and export destinations are showing empty

Tested on Strapi 3.6.1 version

Content types are not showing for import and export destinations.

Screenshot 2021-05-01 at 1 23 47 AM

while debugging file (admin/src/hooks/useContentTypes.js), noticed that after checking hasPermission for each content type getting empty array for contentTypesWithPermissions

Any solution? as an administrator, all permissions are updated properly.

Unable to start when using node < 14

Due to optional chaining operator used in this plugin, it's not compatible with node versions below 14.
In my case with v12:

[2021-05-28T07:17:32.950Z] info The server is restarting                                                                                                                                                                                                                        
                                                                                                                                                                                                                                                                                
[2021-05-28T07:17:36.527Z] debug ⛔️ Server wasn't able to start properly.                                                                                                                                                                                                       
[2021-05-28T07:17:36.528Z] error /node_modules/strapi-plugin-import-export-content/services/utils/contentChecker.js:7                                                                                        
  return value?.id || null;                                                                                                                                                                                                                                                     
               ^                                                                                                                                                                                                                                                                
                                                                                                                                                                                                                                                                                
SyntaxError: Unexpected token '.'                                                                                                                                                                                                                                               
    at wrapSafe (internal/modules/cjs/loader.js:915:16)                                                                                                                                                                                                                         
    at Module._compile (internal/modules/cjs/loader.js:963:27)                                                                                                                                                                                                                  
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)                                                                                                                                                                                                   
    at Module.load (internal/modules/cjs/loader.js:863:32)                                                                                                                                                                                                                      
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)                                                                                                                                                                                                            
    at Module.require (internal/modules/cjs/loader.js:887:19)                                                                                                                                                                                                                   
    at require (internal/modules/cjs/helpers.js:74:18)                                                                                                                                                                                                                          
    at Object.<anonymous> (/node_modules/strapi-plugin-import-export-content/services/utils/fieldUtils.js:8:29)                                                                                              
    at Module._compile (internal/modules/cjs/loader.js:999:30)                                                                                                                                                                                                                  
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)                                                                                                                                                                                                   
error Command failed with exit code 1.      
```

It would be good to update the docs accordingly or handle older versions or catch such version related errors.

I can make a PR for this, what are your thoughts?

Unable to import CSV: Unexpected mixed field

For example, a username field with 2 records and trying to import to strapi text field:

Username
user1
22222

The above case counted as a mixed field import and the import was failed.

Suspected the 2222 hasn't counted as a text field.

image

freezing at import

when either pasting or selecting file to import, browser freezes,
this issue occurs 6 out 10 times at least

Export only maximum 100 entries & utf-8 characters

Hello,

We have tried your plugin, we find bugs and things that would be cool.

  • We can import and export data but when we want to export more than 100 entries, we can't. (issue)
  • For the French, it would be nice to accept all utf-8 characters when doing an import (issue).

Then, it would be a really good idea to be able to update the existing contents (linked to the id on the new import).

Use Strapi Hash when importing password field

Hi Edison,

I've successfully import the data to users collection using your advice.
Now I'm facing another issue, which is to hash the password field before storing it to database.

I was thinking to override the controller, importItems, to alter the password field if any, then store the data. I tried override the controller but it's failed.

Any idea where should I do the hash before send?

Licence for import export plugin

We would like to use this plugin in one of our projects and also contribute to the development. As there is no licence on this project currently we are not aloowed to use it in commercial products.

It would be great if you could add a licence to clarify if you want to publish this plugin as OSS or not.

Export feature not working

Hi there,

amazing plugin, great work!

The export feature does not to seem to work as expected at the moment. I am getting the following error:

export.items.error

Bildschirmfoto 2021-05-30 um 10 35 19

Import timestamps

It would be really helpful to be able to include the created and updated timestamps in the import. Because when importing for example Articles from another CMS, one will probably want to include the time at which the article was created and updated. Using custom fields does not make so much sense since then the dates always need to be added manually.

I realize that this is currently not possible as strapi does not allow providing these on creation (AFAIK). Anyhow this issue should serve as a reminder that this should be implemented once strapi allows it.

A workaround with direct database manipulation might be possible but I'm quite sure that this is not best practice

[Solved] No Items Imported

I've came across this error message when importing a csv file: No Items Imported.

As this does not provide any feedback on what's going wrong in the import process, I had to investigate by myself. What did expose the errors that were happening was the console.error in this file:

node_modules/strapi-plugin-import-export-content/services/importer/importUtils.js

const importToCollectionType = async (uid, item) => {
  try {
    await strapi.entityService.create({ data: item }, { model: uid });
    // await strapi.query(uid).create(item);
    return true;
  } catch (error) {
    console.error({e: error.data.errors})
    return false;
  }
};

Hope this helps anyone having the same issue. Wish y`all the best.

v4 support

Does this plugin work with v4? If not, is there any plan to update? Thank you.

Error 405 when clicking on Analyze in prod

Hi!
I was trying to import data in prod but I get this error every time I click on "Analyze". I get the message "import.analyze.error":
image

But it works in localhost, so I guess I need some permissions.
Do you have any idea why this is happening?

Thanks.

Import plugin unexpectedly skipping valid data during import

Hi. I have been using this plugin for CSV imports in Strapi and after some usage, I noticed that it skipped some records even though it was perfectly valid.

For eg.
This is my collection:
image

And when I try to upload this RAW CSV row, it works
IN30306910082165,MANOHARAN V M,100,8/31/2019

But if I try to upload this RAW CSV row, it does not import
1201060000935294,GULSHAN GURLI ,1,8/18/2020

But works if I change the first field to alpha numeric to something like this:
ABCD,GULSHAN GURLI ,1,8/18/2020

This shows that there is some typecasting happening which makes it interpret it as integer somehow internally even though the schema declares the field as Text.

Any help appreciated in resolving this. Thanks.

PS: On a side note, may I know where can I increase the limit to number of rows which can be uploaded at once in a CSV cause uploading big files does not work.

CC: @EdisonPeM

Spread JSON objects in CSV outputs to make them more readable

If I have following data in my Strapi database:

Name School
Nirmal {name: xyz, area:abc}

This plugin would export it exactly like above: keeping the JSON field in a stringified form.. I wish to understand why it can't be like this instead:

Name School.name School.area
Nirmal xyz abc

At this line you can add a check whether the element variable is an array or an object. In case if it is an object, you can create extra columns and save the data appropriately.

I have a hunch you knew/ tried to do something like this, so I'm wondering is there anything I'm missing in my understanding of Strapi/ this plugin to figure this out. I'd love to hear some clarification on this.

Awesome job, thanks ✌️

API tokens not working for the plugin endpoints

Hello!

I am trying to use the built in 'API Tokens' functionality for the plugin's endpoints but I get a 401 error. Should this functionality work with the plugin's endpoints? Update: as I read in the developer docs, this functionality is mean for the built in collection API endpoints. Is there an easy way to generate tokens for the import/export plugin's endpoints?

Thanks!

Ability to update and delete records

I have created a fork with the ability to update and delete records.

https://github.com/ztech-company/strapi-plugin-import-export-content/tree/update-delete

To isolate this behaviour (in case users want to keep the existing behavior), a checkbox "Allow Update and Delete" is added to the upload screen.

If the checkbox is selected then:

  1. For each record, a check is made whether it exists (based on id) and if so it is updated rather than created
  2. If the record has an attribute operation_delete then the record is delete (based on id). Records with this attribute still need to be valid records, because they are validated as part of the field mapping (I use this for bulk deleting, although the way it is done is not ideal).

Some of this design could possibly be improved, but am keen to get feedback on it and to get your thoughts on whether you see benefit in merging this into your repo?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.