autotelik / datashift Goto Github PK

delegate_belongs_to fields do not appear in the template output

For example see Spree, the model Spree::Product has following definition

    delegate_belongs_to :master, :sku, :price, :currency, :display_amount, :display_price, :weight, :height, :width, :depth,
:is_master, :has_default_price?, :cost_currency, :price_in, :amount_in, :cost_price, :images

But none of these appear when generating an Excel template,

thor datashift:generate:excel -m Spree::Product -r tmp/product_template.xls

Suspect these will appear at least as instance methods so we need to make sure the catalogue class method can be configured to add those if requested - for example binder does not currently pass any options

model_method_mgr = ModelMethods::Manager.catalog_class(klass)

hi @autotelik

I've been recently giving datashift a shot with my spree and it is really nice it works, really nice job thx!

However I am struggling with my product import which has the following option_types:
-size:small,medium
-color:black,red

it creates one variant per option value as described in the wiki:

• small
• medium
• black
• red

What if I want variants to look like this :

• small & black
• small & red
• medium & black
• medium & red

Thanks

Stephane.

My install is failing, using jruby 1.6.5., therubyrhino, rails 3.2.3 and Spree 1.1.1.

Bailing out on the installation of Spree with a

 Invalid gemspec in [/home/jet/.rvm/gems/jruby-1.6.5@datashift/specifications/spree_auth-1.1.1.gemspec]: Illformed requirement ["#<YAML::Yecht::DefaultKey:0x1f62951> 1.1.1"]

Tried getting by this by running,

 jruby --1.9 -S bundle install                                  (which if my version is only compatible with rails v1.8.7.p320, makes sense?)

...but failed at:

NoMethodError: undefined method `read_nonblock' for #OpenSSL::SSL::SSLSocket:0x1942a3c
An error occured while installing rake (0.9.2.2).

So, what pieces are you using to get this working?

Cheers,

Jet

What is this? No docs? What's the right syntax to use please

As per the demo spreadsheets which were created manually, when a template is generated, see if we can dynamically write similar comments into the heading of each column with details of the association type and the supported syntax for specifying lookups etc

In my Gemfile:
gem 'datashift', github: 'autotelik/datashift'
gem 'datashift_spree', github: 'autotelik/datashift_spree'

When running
bundle exec thor datashift_spree:load:attach_images --input=/vagrant/imageimport/import.xls

I receive the error:
/var/lib/gems/1.9.1/gems/datashift_spree-0.5.0/lib/loaders/spree/image_loader.rb:19:in <module:SpreeHelper>': uninitialized constant DataShift::SpreeHelper::SpreeBaseLoader (NameError) from /var/lib/gems/1.9.1/gems/datashift_spree-0.5.0/lib/loaders/spree/image_loader.rb:10:inmodule:DataShift'
from /var/lib/gems/1.9.1/gems/datashift_spree-0.5.0/lib/loaders/spree/image_loader.rb:8:in <top (required)>' from /var/lib/gems/1.9.1/gems/polyglot-0.3.5/lib/polyglot.rb:65:inrequire'
from /var/lib/gems/1.9.1/gems/polyglot-0.3.5/lib/polyglot.rb:65:in require' from /var/lib/gems/1.9.1/gems/datashift_spree-0.5.0/lib/thor/spree/products_images.thor:77:inattach_images'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/command.rb:27:in run' from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/invocation.rb:126:ininvoke_command'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor.rb:359:in dispatch' from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/base.rb:440:instart'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/runner.rb:36:in method_missing' from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/command.rb:29:inrun'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/command.rb:126:in run' from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/invocation.rb:126:ininvoke_command'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor.rb:359:in dispatch' from /var/lib/gems/1.9.1/gems/thor-0.19.1/lib/thor/base.rb:440:instart'
from /var/lib/gems/1.9.1/gems/thor-0.19.1/bin/thor:6:in <top (required)>' from /usr/local/bin/thor:23:inload'
from /usr/local/bin/thor:23:in `

Getting this error when running datashift:generate:csv

> bundle exec thor datashift:generate:csv -m Spree::Product -r spree_export_template.csv -a
warning: parser/current is loading parser/ruby21, which recognizes
warning: 2.1.5-compliant syntax, but you are running 2.1.0.
uninitialized constant Thor::Sandbox::Datashift::Generate::ModelMapper
ERROR: No such Model [Spree::Product] found - check valid model supplied

While I'm trying to parse csv file I received #<NoMethodError: undefined method `type_cast_from_database' for #<ActiveRecord::ConnectionAdapters::Column:0x007fd36dc5f358>>
Probably it's problem with rails 5 where this method for column is removed. Do you know any simple solution to fix it? Do you plan to release new, rails 5 compatible, datashift version?

Currently, the gem is dependent upon Rails 4.2

Any plan to support Rails 5? Thx :)

Version: current latest on RubyGems, 0.40.0

Script:

require 'thor'
require 'datashift'

Error Message:

Problem initializing gem #<NameError: uninitialized constant DataShift::LoaderBase::Forwardable>
Problem loading datashift file /Users/songyy/.rvm/gems/ruby-2.3.1/gems/datashift-0.40.0/lib/exporters/csv_exporter.rb - #<NameError: uninitialized constant DataShift::ExporterBase
Did you mean?  DataShift::Exporters>
Problem loading datashift file /Users/songyy/.rvm/gems/ruby-2.3.1/gems/datashift-0.40.0/lib/exporters/excel_exporter.rb - #<NameError: uninitialized constant DataShift::ExporterBase
Did you mean?  DataShift::Exporters>

Is it possible to create a has_many record on import? Do i have to create a custom loader (like the Spree Product loader)?

I've been having some problems similar to other users with loading images. I finally found and followed your recipe in Issue #6 and got it to load products and images from 'SpreeProductsWithImages.csv'. It doesn't seem to work right when I try to do the images separately.

My file arrangement is:

spreestore/ (rails root)
spreestore/db/datashift/SpreeProductsWithImages.csv
spreestore/fixtures/images/{the image files}

SpreeProductsWithImages.csv has paths that are: 'fixtures/images/filename.jpeg'

So I sorted out that the file path is relative to where thor is running from, rather than from the directory the csv is in.

When I try to do it by loading products first using 'SpreeProducts.csv' (in the same directory as 'SpreeProductsWithImages.csv') I can load the products just fine, but then when I try to load images with:

bundle exec thor datashift:spree:images -i fixtures/images -s

I get this set of messages and no images in the store:
Using SKU for lookup
CONFIG: {"split_file_name_on"=>"", "input"=>"fixtures/images", "sku"=>"sku", "load"=>false}
OPTIONS {"split_file_name_on"=>"", "input"=>"fixtures/images", "sku"=>"sku"}
Will scan image names splitting on delimiter : _
WARNING : 1 of 1 images could not be attached to a Product
Copying all images with MISSING Records to ./MissingRecords >>
Copy fixtures/images/DEMO_002_Powerstation.jpg to MissingRecords folder

I'm running Rails 3.2.6, Spree 1.1.2, and datashift 0.80 from git. I've tweaking the datashift docs a bit as I figure out how it works-- I'd like to extend it to use it for loading products and digital files for spree_digital.

way I've implemented options causes issues with thor help/usage

$ th datashift:import:csv
No value provided for required options '--model', '--input'

My csv like
name,sku,price,images
Aka,EBK.3435,10.00,fixtures/images/DEMO_001_ror_bag.jpeg|fixtures/images/DEMO_002_ror_bag.jpeg

when i run
bundle exec thor datashift_spree:load:products -i, --input=Sample_image.csv

It gives error

Loading from CSV file: Sample.csv
Processing 1 rows
wrong number of arguments (2 for 1)

Processing Summary Report

Processed total of 1 entry
0 objects were succesfully processed.
WARNING : Check logs : 1 rows contained errors and 1 records NOT created.

Can you please tell me what is the issue here??
its urgent.

Thanks a lot

Hello.
Recently you had a plans in todo to make a gui for importing/exporting.
Could we know some news about it?
Thanks.

Is this function working correctly?

I cannot seem to get it working. The "f" option gives me a lookup error.

spree 3

Following https://github.com/autotelik/datashift/wiki/Walkthroughs but keep getting errors in the datashift.log about Cannot Save Spree::Product : ["Price can't be blank"] whereas the prices column in the spreadsheet is set with a numeric value.

datashift_log.txt

product_newtest.xlsx

ndj-mbp:imazen-store nathanael$ b thor datashift:export:csv -m=Spree::Product -r=./
Please supply array of records to export
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/datashift-0.13.0/lib/exporters/csv_exporter.rb:38:in `export'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/datashift-0.13.0/lib/thor/export.thor:100:in `csv'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/runner.rb:36:in `method_missing'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/command.rb:29:in `run'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/command.rb:126:in `run'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
/Users/nathanael/.rbenv/versions/2.0.0-p353/lib/ruby/gems/2.0.0/gems/thor-0.19.1/bin/thor:6:in `<top (required)>'
/Users/nathanael/.rbenv/versions/2.0.0-p353/bin/thor:23:in `load'
/Users/nathanael/.rbenv/versions/2.0.0-p353/bin/thor:23:in `<main>'
Warning: Error during export, data may be incomplete

Using these versions of gems.

Looks like specifying a default available_on for Product not being picked up in ProductLoader

.. pretty sure general setting defaulots from config specs pass though so maybe something specific to db dates ?

Currently we have wiki: Options and Configuration, but it's quite incomplete.

I want to learn how to:

• configure the loader to map data into specific field / nested resource, from the excel column
• configure the mapper to output the data

Hello again,

I am now working on variants and I would like to set specific images (and a specific price and SKU by the way) on each variant. I am not sure it can be done right now... Or could it be ?

Thanks,
Manu

I'm working on a Rails 4 app and needed the latest code from datashift. As it's not released yet, my Gemfile directly links to a commit from the repo.

Everything worked fine until I tried to deploy on Heroku:

The slug size exceeds 300MB, and the main reason is that the datashift git repo needs more than 200MB.

I've tracked down the problem to this folder.

231M vendor/bundle/ruby/2.0.0/bundler/gems/datashift-bfada2fd1319/.git/objects/pack

I don't have currently the time to find which commit is responsible.

My fault. Sorry for posting.

Better loading of lib files e.g

Gem.find_files("gem_name/**/*.rb").each { |path| require path }

Hello,

Thanks for datashift. The first contact with it is neat and efficient !

I guees I have come accros a bug when using several subcategories of same name. For instance
lucky_product : A>mysubcat
bad_luck : B>mysubcat

The bad_luck product is created with in B and somehow pushed into A>mysubcat.
The category B>mysubcat is not created at all...

Hope this is clear enough,
Manu

Currently, the file structure is like this:

/path/to/datashift/
▾ lib/
  ▸ applications/
  ▸ datashift/
  ▸ exporters/
  ▸ generators/
  ▸ java/
  ▸ loaders/
  ▸ tasks/
    datashift.rb
▸ spec/
  CHANGELOG.md
  datashift.gemspec
  datashift.thor
  Gemfile
  LICENSE
  md_style.rb
  Rakefile
  README.md

As we can see, folders like applications are under the lib root path directly. Following this post about best practices, we can see that the current approach would pollute the global load path:

suppose there's another library called loaders, then the current list of files in loaders/files would pollute the original java library.

I'd suggest putting all files under lib/datashift/folder.

I exported my list of products ( not spree but custom made application) that have all an ID, when I import the excel file again with my changes ( description ), he tells me that there is a duplicate entry with ID x.

Is it possible with datashift to map an existing record with the ID, so when the ID exists that he updates the row with that specific ID with the values? We want to use the excel file to keep the products up to date on the website because the client is working internaly with this excel sheet.

Could not find spec/fixtures/spree directory for browsing examples.

Hey thanks for providing this great stuff, I have query when i upload some products via xls. I got error

WARNING : Check logs : 1 rows contained errors
which is expected as it was failing spree validation. I am sending a mail as soon the background process completes.
But the issue is that warning is not raising any exception such that i can rescue it and send failure mail.

In simple term, i want to send success mail when product uploaded successfully and failure mail when the upload fails

I am slowly looking at implementing this.

Need to create a better loader object management layer and then refactor the perform.._load methods (e.g perform_excel_load) as these currently do too much.

Look for a good pattern for Object Management, move out of loaders/Populator, into separate area.

Enable loaders to reach into this to grab current object, split out from the generic loaders, and provide hooks in those generic Loaders to enable custom loaders to more easily manage the object lifecycle themselves. Currently far too rigid, i.e just creates a new object for each row processed.

bundle execute thor datashift_spree:load:products -i ~/datashift_autotelik/spec/fixtures/SpreeProductsMandatoryOnly.xls 

DataShift::Product starting upload from file: /root/datashift_autotelik/spec/fixtures/SpreeProductsMandatoryOnly.xls

Loading from Excel file: /root/datashift_autotelik/spec/fixtures/SpreeProductsMandatoryOnly.xls
["sku", "name", "price"]
["sku", "name", "description", "shipping_category", "price", "option_types"]
ERROR: Excel loading failed : #<NoMethodError: undefined method `pry' for #<Binding:0x3c99ea87>>

Processing Summary Report
>>>>>>>>>>>>>>>>>>>>>>>>>
Processed total of 0 inbound entry
0   database objects were successfully processed.
0   inbound rows were successfully processed.
There were NO failures.
NoMethodError: undefined method `pry' for #<Binding:0x3c99ea87>
      perform_excel_load at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/bundler/gems/datashift-91b44de22c91/lib/loaders/excel_loader.rb:92
         each_with_index at file:/usr/local/rvm/rubies/jruby-1.7.19/lib/jruby.jar!/jruby/java/java_ext/java.lang.rb:19
      perform_excel_load at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/bundler/gems/datashift-91b44de22c91/lib/loaders/excel_loader.rb:80
             transaction at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/activerecord-4.1.11/lib/active_record/connection_adapters/abstract/database_statements.rb:201
  within_new_transaction at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/activerecord-4.1.11/lib/active_record/connection_adapters/abstract/database_statements.rb:209
             transaction at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/activerecord-4.1.11/lib/active_record/connection_adapters/abstract/database_statements.rb:201
             transaction at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/activerecord-4.1.11/lib/active_record/transactions.rb:208
      perform_excel_load at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/bundler/gems/datashift-91b44de22c91/lib/loaders/excel_loader.rb:78
            perform_load at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/bundler/gems/datashift-91b44de22c91/lib/loaders/loader_base.rb:118
            perform_load at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/bundler/gems/datashift_spree-37f79a7bf671/lib/loaders/spree/spree_base_loader.rb:55
            perform_load at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/bundler/gems/datashift_spree-37f79a7bf671/lib/loaders/spree/product_loader.rb:58
                products at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/bundler/gems/datashift_spree-37f79a7bf671/lib/thor/spree/products_images.thor:69
                     run at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/thor-0.19.1/lib/thor/command.rb:27
          invoke_command at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/thor-0.19.1/lib/thor/invocation.rb:126
                dispatch at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/thor-0.19.1/lib/thor.rb:359
                   start at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/thor-0.19.1/lib/thor/base.rb:440
          method_missing at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/thor-0.19.1/lib/thor/runner.rb:36
                     run at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/thor-0.19.1/lib/thor/command.rb:29
                     run at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/thor-0.19.1/lib/thor/command.rb:126
          invoke_command at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/thor-0.19.1/lib/thor/invocation.rb:126
                dispatch at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/thor-0.19.1/lib/thor.rb:359
                   start at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/thor-0.19.1/lib/thor/base.rb:440
                  (root) at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/gems/thor-0.19.1/bin/thor:6
                    load at org/jruby/RubyKernel.java:1087
                  (root) at /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/bin/thor:23

I used a pry statement in lib/loaders/excel_loader.rb

# be thor datashift_spree:load:products -i ~/datashift_autotelik/spec/fixtures/SpreeProductsMandatoryOnly.xls 
[WARNING] You are not setting Devise.secret_key within your application!
You must set this in config/initializers/devise.rb. Here's an example:

Devise.secret_key = "0d17b9c9239f700e20e4b526459e55257301cd84c46f0654e70821ddbff6b1da631f1d011f59bc2f270d2c5ae6d0169c6903"
Instance method "open" is already defined in Object, use generic helper instead or set StateMachine::Machine.ignore_method_conflicts = true.
DataShift::Product starting upload from file: /root/datashift_autotelik/spec/fixtures/SpreeProductsMandatoryOnly.xls



Loading from Excel file: /root/datashift_autotelik/spec/fixtures/SpreeProductsMandatoryOnly.xls
["sku", "name", "price"]
["sku", "name", "description", "shipping_category", "price", "option_types"]

From: /usr/local/rvm/gems/jruby-1.7.19@mystore20151008/bundler/gems/datashift-91b44de22c91/lib/loaders/excel_loader.rb @ line 92 DataShift::ExcelLoading#perform_excel_load:

    87:             # Excel num_rows seems to return all 'visible' rows, which appears to be greater than the actual data rows
    88:             # (TODO - write spec to process .xls with a huge number of rows)
    89:             #
    90:             # This is rubbish but currently manually detect when actual data ends, this isn't very smart but
    91:             # got no better idea than ending once we hit the first completely empty row
 => 92:             binding.pry
    93:             break if(@current_row.nil? || @current_row.compact.empty?)
    94: 
    95:             logger.info "Processing Row #{current_row_idx} : #{@current_row}"
    96: 
    97:             @contains_data = false

[1] pry(#<DataShift::SpreeEcom::ProductLoader>)> @current_row
=> #<Java::OrgApachePoiHssfUsermodel::HSSFRow:0x762c1d96>
[2] pry(#<DataShift::SpreeEcom::ProductLoader>)> @current_row.nil?
=> false
[3] pry(#<DataShift::SpreeEcom::ProductLoader>)> @current_row.compact
NoMethodError: undefined method `compact' for #<Java::OrgApachePoiHssfUsermodel::HSSFRow:0x762c1d96>
from (pry):3:in `evaluate_ruby'

environment: jruby 1.7.19, java version "1.7.0_95", rails (4.1.11)

hi Tom

I just have installed datashift and datashift_spree from rubygems with Spree 2-1-stable.
and created mysite.thor as README.

require 'datashift'
require 'datashift_spree'

DataShift::load_commands

When command "bundle exec thor list datashift" , got uninitialized constant.
Is there a way to fix this?

bundle exec thor list datashift
WARNING: unable to load thorfile "/home/digitalm/public_html/railsapp/spreestore/mysite.thor": uninitialized constant DataShift::Logging
/home/digitalm/public_html/railsapp/spreestore/vendor/bundle/ruby/1.9.1/gems/datashift-0.14.0/lib/datashift/method_mapper.rb:22:in `<class:MethodMapper>'
No Thor commands available

In addition , github repo cannot be installed currently.

bundle install
Updating git://github.com/autotelik/datashift.git
There was a NameError while loading datashift.gemspec:
uninitialized constant DataShift::Logging from
  /home/digitalm/public_html/railsapp/spreestore/vendor/bundle/ruby/1.9.1/bundler/gems/datashift-14ca51f5f51e/datashift.gemspec:7:in `<main>'

Just for information : Rails 4.0.2 / ruby 1.9.3p484

Best regards,
Mika

Hi there,
I'm running into an error every time I try to generate a template with bundle exec thor datashift:generate:excel --model Spree:Product --result /home/devs/excel.xls --assoc.

devs@barakah:~/artilujos$ bundle exec thor datashift:generate:excel --model Spree:Product --result /home/devs/excel.xls --assoc
/home/devs/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/durable_decorator-0.2.2/lib/durable_decorator/util.rb:46:in `logger': uninitialized constant Logging (NameError)
    from /home/devs/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/durable_decorator-0.2.2/lib/durable_decorator/validator.rb:34:in `handle_soft_fault'
    from /home/devs/.rbenv/versions/2.1.3/lib/ruby/gems/2.1.0/gems/durable_decorator-0.2.2/lib/durable_decorator/validator.rb:22:in `validate_decoration_meta'

I've seen a few similar issues reported and tried many things: adding 'include DataShift::Logging' and 'require 'logging'' to the util.rb file, fiddling with different versions and making the changes suggested in this thread autotelik/datashift_spree#1

I've run out of ideas and so I'd appreciate if you could point me in the right direction.

Gems are listed like this, at the bottom of my gemfile:

gem 'datashift', :git => 'http://github.com/autotelik/datashift'
gem 'datashift_spree', :git => 'http://github.com/autotelik/datashift_spree'

Thanks!

Not sure this is an issue or me missing a step/clue, but I don't seem to have any thor task that can export or import as CSV.

$ be thor list datashift
WARNING: unable to load thorfile "/Users/sbeam/.rvm/gems/jruby-1.6.7.2@global/gems/datashift-0.8.0/lib/thor/spree/bootstrap_cleanup.thor": admin_interface_log preference not defined
/Users/sbeam/.rvm/gems/jruby-1.6.7.2@global/gems/spree_core-1.1.1/app/models/spree/preferences/preferable.rb:60:in `has_preference!'
datashift
---------
thor datashift:export:excel -m, --model=MODEL -r, --result=RESULT # export...
thor datashift:generate:excel -m, --model=MODEL -r, --result=RESULT # genera...
thor datashift:import:excel -i, --input=INPUT -m, --model=MODEL # import...
thor datashift:reports:missing_images # Spree ...
thor datashift:spree:attach_images -i, --input=INPUT # Popula...
thor datashift:spree:images -i, --input=INPUT # Popula...
thor datashift:spree:products -i, --input=INPUT # Popula...

we actually don't have any requirement to use Excel files, so was hoping to use the CSV functionality (though it might be nice, but we were planning on deploying under MRI 1.9.3 so jruby compatibility might be an issue)

If the Price column is a currency type containing symbol get a very nasty bug where the Product saves and save returns true but there is no master Variant. This is particularly nasty as will crash the main site as Products found, but access to master causes no method errors on nil.

Presume the price field is set to something on Master, not nil, so Product saves but then the underlying Variant fails to save when checks done on validity of Price field.

Odd since if you leave the Price column completely blank then the Product itself will not save, validation error something like 'Price must be set'

Hello Tom,

I gave a new try on my file with multiple sub-categories of same name.

I have the following error :

$ bundle exec thor datashift:spree:products -i Omega_checked.csv 
/home/emb/.../bundler/gems/datashift-6b6bd8c89874/lib/loaders/paperclip/image_loader.rb:14:in `<module:ImageLoading>': uninitialized constant DataShift::Paperclip (NameError)
    from /home/emb/.../bundler/gems/datashift-6b6bd8c89874/lib/loaders/paperclip/image_loader.rb:12:in `<module:DataShift>'
    from /home/emb/.../bundler/gems/datashift-6b6bd8c89874/lib/loaders/paperclip/image_loader.rb:10:in `<top (required)>'
    from /home/emb/.../gems/polyglot-0.3.3/lib/polyglot.rb:63:in `require'
    from /home/emb/.../gems/polyglot-0.3.3/lib/polyglot.rb:63:in `require'
    from /home/emb/.../bundler/gems/datashift-6b6bd8c89874/lib/loaders/spree/product_loader.rb:11:in `<top (required)>'
    from /home/emb/.../gems/polyglot-0.3.3/lib/polyglot.rb:63:in `require'
    from /home/emb/.../gems/polyglot-0.3.3/lib/polyglot.rb:63:in `require'
    from /home/emb/.../bundler/gems/datashift-6b6bd8c89874/lib/thor/spree/products_images.thor:36:in `products'
    from /home/emb/.../gems/thor-0.16.0/lib/thor/task.rb:27:in `run'
    from /home/emb/.../gems/thor-0.16.0/lib/thor/invocation.rb:120:in `invoke_task'
    from /home/emb/.../gems/thor-0.16.0/lib/thor.rb:275:in `dispatch'
    from /home/emb/.../gems/thor-0.16.0/lib/thor/base.rb:425:in `start'
    from /home/emb/.../gems/thor-0.16.0/lib/thor/runner.rb:36:in `method_missing'
    from /home/emb/.../gems/thor-0.16.0/lib/thor/task.rb:29:in `run'
    from /home/emb/.../gems/thor-0.16.0/lib/thor/task.rb:126:in `run'
    from /home/emb/.../gems/thor-0.16.0/lib/thor/invocation.rb:120:in `invoke_task'
    from /home/emb/.../gems/thor-0.16.0/lib/thor.rb:275:in `dispatch'
    from /home/emb/.../gems/thor-0.16.0/lib/thor/base.rb:425:in `start'
    from /home/emb/.../gems/thor-0.16.0/bin/thor:6:in `<top (required)>'
    from /home/emb/.../bin/thor:19:in `load'
    from /home/emb/.../bin/thor:19:in `<main>'
    from /home/emb/.../bin/ruby_noexec_wrapper:14:in `eval'
    from /home/emb/.../bin/ruby_noexec_wrapper:14:in `<main>'

The file was correctly imported with stable version.
Please, tell me which information you need, if you need any more.

Cheers,
Manu

Sorry for noobish question.
But. how to attach images to product ?
I have file myapp/db/datashift/SpreeProductsWithImages.csv which I've copied from https://github.com/autotelik/datashift/blob/master/spec/fixtures/spree/SpreeProductsWithImages.csv

And folder with images, myapp/db/datashift/fixtures/images tooked from there
https://github.com/autotelik/datashift/tree/master/spec/fixtures/images

I run:
$> bundle exec thor datashift:spree:products -i db/datashift/SpreeProductsWithImages.csv
There I get products which migrated right but with no images.

DataShift::Product starting upload from file: db/datashift/SpreeProductsWithImages.csv
WARNING: Following column headings could not be mapped : ["Images"]



Loading from CSV file: db/datashift/SpreeProductsWithImages.csv
Processing 3 rows
Created missing OptionType #<Spree::OptionType id: 5, name: "mime_type", presentation: "Mime type", created_at: "2012-07-08 22:32:08", updated_at: "2012-07-08 22:32:08", position: 0>
Created missing OptionType #<Spree::OptionType id: 6, name: "print_type", presentation: "Print type", created_at: "2012-07-08 22:32:09", updated_at: "2012-07-08 22:32:09", position: 0>

Then i run
$> bundle exec thor datashift:spree:images -i db/datashift/fixtures/images

So, how to upload images using specs tamplates ?
In which folder images must be located and what's way to declare to each image?
Thanks for awesome gem and your work!

Hi Tom,

I'm a newbie to Rails and Datashift and am trying to use some of the excel commands but continue to get
'Apologies but DataShift Excel facilities currently need JRuby. Please switch to, or install JRuby'

However, I have JRuby 1.6.7.2 installed on my machine. Is there something that I'm missing to get Datashift to recognize this?

Also, alternatively is there a way to export tables into a csv format? I see there's a way using import but none for export.

Thanks so much for your help and this tool is great!

-Jason

NoMethodError: undefined methodtext_delim=' for DataShift::Delimiters:Module lib/exporters/csv_exporter.rb:42:in export'

exporter = DataShift::CsvExporter.new(file_path)
exporter.export(Foo.all, text_delim: "\t")

I'm hoping to use this for a project where I have some rather complicated import/export logic, and am wondering if there are any examples of this nature (where you actually extend the generator/importer). If not, would it be possible to create an example, or point me in the right direction and I can maybe write one? :)

Consider the use case of a Post that belongs to many Sub Categories, with each Sub Category belonging to a Parent Category. In my case, the logic is such that a Post must belong to one (and only one) of each Parent Category's Sub Categories. Say that we have a "Department" Parent Category, and a "Project" Parent Category. Each Post must belong to one of "Department"'s Sub Categories, and one of "Project"'s Sub Categories. However, the data is manipulated by humans as part of the import/export process, so I'm hoping to have an excel sheet that looks something like the following:

My thor file looks like this:

require 'datashift'
require 'datashift_spree'
DataShift::load_commands
DataShift::SpreeHelper::load_commands

My Gemfile looks like this

gem 'spree', github: 'spree/spree', branch: '3-0-stable'
gem 'spree_gateway', github: 'spree/spree_gateway', branch: '3-0-stable'
gem 'spree_auth_devise', github: 'spree/spree_auth_devise', branch: '3-0-stable'
gem 'datashift'
gem 'datashift_spree'

When I do bundle exec thor list datashift
I get

WARNING: unable to load thorfile "/home/sanjay/git/growchef/growchef.thor": uninitialized constant DataShift::Logging
/home/sanjay/.rvm/gems/ruby-2.2.1/gems/datashift-0.15.0/lib/datashift/method_mapper.rb:22:in `class:MethodMapper'
No Thor commands available

Please help.

Thanks
Sanjay