jnunemaker / happymapper

[unmaintained] XML to object mapping library

NOTE: This project is no longer maintained. If you are interested in helping, let me know. There does seem to be a maintained fork (github.com/dam5s/happymapper).

happymapper¶ ↑

DESCRIPTION:¶ ↑

XML to object mapping library. I have included examples to help get you going. The specs should also point you in the right direction.

FEATURES:¶ ↑

• Easy to define xml attributes and elements for an object

• Fast because it uses libxml-ruby under the hood

• Automatic conversion of xml to defined objects

EXAMPLES:¶ ↑

Here is a simple example that maps Twitter statuses and users.

class User
  include HappyMapper

  element :id, Integer
  element :name, String
  element :screen_name, String
  element :location, String
  element :description, String
  element :profile_image_url, String
  element :url, String
  element :protected, Boolean
  element :followers_count, Integer
end

class Status
  include HappyMapper

  element :id, Integer
  element :text, String
  element :created_at, Time
  element :source, String
  element :truncated, Boolean
  element :in_reply_to_status_id, Integer
  element :in_reply_to_user_id, Integer
  element :favorited, Boolean
  has_one :user, User
end

See examples directory in the gem for more examples.

github.com/jnunemaker/happymapper/tree/master/examples/

INSTALL:¶ ↑

• gem install happymapper

DOCS:¶ ↑

rdoc.info/projects/jnunemaker/happymapper

happymapper's Issues

Cannot have multiple child elements with a default namespace

HappyMapper 0.2.2 (commit 545ebcd8a8b6dec66a526244321fc1dfb171de05) libxml 0.9.8

If an object including HappyMapper has multiple child elements (via has_one or has_many) and a default namespace, an exception is raised when processing the second child element.

I’ve recreated the issue in the specs, see attached diff.

It seems trying to assign a default namespace prefix to a node fails in libxml, but HappyMapper should probably not try to do this anyway.

I’ve fixed this ticket with http://github.com/lightningdb/happymapper/commit/ac6510bcec0314435d29ae092e857e965c0e1c28

The issue is that libxml falls over if the default prefix is assigned more than once. Seems to be a bug with libxml, but likewise it is probably something that happymapper should avoid anyway.

So before assigning the default prefix, we check whether a default prefix has already been assigned.

I’ve updated the tests to reflect this situation, by adding an extra ‘has_one’, which invokes the nested parsing that caused the error in the first place. I moved the Address class mapper definition up and added an address node to the product (I realise this doesn’t make ‘domain’ sense, but it proves the issue and the fix).

Hopefully John approves and can pull into the main happymapper?

Is it possible to get back to the parent?

So if I have a book xml like

<book title="Footown Usa">
  <chapter title="The showdown" />
  <chapter title="The godown" />
  <chapter title="The outcome" />
  <chapter title="The finale" />
  <chapter title="The wrapup" />
</book>

class Book
  include HappyMapper
  attribute :title, String
  has_many :chapters, Chapter
end

class Chapter
  include HappyMapper
  attribute :title, String
end

Would it be possible to get back to the book object from a chapter object? Maybe something like chapter.parent? Except I already know that that won't work. Is there some other magical method I'm missing here?

Support for assigning a node's content to an attribute.

I was working on a wrapper for backpack's API, and they return XML like:

<note id='1132' title='My world!' created_at='2007-07-03 19:15:55'>
  It's a pretty place
</note>

A limitation of happymapper's API is that it doesn't let you easily get at "It's a pretty place". So, I slung together some code to let you:

module Backpack
  class Note
    include HappyMapper
    content :body
  end
end

I went with the term 'content' to match LibXML::XML::Node's naming.

Check it out: http://github.com/technicalpickles/happymapper/tree/content

Elements with 'false' value not generated as xml

In lib/happymapper.rb, line 285:

If the value of my element is false, the condition at this line is false. I changed it locally to !item.nil? and then it works.

libxml-ruby is broken, use nokogiri

i've run into multiple libxml-ruby bugs which are documented at http://github.com/cfis/libxml-ruby/issues

there's also a huge list of unresolved issues at http://rubyforge.org/tracker/?atid=1971&group_id=494&func=browse

this fork switches to nokogiri: http://github.com/dam5s/happymapper

Allowing reusable mapping classes

In the existing codebase, the xpath for non-primitive elements is from the downcased classname of the non-primitive class. This works well, but there are several cases where some more flexibility would be good.

For instance, consider this XML:

```

1
22

4:40:15 PM 5:18:53 PM 6:06:17 PM 6:41:49 PM

```

There is an obvious problem with this xml: the q1, q2, q3, q4 elements should be ‘quarter’ elements, with an attribute containing the quarter number. However, we usually don’t have control of the format of external xml, so we just need to work with it. Anyway, we might map these with the following classes:

```
class Quarter
include HappyMapper

element :start, String

end

class Details
include HappyMapper

element :round, Integer element :quarter, Integer

end

class Game
include HappyMapper

has_one :details, QuarterTest::Details has_one :q1, QuarterTest::Quarter has_one :q2, QuarterTest::Quarter has_one :q3, QuarterTest::Quarter has_one :q4, QuarterTest::Quarter

end
```

The problem is that the elements q1, q2, q3 and q4 will not be able to be found using the current HappyMapper implementation, since the xpath used will be ‘quarter’, which will find game/details/quarter in each case. Adding :tag => ‘q1’ etc will not help, since the tag name is derived from the class name.

We can fix this and allow mapping classes to be resuable. Instead of using the classname as the default tag name, we can use the following finders, in order:
1. specified tag (e.g. :tag => ‘q1’)
2. name of element (e.g. from has_one :q1)
3. tag_name (derived from class name by default)

Using this method, the Game class changes to:

```
class Game
include HappyMapper

has_one :details, QuarterTest::Details has_one :q1, QuarterTest::Quarter, :tag => ‘q1’ has_one :q2, QuarterTest::Quarter, :tag => ‘q2’ has_one :q3, QuarterTest::Quarter, :tag => ‘q3’ has_one :q4, QuarterTest::Quarter, :tag => ‘q4’

end
```

This is a bit of a change, but the additional flexibility and reusability has worked really well for me in a variety of situations. In the existing specs, I only had to make one change for everything to pass:

```
class Radar
include HappyMapper

1. OLD: has_many :places, Place
2. NEW:
   has_many :places, Place, :tag => :place
   end
   ```

I’ve made the changes needed in my fork (http://github.com/lightningdb/happymapper/tree/master, commits ce59623df1ce0b16bbbee89e6fac2b2473756a43 and 3251509d1e68fb652227a42b1e5593ee8bc19d5b), and updated the gemspec version to 0.3.0 to reflect that the change might need minor changes to existing mappings. I think the flexibility is worth it though.

Any thoughts or questions?

happymapper 0.4.0 crashes ruby.exe on windows XP

Ruby version 1.9.2
Rails 3.1.3
libxml-ruby (2.3.0 x86-mingw32)

Code causing problem (added the following code in application_controller.rb):

require 'rubygems'
require 'xml'
require 'happymapper'

xml = "[email protected]111abc"
node = XML::Parser.string(xml).parse.root

Use libxml-ruby version 2.1 for ruby 1.9.3 compatibility

Can you upgrade your require version of libxml for a compatibility with ruby 1.9.3 (preview at this time)

Because installed libyml-ruby version 1.X doesn't work. And it's fix with version 2.1

Thx

Using namespaces is not documented

The documentation for declaring a namespace in HappyMapper is to use the value of the namespace. For example:

class Foo
tag 'foo'
namespace 'n'
end

which one would expect to map to <n:foo/>, but in fact you must now provide the namespace declaration the path of the namespace. The above then becomes:

class Foo
tag 'foo'
namespace 'http://www.n.com'
end

1. Are there good reasons for being less explicit?
2. This isn't documented anywhere, and it's not obvious one must use the uri for the namespace.

Installation and setup

Hi,

I'm really sorry for such a question, but I can't get happymapper to work =(

I'm currently using 3.1 and that's what I did:

1. Added gem 'happymapper' to gemfile and Bundled it
2. Inside my controller file added these two:

class Reference
  include HappyMapper
  element :listName, String
  element :itemCount, Integer
end

class Item
  include HappyMapper
  element :id, Integer
  element :name, String
end

3)Then, in my controller action:

    file_contents = File.read('tmp/spo.xml')
    refs = Reference.parse(file_contents)
    @id = refs.item.first.id
    @name = refs.item.first.id

4. It throws uninitialized constant Reference::HappyMapper

PS I also tried to require it, but it says no such file to load -- happymapper

Thanks a lot! Great job!

how should one handle camel case in xml?

overwrite the tag method on the class?
I would rather not set the tag on every element.

Trouble when same element is nested within itself

I'm having some trouble with an XML layout that user an "item" element within an "item" element within an "item" element. I can't figure out how to access the "items" at each level without pulling in the children "items" at the same time.

This gist shows more details (read from the bottom up): https://gist.github.com/788860

Not really an "issue", I'm just completely stuck. I appreciate any assistance anyone can provide.

How do I use the tag method to identify multiple potential tags

Within the XML schema I'm using there are several conditions where two related schema will contain very similar definition.
One may be an BoxSummary and another an BoxConfiguration. Both will contain most or all of the the BoxSummary information. I would like to create an Box class that encompases the BoxSummary definition that can also be call with a has_one method to point the BoxConfiguration back to the Box object. The problem is that the BoxSummary has a tag "BoxSummary" and the tag for the BoxConfiguration is tag "BoxConfigurationBoxSummary".

Can I place both potential box tags in the Box object? If so... how?

has_one mapping

Given the following xml-

<?xml version="1.0" encoding="UTF-8"?>
<stories type="array">
  <story>
    <id type="integer">46499703</id>
    <project_id type="integer">784123</project_id>
    <story_type>feature</story_type>
    <url>http://www.pivotaltracker.com/story/show/46499703</url>
    <estimate type="float">1</estimate>
    <current_state>unstarted</current_state>
    <description>Sed et nisi sit amet leo tempus pulvinar nec vel magna. Pellentesque arcu metus, iaculis eget sollicitudin sit amet, scelerisque ut quam. Nullam mollis ligula at massa viverra luctus. Nulla non erat id arcu feugiat feugiat quis eget augue. Donec vestibulum pretium mi non porttitor. Nulla dapibus gravida mauris vel posuere. Sed dictum ultricies tellus vel venenatis.

&#402;&#172;&#729;&#8706;&#730;&#223;&#8710;&#729;&#402;</description>
    <name>Just a little test</name>
    <owned_by>
      <person>
        <id type="integer">126384</id>
        <name>Brandon Hansen</name>
        <initials>BH</initials>
      </person>
    </owned_by>
    <requested_by>
      <person>
        <id type="integer">1233535</id>
        <name>Chris Gratigny</name>
        <initials>CG</initials>
      </person>
    </requested_by>
    <created_at type="datetime">2013/03/19 21:25:26 UTC</created_at>
    <updated_at type="datetime">2013/03/19 21:33:26 UTC</updated_at>
    <labels>123,testing,&#8710;&#730;&#730;&#8706;&#229;&#402;</labels>
    <comments type="array">
      <comment>
        <id type="integer">39331775</id>
        <text>Adding some comments here</text>
        <author>
          <person>
            <id type="integer">126384</id>
            <name>Brandon Hansen</name>
            <initials>BH</initials>
          </person>
        </author>
        <created_at type="datetime">2013/03/19 21:25:27 UTC</created_at>
      </comment>
      <comment>
        <id type="integer">39331777</id>
        <text>And a picture</text>
        <author>
          <person>
            <id type="integer">126384</id>
            <name>Brandon Hansen</name>
            <initials>BH</initials>
          </person>
        </author>
        <created_at type="datetime">2013/03/19 21:25:27 UTC</created_at>
      </comment>
    </comments>
    <attachments type="array">
      <attachment>
        <id type="integer">17148567</id>
        <filename>Screen_Shot_2013-03-19_at_12.01.09_PM.png</filename>
        <uploaded_by>
          <person>
            <id type="integer">126384</id>
            <name>Brandon Hansen</name>
            <initials>BH</initials>
          </person>
        </uploaded_by>
        <s3_resource>
          <url>https://s3.amazonaws.com/prod.tracker2/resource/17148567/Screen_Shot_2013-03-19_at_12.01.09_PM.png?AWSAccessKeyId=AKIAIKWOAN6H4H3QMJ6Q&amp;Expires=1363730677&amp;Signature=5ZiyAer7jZ9eHqZXcxX%2FNKJYahU%3D</url>
          <expires type="datetime">2013/03/19 22:04:37 UTC</expires>
        </s3_resource>
        <uploaded_at type="datetime">2013/03/19 21:25:21 UTC</uploaded_at>
        <url>https://www.pivotaltracker.com/resource/download/17148567</url>
      </attachment>
    </attachments>
    <tasks type="array">
      <task>
        <id type="integer">13346287</id>
        <description>Get</description>
        <position type="integer">1</position>
        <complete type="boolean">false</complete>
        <created_at type="datetime">2013/03/19 21:25:27 UTC</created_at>
      </task>
      <task>
        <id type="integer">13346289</id>
        <description>It</description>
        <position type="integer">2</position>
        <complete type="boolean">false</complete>
        <created_at type="datetime">2013/03/19 21:25:27 UTC</created_at>
      </task>
      <task>
        <id type="integer">13346291</id>
        <description>Done</description>
        <position type="integer">3</position>
        <complete type="boolean">true</complete>
        <created_at type="datetime">2013/03/19 21:25:27 UTC</created_at>
      </task>
    </tasks>
  </story>
</stories>

And the following mappings-

class Story
  include HappyMapper

  element :id, Integer
  element :project_id, Integer
  element :story_type, String
  element :url, String
  element :estimate, Float
  element :current_state, String
  element :description, String
  element :name, String
  element :created_at, DateTime
  element :updated_at, DateTime
  element :accepted_at, DateTime
  element :labels, String
  has_one :owned_by, Person
  has_one :requested_by, Person
  has_many :attachments, Attachment
  has_many :comments, Comment

end

class Person
  include HappyMapper

  element :id, Integer, deep: true
  element :email, String, deep: true
  element :name, String, deep: true
  element :initials, String, deep: true
end

Both :owned_by, Person, and :requested_by, Person return the person mapped under owned_by. If I switch the order of owned_by and requested_by in the xml, the requested_by will be returned for both instances of Person, which makes me think that we have an issue with the xpath.

What am I missing here?

XML escape ?

(obj is using HappyMapper)

obj.myattr = 'A&BC'
render :inline => obj.to_xml

gettting error "error : unterminated entity reference BC"

am I missing something ?

Cannot get attribute when there is element of same name

Cannot declare has_many with primitive type

I have the following mapping for the attached xml:

```
module FamilySearch
class AlternateIds
include HappyMapper

tag ‘alternateIds’ has_many :ids, String, :tag => ‘id’ end class Information include HappyMapper has_one :alternateIds, AlternateIds end class Person include HappyMapper attribute :version, String attribute :modified, Time attribute :id, String has_one :information, Information end class Persons include HappyMapper has_many :person, Person end class FamilyTree include HappyMapper tag ‘familytree’ attribute :version, String attribute :status_message, String, :tag => ‘statusMessage’ attribute :status_code, String, :tag => ‘statusCode’ has_one :persons, Persons end

end
```

Notice the AlternateIds class declares a has_many with type String, which should assign ids as a collection of Strings. However, ids ends up being a String of the first object.

I have a failing spec in my fork that represents this situation:
http://github.com/jimmyz/happymapper/commit/496365548e20a15707a46be9a756b1035301773e

support for LibXML's ::SaxParser and ::Reader

it would be great if happymapper allowed usage of LibXML's SaxParser or Reader classes for processing huge XML files (that cause tree-based Parser quit with fatal out-of-memory error).

does anyone know how to do it now via some monkeypatching?

LibXML::XML::Error: LibXML::XML::Error at :0.

I'm getting this error only when I use has_many or has_one method.

The code and xml is here: http://gist.github.com/105430

Incompatible with Rails 3.2.3

! rails c
/Users/sdhull/.rvm/gems/ruby-1.9.3-p194/gems/happymapper-0.4.0/lib/happymapper.rb:6:in `<top (required)>': Boolean is not a class (TypeError)
  from /Users/sdhull/.rvm/gems/ruby-1.9.3-p194/gems/bundler-1.1.4/lib/bundler/runtime.rb:68:in `require'
  from /Users/sdhull/.rvm/gems/ruby-1.9.3-p194/gems/bundler-1.1.4/lib/bundler/runtime.rb:68:in `block (2 levels) in require'
  from /Users/sdhull/.rvm/gems/ruby-1.9.3-p194/gems/bundler-1.1.4/lib/bundler/runtime.rb:66:in `each'
  from /Users/sdhull/.rvm/gems/ruby-1.9.3-p194/gems/bundler-1.1.4/lib/bundler/runtime.rb:66:in `block in require'
  from /Users/sdhull/.rvm/gems/ruby-1.9.3-p194/gems/bundler-1.1.4/lib/bundler/runtime.rb:55:in `each'
  from /Users/sdhull/.rvm/gems/ruby-1.9.3-p194/gems/bundler-1.1.4/lib/bundler/runtime.rb:55:in `require'
  from /Users/sdhull/.rvm/gems/ruby-1.9.3-p194/gems/bundler-1.1.4/lib/bundler.rb:119:in `require'
  from /Users/sdhull/Dev/project/config/application.rb:13:in `<top (required)>'

Just for reference, line 13 of application.rb is:

Bundler.require(*Rails.groups(:assets => %w(development test)))

JRuby failure

It could be nice if you let the gem decide to include libxml-jruby over libxml-ruby when jruby is used
I guess i could fork the project and test on JRuby

using xpath expression to grab contents of tag B based on value of tag A

I need to grab the value of when the is 61. The following works at http://www.bit-101.com/xpath/ ...

//metadata[fieldId=61]/value/s

I've tried every syntax I can think of to get the value, but I always get nil. Any thoughts?

Here's the xml:

<document>
  <metadata>
    <fieldId>60</fieldId>
     <value>
        <s>AG-569R-S-12-0010</s>
     </value>
  </metadata>
  <metadata>
    <fieldId>61</fieldId>
      <value>
        <s>Solicitation 1</s>
     </value>
  </metadata>
</document>

Cannot parse recursive structure

The following code gives a ‘stack level too deep’ error message when run on the attached data file:

```
require ‘rubygems’
require ‘happymapper’

class Node
include HappyMapper
element :node_name, String
has_many :node, ::Node
end

class Taxonomy
include HappyMapper
element :taxonomy_name, String
has_many :node, ::Node
end

class Taxonomies
include HappyMapper
has_many :taxonomy, ::Taxonomy
end

file_contents = File.read(‘worldguide_data/taxonomy_short.xml’)
taxonomies = Taxonomies.parse(file_contents)

/opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:93:in `create_collection’: stack level too deep (SystemStackError)
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:93:in `each’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:93:in `create_collection’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:107:in `inject’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:90:in `each’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:90:in `inject’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:90:in `create_collection’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:79:in `parse’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper/item.rb:29:in `from_xml_node’
… 12965 levels…
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:90:in `inject’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:90:in `create_collection’
from /opt/local/lib/ruby/gems/1.8/gems/jnunemaker-happymapper-0.1.5/lib/happymapper.rb:79:in `parse’
from test.rb:22
```

Namespace doesn't work

I'm trying to wrap my head around namespaces and noticed they don't work as described in the examples. If you run the Amazon example (and actually print out the value that has a namespace: #28), it's empty.