farling42 / csv2rw Goto Github PK

View Code? Open in Web Editor NEW

3.0 3.0 1.0 405 KB

Migrate data from CSV or XLSX files into Realm Works

License: GNU General Public License v3.0

QMake 0.99% C++ 92.63% C 1.39% XSLT 3.77% NSIS 1.22%

qt5 realmworks

csv2rw's Introduction

Welcome

csv2rw's People

Contributors

Stargazers

Watchers

Forkers

ramccor

csv2rw's Issues

Multiple parent topics

http://forums.wolflair.com/showpost.php?p=261563&postcount=168

Allow a single CSV column to be dragged across to use as the parent.

Each unique value in the CSV column will create a different parent, and will contain only those rows which have that "key" value in the nominated CSV column.

Allow import of images

Find a way to have images encoded in CSV files which could be imported into picture snippets.

Allow the addition of additional text (and other) snippets to an imported category.

Blank line in CSV creates a topic with no public_name

A CSV file with a blank line at the end creates a topic entry in the RWEXPORT file which is completely blank. This includes the "public_name" being set to an empty string.

Realm Works complains about the file being invalid in this case.

Support for multi-domain tag entries

Allow more than one value to be put into a multi-domain tag snippet.

Report missing TAG enumerations

When converting a source field into a TAG, report in the error window any source fields which do not match a tag.

Read formatting from CSV file

Convert some sort of mark-up language into the output required by Realm Works.

One possibility is to allow basic HTML formatting, e.g. <b> and <i>

Use different delimiter in CSV files

Allow a different character from "," to be used to delimit fields in a CSV file.

Realm Works crashes if the topic name matches an alias

ErinRigh has reported that when an alias matches the main topic title, then Realm Works crashes on import.

See http://forums.wolflair.com/showpost.php?p=274264&postcount=441

A simple check during RWexport creation to check that the main topic title, and/or aliases have the same text, and either report that the import will fail - or ignore the offending alias.

+Name button jumps around

http://forums.wolflair.com/showpost.php?p=269245&postcount=350

I noticed when adding various "Other Names" that the button allowing you to reveal it will jump around, only allowing you to reveal one of the other names.

Support time fields

Provide support for entering date and/or time into Date_Game and Date_Range snippet types.

<game_date
    canonical="000000001:50:66:+000956:50:21:0017:004:002:005"
    gregorian="0956-03-18 04:02:05" />

Format is "YYYY-MM-DD 00:00:00 [ BCE]"
Note that RW is raising an error about the gregorian time not being present, but it is present.

Use same Category for multiple parents

Some people want to use multiple parents, but want the same category to be used for each parent.

Use existing topic as parent topic

Allow import of a partial or full realm export so that an already-existing topic can be used as a parent for the topics being created by a CSV file.

single quote in titles are wrong

A single quote character in a topic title gets imported as â€™, e.g. "A'eouya" becomes "Aâ€™eouya"

Either the CSV Import is wrong, or the public_name attribute doesn't allow single quotes, or XML is treating the single quote as a special marker/character.

Set 'Flavor' on text snippets.

If it isn't possible to set the flavor of text snippets in the structure export file, then add a method of setting this via the CSV2RW tool.

Add missing optional attributes to generated topics

Add missing optional attributes to generated topics for the MAIN topic name:

is_auto_accept (true or false)
case_matching (Ignore or Sensitive or Correction)
match_priority (Normal or Avoid or Prefer or Never)
is_show_nav_pane (true or false)
add the name-specific attributes to the main topic title

Crash when adding multiple names

Describe the bug
There's an issue with adding multiple names. Joe and I have both experienced crashes with it.

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
http://forums.wolflair.com/showpost.php?p=272880&postcount=415

Desktop (please complete the following information):

Windows Version:

Additional context
Add any other context about the problem here.

Support purpose attribute of each snippet

The "purpose" attribute for each snippet can be one of three values: Story_Only, Directions_Only, Both.

The default is "Story_Only".

Annotations don't work for all snippet types

http://forums.wolflair.com/showpost.php?p=275372&postcount=454

I've run into an odd situation where annotations are causing the import to error out. Specifically, it seems the error occurs when placing an annotation on a numeric facet (Label: Numeric field Annotation text box). The numeric field goes in fine, but unless I leave the annotation out of the import, the import hits about 39% and then the application quits with a lovely bug report dialog.

The only other field I've noticed anything strange with is the same annotation on pictures, though that may be unfamiliarity with what data it's expecting, since there was an import error on the report about a name being too short.

http://forums.wolflair.com/showpost.php?p=275427&postcount=456

The annotation field should be plain text. Mostly it's just extra notes explaining more about the numeric field - as an example, the numeric field might be the number of attacks per combat turn for an entry, and the annotation might be that the attacks are 2 punches and a head butt. The annotations were all hand-typed, so the only formatting would be whatever excel puts in by default for a general text cell. None of the annotations are larger than about 75 characters, and I avoided both single and double quotes in them. Other special characters might include +, -, or parentheses, but nothing outside of normal punctuation.

I did test this out prior to posting in a test realm - tag and hybrid text annotations work perfectly fine (those are the other two I use in this particular article set), the picture annotate I'm not too concerned about, I included it to be thorough.

Allow manual edit of metadata

Allow the various fields in the header of the RW import file to be specified via a sub-dialog form.

Very large numbers get imported as 4.35e+08

See http://forums.wolflair.com/showpost.php?p=276758&postcount=477

"I had one cell formatted like this: 999.999.999,99 -> it is a price column and excel showed the numbers in a convinient way. Very high prices were also imported as 4.35e+08 for example."

Multiple CSV files into a single RWexport file

http://forums.wolflair.com/showpost.php?p=261563&postcount=168

Allow multiple CSV files and topic configurations to be saved in a single RWexport file.

(Maybe a merge tool for multiple RWexport files?

Default path for files in CSV files should be the CSV path

When reading files for inclusion in the CSV, the default path should be set to the same folder that contains the CSV file. This will shrink the size of the CSV file by not requiring full path for each file.

Too many topics created

The tool is generating too many topics, where when there are two (or more) titles and one title is a shorter version of another.

Detect URLs

When importing text, detect URLs and automatically put the correct markers around them.

Separate out contents data from underlying category

The current way that the data field allocation is stored inside the RWBaseItem is quite limiting.

If the "topic" and "snippets" were created as a separate layer from the underlying structure information, then this will make it easier to support multiple different topic types (each of a different or the same category but with a different mapping) and to support using the same category for a hierarchy of parents.

Allow predefined text and CSV field names to be mixed together within certain text snippet types.

Text with < fails to import

RW generates an error during import if a contents is supposed to contain the text:

1 object (<1000# total)

The RWexport file contains the text:

&lt;p class=&quot;RWDefault&quot;&gt;&lt;span class=&quot;RWSnippet&quot;&gt;1 object (&lt;1000# total)&lt;/span&gt;&lt;/p&gt;

However the < before 1000 is being expanded during the first step of expansion which means it is seen as the start of a new tag.

Any "<" needs encoding as < BEFORE the entire string then gets converted into this form of text, so that Qt will probably end up putting &lt; into the output RWexport file.

Better checking of imported HTML

Some people have had problems importing data that contains their own html.

One user suggested that they need to include at the start and at the end of the cell in order to get "<b>" working properly. (I think it is just that the cell needs to start with < and end with > to work properly.

Better parsing to reduce the likelihood of the "invalid span" error being reported by Realm Works during import would be helpful.

XML output file has "</>" in it

The XML output file has "</>" in the contents section where the structure has an "<overlay">

Support Numeric snippet types

Numeric snippet types currently aren't supported, so they generate a command-line error of "unsupported snippet type: 2".

There is no specific format for numeric fields that I can see, so test whether it takes a normal "contents" element or something more basic.

Allow the addition of new sections and subsections to an imported category.

Allow predefined text to be placed into certain text snippet types.

Implemented in v1.5

Provide support for Labelled_Text fields.

Note the "label" attribute:

<snippet type="Labeled_Text" label="The Label" >
		<contents>&lt;p class="RWDefault"&gt;&lt;span class="RWSnippet"&gt;The information to go with the label.&lt;/span&gt;&lt;/p&gt;\</contents>
</snippet>

Allow alternate names to be imported from a CSV file.

http://forums.wolflair.com/showthread.php?p=261259#post261259 about importing a dungeon:

"My first thought was building the skeleton of the adventure in Excel and exporting a CSV to use the community created CSV import tool, but that only seems to allow a single layer of containment (I think I'm going to need at least 2 layers), and it doesn't seem to allow for adding additional names (key to building the linking since all references in the text are to the Prefix IDs). I suppose I could make a bunch of CSVs that each import into individual buckets, but if I have to go in and add alternate names after the fact anyway, it doesn't buy me much in the way of time saving."

So...

Provide a method to allow alternative names to be entered into a topic from additional fields.

Add images to the wiki

The wiki could do with some images.

<overlay/> is appearing in some exports

A random "<overlay/>" is appearing as the very first element within a topic.

Create multiple topic categories from a single CSV file

Now that the contents information has been separated out from the structure information it should be possible to have multiple topic categories configured on the right-hand side in order to allow a single CSV file to produce more than one category's worth of information.

The simplest method is to have a "KEY" icon in the topic title which can be used to specify which column of the model should be used to match with a "fixed text" value to match in order for the CSV row to be used to generate a topic of that category.
(Present two combo boxes: the first combo-box lists the column titles; selecting a column will then populate the second combo box with all the unique values found in that column.)

The drop-down menu will also need to indicate (with a leading "*") which categories have been configured to have data generated from the CSV file.

Enforce correct parentage for Articles/Topics

When choosing categories to be parents, examine the "category_role" attribute to ensure that only the subset of categories are displayed: either Article or Topic

Create connections during import

Use a defined column to contain the name that matches a topic being created.

The column containing the reference needs to be copied to a new "connection" object

A connection can be attached to a topic or a plot

<connection target_id="id" target_role="role" nature="nature" 
    rating="rating(-3 to 3)" attitude="attitude"qualifier_tag_id="id" 
    qualifier="text" is_revealed="true(default false)" reveal_date="date">
<annotation>description</annotation>
</connection>

role (record_role) (default: Topic): Topic, Plot, Ext_Object, Smart_Image, Package, Snippet, Section, Plot_Node, Facet, Map_Pin, Partition, Tag, Alias, Connection, Asset, Plot_Group, Category, Domain

nature: Arbitrary, Generic, Union, Parent_To_Offspring, Offspring_To_Parent, Master_To_Minion, Minion_To_Master, Public_Attitude_Towards, Private_Attitude_Towards

attitude: Hostile, Angry, Annoyed, Neutral, Pleased, Friendly, Gracious

qualifier = dependent on role: from one of the domains in the structure definition:

Comprises Relationship Types
Generic Relationship Types

NOTE: Whilst some links are two-way and others are one-way, Realm Works will create the reverse link for all connections; so the RWEXPORT file only needs to contain one of the links (which makes code A LOT easier).

<connection target_id="Topic_3" nature="Master_To_Minion" qualifier_tag_id="Tag_212" qualifier="Residence / Resident"/>
<connection target_id="Topic_7" nature="Arbitrary"/>
<connection target_id="Topic_10" nature="Public_Attitude_Towards" rating="2" attitude="Friendly"/>
<connection target_id="Topic_11" nature="Private_Attitude_Towards" rating="0" attitude="Neutral"/>
<connection target_id="Topic_33" nature="Minion_To_Master" qualifier_tag_id="Tag_210" qualifier="Owner / Subsidiary"/>
<connection target_id="Topic_9" nature="Union"/>
<connection target_id="Topic_31" nature="Generic" qualifier_tag_id="Tag_367" qualifier="General Family"/>

<domain_global domain_id="Domain_20" name="Comprises Relationship Types">
	<tag_global tag_id="Tag_205" name="Affiliation / Member" />
	<tag_global tag_id="Tag_206" name="Depends On / Depended Upon By" />
	<tag_global tag_id="Tag_207" name="Employer / Employee"/>
	<tag_global tag_id="Tag_208" name="Master / Minion"/>
	<tag_global tag_id="Tag_209" name="Organization / Chapter"/>
	<tag_global tag_id="Tag_210" name="Owner / Subsidiary" />
	<tag_global tag_id="Tag_211" name="Parent / Child"  />
	<tag_global tag_id="Tag_212" name="Residence / Resident" />
	<tag_global tag_id="Tag_213" name="Supervisor / Subordinate" />
</domain_global>

<domain_global domain_id="Domain_34" name="Generic Relationship Types">
	<tag_global tag_id="Tag_366" name="Equivalence"/>
	<tag_global tag_id="Tag_367" name="General Family"/>
	<tag_global tag_id="Tag_368" name="Peer"/>
	<tag_global tag_id="Tag_369" name="Personal"/>
</domain_global>

Comprises or Encompasses / Belongs To or Within

Residence / Resident
Depends On / Depended Upon By
Affiliation / Member
Master / Minion
Supervisor / Subordinate
Employer / Employee
Owner / Subsidiary
Organization / Chapter
Parent / Child

Two way connection

<connection target_id="Topic_3" nature="Master_To_Minion" qualifier_tag_id="Tag_212" qualifier="Residence / Resident">
<connection target_id="Topic_33" nature="Minion_To_Master" qualifier_tag_id="Tag_210" qualifier="Owner / Subsidiary">

<domain_global domain_id="Domain_20" name="Comprises Relationship Types">
	<tag_global tag_id="Tag_205" name="Affiliation / Member" />
	<tag_global tag_id="Tag_206" name="Depends On / Depended Upon By" />
	<tag_global tag_id="Tag_207" name="Employer / Employee"/>
	<tag_global tag_id="Tag_208" name="Master / Minion"/>
	<tag_global tag_id="Tag_209" name="Organization / Chapter"/>
	<tag_global tag_id="Tag_210" name="Owner / Subsidiary" />
	<tag_global tag_id="Tag_211" name="Parent / Child"  />
	<tag_global tag_id="Tag_212" name="Residence / Resident" />
	<tag_global tag_id="Tag_213" name="Supervisor / Subordinate" />
</domain_global>

Family Relationship To

Parent of (actually stored as "Belongs To or Within - Parent/Child")
Child of (actually stored as "Belongs To or Within - Parent/Child")
Immediate Ancestor of
Offspring of
Other Family of (actually stored as "Simple connection to - General Family")
Union with

On creating the "Family Relationship To/Parent of" within RW, it is immediately changed to "Belongs To or Within - Parent/Child".

Two way connection

<connection target_id="Topic_9" nature="Union"/> for "Union"
<connection target_id="Topic_9" nature="Parent_To_Offspring"/> for "Immediate Ancestor Of"

Public Attitude Towards / Private Attitude Towards

attitude attribute: Graciously, Friendly, Pleased, Neutral, Annoyed, Angry Hostile

rating attribute: 3 to -3

The connection is one-way (no connection back from target to source).

<connection target_id="Topic_10" nature="Public_Attitude_Towards" rating="2" attitude="Friendly">
<connection target_id="Topic_11" nature="Private_Attitude_Towards" rating="0" attitude="Neutral">

Simple Connection to

Qualifiers: Peer, Equivalance, Personal, General Family

The same qualifier_tag_id is used in both topics

<connection target_id="Topic_31" nature="Generic" qualifier_tag_id="Tag_367" qualifier="General Family"/>

<domain_global domain_id="Domain_34" name="Generic Relationship Types">
	<tag_global tag_id="Tag_366" name="Equivalence"/>
	<tag_global tag_id="Tag_367" name="General Family"/>
	<tag_global tag_id="Tag_368" name="Peer"/>
	<tag_global tag_id="Tag_369" name="Personal"/>
</domain_global>

Arbitrary connection to

No qualifiers

The connection is one-way (no connection back from target to source).

<connection target_id="Topic_7" nature="Arbitrary">

Sub-categories are not listed in the categories list

Only top-level categories are listed in the categories list, sub-categories do not appear.

Support Veracity of each snippet

The veracity attribute of each snippet can be one of three values: Truth, Partial, Lie

Don't show categories which are "hidden" in Realm Works

In Realm Works it is possible to hide categories.

The CSV2RW tool should not display these in the list of available categories.

More file types to be imported

Extend the import of the file contents to include:
Statblock (RTF files)
Portfolio (.por files)
Rich_Text
PDF
Audio
Video
HTML
Foreign

Crash during import if an Excel cell has multiple paragraphs

Describe the bug
Crash during import if an Excel cell has multiple paragraphs.

To Reproduce
See http://forums.wolflair.com/showpost.php?p=268755&postcount=333