Coder Social home page Coder Social logo

feed-pull's Introduction

Feed Pull Dockunit Status

This is a simple WordPress plugin that is largely based off of Automattic's Syndication plugin. Easily setup feeds to be syndicated into WordPress. You can map feed attributes to post fields or post meta. Syndicated content can be mapped to any post type you want. The plugin comes with a convenient settings page so you can configure things like when feeds are syndicated.

Feed Pull vs. Syndication

Feed Pull takes a lot of concepts from Syndication. Syndication is a more advanced plugin that offers more than just feed pulling. Here are some differences between the two plugins

  • Feed Pull is a plugin for pulling content out of XML feeds. Syndication has feed pulling as well as content pushing features
  • Feed Pull has a much friendlier feed management screen. Both Syndication and Feed Pull use the WordPress post edit, screen, however Feed Pull offers a much cleaner experience from the way options are presented to the instructions provided. At the moment Feed Pull does not have all the features of Syndication such as taxonomy and constant field mapping. However, Feed Pull probably has everything you need and is very extensible.
  • Feed Pull has far superior error logging to Syndication.
  • Feed Pull allows you to schedule content pulling in the future.
  • Feed Pull allows you to do manual pulls using AJAX rather than a sometimes frustrating one-time cron job.

Setup Instructions

  1. Upload and activate the plugin.
  2. A "Feed Pull" settings page has been added as a sub-item of the "Settings" admin menu.
  3. Make sure "Pull feeds" is set to yes. Feeds are syndicated using WordPress cron. This means your content will be pulled in the background. By default feed pulls occur once per hour.

Configure a Feed

  1. After activating the plugin, you should now see a "Source Feeds" menu item in your admin menu. Add a new source feed.
  2. Enter the URL of any XML feed in the feed url input box.
  3. Enter an XPath to the posts within your feed. This tells the plugin where each piece of content lives in your feed. For most RSS feeds, channel/item will do just fine. Here is a bland tutorial if you want to learn more about XPath: http://www.w3schools.com/XPath/
  4. Setup the new content to your liking. The defaults are good to start with.
  5. Finally, we need to map fields in the feed to your new posts. Title and GUID are required. Title is self-explanatory. A GUID is a unique identifier for posts. Items within your feed should have some sort of GUID or permalink that you can map to the post GUID. GUID's allow the plugin to determine if a piece of content has already been syndicated or not. There is an in-depth section on field mapping below.

Atom Feeds and Custom Namespaces

Sometimes feeds make use of prefixes on certain elements. This happens for a variety of reasons; one is to avoid conflicts. Prefixes should have a namespace defined somewhere using an "xmlns" attribute. Feed Pull lets you define custom namespaces. This feature is advanced and not needed in most cases.

However, certain feeds, like the Atom feeds outputted by WordPress define a document level namespace without a prefix. This is totally fine except it makes XPath queries difficult. Feed Pull should automatically detect if this situation is happening and create a namespace with the prefix "default" and the url "http://www.w3.org/2005/Atom". You should use the prefix in your XPath queries. For example, instead of "//feed/entry", your query should be "//default:feed/default:entry". Instead of "title" in your field map, your query should probably be "default:title".

You can learn more about namespaces here: http://www.w3schools.com/xml/xml_namespaces.asp

Field Mapping

When configuring a source feed, you need to tell Feed Pull which XML nodes map to where within WordPress. Here are the current mapping types supported by the plugin:

  1. Post Field (Map content to fields within the post table)
  2. Post Meta (Map content to fields within the post_meta table that refer to the post being created)
  3. Taxonomy (Map content to terms in any taxonomy that relate to the post being created)

Let's see these mapping types in action. Here is a super simple feed structure:

<channel>
   <item>
      <title>Post Title!</title>
      <guid>http://yoursite.com/post-title</guid>
      <copyright>CNN.com</copyright>
      <tag>United States</tag>
      <tag>Politics</tag>
   </item>
   <item>
      <title>Another Post!</title>
      <guid>http://yoursite.com/another-post</guid>
      <copyright>CNN.com</copyright>
      <tag>Celebrities</tag>
   </item>
</channel>

As you can see our simple XML document contains two "items". We installed Feed Pull because we want to create posts within WordPress for each of those items. Now we need to map XML nodes to places within each new post being created. Within the Field Mapping meta box there are two required mappings: Title and GUID. Therefore we MUST pick XML nodes in our feed to map to these things. For post_title, our "Source Field" will be "title". For guid, our "Source Field" will be guid. Simple right?

Now we can map whatever nodes we want to post meta in our feed. For educationally purposes, let's say we want to map "copyright" to post meta for each new post being created. To do this our "Source Field" would be "copyright". Our new post location can be named whatever we want; let's say "post_copyright". For "Mapping Type", we choose "Post Meta".

Like post meta there are no required taxonomy mappings. Let's create one anyway! We want the "tag" nodes in our feed to map to the "post_tag" taxonomy in WordPress. Therefore we create a new field mapping row with "tag" as "Source Field", "post_tag" as "New Post Location", and "Taxonomy" as "Mapping Type".

Development

Setup

Follow the configuration instructions above to setup the plugin.

Testing

Within the terminal change directories to the plugin folder. Initialize your testing environment by running the following command:

For VVV users:

bash bin/install-wp-tests.sh wordpress_test root root localhost latest

For VIP Quickstart users:

bash bin/install-wp-tests.sh wordpress_test root '' localhost latest

where:

  • wordpress_test is the name of the test database (all data will be deleted!)
  • root is the MySQL user name
  • root is the MySQL user password (if you're running VVV). Blank if you're running VIP Quickstart.
  • localhost is the MySQL server host
  • latest is the WordPress version; could also be 3.7, 3.6.2 etc.

Run the plugin test suite:

phpunit
Dockunit

This plugin contains a valid Dockunit file for running unit tests across a variety of environments locally (PHP 5.2 and 5.5). You can use Dockunit (after installing it via npm) by running:

dockunit

Issues

If you identify any errors or have an idea for improving the plugin, please open an issue.

License

Feed Pull is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

feed-pull's People

Contributors

danielbachhuber avatar dtygel avatar mattsims avatar sc0ttkclark avatar tannerm avatar tlovett1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

feed-pull's Issues

Help request; feed-pull content lay-out

Hi,

I have a question. I installed the feedpull plugin and I am now able to retrieve data from the XML-feed into a post or page.
It now creates for each post a new page, which is okay. But my question is how can I create a lay-out so the main content in the post or page is displayed for each item in the same way?
Can I create some sort of template? Please advise. I never worked with this before so I hope you are willing to explain a little bit how I can do this.

Thanks in advance! :-)

Be able to select whether feed is from WordPress or not

When dealing with entering many WordPress feeds, it'd be great to be able to click a button to apply the WordPress XPath / Namespaces (dc/content) / Field mapping etc.

It could be JS or PHP based, so you add a new feed, then you can click "This is a WordPress Feed" and it will load up the defaults from a local JS variable (not AJAX, since we could know what they are since they're all the same).

Batch adding feed sources

Hi,

I have loads of feed sources to load. Is there a way to programatically add feed sources from a csv file? I already have this table with the feed source name, url, XPath, and mapping. Doing it manually will be really tough.

Thanks!

Adding translations

Hi,
I've just translated the feed-pull.pot file and saved to feed-pull-pt_BR, but it doesn't get loaded. Is there anything that must be set in the plugin code to load the language files from the language set in the wordpress site?
Ah: and how do I send you my translation to contribute to your plugin?
Thanks,
daniel

Question: dealing with HTML entities in nodes

Hi, thanks for a great plugin which I am using to pull in an RSS feed from the main University news feed into our departmental website. Rather than using a [media] node, the featured image is included in the [description] node.

I am trying to pull the src of the image with //item/description//img[@Class='cam-scale-with-grid']/@src and mapping it to post_content as a test.

Unfortunately it would appear that because the feed source uses HTML entity names the XPath above does not work. I have duplicated this with a couple of XPATH validators, and it would seem that if I replace the HTML entity names with, for example "/, < or > the XPATH works.

Am I missing something, or I am attempting the impossible. Please feel free to delete this post if this is the wrong place to post it!

Get post tags from a comma separated XML term

The XML feed provide the keywords/tags for a post like this:

tag1, tag2, tag3, tag4

I tried to use /keywords/text() and yes, I will get ONE tag like "tag1, tag2, tag3, tag4", but unfortunately Wordpress do not understand this term as four separate tags.

So, how could I extract the tags in a way Wordpress can handle them?

Best
Sebastian

Error while Pull

Hi,

We are receiving following error while doing feed pull

Status : Pulling feed
Warning : Xpath to source field returns nothing for HtmlBody
Status : Attempting to update post with guid
Status : Updated post - Edit Post
Warning : Xpath to source field returns nothing for HtmlIntro - Edit Post
Warning : Xpath to source field returns nothing for HtmlContact - Edit Post
Warning : Xpath to source field returns nothing for HtmlBody

Feed Pull with WP Fastest Cache

It seems there are some problems with the cron job resp. the publishing of my posts using Feed Pull together with activated WP Fastest Cache. Could this be possible caused an issue? And, of course, what could I do to run the Feed Pull cron successfully?

Best
Sebastian

Error: Could not create post: Content, title, and excerpt are empty.

Hi,

Trying to use your plugin to pull in this feed:
http://tickets.courttheatre.org/feed/events/

Every record fails, giving me the error I put as the subject for each record. I've tried simplifying the field mapping, changing the post type to standard post (was trying to create event records)

The XML file looks fine, there's definitely data in the XPaths I'm using, but still I can't create posts because it says the above error.

Any suggestions?

Edit: Just noticing that this is probably worth mentioning too: I am using a node called DirectLink in the XML which is a url formatted link to a page for the guid. However, every record I am trying to insert says:

Status : Attempting to update post with guid Array

It's seeing an array for the guid, not the node value. Using a different node did not change it.

Thanks.

Mapping a field attribute?

I know you can map an element such as The Catcher in the Rye

but are you able to reference the attribute of an element such as

In the above example I want to map the name attribute to a field. I cannot seem to find an answer to this. Is this existing functionality? I have tried mapping person["name"] but it doesn't map anything.

Suggestion: grab first image in content as featured image

Hi,

I would like to make a suggestion, which I could try to help programming: In the feed source edition page, there could be an input called "XPath to grab featured image". Then the user can put a XPath, and feed pull will first see if the content is html or simply an url of an image. If it's HTML, feed pull will parse the first "img" tag and get the 'src' attribute. If it's only an url, this will be the image url.

Then feed pull would add this url as featured image for the posts loaded on each feed pull.

Is this interesting or under the scope of the plugin?

Multiple category

Is it possible to map the category by feed?

If I choose one category in "Automatically Add New Posts to Categories:", could I also add another category by feed? I tested that but the category by feed override the "Automatically Add New Posts to Categories:".

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.