automattic / aggregator

Synchronise posts between blogs in a multisite network.

Aggregator

Synchronise posts between blogs in a multisite network.

Basic Usage

Network activate the plugin and you'll find a new options screen in Network Admin > Settings > Aggregator.

Here you can create what we call "aggregator jobs". An aggregator job describes a single portal and source, and what should be pushed between them. You can create multiple aggregator jobs for each portal and source, but you can't have multiple jobs describing the same connection.

For example, if I have two sites, I can create a maximum of two jobs;

Portals	Sources
Site A	Site B
Site B	Site A

You can't do this;

Portals	Sources
Site A	Site B
Site B	Site A
Site B	Site A

The second and third jobs here conflict with each other. This is open to change, if you want to send in a pull request.

Job settings

When you create an aggregator job, you choose the post types, taxonomies and terms you want to sync. Only posts that match the settings will be synced. Here's some examples;

Job	Post Types	Taxonomies	Terms	Outcome
A	Posts	Categories	[None]	All posts will be pushed along with any assigned categories, tags will be ignored.
B	Posts	Categories	Categories: News, Reviews	Only posts that belong in either the News or Reviews categories will be pushed. Tags ignored.
C	Posts	Categories, Tags	[None]	All posts will be pushed along with any assigned categories and tags.
D	Posts	Categories, Tags	Categories: News; Tags: Featured	Only posts in the News category and tagged 'Featured' will be pushed.

You'll also need set an "author". This is the author on the portal to whom pushed posts will be assigned. I.e., the plugin doesn't check if a post's author exists on the portal and retains the association. Instead, each post that is pushed will be given the same author.

Technical stuff

Class: Aggregate

Sets up the plugin, creating the admin area and some other bits and bobs.

Class: Aggregator_Job

This represents the aggregator jobs you create in the network admin. To create/retrieve an Aggregator_Job object, the class is called with the IDs of the portal and source blogs as parameters. The Class does all the hard work of retrieving the settings so that they can be checked during a 'push'.

Class: Aggregate

This is where the pushing of posts actually takes place. The push_post_data_to_blogs function starts the grunt work, retrieving the Aggregator_Job object and checking if the saved post should be pushed based on the job settings.

Classes: Aggregator_Jobs_List_tables and Aggregator_Portals_List_table

These two are just extensions of the WP_List_Table class and create the tables you see in Network Admin.

Site Options

There are a couple of pretty important site options (as in get_site_option) that the plugin relies on.

• aggregator_{$blog_id}_source_blogs - Stores a simple array of blog IDs. In this case, the IDs of blogs that posts will be pushed from where $blog_id is a portal
• aggregator_{$blog_id}_portal_blogs - Vice versa above, so $blog_id is the ID of a source blog, and the value is an array portal blog IDs.

Filters

There are some strategically placed filters that you can use to overwrite some of Aggregator's behaviour. These are well documented inline, so they are listed below and linked to the code.

• aggregator_sync_meta_key - class-aggregate.php:91-102 - Decide whether to push some meta data to a portal site.
• aggregator_allowed_post_types - class-aggregate.php:118-127 - Override the allowed post types as set by a sync job.
• aggregator_taxonomy_terms - class-aggregate.php:167-173 - Allow overriding of non-whitelisted taxonomies and terms.
• aggregator_taxonomy_terms - class-aggregate.php:243-249 - Allow overriding of non-whitelisted taxonomies and terms.
• aggregator_orig_post_data - class-aggregate.php:462-470 - Alter the post data before syncing.
• aggregator_orig_meta_data - class-aggregate.php:498-507 - Alter the meta data before syncing.
• aggregator_portal_blogs - class-aggregator.php:217-226 - Filters the list of blogs to push to.
• aggregator_source_blogs - class-aggregator.php:247-255 - Filters the list of blogs to push from.

Step-by-step instructions

Pushing posts to the national site is done using a plugin called Aggregator. Here's how to set that up:

1. Log in to WordPress
2. Hover over "My Sites" in the top left to reveal the sites menu, and navigate to Network Admin
3. In Network Admin, under Settings, choose "Aggregator"
4. Click "Add New"
5. From the first drop down, choose the site you wish posts to be pushed to – e.g. the national site.
6. From the second, choose the source of posts
7. You will now be able to set the criteria for this "sync job" - this works on a whitelist system. E.g., you choose the post types, taxonaomies and terms that should be pushed. Anything that isn't selected won't be pushed.
8. Select your preferred post types, taxonomies, and terms
9. Click Save

aggregator's Issues

Not all sites in network are being returned.

When I go to create a new sync job I can only select one blog as the "portal" and "source". My network has a total of 7 blogs but I can only select 1. The one I can select happens to be the last blog in my network.

If I change $blogs = Aggregator::get_sites(); to $blogs = get_sites(); in network-admin-setup.php all of the blogs are returned, but I am unable to to save the sync job.

Any reason why that might be? Is there anything you would like me to do to help troubleshoot?

Add metavalue and indicator when post is detached

We have an indicator for when a post is aggregated, but nothing that indicates that a post originated on another site and has been detached. It would be useful to add a meta value that can be used for other purposes (for example it would be useful to know in our page builder that a post has been detached) and a filterable indicator that would display in the post list view indicating that the post has been detached

Roughly something along the lines of:
Meta: _aggregator_detached => {original blog or post ID, or truthy value}
Indicator: add_filter( 'aggregator_detached_label', __( 'Detached' ), $post_id )

Invalid callback for `post_link` hook

This line https://github.com/Automattic/aggregator/blob/master/class-aggregator.php#L64 is incorrect, and produces a warning call_user_func_array() expects parameter 1 to be a valid callback, second array member is not a valid method.

View/Preview links on the cloned post link to the origin site

When viewing a cloned post in the post list screen the action links link to the origin site instead of the cloned-to site, even if the post has been published. This is confusing for users as they would expect to view the post as it is on the cloned-to site.

Terms not assigned on cloned post

Terms are not being assigned on a post cloned from the origin site. The data is being carried over but the post displays in the list table with Uncategorized instead of the proper terms.

Oddly, if you clone a post (using a post cloning plugin we've written - uses wp_set_object_terms() to assign the same terms to the newly cloned post) the terms/taxonomies will display correctly on the new-new post.

$aggregator not in global scope causing fatal error in class-aggregate.php

Whenever the save_post action is triggered we end up with a fatal 500 error, caught by Query Monitor that results from attempting to pull a non-global variable as global. This is 500ing on our local installs and causing posts not to update if an aggregator job is triggering.

The issue comes from using $aggregator as a global in class-aggregate.php line 40. The variable is not available within global scope which leads to class variable resolving as null when called.

I'd recommend either calling the Aggregator class anew when loading the Aggregate class or using a static instance of the class and not relying on globals because globals kinda suck.

Error and Trace:

Xdebug: Fatal error: Uncaught Error: Call to a member function get_portals() on null in /srv/www/thesun/wp-content/plugins/aggregator/class-aggregate.php:992 
Stack trace: 
#0 /srv/www/thesun/wp-includes/plugin.php(524): Aggregate->save_post(1373719, Object(WP_Post)) 
#1 /srv/www/thesun/wp-includes/post.php(3409): do_action('save_post', 1373719, Object(WP_Post), true) 
#2 /srv/www/thesun/wp-includes/post.php(3482): wp_insert_post(Array, false) 
#3 /srv/www/thesun/wp-admin/includes/post.php(376): wp_update_post(Array) 
#4 /srv/www/thesun/wp-admin/post.php(193): edit_post() 
#5 {main} thrown in /srv/www/thesun/wp-content/plugins/aggregator/class-aggregate.php on line 992. 
Output triggered in /srv/www/thesun/wp-content/mu-plugins/query-monitor/collectors/php_errors.php on line 143

PHP Notice due to lack of parsing in prepare_terms

Notice & Trace:

[05-Oct-2016 21:28:18 UTC] PHP Warning:  Invalid argument supplied for foreach() in /srv/www/site/wp-content/plugins/aggregator-fork/class-aggregate.php on line 613
[05-Oct-2016 21:28:18 UTC] PHP Stack trace:
[05-Oct-2016 21:28:18 UTC] PHP   1. {main}() /srv/www/site/wp-admin/post.php:0
[05-Oct-2016 21:28:18 UTC] PHP   2. edit_post($post_data = *uninitialized*) /srv/www/site/wp-admin/post.php:193
[05-Oct-2016 21:28:18 UTC] PHP   3. wp_update_post($postarr = array ('ID' => 1373690, 'post_author' => 1, 'post_date' => '2016-10-05 22:28:16', 'post_date_gmt' => '', 'post_content' => '', 'post_title' => 'New Post', 'post_excerpt' => '', 'post_status' => 'publish', 'comment_status' => 'open', 'ping_status' => 'closed', 'post_password' => '', 'post_name' => '', 'to_ping' => '', 'pinged' => '', 'post_modified' => '2016-10-05 22:28:03', 'post_modified_gmt' => '2016-10-05 21:28:03', 'post_content_filtered' => '', 'post_parent' => '0', 'guid' => 'https://site.dev/?p=1373690', 'menu_order' => '0', 'post_type' => 'post', 'post_mime_type' => '', 'comment_count' => '0', 'filter' => 'raw', 'ancestors' => array (), 'post_category' => array (0 => '0'), 'tags_input' => array (), '_wpnonce' => '55cf6faede', '_wp_http_referer' => '/wp-admin/post-new.php', 'user_ID' => 1, 'action' => 'editpost', 'originalaction' => 'editpost', 'original_post_status' => 'auto-draft', 'referredby' => 'https://site.dev/wp-admin/', '_wp_original_http_referer' => 'https://site.dev/wp-admin/', 'auto_draft' => '', 'post_ID' => '1373690', 'meta-box-order-nonce' => 'fd5a5ac2db', 'closedpostboxesnonce' => '5fb5502a9c', 'samplepermalinknonce' => 'becf153c50', 'fieldmanager-iets_title_metabox_group-nonce' => 'fb088e3ba9', 'teis_title_metabox_group' => array ('sub_headline' => '', 'byline' => ''), 'content' => '', 'syndicate_noncename' => '1744d14ad3', 'wp-preview' => '', 'hidden_post_status' => 'draft', 'hidden_post_password' => '', 'hidden_post_visibility' => 'public', 'visibility' => 'public', 'mm' => '10', 'jj' => '05', 'aa' => '2016', 'hh' => '22', 'mn' => '27', 'ss' => '49', 'hidden_mm' => '10', 'cur_mm' => '10', 'hidden_jj' => '05', 'cur_jj' => '05', 'hidden_aa' => '2016', 'cur_aa' => '2016', 'hidden_hh' => '22', 'cur_hh' => '22', 'hidden_mn' => '27', 'cur_mn' => '27', 'original_publish' => 'Publish', 'publish' => 'Publish', 'consumer_site-restricted-taxonomies-add-term-nonce' => '94290b6e3e', 'tax_input' => array ('consumer_site' => array (0 => 258), 'post_tag' => array (), 'hashtags' => array (), 'football-team-taxo' => array (), 'who' => array (), 'where' => array (), 'feeds' => array (0 => '0')), 'newtag' => array ('consumer_site' => '', 'post_tag' => '', 'hashtags' => '', 'football-team-taxo' => '', 'who' => '', 'where' => ''), 'newcategory' => 'New Section Name', 'newcategory_parent' => '-1', '_ajax_nonce-add-category' => '7f4e48f7eb', 'football-team-taxo-restricted-taxonomies-add-term-nonce' => '3f3f101f75', 'who-restricted-taxonomies-add-term-nonce' => '1202659cf8', 'where-restricted-taxonomies-add-term-nonce' => '220a47fa03', 'newfeeds' => 'New Feed Name', 'newfeeds_parent' => '-1', '_ajax_nonce-add-feeds' => '164734bc76', '_thumbnail_id' => '-1', 'fieldmanager-site_image_metabox_group-nonce' => '9ee0c5a813', 'site_image_metabox_group' => array ('landscape_image' => '', 'portrait_image' => '', 'hero_image' => '', 'mobile_image' => '', 'gallery_image' => '', 'dist_content_image' => ''), 'fieldmanager-site_teaser_metabox_group-nonce' => 'ab11f181b4', 'site_teaser_metabox_group' => array ('teaser_kicker' => '', 'teaser_headline' => '', 'teaser_lead' => ''), 'fieldmanager-site_article_metabox_group-nonce' => '0066d363e9', site_article_metabox_group' => array ('article_kicker' => '', 'article_slug' => '', 'featured_video_id' => '', 'featured_video_duration' => '', 'hide_headline' => '', 'hide_sharing' => '', 'hide_ads' => '', 'show_updated' => '', 'full_width' => '', 'override_recent_articles_count' => ''), 'fieldmanager-feed_override_fields-nonce' => 'fa7610f767', 'feed_override_fields' => array ('smartphone' => array ('override_pub_date' => array ('date' => '', 'hour' => '', 'minute' => ''), 'headline' => '', 'sub_headline' => '', 'byline' => '', 'teaser_text' => '', 'teaser_kicker' => '', 'teaser_headline' => '', 'article_slug' => '', 'show_image_as_gallery' => '', 'body_override' => '', 'hide_landscape' => '1', 'main_image' => '', 'landscape_image' => '', 'portrait_image' => '', 'smartphone_image' => '', 'favourite_image' => '', 'hi_res_image' => '', 'linked_appcrossref' => ''), 'goalsapp' => array ('override_pub_date' => array ('date' => '', 'hour' => '', 'minute' => ''), 'headline' => '', 'sub_headline' => '', 'byline' => '', 'teaser_text' => '', 'teaser_kicker' => '', 'teaser_headline' => '', 'article_slug' => '', 'show_image_as_gallery' => '', 'body_override' => '', 'hide_landscape' => '', 'main_image' => '', 'landscape_image' => '', 'portrait_image' => '', 'smartphone_image' => '', 'favourite_image' => '', 'hi_res_image' => '', 'linked_appcrossref' => '', 'linked_match' => ''), 'favourite' => array ('override_pub_date' => array ('date' => '', 'hour' => '', 'minute' => ''), 'headline' => '', 'sub_headline' => '', 'byline' => '', 'teaser_text' => '', 'teaser_kicker' => '', 'teaser_headline' => '', 'article_slug' => '', 'show_image_as_gallery' => '', 'body_override' => '', 'hide_landscape' => '1', 'main_image' => '', 'landscape_image' => '', 'portrait_image' => '', 'smartphone_image' => '', 'favourite_image' => '', 'hi_res_image' => '', 'linked_appcrossref' => '', 'linked_tipdata' => array ('proto' => '', 0 => ''), 'headlineDisplay' => 'Default', 'tipData' => '', 'staticOdds' => '', 'eventDate' => array ('date' => ''), 'tipFlag' => '', 'tipSubFlag' => '', 'previewOnly' => ''), 'applenews' => array ('an_headline' => '', 'an_subdeck' => '', 'an_flag' => '', 'an_teaser_headline' => '')), 'coauthors-main' => 'admin', 'coauthors' => array (0 => 'admin'), 'coauthorsinput' => array (0 => 'Search for an author'), 'coauthors-nonce' => '9b12c381ac', 'wp-seo-nonce' => 'b9ef04e6c1', 'seo_meta' => array ('title' => '', 'description' => '', 'keywords' => ''), 'excerpt' => '', 'advanced_view' => '1', 'liveblog-key-template-name' => 'timeline', 'liveblog-key-template-format' => 'first-linebreak', 'liveblog-key-limit' => ''), $wp_error = *uninitialized*) /srv/www/site/wp-admin/includes/post.php:376
[05-Oct-2016 21:28:18 UTC] PHP   4. wp_insert_post($postarr = array ('post_author' => 1, 'post_content' => '', 'post_content_filtered' => '', 'post_title' => 'New Post', 'post_excerpt' => '', 'post_status' => 'publish', 'post_type' => 'post', 'comment_status' => 'open', 'ping_status' => 'closed', 'post_password' => '', 'to_ping' => '', 'pinged' => '', 'post_parent' => 0, 'menu_order' => 0, 'guid' => 'https://site.dev/?p=1373690', 'import_id' => 0, 'context' => '', 'ID' => 1373690, 'post_date' => '2016-10-05 22:28:16', 'post_date_gmt' => '', 'post_name' => '', 'post_modified' => '2016-10-05 22:28:03', 'post_modified_gmt' => '2016-10-05 21:28:03', 'post_mime_type' => '', 'comment_count' => '0', 'ancestors' => array (), 'post_category' => array (0 => '0'), 'tags_input' => array (), '_wpnonce' => '55cf6faede', '_wp_http_referer' => '/wp-admin/post-new.php', 'user_ID' => 1, 'action' => 'editpost', 'originalaction' => 'editpost', 'original_post_status' => 'auto-draft', 'referredby' => 'https://site.dev/wp-admin/', '_wp_original_http_referer' => 'https://site.dev/wp-admin/', 'auto_draft' => '', 'post_ID' => '1373690', 'meta-box-order-nonce' => 'fd5a5ac2db', 'closedpostboxesnonce' => '5fb5502a9c', 'samplepermalinknonce' => 'becf153c50', 'fieldmanager-site_title_metabox_group-nonce' => 'fb088e3ba9', 'site_title_metabox_group' => array ('sub_headline' => '', 'byline' => ''), 'content' => '', 'syndicate_noncename' => '1744d14ad3', 'wp-preview' => '', 'hidden_post_status' => 'draft', 'hidden_post_password' => '', 'hidden_post_visibility' => 'public', 'visibility' => 'public', 'mm' => '10', 'jj' => '05', 'aa' => '2016', 'hh' => '22', 'mn' => '27', 'ss' => '49', 'hidden_mm' => '10', 'cur_mm' => '10', 'hidden_jj' => '05', 'cur_jj' => '05', 'hidden_aa' => '2016', 'cur_aa' => '2016', 'hidden_hh' => '22', 'cur_hh' => '22', 'hidden_mn' => '27', 'cur_mn' => '27', 'original_publish' => 'Publish', 'publish' => 'Publish', 'consumer_site-restricted-taxonomies-add-term-nonce' => '94290b6e3e', 'tax_input' => array ('consumer_site' => array (0 => 258), 'post_tag' => array (), 'hashtags' => array (), 'football-team-taxo' => array (), 'who' => array (), 'where' => array (), 'feeds' => array (0 => '0')), 'newtag' => array ('consumer_site' => '', 'post_tag' => '', 'hashtags' => '', 'football-team-taxo' => '', 'who' => '', 'where' => ''), 'newcategory' => 'New Section Name', 'newcategory_parent' => '-1', '_ajax_nonce-add-category' => '7f4e48f7eb', 'football-team-taxo-restricted-taxonomies-add-term-nonce' => '3f3f101f75', 'who-restricted-taxonomies-add-term-nonce' => '1202659cf8', 'where-restricted-taxonomies-add-term-nonce' => '220a47fa03', 'newfeeds' => 'New Feed Name', 'newfeeds_parent' => '-1', '_ajax_nonce-add-feeds' => '164734bc76', '_thumbnail_id' => '-1', 'fieldmanager-site_image_metabox_group-nonce' => '9ee0c5a813', 'site_image_metabox_group' => array ('landscape_image' => '', 'portrait_image' => '', 'hero_image' => '', 'mobile_image' => '', 'gallery_image' => '', 'dist_content_image' => ''), 'fieldmanager-site_teaser_metabox_group-nonce' => 'ab11f181b4', 'site_teaser_metabox_group' => array ('teaser_kicker' => '', 'teaser_headline' => '', 'teaser_lead' => ''), 'fieldmanager-site_article_metabox_group-nonce' => '0066d363e9', 'site_article_metabox_group' => array ('article_kicker' => '', 'article_slug' => '', 'featured_video_id' => '', 'featured_video_duration' => '', 'hide_headline' => '', 'hide_sharing' => '', 'hide_ads' => '', 'show_updated' => '', 'full_width' => '', 'override_recent_articles_count' => ''), 'fieldmanager-feed_override_fields-nonce' => 'fa7610f767', 'feed_override_fields' => array ('smartphone' => array ('override_pub_date' => array ('date' => '', 'hour' => '', 'minute' => ''), 'headline' => '', 'sub_headline' => '', 'byline' => '', 'teaser_text' => '', 'teaser_kicker' => '', 'teaser_headline' => '', 'article_slug' => '', 'show_image_as_gallery' => '', 'body_override' => '', 'hide_landscape' => '1', 'main_image' => '', 'landscape_image' => '', 'portrait_image' => '', 'smartphone_image' => '', 'favourite_image' => '', 'hi_res_image' => '', 'linked_appcrossref' => ''), 'goalsapp' => array ('override_pub_date' => array ('date' => '', 'hour' => '', 'minute' => ''), 'headline' => '', 'sub_headline' => '', 'byline' => '', 'teaser_text' => '', 'teaser_kicker' => '', 'teaser_headline' => '', 'article_slug' => '', 'show_image_as_gallery' => '', 'body_override' => '', 'hide_landscape' => '', 'main_image' => '', 'landscape_image' => '', 'portrait_image' => '', 'smartphone_image' => '', 'favourite_image' => '', 'hi_res_image' => '', 'linked_appcrossref' => '', 'linked_match' => ''), 'favourite' => array ('override_pub_date' => array ('date' => '', 'hour' => '', 'minute' => ''), 'headline' => '', 'sub_headline' => '', 'byline' => '', 'teaser_text' => '', 'teaser_kicker' => '', 'teaser_headline' => '', 'article_slug' => '', 'show_image_as_gallery' => '', 'body_override' => '', 'hide_landscape' => '1', 'main_image' => '', 'landscape_image' => '', 'portrait_image' => '', 'smartphone_image' => '', 'favourite_image' => '', 'hi_res_image' => '', 'linked_appcrossref' => '', 'linked_tipdata' => array ('proto' => '', 0 => ''), 'headlineDisplay' => 'Default', 'tipData' => '', 'staticOdds' => '', 'eventDate' => array ('date' => ''), 'tipFlag' => '', 'tipSubFlag' => '', 'previewOnly' => ''), 'applenews' => array ('an_headline' => '', 'an_subdeck' => '', 'an_flag' => '', 'an_teaser_headline' => '')), 'coauthors-main' => 'admin', 'coauthors' => array (0 => 'admin'), 'coauthorsinput' => array (0 => 'Search for an author'), 'coauthors-nonce' => '9b12c381ac', 'wp-seo-nonce' => 'b9ef04e6c1', 'seo_meta' => array ('title' => '', 'description' => '', 'keywords' => ''), 'excerpt' => '', 'advanced_view' => '1', 'liveblog-key-template-name' => 'timeline', 'liveblog-key-template-format' => 'first-linebreak', 'liveblog-key-limit' => '', 'filter' => 'db'), $wp_error = FALSE) /srv/www/site/wp-includes/post.php:3482
[05-Oct-2016 21:28:18 UTC] PHP   5. do_action($tag = 'save_post', $arg = 1373690, class WP_Post { public $ID = 1373690; public $post_author = '1'; public $post_date = '2016-10-05 22:28:16'; public $post_date_gmt = '2016-10-05 21:28:16'; public $post_content = ''; public $post_title = 'New Post'; public $post_excerpt = ''; public $post_status = 'publish'; public $comment_status = 'open'; public $ping_status = 'closed'; public $post_password = ''; public $post_name = 'new-post'; public $to_ping = ''; public $pinged = ''; public $post_modified = '2016-10-05 22:28:16'; public $post_modified_gmt = '2016-10-05 21:28:16'; public $post_content_filtered = ''; public $post_parent = 0; public $guid = 'https://site.dev/?p=1373690'; public $menu_order = 0; public $post_type = 'post'; public $post_mime_type = ''; public $comment_count = '0'; public $filter = 'raw' }, TRUE) /srv/www/site/wp-includes/post.php:3409
[05-Oct-2016 21:28:18 UTC] PHP   6. Aggregate->save_post($orig_post_id = 1373690, $orig_post = class WP_Post { public $ID = 1373690; public $post_author = '1'; public $post_date = '2016-10-05 22:28:16'; public $post_date_gmt = '2016-10-05 21:28:16'; public $post_content = ''; public $post_title = 'New Post'; public $post_excerpt = ''; public $post_status = 'publish'; public $comment_status = 'open'; public $ping_status = 'closed'; public $post_password = ''; public $post_name = 'new-post'; public $to_ping = ''; public $pinged = ''; public $post_modified = '2016-10-05 22:28:16'; public $post_modified_gmt = '2016-10-05 21:28:16'; public $post_content_filtered = ''; public $post_parent = 0; public $guid = 'https://site.dev/?p=1373690'; public $menu_order = 0; public $post_type = 'post'; public $post_mime_type = ''; public $comment_count = '0'; public $filter = 'raw' }) /srv/www/site/wp-includes/plugin.php:524
[05-Oct-2016 21:28:18 UTC] PHP   7. Aggregate->push_post_data_to_blogs($orig_post_id = 1373690, $orig_post = class WP_Post { public $ID = 1373690; public $post_author = '1'; public $post_date = '2016-10-05 22:28:16'; public $post_date_gmt = '2016-10-05 21:28:16'; public $post_content = ''; public $post_title = 'New Post'; public $post_excerpt = ''; public $post_status = 'publish'; public $comment_status = 'open'; public $ping_status = 'closed'; public $post_password = ''; public $post_name = 'new-post'; public $to_ping = ''; public $pinged = ''; public $post_modified = '2016-10-05 22:28:16'; public $post_modified_gmt = '2016-10-05 21:28:16'; public $post_content_filtered = ''; public $post_parent = 0; public $guid = 'https://site.dev/?p=1373690'; public $menu_order = 0; public $post_type = 'post'; public $post_mime_type = ''; public $comment_count = '0'; public $filter = 'raw' }) /srv/www/site/wp-content/plugins/aggregator-fork/class-aggregate.php:960
[05-Oct-2016 21:28:18 UTC] PHP   8. Aggregate->prepare_terms($post_id = 1373690, $post = class WP_Post { public $ID = 1373690; public $post_author = '1'; public $post_date = '2016-10-05 22:28:16'; public $post_date_gmt = '2016-10-05 21:28:16'; public $post_content = ''; public $post_title = 'New Post'; public $post_excerpt = ''; public $post_status = 'publish'; public $comment_status = 'open'; public $ping_status = 'closed'; public $post_password = ''; public $post_name = 'new-post'; public $to_ping = ''; public $pinged = ''; public $post_modified = '2016-10-05 22:28:16'; public $post_modified_gmt = '2016-10-05 21:28:16'; public $post_content_filtered = ''; public $post_parent = 0; public $guid = 'https://site.dev/?p=1373690'; public $menu_order = 0; public $post_type = 'post'; public $post_mime_type = ''; public $comment_count = '0'; public $filter = 'raw' })

Taxonomies:
Taxonomy A
Taxonomy B

Post Types:
Default posts

Sites:
Main Site (master on all jobs, main site)
Site B

Jobs:
Main Site post -> Site B post

To Reproduce:

• Create sites, taxonomy, and jobs per the above
• Add a post to Main Site with a term in Taxonomy A assigned, but not a term in Taxonomy B
• Watch the error log for the above error

What's Happening
Prepare terms is pulling terms assigned to a post using get_the_terms() on each taxonomy available to the post, but if no term has been assigned to said post in the taxonomy get_the_terms() will return false (or WP_Error on further issue), but Aggregate->prepare_terms() chugs on regardless and attempts to loop through the value of false or WP_Error

See class-aggregate.php starting at line 601 to see what I mean:
https://github.com/Automattic/aggregator/blob/master/class-aggregate.php#L600

"Detach" feature for breaking the link between synced posts

A network may want to allow for posts that have been synced to be edited by users on the portal site.

Currently this is not possible, because Aggregator assumes that any changes on the source must be synced to the portal.

Breaking the link should be simple in programattic terms. On the portal site, the post will have two pieces of meta - _aggregator_orig_post_id and _aggregator_orig_blog_id. Deleting these two pieces of meta will detach the post from the source.

The best way to do that is probably to provide a post action link on the portal site labelled "Detach" that simply triggers the post meta deletion.

Questions on functionality

Hello, I just have a few questions around functionality:

1. I have 7 sites in my network. I would like each site to have the ability to post to any other site in the network. So a post can be synced to 1, 2, 3, or all 7 sites if the appropriate categories are checked. Is this possible?

2. When a post is synced to another site, clicking on the article takes you back to the original site the article was posted on. Is it possible that the user remains on the site the article was pushed to?

3. When a post is synced to another site it gets set as a pending. Can the post be published automatically?

Wrong permalinks on detached posts

After detaching a post (since v1.2) the permalink still points to the parent site.

It looks like this is because we also need to remove the _aggregator_permalink meta when detaching.

Media files not Pulled from Origin Site

When copying over a post that has media in the post, a thumbnail attached, or has meta with an attachment ID the original image does not come over with the post. This means that crops cannot be changed and the media is permanently attached to the origin site and could go away if deleted on the origin site - even if it's still used on the cloned-to site.

Sideloading the file into the cloned-to site (preferrably with an action or filter to allow us to specify files used in metadata) would be very useful.

Main Multisite Site Not Available

When enabling this service, it would appear as the "Portal Site" cannot be the main site in a multisite setup. I can select any other site, but not the main site.

In our use case, we would like to aggregate all posts from every sub-site to the main site in the network. We are using this as an intranet in which every internal department as their own sub-site. Aggregating blogs entries to the main site is essential.

wp_get_sites() is deprecated since version 4.6.0!

We need to switch to using get_sites() in v4.6 and up (I presume)

Gutenberg Support

Awesome plugin @philipjohn, just opening a ticket to log issue with Gutenberg.

Gonna do some digging into it but it looks like we'll need to add a few other hooks for it to fire and support publish events there.

Better naming for Jobs

On the Jobs list, the names are all the same/similar and are unhelpful in showing what each job does.

We should fix this in two ways;

• Include the relationship in the name for existing jobs
• Allow new jobs to have a title.

Replace uses of $_REQUEST where used

We should be using $_GET or $_POST instead of $_REQUEST. Find instances and replace.

Bulk detach posts

It'd be nice to be able to bulk detach posts on a portal site.

Simplify the network admin interface

A hangover of an early alpha version is that there are two screens in the network admin. This is silly, and should be fixed.

Post Meta not Updating

Post meta is not currently updating due to an incorrect post ID assignment when a target post already exists. Starting on line 866, you'll see that the $target_post_id is being overridden instead of being assigned to the old post data being carried through.

This means that when updating a post (new posts work fine) the child post never receives the updated fields.

These variables should be switched.

if ( false !== $portal_target_post_id ) {
    $target_post_id = $orig_post_data['ID'];
    wp_update_post( $orig_post_data );
}

Allow posts to retain original author

At present, the plugin doesn't check who authored a post. Every synced post is simply attached to one author.

This is a lazy way of getting around the problem that we need authors to be added to both the source and target sites.

Fatal error when attempting to run Behat tests

When running bin/behat, the following fatal error occurs:

PHP Fatal error: Call to a member function getSession() on a non-object in /home/philipjohn/vagrants/cftp/www/wordpress-default/wp-content/plugins/aggregator/vendor/behat/mink-extension/src/Behat/MinkExtension/Context/RawMinkContext.php on line 103

See aggregator.feature

Use select2 for selecting sites when adding jobs

On the "Add New Aggregation Job" screen we use a standard drop down list of sites.

This isn't going to cut it in large networks, so we should replace that with select2 to allow for a short, searchable drop down list instead.

VIP compat

The plugin needs a full VIP review to make sure it's compatible with living on VIP.

Adding More Data to aggregator_post_status Filter

We have a need to push different post statuses contextually dependent upon the original post. I'd like to pass through the entire post data and possibly the original blog ID to be able to find out this information and send a different post status through depending on certain conditions.

Current Filter
apply_filters( 'aggregator_post_status', $orig_post_data['post_status'] );

However, adding all this extra data feels a little gross, especially when we're already passing through the status. I don't want to change the data passed through now because backwards compatibility but essentially doubling up on post data is also not preferential. #53 could potentially cover our needs but also feels heavy and we would need to update the post again that this filter could have just set.

Any thoughts on changes to this filter @philipjohn?

For reference, this is the logic we would want to implement:
We have Categories A and B on posts that could be pushed. If a post assigned to Category A is pushed it should be published right away on the portal (child) site. If a post assigned to Category B is pushed it should be set to draft.

Stop using switch_to_blog()

For large/high-traffic sites, switch_to_blog() just isn't going to cut the mustard. We need Another Way.

My immediate thought is to use custom WP-API endpoints, but that relies on WP-API being adopted into core which will happen.... one day.... far in the future.... maybe.

Set post as publish instead of pending

When a post is synced to another site it gets set as a pending. I want the post to be automatically set publish instead of pending. How can I do this?

PHP notice due to Aggregator_Jobs_List_Table being called on AJAX request

Aggregator->admin_init() is getting called on AJAX actions which is causing an error in the Aggregator_Jobs_List_Table where screen isn't being set (since it shouldn't be loaded in AJAX and has no screen on a heartbeat call for example) which is throwing a PHP notice.

Notice and Trace

[05-Oct-2016 21:16:51 UTC] PHP Notice:  Undefined index: hook_suffix in /srv/www/site/wp-admin/includes/class-wp-screen.php on line 229
[05-Oct-2016 21:16:51 UTC] PHP Stack trace:
[05-Oct-2016 21:16:51 UTC] PHP   1. {main}() /srv/www/site/wp-admin/admin-ajax.php:0
[05-Oct-2016 21:16:51 UTC] PHP   2. do_action($tag = 'admin_init', $arg = *uninitialized*) /srv/www/site/wp-admin/admin-ajax.php:44
[05-Oct-2016 21:16:51 UTC] PHP   3. Aggregator->admin_init('') /srv/www/site/wp-includes/plugin.php:524
[05-Oct-2016 21:16:51 UTC] PHP   4. Aggregator_Jobs_List_Table->__construct() /srv/www/site/wp-content/plugins/aggregator-fork/class-aggregator.php:77
[05-Oct-2016 21:16:51 UTC] PHP   5. WP_List_Table->__construct($args = array ('plural' => 'wp_list_aggregator_jobs', 'singular' => 'wp_list_aggregator_job', 'ajax' => FALSE, 'screen' => NULL)) /srv/www/site/wp-content/plugins/aggregator-fork/class-aggregator-jobs-list-table.php:20
[05-Oct-2016 21:16:51 UTC] PHP   6. convert_to_screen($hook_name = NULL) /srv/www/site/wp-admin/includes/class-wp-list-table.php:143
[05-Oct-2016 21:16:51 UTC] PHP   7. WP_Screen::get($hook_name = NULL) /srv/www/site/wp-admin/includes/template.php:2003

Think of a better name.

"Aggregator" is kind of boring and doesn't explain well what the plugin does.

We should come up with something better.

Found issues

hi @philipjohn,

As discussed during our call the other day.

I have attached all the issues the we are facing. It seems to create the content however, it is throwing Wordpress errors as well.

Please do let me know if you need me to update the code or send you a snippet.

Thanks,

Move classes into a `classes` directory

The file structure is ugly :(

We should move all the class-* files into a classes/ directory instead.

Attachment meta data not syncing

Currently, the attachment post type is not synced. When an attachment needs to be pushed a new attachment is created on the portal site, and attached to the pushed post that needs it.

There are two ways to make sure all attachment meta data is pushed across...

1. Push the attachment post type

To do this we'd need to find a way to check if we actually need to push the attachment to the portal (i.e. is it on a pushed post/being pushed?) otherwise each portal will end up with every attachment from every source site pushed to it, which is unnecessary and soaks up disk space.

2. Copy attachment meta data

This way, we still create attachments only when needed but add the meta data. However, we'd then need a task to update that meta data if the source attachment changes. The easy solution to that is to sync the attachment post type, but that has it's own issues.

Needs more thought

Use "destinations" rather than "portals"

The plugin currently uses the following terminology:

• source = the site content comes from
• portal = the site content goes to

I can't remember why we used "portal" but it doesn't make much sense anymore. I propose:

• source = the site content comes from
• destination = the site content goes to

The plan is to make this change in v1.3. Function and hook names will change! We can add backwards-compatibility though.

Behat tests

We need to have good Behat tests for all the things

"Do you want to leave this site" when creating Jobs

When creating a new Job, clicking Save prompts WordPress to pop up the message "Do you want to leave this site?".

This suggests WP thinks there are unsaved changes that will be lost by clicking Save.

Debugging shows this bug has been introduced by #36, but it's not clear yet why.

Multiple jobs can be created

At the moment, the plugin doesn't support multiple jobs for the same portal/source combination. However, it is possible to create new ones and that leads to unexpected behaviour including;

• Deleting a job appears to not delete a job, because after deletion another job is shown which appears identical
• Posts are duplicated

Refactor uses of `extract()`

We shouldn't be using extract() but it's used at least twice in the plugin. We need to fix that.

_orig_terms kept on detached post

Not sure if this is intentional or not, but thought I'd bring it up - when a post is detached, the _aggregator_orig_post_id and _aggregator_orig_blog_id meta keys are deleted, but _orig_terms is left behind. I would have expected that this value would be deleted alongside the others.

Rename Aggregator

There is already a plugin on WordPress.org called Aggregator, and "aggregator" doesn't really say very well what this plugin does. We need a better name. Suggestions welcome!

500 Error when specifying terms in a taxonomy

I've identified an error in the way that terms are run when using multiple jobs based off of terms in a single taxonomy.

I'm going to explain and make reproduction steps as best I can but do feel free to ask me questions.

Error

Catchable fatal error: Argument 1 passed to Aggregate::allowed_taxonomies() must be of the type array, object given, called in /srv/www/thesun/wp-content/plugins/aggregator-fork/class-aggregate.php on line 801 and defined in /srv/www/thesun/wp-content/plugins/aggregator-fork/class-aggregate.php on line 140

Taxonomies:
Consumer Sites - one term for Site B, one term for Site C

Post Types:
Default posts

Sites:

• Main Site (master on all jobs, main site)
• Site B
• Site C

Jobs:

• Main Site post -> Site B when Consumer Sites set to Site B on post
• Main Site post -> Site C when Consumer Sites set to Site C on post

To Reproduce:

1. Create sites, taxonomy, and jobs per the above (set up the jobs in the above order)
2. Add a new post with Consumer Sites set to Site B - article should correctly populate to the site with no errors (using as a comparison and proof of identified issue)
3. Add a new post with Consumer Sites set to Site C - page will 500 with the above error

What's Happening:
There is a cross in variables causing the variable $orig_terms that's defined outside the foreach loop to be constantly overridden as you loop through jobs in push_post_data_to_blogs(). As you loop through jobs $orig_terms is constantly modified by the code inside the foreach loop causing the next iteration to use the last iteration's modified and incorrect values. In this case if a job is bad on the first loop, but good on the second loop, the second loop attempts to push a WP_Error into prepare_terms.

Look at this line through ~805 to see what I'm talking about:
https://github.com/Automattic/aggregator/blob/master/class-aggregate.php#L776

Actions links missing on source

On the source site's Posts page (admin) the action links (edit | delete etc) are missing.