Ticket #2864 (closed enhancement: invalid)

Opened 2 years ago

Last modified 2 years ago

Magpie falling behind feed developments

Reported by: skeltoac Assigned to: link92
Priority: normal Milestone:
Component: Optimization Version: 2.0.3
Severity: normal Keywords: magpie rss atom rdf xml bg|has-patch bg|needs-testing
Cc:

Description

MagpieRSS doesn't know how to handle new elements in XML. It doesn't understand Atom 1.0 or RSS 2.0 very well. It is lossy. It doesn't use wp_object_cache. It needs an update.

Rather than tweak Magpie to handle each new thing that comes along, I've put together an XML parser that handles Atom, RSS and RDF feeds and provides an way to add support for new kinds of XML elements. It has default logic to handle any XML data in a feed and can be extended via plugins to add logic for handling special elements. It uses wp_object_cache whenever possible; otherwise it falls back on the database.

fetch_rss is emulated very well and much extra data is available in the array returned but some elements, such as Atom elements with both attributes and inner text, may lose some of their meaning. It may be best to start using a lossless format or tweak the array so that attributes are always preserved.

This code is in use on wordpress.com right now. It needs scrutiny and testing on other platforms as well.

Attachments

rss.php (32.9 kB) - added by skeltoac on 06/27/06 00:31:59.
rss.2.php (1.5 kB) - added by link92 on 10/09/06 19:59:50.
SimplePie?-clad wp-includes/rss.php
simplepie.patch (3.6 kB) - added by link92 on 10/09/06 20:00:28.
rss.3.php (1.5 kB) - added by link92 on 10/10/06 19:38:37.
Fix breach of WP coding standards in rss.2.php

Change History

06/26/06 23:30:35 changed by skeltoac

New rss.php uploaded.

Incutio's HTTP client need not be used, but it does an efficient job and allows us to handle HTTP codes >200 ourselves. For instance, we can cache redirects. Proposals welcome.

06/27/06 00:31:59 changed by skeltoac

  • attachment rss.php added.

06/27/06 01:44:33 changed by filosofo

In magpie, the object member 'summary' held the value of whatever was between 'description' or 'content' tags. How about including that feature in your parser, for backwards-compatibility?

07/24/06 11:33:07 changed by link92

Just quickly skimming through the code, I think I can see some pretty major shortcomings within the parser. Atom 1.0 has got a "type" attribute(and Atom 0.3 has "mode"), which must be used to format the data correctly (eg. decoding entities).

I personally am one of the two developers of SimplePie, another PHP feed parser, and I may well try and hack it into WP sometime before term starts (22nd Aug). Catch me on IRC sometime (I'm gsnedders, this is just a nick I used a while ago).

10/09/06 19:59:50 changed by link92

  • attachment rss.2.php added.

SimplePie?-clad wp-includes/rss.php

10/09/06 20:00:28 changed by link92

  • attachment simplepie.patch added.

10/09/06 20:07:07 changed by link92

I've attached two files: a new rss.php, and simplepie.patch (which patches wp-admin/index.php, due to the completely different syntax). It assumes there's simplepie.inc in wp-includes, and that needs to be a recent SVN copy, due to it using some features that aren't in the latest release. It's very stable at the moment (it's actually massively less buggy than the latest "stable" release), and probably won't break for quite a while. So, till SP 1.0 Beta 3 is out, just grab the SVN copy.

One of the things about this, and should be obvious from the fact I'm not including simplepie.inc here, is that we now don't need any modifications to the feed parser itself, undoubtedly easier from a maintenance point of view.

10/09/06 20:09:47 changed by link92

  • keywords changed from magpie rss atom rdf xml to magpie rss atom rdf xml bg|has-patch bg|needs-testing.

10/10/06 19:38:37 changed by link92

  • attachment rss.3.php added.

Fix breach of WP coding standards in rss.2.php

10/11/06 21:29:43 changed by link92

  • owner changed from skeltoac to link92.
  • status changed from new to assigned.

10/11/06 23:03:20 changed by markjaquith

  • status changed from assigned to closed.
  • resolution set to fixed.

(In [4378]) The changing of the pies. Now using SimplePie?. props to gsnedders. fixes #2864

10/15/06 01:59:31 changed by matt

  • status changed from closed to reopened.
  • resolution deleted.

This is entirely unacceptable.

We just broke everything that uses our RSS libraries in the world. If simplepie can't maintain the same external functions, like fetch_rss and the old object structure, then we shouldn't use it. Nothing in WP should have changed except the library.

Also, the dashboard is now loading entire posts rather than descriptions.

10/16/06 04:41:10 changed by markjaquith

Have reverted back, for now.

Will consider patches with full backwards compat.

11/30/06 01:13:05 changed by matt

  • milestone changed from 2.1 to 2.2.

11/30/06 01:13:18 changed by matt

  • status changed from reopened to closed.
  • resolution set to invalid.

Closing awaiting new patches.

01/08/07 21:55:50 changed by foolswisdom

  • milestone deleted.