Ticket #4421 (closed defect: fixed)

Opened 1 year ago

Last modified 9 months ago

MT and WP Importers memory usage 2.3 fixes

Reported by: ryan Assigned to: ryan
Priority: normal Milestone: 2.3
Component: Administration Version: 2.2
Severity: normal Keywords: import importer memory performance
Cc:

Description

Most of the importers slurp entire files into memory and store the results of parsing into memory, making for a big footprint. They also use lots of unnecessary regular expressions. Importers should parse files one line at a time and write posts to the DB as they are processed to help keep us within memory limits.

Attachments

mt.diff (16.4 kB) - added by ryan on 06/08/07 07:01:47.
Reworked MT importer
import-memory-reduction-r6606.patch (2.1 kB) - added by tellyworth on 07/09/07 23:40:35.

Change History

06/08/07 07:01:47 changed by ryan

  • attachment mt.diff added.

Reworked MT importer

06/12/07 19:49:30 changed by ryan

(In [5684]) MT importer rework. see #4421

07/09/07 23:40:35 changed by tellyworth

  • attachment import-memory-reduction-r6606.patch added.

07/09/07 23:42:14 changed by tellyworth

import-memory-reduction-r6606.patch reduces memory usage in the WP importer by avoiding the use of file(). It still doesn't process entries one by one, but it's a start.

07/15/07 17:55:12 changed by ryan

(In [5802]) Don't slurp in entire file. Props tellyworth. see #4421

08/25/07 20:13:56 changed by foolswisdom

  • status changed from new to closed.
  • resolution set to fixed.
  • summary changed from Importers are memory hungry to MT and WP Importers memory performance 2.3 fixes.

08/25/07 20:17:49 changed by foolswisdom

  • summary changed from MT and WP Importers memory performance 2.3 fixes to MT and WP Importers memory usage 2.3 fixes.