Make WordPress Core

Opened 18 years ago

Closed 16 years ago

#3141 closed defect (bug) (fixed)

explicit memory check before starting install

Reported by: looksaus's profile looksaus Owned by:
Milestone: 2.5 Priority: normal
Severity: normal Version: 2.1
Component: Administration Keywords: has-patch commit tested
Focuses: Cc:

Description

When installing wordpress on a vanilla 64 bit Debian Sarge LAMP system, I bumped into the default 8M php memory limit. No feedback at all, just the install.php page that refused to load. I had to fiddle with executing it on the command line and seeing it run out of memory to see wat the problem was.

An installer script with a builtin php memory check would be nice. That would save quite a bit of hassle.

Attachments (6)

memory_limit_2_4.diff (478 bytes) - added by ffemtcj 16 years ago.
Changes the memory limit for version 2.4
memory_limit_2_3.diff (478 bytes) - added by ffemtcj 16 years ago.
Changes the memory limit for version 2.3.X
3141.prototype.r6528.patch (3.2 KB) - added by darkdragon 16 years ago.
Prototype of increasing memory limit API
3141.r6601.diff (3.8 KB) - added by darkdragon 16 years ago.
Tested patch against r6601
3141.r6668.diff (4.1 KB) - added by darkdragon 16 years ago.
updated patch which moves a function to pluggable, so that plugins can replace the functionality in the function. Based on r6668
mem_limit.diff (508 bytes) - added by ryan 16 years ago.

Download all attachments as: .zip

Change History (63)

#1 @darkfate
18 years ago

I have my php setup as 8MB memory limit and the installer runs fine.

#2 @jamesgregory
17 years ago

I also ran into the memory limit just by running the install script, and I also run 64 bit Linux - 64 bit computers use more memory to do the same thing.

#3 @jamesgregory
17 years ago

  • Version changed from 2.0.4 to 2.1

#4 @foolswisdom
17 years ago

  • Milestone set to 2.4

#5 @ffemtcj
16 years ago

I don't see an actual memory limit that WP requires or even suggests. If there is one, can someone post a link? If there isn't one, then it all depends on the individual server as to weather or not it will work, so there is no point in doing a memory check if there is no defined requirement.

#6 @darkdragon
16 years ago

The memory limit is set by PHP and not by WordPress. WordPress will happily consume all the available memory.

However, the problem is that, a lot of PHP 4 hosts and installations are stuck at 8 MBs which is just not enough for all that the installation does and I believe even for a large blog would crash.

I often thought that WordPress should first try to attempt to increase the memory dependent on a few factors.

  • ini_set() would have to be enabled, some hosts disable this.
  • Increasing the memory might fail given some other PHP or server configuration.

I've seen other PHP applications attempt this before failing. WordPress would need a new function in functions.php that first attempts to increase the memory limit before failing.

This could make the process more difficult since you would have to first check how much memory is consumed during loops and sections which cause the most memory overhead. However, the workaround would be to just have a DEFINE or option that allows the user to increase the memory limit to that in wp-settings.php to a default of 32MB. It would be far simple solution.

My memory limit is set to >128MB however, so I have no problems.

#7 @ffemtcj
16 years ago

So would the suggestion be then to have the install script initially check the php_memory_limit and see if it is greater than 8MB (or some other number) and if it isn't provide a page with details on how to increase the memory limit if it is not possible to do it automatically, and then give the option to continue with install hoping that it works?

#8 @darkdragon
16 years ago

Well, unless you have access to the php.ini file, which most shared hosters don't (one good thing about Godaddy, however I'm not recommending them) because of the probable security implications of having access to the php.ini file (think godaddy restricts what you can put into the file).

You can also change it if you have mod_php, which for the hosts that have PHP 4 or PHP 5, it would be possible in the .htaccess file.

However, the only way that WordPress has complete control is by using ini_set().

I would try to set the limit to 32MB. The topic is pretty advanced, so users might need either really detailed instructions or someone else to do it. I suppose the first is possible to place on the codex, so no problems there.

#9 @ffemtcj
16 years ago

I know that Dreamhost has access to it as well.. I learned how to do it based upon reading.. Both in the .htaccess and the php.ini.. I'll look into how to possibly do this using the ini_set()and then as a backup for cases where that doesn't work, some sort of instructions in the codex.

#10 @darkdragon
16 years ago

ini_set('memory_limit', '32M');

#11 @darkdragon
16 years ago

I did not know Dreamhost allowed this! That is pretty sweet. Are you sure it isn't from installing PHP yourself?

@ffemtcj
16 years ago

Changes the memory limit for version 2.4

@ffemtcj
16 years ago

Changes the memory limit for version 2.3.X

#12 @ffemtcj
16 years ago

  • Cc ffemtcj added
  • Keywords has-patch needs-testing added
  • Owner changed from anonymous to ffemtcj
  • Status changed from new to assigned

These will hopefully work provided that the server supports it. I'll have to see if there is a way to figure out if it actually does work on each individual setup and if it doesn't give a notification with a link to the codex that explains how to change it manually. If some of the people who had the problem could check this and see if it works for them and leave feedback please.

#13 follow-ups: @darkdragon
16 years ago

Place it in wp-settings.php.

#14 follow-up: @darkdragon
16 years ago

ALso check to see if the memory limit is less than 32MB and then try to set it. I believe WordPress has a function for getting the bytes from like '8M', I forget where I found it.

#15 in reply to: ↑ 13 ; follow-up: @darkfate
16 years ago

Replying to darkdragon:

Place it in wp-settings.php.

Well I'm not completely sure, but that ini_set only works on the current script. I don't know if it works when the file is included like it would be.

#16 in reply to: ↑ 13 ; follow-up: @ffemtcj
16 years ago

Replying to darkdragon:

Place it in wp-settings.php.

I don't see wp-settings.php called for in install.php...

#17 in reply to: ↑ 15 @ffemtcj
16 years ago

Replying to darkfate:

Replying to darkdragon:

Place it in wp-settings.php.

Well I'm not completely sure, but that ini_set only works on the current script. I don't know if it works when the file is included like it would be.

Looking through the WP forums, there is a plugin that is using the same method that I used, and they are saying that there are good results.

#18 in reply to: ↑ 14 @ffemtcj
16 years ago

Replying to darkdragon:

ALso check to see if the memory limit is less than 32MB and then try to set it. I believe WordPress has a function for getting the bytes from like '8M', I forget where I found it.

Are you thinking something more like this:

if ( memory_get_usage() / 1024 < 32 ) {
@ini_set("memory_limit","32M");
}

#19 follow-up: @darkdragon
16 years ago

Replying to both:

  1. Placing it in wp-config-sample.php would work for new installs, but the user would have to manually copy it wp-config.php if they already installed.
  2. wp-config.php is already included, therefore your statement is incorrect. The entire WordPress run is one process, therefore it makes no difference to PHP if the file is included or not.
  3. The code snippet is accurate, because you want to make sure that the need to increase memory is needed and then you can increase it if it is needed. However, you would need to place that code snippet in many different places that have known high memory usage. It would also be better to place it in a function.
  4. If the memory limit is set to >32M then you are actually lowering the memory limit, which is not required. You only want to raise the memory limit, not lower it.

If you need a demonstration, then I can create a patch that does all of this.

#20 in reply to: ↑ 19 ; follow-up: @ffemtcj
16 years ago

Replying to darkdragon:

Replying to both:

  1. Placing it in wp-config-sample.php would work for new installs, but the user would have to manually copy it wp-config.php if they already installed.
  2. wp-config.php is already included, therefore your statement is incorrect. The entire WordPress run is one process, therefore it makes no difference to PHP if the file is included or not.
  3. The code snippet is accurate, because you want to make sure that the need to increase memory is needed and then you can increase it if it is needed. However, you would need to place that code snippet in many different places that have known high memory usage. It would also be better to place it in a function.
  4. If the memory limit is set to >32M then you are actually lowering the memory limit, which is not required. You only want to raise the memory limit, not lower it.

If you need a demonstration, then I can create a patch that does all of this.

Unless I am not understanding something, I believe that the code that I put in here only raises it, not lowering it.. I'll have to wait until tomorrow to work on this more. Since this is one of my first patches, I'd like to be able to figure it out.. Thanks for helping me with it though.

#21 in reply to: ↑ 16 @darkdragon
16 years ago

Replying to ffemtcj:

Replying to darkdragon:

Place it in wp-settings.php.

I don't see wp-settings.php called for in install.php...

install.php most likely calls wp-config.php which does call wp-settings.php.

#22 in reply to: ↑ 20 @darkdragon
16 years ago

Replying to ffemtcj:

Unless I am not understanding something, I believe that the code that I put in here only raises it, not lowering it.. I'll have to wait until tomorrow to work on this more. Since this is one of my first patches, I'd like to be able to figure it out.. Thanks for helping me with it though.

I'm saying that since my memory limit is set to 128MB, when I run your patch, it will lower to 32MB. I rather like my memory limit to stay where it is at.

#23 @darkdragon
16 years ago

Well, if I copied your code over to my wp-config.php

#24 @darkdragon
16 years ago

Well, thanks for taking ownership of this problem. I've forgotten about this problem. It has bite me before, so a good resolution to the problem would be great.

#25 @ffemtcj
16 years ago

I'm not sure I understand... The way that I read it is that if memory_get_usage is less than 32, then raise it to 32.. Which shouldn't do anything if it is more than 32.. Am I missing something?

#26 follow-up: @darkdragon
16 years ago

I was referring to your current patch and not to the code snippet. The code snippet is okay, as long as it is further down the execution path and raises it dynamically to no more than double the current memory limit.

I'm saying that you should always check to the current memory limit to make sure that you never decrease the memory limit.

If you put the code snippet at the beginning, it would again lower my memory limit to 32, since when WordPress first starts, it does not use 32 MB, therefore your code block will run, when it should not on my system.

memory_get_usage() gets the current process usage at that given time it was called. Not what the current limit is. Also, as an aside, you won't be able to use memory get_usage() since you have to support PHP 4.2.0 and memory_get_usage() was added in PHP 4.3.2.

#27 @darkdragon
16 years ago

It appears that I was thinking of size_format() which does the inverse of what I was thinking of. A new function would have to be created, which does the inverse of size_format().

#28 in reply to: ↑ 26 @DD32
16 years ago

Replying to darkdragon:

It appears that I was thinking of size_format() which does the inverse of what I was thinking of. A new function would have to be created, which does the inverse of size_format().

As the memory limit is allways a MB figure, you can simply cast it to int:

echo (int) '32M'; //Prints 32

Replying to darkdragon:

Also, as an aside, you won't be able to use memory get_usage() since you have to support PHP 4.2.0 and memory_get_usage() was added in PHP 4.3.2.

As a complete aside, It hasnt stopped 4.3+ functions elsewhere: #5416

#29 @darkdragon
16 years ago

Cool, DD32. I'm thinking I'm going to submit a patch. Thanks for the info.

@darkdragon
16 years ago

Prototype of increasing memory limit API

#30 @darkdragon
16 years ago

Patch against r6528 and is a prototype of the points I pointed out.

Includes another constant that can be used to set the memory limit in wp-config.php in case 32MB is still too low.

Prototype code needs testing.

#31 @darkdragon
16 years ago

Includes a function which could be used in later WordPress versions and by Plugins to attempt to increase memory limit when needed in the case that the memory is about to be exhausted.

The algorithm is but a prototype, which increases the memory limit by 75% whenever the memory limit is 75% or higher of the current memory limit. If the function to get the current peak memory usage is unavailable the function will return 'null' and will do nothing.

So basically, you can use my patch as a guide for what I was trying to explain or ignore it. It is what I've been thinking would be great to solve this problem, but never got around to doing so. Having the increasing the memory based on usage in WordPress would allow future plugins and future WordPress versions to use it to get rid of those pesky memory has been exhausted errors.

#32 @ffemtcj
16 years ago

That looks good.. I don't know of anything else to do, I think it is above my skill level at this point.

#33 @darkdragon
16 years ago

Can someone look over this patch and implementation to see if the prototype is any good?

@darkdragon
16 years ago

Tested patch against r6601

#34 @darkdragon
16 years ago

  • Keywords tested added; needs-testing removed

I tested the patch against low memory limit and it increased the memory limit to 32MiB.

I've found that PHP systems that don't have --enable-memory-limit while PHP was installed won't benefit from this system. Also, those PHP hosts that disable ini_set() will not benefit.

I believe that it won't be a problem, since the functions make checks to see if increasing the memory is available and do nothing it is isn't. So those systems where either ini_set() is disabled or --enable-memory-limit wasn't used during the PHP install won't be affected.

If someone can confirm, then I think the patch will be a good jump start towards improving the memory limit

#35 @darkdragon
16 years ago

Pretty sure the patch is probably stale, so will create a new one. I think the second function should be pluggable, so I'm going to make it pluggable.

@darkdragon
16 years ago

updated patch which moves a function to pluggable, so that plugins can replace the functionality in the function. Based on r6668

#36 @darkdragon
16 years ago

  • Keywords commit added

Please commit new patch.

#37 @ryan
16 years ago

Shared hosts won't let you touch the memory limit, making the set code worthless.

We should make install.php use less memory, if possible, before messing with all of this. Has anyone profiled memory usage in install.php to see what is sucking up memory? Not being able to install within 8MB is pretty crappy.

See also Drupal's discussion about this:

http://drupal.org/node/197720

#38 @darkdragon
16 years ago

Dreamhost does, so I assume that at least several others will, as well as dedicated. If the increase can not be done, then the functions are backed out of and almost no time is wasted.

Yeah, it will only help a fraction, but a fraction that probably needs it.

#39 @ffemtcj
16 years ago

  • Owner ffemtcj deleted
  • Status changed from assigned to new

#40 @darkdragon
16 years ago

It isn't just install, it is actual running of WordPress that does not fall into 8MB of memory. I believe Godaddy also allows for this.

So I mean, at least two shared hosts, that are popular do this. What disadvantage is there?

Not including it: Everyone that could have benefited now has to go to support.
Including it: Those that can't increase memory, won't, so it is optimized for them and they won't feel the burn. Those that can, will have a minor decrease in performance, but will be able to not need support.

It is the similar method used in openads, so while I'm not exactly sure what their code looks like, I know at least that the algorithm is similar.

#41 @darkdragon
16 years ago

WTF? The ONLY TWO reasons this would not work on any host is if --enable-memory-limit switch was not used during the PHP installation. The second is if ini_set() is in the disabled function list.

There is no other way to disable setting of the memory limit. Some hosts DO in fact disable ini_set() which is a problem with support tickets. However, your reasoning doesn't seem to me to be valid in the sense that either one of us has any statistics on the exact number of hosts.

It will hurt no one by committing this patch, and will help some people if you do. So I mean, do you want to at least help some people that will benefit? You do want to help thost that will benefit don't you?

#42 @ryan
16 years ago

I just want to make sure we're being good citizens on shared hosts.

Anyhow, let's give it a try. I'd like to forego the function additions until we actually use them somewhere. Let's keep it simple and set the limit at the top of wp-settings.php before we load anything. That way we'll have room to breathe from the start. See reduced patch.

@ryan
16 years ago

#43 @darkdragon
16 years ago

Beautiful. Just beautiful.

#44 @ryan
16 years ago

(In [6681]) Increase memory limit. Props darkdragon. see #3141

#45 @ruckus
16 years ago

Why does the code in [6681] check for the memory_get_usage() function to exist? That function is not used.

#46 @ruckus
16 years ago

  • Cc ruckus added

#47 @ryan
16 years ago

It's used to determine if PHP was compiled with --enable-memory-limit.

#48 @ruckus
16 years ago

Thanks for the explanation. That might be a good comment to add to the code. I would qualify that as a non-obvious piece of code. :)

#49 follow-up: @MichaelH
16 years ago

Confirming that after experiencing an "Allowed memory size of 8388608 bytes exhausted" trying to install on a OVH.COM hosted account, the code in changeset [6681] allowed 2.3.2 to be installed successfully.

PHP 5.0.4
Linux atvantage.atvantage.com 2.6.17-1.2142_FC4smp #1 SMP Tue Jul 11 22:57:02 EDT 2006 i686

#50 @darkdragon
16 years ago

I could die today and my life would be complete. Thank you Michael!

Does this mean this should also be applied to the 2.3 branch?

#51 in reply to: ↑ 49 ; follow-up: @bentrem
16 years ago

*I'm new to this bug set; please don't hit me hard*

Replying to MichaelH:

Confirming that after experiencing an "Allowed memory size of 8388608 bytes exhausted" trying to install on a OVH.COM hosted account ... changeset [6681] allowed 2.3.2 to be installed successfully.

Does this respond at all to "Large blogs fatal error in cache" Ticket #4045?

#52 in reply to: ↑ 51 ; follow-up: @darkdragon
16 years ago

Replying to bentrem:

*I'm new to this bug set; please don't hit me hard*

Replying to MichaelH:

Confirming that after experiencing an "Allowed memory size of 8388608 bytes exhausted" trying to install on a OVH.COM hosted account ... changeset [6681] allowed 2.3.2 to be installed successfully.

Does this respond at all to "Large blogs fatal error in cache" Ticket #4045?


Yes, it should fix it. If not, then just plug this code into wp-config.php

define('WP_MEMORY_LIMIT', '64MB');

and if that doesn't work, then your PHP installation doesn't allow for increasing the hard coded PHP Memory Limit.

#53 in reply to: ↑ 52 @bentrem
16 years ago

Replying to darkdragon:

Yes, it should fix it. If not, then just plug this code into wp-config.php


define('WP_MEMORY_LIMIT', '64MB');

and if that doesn't work, then your PHP installation doesn't allow for increasing the hard coded PHP Memory Limit.

Very nice; that's pretty much exhaustive. (That's the snippet I was searching for when I noticed the two tickets!)
Now, out of curiosity: how do the folk involved with #4045 find out about it?

#54 @darkdragon
16 years ago

By waiting for 2.5 to come out in two months...? Or checking out trunk. Or by adding the following code to their wp-settings.php file. I'm writing up some articles for the codex and will be creating several pages for when 2.5 comes out.

#55 @pishmishy
16 years ago

It reads to me like a fix has been found for this problem. Can we close it?

#56 @ruckus
16 years ago

  • Cc ruckus removed

#57 @darkdragon
16 years ago

  • Resolution set to fixed
  • Status changed from new to closed

Fixed in [6681], create a new ticket on any failure.

Note: See TracTickets for help on using tickets.