Make WordPress Core

Opened 16 years ago

Closed 16 years ago

Last modified 15 years ago

#5560 closed enhancement (fixed)

Automatic Upgrade

Reported by: keithdsouza's profile keithdsouza Owned by: ryan's profile ryan
Milestone: 2.7 Priority: normal
Severity: normal Version: 2.5
Component: General Keywords: blessed
Focuses: Cc:

Description (last modified by ryan)

Automatically upgrade WordPress to the latest version.

Attachments (3)

5560.diff (6.1 KB) - added by ryan 16 years ago.
wp_upgrade_core()
5560.2.diff (9.3 KB) - added by ryan 16 years ago.
update-core.php and maintenance mode
5560.3.diff (491 bytes) - added by DD32 16 years ago.

Download all attachments as: .zip

Change History (67)

#1 follow-up: @ryan
16 years ago

Let's go for it.

#2 @ryan
16 years ago

To do:

  • Move files into wp-admin, wp-admin/includes, wp-admin/js, etc. as appropriate
  • Change prefix from wpau_ to wp_au or wp_
  • Adopt the WP coding style, underscores instead of CamelCase in function and method names, for example
  • Use latest 2.4 API such as activate_plugin() and deactivate_plugins()
  • Make generally useful wpau methods into top-level functions. Activate and deactivate all come to mind
  • Call wp_upgrade directly from automatic upgrade rather than sending user to upgrade.php
  • Use the jquery bundled with WP

#3 in reply to: ↑ 1 @filosofo
16 years ago

Replying to ryan:

Let's go for it.


Making this plugin core seems like a mistake to me.

  • It goes against WordPress's philosophy of having a light, extensible core. The current plugin's files would increase WP core code size about 13% to solve something that isn't a problem for a majority of users. Those who need this feature should use the plugin.
  • It has the potential to introduce lots of security headaches. I'm no security expert, but it seems to me that posting unencrypted ftp info, making lots of files world-writable, downloading and installing remote files, and saving zipped databases on the server are some ways to ask for trouble.
  • It will encourage people who shouldn't be upgrading to upgrade. Those with admin rights but little technical knowledge (who are the main beneficiaries of something like this) could end up making a big mess of things when essential plugins are incompatible with the latest WP version.

#4 @ryan
16 years ago

Security will definitely require some care and caution. One of the main reasons for doing this is so that blogs are updated when security releases come out. Such a good intention would be worth little if we introduce lots of new security bugs.

The extra code doesn't overly worry me since it is not loaded in the critical path. This won't affect front page loads or even current admin pages. That said, we should keep it as light as possible.

I'd like to couple this with code that checks the active theme and plugins for known incompatibilities. That'll help prevent messes. The fact that the plugin backs up files and tables helps recover from messes. This seems like a big help when it comes to offering support for upgraders.

This is something that's been in consideration for awhile. I think developing a tightly integrated automatic upgrade is a worthwhile pursuit.

#5 @westi
16 years ago

  • Milestone changed from 2.4 to 2.5

While I agree that including something of this ilk in the core is a good idea for the large majority of the user-base that want an easy upgrade path I think that this is a large feature with major security implications and should wait for 2.5 seeing as we are now in the last 30 days before release.

We need to ensure that we have an automatic upgrade capability that is:

  • easy to use
  • secure
  • easy to disable - for the paranoid people
  • easy to maintain

#6 @johnbillion
16 years ago

  • Summary changed from Integration of WordPress Automatic Plugin Into Core WordPress Code to Integration of WordPress Automatic Upgrade Plugin Into Core WordPress Code

#7 @matt
16 years ago

Fundamentally, I think the reason this should be core is that WordPresss being use as, and has the responsibilities of, a platform. Therefore it's useful to look at the evolution of a few of the other most successful platforms out there, I would consider the top 4 to be Windows, Mac OS X, Flash, and Firefox. Each has a built-in update mechanism that is essential to the security of its users and gives it, to varying degrees of success, extremely good upgrade rates. (I think Firefox is our closest analogue.)

We are all familiar with the high-profile WP blogs running old versions of the software, and what happens to them. If a blog is hacked or compromised people don't care if it's a 2-year-old version of WP, it reflects badly on us and permanently damages our reputation in their eyes. IMO it's the biggest threat to WP today.

My biggest regret is that this issue has festered for so long, I think partially because I switched to SVN-managing all my blogs a long time ago and forgot what a pain it was to upgrade/install, even as streamlined as we've made the process.

Like anything, I expect this to be a feature iterated on and improved in future version of WP after the one it ships in, for example it might be cool to have it utilize native SVN commands if available, and a file MD5 check UI and diff viewer.

We've hooked a huge number of people on the power of WordPress because it's so easy to get started with, let's keep it working for them and not shooting them in the foot.

#8 @ryan
16 years ago

  • Summary changed from Integration of WordPress Automatic Upgrade Plugin Into Core WordPress Code to Integrate Automatic Upgrade

#9 follow-up: @ryan
16 years ago

To survey what else is available, there is also this plugin:

http://www.zirona.com/software/wordpress-instant-upgrade/

#10 follow-ups: @filosofo
16 years ago

Are there any other open-source PHP apps that are doing this? It might be helpful to see what they've done.

#11 @darkdragon
16 years ago

  • Milestone changed from 2.6 to 2.5

Having looked at a few of the plugins I disagree with the security problems they can cause. I also think that Subversion should be supported and recommended. That doesn't mean that the user should manually do it.

I'll be working on this since my curiosity has peaked again on this topic. I would also like to not have to log into shell every time to update my checkout of WordPress.

Given my past attention span with this topic, it is unlikely I'll be offering a complete solution. However, whatever I submit will most likely NOT support PHP 4.2 and will be only PHP 4.3 and higher. I'll most likely be wrapping my other ticket for HTTP requests into my patch and will have dependency on that ticket.

I'm not sure my patch won't suck, but I'm sure going to try.

#12 in reply to: ↑ 10 @darkdragon
16 years ago

Replying to filosofo:

Are there any other open-source PHP apps that are doing this? It might be helpful to see what they've done.


See Gallery 2. Well, it has automatic Plugin download install/update built in to the Administration.

However, the upgrader is difficult, since it would have to exist outside the main WordPress execution path.

#13 @darkdragon
16 years ago

  • Milestone changed from 2.5 to 2.6

Oops, didn't mean to change the milestone.

#14 in reply to: ↑ 10 @fitztrev
16 years ago

Replying to filosofo:

Are there any other open-source PHP apps that are doing this? It might be helpful to see what they've done.

Magento just announced one-click upgrades.

#15 @darkdragon
16 years ago

Okay, I think doing this in stages would be a good idea.

The first stage, since it will be many times easier, would be for supporting Subversion checkouts of the WordPress repository. I would absolutely love to just press a button and have everything update instead of having to log into shell and manually "svn up" that way.

While I was hoping to build an installer, I think that can wait until later. I'm uncertain, but I'll check into a plugin repository listings that goes along with the svn upgrader that allows for checking out plugins (if possible).

Will post patch sometime (in case I'm unable to post patch, I would very much like solution to support "svn up").

#16 @ryan
16 years ago

see #5586 for plugin and theme update discussion

#17 @ryan
16 years ago

Ideally we'd have different backends that can be used. DD32's plugin updater has some nicely organized filesystem backends.

http://wordpress-soc-2007.googlecode.com/svn/trunk/dd32.crazyman/trunk/includes/

svn support is well and good, but most people are going to be using ftp. If you have svn and a shell you can just write a one-liner that calls svn up over ssh and be done. :-)

#18 @darkdragon
16 years ago

I'm moving towards pluggable functions, since this would allow for my svn system (I would want to use the WordPress Plugin Respository to checkout and update the plugins). So basically, it could probably forward.

#4779 has working, unit tested code that fully supports many file system methods and is extensible. So I haven't received any feedback on that ticket on what improvements need to be made. It functionality into the core would allow for downloading a zip file.

However there are only a few methods to unzip on PHP systems.

  1. If *nix, using the system function (if not disabled) to unzip the file.
  2. Two zip extensions are available to test for. The Zip extension does quite a good job and is very easy. I would recommend using that.
  3. Some PHP class that combines all three above methods.

The reason I would advocate SVN is that even through, many users won't use it, building an installer for SVN can come later and be a plugin or the user can checkout the repository by shell and manage WordPress administration from there. Many testers also use Subversion, so haven't a one click upgrader that no one else can find and run would be useful.

It is also to be complete. The only other method besides FTP/HTTP and Subversion is PEAR. I also think Subversion upgrader would be both the easiest and more secure version (the upgrader can automatically add .htaccess rules to prevent looking into the .svn folders for the user).

#19 @ryan
16 years ago

Even a subversion updater would need FTP or direct fs access to manipulate non-svn files such as the backups saved by the Automatic Upgrade plugin or the htaccess files you mention. DD32's filesystem backends look promising for that.

#20 @RanYanivHartstein
16 years ago

  • Resolution set to fixed
  • Status changed from new to closed

Don't forget the users of localized versions.

The auto-update mechanism should also update localized installations, possibly using the wp-i18n repository, or alternatively using a custom repository to be defined manually by the localization group.

Potentially, the auto-update mechanism should only notify of new versions if the localized version has already been updated as well.

#21 @RanYanivHartstein
16 years ago

  • Resolution fixed deleted
  • Status changed from closed to reopened

sorry, closed accidentally

#22 @lelion
16 years ago

  • Version 2.5 deleted

The idea of an automatic upgrade option in WordPress looks very promising!

Currently, I make my WordPress upgrades using a 'mixed' system of my own - I make a backup of all my current WordPress files using SSH (the 'cp -rf' command, simply copy all files in a folder *outside* of the web-accessible part of the site), also a backup of the database using the excellent wp-db-backup plugin, then I download the latest WordPress version (wget, tar), delete '/wp-content/' in it (so I don't over-write by mistake any custom theme of mine or any plugins), after that I simply delete all current WP files and folders, *except* /wp-content/, wp-config.php, .htaccess, then I copy the new files and folders to my current WP install dir, open upgrade.php in my browser, all done:)

Doesn't take more than a couple of minutes, and I also have complete control and understanding of what's happening at the moment of the upgrade.

In case something goes wrong, I can restore my previous WP install in just a second, using my backups: copy of the WP files & folders and MySQL database snapshot, made right before the upgrade...

Would be great, if the Automatic Upgrade option, which is planned for WP 2.6+, will work in a similar, transparent and user-friendly way... or at least, for the power user it'll work this way... :-)

Cheers!

#23 @lelion
16 years ago

  • Version set to 2.5

PS Sorry, accidentally delete 'version', restoring back to 2.5...

#24 in reply to: ↑ 9 @mastermind
16 years ago

Replying to ryan:

To survey what else is available, there is also this plugin:

http://www.zirona.com/software/wordpress-instant-upgrade/

Just FYI, Instant Upgrade is available in a new version, now also with FTP support and many other features. (A first alpha with FTP support had already been released in May 2007, but this beta is really mature. I think it will be released "final" soon.)

Please also consider this plugin, when you think about inclusion of an automatic upgrading solution into the WP core.

Regardless of which one will be taken; I also (like filisofo) have concerns about taking such a complex piece of software into the WP core. I've written about this topic a while ago (Note that the security-related statements in that entry refer to the old IU version). However, if the core developers want to include one, I wouldn't mind seeing mine being taken. ;)

#25 follow-up: @darkdragon
16 years ago

The current upgrade system is not up to my personal standards, which means it does not work on my system and is throwing errors. It appears that it wants to traverse all directories leading up to the public one. I take issue with that since I do not have permissions to the directories above mine. Not quite true, but in most cases I don't.

#26 in reply to: ↑ 25 @ryan
16 years ago

Replying to darkdragon:

The current upgrade system is not up to my personal standards, which means it does not work on my system and is throwing errors. It appears that it wants to traverse all directories leading up to the public one. I take issue with that since I do not have permissions to the directories above mine. Not quite true, but in most cases I don't.

You mean the stuff for #5586? If so, yes, that needs to be fixed.

#27 follow-up: @frbobscorner
16 years ago

I had problems with my host after installing. The plugin never worked correctly, and I finally deleted it. I have a folder called "wpau-backup" in my root index. Can I delete it without causing serius problems? Does anyone know?

#28 in reply to: ↑ 27 @lloydbudd
16 years ago

Replying to frbobscorner:

I had problems with my host after installing.

Which plugin? Anyway, this isn't the place for plugin support.

#29 @matt
16 years ago

  • Keywords blessed added
  • Owner changed from anonymous to ryan
  • Status changed from reopened to new

#30 @ryan
16 years ago

  • Description modified (diff)
  • Summary changed from Integrate Automatic Upgrade to Automatic Upgrade

@ryan
16 years ago

wp_upgrade_core()

#31 @ryan
16 years ago

Patch adds a very simple-minded first attempt at wp_upgrade_core() that is patterned after wp_upgrade_plugin(). Right now the new WP files are simply copied over the top of existing ones. Old files aren't deleted, yet. It doesn't do backups or put up maintenance messages.

#32 follow-up: @santosj
16 years ago

One suggestion is that you backup the old files first and then copy over.

#33 follow-up: @ryan
16 years ago

I'm thinking wp_update_core() should grab the updated version of WP, copy in wp-admin/includes/update-core.php from the updated version, and then include that and let it take over the upgrade from there. That way it can do things only the new version knows about.

update-core.php can have a master list of files that are no longer part of WP. We could just diff the current installation with the contents of the zip and delete files not in the zip, but we should be conservative with what we delete and only delete files that were once included with WP.

Upgrade could add a special file to the content dir that wp-settings.php checks to see if an upgrade is in progress. If so it throws up a 503 message and stops loading WP. We can serve a custom 503.php or the like from the content directory, if the user has supplied such.

#34 in reply to: ↑ 32 ; follow-up: @ryan
16 years ago

Replying to santosj:

One suggestion is that you backup the old files first and then copy over.

Doing a zip or tar archive would be nice.

#35 in reply to: ↑ 33 @santosj
16 years ago

Replying to ryan:

I'm thinking wp_update_core() should grab the updated version of WP, copy in wp-admin/includes/update-core.php from the updated version, and then include that and let it take over the upgrade from there. That way it can do things only the new version knows about.

update-core.php can have a master list of files that are no longer part of WP. We could just diff the current installation with the contents of the zip and delete files not in the zip, but we should be conservative with what we delete and only delete files that were once included with WP.

I like this. I was thinking about something like this myself, I just didn't know how to abstract it. It would be nice if this file had abstraction, so that you only have to call functions to do tasks in order instead of calling the PHP functions in one long script.

I'll be willing to support something like this, both with testing and development. It is something I also had in mind, so I would like to see it done. Well anyway would be good.

Upgrade could add a special file to the content dir that wp-settings.php checks to see if an upgrade is in progress. If so it throws up a 503 message and stops loading WP. We can serve a custom 503.php or the like from the content directory, if the user has supplied such.

I like using a temp file for this. Something like .upgrading in the WordPress base folder and check for it and bail if it is found. We can use a lot of the current plugin upgrade code. It is a lot easier to check for the existance of a file, than a constant in some cases.

#36 in reply to: ↑ 34 @santosj
16 years ago

Replying to ryan:

Replying to santosj:

One suggestion is that you backup the old files first and then copy over.

Doing a zip or tar archive would be nice.

The folder that this is put in, would have to have permissions changed so that some common folk can't come by and download the entire site (with wp-config.php).

#38 @santosj
16 years ago

So basically:

  1. Check for update, if update display link for automatic upgrading.
  2. Check for locale: if not en, then check locale website for new update. If available then download that version instead of en version of WordPress.
  3. Download latest version.
  4. Copy to upgrader folder in base WordPress folder.
  5. Send User to upgrader script in new folder.
  6. Zip up old installation to upgrader/old, create .htaccess and set permissions to restrict access.
  7. Ask user if they want to write new default theme files, and new akismet files over to the wp-content folders as well.
  8. Run custom upgrader script to copy new files over to old installation, deleting old files based on custom upgrader script.
  9. If successful, then delete all files in upgrader folder, but keep upgrader/old.
  10. Redirect to base/wp-admin/upgrade.php to upgrade database and whatever, if needed.
  11. Ask user if they want to keep the old files or delete them as well. If yes, then delete the upgrader/old, as well as upgrader directories.

@ryan
16 years ago

update-core.php and maintenance mode

#39 @ryan
16 years ago

We already send the locale to api.wordpress.org when checking the version, but api doesn't do anything with it yet. We'd have to put the locale packages of WordPress on the same file serving infrastructure as the regular packages.

The zip can be given a random filename and tucked in a dir with an index.php in place to prevent directory listings.

I think we should always upgrade the two bundled plugins but leave themes as an option.

The current code does the upgrade on the spot without redirecting. That might need to change in the future, but for now it works.

Feedback is ugly and we need to worry about timing out if the script runs too long. When using the direct fs the update is very fast, however.

#40 @ryan
16 years ago

  • Milestone changed from 2.9 to 2.7

#41 @santosj
16 years ago

I'm trying to test it, because I see several areas where there are comments. I would totally love to test this. Can't wait!

#42 @ryan
16 years ago

It's pretty ghetto, so be careful. :-)

I'm hoping to get the nightly builds running again and have upgrade pull the latest nightly during the 2.7 dev cycle.

#43 @DD32
16 years ago

I think we should always upgrade the two bundled plugins but leave themes as an option.

Well, The wp-content folder could be ignored entirely, The Plugin upgrader can be used to upgrade the plugins, Akismet is usually at the latest version on peoples installs now (if they use it) thanks to the Automatic plugin upgrades. Maybe an option for upgrading default themes and plugins should be provided?

Glad to see most of the plugin upgrader's code could be reused, Its a shame its' being duplicated so much though, Once this is a bit smoother, I might look into moving some of the functionality into some functions (ie. checking for filesystem access)

#44 @ryan
16 years ago

Yeah, the plugin upgrader will see Akismet updates before they even upgrade WP. If we end up doing upgrade/install for themes, we could make the default and classic themes svn externals and treat them just like any other theme.

We can reduce duplication as we iterate. This is still very much first draft.

#45 @ryan
16 years ago

Working on updating api.wordpress.org to return the locale specific WP packages. Example, http://ja.wordpress.org/wordpress-2.6-ja.zip would be served up for ja* locales.

#46 @DD32
16 years ago

We can reduce duplication as we iterate. This is still very much first draft.

Yeah, What i meant was later in the dev cycle when all the seperate functionalities are implemented fully and for the most part, working nicely.

#47 @ryan
16 years ago

I updated the version-check api to handle locales and to point to the latest nightly build if you are using a development version.

Development:

http://api.wordpress.org/core/version-check/1.2/?version=2.7-bleeding

Locale example:

http://api.wordpress.org/core/version-check/1.2/?version=2.6&locale=ja

The locale has to have an lc.wordpress.org site and a downloadable 2.6 package.

The 1.2 version-check is experimental and may change, so don't rely on it yet.

#48 @ryan
16 years ago

(In [8595]) Automatic upgrade, first cut. see #5560

#49 @ryan
16 years ago

If you're running 2.7-bleeding, you will now see an update link in the footer that will take you to update.php and perform an automatic upgrade, if possible. It upgrades to the latest nightly build. At the moment, the latest nightly doesn't have update-core.php in it, so things will break. Wait a night before testing.

update.php needs a landing page that offers a choice to automatically upgrade or download and install manually. Currently it launches straight into auto upgrade, which can be surprising.

#50 @ryan
16 years ago

(In [8596]) Remove old files. see #5560

#51 @ryan
16 years ago

The nightly build is now at [8596]. It's ready for some testing, if you're brave or foolish. Needless to say, don't try this on a production blog.

#52 @ryan
16 years ago

If you don't see the "stay updated" link in your footer, you need to delete the update_core option. It will regenerate on the next page load and the link should then appear. You have to be an admin to see the link. Clicking "stay updated" will immediately start an auto upgrade. If you are using FTP, it should prop for credentials first.

#53 @ryan
16 years ago

(In [8597]) Landing page for auto upgrade. see #5560

#54 @ryan
16 years ago

(In [8599]) upload.php is not a retired file. see #5560

#55 @ryan
16 years ago

(In [8759]) Upgrade db with remote post during core update. see #5560

@DD32
16 years ago

#56 @DD32
16 years ago

attachment 5560.3.diff added.

I was getting redirected back to the 'Uprade Automatically | Download nightly build' page upon entering FTP details.. Said patch fixes it.

#57 @ryan
16 years ago

  • Resolution set to fixed
  • Status changed from new to closed

(In [8885]) Fix core upgrade redirect. Props DD32. fixes #5560

#58 @ryan
15 years ago

(In [9085]) Don't die with 503 if WP_INSTALLING. Don't load plugins if WP_INSTALLING. Remove old files and working dir before exiting maintenance mode. see #5560

#59 @ryan
15 years ago

(In [9086]) Remove old files before upgrading DB. see #5560

#60 @ryan
15 years ago

Steps performed during upgrade.

  • Download the package from wordpress.org to the system temp directory
  • Create working dir, usually wp-content/upgrade/core
  • Unzip the package into our working dir
  • Copy wp-admin/includes/update-core.php from the new version unzipped in the working dir into the current WP install
  • Include the new update-core.php
  • Call update_core(), which is defined in update-core.php

update_core() takes over and does the following:

  • Sanity check the new version of WP sitting in the working dir
  • Create a .maintenance file in the WP root dir. wp-settings.php checks this and prevents WP from fully loading if it exists. We don't want WP loading while files are being upgraded.
  • Overwrite old WP files with the new ones
  • Remove files that are no longer part of the WP distro
  • Send a request to ugrade.php to perform a DB upgrade. We do this as a separate request instead of calling the upgrade function directly because we need a fresh request that has all of the new stuff loaded.
  • Delete the working dir
  • Delete .maintenance
  • Done

#61 @ryan
15 years ago

Locales must have a wordpress.org subdomain for update to get the locale-specific package.

http://codex.wordpress.org/International_WordPress_Sites

#62 @ryan
15 years ago

(In [9164]) Use set_time_limit() to buy some time during upgrade. see #5560

#63 @bradt
15 years ago

@ryan How are you addressing the file permissions issue that will be a problem for this installer on the majority of web hosting configurations? I have done quite a bit of research and thinking around automatic upgrades.

#64 @ryan
15 years ago

All file operations are performed as the user that owns the files, not as the web server's user. All files are set to 0644 and all directories are set to 0755. Writable by only the user and readable by everyone else, including the web server.

Note: See TracTickets for help on using tickets.