Make WordPress Core

Opened 15 years ago

Closed 15 years ago

Last modified 15 years ago

#8335 closed defect (bug) (fixed)

[WP 2.7-beta3-9858] Edit pages cuts off special characters

Reported by: anders-dk's profile anders-dk Owned by:
Milestone: 2.7 Priority: high
Severity: major Version: 2.7
Component: General Keywords: reporter-feedback
Focuses: Cc:

Description

EDIT PAGE

When clicking Update Page, WP returns a page with text cut off just before the first special character.

Try pasting this in:
Special characters in Danish: Text example blåbærgrød.
Special characters in German: Text example München.

BROWSERS

All

Attachments (3)

screenshot.png (30.6 KB) - added by vladimir_kolesnikov 15 years ago.
die_on_update.diff (364 bytes) - added by ryan 15 years ago.
Output post_content and die when creating or saving posts.
remove-blank-lines.diff (323 bytes) - added by vladimir_kolesnikov 15 years ago.
Removes blank lines before !DOCTYPE in wp-admin/page-new.php

Download all attachments as: .zip

Change History (51)

#1 @azaozz
15 years ago

  • Keywords reporter-feedback added; special characters removed

Works properly here. If special characters were causing problems, a lot of people would have been affected. Are you using any plugins that could be manipulating the content for editing? Perhaps try with all plugins disabled and the default theme.

#2 @anders-dk
15 years ago

I've tried disabling all plugins and applying the default theme - still no special characters when updating page.

#3 follow-up: @vladimir_kolesnikov
15 years ago

This bug does not seem to exist in WordPress 2.7-beta3-9889, please see the screen shot attached.

#4 in reply to: ↑ 3 ; follow-up: @anders-dk
15 years ago

I've just upgraded to beta3-9889 (new icons!)and the bug still exists - unfortunately. It's still the demo theme with all plugins disabled.

I've tried both updating and publishing a new page.

#5 in reply to: ↑ 4 ; follow-up: @vladimir_kolesnikov
15 years ago

Replying to anders-dk

Andreas, what are DB_CHARSET and DB_COLLATE in wp-config.php, what is "Encoding for pages and feeds" (Admin Panel -> Settings -> Reading) and in what character set wp_posts table is? I suspect that these are charset conversion issues.

#6 in reply to: ↑ 5 ; follow-up: @anders-dk
15 years ago

Replying to vladimir_kolesnikov:

/ Database Charset to use in creating database tables. */
define('DB_CHARSET', 'utf8');

/ The Database Collate type. Don't change this if in doubt. */
define('DB_COLLATE', );

Encoding for pages and feeds: UTF-8

wp_posts: utf8_general_ci

I haven't changed any of these. And there's no problem with special characters in posts - only pages.

#7 in reply to: ↑ 6 ; follow-up: @vladimir_kolesnikov
15 years ago

Replying to anders-dk:
Does that happen in WYSIWYG editor, HTML editor or both?

#8 in reply to: ↑ 7 ; follow-ups: @anders-dk
15 years ago

Replying to vladimir_kolesnikov:

Both.

I've just installed another test site (beta3-9909). Same problem.

#9 in reply to: ↑ 8 @vladimir_kolesnikov
15 years ago

Replying to anders-dk:

Is it possible somehow to have a look at the site, please?

#10 in reply to: ↑ 8 ; follow-up: @azaozz
15 years ago

Replying to anders-dk:

Both.

I've just installed another test site (beta3-9909). Same problem.

Posts and pages are handled almost identically in WordPress, so it's very strange that this would affect pages only. Does it happen on attachment posts too (the "content" of an attachment post is the "Description" when you edit it from the Medial Library)?

#11 in reply to: ↑ 10 ; follow-ups: @anders-dk
15 years ago

Replying to azaozz:

Does it happen on attachment posts too (the "content" of an attachment post is the "Description" when you edit it from the Medial Library)?

Posts: The image with caption and description "Blåbærgrød" stays after updating.

Pages: When inserting the same image from the media library (with caption and description prefilled of course), the image is removed.

#12 in reply to: ↑ 11 ; follow-up: @vladimir_kolesnikov
15 years ago

Replying to anders-dk:

Weird... Are you sure you have no 3rd party plugins/javascript enabled? And 'svn status' shows no modified files?

#13 in reply to: ↑ 11 ; follow-up: @azaozz
15 years ago

Replying to anders-dk:

Posts: The image with caption and description "Blåbærgrød" stays after updating.

Pages: When inserting the same image from the media library (with caption and description prefilled of course), the image is removed.

Are you logged in as admin? This sounds like some kind of filtering if the whole image is removed...

#14 in reply to: ↑ 12 @anders-dk
15 years ago

Replying to vladimir_kolesnikov:

Weird... Are you sure you have no 3rd party plugins/javascript enabled? And 'svn status' shows no modified files?

No plugins but Akismet and Hello Dolly (deactivated).

'svn status'?

#15 in reply to: ↑ 13 @anders-dk
15 years ago

Replying to azaozz:

Are you logged in as admin? This sounds like some kind of filtering if the whole image is removed...

I also logged in as an editor and edited a page. Same same...

#16 @anders-dk
15 years ago

Anyone? 2.7-almost-rc-9997 still has this problem :-(

#17 follow-ups: @xodeus
15 years ago

Hi all.
I must say as I mentioned in the wp-testers mailing list that I can reproduce this too.
FF3 and Opera9 in Fedora and IE8 in Win7.
The issue only occurs when editing/creating pages.
I'm now using WordPress 2.7-RC1.
It doesn't matter whether the plugins are active or disabled.

#18 @xodeus
15 years ago

You can see the php_info of my host here http://xodeus.dk/info.php

#19 in reply to: ↑ 17 @vladimir_kolesnikov
15 years ago

Replying to xodeus:

Hi all.
I must say as I mentioned in the wp-testers mailing list that I can reproduce this too.
FF3 and Opera9 in Fedora and IE8 in Win7.
The issue only occurs when editing/creating pages.
I'm now using WordPress 2.7-RC1.
It doesn't matter whether the plugins are active or disabled.

#20 in reply to: ↑ 17 @vladimir_kolesnikov
15 years ago

Replying to xodeus:
Is it possible to take a look at the database dump (with all CREATE DATABASE etc - preferably created with mysqldump)?

#21 @DD32
15 years ago

since i replied on the wp-testers list: I cant reproduce it on posts/pages/attachments using both content and titles either. running PHP 5.2.4

#22 follow-up: @azaozz
15 years ago

When all plugins are disabled and the default theme used, there are still 2 places that may affect this: a [locale].php file in wp-content/languages (or perhaps wp-includes/languages) and the functions.php file in the default theme (if localized).

Can you check if there is a [locale].php file and temporary remove it, and also replace the default theme with the English version and try again.

#23 in reply to: ↑ 22 ; follow-up: @anders-dk
15 years ago

Replying to azaozz:

As vladimir_kolesnikov says: The issue only occurs when editing/creating pages.


Can you check if there is a [locale].php file and temporary remove it, and also replace the default theme with the English version and try again.


WordPress 2.7-RC1-10041: I renamed /wp-includes/locale.php to old_local.php and got this error:

Warning: require_once(/home/www/mydomain.dk/wp27beta/wp-includes/locale.php) [function.require-once]: failed to open stream: No such file or directory in /home/www/mydomain.dk/wp27beta/wp-settings.php on line 563

Fatal error: require_once() [function.require]: Failed opening required '/home/www/mydomain.dk/wp27beta/wp-includes/locale.php' (include_path='.:/usr/share/php:/usr/share/pear') in /home/www/mydomain.dk/wp27beta/wp-settings.php on line 563

#24 in reply to: ↑ 23 ; follow-up: @azaozz
15 years ago

Replying to anders-dk:

WordPress 2.7-RC1-10041: I renamed /wp-includes/locale.php to old_local.php and got this error:

Perhaps I wasn't clear: the [locale].php file would be in wp-includes/languages/[your_locale].php or wp-content/languages/[your_locale].php. For Danish the name would be dk_DK.php.

Also, if the default theme is localized (translated), try replacing it with a non-translated one from the English distribution package.

#25 in reply to: ↑ 24 @anders-dk
15 years ago

Replying to azaozz:

Perhaps I wasn't clear: the [locale].php file would be in wp-includes/languages/[your_locale].php or wp-content/languages/[your_locale].php. For Danish the name would be dk_DK.php.

Also, if the default theme is localized (translated), try replacing it with a non-translated one from the English distribution package.

Ok. It's a clean installation - it's not localized at all...

#26 follow-up: @xodeus
15 years ago

For me it's a default not localized installation too.

anders-dk: Is you host gigahost?

#27 in reply to: ↑ 26 ; follow-up: @anders-dk
15 years ago

Replying to xodeus:

anders-dk: Is you host gigahost?

Yep.

#28 in reply to: ↑ 27 ; follow-up: @xodeus
15 years ago

Replying to anders-dk:

Replying to xodeus:

anders-dk: Is you host gigahost?

Yep.

That's mine host to. Then it maybe is host related? How do we track this one?

#29 in reply to: ↑ 28 @fob
15 years ago

  • Priority changed from normal to high

That's mine host to. Then it maybe is host related? How do we track this one?

Sorry, but this is NOT host related. I have the same problem with German Umlauts on a dedicated server (Debian, PHP and mySQL all up-to-date). The first German Umlaut within the text of any page you want to update cuts everything behind it. I would suggest to use high priority for this bug.

#30 @ryan
15 years ago

What is "Encoding for pages and feeds" in the Settings -> Reading screen set to? What is DB_CHARSET in wp-config.php set to (if set at all)?

Try die()ing with post_content in the wp_insert_post() function in wp-includes/post.php. See attached diff.

@ryan
15 years ago

Output post_content and die when creating or saving posts.

#31 @ryan
15 years ago

That will output post_content right before inserting it into the DB. Let's see if it is chopped at that point.

#32 follow-up: @fob
15 years ago

FYI:

Settings -> Reading -> CHARSET = UTF-8

wp-config.php:

define('DB_CHARSET', 'utf8');
define('DB_COLLATE', );
define ('WPLANG', 'de_DE');

#33 in reply to: ↑ 32 @vladimir_kolesnikov
15 years ago

Replying to fob:

Could you please post the result of these two MySQL queries:

SHOW CREATE DATABASE your_wordpress_database_name_here

and

SHOW CREATE TABLE wp_posts

Just to make sure that both the database and wp_posts table have UTF-8 charset.

#34 @fob
15 years ago

DB is utf8_general_ci - WP 2.3 and several upgrades.

CREATE DATABASE usr_xxx1_2 /*!40100 DEFAULT CHAR...
(usr_input changed as well as tabel prefix below)

usr_xxx1_2 CREATE DATABASE usr_xxx1_2 /* !40100 DEFAULT CHARACTER SET utf8 */

CREATE TABLE xx12_posts (

ID bigint(20) unsigned NOT NULL auto_increment,
post_author bigint(20) NOT NULL default '0',
post_date datetime NOT NULL default '0000-00-00 00:00:00',
post_date_gmt datetime NOT NULL default '0000-00-00 00:00:00',
post_content longtext NOT NULL,
post_title text NOT NULL,
post_category int(4) NOT NULL default '0',
post_excerpt text NOT NULL,
post_status varchar(20) NOT NULL default 'publish',
comment_status varchar(20) NOT NULL default 'open',
ping_status varchar(20) NOT NULL default 'open',
post_password varchar(20) NOT NULL default ,
post_name varchar(200) NOT NULL default
,
to_ping text NOT NULL,
pinged text NOT NULL,
post_modified datetime NOT NULL default '0000-00-00 00:00:00',
post_modified_gmt datetime NOT NULL default '0000-00-00 00:00:00',
post_content_filtered text NOT NULL,
post_parent bigint(20) NOT NULL default '0',
guid varchar(255) NOT NULL default ,
menu_order int(11) NOT NULL default '0',
post_type varchar(20) NOT NULL default 'post',
post_mime_type varchar(100) NOT NULL default
,
comment_count bigint(20) NOT NULL default '0',
PRIMARY KEY (ID),
KEY post_name (post_name),
KEY type_status_date (post_type,post_status,post_date,ID),
KEY post_date (post_date),
KEY post_date_gmt (post_date_gmt),
KEY post_parent (post_parent)

) ENGINE=MyISAM AUTO_INCREMENT=219 DEFAULT CHARSET=utf8

#35 @fob
15 years ago

The fix provided by Ryan (at the top of this page) does not work. I made some tests with it but saw additional errors not being able to save anything.

I guess this is an easy browser issue, a problem with automatic browser changes done by Firefox and other modern browsers.

If you want to edit pages the browser encoding jumps from UTF-8 to ISO-8859-1 (German default format) so that data will be saved with invalid format or can not be saved at all. If you click on articles again you will recognize that your browser settings are switching back to UTF-8 automatically. That`s why you can read and save posts without any irritations. Opening elder pages that have been saved in former WordPress versions (UTF-8 style) will show UTF-8 errors, cryptic things. Opening posts is not a problem because database and browser encoding is quite the same.

So we seem to have a DOCUMENTTYPE ISSUE. The doctype is always the same - but on pages we have a gap at the top. May be deleting the empty line already fixes the validation problem but unfortunately I can not find out where to to do it to make a test.

Does anybody know where to change the doctype resp. header information for "Edit Page" pages only?

#36 follow-up: @fob
15 years ago

Does anyone know how to get rid of the white space in the first row of the HTML source code? I would love to see what happens if the document starts with doctype delaraton.

#37 in reply to: ↑ 36 @vladimir_kolesnikov
15 years ago

Replying to fob:

Does anyone know how to get rid of the white space in the first row of the HTML source code? I would love to see what happens if the document starts with doctype delaraton.

I know. Please apply the attached patch

@vladimir_kolesnikov
15 years ago

Removes blank lines before !DOCTYPE in wp-admin/page-new.php

#38 follow-up: @fob
15 years ago

Yes man, thank you VERY MUCH! I`m so happy that this one works for me. ;-) Not in wp-admin/page-new.php but in wp-admin/edit-page-form.php. Thanks a lot!

#39 @azaozz
15 years ago

(In [10091]) Remove blank line from edit-page-form.php, props vladimir_kolesnikov, see #8335

#40 in reply to: ↑ 38 @vladimir_kolesnikov
15 years ago

Replying to fob:

Yes man, thank you VERY MUCH! I`m so happy that this one works for me. ;-) Not in wp-admin/page-new.php but in wp-admin/edit-page-form.php. Thanks a lot!

You are always welcome! I assume that worked and special characters are not cut off anymore?

#41 follow-up: @fob
15 years ago

Everything works as it should work now. This includes saving photos with the flash uploader. Well - it works! ;-)

Another one, just FYI: When I was searching for the reason of this problem I recognized another problem with /wp-admin/includes/post.php. Within my copy of nightly builds I recognized that it doesn`t end up with a " } ?> ". May be this little mistake has not been fixed yet?

#42 in reply to: ↑ 41 ; follow-up: @vladimir_kolesnikov
15 years ago

Replying to fob:

Another one, just FYI: When I was searching for the reason of this problem I recognized another problem with /wp-admin/includes/post.php. Within my copy of nightly builds I recognized that it doesn`t end up with a " } ?> ". May be this little mistake has not been fixed yet?

Actually, PHP files are not required to end with ?>.

"php -l ./wp-admin/includes/post.php" says "no syntax errors detected"

#43 in reply to: ↑ 42 ; follow-up: @fob
15 years ago

Replying to vladimir_kolesnikov]:

Actually, PHP files are not required to end with ?>.
"php -l ./wp-admin/includes/post.php" says "no syntax errors detected"

Oops? That`s interesting.

In my opinion we could set the action status for this ticket to resolved now.
Would you like to do that?

#44 in reply to: ↑ 43 @vladimir_kolesnikov
15 years ago

  • Resolution set to fixed
  • Status changed from new to closed

Replying to fob:

In my opinion we could set the action status for this ticket to resolved now.
Would you like to do that?

NP. I thought that the ticket opener marks it as resolved.

#45 follow-up: @fob
15 years ago

Im sure the solution will help but not sure if the ticket opener is with us in this thread. If it doesnt help him too he could reopen the ticket. So I think it`s fine as it is... Thank you very much for you help!

#46 @DD32
15 years ago

NP. I thought that the ticket opener marks it as resolved.

Just a quick note, Tickets are usually closed as fixed by whoever commits the fix for the ticket, In some cases where its more of a trial-commit to see if it fixes it then it gets closed as fixed when the consensus is that it has fixed the issue for those affected. Just as long as its closed as fixed *after* the fix has been commited to SVN, then its OK

#47 @xodeus
15 years ago

Hi there. I'm not the author of this post. But my installation is the same as anders-dk's on the same host. I've applied the change by vladimir [10091] and it's working. So I would suggest to close this topic.

#48 in reply to: ↑ 45 @anders-dk
15 years ago

Replying to fob:

I`m sure the solution will help but not sure if the ticket opener is with us in this thread.

I'm with you. I'm also drinking beer once in a while... :-)

I've upgraded to 2.7-RC1-10073 and new/edit page works now - many thanks!

Note: See TracTickets for help on using tickets.