| 1 |
(11:48:55) photomatt: okay, going to flood to catch ryan up :) |
|---|
| 2 |
(11:49:03) photomatt: did you guys see the mysql guy in #wordpress last night? |
|---|
| 3 |
(11:49:03) photomatt: [11:32AM] photomatt: he suggested some really interesting improvements |
|---|
| 4 |
(11:49:03) photomatt: [11:32AM] photomatt: in my tests it took the main post query from 0.12 to 0.05 |
|---|
| 5 |
(11:49:07) photomatt: [11:32AM] photomatt: on average |
|---|
| 6 |
(11:49:09) photomatt: [11:32AM] photomatt: http://p.defau.lt/?tiJutSuV86iBJfUPGq_h0g |
|---|
| 7 |
(11:49:12) photomatt: [11:33AM] photomatt: http://p.defau.lt/?_hp_YagOMRNnFgeXdhpbpw |
|---|
| 8 |
(11:49:14) photomatt: [11:33AM] photomatt: (the second is 4.1 only though) |
|---|
| 9 |
(11:49:33) photomatt: rboren: if we could eliminate the DISTINCT and GROUP BY from post queries that don't need it |
|---|
| 10 |
(11:49:43) photomatt: and add this suggested index |
|---|
| 11 |
(11:49:53) photomatt: (he also suggested making post_type a shorter varchar, maybe 10-20) |
|---|
| 12 |
(11:50:10) photomatt: well, try out the queries, it helps a lot |
|---|
| 13 |
(11:52:51) domas [n=midom@wikipedia/Midom] entered the room. |
|---|
| 14 |
(11:53:00) domas: hello! can I lurk here? :) |
|---|
| 15 |
(11:53:17) photomatt: totally |
|---|
| 16 |
(11:53:43) photomatt: domas is the fellow who suggested the changes I pasted above |
|---|
| 17 |
(11:53:59) domas: did you collect ones that I pasted afterwards in channel? |
|---|
| 18 |
(11:54:16) photomatt: got two from p.defau.lt |
|---|
| 19 |
(11:54:30) rboren: I'm making the following changes: |
|---|
| 20 |
(11:54:31) rboren: http://p.defau.lt/?tiJutSuV86iBJfUPGq_h0g |
|---|
| 21 |
(11:54:47) rboren: And shortening the varchars for post_type and post_status |
|---|
| 22 |
(11:55:06) domas: category query is important too |
|---|
| 23 |
(11:55:14) photomatt: domas: out of curiosity, why is the index in that order? |
|---|
| 24 |
(11:55:24) photomatt: (post_status,post_type,post_date) |
|---|
| 25 |
(11:55:31) domas: photomatt: it could be type,status,date |
|---|
| 26 |
(11:55:39) domas: status,type - for WHERE, date for ORDER BY |
|---|
| 27 |
(11:56:31) domas: maybe you need to add ID there |
|---|
| 28 |
(11:56:44) domas: at the end |
|---|
| 29 |
(11:56:56) domas: (so you could GROUP BY date DESC, ID DESC |
|---|
| 30 |
(11:57:12) domas: of course, if post_date was unique, you could just GROUP BY date |
|---|
| 31 |
(11:57:21) domas: but you'd have to code according to that |
|---|
| 32 |
(11:57:23) photomatt: mysql can only use one key per query? |
|---|
| 33 |
(11:57:38) photomatt: post_date might not be unique |
|---|
| 34 |
(11:57:48) domas: then append ID at the end. |
|---|
| 35 |
(11:58:12) domas: photomatt: not really. mysql in 5.0 can do index merge for multiple keys, but it won't compile a composite key out of separate indexes |
|---|
| 36 |
(11:58:19) domas: (and that would be painful operation anyway) |
|---|
| 37 |
(11:58:30) photomatt: k |
|---|
| 38 |
(11:58:56) domas: composite keys are used everywhere |
|---|
| 39 |
(12:00:40) domas: these two queries: |
|---|
| 40 |
(12:00:40) domas: 66 Query SELECT * FROM wp_categories WHERE cat_ID > 0 HAVING category_count > 0 ORDER BY cat_name asc |
|---|
| 41 |
(12:00:40) domas: 66 Query SELECT * FROM wp_categories WHERE cat_ID > 0 HAVING link_count > 0 ORDER BY cat_name ASC |
|---|
| 42 |
(12:00:55) domas: could possibly be rewritten into link_count>0 or category_count>0 |
|---|
| 43 |
(12:01:21) domas: this one: |
|---|
| 44 |
(12:01:22) domas: 66 Query SELECT COUNT(DISTINCT ID) FROM wp_posts WHERE 1=1 AND (post_type = 'post' AND (post_status = 'publish')) |
|---|
| 45 |
(12:01:30) domas: can be eliminated too, of course :) |
|---|
| 46 |
(12:02:09) domas: I didn't look yet at the object cache code |
|---|
| 47 |
(12:02:31) photomatt: for post queries we also sometimes have "(post_author = 1 AND post_status = 'private'" |
|---|
| 48 |
(12:02:38) photomatt: to get private posts for the current logged in user |
|---|
| 49 |
(12:03:02) photomatt: but that might be a rare enough case that it doesn't need to be optimized for |
|---|
| 50 |
(12:03:15) domas: *nod* |
|---|
| 51 |
(12:03:18) domas: I'm looking at anon hits |
|---|
| 52 |
(12:04:21) photomatt: but it might be easier just to create a separate section in the admin for private posts and leave them out of public listings all-together |
|---|
| 53 |
(12:04:27) photomatt: it's a weird feature anyway |
|---|
| 54 |
(12:04:57) domas: 'cron' and 'rewrite_rules' should go into autoload options :) |
|---|
| 55 |
(12:05:22) domas: as for categories, this seems fun: |
|---|
| 56 |
(12:05:23) domas: 68 Query SELECT * FROM wp_categories |
|---|
| 57 |
(12:05:23) domas: 68 Query SELECT cat_ID FROM wp_categories |
|---|
| 58 |
(12:06:07) domas: btw, for categories, this query is executed twice too: |
|---|
| 59 |
(12:06:08) domas: 68 Query SELECT COUNT(DISTINCT ID) FROM wp_posts LEFT JOIN wp_post2cat ON (wp_posts.ID = wp_post2cat.post_id) WHERE 1=1 AND (category_id = 4) AND (post_type = 'post' AND (post_status = 'publish')) |
|---|
| 60 |
(12:06:53) domas: it's fast enough and using the (`post_status`,`post_type`,`post_date`,`ID`) index, but still :) |
|---|
| 61 |
(12:08:37) MarkJaquith: photomatt: New wp.org/support/ logins don't seem to be syncing with Trac... you aware? |
|---|
| 62 |
(12:08:47) photomatt: yes |
|---|
| 63 |
(12:09:01) photomatt: haven't been since the server move, I'm going to look at it ina bit |
|---|
| 64 |
(12:09:02) MarkJaquith: Cool. I figured it had something to do with the server move or something |
|---|
| 65 |
(12:09:21) domas: for rss feeds: |
|---|
| 66 |
(12:09:21) domas: 70 Query SELECT post_modified_gmt FROM wp_posts WHERE post_status = 'publish' ORDER BY post_modified_gmt DESC LIMIT 1 |
|---|
| 67 |
(12:09:21) domas: 70 Query SELECT post_date_gmt FROM wp_posts WHERE post_status = 'publish' ORDER BY post_date_gmt DESC LIMIT 1 |
|---|
| 68 |
(12:09:29) domas: both fields have no indexes. |
|---|
| 69 |
(12:10:00) domas: so either these values have to be maintained somewhere else |
|---|
| 70 |
(12:10:23) domas: on the second query you may just use post_date |
|---|
| 71 |
(12:10:31) domas: instead of _gmt, as for ordering value should be the same |
|---|
| 72 |
(12:11:03) MarkJaquith: there is an edge case where they aren't... not sure it matters |
|---|
| 73 |
(12:11:22) domas: well, it's rss feed |
|---|
| 74 |
(12:11:43) domas: on every hit to rss feed you order the column again and again. |
|---|
| 75 |
(12:11:49) domas: it's optimized by mysql though |
|---|
| 76 |
(12:11:57) domas: that is, only one row is used for a sort |
|---|
| 77 |
(12:13:38) domas: woo, article view. |
|---|
| 78 |
(12:14:18) domas: the index I created was used again for several queries |
|---|
| 79 |
(12:15:28) MarkJaquith: what'd you index? |
|---|
| 80 |
(12:15:29) domas: comments: |
|---|
| 81 |
(12:15:51) domas: MarkJaquith: on posts: type,status,date,id |
|---|
| 82 |
(12:16:10) domas: now comments ask for this index: postid,approved,date |
|---|
| 83 |
(12:18:13) mdawaffe is now known as mdawaffe|lunch |
|---|
| 84 |
(12:19:37) domas: with this index comments would not be sorted :) |
|---|
| 85 |
(12:19:46) domas: filesorted, just an index range scan |
|---|
| 86 |
(12:20:17) domas: it picks comments_aproved index |
|---|
| 87 |
(12:20:28) domas: I provided a hint to ignore it |
|---|
| 88 |
(12:21:19) domas: http://p.defau.lt/?DRnl1EN_6SB7UhBfgzLdrw |
|---|
| 89 |
(12:21:56) domas: the /*!4000.. specifies that the ignore index clause should be used on 4.0 or newer |
|---|
| 90 |
(12:22:04) domas: I don't remember if 3.23 actually supported IGNORE INDEX :) |
|---|
| 91 |
(12:22:20) photomatt: wow, you weren't kidding about the mysqlisms ;) |
|---|
| 92 |
(12:22:46) domas: well, /* is official SQL comment |
|---|
| 93 |
(12:22:56) domas: so, on any other platform it would not be executed |
|---|
| 94 |
(12:23:01) domas: oracle uses /*+blahblah |
|---|
| 95 |
(12:23:48) photomatt: got it |
|---|
| 96 |
(12:24:11) domas: anyway, on proper deployments you don't need this hint |
|---|
| 97 |
(12:24:22) domas: it's just my test dataset of single article provides skewed view to optimizer |
|---|
| 98 |
(12:24:37) domas: not much of optimizations can be done with single row, can it? :) |
|---|
| 99 |
(12:25:10) MarkJaquith: heh... at least not any that'll add up to what you can do while optimizing one of the posts/category queries |
|---|
| 100 |
(12:25:41) domas: hehe |
|---|
| 101 |
(12:25:51) rboren: FYI: http://trac.wordpress.org/ticket/2604 |
|---|
| 102 |
(12:26:01) domas: well, I did that analysis post on my real dataset |
|---|
| 103 |
(12:26:08) domas: which at least has >20 posts and >20 comments, ha ha ha |
|---|
| 104 |
(12:27:12) BigJibby [n=matt@mtl58-2-131-132.dialup.sprint-canada.net] entered the room. |
|---|
| 105 |
(12:27:17) MarkJaquith: domas: running 1.5.2, right? |
|---|
| 106 |
(12:27:49) domas: MarkJaquith: yessir |
|---|
| 107 |
(12:28:05) MarkJaquith: (just reading your blog post now) |
|---|
| 108 |
(12:28:39) domas: hehe, sorry for sarcasm, it wasn't directed towards any of you, just for general problem of not dealing with performance |
|---|
| 109 |
(12:29:07) domas: I'm doing performance engineering for opensource project, so every time I have to LART people about various issues |
|---|
| 110 |
(12:29:08) MarkJaquith: DISTINCT needs to go away for single-table post queries for sure |
|---|
| 111 |
(12:29:22) domas: MarkJaquith: for multiple-table you'd be better using proper GROUP BY |
|---|
| 112 |
(12:30:13) domas: http://bugs.mysql.com/bug.php?id=18182 <--- this one if fixed would improve your category query performance even more :) |
|---|
| 113 |
(12:30:37) MarkJaquith: "I’d really like if it just used "publish" status for front page and would not care about anything else. " Good news... that's what's happening for 2.1 |
|---|
| 114 |
(12:31:14) domas: hehe, I just looked at our bugs system |
|---|
| 115 |
(12:31:23) domas: I've opened 6 bugs with words 'index' in description :) |
|---|
| 116 |
(12:31:26) domas: *word |
|---|
| 117 |
(12:31:58) MarkJaquith: The hidden benefit there is that you can now cache the queries, because you don't have that dynamic timestamp variable in each query |
|---|
| 118 |
(12:32:13) ***domas enjoys the fading yellow |
|---|
| 119 |
(12:33:24) domas: caching is nice for small sites enjoying lots of reads and no writes |
|---|
| 120 |
(12:33:37) domas: none of sites I work with or clients of mine have such situations :) |
|---|
| 121 |
(12:33:45) photomatt: that's the bulk of WP installations |
|---|
| 122 |
(12:33:58) photomatt: if you think we have perf problems now, you should have seen 0.72 |
|---|
| 123 |
(12:34:00) photomatt: :) |
|---|
| 124 |
(12:34:31) rboren: Not many people were around to witness 0.72 perf problems. :-) |
|---|
| 125 |
(12:34:43) photomatt: I remember doing a profile and finding 60% of our time being spent on convert_char or something like that |
|---|
| 126 |
(12:34:44) domas: haha, I'm fascinated enough to remember mediawiki 1.3/1.4 ;-) |
|---|
| 127 |
(12:35:03) photomatt: which was individually looping over EVERY SINGLE CHARACTER in every post |
|---|
| 128 |
(12:35:04) domas: photomatt: well, an example of badly optimized situation was |
|---|
| 129 |
(12:35:23) domas: when for interwiki prefixes, for every link there was a hit to our memcached cluster |
|---|
| 130 |
(12:35:31) ***MarkJaquith is so glad he only got involved with WordPress right before 1.2 release |
|---|
| 131 |
(12:35:34) domas: and it takes ~2ms each |
|---|
| 132 |
(12:36:08) domas: so it used to spend 50ms or more on fetching absolutely static data |
|---|
| 133 |
(12:36:27) domas: and the interesting part, why it was not noticed |
|---|
| 134 |
(12:36:40) domas: it always was hidden beneath other tasks that appeared 'long' in profiling trees |
|---|
| 135 |
(12:36:52) domas: only proper kcachegrind analysis allowed to track such issues |
|---|
| 136 |
(12:37:12) domas: moral of the story: do not assume the task is slow until you know what is slow in it. |
|---|
| 137 |
(12:37:50) ***ringmaster re-reads the story of MediaWiki innards and impales himself on a fork. |
|---|
| 138 |
(12:37:50) photomatt: what's kcachegrind? |
|---|
| 139 |
(12:38:17) domas: ringmaster: which story? |
|---|
| 140 |
(12:38:26) ringmaster: The one you just told. :) |
|---|
| 141 |
(12:38:51) domas: photomatt: http://dammit.lt/2006/01/18/mediawiki-graphic-profile/ |
|---|
| 142 |
(12:39:03) domas: photomatt: one (two actually) pictures are better than thousand words. |
|---|
| 143 |
(12:39:05) MarkJaquith: domas: in 2.x, WP now stores comment_count in the posts table, as you suggested (still reading!) |
|---|
| 144 |
(12:39:16) photomatt: that's purty |
|---|
| 145 |
(12:39:36) domas: photomatt: I was working on cpu profiling instead of real time profiling |
|---|
| 146 |
(12:39:42) domas: but ditched that for a while |
|---|
| 147 |
(12:39:45) domas: may resume the effort some day |
|---|
| 148 |
(12:45:49) domas: hehe, wanna see a pic of wordpress kcachegrind graph? :) |
|---|
| 149 |
(12:46:01) photomatt: sounds perfect, I'll definitely try to make it |
|---|
| 150 |
(12:46:40) ***MarkJaquith puts his money down on smiley regexes |
|---|
| 151 |
(12:46:44) MarkJaquith: (kidding) |
|---|
| 152 |
(12:46:55) domas: haha, someone asked for that |
|---|
| 153 |
(12:46:59) domas: in mediawiki |
|---|
| 154 |
(12:47:41) photomatt: sure |
|---|
| 155 |
(12:48:15) domas: I hate such features :) |
|---|
| 156 |
(12:48:18) domas: pain to maintain |
|---|
| 157 |
(12:48:22) domas: breaks other syntax |
|---|
| 158 |
(12:48:25) domas: performance hog |
|---|
| 159 |
(12:49:07) MarkJaquith: 1) wait 2) wait 3) wait 4) wait 5) wait 6) wait 7) almost there 8) SUNGLASSES! |
|---|
| 160 |
(12:50:09) photomatt: ha! |
|---|
| 161 |
(12:50:15) photomatt: LOL |
|---|
| 162 |
(12:51:38) domas: ;-D |
|---|
| 163 |
(12:52:49) MarkJaquith: error_bot: google wordpress smilies |
|---|
| 164 |
(12:52:52) error_bot: MarkJaquith: Search took 0.17 seconds: WordPress Smilies [ Tempus Fugit | TxFx.net ]: <http://txfx.net/2004/08/18/wordpress-smilies/>; WordPress Smilies: <http://www.tamba2.org.uk/wordpress/smilies/>; Using Smilies « WordPress Codex: <http://codex.wordpress.org/Using_Smilies> |
|---|
| 165 |
(12:53:16) MarkJaquith: (that first result would be mine, domas) |
|---|
| 166 |
(12:55:49) photomatt: domas: would love to see kcachegrind output for WP |
|---|
| 167 |
(12:56:43) domas: photomatt: hehe |
|---|
| 168 |
(12:56:47) domas: ---> Fetching graphviz |
|---|
| 169 |
(12:56:48) domas: ---> Attempting to fetch graphviz-2.6.tar.gz from http://www.graphviz.org/pub/graphviz/ARCHIVE/ |
|---|
| 170 |
(12:56:52) domas: forgot to install some deps before |
|---|
| 171 |
(12:56:55) domas: on this machine |
|---|
| 172 |
(12:57:18) domas: well, I don't have enough data |
|---|
| 173 |
(12:57:23) domas: so the results would be... skewed again :) |
|---|
| 174 |
(12:57:33) domas: oh, downloaded |
|---|
| 175 |
(12:57:52) domas: this LCD compiles things really fast :) |
|---|
| 176 |
(13:00:11) photomatt: oh that's right you still need the dump file |
|---|
| 177 |
(13:00:23) photomatt: rboren: do you still have that giant MT dump I sent you? I can't find it |
|---|
| 178 |
(13:00:34) rboren: Lemme look... |
|---|
| 179 |
(13:03:54) rboren: Found it. |
|---|
| 180 |
(13:04:02) rboren: I'll upload it somewhere... |
|---|
| 181 |
(13:05:31) BigJibby left the room (quit: Read error: 104 (Connection reset by peer)). |
|---|
| 182 |
(13:05:41) domas: compress it too, thanks! |
|---|
| 183 |
(13:11:30) domas: damn, it got mad with wp profile |
|---|
| 184 |
(13:12:12) domas: there's some code that is executed directly via require_once() instead of calling as functions |
|---|
| 185 |
(13:12:40) photomatt: ? |
|---|
| 186 |
(13:15:09) domas: well, the most expensive functions are require_once |
|---|
| 187 |
(13:15:12) domas: and require |
|---|
| 188 |
(13:15:35) domas: instead of loading file with functions and calling them, you execute it directly |
|---|
| 189 |
(13:15:49) domas: so there's no difference between actually loading file and executing code |
|---|
| 190 |
(13:15:50) domas: in the profile |
|---|
| 191 |
(13:16:02) masquerade [n=masquera@c-68-49-145-242.hsd1.de.comcast.net] entered the room. |
|---|
| 192 |
(13:16:05) photomatt: huh |
|---|
| 193 |
(13:16:25) photomatt: well because of the amount of code, we've always suspected that the biggest overhead was simply parsing it all |
|---|
| 194 |
(13:16:31) domas: not really |
|---|
| 195 |
(13:16:33) photomatt: which is why an opcode cache helps so much |
|---|
| 196 |
(13:16:53) domas: but yeah, opcode helps |
|---|
| 197 |
(13:17:56) domas: it spent 5% of real time on mysql ;-D |
|---|
| 198 |
(13:18:44) domas: eh, I can't provide you with a nice graph, just several small ones probably |
|---|
| 199 |
(13:19:57) photomatt: too much junk? |
|---|
| 200 |
(13:20:36) domas: ah, or doesn't show too much |
|---|
| 201 |
(13:20:40) domas: it's much better if interactive |
|---|
| 202 |
(13:20:49) domas: for analysis of details |
|---|
| 203 |
(13:21:23) domas: and maybe xdebug acting funny on this box :( |
|---|
| 204 |
(13:21:31) domas: ok, I'll do one export |
|---|
| 205 |
(13:23:27) domas: http://dammit.lt/wordpress-kcg.png |
|---|
| 206 |
(13:24:08) domas: most of code is hidden in require or require_once calls |
|---|
| 207 |
(13:24:10) domas: :( |
|---|
| 208 |
(13:24:57) photomatt: weird |
|---|
| 209 |
(13:25:05) photomatt: I'm not sure what we're doing that would make it appear so weird |
|---|
| 210 |
(13:25:32) masquerade: Anyone have Zend Studio's Debug Server setup? Its profiling might help out in this case |
|---|
| 211 |
(13:25:50) photomatt: lots of time in load_template, it looks like |
|---|
| 212 |
(13:26:09) MarkJaquith: lot of file_exists calls eh? |
|---|
| 213 |
(13:26:44) domas: you should really move away execution from script loading |
|---|
| 214 |
(13:27:03) photomatt: I still don't understand exactly what you mean by that |
|---|
| 215 |
(13:27:09) domas: require();require(); dothis(); dothat(); |
|---|
| 216 |
(13:27:21) photomatt: oh I see |
|---|
| 217 |
(13:27:29) photomatt: so the action is happening inside the requires |
|---|
| 218 |
(13:27:34) ringmaster: As opposed to require(<? dothis());require(<? dothat()); |
|---|
| 219 |
(13:28:03) domas: well, not really |
|---|
| 220 |
(13:28:11) photomatt: I think he means that a file includes another file which has the executing code in it |
|---|
| 221 |
(13:28:11) domas: the problem is require(code;code;no;functions) |
|---|
| 222 |
(13:28:28) domas: e.g. wp-settings |
|---|
| 223 |
(13:28:28) photomatt: so we don't execute anything inside of index.php, for example |
|---|
| 224 |
(13:28:43) domas: has lots of code but 3 tiny functions |
|---|
| 225 |
(13:29:19) photomatt: nothing executes from wp-includes, I think |
|---|
| 226 |
(13:29:23) photomatt: but other files do |
|---|
| 227 |
(13:29:47) photomatt: I take that back, nothing in functions.php, template* executes any code, just loads it |
|---|
| 228 |
(13:30:13) masquerade: I fail to see how any of that is or could be a performance trap, though |
|---|
| 229 |
(13:30:42) domas: not much of performance trap |
|---|
| 230 |
(13:30:45) MarkJaquith: masquerade: it's just a performance mask... with this tool. |
|---|
| 231 |
(13:30:49) domas: it just obstructs clear view of what is happening where |
|---|
| 232 |
(13:31:20) domas: anyway, branches can still be analyzed |
|---|
| 233 |
(13:31:21) masquerade: try APD |
|---|
| 234 |
(13:31:33) domas: masquerade: it won't give you graphic profile :) |
|---|
| 235 |
(13:31:46) domas: you should _try_ kcachegrind :) |
|---|
| 236 |
(13:31:53) masquerade: I'd rather have the text than that mess any day, for PHP |
|---|
| 237 |
(13:32:49) domas: ah: http://flake.defau.lt/pics/mediawiki.png |
|---|
| 238 |
(13:32:54) domas: this is how it usually outputs stuff |
|---|
| 239 |
(13:33:16) domas: (that picture details the situation I talked about here ;-) |
|---|
| 240 |
(13:33:49) domas: masquerade: the nice thing about that is 'where is this method called, what is called by it, how much time was spent in multiple places, etc' |
|---|
| 241 |
(13:35:34) domas: e.g. this is all database activity profiled: http://dammit.lt/wordpress-kcg-sql.png |
|---|
| 242 |
(13:35:52) domas: (on an empty db though, I'll try importing a dump soon) |
|---|
| 243 |
(13:36:50) MarkJaquith: photomatt: the load_template just does require_once... so that's misleading. |
|---|
| 244 |
(13:37:44) masquerade: I'm spoiled by Zend Studio's profiler |
|---|
| 245 |
(13:37:46) photomatt: yep |
|---|
| 246 |
(13:37:57) photomatt: masquerade: andy has been using that |
|---|
| 247 |
(13:38:19) masquerade: I'd pick it any day over callgrind/kcachegrind style dumps for PHP source |
|---|
| 248 |
(13:38:42) masquerade: (Although I do understand the value of callgrind, its invaluable when working with C) |
|---|
| 249 |
(13:47:34) domas: anyway, I'm not selling anything here, hehehe |
|---|
| 250 |
(13:49:27) domas: ok, with more data |
|---|
| 251 |
(13:49:47) domas: http://p.defau.lt/?GyzOyrdWQ1eScjMefMXOoQ |
|---|
| 252 |
(13:49:55) domas: this may be probably loaded linked to posts read |
|---|
| 253 |
(13:49:57) domas: joined |
|---|
| 254 |
(13:51:59) MarkJaquith: Wait, it did that all in one load? |
|---|
| 255 |
(13:52:19) domas: yes |
|---|
| 256 |
(13:52:27) domas: for anon |
|---|
| 257 |
(13:53:00) MarkJaquith: You have the object cache disabled, or is wp-content not server writable? |
|---|
| 258 |
(13:53:07) masquerade: MarkJaquith, he's on 1.5 |
|---|
| 259 |
(13:53:20) masquerade: (or according to the blog post, not 2.0 at least) |
|---|
| 260 |
(13:53:25) domas: masquerade: 2.1-alpha |
|---|
| 261 |
(13:53:25) MarkJaquith: masquerade: wp_usermeta |
|---|
| 262 |
(13:53:38) masquerade: ahh, I walked in halfway through all this, excuse me |
|---|
| 263 |
(13:53:39) domas: MarkJaquith: I guess it's object cache disabled |
|---|
| 264 |
(13:54:00) domas: MarkJaquith: it's "default install" |
|---|
| 265 |
(13:54:02) domas: of 2.1 alpha |
|---|
| 266 |
(13:54:07) domas: and wp-content is writable |
|---|
| 267 |
(13:54:10) domas: ok, I'll enable it |
|---|
| 268 |
(13:54:43) MarkJaquith: domas: ah yeah, it is disabled by default in 2.1 |
|---|
| 269 |
(13:55:11) MarkJaquith: so you have posts by each of these authors on the loaded page, right? |
|---|
| 270 |
(13:55:20) domas: yes |
|---|
| 271 |
(13:55:41) ***domas notices cache on filesystem sucks if NFS is used ;-) |
|---|
| 272 |
(13:56:08) MarkJaquith: So it should be grabbing the user IDs from the posts query and doing one query with ID IN (1,2,3,4,5) eh? |
|---|
| 273 |
(13:56:11) domas: but I guess people are already sane at these times |
|---|
| 274 |
(13:56:24) domas: MarkJaquith: you can just join user directly to posts. |
|---|
| 275 |
(13:56:36) domas: SELECT * from wp_posts join wp_users ON (.. |
|---|
| 276 |
(13:56:41) domas: well, * is evil |
|---|
| 277 |
(13:56:42) domas: but anyway |
|---|
| 278 |
(13:57:24) MarkJaquith: hm, only problem with that is that multiple post queries would do possibly redundant user joins |
|---|
| 279 |
(13:58:04) domas: that's why you'd avoid multiple post queries :) |
|---|
| 280 |
(13:58:27) MarkJaquith: well yes, not in the default template |
|---|
| 281 |
(13:59:26) domas: on the other hand |
|---|
| 282 |
(13:59:28) domas: it's much cheaper |
|---|
| 283 |
(13:59:45) domas: than having multiple queries, ipc, .. |
|---|
| 284 |
(14:00:14) domas: I voted for GROUP_CONCAT() for categories too, but this bastard is in mysql 4.1 and later only |
|---|
| 285 |
(14:00:14) masquerade: heh, just about every other theme and plugin on the planet uses query_posts() for a second post query, so I'm not really sure avoiding multiple post queries is an option |
|---|
| 286 |
(14:00:30) domas: masquerade: but you should not care about join price in this case. |
|---|
| 287 |
(14:00:34) domas: a join is done on an index |
|---|
| 288 |
(14:00:36) domas: and on hot data |
|---|
| 289 |
(14:00:39) domas: it's _cheap_ |
|---|
| 290 |
(14:00:47) domas: whatever people tell to you otherwise, don't listen. |
|---|
| 291 |
(14:01:05) MarkJaquith: heh, I expect you know what you're talking about here |
|---|
| 292 |
(14:01:36) domas: well, when you see a join of 50 tables |
|---|
| 293 |
(14:01:43) domas: then you think about better in-db optimization techniques |
|---|
| 294 |
(14:01:47) domas: rather than splitting it |
|---|
| 295 |
(14:01:51) domas: :) |
|---|
| 296 |
(14:02:30) MarkJaquith: this would be a massive code change. |
|---|
| 297 |
(14:02:30) photomatt: domas: hot data? |
|---|
| 298 |
(14:02:43) domas: photomatt: it's in RAM |
|---|
| 299 |
(14:02:58) domas: photomatt: you've read it miliseconds ago, it's in RAM! :) |
|---|
| 300 |
(14:03:03) domas: ergh, nanoseconds even |
|---|
| 301 |
(14:04:29) domas: heh, kubrick fetches are annoying |
|---|
| 302 |
(14:05:14) domas: http://p.defau.lt/?6kSNMyxeSoWHy6ute45mow |
|---|
| 303 |
(14:05:38) MarkJaquith: domas: I filed a bug for that |
|---|
| 304 |
(14:05:42) photomatt: we don't cache "cache misses" |
|---|
| 305 |
(14:05:50) photomatt: right now |
|---|
| 306 |
(14:06:11) domas: well, my original issue about rss feed inside options table still matters :) |
|---|
| 307 |
(14:06:21) domas: photomatt: it should be autoloaded |
|---|
| 308 |
(14:06:22) MarkJaquith: yeah, why are those autoloaded? |
|---|
| 309 |
(14:06:34) photomatt: the options don't exist |
|---|
| 310 |
(14:06:35) MarkJaquith: domas: er, should be? |
|---|
| 311 |
(14:07:00) domas: ah, right |
|---|
| 312 |
(14:07:05) domas: get it :) |
|---|
| 313 |
(14:07:10) photomatt: the RSS cache in the DB is a poor man's cache, writing to the filesystem is a real pain in the enviroments we run in |
|---|
| 314 |
(14:07:24) domas: photomatt: use other DB for that |
|---|
| 315 |
(14:07:28) domas: pew |
|---|
| 316 |
(14:07:29) domas: table. |
|---|
| 317 |
(14:07:30) photomatt: and loading the remote RSS all the time is not nice and slow |
|---|
| 318 |
(14:07:44) photomatt: why? |
|---|
| 319 |
(14:07:57) photomatt: more data in the options table doesn't slow it down, does it? |
|---|
| 320 |
(14:08:31) domas: well, optimizer sees 70 rows |
|---|
| 321 |
(14:08:40) domas: and notices that most of them are autoloaded |
|---|
| 322 |
(14:08:47) domas: so it doesn't even pick autoload index |
|---|
| 323 |
(14:09:02) domas: but there's _one_ row that weights 100x more than all other rows combined |
|---|
| 324 |
(14:09:23) domas: of course, it's negligible data amount ;-) |
|---|
| 325 |
(14:09:25) MarkJaquith: there will still be autoload = no rows even if RSS cache was moved |
|---|
| 326 |
(14:09:37) domas: MarkJaquith: those would not weight 100k |
|---|
| 327 |
(14:09:57) MarkJaquith: hrm, so it's the size of it that matters? |
|---|
| 328 |
(14:09:58) domas: I'm a bit anal here about that, maybe I shouldn't stress it too much |
|---|
| 329 |
(14:10:17) domas: I usually explain this situation as 'the elephant among the ants' |
|---|
| 330 |
(14:10:50) domas: you just can't apply neither elephant rules to all environment, nor ant rules |
|---|
| 331 |
(14:10:51) MarkJaquith: Heh. Well, I'll say this... SELECT * FROM wp_options; is hell from the command line because of those feeds ;-) |
|---|
| 332 |
(14:11:02) domas: MarkJaquith: too. |
|---|
| 333 |
(14:11:03) masquerade: just wait until you see the amount of data plugins could and do store ;-) |
|---|
| 334 |
(14:11:13) domas: haha, lol |
|---|
| 335 |
(14:11:14) domas: screw plugins |
|---|
| 336 |
(14:11:47) mdawaffe|lunch is now known as mdawaffe |
|---|
| 337 |
(14:11:57) domas: soo... |
|---|
| 338 |
(14:11:57) domas: SELECT DISTINCT YEAR(post_date) AS `year`, MONTH(post_date) AS `month`, count(ID) as posts FROM wp_posts WHERE post_type = 'post' AND post_status = 'publish' GROUP BY YEAR(post_date), MONTH(post_date) ORDER BY post_date DESC |
|---|
| 339 |
(14:11:59) domas: this one |
|---|
| 340 |
(14:12:00) mdawaffe: MarkJaquith: did you get a chance to look at 2561? |
|---|
| 341 |
(14:12:02) domas: is the one I hate most actually |
|---|
| 342 |
(14:12:06) MarkJaquith: mdawaffe: not yet |
|---|
| 343 |
(14:12:11) mdawaffe: no worries |
|---|
| 344 |
(14:12:11) photomatt: haha me too |
|---|
| 345 |
(14:12:45) domas: use per-month cache in a table. :) |
|---|
| 346 |
(14:13:04) domas: publish an article - increase. delete an article - decrease. \o/ |
|---|
| 347 |
(14:15:00) domas: oh wait, query cache :) |
|---|
| 348 |
(14:16:00) MarkJaquith: Here's that non-existent option bug report: http://trac.wordpress.org/ticket/2268 |
|---|
| 349 |
(14:16:18) domas: as the timestamp is not put into queries, it's much more efficient with query cache of course ;-) |
|---|
| 350 |
(14:16:32) domas: hehe, wanna add one more mysqlism? :) |
|---|
| 351 |
(14:17:01) ringmaster left the room (quit: "Find me at http://asymptomatic.net, where I remain ambivalent."). |
|---|
| 352 |
(14:17:11) MarkJaquith: go for it |
|---|
| 353 |
(14:17:16) domas: SELECT /*! 40000 SQL_CACHE */ ... |
|---|
| 354 |
(14:17:17) domas: ;-) |
|---|
| 355 |
(14:17:29) domas: for queries you really want to cache. |
|---|
| 356 |
(14:18:04) domas: there's a setting for mysqld, which caches only queries with such tags. |
|---|
| 357 |
(14:18:34) domas: so, it'd cache in such cases :) |
|---|
| 358 |
(14:20:36) MarkJaquith: Would require people to have an appropriate setup though |
|---|
| 359 |
(14:20:41) domas: yes |
|---|
| 360 |
(14:21:03) domas: minor issue anyway :) |
|---|