Business Economics Gadgets Health PHP Post Programming Religion Science The Internet Web Design WordPress

Lazy Girl Posts Pt 2

So, do you believe in God?  This video will tell you why you shouldn't.  Watch it.  Or don't. But if you don't, Santa will know.  He knows who is naughty and who is nice. [youtube]GxA8_NIxQZc[/youtube]

Fighting Spam on a Diet – How to fix Akismet Performance Problems

Running into strange WordPress performance problems and database errors?  Akismet could be the culprit, but we're in luck, it's an easy fix. Earlier I wrote a bit about our encounter with vicious, robotic Chinese comment spammers.  Since then we've had a few further issues, and I think I've found the culprit - Akismet, the plugin we've been using to fight the spam. First off, let me say that I think Akismet is a great plugin.  While we had hundreds of spams come in for a few days in a row, not one made it out to the public.  Very nice.  But it is a bit too aggressive in one spot, and that can slow down your blog or lock up the comment table, filling your max_connections. The problem is in akismet.php, specifically the akismet_delete_old() function.  Look for the following lines:
$n = mt_rand(1, 5); if ( $n % 5 ) $wpdb->query("OPTIMIZE TABLE $wpdb->comments");
Those of you with PHP / MySQL experience will recognize the problem immediately.  For the less code-literate, this is creating a random number between 1 and 5, and if the number has a remainder after being divided by 5, it runs and OPTIMIZE TABLE on the comments table.  That means that at random, it will lock the entire table and compute statistics after 80% off all deletes. Now, it's a good idea to optimize your tables after a large number of deletes.  But it is a pretty expensive operation, because it could be rearranging things on disk to free up space. Now, imagine you get hit by a spam bot and end up with a couple hundred spam comments.  Akismet catches them all, and 15 days later tries to delete them all in one big loop.  One big loop filled with a couple hundred table-locking, disk-intensive database operations. But it's easy to fix.  Replace the lines above with this:
$n = mt_rand(1, 100); if ( $n == 42 ) $wpdb->query("OPTIMIZE TABLE $wpdb->comments");
That will only optimize the table on average once out of 100 comments deleted.  Why 100?  It's an educated guess.  According to the MySQL documentation, at most you will need to optimize a table once a month or so, maybe once a week if you have a large number of deletes or edits on varchar fields. Why did I pick 42 for the one value out of a hundred that triggers an optimization?  You're asking the wrong question.

Science Projects: The Joy of Non-Newtonian Fluids

Have you ever wanted to walk on water, without the downside of being inevitably betrayed and denied by your closest friends? [youtube]f2XQ97XHjVw[/youtube] In elementary school we learn the difference between the three conventional states of matter: solid, liquid, and gas. But as lucky students with cool science teachers (and nerdy kids with the Internet) know, not all fluids are the same. Non-Newtonian fluids change viscosity under strain. Poke, shake, or smash them, and they act like a solid. Let them sit or move them around slowly, and they act like a fluid. This has some interesting applications, like body armor and custard. But who needs practical applications when you can create an evil cornstarch monster: [youtube]WnDKOc0Ag28[/youtube] Want to make some non-Newtonian fluids of your own? There are plenty of recipes online, but the easiest way to do it is to get some water in a bowl and slowly stir in some cornstarch. It should get kinda think and even, and when you get to the point where you can stir slowly, but not quickly, voila -- you have broken the laws of 4th grade science. The technical term is oobleck. Here are further instructions, one set for geeks: [youtube]BmL2WEOmEDw[/youtube] And one set for little kids: [youtube]fazPiaHvFcg[/youtube] And here's one more for the road: [youtube]NKxKVpHZe5Q[/youtube] And of course, since this is the Internet, there are even more videos to watch.

How to survive the Christmas Crisis!

As you know, we are only a few weeks away from that time of year where people lose whatever sanity they had and head out into the icy world in search for the elusive TMX Elmo or the PS3. These herds of mindless drones pushed on by consumer desire and 10 year olds are the most dangerous creatures known to mankind. The most dreaded/anticipated day of the year is coming up, The Day After Thanksgiving. The day that, for most retail workers, must not be named. So, from here on out, we will refer to it as Tdat. Or, as they call it in the world of retail, Blitz. Most of the mindless drones will spend the whole day of Thanksgiving plotting their course through the rapids of early morning shopping to get the best deals on toasters and TV sets, to find the toy of their spoiled rotten child's dream of the minute or whatever they happen to be standing in line at 4 in the morning to buy. Where will they be, you ask? Probably at one of the major retailers that you happen to have in your hometown area. I believe that your choices are displayed in the picture below. I don't know which one is Walmart or Target or whatever but I know that they are all there. Personally I like to think of the one in the green bikini as Walmart and the one with the Leopard print as Target.walmart.jpg So, in order to help you through the holiday season, I am going to put up a series of posts to guide you through this otherwise tramatic and dangerous season of getting cut off pulling into a parking space, having the last basket pulled from your hands by trailer trash, seeing the toy/item you really wanted stolen from your shopping cart while you wait to purchase it, punching said trailer trash in the face for doing all of these things to you and then spending the night in jail next to your mom, who inadvertantly did the same things as you so she can't bail you out of jail. So, stay tuned!

Comment Spam Deluge – Did our Captcha get Hacked?

Have you been having trouble reading Unsought Input lately? You're in good company – I've been having trouble writing for it.

We've been having issues with MySQL to the point of hanging connections and pleasant, but not very helpful WordPress error messages. It's nice that user-friendly errors are built-in to WordPress, since you never want to give users cryptic, blue-screen-of-death style errors. But I needed to get to the root of the problem.

I quickly put on my detective cap and tried to log in with phpMyAdmin – no luck, but this time the error message was a little more useful:

#1040 - Too many connections

Normally you encounter this error for one of two reasons: either you are being Slashdotted, or you are opening up persistent connections (with PHP's mysql_pconnect(), for example) and they are not being closed properly. In the first case, there are just too many queries at once and it fills up the connection limit, and in the second case they build up over time.

I didn't think possibility number 1 was very likely, since we don't write anything cool and geeky enough to get on Slashdot. The story about the Canadian geologist was probably our best bet. I knew I hadn't written any code to use persistent connections, but what about the rest of WordPress?

No such luck. Not a single pconnect in any of the WordPress or plugin code. Back to the first possibility – is it possible we were being hit but a distributed denial of service attack (DDoS)? More specifically (and more likely), we were being effectively DDoS'ed by comment spammers.

How did I figure it out? The connection limit for MySQL is set in the config file, my.cnf in Apache (or possibly my.ini in Windows/IIS):

[mysqld] set-variable=max_connections=100

The default is 100 and that should be enough for most sites. I needed to see what was actually being run, so I connected as a user with administrative rights and sent MySQL this command:

SHOW FULL PROCESSLIST

I got back a list of 200 locked queries, all dealing with selecting or deleting comments!

We have two measures in place to combat comment spam. One is Askimet, which is a standard plugin for WordPress. I have no hard data but I would guess almost everyone uses it. The other is a captcha plugin called Did You Pass Math?

The idea behind captchas is to give visitors a small task that is easy for humans but harder for machines. That's where those fancy images with the wavy letters and numbers come from. I wanted to use something a little simpler, so I went with Did You Pass Math. From what I've read, a big part of the power of captchas is just having something there at all to make your submit form non-standard and break the really naïve spamming scripts (see Jeff Atwood's story about his captcha in Coding Horror). It worked really well for a while.

But not any more. Askimet now reports an order of magnitude more spam blocked than ever before.

Is Did You Pass Math officially broken? It seems like I'll need to upgrade or find something different. Maybe I can hack it a bit to ask about more than just addition.

Jess B was kind enough to look through our logs and she found a ton of hits from the same IP range, and the IPs all went to spammy sites filled with more spam. Ugh.

Has anyone else noticed this with Did You Pass Math, or any other captcha plugin?