Theo wrote:There was a problem with the database today. For some reason the mysql database decided that it would continously crash because of a bad innodb page. If this sounds greek, then you're part of the 99.99% of the world who doesn't need to know this. To fix the problem I had to stop the web services, stop mysql, add a special line to the mysql config file to allow innodb recovery, do a full export of the entire database (which wasn't that bad), renamed the mysql data folder, created a new mysql data folder, ran the mysql_install_db script, updated the permissions on the folder (and it's contents), removed the line that I added to the mysql config, started mysql, performed a full import of the data (which was slow as it's a large database), and then finally started the web services.
Why is this important. It's not, but if someone runs into the same problem, google would have indexed this page and they might just find the answer they were looking for.
So, the keywords of the day are: mysql innodb corrupt table recovery shot who in the foot and didn't know it tacobell and things you can buy theo for xmas
Would this be also be the reason why I was unable to access the Board index? Google claimed it was a reported attack site for malware.
I don't think that had anything to do with it. Google Webmaster Tools couldn't pinpoint any cause, so I submitted for a re-check and apparently it came out good because there is no more malware warning.
I had the malware warning as well lust 5ish minutes after Athos did, then it went away just as randomly as it came back.. Really odd.
"You dont have to be alive to make yourself relevant.And you dont have to be a good person to be a hero.You just have to know who you are and stay true to that.So Im going to keep fighting for people the only way I ever knew how"-Vriska Serket
Major change to the networking of AA today. Prior to today we used to have everything running behind a NAT'ed firewall. Today we are running it through a bridged firewall. This should once and for all solve global warming. Once and for all!
As always, if any of the moderators find a problem with the site, email Ken (or myself if you are one of the privileges people who ever wrote down my email address).
If you are a moderator and need my email, just PM me.
Theo wrote:Major change to the networking of AA today. Prior to today we used to have everything running behind a NAT'ed firewall. Today we are running it through a bridged firewall. This should once and for all solve global warming. Once and for all!
As always, if any of the moderators find a problem with the site, email Ken (or myself if you are one of the privileges people who ever wrote down my email address).
If you are a moderator and need my email, just PM me.
I do require your email - but mostly for the purpose of shenanigans and nothing productive.
The server had a hiccup this week. We received an alert saying one of the mirrored drives was having some problems. So, like the good boys we are, we looking into the problem. The drive reporting the problem would not succesfully complete a disk check. The mirror was still fine. The other disk had an error form before, but selfcheck worked fine.
So I created a temporary virtual server and replicated all files/config/data/etc from the normal server to the temp server. I've done it a few times today. After the last replication, the main server stopped responding. So, we failed over to the replicated copy. There may be a 20 minutes window where forum topics were missed.
Hopefully I can get these drives replaced ASAP, assuming that's the problem. The virtual server doesn't have the ram I would like it to have to keep it on that server (it's much faster, just not enough memory). What really sucks is these drives were purchase and installed about a year ago. They have around 400 days on them and they both have errors now (one about 3 months ago, and the one this week).
There was a crash last night. For some reason we received from Segmentation fault errors in the apache log. A backtrace showed "zend_mm_heap corrupted", which, after doing some research, appears to be a non-fixed long standing PHP bug. I'm not sure why we haven't received this type of error before, in the 10+ years of running this site, I think it's the first time we have seen this bug. We have seen other bugs, just not this one. Anyway, I have implemented the suggest workaround from the PHP site for it.
Anyway, if all this sounds greek to you, well, it just means that we had a hiccup...
Theo wrote:There was a crash last night. For some reason we received from Segmentation fault errors in the apache log. A backtrace showed "zend_mm_heap corrupted", which, after doing some research, appears to be a non-fixed long standing PHP bug. I'm not sure why we haven't received this type of error before, in the 10+ years of running this site, I think it's the first time we have seen this bug. We have seen other bugs, just not this one. Anyway, I have implemented the suggest workaround from the PHP site for it.
Anyway, if all this sounds greek to you, well, it just means that we had a hiccup...
I know latin, that just sounded nerdy.
I felt like a race horse in a world without racetracks or a champion college footballer suddenly confronted by Wall Street and a business suit, his days of glory shrunk to a little gold cup on his mantel with a date engraved on it like a tombstone.