Site Status

Talk about the website, offer suggestions, express opinions, and get help with understanding and using the forums.
User avatar
Dretlin
Moderator
Moderator
Posts: 2898
Joined: Oct 29, 2004 8:00am
Gender: Male
Location: Scotland

Re: Site Status

Post by Dretlin »

Athos wrote:Awesome. And didn't even need to make that phone call. :)
Again. =D>

Theo
Host Admin
Host Admin
Posts: 276
Joined: Aug 27, 1973 12:03am
Gender: Male
Location: CA, USA, Earth, Milkyway
Contact:

Re: Site Status

Post by Theo »

As you might have seem, website was offline. For some odd reason, we were seeing some problems with the web server having apache hang recently. Nothing has changed so I don't know why it's happening. I'm writing it off as hardware. If you have something that hasn't changed software wise in 18 months, it has to be something else.

We had a dual xeon SuperMicro with 8/gb laying around so I set that up. When you are able to read this, your DNS has replicated and maintenance for you is over. If you can't read this then, well, you don't know that I just said that.

This particular SuperMicro server has an interesting flaw. It was purchased to become a Windows 2003 Small Business Server but the network kept hanging. After a year not being able to use it, it was shelved. So, last year I did some research to find that there was a bug in the design. Actually, in the Intel chipset. The newer drivers will not work on it if you have more than 4GB of ram. We have 8GB. So one of the authors of the E1000 linux driver patched it to handle the case of more than 4GB of ram. So, now it's usable, but only under linux and you have to compile the driver yourself. Luckily it can be compiled without compiling the entire kernel.

So, this machine has life again. I'll pull the old one apart and play around with it. It might come back as a database/file server once I figure it out.

Theo
Host Admin
Host Admin
Posts: 276
Joined: Aug 27, 1973 12:03am
Gender: Male
Location: CA, USA, Earth, Milkyway
Contact:

Re: Site Status

Post by Theo »

Small changes. After the server move, we maxed out connections to the server. It appears that we have leachers. Normally, I wouldn't care. Ken might though. The problem is these leachers are downloading max connections at max speed. One in particular had 200 threads going. Normal browsers have 2 threads. We have the web server set for 200 threads. So, this caused a significant slowdown for many users. AS A RESULT, we are putting a connection limiter back in place. Normally I dislike these are they have negetive affects on site performance, but the tradeoff is necessary. I have set the site for 800KB/s and 10 connections per IP. Note, this if for content. Images are housed differnetly. It should also be noted that we can tweak these limits down per IP/range. So, please repect the content, the site, the people, and the animals as well, since they can't really force your respect.

Now, with that said, if you have a particular need to want to mirror parts of the content, it would make more sense to contact Ken. I would be much easier to zip up content and have 80% of the bandwidth and servers resouces.

Time for bed now.

Theo
Host Admin
Host Admin
Posts: 276
Joined: Aug 27, 1973 12:03am
Gender: Male
Location: CA, USA, Earth, Milkyway
Contact:

Re: Site Status

Post by Theo »

Up, Down, Up, Down, Up... This is partly caused be me. The I've had to stop/start the site several times to reload the config. I'm slowly tweaking the config to see what the best combo is for the site. The last one seemed to be working well performance wise but for some reason Apache was taking 8GB of ram + some swap. It's probably because I spawned max instances. Anyway, tweked it work to about 75% of what it was. I will check it from there. I think the magic number is probably 50% of max. Unfortunately, to make changes I have to stop and start the web service which usually takes about 10 seconds.

It looks like the site is doing very well today besides my interruptions.

User avatar
SASSZAIN
Moderator
Moderator
Posts: 5302
Joined: Jan 27, 2006 8:09am
Gender: Female
Location: Everywhere and nowhere
Contact:

Re: Site Status

Post by SASSZAIN »

Sounds like you've just been having fun ;)
"You dont have to be alive to make yourself relevant.And you dont have to be a good person to be a hero.You just have to know who you are and stay true to that.So Im going to keep fighting for people the only way I ever knew how"-Vriska Serket

BigKwell
Inactive
Inactive
Posts: 218
Joined: Jul 27, 2004 5:09pm
Gender: Male
Location: Kennewick, WA, U.S.A.
Contact:

Re: Site Status

Post by BigKwell »

Heard that the site can now link to Facebook. I've got a Facebook account, but I didn't see a Facebook icon on the Absolute Anime site. Any way I can find it?
"When I hear of Schrödinger's cat, I reach for my pistol."

Stephen W. Hawking

User avatar
webmaster
Site Admin
Site Admin
Posts: 1054
Joined: Jul 24, 2004 10:25pm
Gender: Male
Location: Redding, CA
Contact:

Re: Site Status

Post by webmaster »

For right now, it only shows up when you're logged out. It'll be up near the quick login.

Theo
Host Admin
Host Admin
Posts: 276
Joined: Aug 27, 1973 12:03am
Gender: Male
Location: CA, USA, Earth, Milkyway
Contact:

Re: Site Status

Post by Theo »

Argh. I migrated one of the old web domains that nobody used anymore and my image icon is broken... I'll have to find it again.

User avatar
ANGEL OF HELL
Absolute Otaku
Posts: 1274
Joined: Jun 22, 2005 2:15pm
Gender: Male
Location: I dont even know anymore
Contact:

Re: Site Status

Post by ANGEL OF HELL »

AA has taken over my life. Every. Aspect.
"Somebody's been Funk'n with my Funk! Funk'n 'round Funk'n with it untill they Funk it up! (Don't Funk with my Funk!)" - Bootsy Collins and the RubberFans

Nyx is the wife, ya digg?

Theo
Host Admin
Host Admin
Posts: 276
Joined: Aug 27, 1973 12:03am
Gender: Male
Location: CA, USA, Earth, Milkyway
Contact:

Re: Site Status

Post by Theo »

Yeah, I found my picture on one of the old hard drives I removed from the old server.

User avatar
Dretlin
Moderator
Moderator
Posts: 2898
Joined: Oct 29, 2004 8:00am
Gender: Male
Location: Scotland

Re: Site Status

Post by Dretlin »

With the website name, I am really surprised we suffer from a lack of consistant users.

User avatar
SASSZAIN
Moderator
Moderator
Posts: 5302
Joined: Jan 27, 2006 8:09am
Gender: Female
Location: Everywhere and nowhere
Contact:

Re: Site Status

Post by SASSZAIN »

same here :( I think we get a lot of frequent visitors, though none of them stay..
"You dont have to be alive to make yourself relevant.And you dont have to be a good person to be a hero.You just have to know who you are and stay true to that.So Im going to keep fighting for people the only way I ever knew how"-Vriska Serket

Theo
Host Admin
Host Admin
Posts: 276
Joined: Aug 27, 1973 12:03am
Gender: Male
Location: CA, USA, Earth, Milkyway
Contact:

Re: Site Status

Post by Theo »

My bad... I implemented a firewall change while trying to fix a problem and it blocked a bunch of people. It's been fixed. Anyway, we have been seeing a couple incidents where the Apache web service hangs. When it happens, everything looks fine. The web service just isn't serving pages anymore. We hate to have to restart it on a regular basis, but we might have to in order to solve the problem. Luckily a graceful restart solves the problem, so hopefully no users will notice a regular restart.

Theo
Host Admin
Host Admin
Posts: 276
Joined: Aug 27, 1973 12:03am
Gender: Male
Location: CA, USA, Earth, Milkyway
Contact:

Re: Site Status

Post by Theo »

Server maintenance is being done today. I have to update some core OS files on the server. This is a must. Sicne the server is 50 miles away, if it goes bad, It will be offline for 12 hours. I don't expect this to be the case though. Server will be rebooted at 09:45 PST (about 39 minutes from the time of this posting).

Theo
Host Admin
Host Admin
Posts: 276
Joined: Aug 27, 1973 12:03am
Gender: Male
Location: CA, USA, Earth, Milkyway
Contact:

Re: Site Status

Post by Theo »

Small problems were encountered. I had to reboot the server a couple more times than expected. There was a problem with the firewall that was fixed but the changes were never saved (plus I had to go for a walk for 15 minutes). Anyway, all should be well now. If you do encounter a problem, please email me directly (gary.smith@holdstead.com) with "absoluteanime.com" somewhere in the subject.

Theo
Host Admin
Host Admin
Posts: 276
Joined: Aug 27, 1973 12:03am
Gender: Male
Location: CA, USA, Earth, Milkyway
Contact:

Re: Site Status

Post by Theo »

There was a problem with the database today. For some reason the mysql database decided that it would continously crash because of a bad innodb page. If this sounds greek, then you're part of the 99.99% of the world who doesn't need to know this. To fix the problem I had to stop the web services, stop mysql, add a special line to the mysql config file to allow innodb recovery, do a full export of the entire database (which wasn't that bad), renamed the mysql data folder, created a new mysql data folder, ran the mysql_install_db script, updated the permissions on the folder (and it's contents), removed the line that I added to the mysql config, started mysql, performed a full import of the data (which was slow as it's a large database), and then finally started the web services.

Why is this important. It's not, but if someone runs into the same problem, google would have indexed this page and they might just find the answer they were looking for.

So, the keywords of the day are: mysql innodb corrupt table recovery shot who in the foot and didn't know it tacobell and things you can buy theo for xmas

Post Reply