Server stuff
- Hi, Matt. I got the message you left here and I'd be more than happy to lend some suggestions. I'll leave the entire discussion here, you can point any interested parties to your talk page. I have some questions for you guys for starters. If you don't want some of this info public for some reason, then my email address is on my Heroes Wiki user page:
- You said you have two servers. Could you explain what pertinent services are running on each (i.e. are both webservers, or is one a db server and one a web server)? What are the memory and processor specs of each one?
- How many hits are you seeing during your peak day (this can include images, pages, etc.)?
- During your periods of high load, what metrics are you seeing? High load average, high cpu utilization, high IO utilization, or simply sluggish response time? If it's high cpu utilization can you pinpoint a single process or group of processes that are taking all the CPU time? Is this occurring on one server of both?
- Do you have any PHP caching extensions installed?
- Are you using any web-accelerators (Squid, etc.)?
--HeroesWiki 17:50, 5 December 2007 (CST)
- Hi Brian! Our server specs are located here. "Apollo" is our apache server, while "Athena" is our database server. Also, I posted a blog entry on our numbers for 11/24, and I'll get you the following days numbers as soon as Webmin can generate them from the logfiles. As for the server lag, it happens on the "Apollo" server exclusively, and I highly supsect the culprit is apache.
- At one time, we did have Squid installed, but it caused problems with the blog and, since we had upgraded to newer servers at the time, we haven't used it since. -- Joe Beaudoin So say we all - Donate - Sanctuary Wiki — New 23:10, 5 December 2007 (CST)
- We're not using Squid :O ? We should be. It also seems we don't have memcached, which we really should be running. --Catrope(Talk to me or e-mail me) 08:34, 6 December 2007 (CST)
- It doesn't sound like you need Squid too much at this point. If your read activitiy started causing problems then Squid would be a good idea. It looks like you're running into a similar problem that I did back in May with my old server (which is comparable to the web server you're using). I strongly recommend looking into APC. PHP is an interpreted language and when you have a lot of people hitting the site and editing articles then PHP can consume a lot of resources. APC will allow PHP to cache the MediaWiki PHP opcode. Before APC the server was very sluggish under load. During the season one finale the load average was between 13-26 constantly. By simply dropping APC in the average dropped down to about 1-2 and the site responded so well. Once you see how the site performs with APC then it'll be easier to determine if any changes in hardware are required. For reference Heroes Wiki is using dual quad-core 1.6ghz processors and never even breaks a sweat (though I purposely over-speced it to allow for significant growth). --HeroesWiki 08:34, 7 December 2007 (CST)
- Interesting. I wonder how that will work with the dynamic PHP of the forums and the blog. :o Shane (T - C - E) 13:27, 7 December 2007 (CST)
- Let's find out! Maybe try to work it into a Tuesday/maintenance day? It sounds like it may be just the ticket... --Steelviper 14:56, 7 December 2007 (CST)
- PHP itself is typically rather static while content it's used to display is very dynamic. Where Squid may cause issues with stale dynamic content, APC is only caching the php opcode itself and not the content that it's returning so it won't affect it. APC is also a generic PHP opcode cache so it's not tied to MediaWiki for instance. It will accelerate any heavily used PHP application. In fact if the application has hooks for APC built in it can further take advantage of it by using shared memory for holding global variables (MediaWiki does include specific hooks for APC to take advantage of this as well, but your biggest improvement will be the opcode caching alone). FYI I use apc.shm_size=512 to allocate 512MB of RAM for the opcode cache. If you try out APC you may need to tweak this value to reach the right balance of performance vs memory utilization. --HeroesWiki 15:01, 7 December 2007 (CST)
- Interesting. I wonder how that will work with the dynamic PHP of the forums and the blog. :o Shane (T - C - E) 13:27, 7 December 2007 (CST)
- It doesn't sound like you need Squid too much at this point. If your read activitiy started causing problems then Squid would be a good idea. It looks like you're running into a similar problem that I did back in May with my old server (which is comparable to the web server you're using). I strongly recommend looking into APC. PHP is an interpreted language and when you have a lot of people hitting the site and editing articles then PHP can consume a lot of resources. APC will allow PHP to cache the MediaWiki PHP opcode. Before APC the server was very sluggish under load. During the season one finale the load average was between 13-26 constantly. By simply dropping APC in the average dropped down to about 1-2 and the site responded so well. Once you see how the site performs with APC then it'll be easier to determine if any changes in hardware are required. For reference Heroes Wiki is using dual quad-core 1.6ghz processors and never even breaks a sweat (though I purposely over-speced it to allow for significant growth). --HeroesWiki 08:34, 7 December 2007 (CST)
- We're not using Squid :O ? We should be. It also seems we don't have memcached, which we really should be running. --Catrope(Talk to me or e-mail me) 08:34, 6 December 2007 (CST)