Bandwidth in computer networking refers to the data rate supported by a network connection or interface. One most commonly expresses bandwidth in terms of bits per second (bps). The term comes from the field of electrical engineering, where bandwidth represents the total distance or range between the highest and lowest signals on the communication channel (band).
Bandwidth represents the capacity of the connection. The greater the capacity, the more likely that greater performance will follow, though overall performance also depends on other factors, such as latency.
Here is the list of bandwidth monitoring tools for your network bandwidth
bmon
bwbar
bwm
bwm-ng
iftop
iperf
ipfm
speedometer
cbm
ibmonitor
pktstat
mactrack
MRTG
Cacti
Now we will see each tool separately
bmon
bmon is a portable bandwidth monitor and rate estimator running on various operating systems. It supports various input methods for different architectures.
Various output modes exist including an interactive curses interface, lightweight HTML output but also formatable ASCII output
Current Stable Version :- 2.1.0
Install bmon in Ubuntu
sudo apt-get install bmon
This will complete the installation.
If you want to open the application you need to enter the following command
bmon
you should see the following output
If i select the eth0 i should see all the traffic details
If you want to know more available options check man page
bwbar
This program will output a PNG and a text file that can be used in scripts or be included in web pages to show current bandwidth usage. The amount of total bandwidth can be customized. The PNG output appears as a bar graph showing maximum possible usage with the current inbound or outbound usage shown as a differently colored bar.
Current Stable Version :- 1.2.3
Install bwbar in ubuntu
sudo apt-get install bwbar
This will complete the installation
There is a /etc/default/bwbar file to configure before this package will work and here is my config
# Directory to put files into
DIR=/var/www
# Who to run as
RUNAS=www-data
# What are the options (eth0, scale of 1.5mbps)
OPTIONS=”eth0 1.5″
Now you need to chnage the permissions using the following command
sudo chown -R www-data:www-data /var/www/bwbar/
Start the bwbar using the following command
sudo /etc/init.d/bwbar start
You can also launch bwbar using the following command
bwbar eth0 100 -x 200 -y 7 -t 10 -p network.png -f network.txt -d /var/www/bwbar &
Now you need to go to http://serverip/bwbar you should see the bandwidth bar as follows
If you want to know about available option check man page
bwm
BandWidth Monitor This is a very tiny bandwidth monitor (not X11). Can monitor up to 16 interfaces in the in the same time, and shows totals too.
Current Stable Version :- 1.1.0
Install bwm ubuntu
sudo apt-get install bwm
This will complete the installation now
If you want to run the application use the following command
bwm
Output looks like below
If you want more details and available options check man page
bwm-ng
small and simple console-based bandwidth monitor.Bandwidth Monitor NG is a small and simple console-based live bandwidth monitor.
Current Stable Version :- 0.5
features
supports /proc/net/dev, netstat, getifaddr, sysctl, kstat and libstatgrab
unlimited number of interfaces supported
interfaces are added or removed dynamically from list
white-/blacklist of interfaces
output of KB/s, Kb/s, packets, errors, average, max and total sum
output in curses, plain console, CSV or HTML
configfile
Install bwm-ng in ubuntu
sudo apt-get install bwm-ng
This will complete the installation
If you want to run the application you need to use the following command
bwm-ng
Output looks like below
If you want more details and available options check man page
iftop
iftop does for network usage what top does for CPU usage. It listens to network traffic on a named interface and displays a table of current bandwidth usage by pairs of hosts.
Current Stable Version :- 0.17
Install iftop in ubuntu
sudo apt-get install iftop
This will complete the installation
If you want to run the application you need to use the following command
iftop
Output looks like below
If you want more details and available options check man page
ipfm
IP Flow Meter (IPFM) is a bandwidth analysis tool, that measures how much bandwidth specified hosts use on their Internet link.
Current Stable Version :- 0.11.5
Install ipfm in Ubuntu
sudo apt-get install ipfm
This will complete the installation
Now if you want to configure this for your network you need to copy example configuration file from /usr/share/doc/ipfm/examples if you want to check this file click here
once you configure this file you need to start the service using the following command
sudo /etc/init.d/ipfm start
If you want more details and available options check man page
Speedometer
Measure and display the rate of data across a network connection or data being stored in a file.
Current Stable Version :- 2.4
Install speedometer in ubuntu
Check which version of python is the default by running
python -V
Then issue the following commands as root to install speedometer (choose the correct Urwid package for your python version, ie. if python -V reports version 2.3.X then install python2.3-urwid)
apt-get install python2.4-urwid
Download the speedometer.py source file.
As user issue the following commands in the directory that you downloaded the source file
sudo cp speedometer.py /usr/local/bin/speedometer
sudo chown root: /usr/local/bin/speedometer
sudo chmod 755 /usr/local/bin/speedometer
Now you can run the speedometer application using the following
/usr/local/bin/speedometer
Now you should see the following screen output
Speedometer Usage
Usage: speedometer [options] tap [[-c] tap]
Available options
speedometer -h
Usage: speedometer [options] tap [[-c] tap]…
Monitor network traffic or speed/progress of a file transfer. At least one tap must be entered. -c starts a new column, otherwise taps are piled vertically.
Taps:
[-f] filename [size] display download speed [with progress bar]
-f must be used if directly following another
file tap without an expected size specified
-rx network-interface display bytes received on network-interface
-tx network-interface display bytes transmitted on network-interface
Options:
-i interval-in-seconds eg. “5″ or “0.25″ default: “1″
-p use plain-text display (one tap only)
-b use old blocky display instead of smoothed display even when UTF-8 encoding is detected
-z report zero size on files that don’t exist instead of waiting for them to be created
Usage Examples
How long it will take for my 38MB transfer to finish?
speedometer favorite_episode.rm $((38*1024*1024))
How quickly is another transfer going?
speedometer dl/big.avi
How fast is this LAN?
$ cat /dev/zero | nc -l -p 12345
$ nc host-a 12345 > /dev/null
$ speedometer -rx eth0
How fast is the upstream on this ADSL line?
speedometer -tx ppp0
How fast can I write data to my filesystem? (with at least 1GB free)
dd bs=1000000 count=1000 if=/dev/zero of=big_nothing &
speedometer big_nothing
cbm
cbm — the Color Bandwidth Meter — displays the current traffic on all network devices.
Current Stable Version :- 0.1-1
Install cbm in ubuntu
First you need to download the .deb package from here
once you have the .deb package you need to install using the following comamnd
sudo dpkg -i cbm_0.1-1_i386.deb
this will complete the installation now if you want to use the appltcation use the following command
cbm
Output looks like below
pktstat
pktstat listens to the network and shows the bandwidth being consumed by packets of various kinds in realtime. It understands some protocols (including FTP,HTTP, and X11) and adds a descriptive name next to the entry (e.g., ‘RETR cd8.iso’, ‘GET http://slashdot.org/’ or ‘xclock -fg blue’).
Current Stable Version :- 1.8.3
Install pktstat in Ubuntu
First you need to download .rpm package from here once you have the .rpm package you need to convert this .rpm file to .deb file using alien
Install alien
sudo apt-get install alien
Now you need to use the follwoing command to convert .rpm to .deb
sudo alien -k pktstat-1.7.2q-0.i386.rpm
Now you should be having pktstat_1.7.2q-0_i386.deb package
Install pktstat in Ubuntu
sudo dpkg -i pktstat_1.7.2q-0_i386.deb
This will complete the installation now you can open the application using the following command
pktstat
output looks like below
If you want more available options for pktstat check man page
ibmonitor
ibmonitor is an interactive linux console application which shows bandwidth consumed and total data transferred on all interfaces.
Current Stable Version :- 1.4
Its main features are:
Shows received, transmitted and total bandwidth of each interface
Calculates and displays the combined value of all interfaces
Displays total data transferred per interface in KB/MB/GB
Values can be displayed in Kbits/sec(Kbps) and/or KBytes/sec(KBps)
Can show maximum bandwidth consumed on each interface since start of utility
Can show average bandwidth consumption on each interface since start of utility
The output with all features (max, avg and display in Kbps and KBps) easily fits on a 80×24 console or xterm
Can interactively change its output display format depending on key pressed by user.
Install ibmonitor in Ubuntu
First you need to download the latest version from here
wget http://ovh.dl.sourceforge.net/sourceforge/ibmonitor/ibmonitor-1.4.tar.gz
Now you have ibmonitor-1.4.tar.gz
Extract this file using the following commands
tar xvfz ibmonitor-1.4.tar.gz
cd ibmonitor
If you want to run the application use the following command
Once you are in ibmonitor folder use
./ibmonitor
Output looks like the following screen
iperf
While tools to measure network performance, such as ttcp, exist, most are very old and have confusing options. Iperf was developed as a modern alternative for measuring TCP and UDP bandwidth performance.
Iperf is a tool to measure maximum TCP bandwidth, allowing the tuning of various parameters and UDP characteristics. Iperf reports bandwidth, delay jitter, datagram loss.
Current Stable Version :- 2.0.2
Install iperf in ubuntu
sudo apt-get install iperf
iperf Syntax
iperf [-s|-c host] [options]
Example
iperf -c server address -F file-name
iperf -c server address -I
The -F option is for file input.
The -I option is for input from stdin.
If you want more details and available options check man page
tcptrack
tcptrack is a sniffer which displays information about TCP connections it sees on a network interface. It passively watches for connections on the network interface, keeps track of their state and displays a list of connections in a manner similar to the unix ‘top’ command. It displays source and destination addresses and ports, connection state, idle time, and bandwidth usage.
Current Stable Version :- 1.1.5
Install tcptrack in Ubuntu
sudo apt-get install tcptrack
this will complete the installation
tcptrack Syntax
tcptrack [-dfhvp] [-r ] -i []
Examples
tcptrack requires only one parameter to run: the -i flag followed by an interface name that you want tcptrack to monitor. This is the most basic way to run tcptrack
tcptrack -i eth0
tcptrack can also take a pcap filter expression as an argument. The format of this filter expression is the same as that of tcpdump and other
libpcap-based sniffers. The following example will only show connections from host 10.45.165.2
tcptrack -i eth0 src or dst 10.45.165.2
The next example will only show web traffic (ie, traffic on port 80)
tcptrack -i eth0 port 80
The following output screen will show you more details
MRTG
The Multi Router Traffic Grapher or just simply MRTG is free software for monitoring the traffic load on network links. It allows the user to see traffic load on a network over time in graphical form.
Current Stable Version :- 2.15.0
Project Homepage
http://oss.oetiker.ch/mrtg/
Cacti
Cacti is a complete network graphing solution designed to harness the power of RRDTool’s data storage and graphing functionality. Cacti provides a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive,easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices.
Current Stable Version :- 0.8.6i
Project Homepage
http://www.cacti.net
Regarding MRTG and Cacti i am going write detailed articles in future
Monday, November 23, 2009
Friday, November 6, 2009
Zend Performance
Let me first start off by saying that the Zend Framework has been very good to us.
It enabled us to build a kick-ass application in a relatively short amount of time. On top of that, we followed the conventions from Zend and PEAR and essentially have a very maintainable piece of software which I don't hate looking at every day (which is as one can imagine, a huge plus).
The other day our servers were overwhelmed with the rising traffic and I started profiling my application through Xdebug. Initially I tried to use Zend Studio and the Zend_Debugger but Zend doesn't like my (awesome) operating system (FreeBSD) and only provides Linux and Windows extensions. Xdebug, while being free and awesome in general, doesn't know this prejudice. :-)
On this project we currently run with 100,000 visitors per day on average, our peak is Sunday night where we get a ton more traffic than usually. We run the latest PHP (5.2.6 at this time), etc.. The software comes from FreeBSD ports, there are no magic secret patches. I'm picky about the modules I compile and load but the list is far from optimized.
To our defense, we just relaunched over summer and since we are a team of four total and only two of those four people write code. Since we started off slow with 60,000-80,000 visitors per day since summer, we never really had a chance or need to optimize and tried to avoid all premature optimization.
We currently use 50-some Zend-classes. I wish I could provide a better number, but as you may know, the Zend Framework is only bundled as a whole and figuring out which classes are all in the mix is tricky. So the 50 is an estimation based on grepping through our own class code mostly.
On the server we run Apache 1.3. Currently we have a total of four webservers (two older (dual core, 6 GB RAM, slow disks (7.5k rpm)), two newer models (eight cores and 6 GB RAM, faster disks (15k rpm)). The backend consists of two powerful workhouses with eight cores, more RAM (than the frontends) and a lot of disk (at 12k rpm each).
Prior to starting the quest for performance one of our older servers was able to handle ten (10) requests/second at peaks, now we are at 42 requests/second. (Give or take a few.) In regard to page loading time, we went with a few optimizations from 340 ms to 76 ms in no time (all figures according to Xdebug). So I feel like we are right on the right track to Getting rich with PHP. (Where's my Lexus at? :-))
We benchmarked using Apache Bench (at moderate ab -n 1000 -c 100 http://url) and Siege, which are both really awesome tools and provide you with an instant DoS attack on your servers. I might add that you are better off running those tools from "localhost" vs. remote as you might trigger your providers IDS/snort/DoS protection otherwise.
Here are a few things, that helped us. Suggestions are in no particular order and I should add that whatever is applicable for my suitation, doesn't have to work for you. Also, my number game could be off and if you have suggestions on how to improve, please comment or drop me a line.
1) APC
It cannot be stressed enough. Please run APC. Please take into account to adjust the default settings, also check apc.slam_defense, apc.write_lock and apc.stat.
We had APC before, but I felt like I needed to mention it on the list of things.
Also, apc_fetch, apc_store etc. are great ways to add little caches throughout your application.
And they almost require zero time to implement. I may suggest you use apc_fetch/apc_store directly vs. wrapping the Zend_Cache layer around it which provides (IMHO) little added value and benefit but just adds more class code around the obvious.
2) Adjust PHP's realpath.cache setting and .ttl
This helps, somewhat.
3) Get rid off require_once, use __autoload (and the Zend_Loader)
This might be a hassle when during development, because require_once evaluates each include, thus letting you know if it finds a parse error, and also where.
With include_once (which Zend_Loader essentially uses), it's a bit tricky at times. A good idea here would be a phing task (or some other script) which strips out or replaces require_once when you deploy your application to production.
Removing require_once in favour of __autoload shows one of the biggest performance improvements in my entire application - I shaved off roughly 220 milliseconds by removing about 15 (or so) calls to require_once in my bootstrap.php file. And that's with APC enabled, and a decent sized realpath.cache (and .ttl).
Beyond weird coding conventions (I shall bitch about those in another blog post), require_once is also the number one performance killer from the entire Zend_* code base. The before/after is amazing. Without any of those enhancements from the list just by stripping out require_once from our ZendFramework "install", we went from 9-10 requests/second to 27 requests/second.
Use the following shellscript to strip them:
grep -rl require_once . | grep -v svn |grep -v Loader | xargs perl -pi~ -e 's/require_once/#require_once/'
3) Zend_Loader
I know, I just recommended using the Zend_Loader but with no offense, the Zend_Loader sucks is not so great when it comes to general performance. Obviously I did not write it and really no offense meant, but it does some really weird stuff on the inside which I am not sure what the use-case is. But I am sure there is one. ;-)
In order to preserve the API, I extended Zend_Loader and started overwriting functions such as Zend_Loader::_securityCheck(), which runs a regular expression on the name of each file you feed to __autoload/Zend_Loader.
On top of that I switched to usig the Zend_Loader only for models and controllers. But not for Zend_ and Company_-classes. Since Zend (and we) essentially follow the great PEAR coding standard in regard to one class per file and a very explicit naming scheme, all you have to do in your __autoload is the following:
function __autoload($className) {
include_once $className = str_replace('_', '/', $className) . '.php';
}
Now, that would be the bare minimum and our loader looks slightly more complicated but I haven't stopped there and we are still in the process of "dumbing" it down even further, but so far it saved us between five and 15 ms per page.
4) Cache DB results, avoid queries!
Those tricky, tricky DB queries.
Even though our DB backends idle mostly even when we get beat with traffic, there's a few things to keep in mind.
One of them is - DB queries are really expensive. And by queries I am not talking about the "SELECT * FROM foo"-part, but rather about opening a connection to another server, sending the query, receiving it and so on. Let alone by caching one of those, we roughly gained another 20 ms on the frontpage. And it's not a very complex query either.
I remember looking puzzled when ahem... I was presented with the code that pulls a status message on each request to the homepage but I had forgotten about this already and just noticed it again when it popped up in xDebug with a notable amound of milliseconds.
5) Zend_Db_Table
Zend_Db_Table is very easy to use, in fact most of our models wrap around a couple tables and that's why we got a bunch of them. Now what I did not realize (but thanks to JamesG@#zftalk now I do), is that the meta data the class uses to provide all those nifty interfaces is generated on each request. That's a DESCRIBE TABLE in the background, which is pure overhead.
Zend_Db_Table_Abstract::setDefaultMetadataCache() to the rescue.
5) Apache
5a)
I sometimes hate Apache, but I also can't live without it.
Over the past years, I have tried all sorts of things in the webserver market - Lighttpd and nginx with php-cgi (fastcgi) seem to be no fun. A commercial solution such as Resin or Zeus has never been an option either.
I've always come back to Apache (1.3) for the simple fact that Apache and PHP are really so tightly integrated that nothing ever will go wrong.
Remember that guy Nik who claimed that Apache/PHP sometimes fail and deliver the sourcecode to the browser (because Facebook obviously failed to configure Apache)? Well, that doesn't happen - ever. The only problem with Apache is that Apache and client(browser)-communication is a bitch.
Nginx to the rescue! Fast install, easy to configure (don't let the Russian FAQ scare you, the nginx.conf-dist will teach you all you need!) - just chain your Apache to localhost:8080 and let Nginx proxy all requests to it and your Apaches move from "lockf" status, to "run" and "accept" always.
Whenever Apache receives a request of a slower client, it will have to wait until the slower client is done reading all of the response. While waiting, your 30 MB Apache sits there unable to do anything else. Which nginx in the mix, the Apache sends the response as fast as it can to nginx, thus having more time to take care of what it's supposed to do for you - PHP.
Judging from my poor benchmarks, nginx adds to the number of requests by factor six or seven (6 or 7). It's amazying and I never expected it to have such a great impact. It also doesn't eat away on resources, so beware of the Russians! :-)
Take all "optimizations" into account, Apache 1.3, proxied by Nginx can now handle over 3000 requests/second (ab -n 10000 -c 1000 http://url).
5b)
The obvious quirks, for example check out your default Apache install and unload all the modules and extensions you never use anyway.
For example, we don't have any of those HTTP authentication boxes ever. So why do we need *_auth_* modules. Then, we don't use a user_dir, why load mod_userdir, our Apache does not log - why load mod_config_log, or my most favorite: mod_status.
Make sure mod_status is really disabled because otherwise that's one very, very expensive operation you got right there, with each request.
A good idea is to check top, unload, and look again:
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
42601 root 1 96 0 7092K 2324K CPU1 1 0:00 1.00% top
36242 www 1 20 0 114M 33304K lockf 1 0:22 0.29% httpd
38251 www 1 20 0 114M 32184K lockf 0 0:08 0.29% httpd
42579 www 1 20 0 114M 28016K lockf 1 0:02 0.29% httpd
37975 www 1 20 0 115M 34688K lockf 1 0:18 0.24% httpd
36344 www 1 20 0 115M 34036K lockf 1 0:18 0.24% httpd
Take into account (thanks, Jan), that the size column, is not the real size. It's like the theoretical size including whatever the Apache could use if it had to, but you want to look at RES (resident) instead. Because that's what's in the memory right now.
Another smart move is to put all the rules from .htaccess into your server configuration because otherwise Apache searches for various (!) .htaccess files with each requests and tries to evaluate the rules you have in there.
Imagine this:
/htdocs/foo/bar/index.html
In this request Apache will look for .htaccess in the following directories:
/htdocs
/htdocs/foo
/htdocs/foo/bar
Turn it off (AllowOverride none) instead and move on because when you move all your directives into httpd.conf (or similar) at least they get evaluated once when the server process starts up.
From a deployment perspective it's nicer to have .htaccess because all you need to do is re-deploy one file vs. editing a server config and restarting a server, but this really pays off. With certain APC settings, you will need to restart the server anyway, also, "No pain, no gain!".
It enabled us to build a kick-ass application in a relatively short amount of time. On top of that, we followed the conventions from Zend and PEAR and essentially have a very maintainable piece of software which I don't hate looking at every day (which is as one can imagine, a huge plus).
The other day our servers were overwhelmed with the rising traffic and I started profiling my application through Xdebug. Initially I tried to use Zend Studio and the Zend_Debugger but Zend doesn't like my (awesome) operating system (FreeBSD) and only provides Linux and Windows extensions. Xdebug, while being free and awesome in general, doesn't know this prejudice. :-)
On this project we currently run with 100,000 visitors per day on average, our peak is Sunday night where we get a ton more traffic than usually. We run the latest PHP (5.2.6 at this time), etc.. The software comes from FreeBSD ports, there are no magic secret patches. I'm picky about the modules I compile and load but the list is far from optimized.
To our defense, we just relaunched over summer and since we are a team of four total and only two of those four people write code. Since we started off slow with 60,000-80,000 visitors per day since summer, we never really had a chance or need to optimize and tried to avoid all premature optimization.
We currently use 50-some Zend-classes. I wish I could provide a better number, but as you may know, the Zend Framework is only bundled as a whole and figuring out which classes are all in the mix is tricky. So the 50 is an estimation based on grepping through our own class code mostly.
On the server we run Apache 1.3. Currently we have a total of four webservers (two older (dual core, 6 GB RAM, slow disks (7.5k rpm)), two newer models (eight cores and 6 GB RAM, faster disks (15k rpm)). The backend consists of two powerful workhouses with eight cores, more RAM (than the frontends) and a lot of disk (at 12k rpm each).
Prior to starting the quest for performance one of our older servers was able to handle ten (10) requests/second at peaks, now we are at 42 requests/second. (Give or take a few.) In regard to page loading time, we went with a few optimizations from 340 ms to 76 ms in no time (all figures according to Xdebug). So I feel like we are right on the right track to Getting rich with PHP. (Where's my Lexus at? :-))
We benchmarked using Apache Bench (at moderate ab -n 1000 -c 100 http://url) and Siege, which are both really awesome tools and provide you with an instant DoS attack on your servers. I might add that you are better off running those tools from "localhost" vs. remote as you might trigger your providers IDS/snort/DoS protection otherwise.
Here are a few things, that helped us. Suggestions are in no particular order and I should add that whatever is applicable for my suitation, doesn't have to work for you. Also, my number game could be off and if you have suggestions on how to improve, please comment or drop me a line.
1) APC
It cannot be stressed enough. Please run APC. Please take into account to adjust the default settings, also check apc.slam_defense, apc.write_lock and apc.stat.
We had APC before, but I felt like I needed to mention it on the list of things.
Also, apc_fetch, apc_store etc. are great ways to add little caches throughout your application.
And they almost require zero time to implement. I may suggest you use apc_fetch/apc_store directly vs. wrapping the Zend_Cache layer around it which provides (IMHO) little added value and benefit but just adds more class code around the obvious.
2) Adjust PHP's realpath.cache setting and .ttl
This helps, somewhat.
3) Get rid off require_once, use __autoload (and the Zend_Loader)
This might be a hassle when during development, because require_once evaluates each include, thus letting you know if it finds a parse error, and also where.
With include_once (which Zend_Loader essentially uses), it's a bit tricky at times. A good idea here would be a phing task (or some other script) which strips out or replaces require_once when you deploy your application to production.
Removing require_once in favour of __autoload shows one of the biggest performance improvements in my entire application - I shaved off roughly 220 milliseconds by removing about 15 (or so) calls to require_once in my bootstrap.php file. And that's with APC enabled, and a decent sized realpath.cache (and .ttl).
Beyond weird coding conventions (I shall bitch about those in another blog post), require_once is also the number one performance killer from the entire Zend_* code base. The before/after is amazing. Without any of those enhancements from the list just by stripping out require_once from our ZendFramework "install", we went from 9-10 requests/second to 27 requests/second.
Use the following shellscript to strip them:
grep -rl require_once . | grep -v svn |grep -v Loader | xargs perl -pi~ -e 's/require_once/#require_once/'
3) Zend_Loader
I know, I just recommended using the Zend_Loader but with no offense, the Zend_Loader sucks is not so great when it comes to general performance. Obviously I did not write it and really no offense meant, but it does some really weird stuff on the inside which I am not sure what the use-case is. But I am sure there is one. ;-)
In order to preserve the API, I extended Zend_Loader and started overwriting functions such as Zend_Loader::_securityCheck(), which runs a regular expression on the name of each file you feed to __autoload/Zend_Loader.
On top of that I switched to usig the Zend_Loader only for models and controllers. But not for Zend_ and Company_-classes. Since Zend (and we) essentially follow the great PEAR coding standard in regard to one class per file and a very explicit naming scheme, all you have to do in your __autoload is the following:
function __autoload($className) {
include_once $className = str_replace('_', '/', $className) . '.php';
}
Now, that would be the bare minimum and our loader looks slightly more complicated but I haven't stopped there and we are still in the process of "dumbing" it down even further, but so far it saved us between five and 15 ms per page.
4) Cache DB results, avoid queries!
Those tricky, tricky DB queries.
Even though our DB backends idle mostly even when we get beat with traffic, there's a few things to keep in mind.
One of them is - DB queries are really expensive. And by queries I am not talking about the "SELECT * FROM foo"-part, but rather about opening a connection to another server, sending the query, receiving it and so on. Let alone by caching one of those, we roughly gained another 20 ms on the frontpage. And it's not a very complex query either.
I remember looking puzzled when ahem... I was presented with the code that pulls a status message on each request to the homepage but I had forgotten about this already and just noticed it again when it popped up in xDebug with a notable amound of milliseconds.
5) Zend_Db_Table
Zend_Db_Table is very easy to use, in fact most of our models wrap around a couple tables and that's why we got a bunch of them. Now what I did not realize (but thanks to JamesG@#zftalk now I do), is that the meta data the class uses to provide all those nifty interfaces is generated on each request. That's a DESCRIBE TABLE in the background, which is pure overhead.
Zend_Db_Table_Abstract::setDefaultMetadataCache() to the rescue.
5) Apache
5a)
I sometimes hate Apache, but I also can't live without it.
Over the past years, I have tried all sorts of things in the webserver market - Lighttpd and nginx with php-cgi (fastcgi) seem to be no fun. A commercial solution such as Resin or Zeus has never been an option either.
I've always come back to Apache (1.3) for the simple fact that Apache and PHP are really so tightly integrated that nothing ever will go wrong.
Remember that guy Nik who claimed that Apache/PHP sometimes fail and deliver the sourcecode to the browser (because Facebook obviously failed to configure Apache)? Well, that doesn't happen - ever. The only problem with Apache is that Apache and client(browser)-communication is a bitch.
Nginx to the rescue! Fast install, easy to configure (don't let the Russian FAQ scare you, the nginx.conf-dist will teach you all you need!) - just chain your Apache to localhost:8080 and let Nginx proxy all requests to it and your Apaches move from "lockf" status, to "run" and "accept" always.
Whenever Apache receives a request of a slower client, it will have to wait until the slower client is done reading all of the response. While waiting, your 30 MB Apache sits there unable to do anything else. Which nginx in the mix, the Apache sends the response as fast as it can to nginx, thus having more time to take care of what it's supposed to do for you - PHP.
Judging from my poor benchmarks, nginx adds to the number of requests by factor six or seven (6 or 7). It's amazying and I never expected it to have such a great impact. It also doesn't eat away on resources, so beware of the Russians! :-)
Take all "optimizations" into account, Apache 1.3, proxied by Nginx can now handle over 3000 requests/second (ab -n 10000 -c 1000 http://url).
5b)
The obvious quirks, for example check out your default Apache install and unload all the modules and extensions you never use anyway.
For example, we don't have any of those HTTP authentication boxes ever. So why do we need *_auth_* modules. Then, we don't use a user_dir, why load mod_userdir, our Apache does not log - why load mod_config_log, or my most favorite: mod_status.
Make sure mod_status is really disabled because otherwise that's one very, very expensive operation you got right there, with each request.
A good idea is to check top, unload, and look again:
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
42601 root 1 96 0 7092K 2324K CPU1 1 0:00 1.00% top
36242 www 1 20 0 114M 33304K lockf 1 0:22 0.29% httpd
38251 www 1 20 0 114M 32184K lockf 0 0:08 0.29% httpd
42579 www 1 20 0 114M 28016K lockf 1 0:02 0.29% httpd
37975 www 1 20 0 115M 34688K lockf 1 0:18 0.24% httpd
36344 www 1 20 0 115M 34036K lockf 1 0:18 0.24% httpd
Take into account (thanks, Jan), that the size column, is not the real size. It's like the theoretical size including whatever the Apache could use if it had to, but you want to look at RES (resident) instead. Because that's what's in the memory right now.
Another smart move is to put all the rules from .htaccess into your server configuration because otherwise Apache searches for various (!) .htaccess files with each requests and tries to evaluate the rules you have in there.
Imagine this:
/htdocs/foo/bar/index.html
In this request Apache will look for .htaccess in the following directories:
/htdocs
/htdocs/foo
/htdocs/foo/bar
Turn it off (AllowOverride none) instead and move on because when you move all your directives into httpd.conf (or similar) at least they get evaluated once when the server process starts up.
From a deployment perspective it's nicer to have .htaccess because all you need to do is re-deploy one file vs. editing a server config and restarting a server, but this really pays off. With certain APC settings, you will need to restart the server anyway, also, "No pain, no gain!".
Subscribe to:
Posts (Atom)