Benchmarking Autoload Performance In PHP

I read a blog post a few days ago written by Brandon Savage outlining some of the more worthwhile optimizations one can make to their PHP code and application environment in order to improve performance. One of the optimizations suggested by Brandon is employing the use of an autoloader (for the uninitiated, you can read all about autoloading here). On reading through the comments, I noticed that there seemed to be some uncertainty as to how autoloading stacks up against the use of multiple include files as it relates to speed. After running a quick Google search on the topic, I realized that the uncertainty extended far beyond Brandon’s article. I decided then, to do some benchmarking of my own and post the results.

First of all, let me categorically state that for reasons which I’ll reveal in a later article, I don’t care very much for benchmarks. These tests were carried out merely out of academic interest. Secondly, I am a fan of anything which makes my life easier. Autoloading makes my life easier. Enough said.

The Machine

These tests were carried out on a Sony Vaio notebook with 4GB of RAM and a core 2 duo processor with each core running at 2.0Ghz. The software versions are listed below:

• Windows 7 RC
• PHP 5.3.0
• Apache 2.2.12
• eAccelerator 0.9.6 RC1

The Tests

Two sets of tests were carried out, one without opcode caching and one with opcode caching (courtesy of eAccelerator). Each set of tests include tests using multiple include() calls, tests using the SPL autoloader and tests using the __autoload magic function.

The Code

The code used to perform the tests is extremely simple. It consists of a timer class as defined below:

class Timer 
{ 
private $totalTime; 
public function startTimer() { 
	$this->totalTime = microtime(true); 
} 
public function stopTimer() { 
	$this->totalTime = round(microtime(true)-$this->totalTime,8); 
} 
public function getTime() { 
	return $this->totalTime; 
} 
}

As you can see, the Timer class has Timer::startTimer() and Timer::stopTimer() methods which start and stop the timer respectively. Calls to these methods envelop the code which we are evaluating, i.e. the multiple includes or our autoload functions:

$timer = new Timer;
$timer->startTimer();
 
//Code block used to test multiple includes.
include 'includes/dummy.class.php';
include 'includes/dummytwo.class.php';
include 'includes/dummythree.class.php';
include 'includes/dummyfour.class.php';
include 'includes/dummyfive.class.php';
 
//Code block used to test SPL Autoloader.
function autoload($classname) {
 	$name = strtolower($classname);
 	include 'includes' . DIRECTORY_SEPARATOR . $name . '.class.php';
}
spl_autoload_register('autoload');
 
//Code block used to test __autoload()
function __autoload($classname) {
 	$name = strtolower($classname);
 	include 'includes' . DIRECTORY_SEPARATOR . $name . '.class.php';
}
 
$dummyOne = new Dummy;
$dummyTwo = new DummyTwo;
$dummyThree = new DummyThree;
$dummyFour = new DummyFour;
$dummyFive = new DummyFive;
 
$timer->stopTimer();
echo $timer->getTime();

The classes being included are identical save for their names and simply perform a mathematical addition 50 times each:

class Dummy 
{
public function __construct() {
	for($i=0;$i<50;$i++) {
		$r = $i + 1;
	}
}
}

For tests done while using eAccelerator, the relevant php.ini settings were as follows:

[eAccelerator] 
eaccelerator.shm_size = "0"
eaccelerator.enable = "1"
eaccelerator.debug = 0 
eaccelerator.check_mtime = "1"
eaccelerator.filter = ""
eaccelerator.shm_max = "0" 
eaccelerator.shm_ttl = "0" 
eaccelerator.shm_prune_period = "0"
eaccelerator.shm_only = "0"

The Results

Now onto the results. Take a look:

PHP 5.3.0 Without Opcode Caching

PHP 5.3.0 With Opcode Caching (eAccelerator)

The first set of tests indicate that autoloading is faster than multiple includes. The SPL autoloader wins here with the __autoload() function coming in second and multiple includes quite a way behind in third. The second set of tests however convey the complete opposite. Here we see multiple includes winning out by a clear margin with the __autoload() function coming in second and the SPL autoloader finishing third. I was quite interested to find out why using multiple includes is faster than autoloading when using an opcode caching system and so I did a bit of research. Apparently some opcode caching systems may have issues with caching autoloaded classes. I’m going to have to do a bit more digging to find out why exactly that is.

These tests can be modified and extended in a number of ways to improve the data collected. They could be carried out on Unix, Linux and Mac systems, they could be modified so they test against require(), require_once() and include_once() or they could be tested with differing classes. This data is simply a starting point and is subject to one of the biggest issues I have with benchmarks, they simply don’t effectively mirror real world systems.

Final Thoughts

We have one victory here for the pro-autoload developers and one for the anti-autoload developers. These tests and results however provide quite an effective distraction from the entire point of using an autoloader. Even if autoloading proved to be whole seconds slower than using multiple includes, it is a technique I’d still use. Religiously. Why you ask? The answer is simple. The benefits of using an autoloader are not centered around speed gains. It is about organizational efficiency. Autoloading allows you to write less lines of code, which in turn decreases the probability of bugs creeping up in your code. Think about it, writing less code equals less opportunity to mess up somewhere. Additionally, in cases where you might include 25 files of which only 5 are used autoloading would clearly prove to be the faster option. Somewhere in the region of 5 times faster I’m guessing. Cleaner, smaller and in some cases faster code. That is what autoloading offers. If these results teach us anything, it is that we should definitely use an opcode caching system on our servers. The difference in speed between the two sets of tests is incredible!

I am open to any comments or criticism regarding how these tests were carried out, and I welcome any data which validates or invalidates my own. I have yet to perform these tests on a Linux server, however once I get around to upgrading to PHP 5.3 on my Linux box I’ll be sure to rerun the tests and post the results. I hope that you found this article useful.

Tagged as: autoload, benchmarks, performance, PHP

Subscribe to the RSS feed! You know you want to ;)

Liked this post? Disliked it? Let people know! Share it!