N98-magerun: Writing Tests

Test driven development is all the rage in certain PHP circles (while rage inducing code is all the rage in other PHP circles). Today we’re going to cover using the PHPUnit testing tools that ship with n98-magerun, as well as cover some programatic testing basics.

Creating a test in n98-magerun is simple. First, make sure you’ve installed the composer dev package.

$ composer.phar install --dev

The regular composer installation gives you all the tools you need to run a project. Installing the “dev” package will give you all the tools you need to work on a project. This includes a full phpunit package, which is the testing framework we’ll be using.

While test driven development can get pretty complicated and esoteric, the core idea of testing is super simple. You run a program. It performs a small task. Then it checks that the small task did what you thought it should. A test based workflow takes the mental model we already use to develop code, but creates an artifact about our assumptions that the computer will run later for us.

Eek! Talk about esoteric: Let’s get into some concrete examples so you can see how simple it is. First, create the following file with the following contents

#File: tests/N98/Magento/Command/FirstTest.php 
<?php
namespace N98MagentoCommand;
use N98MagentoCommandPHPUnitTestCase;
class FirstTest extends TestCase
{
    public function testMaths()
    {    
        $result = 2+2;
        $this->assertEquals($result, 5);
    }
}

This is your first test case. It contains a single test, testMaths. To run your test, use the bundled phpunit

$ vendor/bin/phpunit tests/N98/Magento/Command/FirstTest.php 

After running the above, you should see the following output.

Configuration read from /path/to/n98-magerun/phpunit.xml

F

Time: 0 seconds, Memory: 7.25Mb

There was 1 failure:

1) N98MagentoCommandFirstTest::testMaths
Failed asserting that 5 matches expected 4.

/Users/alanstorm/Documents/github_netz98/n98-magerun/tests/N98/Magento/Command/FirstTest.php:9

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

Congratulations, you just wrote you first failing test.

Anatomy of a Test

The PHP class you created is known as a test case. Each test case contains multiple, individual tests.

namespace N98MagentoCommand;

use N98MagentoCommandPHPUnitTestCase;

class FirstTest extends TestCase
{
    public function testMaths()
    {    
        $result = 2+2;
        $this->assertEquals($result, 5);
    }
}

The namespace is a n98-magerun convention (following a Symfony convention). Our full class name is N98MagentoCommandFirstTest. Test case classes should end in the word Test. A test case extends the class N98MagentoCommandPHPUnitTestCase, which has been aliased into the local namespace as TestCase. By giving our class the proper name and extending this base TestCase class, we’ve told the unit testing framework that this class is a test case.

As we said earlier, an individual test case contains multiple tests. Each method of our class that starts with the word test will be interpreted as a test. Our current class contains a single Maths test

public function testMaths()
{    
    $result = 2+2;
    $this->assertEquals($result, 5);
}

As we said earlier, a test performs a simple task, and then checks if the task completed properly. Here, we’ve

1. Performed the task of adding two and two

2. Checked (Asserted) the result was equal to 5

Of course, 2+2 only equals 5 if Gul Madred is in the room. We deliberately wrote a failing test so you could see what a failure looked like. Let’s change things so we’re testing that 2+2 equals 4.

public function testMaths()
{    
    //...
    $this->assertEquals($result, 4);
}

With the above in place, re-run your test

$ vendor/bin/phpunit tests/N98/Magento/Command/FirstTest.php 
PHPUnit 3.7.21 by Sebastian Bergmann.

Configuration read from /oath/to/n98-magerun/phpunit.xml

.

Time: 0 seconds, Memory: 7.00Mb

OK (1 test, 1 assertion)

This time, we’ve ended up with much different results. The single F (which indicated failure) has been replaced with a . (indicating a test passed).

The final test results

OK (1 test, 1 assertion)

indicated our test case passed all its tests. Of course, “all its tests” was just one test. Let’s add another

class FirstTest extends TestCase
{
    //...
    public function testExponents()
    {
        $result = 2^3;
        $this->assertEquals(8,$result);
    }        
}

If we run our tests again

$ vendor/bin/phpunit tests/N98/Magento/Command/FirstTest.php 
#...

.F

Time: 0 seconds, Memory: 7.00Mb

There was 1 failure:

1) N98MagentoCommandFirstTest::testExponents
Failed asserting that 8 matches expected 1.

/Users/alanstorm/Documents/github_netz98/n98-magerun/tests/N98/Magento/Command/FirstTest.php:15

FAILURES!
Tests: 2, Assertions: 2, Failures: 1.

We can see our new test failed. Again, the F

.F

indicates a single test failed, and the report tells us why/which test

There was 1 failure:

1) N98MagentoCommandFirstTest::testExponents
Failed asserting that 8 matches expected 1.

Why did our test fail? Because the ^ operator isn’t PHP’s exponent operator, it’s a bitwise operator. Let’s correct our test.

public function testExponents()
{
    $result = pow(2,3);
    $this->assertEquals($result, 8);
}    

Run the suite with the above in place, and you’ll see more favorable results.

Testing the Application

We could test PHP math functions all day — but that’s not why we’re here. We want to test the functionality of the n98-magerun command line tool. Your mind may be jumping ahead, trying to figure out the best way to run a command from your PHP test case. Fortunately, the n98-magenrun team already has you covered. Give the following new test a try

public function testListCommand()
{
    $command = $this->getApplication()->find('list');

    $commandTester = new CommandTester($command);
    $commandTester->execute(
        array(
            'command' => 'list'
        )
    );        
    $this->assertContains('sys:setup:compare-versions',$commandTester->getDisplay());
}    

The CommandTester class is a special n98-magerun class (actually, its a Symfony class provided by the n98-magerun team) provided to help users run tests against command line console applications. A good testing framework has tools like this in place to make testing code easy. If you need to perform coding gymnastics every-time you write a test, you may need to take a step back and ask yourself if there’s an easier way.

So, what we’ve done above is get a reference to a command object

$command = $this->getApplication()->find('list');

Used that command to instantiate a command tester

$commandTester = new CommandTester($command);

Executed the command

$commandTester->execute(
    array(
        'command' => 'list'
    )
);        

Then asserted that the command results ($command->getDisplay()) contained the text sys:setup:compare-versions (which we’d exptect to see running the list command). This would be equivalent to running the command

$ n98-magerun list

While it’s beyond the scope of this article, you can take a look at other tests in the tests/N98/Magento/Command/ folder to get an idea of how to run other commands with complicated argument strings. When starting on a new project, it’s always best to mimic the successful behavior of other code in the project.

Let’s try running our test case with the above in place

$ vendor/bin/phpunit tests/N98/Magento/Command/FirstTest.php 
PHPUnit 3.7.21 by Sebastian Bergmann.

Configuration read from /Users/alanstorm/Documents/github_netz98/n98-magerun/phpunit.xml

..PHP Fatal error:  Class 'N98MagentoCommandCommandTester' not found in

Whoops! PHP is complaining the CommandTester class doesn’t exist. That’s because we forgot to import it into the main namespace. Let’s take care of that. Add the following to the top of your page under the namespace declaration

use SymfonyComponentConsoleTesterCommandTester;

With the above in place, let’s run the tests again

$ vendor/bin/phpunit tests/N98/Magento/Command/FirstTest.php 
//...

..E

//... 

Well. That’s new. Instead of a ., or an F, our new test returned an E. The E standard for error, and meant PHP encountered an error that made it stop the test. An error state is different than a failure state. A test failure means our code ran, but did something unexpected. An error means our PHP code itself is bad. When PHPUnit, (and other testing frameworks), encounter a non-fatal error state, they’ll attempt to recover and continue running other tests in the case.

If we reveal a bit more of the testing output, we’ll see the error our code created

1) N98MagentoCommandFirstTest::testListCommand
RuntimeException: Please specify environment variable N98_MAGERUN_TEST_MAGENTO_ROOT with path to your test
magento installation!

Ah ha! The n98-magerun command line program expects a Magento installation to be present before running it. If you’ve ever tried to run a command outside a Magento directory, you’re familiar with this

$ cd /tmp
$ n98-magerun sys:check

  [RuntimeException]                    
  Magento folder could not be detected  

This means our test suite needs to know where Magento is located on our system. If you take a look at the readme included with the testing framework, you’ll find instructions on how to do this.

Set the environment variable N98_MAGERUN_TEST_MAGENTO_ROOT with a path to a magento installation which can be used to run tests.

i.e.

export N98_MAGERUN_TEST_MAGENTO_ROOT=/home/myinstallation

Before running real tests, we’ll need to point the shell environmental variable N98_MAGERUN_TEST_MAGENTO_ROOT to a Magento folder. In bash that looks like the following

export N98_MAGERUN_TEST_MAGENTO_ROOT=/path/to/magento

This is a common pattern with testing frameworks — it’s often necessary to perform a special setup routine to get your code running in a testing context vs. a real context. In well organized projects like n98-magerun this process is documented. In less organized environments, making the new programmer figure this out on their own is a common hazing ritual. That, however, is another topic for another time.

After setting our environmental variable, we should be able to run our new test (this run may take longer than the previous ones, as a Magento environment will be bootstrapped for the command runner)

$ vendor/bin/phpunit tests/N98/Magento/Command/FirstTest.php 
PHPUnit 3.7.21 by Sebastian Bergmann.

Configuration read from /Users/alanstorm/Documents/github_netz98/n98-magerun/phpunit.xml

...

Time: 9 seconds, Memory: 33.25Mb

OK (3 tests, 4 assertions)

That, in a nutshell, is how to write tests for n98-magerun. Next time we’ll wrap up the n98-magerun series with a tutorial that shows how I created the config:search command.