A Story of an Individual PHP Composer Dependency Heck and how to Exorcise Said Heck

I checked some code into pestle (my PHP command-line framework and Magento 2 code generation tool) for the first time in a bit, and was greeted with a broken travis build (for PHP 5.6).

Digging into the problem, I saw a sight that’s become increasingly familiar to PHP developers in recent years

$ composer install
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file

Your requirements could not be resolved to an installable set of packages.
Problem 1
    - Installation request for phpdocumentor/reflection-docblock 4.1.1 -> satisfiable by phpdocumentor/reflection-docblock[4.1.1].
    - phpdocumentor/reflection-docblock 4.1.1 requires php ^7.0 -> your PHP version (5.6.31) does not satisfy that requirement.
Problem 2
    - phpdocumentor/reflection-docblock 4.1.1 requires php ^7.0 -> your PHP version (5.6.31) does not satisfy that requirement.
    - phpspec/prophecy 1.7.3 requires phpdocumentor/reflection-docblock ^2.0|^3.0.2|^4.0 -> satisfiable by phpdocumentor/reflection-docblock[4.1.1].
    - Installation request for phpspec/prophecy 1.7.3 -> satisfiable by phpspec/prophecy[1.7.3].

Travis couldn’t get past composer install for the 5.6 test branch. The error messages above tell me that

1. Version 4.1.1 of the phpdocumentor/reflection-docblock package requires PHP 7.0 or above

2. Version 1.7.3 of phpspec/prophecy requires version 4.1.1 phpdocumentor/reflection-docblock

On one hand — it’s pretty handy that composer tells me which of my packages it’s having trouble with. On the other hand — I never added the phpdocumentor/reflection-docblock or phpspec/prophecy packages to my project, so what gives?

What gives is this: Some other package that I’ve required has these packages as a dependency. Fortunately, the composer depends command will tell me which package causes another package to be required. For example, ask composer why it wants phpdocumentor/reflection-docblock

$ composer depends phpdocumentor/reflection-docblock
phpspec/prophecy  1.7.3  requires  phpdocumentor/reflection-docblock (^2.0|^3.0.2|^4.0)  

and composer tells you that phpspec/prophecy wants phpdocumentor/reflection-docblock. Well, phpspec/prophecy still doesn’t look like anything I’ve added to my project, so run composer depends again with phpspec/prophecy

$ composer depends phpspec/prophecy
phpunit/phpunit  4.8.36  requires  phpspec/prophecy (^1.3.1)           

Ah ha! The phpunit/phpunit package is a package I’ve added to my project. We’ve found the culprit.

What to Do

I deliberately chose an old and stable version of PHPUnit to avoid these sorts of problems. So, the next question: What did I do wrong, and how can I fix this?

If you take a look at PHPUnit’s composer.json file at the 4.8.36 tag you’ll see the PHP version is fixed at 5.3.3 or greater.

//File: composer.json
"require": {
    "php": ">=5.3.3",

So, it’s the intent of PHPUnit that version 4.8.36 stay usable for older version of PHP. No problem there.

The problem with this composer.json lies here.

//File: composer.json
"require": {
    /* ... */
    "phpspec/prophecy": "^1.3.1",
     /* ... */
}

This may look like PHPUnit’s asking for version 1.3.1 of phpspec/prophecy, but its not. The leading ^ means this line is asking for

the most recent version of phpspec/prophecy greater than or equal to 1.3.1, and less then (but not equal to) 2.0.0

At the time my broken travis build ran, this was version 1.7.3. If we look at this version’s composer.json file

"require": {
    /* ... */
    "phpdocumentor/reflection-docblock": "^2.0|^3.0.2|^4.0",
    /* ... */
},

We see they’re pulling in phpdocumentor/reflection-docblock with the ^2.0|^3.0.2|^4.0 SemVer string. While this allows a wide variety of phpdocumentor/reflection-docblock versions, it causes a request for phpspec/prophecy:1.7.3 to ask for the most up-to-date version of phpdocumentor/reflection-docblock in the 4.x branch, which is phpdocumentor/reflection-docblock:4.1.1.

When we take a look at phpdocumentor/reflection-docblock:4.1.1‘s composer.json file

"require": {
    "php": "^7.0",
    /* ... */
},          

we see a minimum PHP version 7.0 — i.e. pestle’s PHP 5.6 test build is rejected outright.

The Fix

All of the above took me a few hours of on-and-off weekend development to trace out. At a certain point, I did a reality check — the build from a few weeks back was fine, and when I looked at the versions of things composer downloaded there I saw

- Installing phpspec/prophecy (v1.7.0): Downloading (100%) 

i.e. phpspec/prophecy:1.7.0 seemed fine. So, a quick hard coding of 1.7.0 in my composer.json fixed things up. The mind exercise of Why did this beak now but not then remains a powerful debugging technique.

The phpspec/prophecy:v1.7.0 version worked for me because its composer.json file

"require": {
    /* ... */
    "phpdocumentor/reflection-docblock": "^2.0|^3.0.2",
    /* ... */
},

didn’t want a 4.x version of phpdocumentor/reflection-docblock, and phpspec/prophecy:1.7.0 was acceptable to PHPUnit’s "phpspec/prophecy": "^1.3.1" dependency string.

Even though our problem is solved — something’s still not quite right. First, why did my build from 15 days ago grab phpspec/prophecy:1.7.0 even though phpspec/prophecy:1.7.0 was released well before that? Also — composer’s supposed to take the version of PHP you’re running into account when it builds out a dependency tree. In fact, if use the older version of PHP 5 that ships with my Apple laptop, composer is smart enough to grab the older versions of the packages that are supported

$ /usr/bin/php -v
PHP 5.5.36 (cli) (built: May 29 2016 01:07:06) 
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2015 Zend Technologies

$ /usr/bin/php ~/bin/composer require phpunit/phpunit:4.8.36
./composer.json has been created
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 21 installs, 0 updates, 0 removals
  - Installing symfony/yaml (v3.3.13): Loading from cache
  - Installing sebastian/version (1.0.6): Loading from cache
  - Installing sebastian/global-state (1.1.1): Loading from cache
  - Installing sebastian/recursion-context (1.0.5): Loading from cache
  - Installing sebastian/exporter (1.2.2): Loading from cache
  - Installing sebastian/environment (1.3.8): Loading from cache
  - Installing sebastian/diff (1.4.3): Loading from cache
  - Installing sebastian/comparator (1.2.4): Loading from cache
  - Installing doctrine/instantiator (1.0.5): Loading from cache
  - Installing phpunit/php-text-template (1.2.1): Loading from cache
  - Installing phpunit/phpunit-mock-objects (2.3.8): Loading from cache
  - Installing phpunit/php-timer (1.0.9): Loading from cache
  - Installing phpunit/php-file-iterator (1.4.5): Loading from cache
  - Installing phpunit/php-token-stream (1.4.11): Loading from cache
  - Installing phpunit/php-code-coverage (2.2.4): Loading from cache
  - Installing webmozart/assert (1.2.0): Loading from cache
  - Installing phpdocumentor/reflection-common (1.0.1): Loading from cache
  - Installing phpdocumentor/type-resolver (0.3.0): Loading from cache
  - Installing phpdocumentor/reflection-docblock (3.2.2): Loading from cache
  - Installing phpspec/prophecy (1.7.3): Loading from cache
  - Installing phpunit/phpunit (4.8.36): Loading from cache
symfony/yaml suggests installing symfony/console (For validating YAML files using the lint command)
sebastian/global-state suggests installing ext-uopz (*)
phpunit/php-code-coverage suggests installing ext-xdebug (>=2.2.1)
phpunit/phpunit suggests installing phpunit/php-invoker (~1.1)
Writing lock file
Generating autoload files

So we’re back to what gives? Why is the PHP on my travis box trying to grab packages that are too new for it? The real culprit is my composer.lock file. Composer generates a composer.lock file whenever you run composer install or composer update (with new packages), and it contains the exact version (no SemVer strings) of each package installed. The intention of the the composer.lock file is that you commit it to source control, and then anyone (human or deployment robot) that grabs your install gets the exact same packages as you. The lock files always wins.

While this works great for keeping applications you deploy to servers in sync — it presents a bit of a sticky wicket for someone like me whose trying to provide an application (pestle) that other people with unknown versions of PHP can download and use. Because I ran composer update on a machine with PHP 7.0, my lock files has packages that were not suitable (and would not have been downloaded) for PHP 5.6. When travis ran composer install with my lock file, it dutifully tried to grab phpspec/prophecy:1.7.3, saw that it couldn’t use it, and then gave up.

When I changed my composer.json to use a hard coded phpspec/prophecy hard coded to 1.7.0 and ran composer update again, my composer.lock file was also updated, and travis was happy again. There’s probably a case to be made that a “runs in multiple versions of PHP” tool like pestle shouldn’t commit its composer.lock file — but for now the hard coded version has the build system happy while I think about longer term plans.

Software vs. Service

Composer is probably the best thing to happen to PHP in the past five years — but the semi-stable nature of these large dependency trees and a cultural I’m not sure how all this works but yolo approach to semantic versioning means PHP projects that adopt composer are shifting away from being stable and known pieces of software and towards being services that may or may not run on any given day.

Whether that’s a good thing or a bad thing probably depends on what sort of software you’re making.