Magento 2: Composer Plugins

In our last article, we talked a bit about Magento 2’s use of Composer, and touched on the Composer meta-package installation method. One high level take away was, when you use Composer’s create-project method to start a new Magento project, you are

1. Fetching the latest version of a specific Composer package (magento/project-community-edition) hosted at repo.magento.com

2. Running composer install for that project’s composer.json file

The curious among you may have noticed something strange about this. If we use the --no-install option to skip step #2 above (i.e. only fetch magento/project-community-edition),

$ composer create-project --no-install --repository-url=https://repo.magento.com/ magento/project-community-edition

we’ll see a pretty sparse project folder

$ ls project-community-edition
README.md    composer.json    update    

The README.md file is a basic Welcome to Magento affair, and the composer.json file contains the actual Composer packages you’ll need to install Magento. That’s why magento/project-community-edition is called a “meta” package — it’s not he actual package, and most software engineers aren’t english or philosophy majors and enjoy stretching the definition of “meta”.

The update folder contains a snapshot of the Magento Component Manager application (i.e Market Place updater), which is a separate project from the Magento core. Why this feature is a separate application, and why Magento distributes it like this is a story for another time.

All in all, relatively straight forward. However, after running composer install (or running create-project without the --no-install flag), we end up with the following

$ ls project-community-edition/
CHANGELOG.md                        dev
CONTRIBUTING.md                     index.php
CONTRIBUTOR_LICENSE_AGREEMENT.html  lib
COPYING.txt                         nginx.conf.sample
Gruntfile.js                        package.json
LICENSE.txt                         php.ini.sample
LICENSE_AFL.txt                     phpserver
README.md                           pub
app                                 setup
bin                                 update
composer.json                       var
composer.lock                       vendor

That’s a huge difference. A plethora of files and folders. Many of these files are necessary for Magento to run, others are informational, and still others are sample configurations. However, if you’re new to the Composer eco-system, you may be wondering

Where the heck did all these extra files and folders come from? I thought Composer would only update files in /vendor.

Today we’re going to explain how these files get here, and why you’ll need to be hyper aware of this as a Magento 2 developer. To start, we’ll need to dive into some less known features of Composer

Composer: Plugins and Scripts

Composer bills itself as a Dependency Manager for PHP. While this is true, and dependency management is an important part of a PHP project, Composer is really a foundational framework for PHP development, and serves the same role that linkers do in the C/C++ world.

Yes yes, I know, from a computer science point of view linkers and Composer couldn’t be further apart. However, the end result of a linker is, the C programmer stops needing to worry about how they incorporate code from other libraries into their program. In a similar way, Composer does the same thing for PHP — if a project conforms to what Composer expects in terms of directory structure and autoloading, and a PHP developer conforms to what Composer expects from a PHP program (i.e., includes the Composer autoloader), the developer stops needing to worry about how they should include other people’s code in their own systems.

When considered from this point of view — that Composer is, itself, just another programmatic framework that your code sits on top of — it makes more sense that Composer would have a plugin system for changing, altering, and extending its behavior. There are two main systems programmers have for altering the behavior of Composer. These systems are scripts, and plugins.

Scripts and plugins share a base set of concepts, but have a few key distinctions. Scripts provide a way, in the project composer.json file, to take additional programmatic action when composer triggers certain events. These events are listed in the Composer manual, and include things like pre-and-post composer install running.

Plugins, on the other hand, provide the same mechanism for individual packages that are part of a larger project. In addition to listening for Composer events, plugins also have the ability to modify composer’s installation behavior.

Put another way, you configure scripts in your main composer.json file, you (or third parties) configure plugins in composer.json files that live in the vendor/ folder.

While understanding both systems is important for a well rounded Composer developer, today we’re going to focus on the plugin system.

Plugin Example

Rather than try to describe things from scratch, we’ve created a simple Composer plugin that should demonstrate the plugin lifecycle, and help you understand what Magento 2 is doing with Composer plugins.

If you take a look at the plugin class

#File: src/Plugin.php
//...
public static function getSubscribedEvents()
{
    return array(
        'post-install-cmd' => 'installOrUpdate',
        'post-update-cmd' => 'installOrUpdate',            
    );
}    
//...    
public function installOrUpdate($event)
{
    file_put_contents('/tmp/composer.log', __METHOD__ . "\n",FILE_APPEND);
    file_put_contents('/tmp/composer.log', get_class($event) . "\n",FILE_APPEND);            
}

you can see that this plugin listens for the post-install-cmd and post-update-cmd events. You tell a plugin which events it should listen to by defining a getSubscribedEvent method that returns an array in the above format. Keys are the event, and values are the method, (on the plugin class), that Composer calls as an observer.

In our case, both the post install and post update events call the installOrUpdate method, and this method logs some simple information to the /tmp/composer.log file in our temp directory.

The plugin class also has an activate method.

#File: src/Plugin.php
public function activate(Composer $composer, IOInterface $io)
{
    file_put_contents('/tmp/composer.log', __METHOD__ . "\n",FILE_APPEND);
}

Composer calls the activate method when it detects the plugin every time Composer runs. The activate method is where you instantiate any other objects your plugin will need. In our case, we’ve added a line to log when the method is called.

All in all, a mostly useless plugin, but one that’s useful to diagnose how plugins work.

Adding a Plugin to Your Project

Adding a plugin to your project is the same as adding any other Composer package to your project. Create a new folder

$ mkdir test-plugin
$ cd test-plugin

and then create the following composer.json file in that folder.

//File: composer.json
{
    "repositories":[
        {
            "type":"vcs",
            "url":"git@github.com:astorm/composer-plugin-example.git"
        }
    ],
    "require":{
        "pulsestorm/composer-plugin-example":"0.0.1"
    }
}

In the require section we’ve added our plugin (pulsestorm/composer-plugin-example) and the desired version (0.0.1). The plugin’s name comes from the plugin’s composer.json file file. The version, 0.0.1, comes from the tagged releases.

Since I didn’t create a packagist.org listing for this package, we need the repositories section. This tells Composer to use the git (vcs/version control system) repository at the provided URL as a repository.

This Composer file in place, there’s one last thing we’ll want to do before we run composer install. In a separate terminal window, run the following.

$ touch /tmp/composer.log
$ tail -f /tmp/composer.log

This creates our composer.log file, and then tails it. Tailing a file means showing the last few lines of output. When we run tail with the -f option, we’re telling tail to show us the last line of the file whenever the file is changed. This is a decades old technique for monitoring log files in the *nix world.

Composer Plugin Lifecycle

OK! We’re ready to install our simple project. Run

$ composer install

and composer will install the plugin.

$ composer install
Loading composer repositories with package information
Updating dependencies (including require-dev)                       
  - Installing pulsestorm/composer-plugin-example (0.0.1)
    Loading from cache

Writing lock file
Generating autoload files

More interesting to us though is the output in our /tmp/composer.log file.

Pulsestorm\Composer\Example\Plugin::activate
Pulsestorm\Composer\Example\Plugin::installOrUpdate
Composer\Script\Event

Here, we see Composer called the activate method, and then (per our events) called the installOrUpdate method. If we were to run update

$ composer update

We’d see the same lines

Pulsestorm\Composer\Example\Plugin::activate
Pulsestorm\Composer\Example\Plugin::installOrUpdate
Composer\Script\Event    

because we’re also listening for the update event.

A Composer plugin developer can, (via the Composer\Script\Event object Composer passed to our handler or the Composer\Composer object Composer passes to the active method), examine and change Composer’s state at run time, implementing all sorts of extra functionality whenever a Composer project updates.

Covering that functionality in full is beyond the scope of this article, but with Composer being open source, there’s nothing stopping you from diving right in.

What Makes a Package a Plugin?

As we mentioned earlier, a Composer plugin is just a standard Composer package. However, it’s a standard Composer package with a special composer.json file. Let’s take a look at our plugin’s composer.json file.

//File: composer.json
{
    //...
    "type": "composer-plugin",
    //...        
    "require": {
        "composer-plugin-api": "^1.0"
    },
    "autoload":{
        "psr-4":{
            "Pulsestorm\\Composer\\Example\\":"src/"
        }
    },    
    "extra":{     
        "class":"Pulsestorm\\Composer\\Example\\Plugin"
    }
    //...
}

The first configuration a plugin package needs is the following

//File: composer.json

"type": "composer-plugin"

This tells Composer that this is a plugin package.

The second configuration a plugin package needs is

//File: composer.json

"require": {
    "composer-plugin-api": "^1.0"
},    

This looks like a standard Composer require — but it’s not. When Composer encounters a package named composer-plugin-api, this indicated which Plugin API version your plugin targets.

Finally, in the extra section, the following configuration

//File: composer.json

"extra":{     
    "class":"Pulsestorm\\Composer\\Example\\Plugin"
}

points to our plugin class (Pulsestorm\Composer\Example\Plugin). Since Composer will need to instantiate this class, that means you’ll need something in your autoload section that ensures PHP will load the class definition file. In our case, we used a standard PSR-4 autoloader

//File: composer.json

"autoload":{
    "psr-4":{
        "Pulsestorm\\Composer\\Example\\":"src/"
    }
}

That’s all you’ll need for a Composer plugin!

Finding Magento 2’s Composer Plugins

Now that we have a better understanding of Composer plugins, we can come back to our Magento problem. As a reminder, we’re trying to figure out how the stock create-project files

$ ls project-community-edition
README.md    composer.json    update 

become a full fledged, many files outside of vendor, Magento installation

$ ls project-community-edition/
CHANGELOG.md                        dev
CONTRIBUTING.md                     index.php
CONTRIBUTOR_LICENSE_AGREEMENT.html  lib
COPYING.txt                         nginx.conf.sample
Gruntfile.js                        package.json
LICENSE.txt                         php.ini.sample
LICENSE_AFL.txt                     phpserver
README.md                           pub
app                                 setup
bin                                 update
composer.json                       var
composer.lock                       vendor

If it’s not obvious by now, these additional files are placed here by a plugin in one of the Composer packages that make up Magento 2.

Unfortunately, Composer doesn’t provide an easy way to check your project for any installed plugins. You’ll need to use some good old fashioned unix command line searching to figure out which Magento packages have plugins.

In plain english, we’ll want to

1. Create a list of all our project’s composer.json files
2. Search those files for the all important "type": "composer-plugin", text

In unix english, that’s

$ find vendor/ -name composer.json | xargs grep 'composer-plugin'
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v1/composer.json:    "type": "composer-plugin",
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v1/composer.json:        "composer-plugin-api": "1.0.0"
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v2/composer.json:    "type": "composer-plugin",
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v2/composer.json:        "composer-plugin-api": "1.0.0"
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v3/composer.json:    "type": "composer-plugin",
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v3/composer.json:        "composer-plugin-api": "1.0.0"
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v4/composer.json:    "type": "composer-plugin",
vendor//composer/composer/tests/Composer/Test/Plugin/Fixtures/plugin-v4/composer.json:        "composer-plugin-api": "1.0.0"
vendor//magento/magento-composer-installer/composer.json:    "type":"composer-plugin",
vendor//magento/magento-composer-installer/composer.json:        "composer-plugin-api": "^1.0"

We can safely ignore the results in composer/composer/tests — these are tests in the main Composer package. The result we are interested in is

vendor//magento/magento-composer-installer/composer.json

It looks like the magento/magento-composer-installer package is actually a Composer plugin. If we take a look at the contents of this composer.json file

#File: vendor//magento/magento-composer-installer/composer.json 
{
    //...

    "type":"composer-plugin",

    //...

    "extra":{
        //...
        "class":"MagentoHackathon\\Composer\\Magento\\Plugin"
    }
}

We see the composer-plugin type-tag our command line searching found, as well as the required extra configuration that configures the MagentoHackathon\Composer\Magento\Plugin class as a plugin.

Without getting into the specific technical details, this is the plugin that installs those extra files at the root level, above the vendor folder. In short, the MagentoHackathon\Composer\Magento\Plugin will

1. Listen for composer install and composer update events

2. Look at the extra->map section(s) for any composer.json file in the just installed or updated composer vendor packages

3. Use that information to copy file from the installed package, to the root level project folder

If that didn’t make sense, let’s walk through it. First, let’s find any composer.json files with a "map" section.

$ find vendor/ -name composer.json | xargs ack '"map"'
vendor/magento/magento2-base/composer.json
75:        "map": [

vendor/magento/magento2-base/dev/tests/api-functional/_files/Magento/TestModuleIntegrationFromConfig/composer.json
12:    "map": [

vendor/magento/magento2-base/dev/tests/api-functional/_files/Magento/TestModuleJoinDirectives/composer.json
12:    "map": [

Again, we can safely ignore the files in the tests folder — this leaves us (at the time of this writing) with a single result in the magento/magento2-base package. If we look at a snippet of this file

//File: vendor/magento/magento2-base/composer.json
{
    "name": "magento/magento2-base",
    //...
    "extra": {
        //...
        "map": [
            [
                "lib/internal/Cm",
                "lib/internal/Cm"
            ],
            [
                "lib/internal/LinLibertineFont",
                "lib/internal/LinLibertineFont"
            ],
            [
                "lib/internal/Credis",
                "lib/internal/Credis"
            ],
            //...
            [
                "LICENSE_AFL.txt",
                "LICENSE_AFL.txt"
            ],
            [
                "vendor/.htaccess",
                "vendor/.htaccess"
            ]
        ]
    }
}

When the MagentoHackathon\Composer\Magento\Plugin finds the above map section, it will start running PHP code that’s roughly equivalent to

cp -r vendor/magento/magento2-base/lib/internal/Cm lib/internal/Cm
cp -r vendor/magento/magento2-base/lib/internal/LinLibertineFont lib/internal/LinLibertineFont

cp -r vendor/magento/magento2-base/lib/internal/Credis lib/internal/Credis

//...
cp vendor/magento/magento2-base/LICENSE_AFL.txt LICENSE_AFL.txt

cp vendor/magento/magento2-base/vendor/.htaccess vendor/.htaccess"

This is how the non-vendor files Magento needs to operate get from Magento core vendor packages into the root folder of your project.

History of magento/magento-composer-installer

Before we wrap up, it’s worth noting that the magento/magento-composer-installer Composer plugin is a fork of the original Magento 1 Composer installer plugin built at a Magento hackathon, promoted by Firegento, and maintained by Daniel Fahlke. The original goals of this plugin were to build a system that allowed developers to use Composer to fetch Magento 1 plugins into vendor, and then install them into a Magento 1 system via a number of different strategies. This was necessary since Magento 1 never officially adopted Composer.

The Magento core team has repurposed the project as an automatic installer which, on one hand, shows the power and usefulness of open source. On the other hand, if you’re not familiar with the project history and you start exploring the plugin’s implementation in

vendor/magento/magento-composer-installer//src/MagentoHackathon/Composer/Magento/Plugin.php

you may be left scratching your head.

However, if you keep the project’s original goals in mind, the source should make a little more sense.

Consequences

Between this article and last week’s, you should have a pretty good understanding of where all Magento’s files come from in a Composer “meta-package” installation. Understanding this is critical for Magento 2 developers and consultants looking to use and develop day-to-day on Magento’s systems. For example, it may be temping to drop some functions into

app/functions.php

as a shortcut way to get your code into a Magento system, but once you understand that any future composer update will wipe these changes away, the true cost of such short cuts become apparent. Don’t edit the core is as true as it ever was, but with Magento 2 it’s not always clear what is, and what isn’t, a core Magento file.

Next time we’ll be diving into Magento 2’s component system — the system that makes Composer distribution possible.