Magento 2 Object Manager: Proxy Objects

With their take on an object manager/container and dependency injection, Magento have created a new way for PHP programmers to work. With that new way of working comes a new set of unanticipated challenges. Put another way, new patterns create new problems.

Fortunately, Magento 2’s extended development cycle has given the core team time to address some of these problems with — you guessed it — more design patterns and more object manager features.

Today we’re going to take a look the problem of ill-performant code while using other people’s objects. Along the way we’ll discuss Magento 2’s use of the proxy pattern to solve this problem, as well as our first real discussion of Magento 2’s code generation features.

We’re deep in the weeds here — while you may find something of value as a newcomer, you’ll definitely want to get familiar with the basics of Magento’s object system to get the full value out of the article.

Installing the Module

Rather than have you spend 30 minutes copying and pasting code, we’ve created two modules that simulate a common problem you’ll run into with Magento automatic constructor dependency injection. The modules (Pulsestorm_TutorialProxy1 and Pulsestorm_TutorialProxy2) are available on GitHub

The official installation procedure for a Magento module is still being worked out, so we recommend installing these tutorial modules manually using the latest tagged release. If you’re not sure how to install a module manually, the first article in this series has the instructions you’re looking for.

To test that you’ve installed the module correctly, try running the following command

$ php bin/magento ps:tutorial-proxy
You've installed Pulsestorm_TutorialProxy2!
You've also installed Pulsestorm_TutorialProxy1!

If you see output telling you you’ve installed both modules, you’re ready to go!

Slow Loading Dependencies

These two modules simulate a common situation when using other people’s code — you’d like to make use of the their objects, but their code includes slow loading dependencies that you don’t need.

Let’s open up the command class and comment out our install check and uncomment the other lines

#File: app/code/Pulsestorm/TutorialProxy2/Command/Testbed.php
protected function execute(InputInterface $input, OutputInterface $output)
{
    //$this->installedCheck($output);
    $service = $this->createService($output);        
    $this->sayHelloWithFastObject($service, $output);
    $this->sayHelloWithSlowObject($service, $output);
}

The first thing this code does ($this->createService($output);), is create a Pulsestorm\TutorialProxy1\Model\Example object using the object manager.

#File: app/code/Pulsestorm/TutorialProxy2/Command/Testbed.php
protected function createService($output)
{
    //...
    $om = $this->getObjectManager();
    //...
    $service = $om->get('Pulsestorm\TutorialProxy1\Model\Example');
    //...
    return $service;    
}

We’re using the object manager directly here for simplicity’s sake — in real Magento 2 code you’ll create most of your objects via automatic constructor dependency injection. Since automatic constructor dependency injection uses the object manager, everything we show you today will be available to your injected objects as well.

After instantiating that object, we call the two sayHello... methods

#File: app/code/Pulsestorm/TutorialProxy2/Command/Testbed.php
protected function sayHelloWithFastObject($service, $output)
{
    //...
    $service->sayHelloWithFastObject();
    //...
}

protected function sayHelloWithSlowObject($service, $output)
{
    //...
    $service->sayHelloWithSlowObject();
    //...        
}

All these methods do is pass on method calls to the Pulsestorm\TutorialProxy1\Model\Example object. Also, and omitted above, these methods contain some simple profiling code that will let us know how long each method took to run. We’ve also dropped similar profiling code in the Example service object and its dependencies.

If we run our command, we should see output something like the following

$ php bin/magento ps:tutorial-proxy
About to Create Service
Constructing FastLoading Object
Constructing SlowLoading Object
Created Service, approximate time to load: 3005.656 ms

About to say hello with fast object
Hello
Said hello with fast object, approximate time to load: 0.0172 ms

About to say hello with slow object
Hello
Said hello with slow object, approximate time to load: 0.0491 ms

While we’ll eventually get to all the output, the line we care about right now is this one

Created Service, approximate time to load: 3005.656 ms

Something is taking over 3 seconds (3000 milliseconds) to load. In our example, this three second load is simulated using a PHP sleep statement — but in the real world slow loading constructors can be murder to objects musing dependency injection.

Digging Deeper

Let’s take another look at our execute method

#File: app/code/Pulsestorm/TutorialProxy2/Command/Testbed.php
protected function execute(InputInterface $input, OutputInterface $output)
{
    //this->installedCheck($output);
    $service = $this->createService($output);        
    $this->sayHelloWithFastObject($service, $output);
    $this->sayHelloWithSlowObject($service, $output);
}

At this level of abstraction, all we’re doing is

• Creating a Pulsestorm\TutorialProxy1\Model\Example object
• Calling that object’s sayHelloWithSlowObject and sayHelloWithFastObject methods

If we take a quick look at the source for our Example object

#File: app/code/Pulsestorm/TutorialProxy1/Model/Example.php
namespace Pulsestorm\TutorialProxy1\Model;
//...
public function __construct(FastLoading $fast, SlowLoading $slow)
{
    $this->fast = $fast;
    $this->slow = $slow;
}

We see it has two dependencies — Pulsestorm\TutorialProxy1\Model\FastLoading and Pulsestorm\TutorialProxy1\Model\SlowLoading. In turn, the sayHelloWithFastObject and sayHelloWithSlowObject methods

#File: app/code/Pulsestorm/TutorialProxy1/Model/Example.php
public function sayHelloWithFastObject()
{
    $this->fast->hello();
}

public function sayHelloWithSlowObject()
{
    $this->slow->hello();
}  

pass on a call to the hello method of the fast and slow arguments (now stored as object properties).

We’ve also added profiling to

1. The creation of the Example object
2. The calling of both sayHelloWithFastObject and sayHelloWithSlowObject

We’ve also added some debugging messages to the constructors of our FastLoading and SlowLoading classes.

#File: app/code/Pulsestorm/TutorialProxy1/Model/FastLoading.php
public function __construct()
{
    echo "Constructing FastLoading Object","\n";
}

#File: app/code/Pulsestorm/TutorialProxy1/Model/SlowLoading.php
public function __construct()
{
    echo "Constructing SlowLoad Object","\n";
    //...
}

Finally, in the __construct method of the SlowLoading class, we’ll see the following

#File: app/code/Pulsestorm/TutorialProxy1/Model/SlowLoading.php

public function __construct()
{
    echo "Constructing SlowLoading Object","\n";
    //simulate slow loading object with sleep
    sleep(3);
}

That is — we’ve placed a three second sleep in the constructor to simulate a slow loading object.

The end result is, when we run our command, we have a detailed report of what’s going on, and how long it’s taking.

$ php bin/magento ps:tutorial-proxy
About to Create Service
Constructing FastLoading Object
Constructing SlowLoading Object
Created Service, aproximate time to load: 3000.7651 ms

About to say hello with fast object
Hello
Said hello with fast object, approximate time to load: 0.0162 ms

About to say hello with slow object
Hello
Said hello with slow object, approximate time to load: 0.052 ms

Based on the above, it’s taking approximately three seconds to create a Pulsestorm\TutorialProxy1\Model\Example object. We now have a crude, if exaggerated, version of the sort of performance problem you might run into in the real world.

Why so Slow?

The first question we need to answer is

Why is the Example class loading slowly?

Based on what we’ve learned in this article so far, it’s the SlowLoading objects that are our slow ones, not the Example object.

The problem here is Magento’s automatic constructor dependency injection system. If we take another look at the Example object’s constructor

#File: app/code/Pulsestorm/TutorialProxy1/Model/Example.php
namespace Pulsestorm\TutorialProxy1\Model;
//...
public function __construct(FastLoading $fast, SlowLoading $slow)
{
    $this->fast = $fast;
    $this->slow = $slow;
}

We see two objects injected. Per Magento’s automatic constructor dependency injection, we know this means before the object manager instantiates a Pulsestorm\TutorialProxy1\Model\Example object, it will need to instantiate both a Pulsestorm\TutorialProxy1\Model\FastLoading and Pulsestorm\TutorialProxy1\Model\SlowLoading object.

So, our problem isn’t really a slow loading Example object — it’s a slow loading Pulsestorm\TutorialProxy1\Model\SlowLoading argument. This distinction might seem trivial, but it will be important in the solution Magento’s object manager provides.

Magento 2 Proxy Objects

The Magento core team must have run into this sort of a problem a lot — because not only do they have a solution for it, but that solution is baked into the object manager.

Magento’s solution is an implementation of the proxy pattern, used here to defer the loading of an object’s arguments.

If that didn’t make sense, don’t worry, we’ll guide you through the process step by step, and then explain what we did. If that did make sense to you, you’ll still want to proceed carefully — there’s some Magento 2 specific magic that, at first glance, makes everything a little confusing. However, once you’ve created a few proxy objects the magic should start making sense.

As previously discussed, our problem is

When Magento uses a Pulsestorm\TutorialProxy1\Model\SlowLoading object as an argument in a Pulsestorm\TutorialProxy1\Model\Example object, this slows down the instantiation of the Pulsestorm\TutorialProxy1\Model\Example object.

The solution to this is, via Magento’s di.xml configuration, to replace the Pulsestorm\TutorialProxy1\Model\SlowLoading argument with a proxy object. The proxy object will not suffer from the same slow loading as the original argument. Said another way, we’re going to use plain old argument replacement to replace the problem object with a different, special object called a proxy.

To do this, add the following nodes to the Pulsestorm_TutorialProxy2 module’s di.xml file.

#File: app/code/Pulsestorm/TutorialProxy2/etc/di.xml
@highlightsyntax@xml
<config>
    <!-- #File: app/code/Pulsestorm/TutorialProxy2/etc/di.xml -->
    <!-- Notice: we're using `TutorialProxy2`'s di.xml to change the
         behavior of `TutorialProxy1`.  This the far more common usage
         of object manager/dependency-injection than the examples in 
         our tutorial so far -->

    <type name="Pulsestorm\TutorialProxy1\Model\Example">
        <arguments>
            <argument name="slow" xsi:type="object">Pulsestorm\TutorialProxy1\Model\SlowLoading\Proxy</argument>
        </arguments>        
    </type>
</config>

This is (mostly) standard argument replacement. We’re targeting the arguments in the Pulsestorm\TutorialProxy1\Model\Example object, specifically the argument named slow, and replacing it with a Pulsestorm\TutorialProxy1\Model\SlowLoading\Proxy object.

The one new thing is the Pulsestorm\TutorialProxy1\Model\SlowLoading\Proxy class/object. We’re going to need to do a little hand waving and say “trust us” on what this is. All you need to know for now is, since we’re replacing the Pulsestorm\TutorialProxy1\Model\SlowLoading class, we replace it with a Pulsestorm\TutorialProxy1\Model\SlowLoading\Proxy class (i.e. — we append \Proxy to the initial class name).

If you clear your cache

$ php bin/magento cache:clean 
Cleaned cache types:
config
layout
block_html
view_files_fallback
view_files_preprocessing
collections
db_ddl
eav
full_page
translate
config_integration
config_integration_api
config_webservice

and re-run the command with the above in place, you should see the following.

$ php bin/magento ps:tutorial-proxy
About to Create Service
Constructing FastLoading Object
Created Service, aproximate time to load: 6.0711 ms

//...

That is — we’ve gone from 3000 ms to around 6 ms. I’d say that’s a substantial improvement. Run the command again though

$ php bin/magento ps:tutorial-proxy
About to Create Service
Constructing FastLoading Object
Created Service, aproximate time to load: 0.736 ms

//...

And you’ll see an even more dramatic improvement. Your times may vary in their specificity, but the ratios/pattern should be similar. Congratulations! You’ve successfully used a proxy to improve the construction time of a Magento object manager object.

What Just Happened

Of course, this raises myriad questions. A few might be

1. We never defined a Pulsestorm\TutorialProxy1\Model\SlowLoading\Proxy class, so what is it? A virtualType? Something else?

2. The proxy pattern is about substituting one object for another — why does this improve performance?

3. Why did we need to run the command twice to see the full performance improvements?

All valid questions, and they all get to the heart of the magic involved in Magento’s proxy objects.

The answer to the first question: The Pulsestorm\TutorialProxy1\Model\SlowLoading\Proxy configuration is not a virtual type — it’s a plain old PHP class. The reason we didn’t need to create this class on our own is, Magento automatically generates proxy class files for us. Take a look at the following file

#File: var/generation/Pulsestorm/TutorialProxy1/Model/SlowLoading/Proxy.php    
<?php
namespace Pulsestorm\TutorialProxy1\Model\SlowLoading;

/**
 * Proxy class for @see \Pulsestorm\TutorialProxy1\Model\SlowLoading
 */
class Proxy extends \Pulsestorm\TutorialProxy1\Model\SlowLoading
{
    //...
}

This is the Pulsestorm\TutorialProxy1\Model\SlowLoading\Proxy class we configured, and Magento generated. Notice it’s in the var/generation folder.

When Magento’s object manage encounters a class whose “short name” is Proxy (i.e. whose full class name Ends\In\Proxy), it will automatically generate a proxy class like this one.

If you don’t believe us, try deleting this file

$ rm var/generation/Pulsestorm/TutorialProxy1/Model/SlowLoading/Proxy.php
$ ls -l var/generation/Pulsestorm/TutorialProxy1/Model/SlowLoading/Proxy.php
ls: var/generation/Pulsestorm/TutorialProxy1/Model/SlowLoading/Proxy.php: No such file or directory

and then re-run the command

$ php bin/magento ps:tutorial-proxy
//...
$ ls -l var/generation/Pulsestorm/TutorialProxy1/Model/SlowLoading/Proxy.php
-rw-rw-rw-  1 alanstorm  staff  2350 Aug  9 16:32 var/generation/Pulsestorm/TutorialProxy1/Model/SlowLoading/Proxy.php        

The file’s been restored after running our command! It’s not just di.xml configuration that will trigger Magento code generation — it’s anytime the object Manager encounters a class whose short name is Proxy. Give the following a try — add the following temporary code to our execute method

#File: app/code/Pulsestorm/TutorialProxy2/Command/Testbed.php
protected function execute(InputInterface $input, OutputInterface $output)
{

    $object_manager = $this->getObjectManager();
    $object = $object_manager->create('Pulsestorm\TutorialProxy1\Model\Example\Proxy');
    $output->writeln('You just instantiated a ' . get_class($object) . ' class'); 
    $output->writeln('You also indirectly told Magento to create this class in the following location');
    $r = new \ReflectionClass($object);
    $output->writeln($r->getFilename());        
    exit;  
    //...
}

and then run it.

$ php bin/magento ps:tutorial-proxy
You just instantiated a Pulsestorm\TutorialProxy1\Model\Example\Proxy class
You also indirectly told Magento to create this class in the following location
/path/to/magento/var/generation/Pulsestorm/TutorialProxy1/Model/Example/Proxy.php

So, above we told the object manager to create a Pulsestorm\TutorialProxy1\Model\Example\Proxy object. The object manager, seeing that the class name ended in Proxy, automatically created a proxy object that extends Pulsestorm\TutorialProxy1\Model\Example

#File: var/generation/Pulsestorm/TutorialProxy1/Model/Example/Proxy.php
<?php
namespace Pulsestorm\TutorialProxy1\Model\Example;

/**
 * Proxy class for @see \Pulsestorm\TutorialProxy1\Model\Example
 */
class Proxy extends \Pulsestorm\TutorialProxy1\Model\Example
{
}    

Be careful though, if you try to create a proxy for an object that doesn’t exist,

$object_manager->create('Pulsestorm\TutorialProxy1\Model\Ghost\Proxy');

the object manager will yell at you.

$ php bin/magento ps:tutorial-proxy

[Magento\Framework\Exception\LocalizedException]
Source class "\Pulsestorm\TutorialProxy1\Model\Ghost" 
for "Pulsestorm\TutorialProxy1\Model\Ghost\Proxy" generation does not exist.  

While PHP frameworks have used automatic code generation for a while, it’s usually at the explicit request of a user (i.e. “make me a file”). In Magento 2, code generation exists to save a developer the time of writing a lot of the boiler plate code that accompanies “design patterns” style programming.

This leads in well to our next topic: Why does a proxy object improve performance? Before we continue, don’t forget to remove the temporary object manager code from the execute method.

Magento’s Generated Proxy Objects

From Wikipedia — the proxy pattern

in its most general form, is a class functioning as an interface to something else. The proxy could interface to anything: a network connection, a large object in memory, a file, or some other resource that is expensive or impossible to duplicate.

If we take a look at the generated code, we’ll immediately see why a proxy object gave our object instantiation the performance boost it did.

#File: var/generation/Pulsestorm/TutorialProxy1/Model/SlowLoading/Proxy.php
<?php
namespace Pulsestorm\TutorialProxy1\Model\SlowLoading;

class Proxy extends \Pulsestorm\TutorialProxy1\Model\SlowLoading
{
    //...
    public function __construct(\Magento\Framework\ObjectManagerInterface $objectManager, $instanceName = '\\Pulsestorm\\TutorialProxy1\\Model\\SlowLoading', $shared = true)
    {
        $this->_objectManager = $objectManager;
        $this->_instanceName = $instanceName;
        $this->_isShared = $shared;
    }
    //...
}

The proxy object completely replaces the constructor of the proxied object, and does not make a parent::__construct call.

Proxy objects (like all objects in Magento) are still subject to dependency injection, and all proxy objects will have the same three arguments

#File: var/generation/Pulsestorm/TutorialProxy1/Model/SlowLoading/Proxy.php
public function __construct(\Magento\Framework\ObjectManagerInterface $objectManager, $instanceName = '\\Pulsestorm\\TutorialProxy1\\Model\\SlowLoading', $shared = true)
{
    $this->_objectManager = $objectManager;
    $this->_instanceName = $instanceName;
    $this->_isShared = $shared;
}

The first is an object manager instance, the second is the name of the class this object proxies, and the third is a shared argument. We’ll cover shared/unshared in a future article — for now just concentrate on the first two arguments.

So, by replacing the entire constructor, we avoid any slow loading behavior.

However — what if important things happen in the __construct method? What about the other properties assigned there? Aren’t we breaking the Pulsestorm\TutorialProxy1\Model\SlowLoading object by doing this?

Fortunately not. In addition to replacing the constructor, our proxy object also replaced each public method of the original object.

#File: var/generation/Pulsestorm/TutorialProxy1/Model/SlowLoading/Proxy.php
/**
 * {@inheritdoc}
 */
public function hello()
{
    return $this->_getSubject()->hello();
}

If someone calls hello, the proxy will call _getSubject and pass on the call to hello. The _getSubject method

#File: var/generation/Pulsestorm/TutorialProxy1/Model/SlowLoading/Proxy.php
protected function _getSubject()
{
    if (!$this->_subject) {
        $this->_subject = true === $this->_isShared
            ? $this->_objectManager->get($this->_instanceName)
            : $this->_objectManager->create($this->_instanceName);
    }
    return $this->_subject;
}

will instantiate an instance of the original object! Remember, the _instanceName variable argument from the constructor? If we use some x-ray vision, this method actually looks like

#File: var/generation/Pulsestorm/TutorialProxy1/Model/SlowLoading/Proxy.php
protected function _getSubject()
{
    if (!$this->_subject) {
        $this->_subject = true === $this->_isShared
            ? $this->_objectManager->get('\\Pulsestorm\\TutorialProxy1\\Model\\SlowLoading')
            : $this->_objectManager->create('\\Pulsestorm\\TutorialProxy1\\Model\\SlowLoading');
    }
    return $this->_subject;
}

In this way, Magento defers the loading of the slow loading object until it’s needed. In fact, if you re-examine our current output in its entirety.

$ php bin/magento ps:tutorial-proxy
About to Create Service
Constructing FastLoading Object
Created Service, aproximate time to load: 0.89 ms

About to say hello with fast object
Hello
Said hello with fast object, approximate time to load: 0.0069 ms

About to say hello with slow object
Constructing SlowLoading Object
Hello
Said hello with slow object, approximate time to load: 3001.014 ms

you’ll notice we still have a 3 second (3000 ms) delay, it’s just been deferred until we actually need the SlowLoading object

About to say hello with slow object
Constructing SlowLoading Object
Hello
Said hello with slow object, approximate time to load: 3001.014 ms

So, in our silly example program, the proxy doesn’t save us much. In the real world, you’ll use proxy objects in situations where

1. You have a slow loading dependency
2. You know your particular use of this code doesn’t need the dependency

Regardless of whether you choose to use proxies — they’re a part of the Magento 2 core system, and likely to show up in other people’s code, so make sure you understand them.

Generation Caveats

So, that’s questions one and two answered. As for the third

Why did we need to run the command twice to see the full performance improvements?

Some readers will have already figured this out. While code generation means we avoid having to create all the boiler plate code in our proxy class over and over again, there is a small performance cost involved (around 6 ms on my development machine). Once the code is generated, it stays generated — Magento 2 will only generate a file if it can’t instantiate the request object.

So the first performance improvement

[THREE SECONDS HERE]
Created Service, aproximate time to load: 6.0711 ms

//...

was the proxy object skipping direct instantiation of the SlowLoading argument.

The second performance improvement

Created Service, aproximate time to load: 6.0711 ms
Created Service, aproximate time to load: 0.736 ms

was PHP not having to regenerate an already generated class.

There’s another gotcha to code generation. If you change the original proxied class, Magento 2 won’t automatically re-generate any new public methods. This will create unpredictable behavior as you’ll still be able to call the new method (since the proxy object extends the original class), but the new methods won’t instantiate the subject object, and will cary a different state.

During development it would be wise to periodically remove your generated folder

var/generation

to trigger re-generation of any code that needs it.

For production systems — I expect this generated code folder will be one of those challenges early adopters will wrestle with. Magento 2 appears to offer two compilation commands) that will pre-generate code for the entire system — but these commands also go above and beyond generating code. Also, for anyone into scalable systems, on-the-fly generated code sets off all sort of trigger warnings for running in a multiple-app/frontend server environment. While these problems will, no doubt, be sorted out over the coming months and year(s), it’s a challenging climb ahead for Magento 2 devops engineers.

Fortunately for us — we’re a series for programmers! Unfortunately for us, we’ve covered a lot of information today (proxies and code generation), and it’s probably a good idea to take a small breather. Next time our code generation exposure will come in handy as we explore shared/unshared objects, as well as Magento 2’s take on the age old factory pattern.