Magento 2: Understanding Object Repositories

Last time we explored database access in Magento via creating simple Magento 2 CRUD objects, and we explored the various source files involved. While some of the window dressing has changed, Magento 1 developers probably felt right at home. Conceptually, Magento 1’s Model/ResourceModel/Collection ORM still exists in Magento 2. Today we’re going to discuss a new feature of Magento 2’s Model layer — repository objects.

The Repository Pattern

One of Magento 2’s goals was a complete overhaul of the API system. Magento needed to

1. Provide a modern RESTful based API with oAuth authentication
2. Keep a SOAP based API for corporate shops/merchants that speak SOAP.
3. Maintain feature parity between the API regardless of REST/SOAP/FutureAPI
4. All while refactoring Magento’s underlying business objects and the ORM layer
5. Do all this with a team that had various levels of experience with Magento 1

One of the tools the core team used to rein in the resulting chaos was the repository pattern. Like many design patterns, the repository pattern is an old one that first came to prominence in java and C++ based systems. These systems needed to map data objects (or “business objects”) to virtual/machine memory. Like all the great 90s design patterns, the repository has been used, abused, and repurposed for different means in the ensuing decades.

To my mind, the best way to understand repository objects is

A repository object is responsible for reading and writing your object information to an object store (i.e. a database, the file system, physical memory, etc.), freeing the repository-user-programmer from worrying about computer problems, and letting them focus on using the data in their business objects and business logic

In other words, a repository becomes the source of truth for fetching and saving objects. With this source of truth in place, the API team was able to use these repositories in the webapi.xml files while other teams worked on the refactoring effort.

Understanding Magento’s use of repositories is an important part of being a Magento 2 programmer, but at this point in Magento 2’s lifecycle, repositories are not ready to shoulder the full burden of Magento 2’s model layer.

This article will explore using Magento 2’s repositories to fetch and manipulate objects, but also serve as a critique of where the repository implementation introduces new unneeded complexity and confusion. Consider this your guide through the fire swamp.

Article Conventions

We’re going to work through some code examples using a Magento command line script. To simplify things, we’ll be fetching and instantiating objects directly via the object manager. At the end of this article, we’ll provide a class that shows how to use automatic constructor dependency injection with these objects. While direct use of the object manager is discouraged for product systems, it can be useful for teaching users about the underlying objects, and for exploring areas of the system you’re not familiar with.

If any of the above, or the following, sounds like ancient greek (and you’re not a time traveler!), you may want to review our object manager series and the articles so far in this series, in particular the first article where we discuss manually creating a Magento module. You should be able to follow along without fully understanding everything we’ve done so far, but there articles might help you fill the gaps in your understanding.

We’ll use pestle to create a module for our command line application.

$ pestle.phar generate_module Pulsestorm RepositoryTutorial 0.0.1

and then add a command class with pestle’s generate_command

$ pestle.phar generate_command Pulsestorm_RepositoryTutorial Examples    

We’ll enable our module with the Magento command line

$ php bin/magento module:enable Pulsestorm_RepositoryTutorial

and then install it with the setup:upgrade command

$ php bin/magento setup:upgrade

Finally, we can test that the command was added successfully by running

$ php bin/magento ps:examples
Hello World

and looking for the “Hello World” output.

Injecting the Object Manager

The example command we just created should be in the following file.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php

We’ll need to modify this command slightly to give us access to the object manager, as well as make a small adjustment related to Magento’s areas. Add the following property and constructor to your command file.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php
class Examples extends Command
{
    //...    
    protected $objectManager;
    public function __construct(
        \Magento\Framework\ObjectManagerInterface $objectManager,
        \Magento\Framework\App\State $appState,
        $name=null
    )
    {
        $this->objectManager = $objectManager;
        $appState->setAreaCode('frontend');
        parent::__construct($name);
    }
    //...
}

This constructor does two things. First, it uses automatic constructor dependency injection to insert an object manager instance we’ll use later. Second, it uses automatic constructor dependency injection to insert a Magento\Framework\App\State object to work around some issues related to Magento areas. The later is beyond the scope of this article, but we’ve talked about it a bit over on Magento Quickies.

For the remainder of this article, we’ll be running code samples from the execute method.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php

protected function execute(InputInterface $input, OutputInterface $output)
{
    $repo = $this->objectManager->get('Magento\Catalog\Model\ProductRepository');
    $page = $repo->getById(2);        
    echo get_class($page),"\n";
}

With that out of the way, we’re ready to get started!

Getting Started with Repositories

To start with, a repository is just another object. For each “business object” (product, category, CMS page, etc), there’s a single corresponding repository object. We’ll start with CMS Page objects (cms/page objects for folks still thinking in terms of Magento 1, Magento\Cms\Model\Page objects for folks thinking in terms of Magento 2). If you’re not familiar with them, CMS page objects hold all the data (content, title, additional layout rules, etc.) related to a single page in Magento’s built-in CMS.

The repository object for CMS objects is a Magento\Cms\Model\PageRepository object. You can use this object to load a CMS Page, by its id, with the getById method.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php

$repo = $this->objectManager->get('Magento\Cms\Model\PageRepository');
$page = $repo->getById(2);

echo $page->getTitle(),"\n";    

If you have a CMS Page object in your system with an ID of 2, the above code will output its title.

When you’re looking for a business object’s corresponding repository object, there’s nothing formal in the system that ties a particular repository to a particular object. Generally speaking though, the pattern of appending Repository to the base business object class name holds

Magento\Cms\Model\Page
Magento\Cms\Model\PageRepository

If you take a look at the source of the PageRepository

#File: vendor/magento/module-cms/Model/PageRepository.php

namespace Magento\Cms\Model;
//...
class PageRepository implements PageRepositoryInterface
{    
    //...
}

you’ll see the repository implements a PageRepositoryInterface. If we take a look at this interface,

#File: vendor/magento/module-cms/Api/PageRepositoryInterface.php

interface PageRepositoryInterface
{

    public function save(\Magento\Cms\Api\Data\PageInterface $page);

    public function getById($pageId);

    public function getList(\Magento\Framework\Api\SearchCriteriaInterface $searchCriteria);

    public function delete(\Magento\Cms\Api\Data\PageInterface $page);

    public function deleteById($pageId);
}

We see there’s five methods the page repository must implement. (save, getById, getList, delete, and deleteById). We’ve already seen one of them — the getById method. This method fetches an object from the system by its assigned database ID. The save method will persist an object to the database, the delete and deleteById methods are for removing an object, and the getList method will fetch a number of objects from the database based on search criteria. The remainder of this article will explore each of these methods.

One last thing before we leave the interface behind — it’s important to note that each repository in Magento 2 implements its own interface.

#File: vendor/magento/module-catalog/Model/ProductRepository.php
class ProductRepository implements \Magento\Catalog\Api\ProductRepositoryInterface
class OrderRepository implements \Magento\Sales\Api\OrderRepositoryInterface    
etc.

While most (if not all?) business object repositories in Magento 2 share the save, getById, getList, delete, and deleteById methods, there’s nothing in the system forcing them to. If you’re having trouble with a particular repository type, it’s always a good idea to investigate the underlying interface, and its implementation in the concrete repository class.

Persisting to/from the Database

Coming back to our client code, so far we’ve loaded an object using the repository.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo = $this->objectManager->get('Magento\Catalog\Model\ProductRepository');
$page = $repo->getById(2);    
echo $page->getTitle(),"\n";   

Next up is saving objects. In the Magento 1 view of the world, this was as simple as calling save on the object you wanted to persist to the database

//magento 1
$page->save();

However, this violates a core repository principle — the logic that saves an object’s data to the system should not be part of the business object. Instead, in Magento 2, you tell your repository to save an object. If you give the following code a try, you’ll find your page object saved with an appended title.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo = $this->objectManager->get('Magento\Cms\Model\PageRepository');
$page = $repo->getById(2);    
echo $page->getTitle(),"\n";  
$page->setTitle($page->getTitle() . ', Edited by code!');
$repo->save($page);

If you wanted to duplicate a page, you could load the page to duplicate, set its ID to NULL, and pass the duplicated object to the repository.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo = $this->objectManager->get('Magento\Cms\Model\PageRepository');
$page = $repo->getById(2);
$page->setId(null);
$page->setTitle('My Duplicated Page');
$repo->save($page);                
echo $page->getId(),"\n";

The delete methods work similarly — either pass the repository a full object to delete, or pass an ID

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo->delete($page);
$repo->deleteById($page_id);

The deleteById method is useful if you have a database ID from a previous operation and don’t want to load a new object just to delete it.

Getting a List of Objects

Nothing we’ve covered so far is too complicated. Instead of objects using their own methods to interact with the database, we’re using the repository’s.

The biggest change you’ll need to overcome with repositories is the syntax/APIs for fetching a list of objects. Magento 1 provided collection objects, and collection objects exposed a SQL-like API for fetching objects.

With repositories, collections have been replaced with a getList method. While using this method may seem simple on the surface, there’s hidden complexity at every step along the way.

If you try calling the getList method (notice we’ve switched to a ProductRepository)

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo = $this->objectManager->get('Magento\Catalog\Model\ProductRepository');        
$repo->getList();

you’ll get an error like the following

Argument 1 passed to Magento\Catalog\Model\ProductRepository\Interceptor::g etList() must implement interface Magento\Framework\Api\SearchCriteriaInter face, none given

Whenever you call getList, you need to pass in a “search criteria” object. We’ll discuss why in a moment, but for now just give the following a try.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$repo = $this->objectManager->get('Magento\Catalog\Model\ProductRepository');        
$search_criteria = $this->objectManager->create(
    'Magento\Framework\Api\SearchCriteriaInterface'
);
$result = $repo->getList($search_criteria);
$products = $result->getItems();
foreach($products as $product)
{
    echo $product->getSku(),"\n";
}

Running our command with the above code in the execute method should output a list of product SKUs. We’ve used the object manager to create a search criteria object and passed that search criteria object in to the getList method. After doing this, you can use the result object’s getItems method to grab a PHP array of the returned product objects, and then foreach over that array to get the actual product objects.

Using Search Criteria

Of course, it’s rare that you want every object of a particular type in the system. More often than not, you want a subset of objects that match some search criteria. For example, you may want all SKUs that start with the text WSH11. This is where we actually use the search criteria object.

A search criteria object contains a number of grouped filter objects. These filter objects control what objects the repository will return.

If that didn’t make sense, a code sample might. The following code implements the “all SKUs that start with the text WSH11” use case we described earlier.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

//create our filter
$filter = $this->objectManager->create('Magento\Framework\Api\Filter');
$filter->setData('field','sku');
$filter->setData('value','WSH11%');
$filter->setData('condition_type','like');

//add our filter(s) to a group
$filter_group = $this->objectManager->create('Magento\Framework\Api\Search\FilterGroup');
$filter_group->setData('filters', [$filter]);

//add the group(s) to the search criteria object
$search_criteria = $this->objectManager->create('Magento\Framework\Api\SearchCriteriaInterface');
$search_criteria->setFilterGroups([$filter_group]);

//query the repository for the object(s)
$repo = $this->objectManager->get('Magento\Catalog\Model\ProductRepository');                        
$result = $repo->getList($search_criteria);
$products = $result->getItems();
foreach($products as $product)
{
    echo $product->getSku(),"\n";
}

The above code

1. Creates a LIKE filter for SKUs using the % wildcard (you can find a list of valid condition_types in vendor/magento/framework/Api/CriteriaInterface.php)
2. Creates a filter group object, and adds our single filter to that group
3. Adds the filter group to our search criteria object
4. Uses the search criteria object to fetch the product objects we want from the repository

If you run the above code, you should get a list of products whose SKUs begin with WSH11.

If all that arbitrary setData seems a little loosey goosey, you’re right. Magento provides Builder objects to build your filter, filter group, and search criteria objects. The above is equivalent to the following, similar code.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

//create our filter
$filter = $this->objectManager
->create('Magento\Framework\Api\FilterBuilder')
->setField('sku')
->setConditionType('like')
->setValue('WSH11%')
->create();

//add our filter(s) to a group
$filter_group = $this->objectManager
->create('Magento\Framework\Api\Search\FilterGroupBuilder')
->addFilter($filter)
->create();
// $filter_group->setData('filters', [$filter]);

//add the group(s) to the search criteria object
$search_criteria = $this->objectManager
->create('Magento\Framework\Api\SearchCriteriaBuilder')
->setFilterGroups([$filter_group])
->create();

When working with the product repository, filters within a group are added as OR filters. For example, the following search criteria would return two products — WHERE sku LIKE 'WSH11-28%Blue' OR sku = 'WSH11-28%Red' (assuming, of course, the sample data is installed)

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

//create our filter
$filter_1 = $this->objectManager
->create('Magento\Framework\Api\FilterBuilder')
->setField('sku')
->setConditionType('like')
->setValue('WSH11-28%Red')    
->create();

$filter_2 = $this->objectManager
->create('Magento\Framework\Api\FilterBuilder')
->setField('sku')
->setConditionType('like')
->setValue('WSH11-28%Blue')    
->create();

//add our filter(s) to a group
$filter_group = $this->objectManager
->create('Magento\Framework\Api\Search\FilterGroupBuilder')
->addFilter($filter_1)
->addFilter($filter_2)
->create();
// $filter_group->setData('filters', [$filter]);

//add the group(s) to the search criteria object
$search_criteria = $this->objectManager
->create('Magento\Framework\Api\SearchCriteriaBuilder')
->setFilterGroups([$filter_group])
->create();

Filter groups, on the other hand, are combined as AND filters (again, when working with product repositories). The following code would return no items, as WHERE sku LIKE 'WSH11-28%Blue' AND sku = 'WSH11-28%Red' is impossible.

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

//create our filter
$filter_1 = $this->objectManager
->create('Magento\Framework\Api\FilterBuilder')
->setField('sku')
->setConditionType('like')
->setValue('WSH11-28%Red')    
->create();

$filter_2 = $this->objectManager
->create('Magento\Framework\Api\FilterBuilder')
->setField('sku')
->setConditionType('like')
->setValue('WSH11-28%Blue')    
->create();

//add our filter(s) to a group
$filter_group_1 = $this->objectManager
->create('Magento\Framework\Api\Search\FilterGroupBuilder')
->addFilter($filter_1)
->create();

$filter_group_2 = $this->objectManager
->create('Magento\Framework\Api\Search\FilterGroupBuilder')
->addFilter($filter_2)
->create();        
// $filter_group->setData('filters', [$filter]);

//add the group(s) to the search criteria object
$search_criteria = $this->objectManager
->create('Magento\Framework\Api\SearchCriteriaBuilder')
->setFilterGroups([$filter_group_1, $filter_group_2])
->create();

Finally, when working with non-complicated queries, the search criteria builder has short cut methods for adding “single-filter” filter groups. For example, the following code will create a search criteria object, with a single filter group, and that single filter group will contain a sku LIKE 'WSH11-28%Blue' filter

#File: app/code/Pulsestorm/RepositoryTutorial/Command/Examples.php    

$search_criteria = $this->objectManager
->create('Magento\Framework\Api\SearchCriteriaBuilder')
->addFilter('sku','WSH11-28%Blue', 'like')
//->addFilter('sku','WSH11-28%Blue', 'like') //additional addFilters will 
                                             //add another group
->create();        

This is convenient for creating a series of simple AND filters.

The Case Against Repositories

It makes sense that Magento 2 chose the repository pattern for its new API work. By starting with a single, simple, unified RepositoryInterface, Magento’s architects gave the system implementors a tool that could abstract away whatever ugliness was necessary on the persistence/database layer to unify Magento 2 into a new API. If Magento 2’s core team was a team of developers familiar with Magento 1, this might not have been necessary. However, since they were getting the a new team up to speed on an existing platform, it made sense to start with repositories on the top.

Repositories also make it easier to write tests — breaking filters out in to a well factored, multiple object hierarchy means there’s a path forward for getting full test coverage on the repository object queries.

All that said, (and while I’m using repository objects in my Magento 2 work), there’s a lot about Magento’s particular implementation of this platform I don’t like, and I’m not afraid to fall back on using Magento’s CRUD models and collections when the repository API fails me.

Too Many Ways to Do The Same Thing

First off, there’s no clear, right way to use the getList method of a repository object. Put more succinctly, there’s too many ways to create a search criteria object. This is on display in our code samples above — we started directly creating filter, group, and criteria objects, but then moved to using Magento’s builder objects. Nothing in the architecture points to a clean, single, API for creating search criteria objects. Even now, I’m not 100% certain my code with the builder objects is right.

Additionally, since the final implementation features multiple repository interfaces, each repository can potentially drift on how search criteria are applied, and may not even provide a getList method. This means a product repository may work differently than a category repository, and they both may work differently than a CMS Page repository.

With systems like Laravel’s Eloquent available, which starts with a unified collection interface for all objects, even modern PHP developers will be looking sideways at the criteria/filter group/filter system. Java developers should feel right at home.

Too Verbose

Consider — Magento 2

$search_criteria = $this->searchCriteriaBuilder
->addFilter('sku','WSH11-28%Blue', 'like')
->create();

$repo->getList($search_criteria);

$items = $repo->getItems();

foreach($items ...){}

vs. Magento 1

$items = Mage::getModel('catalog/product')
->getCollection()
->addFieldToFilter('sku',['like'=>'WSH11-28%Blue']);    

foreach($items ...){}

One expression vs. three expressions for the simplest case. Start adding more complex filters and this difference becomes even more striking. Magento 2’s repository implementation is biased towards making it easier for the core team to implement new SOAP and REST APIS. Magento 1’s collections, and ActiveRecord in general, are biased towards giving programmers easy, SQL like access to a set of objects.

Repositories are Built on top of CRUD models

There’s a few very smart folks in the Magento community who are gung-ho about repositories being the new way forward, and the only way to write new code. It’s hard to take these claims too seriously when Magento repositories, are themselves, based on Magento’s old CRUD models. For example, if you take a look at the product repository source file

#File: vendor/magento/module-catalog/Model/ProductRepository.php
public function __construct(
    //...
    \Magento\Catalog\Model\ResourceModel\Product\CollectionFactory $collectionFactory,
    //...
) {
    $this->collectionFactory = $collectionFactory;
}

public function getList(\Magento\Framework\Api\SearchCriteriaInterface $searchCriteria)
{
    /** @var \Magento\Catalog\Model\ResourceModel\Product\Collection $collection */
    $collection = $this->collectionFactory->create();
    //...
}

You’ll see that getList relies on a converted Magento 1 Collection resource model.

Given the Magento core developers needed to dip into Magento’s old CRUD system, it seems likely Magento system integrators and extension developers will need to as well, if only to debug code related to using Magento 2’s repositories.

On top of that, the objects Magento 2 repositories return still have their own load, save, delete, and getCollection methods. While this was smart from a practical migration point of view, it creates ambiguity as to where and when a Magento 2 programmer should use the new repository classes. We’ll likely see a lot of code where Magento developers are still relying on the old CRUD methods without even realizing the repositories are available. It also makes the whole “we’ve abstracted away persistence from business objects” logic a tough sell when the persistence logic still lives in the business objects.

Now that Magento 2 has shipped, and applications are shipping inside of large corporate enterprise organizations, it seems likely that Magento 2’s core developers will face the same “de-facto frozen API” as Magento 1’s core developers did, or risk breaking what their partner agencies and independent developers are doing with the platform. While there’s no guarantees of the CRUD models not changing, doing so would cause a cascade of problems that could seriously impact Magento’s credibility.

No Mandated Consistent Repository Interfaces

As previously mentioned, each Magento 2 repository object implements its own interface. It’s only convention and process that keeps the consistent getById, getList, save, delete, and deleteById methods available to all repositories. As time goes on, I wouldn’t be surprised to see this consistent API drift, and new future third party repositories not using these methods.

Potential for Inconsistent Filter and Group Interactions

Finally, Magento’s implementation of the repository pattern leaves the system open for inconsistent filter and group behavior between repositories. Earlier we told you that inside a group, filters are applied as OR criteria, but that groups are combined with AND, with a warning that we were talking about the product repository only.

That’s because it’s the responsibility of of each repository’s getList method to implement filtering logic. Again, using the product repository as an example

#File: vendor/magento/module-catalog/Model/ProductRepository.php    

public function getList(\Magento\Framework\Api\SearchCriteriaInterface $searchCriteria)
{
    //...
    //Add filters from root filter group to the collection
    foreach ($searchCriteria->getFilterGroups() as $group) {
        $this->addFilterGroupToCollection($group, $collection);
    }
    //...
}     

protected function addFilterGroupToCollection(
    \Magento\Framework\Api\Search\FilterGroup $filterGroup,
    Collection $collection
) {
    $fields = [];
    $categoryFilter = [];
    foreach ($filterGroup->getFilters() as $filter) {
        $conditionType = $filter->getConditionType() ? $filter->getConditionType() : 'eq';

        if ($filter->getField() == 'category_id') {
            $categoryFilter[$conditionType][] = $filter->getValue();
            continue;
        }
        $fields[] = ['attribute' => $filter->getField(), $conditionType => $filter->getValue()];
    }

    if ($categoryFilter) {
        $collection->addCategoriesFilter($categoryFilter);
    }

    if ($fields) {
        $collection->addFieldToFilter($fields);
    }

}

The only reason product repositories have the AND/OR behavior we mentioned is an implementation detail of the specific repository. A different repository might implement a different bit of filtering logic. i.e. there’s nothing in the framework that forces filters and filter group to implement the same logic.

Helper Class with Dependency Injection

As promised, before we wrap up, here’s a single helper class with examples of how to use automatic constructor dependency injection to grab references to search criteria builders, filter group builders, and filter builders.

#File: app/code/Pulsestorm/RepositoryTutorial/Model/Helper.php
<?php
namespace Pulsestorm\RepositoryTutorial\Model;
/**
* Example of automatic constructor dependency injection
* for repository and filter objects
*/
class Helper
{
    protected $pageRepository;
    protected $productRepository;
    protected $filterBuilder;
    protected $filterGroupBuilder;
    protected $searchCriteriaBuilder;

    public function __construct(
        \Magento\Cms\Model\PageRepository $pageRepository, 
        \Magento\Catalog\Model\ProductRepository $productRepository,
        \Magento\Framework\Api\FilterBuilder $filterBuilder,
        \Magento\Framework\Api\Search\FilterGroupBuilder $filterGroupBuilder,
        \Magento\Framework\Api\SearchCriteriaBuilder $searchCriteriaBuilder    
    )
    {
        $this->pageRepository           = $pageRepository;
        $this->productRepository        = $productRepository;
        $this->filterBuilder            = $filterBuilder;
        $this->filterGroupBuilder       = $filterGroupBuilder;
        $this->searchCriteriaBuilder    = $searchCriteriaBuilder;
    }
}

Wrap Up

While there’s a lot to critique about Magento’s repository implementation, they do seem like the new way forward for Magento 2. It would be wise to become familiar with using them in your own extensions and integration work.

That said, the things you’ll need to do with Magento often go beyond what’s available via the new officially sanctioned API classes — don’t be afraid to dip back into your Magento 1 CRUD knowledge to get your work done. This will mean moving beyond Magento sanctioned @api methods. Getting solid integration and acceptance tests around your work will protect you from any surprises when and if things break in the future.

As Magento’s consultants, its partners, and its unaffiliated developers start deploying Magento across the world, a @defacto-api will start to develop, and the core team will be forced into a decision where they either support the old APIs, or let an untold number of systems break. Unless they’re recklessly irresponsible, conventional wisdom points to the later being the case.