Magento 2: Serving Frontend Files

When Magento 1 was initially released, “front end development” as it exists today was barely a thing. CSS was still largely written by hand, and jQuery vs. PrototypeJS was still a legitimate question for a new software project. Magento 2 enters a world where javascript is the dominant language in the interactive agency world, and the thought of not using a CSS preprocessor is considered barbaric.

While Magento 2 does use many modern front end technologies (RequireJS, LESS for CSS, etc.), at its heart Magento’s still a full stack PHP framework. This means, in addition to understanding these new technologies, a well rounded Magento developer also needs to understand how the Magento core team has integrated these new technologies into the system. This is a huge topic, and one we’ll be covering over the next few articles.

These articles are written against the official, Magento 2.0 released in the fall of 2015. Concepts should hold for future versions of Magento 2, but minor details may change.

Serving CSS and Javascript Files

We’re going to start at the bottom of the front end abstraction tree, and talk about including “raw” front end resource files (.js, .css, etc.) with your Magento module. This may be lower down the stack than you’re used to working as a modern front end developer, but understanding how these fundamentals work will be an important part of your job if you’re working with, or developing modules/themes for, Magento 2 systems.

Before we talk about javascript and CSS though, we need to talk about Magento 2’s root web folder, and Magento 2’s modes.

Magento 2 Root Folder

When you setup a PHP based framework, you need to point your webserver (Apache, nginx, etc) at a “root” folder. In other words, if someone requests URLs like these

http://magento2.example.com/index.php
http://magento2.example.com/path/to/file.js

it means they’re accessing files on your system like these

/var/www/html/index.php
/var/www/html/path/to/file.js

In the examples above /var/www/html is the root web folder. Which folder you pick as root is beyond the scope of this article, and is going to depend on which version of which linux distribution you’re using, and how the packagers have decided to setup Apache, nginx, etc on that system. What is in scope of this article is where in the Magento source tree you choose to put your root web folder.

Specifically, Magento 2 ships with two index.php files.

/path/to/magento2/index.php
/path/to/magento2/pub/index.php

One is at the absolute top level of Magento 2’s distribution folder. The second is inside the “pub” folder. There’s also separate but similar .htaccess files for each of these index.php files.

The file you want to use for your Magento system’s root folder is pub. This is a modern PHP framework convention, and is one layer in a layered approach to protecting your PHP and configuration files from being exposed to the world. However, Magento still ships with the root level index.php because many hosting companies make changing this web root difficult and/or impossible.

If you’re smart enough to seek out online articles about your ecommerce programming framework, you’re smart enough to know that the pub folder is the one you want to use. However, which folder you choose will have consequences for the paths to your front end asset files. Unless we explicitly state otherwise, assume we’ve setup apache to point to the pub folder.

Magento 2 Modes

The second bit of Magento infrastructure we’re interested in is Magento’s “modes”. You may already be familiar with developer mode. If you add the following to your .htaccess file

SetEnv MAGE_MODE developer

You’ll be running Magento in developer mode. In developer mode, Magento’s much more strict about PHP errors, will display raw exceptions, and generally do things that make it easier to develop Magento 2 modules. There are two other modes Magento can run in. If you haven’t set an explicit MAGE_MODE, you’re running in default mode. The other explicit Magento mode is production mode.

If you’re running Magento 2 in production mode, Magento 2 will do everything possible to hide errors and exceptions from end users. Magento 2 will also turn off most (if not all) of its magic code generation. If you’ve read through our series on the object system you know Magento 2 automatically creates many of the boilerplate classes needed for its object system features. There are similar code generation systems for Magento’s front end file serving. These front end file generators will also be turned off when Magento’s running in production mode. We’ll be starting this article in developer mode, and discuss the consequences of production mode as we go.

For completionists — Magento’s default mode is a weird combination of production and developer mode. A better name for it might be demo mode. Similar to production mode, default mode will suppress many of Magento’s unfriendly-but-useful technical error messages, but will automatically generate code.

I imagine default mode came out of a meeting where folks were concerned with the non-tech-savvy deploying a stock system without understanding production mode. Generally speaking, if you’re working with Magento you’ll want to run in developer mode, and deploy to production mode. The default is a weird hybrid that serves no one well.

Serving a Frontend Asset File

Magento infrastructure covered, we’re ready to discuss how Magento serves its javascript and CSS files. As mentioned above, we’re starting with a system whose web root is pointed at the pub folder, and one that’s running in developer mode.

If you view the source of your Magento homepage, you’ll see source code that looks like the following.

<link rel="stylesheet" type="text/css"  media="all" href="http://magento.example.com/static/frontend/Magento/luma/en_US/mage/calendar.css" />
<script  type="text/javascript" src="http://magento.example.com/static/_requirejs/frontend/Magento/luma/en_US/requirejs-config.js"></script>

While the wonders of modern front end systems often remove the need for manually adding javascript and CSS files yourself, their abstractions must, at some level, create HTML to include an HTTP resource on that page. i.e. The following URLs

http://magento.example.com/static/frontend/Magento/luma/en_US/mage/calendar.css
http://magento.example.com/static/_requirejs/frontend/Magento/luma/en_US/requirejs-config.js

need to return plain old javascript and CSS. If you load them in your browser or via a command line program like curl, you should see a file returned

$ curl -i 'http://magento.example.com/static/frontend/Magento/luma/en_US/mage/calendar.css'
HTTP/1.1 200 OK
Date: ...

/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
.ui-datepicker {
    display: none;
    z-index: 999999 !important;
}
/* ... */

If you take a closer look at that URL

http://magento.example.com/static/frontend/Magento/luma/en_US/mage/calendar.css

you’d expect to find a file at

/path/to/magento2/pub/static/frontend/Magento/luma/en_US/mage/calendar.css

If your system is setup correctly, that expectation is correct.

Pretty straight forward — but here’s the first Magento 2 twist. Remember how we said Magento has two index.php files? If you change your web server configuration to point to the base index.php file (not the one in pub/), your request will now return an HTTP status 404 Not Found.

$ curl -I 'http://magento.example.com/static/frontend/Magento/luma/en_US/mage/calendar.css'
HTTP/1.1 404 Not Found

If you go back and load your Magento home page, you’ll notice the href has changed to include the pub folder as a prefix

http://magento.example.com/pub/static/frontend/Magento/luma/en_US/mage/calendar.css

Magento is smart enough to recognize where its web root is, and generate the correct URL. However, this does have consequences for third party Magento developers — mainly that adding javascript and CSS outside of Magento 2’s abstractions is no longer a reliable way to distribute simple code. i.e., in Magento 1, it was common for many extension vendors to simply add a folder to the top level js folder

/path/to/magento/js/packagename_vendor/...

and then hard code link and script tags to this folder

<script src="/js/packagename_vendor/file.js" ... >
<link href="/js/packagename_vendor/file.css" ... >

Since Magento 2 has two possible root folders, this is no longer a reliable way to ship code for wider use.

Also, despite this, it’s also likely that single merchant integrations (i.e. an offshore team hacking together a Magento 2 system for someone) will include javascript/css like this that’s incompatible with one of the two root folders.

If you’re cleaning up a Magento 2 system in the wild — be careful suggesting merchants just switch their cart over to serving files out of pub/. While this is “the right” thing to do, there may be hacked in javascript and CSS files that rely on the “less right” configuration.

Magento 2 Static Asset Serving

So, assuming you’ve re-re-configured your web server to properly point to the pub folder, and you’re still running in developer mode, lets return to our static asset file

http://magento.example.com/static/frontend/Magento/luma/en_US/mage/calendar.css

Depending on your system configuration (locale, theme, etc.) your URL may looks slightly different

http://magento.example.com/static/frontend/Magento/themename/en_UK/mage/calendar.css

However, the non-domain portion of the URL should refer to an actual static file in pub. The rest of this article will assume a theme of luma and a locale of en_US.

Let’s open up our calender.css file and add a comment (/* I am learning how to serve CSS files in Magento 2 */) to the top

/* File: pub/static/frontend/Magento/luma/en_US/mage/calendar.css */

/* I am learning how to serve CSS files in Magento 2 */

/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
.ui-datepicker {/* ... */}
/* ... */

Save the file, and load the calendar.css URL in your browser or via curl. You should see your new comments at the top of the file

$ curl -i 'http://magento.example.com/static/frontend/Magento/luma/en_US/mage/calendar.css' 

HTTP/1.1 200 OK
...

/* I am learning how to serve CSS files in Magento 2 */

/**
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */
.ui-datepicker {
    /*...*/
}
/*...*/

So far, nothing out of the ordinary. Next up, let’s rename the calendar.css file

$ mv pub/static/frontend/Magento/luma/en_US/mage/calendar.css pub/static/frontend/Magento/luma/en_US/mage/calendar.bak

Now, if I asked you what would happen if we tried to download the file again, you’d tell me that the web server would return a 404 Not Found error. However, if you do try to access the file again

$ curl -i 'http://magento.example.com/static/frontend/Magento/luma/en_US/mage/calendar.css'
HTTP/1.1 200 OK
//...

you’ll find the file’s still there. Even more confusing, if you check the file system, you’ll find the calendar.css file has been restored!

$ find pub/static -name 'calendar.css'
//...
pub/static/frontend/Magento/luma/en_US/mage/calendar.css

So what gives? When you’re running your system in developer mode, if Magento can’t find a static asset file, it will automatically copy or symlink that file from that file’s source module. If you don’t believe us, setup your system to run in production mode by editing the .htaccess file.

#File: .htaccess

SetEnv MAGE_MODE production

Then, remove the calendar.css file

$ rm pub/static/frontend/Magento/luma/en_US/mage/calendar.css

Finally, with Magento’s mode set to production, try to download the calendar.css file again.

curl -I 'http://magento.example.com/static/frontend/Magento/luma/en_US/mage/calendar.css'
HTTP/1.1 404 Not Found
//...

Instead of generating a new calendar.css file, Magento 2 returns a 404 Not Found error. As previously mentioned, when Magento’s running in production mode, it will not automatically generate files. This is a smart security precaution on the part of the core team — code generation opens all sorts of surface area for system attacks. By enabling Magento devops folks to turn off code generation, Magento has closed off an entire class of attack vectors into the application.

For Every Answer, More Questions

A few questions that might immediately pop into your mind are

1. How do we ensure all the files for production mode are generated?
2. What sort of magic is Magento doing to enable the developer mode file creation?
3. How can our own modules take advantage of this?

The first question is the easiest to answer. Magento 2’s command line application ships with a command named setup:static-content:deploy. If you run this command, Magento will run through every module in the system

$ php bin/magento setup:static-content:deploy
Requested languages: en_US
=== frontend -> Magento/blank -> en_US ===
//...
Successful: 845 files modified
---
New version of deployed files: 1450920725

After running the above command, you’ll find that calendar.css has been restored to the file system.

The basic idea is that deployment of a Magento 2 system should only happen via a formalized deployment process, and part of that deployment will always include running this command in production — either manually, or via your deployment system. This is similar to the asset pipeline in the Symfony framework.

As for how Magento performs the magic of creating files in developer and default mode, the answer lies in the .htaccess file found in the static folder

#File: pub/static/.htaccess
#...
<IfModule mod_rewrite.c>
    RewriteEngine On

    # Remove signature of the static files that is used to overcome the browser cache
    RewriteRule ^version.+?/(.+)$ $1 [L]

    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-l

    RewriteRule .* ../static.php?resource=$0 [L]
</IfModule>
#...

This is a separate .htaccess file from Magento’s main .htaccess file. The above rewrite rule basically says

If the requested file does not exist on the system, redirect the request to the static.php file one directory up, with the file path included as a resource parameter.

i.e., the following request

http://magento.example.com/static/frontend/Magento/luma/en_US/mage/calendar.css

is identical to this request

http://magento.example.com/static.php?resource=frontend/Magento/luma/en_US/mage/calendar.css

While it’s beyond the scope of this article — the static.php file contains a mini-application built using Magento’s framework code.

#File: pub/static.php     
/**
 * Entry point for static resources (JS, CSS, etc.)
 *
 * Copyright © 2015 Magento. All rights reserved.
 * See COPYING.txt for license details.
 */

require __DIR__ . '/../app/bootstrap.php';
$bootstrap = \Magento\Framework\App\Bootstrap::create(BP, $_SERVER);
/** @var \Magento\Framework\App\StaticResource $app */
$app = $bootstrap->createApplication('Magento\Framework\App\StaticResource');
$bootstrap->run($app);

This “Static Resource” application is the one that’s responsible for copying or symlink-ing files back to their original module folder. If you’re curious in tracing this application’s execution, you can start in the launch method here

#File: vendor/magento/framework/App/StaticResource.php
public function launch()
{
    // disabling profiling when retrieving static resource
    \Magento\Framework\Profiler::reset();
    $appMode = $this->state->getMode();
    if ($appMode == \Magento\Framework\App\State::MODE_PRODUCTION) {
        $this->response->setHttpResponseCode(404);
    } else {
        $path = $this->request->get('resource');
        $params = $this->parsePath($path);
        $this->state->setAreaCode($params['area']);
        $this->objectManager->configure($this->configLoader->load($params['area']));
        $file = $params['file'];
        unset($params['file']);
        $asset = $this->assetRepo->createAsset($file, $params);
        $this->response->setFilePath($asset->getSourceFile());
        $this->publisher->publish($asset);
    }
    return $this->response;
}    

As for the final question

How can our own modules take advantage of this?

That’s one that will need to wait for next time. In this article, we’ve introduced the concept of generated front end files, and how file generation interacts (or doesn’t) with both Magento’s production, default, and development modes and the index.php and pub/index.php deployment approaches. Once that meal digests, you’ll be ready for our next article, which covers adding front end asset files in your own Magento 2 modules.