webpack support about docs

[allyraza]

We should not make this decision, we should leave this to the user to choose if they wanna use webpack, rollup or gulp, user should be able to use whatever is appropriate for them or they are most comfortable with.

Perhaps we can include a package.json file which includes the app dependencies.

[samdark]

It is marked as docs because of that.

[armpogart]

@samdark Not sure if this proposal belongs here, or it needs separate issue in yii2 repo for 2.1 development. But may I propose to rethink how AssetBundles work and/or we use them in framework.
They are great, as they allow you to register asset in any view and be sure that the those assets are there when rendering the view, but they also introduce one big problem when containerizing web application. When you do it properly you usually have separate container for web server (nginx/apache) and separate container for app (php-fpm), db and other containers are not important in this context. And assets usually don't belong to app container, they belong to web server container. So in yii's case, you need to find all the AssetBundles registered in all the views and make sure they are all published and bundled with web server container, this creates some difficulties (very often you forget and miss some assets), this also makes it hard to build containers as you need to build app container first, publish all the assets from yii container, save it in some sort of storage (artifact) and then bundle in web server container.
The same problem arises when using any bundler (like webpack), as we don't know which assets are used without first publishing them.

This is from my own experience when building yii app in CI. See also this issue (yiisoft/yii2#14325). If my example is not clear enough I can show with real code and example that we use in private. (P.S. also in russian, as english is not my native one)

[samdark]

Aren't assets (same as source code) usually not in a container but in FS mounted to containers needed?

[armpogart]

@samdark In case of development - yes (when you run docker on dev machine), but in case of staging/production you have (usually) something like:

• App container (in case of php/yii all source code is bundled here) - it is some version of your application already bundled (usually at some git ref), for go it would be compiled app from source code.
• Web server container (it may be separate for just this app, or some number of apps can share one, and it can also act as an load balancer for app when scaling)
• DB/Queue/... some other containers

And the only thing that app and web server container can share, is some storage (usually in cloud or on some sort of cluster fs) where dynamic assets are (e.g. uploads of the users).

AssetBundles make it easy to add asset in any view, but they also make it hard for devops, as first you don't know your app assets beforehand (or only checking layouts) and second yii publishes assets (it generates random paths for them) only when running (when it is requested for the first time). In an issue mentioned I proposed to add command to AssetBundle to publish all used assets (ideally it would be shell script, something universally working and not written in php) without bundling (bundling is separate issue, you don't always want to bundle, e.g. in case of staging, and even if bundling you need them in web server container, but currently I need to build app container, run it to extract assets and only then build web server container, while normally you must be able to run these 2 tasks in parallel on CI considering the fact that both tasks/jobs have access to the source code when running).
Not sure if it is possible, but for example some shell script (maybe executable written in go) will parse the views (all other parts of application where you usually register assets with yii) and publish them.
As far as I know other frameworks simply don't have magic like AssetBundle, which doesn't create this problem in the first place. Am not stating that yii AssetBundles are bad.

P.S. This was simplified structure, when you want to be able to scale more dynamically, you will have more moving parts.

[samdark]

I see. Would extending options of https://github.com/yiisoft/yii2/blob/master/framework/console/controllers/AssetController.php not to compress / combine assets solve the issue?

[armpogart]

Assets in this case are bundled with web server, as they are also some exact version of your app. Such structure is universally used as it allows to have prebuilt containers for any version of your app you want, as it allows to rollback anything easily, or to update your application in staging/production without any downtime, to explain the latter in simple words, imagine that you have:

• 1 Nginx container with all the assets for version 1.0 of your application running somewhere (bare metal, cloud, kubernetes.... it doesn't matter)
• 1 app container with all the php code in there for version 1.0
• Load balancer somewhere in front of nginx

and when you update your application new nginx container and app container are created there, load balancer is starting to send the requests to the new version of nginx (while the older is still there for current request) and the old one is taken down only when the last request to it was processed.

This simple structure allows you to scale your services to any number you want and anywhere in the cloud (considering that you also have orchestrator).

[armpogart]

@samdark, that's how I'm doing it right now. I have following controller (I proposed to add such command to yii's own AssetController):

namespace console\controllers;

use frontend\assets\AppAsset;
use yii\console\controllers\AssetController as BaseAssetController;
use yii\web\AssetBundle;

class AssetsController extends BaseAssetController
{

    // Additional console command
    public function actionPublish($configFile)
    {
        $this->loadConfiguration($configFile);
        // Loading bundles will publish them automatically
        $this->loadBundles($this->bundles);
        $this->stdout("Publishing assets... \n");

        return self::EXIT_CODE_NORMAL;
    }
}

and I have following in configs:

/**
 * Configuration file for the "yii asset" console command.
 */

// In the console environment, some path aliases may not exist. Please define these:
 Yii::setAlias('@webroot', __DIR__ . '/../../backend/web');
 Yii::setAlias('@web', '/');

return [
    // The list of asset bundles to compress:
    'bundles' => [
         'backend\assets\AppAsset',
    ],
    // Asset manager configuration:
    'assetManager' => [
        'basePath' => '@webroot/assets',
        'baseUrl' => '@web/assets',
    ],
];

It's a simple and fast hack (which can at least be in core) on yii's AssetController to add the command to publish the assets without bundling/combining them. But there's still an issue here that first any team member or the final reviewer need to make sure that all used assets are added to the config itself, and second we still need to build and run app container in CI to publish assets, upload them somewhere (as temporary build artifact) and only then start the web container build task as we won't have new assets if we run them in parallel (one workaround for the latter issue is run the job/task of building nginx container in an environment, in our case docker image, where you have php-fpm to do the asset publishing there before building the image, but I'm not sure that both will have same asset paths, haven't looked how those hash paths are generated, and ideally your CI job that build nginx container doesn't need to have php-fpm).

Not sure if @schmunk42 came across this problem as I haven't seen separation of web and app containers in his templates and don't know if he runs containerized setups in production. Monolith containers are first not recommended by docker (each container must do one specific task and not more) and second are not good for scaling, as you don't usually scale app and nginx equally.

[armpogart]

I can see that there no any easy way to solve the whole issue except not using AssetBundles at all, but including proper way to publish assets (without any combining/bundling, in our case combining/bundling is not the task of yii at all, we use webpack/gulp in CI to do it for production, but we still need to publish assets first before that) in core will still help.

And will you consider to include pure shell script (that can even be added to yii executable itself) that will parse the assetbundles and views and publish them without any dependency on php). Not sure if it will be possible in this case, but I like challenges writing pure shell scripts (i won't work on windows obviously and I'm not familiar with Windows cmd/PowerShell scripting, but it would be another task later) and maybe I can come up with something basic that will work in most common scenario.

[schmunk42]

Related issue about npm in Docker base-images: https://github.com/yiisoft/yii2-docker/issues/30

@armpogart Sorry I didn't notice the mentioning before.

In face we are running a combination of PHP and nginx in production on one image, even though this is not the recommended way by Docker, related issue: yiisoft/yii2-docker#12
In fact PHP-fpm + nginx is kind of a special setup, most other web-servers (also in other languages) are run from your application or you have a setup like the one with Apache, where PHP is a process within Apache.

Especially with asset-publishing it gets really hard if you separate the containers, since your published assets only make sense if they are served only from nginx (or webserver in general), not php. We also bundle our assets during docker build, we also usually test against those bundled versions, which later run in production.

[lav45]

@armpogart yiisoft/yii2#15772

[armpogart]

@lav45 Thanks, haven't noticed that PR, this solves the issue partially as I described earlier.

I still don't see any easy way to solve the issue, but will look through all the options.