symbiote / silverstripe-build

Phing build settings and tasks for the silverstripe-base project

Requirments

• Phing
• php cli
• PEAR (with a version of phpunit supported by silverstripe)

Installation

• Edit scripts/build.xml and change the name attribute to
• Copy scripts/build.properties.sample to scripts/build.properties
• Edit build.properties and change the DB configuration and rewrite.base settings.
• Run phing -f scripts/build.xml to make sure everything installs
• If you require additional modules, add them into the scripts/dependent-modules file, then run phing -f scripts/build.xml update_modules
• Run phing -f scripts/build.xml test to make sure everything's working as expected

Phing targets

Commonly used targets

• build
• test
• package

Helpful for development:

• pak - builds a .sspak file at mysite/build/site.sspak . Useful for CI
• phpstan - Runs PHPStan + PHPStan for SilverStripe if it's installed via Composer. Useful for static analysis of PHP code.
  • Define folders to scan by placing following in your build.xml file, underneath the <project> tag.
    • <property name="phpstan.dir" value="mysite/src mysite/tests" />

Composer scripts

If your themes use yarn based dependencies, you can add the following post-install scripts to be triggered by a composer call, which will ensure the node_modules are installed correctly

    "scripts": {
		"post-update-cmd": [		
	            "Symbiote\\Build\\ThemeBuilder::run"
		],
		"post-install-cmd": [
	            "Symbiote\\Build\\ThemeBuilder::run"
		]
	},

Optional Scripts

There are three scripts that may optionally be used for your projects, and can be done using the following commands.

sh build/scripts/cache

This will clear out any project cache files (for all projects), and is basically the forced equivalent of doing a ?flush for everything in your site.

sh ~/path/to/permissions

This requires you to update the "{user}" and as such will need to be copied out to a location of your choice, and will apply the appropriate owner and permissions to both the cache and repository.

sh build/scripts/recursive-status

This will recursively trigger a "git status" on each module directory found within your repository, primarily so you check for changes that may have been made in a module that hasn't been included in the repository code base.

sh build/scripts/recursive-status assume-unchanged-listing

When you have patched files (resulting in them coming up as being modified during a recursive-status), you can git update-index --assume-unchanged {file_name}, and use this script with a parameter. This will not only list out the files that have been assumed unchanged, but it will also force the files to reflect the upstream. This makes sure you don't have local changes that have been accidentally made, however it will also remove any patches that were put in place. Therefore, you can run this and then phing to ensure the patches remain in place.

Excluding files from packages

Create a {root}/.pkgignore file with a set of paths to be excluded from built up package files

vendor/un-used/module/
themes/mytheme/node_modules/
mysite/cypress/

silverstripe-build's Issues

SilverStripe SQLite module doesn't currently support Polymorphic relationships

See: Polymorphic database fields are bad news for reconstruction #45

SS4 Flush

In SS4, when you run a phing, the /dev/build with flush step doesn't actually trigger a flush. I've noticed that opening a page after phing results in a wall of errors until I append ?flush=1 to the URL. I don't recall this being a problem with SS3, so my assumption is that something underlying has changed.

Huge speedup with "composer install"

On projects with a lot of modules, by switching to --prefer-dist with our build.xml files, we can get a huge speed up.

--prefer-source: 14 mins
--prefer-dist: 7 mins
--prefer-dist + composer cache: 2 mins

Phing build tasks need to error correctly

We need to change the phing buildfile.xml and individual PHP tasks as much as possible to ensure any error outputted during tasks or executions causes a build exception so Jenkins can mark builds as failed.

An example is the unit tests currently throw away output and do not report anything back to Phing/Jenkins at https://github.com/silverstripe-australia/silverstripe-build/blob/master/SilverStripeTestTask.php#L88

This should be changed to capture the output and return code and throw a BuildException() on any failed tests or erroneous output.

All other taskdef's need reviewing to ensure that if their PHP task that runs produces output or errors, that is handled instead of thrown away.

Example of integrating PHPStan

Example use of calling in CLI (when developing on your local):
phing phpstan

Add to phing ci_test target: (to execute in CI)

<!-- Override ci_test from build/buildfile.xml to include phpstan -->
<target name="ci_test" depends="build">
	<property name="flush" value="1" override="true" />
	<property name="build" value="1" override="true" />
	<phingcall target="phpstan" />
	<phingcall target="test" />
</target>

Mirror:

<property name="phpstan.dir" value="mysite/code my-internal-module/code" />
<target name="phpstan" depends="get-composer">
	<!-- 
		Example CLI use: `phing phpstan -Dphpstan.level=2`
	-->
	<if>
		<available file="${project.basedir}/vendor/bin/phpstan" />
		<then>
			<property name="phpstan.installed" value="1" />
		</then>
	</if>
	<if>
		<available file="${project.basedir}/vendor/silbinarywolf/silverstripe-phpstan" type="dir" />
		<then>
			<property name="phpstan.silverstripe_installed" value="1" />
		</then>
	</if>
	<if>
		<isset property="phpstan.debug"/>
		<then>
			<property name="phpstan.debug_cmd" value="--debug" />
		</then>
		<else>
			<property name="phpstan.debug_cmd" value="" />
		</else>
	</if>

	<if>
		<not><isset property="phpstan.disabled"/></not>
		<then>
			<!-- Run PHPStan if installed -->
			<if>
				<isset property="phpstan.installed"/>
				<then>
					<if>
						<isset property="phpstan.silverstripe_installed"/>
						<then>
							<!-- Default values -->
							<if>
								<not><isset property="phpstan.level" /></not>
								<then>
									<property name="phpstan.level" value="2" />
								</then>
							</if>
							<if>
								<not><isset property="phpstan.dir" /></not>
								<then>
									<property name="phpstan.dir" value="mysite" />
								</then>
							</if>
							<!-- Run PHPStan -->
							<exec executable="vendor/bin/phpstan" 
								  passthru="true" 
								  checkreturn="true"
							>
								<arg value="analyse" />
								<arg line="${phpstan.dir}" />
								<arg value="-c" />
								<arg value="phpstan.neon" />
								<arg value="-a" />
								<arg value="vendor/silbinarywolf/silverstripe-phpstan/bootstrap.php" />
								<arg value="--level"/>
								<arg value="${phpstan.level}"/>
								<arg line="${phpstan.debug_cmd}"/>
							</exec>
						</then>
						<else>
							<echo msg="PHPStan SilverStripe extension is not installed in this project and was not executed." />
						</else>
					</if>
				</then>
				<else>
					<echo msg="PHPStan is not installed in this project and was not executed." />
				</else>
			</if>
		</then>
		<else>
			<echo msg="PHPStan has been disabled via the 'phpstan.disabled' property." />
		</else>
	</if>
</target>

"build.properties.sample" copies to "build.properties" within the build module folder

The problem
For some reason, the build.properties.sample file will copy itself to build.properties within the same folder. This caused me to hit an issue where I:

• Ran Phing with an older copy of the build module (build.properties was generated)
• Update the build module
• Deleted my .env file
• Ran phing again (Expecting it to update the .env to use MySQLDatabase, not SQLLite using the build.properties.sample file defined)
• It didn't work.
• After investigation, I discovered that it was trying to pull values from build.properties, which only gets generated when you first spin up the project. Keep in mind this file is generated within the build module folder so it is not committed up.

The solution
Just use build.properties.sample within buildfile.xml and remove all references to build.properties.

Move configs from local.conf.php to _ss_environment.php

Why?
All define() constants done in _ss_environment.php are usable in caching layers or before building the various ConfigManifest files, etc.

By putting database, environment and log settings in this file, we can update our caching layers use these values and/or store cache configs there. (One centralized place for server configurations sounds like a good idea to me!)

Silverstripe versions?
IIRC, _ss_environment.php only works 3.2+, but maybe its in 3.1+ and just not documented.

Fix 404/500 pages generated during dev/build on Build Servers

We should add the following to "non_interactive" so that 404/500 pages get generated properly without an error/warning occuring during dev/build on ErrorPage.php
<mkdir dir="${project.basedir}/assets" />

<target name="non_interactive" if="ni_build">
	<!-- we'll make sure that there's a build.properties file, even if we're just using the default  -->
	<delete file="${project.basedir}/build/build.properties" />
	<copy tofile="${project.basedir}/build/build.properties" file="${project.basedir}/build/build.properties.sample" />
	<delete dir="assets" />
	<delete file="${project.basedir}/mysite/local.conf.php" />
	<delete file="${project.basedir}/mysite/_config/local.yml" />
	<delete file="${project.basedir}/test-assist/testing.conf.php" />
	<property name="properties_exist" value="true" override="true" />
	<property file="${project.basedir}/build/build.properties"></property>
	<mkdir dir="${project.basedir}/silverstripe-cache" />
</target>

ApplyPatchesTask throws false error

silverstripe-build/ApplyPatchesTask.php

Lines 47 to 57 in 68bc7d2

	if ($exec_check_return) {
	exec($exec_command, $exec_output, $exec_return);
	$this->log(implode("\n", $exec_output) . "\n\n");
	if ($exec_return) {
	throw new Exception($this->formatWithColor("fg-white, bg-red", "The patch produced errors"));
	}
	} else {
	$this->log("This patch is already applied - skipping\n", Project::MSG_WARN);
	$this->project->setProperty('patches_applied', true);
	}

• Have a patch file that's already been applied.
• Add another hunk to the patch (yet to be applied). Targeting another file in my case.
• Run phing apply_patches.
• Task prints error due to L52 which halts phing (and no error message).
• Verbose output shows that the patch is applied successfully and no error should be thrown.
• Seems that L51's expectation for the value of $exec_return is incomplete.
  • From what I can see, $exec_check_return is 0 when nothing will be applied, and 1 when a change will be applied. And $exec_return is 1 if anything(?) wasn't applied.

It's proven hard to find just what the return values are for patch so not sure what the correct implementation would be. It's not a major issue, just raising for now for the record.

Backport PHPStan to SilverStripe 3.X build module

Continuing from:
#25

We now have this PHPStan target in our SilverStripe 4 compatible build module, as seen here:

silverstripe-build/buildfile.xml

Line 566 in a227dc8

<target name="phpstan" depends="get-composer">

This exact PHPStan target code worked on a project that was running SilverStripe 3.6 and it just worked after I updated to SilverStripe 4, so in theory backporting should literally be just copying that chunk into the SS 3.X branches.