drupal-modules / ads Goto Github PK

BUILD FAILED
exception 'BuildException' with message '/home/travis/build/mycognitive/ads/build.xml:171:39: '/home/travis/build/mycognitive/ads/src' is not a valid directory' in /usr/share/php/phing/tasks/system/ExecTask.php:198

More details:
https://travis-ci.org/mycognitive/ads/builds/19880822

Builds:
https://travis-ci.org/mycognitive/ads_features/builds/37815410 (previously working)
https://travis-ci.org/mycognitive/ads_features/builds/37846041

The following problems appears during ads_features build:

[exec] There was a problem enabling devel_generate.                         [error]
Execution of target "build-users" failed for the following reason: /home/travis/ads/phing/build-users.xml:21:27: Task exited with code 1

Build: https://travis-ci.org/mycognitive/ads_features/builds/36198908#L5195
File: phing/build-users.xml

When testing build installation process, I had the error:

ERROR 1045 (28000): Access denied for user 'ads@localhost'@'localhost' (using password: YES)
Property ${cmd.return} => 0

Despite of the error, build-ads started to continue with post-install-drupal and ads-install and at the end displaying the successful message.

Possibly we can add additional phing target which should be called before the site-install (as well as separately, e.g. test-db) to test the db credentials.

Despite the error, ads shows the successful built message.

Example case:

$ make
build-ads > build-drupal:

Drush needs to download a library from                               [error]
http://download.pear.php.net/package/Console_Table-1.1.3.tgz in order
to function, and the attempt to download this file automatically
failed because you do not have permission to write to the library
directory /usr/share/php/drush/lib. To continue you will need to
manually download the package from
http://download.pear.php.net/package/Console_Table-1.1.3.tgz, extract
it, and copy the directory into your /usr/share/php/drush/lib
directory.

build-ads > ads-make:


build-ads > build-ads:


BUILD FINISHED

Total time: 0.1570 seconds



Human readable log output:

2014-09-13 16:48:45 --------------------------------------------------------------------------------
2014-09-13 16:48:45 Downloading and building Drupal sources... (it may take a while)
2014-09-13 16:48:45 
2014-09-13 16:48:45 Done
2014-09-13 16:48:45 
2014-09-13 16:48:45 NOTE:
2014-09-13 16:48:45 Distribution can be found in the directory: /home/x/ads/src
2014-09-13 16:48:45 You can move your newly created distribution to your webroot.
2014-09-13 16:48:45 Or you can install it by the following command:
2014-09-13 16:48:45     phing ads-install

This can be fixed by checking if project.dir.src dir exists, if not, display that the build failed.

Target: build-drupal

Currently we're using static path for docroot, we need to find a way to pass SRCDIR environment variable into puppet.

File: puppet/ads.dev.configure-site.pp

Line:

path           => '/var/www/ads',

You can define/pass the global variable in .travis.yml (e.g.: DOCROOT="/var/www/ads"), if global variable is not set, default value should be /var/www

It seems mod_rewrite extension for Apache is not installed by ads.dev.pp. We need this for the clean URLs.

Set-up MySQL error log and slow queries via puppet.

Currently database is created in .travis.yml, it should be created in build.xml before the installation. Or check if drush has this ability.

CREATE DATABASE IF NOT EXISTS DBName; 

Clone https://github.com/mycognitive/ads_drush into sites/all/drush.
Files: build-ads-*-dev.make

Currently on pull request, Travis CI is testing the build by downloading the sources from the main repo. We need to find some way of testing the changed code which is requested.

Examples:
#46
https://travis-ci.org/mycognitive/ads/builds/35371736

Currently all contrib modules are downloaded into sites/all/module.
Contrib modules should be downloaded into sites/all/module/contrib sub-dir.

Build: https://travis-ci.org/mycognitive/ads_features/builds/36707562

Log:

JavaScript 68 passes, 0 fails, 0 exceptions, and 1 debug message
PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away: SELECT last_prefix FROM {simpletest_test_id} WHERE test_id = :test_id LIMIT 0, 1; Array
(
    [:test_id] => 1
)
 in dblog_watchdog() (line 160 of /var/www/ads/modules/dblog/dblog.module).
<h1>Uncaught exception thrown in shutdown function.</h1><p>PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away: DELETE FROM {semaphore} 
WHERE  (value = :db_condition_placeholder_0) ; Array
(
    [:db_condition_placeholder_0] =&amp;gt; 1919012937542b11fbcc3392.02561649
)
 in lock_release_all() (line 269 of /var/www/ads/includes/lock.inc).</p><hr />PHP Fatal error:  Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 2006 MySQL server has gone away' in /var/www/ads/includes/database/database.inc:2171
Stack trace:
#0 /var/www/ads/includes/database/database.inc(2171): PDOStatement->execute(Array)
#1 /var/www/ads/includes/database/database.inc(683): DatabaseStatementBase->execute(Array, Array)
#2 /var/www/ads/includes/database/database.inc(2350): DatabaseConnection->query('SELECT expire, ...', Array, Array)
#3 /var/www/ads/includes/lock.inc(167): db_query('SELECT expire, ...', Array)
#4 /var/www/ads/includes/lock.inc(146): lock_may_be_available('schema:runtime:...')
#5 /var/www/ads/includes/bootstrap.inc(424): lock_acquire('schema:runtime:...')
#6 /var/www/ads/includes/bootstrap.inc(446): DrupalCacheArray->set(Array)
#7 [internal function]: DrupalCacheArray->__destruct()
#8 {main}
  thrown in /var/www/ads/includes/database/database.inc on line 2171
PHP Stack trace:
PHP   1. _drupal_exception_handler() /var/www/ads/includes/bootstrap.inc:0
PHP   2. _drupal_log_error() /var/www/ads/includes/bootstrap.inc:2338
Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 2006 MySQL server has gone away' in /var/www/ads/includes/database/database.inc on line 2171

URL: https://packagist.org/

Currently there is Packaging error on every release:

No makefile for ads-7.x-1.0-alpha2 profile, skipping extended packaging.

See: https://www.drupal.org/node/2367649

In order for extended packaging to take place, the profile must have a file named drupal-org.make in the main directory of their profile.

See:

• https://www.drupal.org/node/642116
• https://www.drupal.org/node/751242

If project.drupal.build.type is set to 'dev', then invoke phing/build-dev-env.xml

Currently project.dir.src is hardcoded to src/ dir. Please find the method of specifying the custom path from the command line (e.g. ../), so it can be different per build.

Not having revision attribute could break the stable releases when new commits/features are introduced in development version of contrib modules.

Here is a good example:

projects[phone][version] = "2.x-dev"
projects[phone][download][revision] = 0df2e253a80f87f3fd886437d9021a06e7202de9

Bad example:

projects[field_group][version] = "1.x-dev"

You can limit revision only to enabled onces.

Use the following feature:
$ drush coder --help
--git Install this coder command as a .git pre-commit hook
to install coder checks as pre-commit hook.

You can create similar script to scripts/update_src.sh

See: https://www.genuineinteractive.com/blog/blog-posts/genuine-thoughts/enforcing-drupal-coding-standards-software-versioning-process/

1. Run: phing
2. Run: phing ads-install

Error:

     [exec] Module ads_advert cannot be enabled because it depends on the        [error]
     [exec] following modules which could not be found: ads_migrate
     [exec] Command dispatch complete                                               [notice]
Execution of target "build-site" failed for the following reason: Task exited with code 1
       [if] Error in IfTask
Execution of target "run-phing-scripts" failed for the following reason: Task exited with code 1
Execution of target "post-install-drupal" failed for the following reason: Task exited with code 1

File: phing/build-configure.xml
grep -P option is not supported on BSD/OSX, so it fails during the build:

     [exec] ads_setup was enabled successfully.                                  [ok]
     [exec] Command dispatch complete                                               [notice]
usage: grep [-abcDEFGHhIiJLlmnOoPqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
    [-e pattern] [-f file] [--binary-files=value] [--color=when]
    [--context[=num]] [--directories=action] [--label] [--line-buffered]
    [--null] [pattern] [file ...]
ads is a required module and can't be disabled.

Try to convert it without -P.

1. Go to: /admin/modules
2. Enable: 'Ads Search via DB' or 'Ads Search Solr'.
   'Ads Search' should be enabled automatically.
   It works from drush, but not from UI on the first time (when modules are not installed).

To re-try on the 2nd time, do first:

$ drush -y dis ads_search ads_search_db
$ drush -y pm-uninstall ads_search_db ads_search 

https://coveralls.io/docs/php

E.g.
https://github.com/zendframework/zf2
https://coveralls.io/r/smalot/drupal-services-client

Google: site:coveralls.io drupal

URL: /admin/structure/types/manage/advert/fields
Hide group_advert_hidden from edit page so field_ad_link_source and field_ad_category_node_ref won't be visible.

After build, the site only displays white page with 'footer' text.

Build property file has the following password:

project.db.password=  aa$2bb!

When when drush executes mysql, it removes $2, so it looks like:

 mysql --database=db_name --host=localhost --port=3306 --user=username --password='aabb!' -e "SELECT 1;"

Currently public & private dirs (set-dir-permission) are created on install. Shouldn't they be created on build, so the whole dir/file structure is ready for usage?

We need to specify datadir in Puppet manifest (file: ads.dev.configure-site.pp) as described below:

Puppet imposes its default settings if you don't override the values. You did not supply a value for datadir, for example, so Puppet would put the module's default value into my.cnf. Again - do compare the my.cnf before and after.

See: http://stackoverflow.com/questions/25964936/why-puppet-is-removing-my-database-in-travis-ci-on-mysql-restart?noredirect=1#comment40766988_25964936

Otherwise the CI build fails.

So basically the problem is when we apply this manifest:

sudo puppet apply puppet/ads.dev.configure-site.pp

the travis_ads_test database is gone, see the description in here.

See:
https://github.com/reload/phing-drupal-template/blob/master/build.xml
Forked at: https://github.com/kenorb/phing-drupal-template
which uses: https://github.com/kasperg/phing-drush-task

Save button shouldn't be visible on the first step of multi-step form.

User: anonymous
URL: /node/add/advert

Error: Invalid parameter restart on Class[Mysql::Server::Service] at /home/travis/ads/puppet/ads.dev.configure-site.pp:49 on node ads.localdomain
Wrapped exception:
Invalid parameter restart

Build: https://travis-ci.org/mycognitive/ads_features/builds/35784622

• [@self.6 ] Unable to clone ads_art from [email protected]:mycognitive/ads_art.git. [error]

• [@self.7 ] Unable to clone ads_travel from [email protected]:mycognitive/ads_travel.git. [error]

• [@self.7 ] Command dispatch complete [notice] [notice]

• [@self.5 ] Unable to clone ads_animals from [email protected]:mycognitive/ads_animals.git. [error]

• [@self.6 ] Unable to clone ads_services from [email protected]:mycognitive/ads_services.git. [error]

• [@self.6 ] Command dispatch complete [notice] [notice]

• [@self.5 ] Unable to clone ads_properties from [email protected]:mycognitive/ads_properties.git. [error]

• [@self.5 ] Command dispatch complete [notice] [ok]

• [@self.3 ] Unable to clone ads_jobs from [email protected]:mycognitive/ads_jobs.git. [error]

• [@self.3 ] Command dispatch complete [notice]

• [@self.2 ] Unable to clone ads_items from [email protected]:mycognitive/ads_items.git. [error]

• [@self.2 ] Command dispatch complete [notice]

• [@self.0 ] Unable to clone ads_finance from [email protected]:mycognitive/ads_finance.git. [error]

• [@self.0 ] Command dispatch complete [notice]

• [@self.1 ] Unable to download https://drupal.org/files/ctools-n1925018-12.patch. [error]

• [@self.1 ] Command dispatch complete [notice]

• [@self.4 ] Unable to clone ads_music from [email protected]:mycognitive/ads_music.git. [error] [ok]

• [@self.1 ] Unable to clone ads_it from [email protected]:mycognitive/ads_it.git. [error] [notice]
  [echo] Done.

  [echo] NOTE:
  [echo] Distribution can be found in the directory: .../html/ads/src
  [echo] You can move your newly created distribution to your webroot.
  [echo] Or you can install it by the following command:
  [echo] phing ads-install

build-ads > ads-make:

build-ads > build-ads:

BUILD FINISHED

Total time: 1 minutes 15.22 seconds

New variable project.drupal.build.search (in build.properties) should have 3 values: solr, db or none.
When solr, enable-search-solr.xml should be invoked (phing -f phing/enable-search-solr.xml).
When db, enable-search-db.xml should be invoked (phing -f phing/enable-search-db.xml).
When none, none of the search modules would be enabled.
By default, set to db.

Use MkdirTask to create directories to be more cross-compatible.

The following commit didn't work as expected:
c45a1d4

Check the build: https://travis-ci.org/mycognitive/ads/builds/35871621

See also:
http://stackoverflow.com/questions/25962022/how-to-create-intermediate-directories-when-required

Travis CI needs to pass.

There should be two messages for successful and unsuccessfully build, depending if 'src' dir is present or not.

File: build.xml

NOTE:
Distribution can be found in the directory: src

ADS build fails on the following error:

$ phing 
...
* [@self.1 ] ctools-7.x-1.3 downloaded.                                                                                                                                                                                                                             [ok]
* [@self.1 ] Unable to download https://drupal.org/files/ctools-n1925018-12.patch.                                                                                                                                                                                  [error]

The patch link is valid.

When downloading via wget, there is following error:

$ wget https://drupal.org/files/ctools-n1925018-12.patch
--2014-03-09 21:41:17--  https://drupal.org/files/ctools-n1925018-12.patch
Resolving drupal.org... 140.211.10.62, 140.211.10.16
Connecting to drupal.org|140.211.10.62|:443... connected.
ERROR: certificate common name `*.drupal.org' doesn't match requested host name `drupal.org'.
To connect to drupal.org insecurely, use `--no-check-certificate'.
Unable to establish SSL connection.

$ wget -V
GNU Wget 1.11.4 Red Hat modified

If environment variable ADS_SEARCH is defined, the value should be set instead of project.drupal.configure.search.
In this way, we can improve our testing with Travis CI by:

env:
  matrix:
    - ADS_SEARCH=db
    - ADS_SEARCH=solr

You need drush command to test it.

Reproducible steps:

drush -y dl ads --dev
cd ads-*
# re-edit/configure the build.properties
phing -D out=..

Currently there is message:

Base path .. already exists.

And the build doesn't proceed and it doesn't fail. Can be this achievable?

Files to change: phing files (*.xml files).

So the scenario is that user can go into public_html/ dir, then download the distribution, then inside the ads folder I want to instal it straight into public_html/ which is actually one directory up.

Like:

cd public_html
drush dl --dev
cd ads-*
phing -D out=..

So the build will generate all www files in public_html (but not in src/), without need to move the files around.

So the goal is to have 2 commands (drush & phing or make) to do the whole site built into public_html/ (without moving the files around). Eventually we could have some parameter to specify the output, so distribution is downloaded outside of public_html/, then making the build in specific folder such as public_html/ (phing -D out=/foo/public_html or out=/foo/public_html make or similar). But this folder should exist.

Errors which are present:

Error: Unknown Puppet subcommand 'module'
$ sudo puppet apply puppet/travis.pp
Could not find class pear for ads.localhost

Build: https://travis-ci.org/mycognitive/ads/builds/35581111

Something that could be related:

• https://answers.launchpad.net/ubuntu/+source/puppet/+question/213144
• https://github.com/treehouseagency/puppet-pear/issues/1

See: drush help archive-backup
Section: after_failure

On build, the following error appears:

Command dispatch complete                                               [notice]
WD search_api: SearchApiException while adding index Adverts to      [error]
server Ads Search Server: SQLSTATE[42S02]: Base table or view not
found: 1146 Table 'ads_dev.search_api_db_ads_adverts_title' doesn't
exist in SearchApiDbService->fieldsUpdated() (line 684 of
/home/kenorb/ads/src/sites/all/modules/search_api_db/service.inc).
WD search_api: SearchApiException while updating the fields of index [error]
Adverts on server Ads Search Server: SQLSTATE[42S02]: Base table or
view not found: 1146 Table 'ads_dev.search_api_db_ads_adverts_title'
doesn't exist in SearchApiDbService->fieldsUpdated() (line 684 of
/home/kenorb/ads/src/sites/all/modules/search_api_db/service.inc).
WD search_api: SearchApiException: Could not index items since       [error]
important pending server tasks could not be performed. in
search_api_index_specific_items() (line 1522 of
/home/kenorb/ads/src/sites/all/modules/search_api/search_api.module).

URL: https://github.com/mycognitive/ads, https://github.com/mycognitive/ads_features
File: README.md

Links in README file should point to the correct branch (page) with list of builds, e.g. https://travis-ci.org/mycognitive/ads/branches, or even more specific (7.x-1.x, master, etc).

Currently they point to the same page.

Est. 1h

Currently vhost is hardcoded to ads.localhost. We need to find the way of passing this variable dynamically.

Links:

• http://serverfault.com/questions/63401/puppet-referencing-variables-from-a-define-in-a-template

Related: #55

Add ability to specify for phing different location of build.properties config file.

Home Page: https://www.openhub.net/

E.g.
https://www.openhub.net/
https://github.com/Wikidata/Wikidata-Toolkit

The build test should stop after BUILD FAILED as currently it's executing all commands on the broken build:

build-ads > install-drupal:
Execution of target "install-drupal" failed for the following reason: /home/travis/build/mycognitive/ads/build.xml:172:39: '/home/travis/build/mycognitive/ads/src' is not a valid directory
BUILD FAILED
/home/travis/build/mycognitive/ads/build.xml:172:39: '/home/travis/build/mycognitive/ads/src' is not a valid directory
make: *** [install] Error 1
The command "sudo make install" exited with 2.
0.01s$ sudo mv $TRAVIS_BUILD_DIR/src/* $SRCDIR/
mv: cannot stat `/home/travis/build/mycognitive/ads/src/*': No such file or directory
The command "sudo mv $TRAVIS_BUILD_DIR/src/* $SRCDIR/" exited with 1.
0.01s$ sudo chmod -R 777 $SRCDIR/
The command "sudo chmod -R 777 $SRCDIR/" exited with 0.
0.02s$ sudo wget -qO- http://ads.localhost/ | cat
The command "sudo wget -qO- http://ads.localhost/ | cat" exited with 0.
0.01s$ pwd
/home/travis/build/mycognitive/ads
The command "pwd" exited with 0.
0.25s$ cd $SRCDIR && pwd && ls && echo "$DRUSH" && $DRUSH -y en simpletest && php ./scripts/run-tests.sh --php `which php` --url http://ads.localhost/ --concurrency 16 --verbose System
/var/www/ads
sudo drush -y -r /var/www/ads --uri=http://ads.localhost/
Command pm-enable needs a higher bootstrap level to run - you will   [error]
need to invoke drush from a more functional Drupal environment to run
this command.
The drush command 'en simpletest' could not be executed.             [error]
The directory /var/www/ads does not contain a valid Drupal           [error]
installation
The command "cd $SRCDIR && pwd && ls && echo "$DRUSH" && $DRUSH -y en simpletest && php ./scripts/run-tests.sh --php `which php` --url http://ads.localhost/ --concurrency 16 --verbose System" exited with 1.
Done. Your build exited with 1.

$ phing ads-install
Starting Drupal installation. This takes a few seconds ...
WD php: Exception: Sorry, the profile you have chosen cannot be loaded. in install_load_profile() (line 1372 of src/includes/install.core.inc).

This error happening with the recent version.
Could be related: #87

Reproducible steps:

Requirements: You need phing and drush installed.

1. Close the repo and cd ads.

2. Build it by phing command then re-edit build.properties (just remove properties.notset), then run phing again. (you don't need to repeat this step again)

3. Create empty db (e.g. CREATE DATABASE ads) if required.

4. Install the profile from the command-line (change your db url to yours):

   $ drush -r src/ --verbose --yes site-install ads --db-url=mysqli://root:root@localhost:3306/ads --account-name=admin --account-mail="[email protected]" --account-pass="admin" --site-mail="[email protected]" --site-name="ADS" install_configure_form.site_default_country="GB" install_configure_form.date_default_timezone="Europe/London" update_notifications.update_status_module="1" variables.file_private_path="sites/default/private/files"
   

You should see the error:

exception 'Exception' with message 'Sorry, the profile you have chosen cannot be loaded.' in /Users/kenorb/temp/ads/ads/src/includes/install.core.inc:1377 [error]

When xdebug is installed, the following Stack trace is produced:

Stack trace:
#0 ~/temp/ads/ads/src/includes/install.core.inc(471): install_load_profile(Array)
#1 ~/temp/ads/ads/src/includes/install.core.inc(339): install_run_task(Array, Array)
#2 ~/temp/ads/ads/src/includes/install.core.inc(77): install_run_tasks(Array)
#3 ~/.composer/vendor/drush/drush/includes/drush.inc(719): install_drupal(Array)
#4 ~/.composer/vendor/drush/drush/includes/drush.inc(705): drush_call_user_func_array('install_drupal', Array)
#5 ~/.composer/vendor/drush/drush/commands/core/drupal/site_install_7.inc(80): drush_op('install_drupal', Array)
#6 ~/.composer/vendor/drush/drush/commands/core/site_install.drush.inc(289): drush_core_site_install_version('ads', Array)
#7 [internal function]: drush_core_site_install('ads', 'install_configu...', 'install_configu...', 'update_notifica...', 'variables.file_...')
#8 ~/.composer/vendor/drush/drush/includes/command.inc(364): call_user_func_array('drush_core_site...', Array)
#9 ~/.composer/vendor/drush/drush/includes/command.inc(215): _drush_invoke_hooks(Array, Array)
#10 [internal function]: drush_command('ads', 'install_configu...', 'install_configu...', 'update_notifica...', 'variables.file_...')
#11 ~/.composer/vendor/drush/drush/includes/command.inc(183): call_user_func_array('drush_command', Array)
#12 ~/.composer/vendor/drush/drush/lib/Drush/Boot/BaseBoot.php(62): drush_dispatch(Array)
#13 ~/.composer/vendor/drush/drush/drush.php(70): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#14 ~/.composer/vendor/drush/drush/drush.php(11): drush_main()
#15 {main}

The same should probably happen when installing from GUI.
The same happens when run: phing ads-install when using project.drupal.build.stable=false in property file.

The broken logic or something missing could be in ads.profile, ads.install or in includes/*.inc files, or somewhere else.

Google: "Sorry, the profile you have chosen cannot be loaded" for more details.

By default the ads build fails on some Linux instances (See: #42), because Console_Table PHP pear is missing.
The dependency should be added to puppet recipe (puppet/ads.dev.pp).

Manual installation:

sudo pear install Console_Table

Disable build.debug.xml by default and move it into phing/includes.
Debug logging can be enabled optionally by one of the project.debug.* option or by running phing -debug.

Create new variable in build.properties to specify the languages by comma which would be automatically imported and enabled during build install.