vbench missing some classes about scikit-learn-benchmarks

[vene]

Did you read the quick-start guide at http://vene.github.com/scikit-learn-speed/quick_start.html ?

It links to a specific branch on my vbench repo. Sorry for having it so convoluted.

[jenkins-scikit-learn]

If you want to reuse vbench, I would appreciate any little help in cleaning
it up! The problem is my changes are no longer backwards compatible with
the pandas vbench suite (and I don't intend them to be, since that version
is pretty hackish) so I couldn't push towards a new official vbench API
without having at least another project (except scikit-learn) using it!

However, the way my branch is designed now allows for vbench to be extended
and used with arbitrary code as long as an appropriate Benchmark subclass
is implemented.

Thanks,
Vlad

On Fri, Feb 22, 2013 at 8:12 AM, Vlad Niculae [email protected]:

Did you read the quick-start guide at
http://vene.github.com/scikit-learn-speed/quick_start.html ?

It links to a specific branch on my vbench repo. Sorry for having it so
convoluted.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1#issuecomment-13932882.

[erg]

Ah! I think the quick start guide doesn't mention the abstract_benchmarks branch, it just links to it.

I don't have any other projects to apply your vbench to at this time. I mainly wanted to add a few more benchmarks.

[erg]

Now I'm getting this error:

[erg@pliny scikit-learn-speed]$ [master*] make PYTHON=python2
python2 benchmarks/run_suite.py quick
Traceback (most recent call last):
  File "benchmarks/run_suite.py", line 16, in <module>
    module_dependencies=dependencies)
  File "/usr/lib/python2.7/site-packages/vbench/runner.py", line 42, in __init__
    self.checksums = [b.checksum for b in benchmarks]
  File "/usr/lib/python2.7/site-packages/vbench/benchmark.py", line 175, in checksum
    return hashlib.md5(self.setup + self.code + self.cleanup).hexdigest()
TypeError: cannot concatenate 'str' and 'list' objects
make: *** [run] Error 1

[vene]

I think you actually need to use this branch. Allow me to update the quick start guide, it was pointing to the wrong branch.

https://github.com/vene/vbench/tree/abstract_multistep_benchmarks

[vene]

Actually the quick start guide is accurate, just not explicit enough: simply click on the link to my repo and it will take you to the correct branch

[erg]

I wasn't observant enough to realize I was in a branch on the github site.

[erg]

scikit-learn-speed requires sphinx-build.

[erg]

Another problem--I don't have a python executable, just python2and the scripts are hardcoded to python instead of using the environment variable PYTHON.

Here's where I hardcoded it to python2.

[erg@pliny scikit-learn-speed]$ [master*] git diff
diff --git a/benchmarks/suite.py b/benchmarks/suite.py
index fb9daea..0be63ea 100644
--- a/benchmarks/suite.py
+++ b/benchmarks/suite.py
@@ -68,10 +68,10 @@ except:
     print "Cannot load configuration file. Reverting to defaults."

 PREPARE = """
-python setup.py clean
+python2 setup.py clean
 """
 BUILD = """
-python setup.py build_ext --inplace
+python2 setup.py build_ext --inplace
 """

 RST_BASE = 'doc'

[erg]

Finally, it sorta runs, but there are a ton of errors and I don't get any benchmark results in index.html.

[erg@pliny scikit-learn-speed]$ [master*] make PYTHON=python2
python2 benchmarks/run_suite.py quick
git clone [email protected]:scikit-learn/scikit-learn.git /tmp/vb_sklearn_tmp
fatal: destination path '/tmp/vb_sklearn_tmp' already exists and is not an empty directory.
Deleting /tmp/vb_sklearn first
rm -rf /tmp/vb_sklearn
git clone /tmp/vb_sklearn_tmp /tmp/vb_sklearn
Cloning into '/tmp/vb_sklearn'...
done.
python2 setup.py clean
Partial import of sklearn during the build process.
/usr/lib/python2.7/site-packages/numpy/distutils/system_info.py:1494: UserWarning: 
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
sklearn/setup.py:64: UserWarning: 
    Blas (http://www.netlib.org/blas/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [blas]) or by setting
    the BLAS environment variable.
  warnings.warn(BlasNotFoundError.__doc__)
blas_opt_info:
blas_mkl_info:
  libraries mkl,vml,guide not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib']
  NOT AVAILABLE

atlas_blas_threads_info:
Setting PTATLAS=ATLAS
  libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib']
  NOT AVAILABLE

atlas_blas_info:
  libraries f77blas,cblas,atlas not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib']
  NOT AVAILABLE

blas_info:
  FOUND:
    libraries = ['blas']
    library_dirs = ['/usr/lib64']
    language = f77

  FOUND:
    libraries = ['blas']
    library_dirs = ['/usr/lib64']
    define_macros = [('NO_ATLAS_INFO', 1)]
    language = f77

running clean

cp -R /usr/lib/python2.7/site-packages/vbench/scripts/vb_run_benchmarks.py /tmp/vb_sklearn
cp -R benchmarks/deps.py /tmp/vb_sklearn
cp -R benchmarks/templates.py /tmp/vb_sklearn
cp -R benchmarks/data /tmp/vb_sklearn
SKIPPING BLACKLISTED edb7592
python2 benchmarks/generate_rst_files.py quick
Generating rst file for Ridge-arcene
Generating rst file for Ridge-madelon
Generating rst file for Lars-minimadelon-oney
Generating rst file for Lars-madelon
Generating rst file for LassoLars-minimadelon-oney
Generating rst file for LassoLars-madelon
Generating rst file for Lasso-minimadelon-oney
Generating rst file for Lasso-madelon
Generating rst file for ElasticNet-minimadelon-oney
Generating rst file for OrthogonalMatchingPursuit-minimadelon
Generating rst file for OrthogonalMatchingPursuit-madelon
Generating rst file for SGDClassifier-madelon
Generating rst file for SGDClassifier-newsgroups
Generating rst file for LogisticRegression-arcene
Generating rst file for LogisticRegression-madelon
Generating rst file for ARDRegression-minimadelon-oney
Generating rst file for ARDRegression-blobs
Generating rst file for BayesianRidge-arcene
Generating rst file for BayesianRidge-madelon
sphinx-build -b html -P -d benchmarks/build/doctrees   doc benchmarks/build/html
Running Sphinx v1.2pre
loading pickled environment... done
building [html]: targets for 2 source files that are out of date
updating environment: 0 added, 2 changed, 0 removed
reading sources... [100%] vb_linear_model                                                                                                                                                     
doc/vbench/Ridge-arcene.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/Ridge-madelon.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/Lars-minimadelon-oney.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/Lars-madelon.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/LassoLars-minimadelon-oney.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/LassoLars-madelon.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/Lasso-minimadelon-oney.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/Lasso-madelon.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/ElasticNet-minimadelon-oney.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/ElasticNet-minimadelon-oney.txt:61: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/OrthogonalMatchingPursuit-minimadelon.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/OrthogonalMatchingPursuit-madelon.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/SGDClassifier-madelon.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/SGDClassifier-madelon.txt:61: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/SGDClassifier-newsgroups.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/SGDClassifier-newsgroups.txt:61: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/LogisticRegression-arcene.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/LogisticRegression-arcene.txt:61: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/LogisticRegression-madelon.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/LogisticRegression-madelon.txt:61: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/ARDRegression-minimadelon-oney.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/ARDRegression-blobs.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/BayesianRidge-arcene.txt:38: ERROR: Content block expected for the "container" directive; none found.
doc/vbench/BayesianRidge-madelon.txt:38: ERROR: Content block expected for the "container" directive; none found.
looking for now-outdated files... none found
pickling environment... done
checking consistency... /home/erg/python/scikit-learn-speed/doc/quick_start.rst:: WARNING: document isn't included in any toctree
done
preparing documents... done
writing output... [100%] vb_linear_model                                                                                                                                                      
writing additional files... genindex search
copying images... [100%] vbench/figures/LogisticRegression-madelon-step1-timing.png                                                                                                           
copying static files... done
dumping search index... done
dumping object inventory... done
build succeeded, 25 warnings.
touch benchmarks/build/html/.nojekyll

Build finished. The HTML pages are in benchmarks/build/html.

[vene]

Judging by the log I believe the current commit is blacklisted because it failed before you changed the hardcoded python call. Try deleting the database and running again.

[jeremiedbb]

This repo is being repurposed to host results of a new asv benchmark suite. I'm closing this.