marmelab / sonataelasticabundle

Power the Sonata Admin list view and filters by an ElasticSearch index to speed up navigation.

Archived Repository This code is no longer maintained. Feel free to fork it, but use it at your own risks.

SonataElastica

Power the Sonata Admin list view and filters by an ElasticSearch index to speed up navigation.

The Sonata Admin Bundle provides a web UI to many types of persistence (RDBMS, MongoDB, PHPCR), some of which have limited query capabilities. If you already have an ElasticSearch index for a given model, this bundle allows you to use this index instead of the native repository query system. This may provide a great performance boost, depending on your data structure and indexes.

Requirements

This bundle depends on:

• sonata-project/admin-bundle
• friendsofsymfony/elastica-bundle

It can be used with Doctrine (ORM, ODM, PHPCR-ODM), or Propel ORM:

• sonata-project/doctrine-orm-admin-bundle - after version @15ed873424fb30af43569014a48f6d216fdefe78
• sonata-project/propel-orm-admin-bundle

Installation

Step 1: Download using composer

Require marmelab/sonata-elastica-bundle in your composer.json file:

{
    "require": {
        "marmelab/sonata-elastica-bundle": "dev-master"
    }
}

Then run composer.phar install as usual.

Step 2: Enable the bundle

Enable the bundle in the kernel:

<?php
// app/AppKernel.php

public function registerBundles()
{
    $bundles = array(
        // ...
        new Marmelab\SonataElasticaBundle\MarmelabSonataElasticaBundle(),
    );
}

Configuration of Elastica Index

For each model that you index, the identifier field (id) must be specified, with the type integer. Configure other fields as multi_field is not required anymore, see UPGRADE-2.0-dev.md.

For string fields, if you want to be able to sort and to search on them, you may want to declare them as multi_field, with two "sub-fields"

• The first one, named after the field, required for filters, must use index: analyzed
• The second one, named raw, required for sorting, must use index: not_analyzed

For more information about this, see ElasticSearch documentation (or this one, as the 1.0 version of ElasticSearch was released recently).

Example

book:
    mappings:
        id: {type: integer}
        title:
            type: multi_field
            fields:
                title: { type: string, index: analyzed }
                raw: { type: string, index: not_analyzed }
        created_at: { type: date }
        ...

Then, in your Admin class, configure the field to use the not_analyzed sub-field:

    protected function configureListFields(ListMapper $listMapper)
    {
        $listMapper
            ->add('title', 'string', array(
                'sortable' => true,
                'sort_field_mapping' => ["fieldName" => "title.raw", "type"=> "string"] // To be able to sort by title.raw which is not_analyzed
            ))
            ...
        ;
    }

Configuration

To enable ElasticSearch for a given model admin, you only need to edit the sonata.admin tag in the services.xml:

• Add a fourth empty argument to the admin service definition.
• Add two attributes in the sonata.admin tag:
  • searcher="elastica"
  • search_index="", set the value of your elastica index type

Example

For a Book entity:

<service id="book.admin" class="Acme\BookBundle\Admin\BookAdmin">
    <argument/>
    <argument>Acme\BookBundle\Entity\Book</argument>
    <argument>AcmeBookBundle:BookCRUD</argument>
    <argument/>
    <tag name="sonata.admin" group="Content" label="Books" manager_type="orm"
         searcher="elastica" search_index="acme.book"/>
</service>

The search_index=acme.book corresponds to the following type of configuration for elastica bundle:

fos_elastica:
    clients:
        default: { host: %elasticsearch_server_url%, port: %elasticsearch_server_port% }
    indexes:
        acme:
            types:
                book:
                    mappings:
                        ...
                author:
                    mappings:
                        ...

Optional: Bypass ORM Hydration Completely

By default, this bundle uses ElasticSearch to find the ids of the entities or documents matching the request, then queries the underlying persistence to get real entities or documents. This should always be a fast query (since it's using a primary key), but it's also a useless query. Indeed, in most cases, all the data required to hydrate the entities is already available in the ElasticSearch response.

This bundle allows to use a custom transformer service to hydrate ElasticSearch results into Entities, therefore saving one query. To enable this transformer, add the fastgrid parameter to the admin tag in services.xml:

Using the "basic" transformer:

 <tag name="sonata.admin" group="Content" label="Books" manager_type="orm"
         fastGrid="true" searcher="elastica" search_index="acme.book"/>

Using your custom transformer:

 <tag name="sonata.admin" group="Content" label="Books" manager_type="orm"
         fastGrid="true" transformer="my.custom.transformer.service" searcher="elastica" search_index="acme.book"/>

The default transformer does basic hydration using setters and makes a few assumptions, like the fact that entities provide a setId() method. You can of course use a custom transformer to implement a more sophisticated hydration logic, by providing your service's id. The transformer class must have a transform method, converting an array of elastica objects into an array of model objects, fetched from the doctrine/propel repository. The transformer class should also have a setter for the objectClass attribute.

Optional: Use mapping for fields

To match fields between Elastica index and your application, you can configure the mapping for your entity as a parameter collection:

<parameter key="book.admin.elastica.mapping" type="collection">
    <parameter key="contentType">_type</parameter>
    <parameter key="publicationDate">publication_timestamp</parameter>
    <parameter key="lastUpdateDate">last_update_timestamp</parameter>
</parameter>

Then specify this parameter in your tag admin

 <tag name="sonata.admin" group="Content" label="Books" manager_type="orm"
         search_index="acme.book"
         fields_mapping="book.admin.elastica.mapping" />

Optional: Define a custom filter for your admin

You can specify a custom filter (using elastica filter classes) for your entity admin. Simply add a getExtraFilter() method in the admin class.

For example, if in my book admin list I want to fetch only the ones that are in a PDF or epub format:

// in Acme\BookBundle\Admin\BookAdmin
use Elastica\Filter\Terms;

...

public function getExtraFilter() {
    $filter = new Terms();
    $filter->setTerms('format', array('pdf', 'epub'));

    return $filter;
}

Optional: Use a custom form filter type

To use a custom form filter class, specify it in the admin tag:

 <tag name="sonata.admin" group="Content" label="Books" manager_type="orm"
         search_index="acme.book"
         search_form="my.custom.filter.form_type" />

License

This bundle is available under the MIT License, courtesy of marmelab.

sonataelasticabundle's Issues

Hi, I want to have a filter (key words) to search in PDF files which related to my entity. Is this possible in the filter admin ?

Composer issue

I have some troubles to download your bundle with composer.
Composer give me the following error:

Your requirements could not be resolved to an installable set of packages.

Problem 1
- The requested package marmelab/sonata-elastica-adapter-bundle could not be found in any version, there may be a typo in the package name.

Potential causes:

• A typo in the package name
• The package is not available in a stable-enough version according to your minimum-stability setting
  see https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion for more details.

Read http://getcomposer.org/doc/articles/troubleshooting.md for further common problems.

So, my "minimum-stability" is set to "alpha". Maybe a typo in the package name?

How to configure it in order to work with sonata global search ?

Forgot something in the documentation?

I try your bundle today but after done all that you write in the documentation, my list page doesn't use elasticsearch? I try to turn off elasticsearch to make sure.

here my service :

report.daily_snapshot:
    class: XXX\ReportBundle\Admin\DailySnapshotAdmin
    tags:
    - { name: sonata.admin, manager_type: orm,  transformer:"marmelab.book.elastica.transformer" , label: "Snapshots", searcher:"elastica", search_index:"website.daily" } 
    arguments: [null,  XXX\ReportBundle\Entity\DailySnapshot, SonataAdminBundle:CRUD, '@fos_elastica.finder.website.daily'] 

I extend my admin class with ElasticaAdmin as well.

something wrong?

findAll in ElasticaProxyRepository

I have an issue when I want to list one of my entities containing 2 fields boolean in the mapping.

It seems the method findAll that consider my booleans like a $param and the query called is "new QueryBool()", the debug bar show the following things:

Path: website/transaction/_search
Method: GET
{"query":{"bool":[]},"from":null,"sort":{"date_time.raw":{"order":"desc"}},"size":10000}
Time: 42.93 ms
Path: website/transaction/_search
Method: GET
{"query":{"bool":[]},"from":null,"sort":{"date_time.raw":{"order":"desc"}},"size":10000}
Time: 2.51 ms

and it returns nothing.

If I "var_dump" the parameter $param in the findAll method, I can see my 2 booleans.

I have an other entity mapped with 1 boolean and it works fine.

search_form set to null

I had this issue while using the bundle :

ContextErrorException: Catchable Fatal Error: Argument 5 passed to Marmelab\SonataElasticaBundle\Builder\ElasticaDatagridBuilder::__construct() must be an instance of Symfony\Component\Form\AbstractType, null given

After analyze i saw this line in DependencyInjection/AdminTagElasticaCompilerPass.php :

$searchForm = isset($attributes['search_form']) ? new Reference($attributes['search_form']) : null;
$datagridService = $this->createDatagridService($container, $id, $ormGuesserService, $proxyRepositoryService, $searchForm);

Whereas the construct of ElasticaDatagridBuilder does not support a null value for $searchForm

I think the null value for the tag search_form should be managed as null everywhere.

Compatibility with Sonata Admin 3.x

Hi,

Do you plan to make this bundle compatible with the last version of Sonata Admin ( 3.x ) ?

Thanks in advance for your answer.

Regards

[question] PHPCR-ODM support

Hi,

I was taking a look at the documentation, and I see that you are supporting PHPCR-ODM.
But when I look to the FOSElasticaBundle documentation (or code), I can't see any note about this driver.

I may need this for one of my project, so I may help you testing the integration, and updating the doc about how to process the interaction between elasticsearch and phpcr.

Best regards,
Yann