Multiple indexed views for object collection? about zodb

[mgedmin]

"Catalog" is the usual term in ZODB-land (because a catalog contains one or more indexes). These aren't provided by ZODB itself, but there are packages that provide them on top of ZODB. E.g. one of those is repoze.catalog.

I haven't used catalogs much personally, so I cannot give recommendations.

[jimfulton]

Good answer. Thanks Marius.

To elaborate a bit, catalogs are objects that maintain multiple indexes on
a collection of objects. They provide for querying on one or more among
multiple indexes. When querying multiple indexes. They rely on some
low-level machinery provided by BTrees for doing set operations on results
from searching multiple indexes.

Jim

On Thu, Feb 11, 2016 at 8:13 AM, Marius Gedminas [email protected]
wrote:

"Catalog" is the usual term in ZODB-land (because a catalog contains one
or more indexes). These aren't provided by ZODB itself, but there are
packages that provide them on top of ZODB. E.g. one of those is
repoze.catalog http://docs.repoze.org/catalog/overview.html.

I haven't used catalogs much personally, so I cannot give recommendations.

—
Reply to this email directly or view it on GitHub
#44 (comment).

Jim Fulton
http://jimfulton.info

[drmalex07]

@jimfulton , @mgedmin Thank you for your quick answer.

I didn't know about repoze.catalog (in fact i thought that repoze had only to do with authn/authz middleware).

I played a bit with it, and it seems that Catalog objects can be persisted into a ZODB database. It also seems that the catalog doesn't own the indexed objects (and so cannot retrieve by docid), so an external mapping (i.e a DocumentMap) must be maintained side-by-side and also be persisted into the database.

Am i correct? Is the following example valid?

class User(Persistent):
     pass # Implement __cmp__, carry a `name` attribute

# Populate our containers

user_catalog = Catalog()
user_catalog['name'] = CatalogFieldIndex('name')
user_map = DocumentMap()

u1 = User('Totos')
u2 = User('Foo')

user_catalog.index_doc(1, u1)
user_map.add(u1, 1)
user_catalog.index_doc(2, u2)
user_map.add(u2, 2)

# Commit to database

db = DB(FileStorage('users.zodb'))
conn = db.open()
root = conn.root()

root['user_map'] = user_map
root['user_catalog'] = user_catalog
transaction.commit()

conn.close

My main concern lies at this warning: http://docs.repoze.org/catalog/usage.html#restrictions.
But maybe this is resolved by subclassing objects directly from object ??

[djay]

You might want to look at souper. Its a single data structure that keeps
internal indexes so you can search it.
https://pypi.python.org/pypi/souper

On Fri, 12 Feb 2016 6:41 pm MichailAlexakis [email protected]
wrote:

@jimfulton https://github.com/jimfulton , @mgedmin
https://github.com/mgedmin Thank you for your quick answer.

I didn't know about repoze.catalog (in fact i thought that repoze had only
to do with authn/authz middleware).

I played a bit with it, and it seems that Catalog objects can be
persisted into a ZODB database. It also seems that the catalog doesn't own
the indexed objects (and so cannot retrieve by docid), so an external
mapping (i.e a DocumentMap) must be maintained side-by-side and also be
persisted into the database.

Am i correct? Is the following example valid?

class User(Persistent):
pass # Implement cmp, carry a name attribute

Populate our containers

user_catalog = Catalog()
user_catalog['name'] = CatalogFieldIndex('name')
user_map = DocumentMap()

u1 = User('Totos')
u2 = User('Foo')

user_catalog.index_doc(1, u1)
user_map.add(u1, 1)
user_catalog.index_doc(2, u2)
user_map.add(u2, 2)

Commit to database

db = DB(FileStorage('users.zodb'))
conn = db.open()
root = conn.root()

root['user_map'] = user_map
root['user_catalog'] = user_catalog
transaction.commit()

conn.close

My main concern lies at this warning:
http://docs.repoze.org/catalog/usage.html#restrictions.
But maybe this is resolved by subclassing objects directly from object ??

—
Reply to this email directly or view it on GitHub
#44 (comment).