Comments (2)
The lock is a very good idea. You've also probably hit the nail on the head
with an issue we had at Sparrho for a bit related to indexing very large
chunks of content. It would take a long while. We never got the time to
investigate the issue.
Concerning the timer, how would you do that?
On Sun, Dec 20, 2015, 22:24 Di Wu [email protected] wrote:
The concept of the buffer is to optimize indexing performance by utilizing
bulk operations, but there's no guarantee that the buffer gets flushed in a
timely fashion.I think it should be necessary to use a timer with a configurable flush
interval to make sure that things don't stay in the buffer indefinitely.Another thing I noticed is that the buffer size check, bulk updating, and
buffer emptying code isn't thread safe, if we have Django running with
Gunicorn thread worker, then we have race condition.There should be a Lock used to prevent any race conditions
—
Reply to this email directly or view it on GitHub
#139.
from bungiesearch.
I think I'll submit another PR to try to deal with the flush issue and then there should be enough bugs fixed to qualify for a new release
from bungiesearch.
Related Issues (20)
- Running test HOT 11
- Coveralls is out of date HOT 1
- why is refresh=True the default? HOT 1
- TransportError when running search_index --create HOT 2
- BulkIndexError when `search_index --update` HOT 8
- Bungiesearch support for ES2.0+
- Documentation for version 2.0
- Upgrade option parser for django management commands HOT 1
- mapping fields HOT 1
- Add SourceClear to travis
- error [_id] is defined twice HOT 14
- How would I query multiple ModelIndexes simultaneously? HOT 8
- BungiesearchManager.__getattr__ tries to instantiate a Bungiesearch instance when not given settings HOT 1
- django-es HOT 1
- Django 1.10 support - BaseCommand has no attribute 'option_list' HOT 1
- Bungiesearch support for elasticsearch 5.x
- Circular import HOT 1
- Slow update times during unit testing HOT 2
- Index not defined in the settings (though it appears to be)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from bungiesearch.