Comments (6)
I can consistently reproduce the issue using:
The fp_code in the query returns a successful result. I ran ab on an c1.medium and the echoprint server is also running on a c1.meduium both are running Ubuntu 12.04
from echoprint-server.
Are you using the web.py thing straight up (using its webserver)? If so can
you try it under a real webserver via wsgi
On Wed, Aug 22, 2012 at 3:19 PM, bgood-clip [email protected]:
I can consistently reproduce the issue using:
ab -n 1000 http://amazon-ec2-host-here:8080/query?fp*
code=eJylllGOazkIRLdkDDawHMBm_0uYys9kFOl5Pp6iPpHcUS5QVThjDKbxgOgLe79g84V4osYL9wXSfGBMekH2C-u8YPXCs1-SeuCvarZ44dAL3Q8Q1QPwzwtTX2B74d3R21d3v0D-wnvOn78_463-x7d_huQD6PqFv-nIzwu1X3jP-X-88Xb7cyO9vVH2wrNm0vPAX7n93ZHKC0-NiPOFx8457HpCbLfKLddcUtvzRKqT581jSiFqHVMZn7tyb9u6rOK668jW4hSX6tl0eTftWIub8uABXTDvjOnUJvN09N4yg5N1TvUlQqtkXSfuqKMVca7lXKbrQqy1blipjuPlp1Aq7bVGVPq5uWlNzXmlvph-lvycfVESxhdtiZbeaTKmy-VOYkI5TTxm5XJlPGdGmlltW5Ycnul8iWVh6LrixhWTi-uzhPSS3B2bdpFG6djBRMP3tXa1KXyIufXUCnROQzK_kL4ZP2dfjErcLB23IMPwiUFDh9Jz9uw2PdACVc6tLhw7MOuoBaObzvC9l81EkTZ7bYznTrppzYM5a6wyl5RgvyLe4ZXX2eZKLo81qciNMLtxjmV_Qd57_Zx9sQ_HsSLiWDH8JmmPZVBQ3VAoX3YbtPCOaMCIPqRbPlXVMFwsyt7MJqrb2bPhTbE17_kod_0qT0gHCbeMla2488vX8KLQsZQsN4zVAWd_AZHafs6-kOKxq4nSBlSmceIaRB2TP2Mks3npxKRW45aI64t4hUXCIJDBEJHPqz8jbvUWm3QSVlHCxwRp8DmhNgV0w6LywHDXPq7Y_9H4uYQdE1VjfoEvbP05-2LvQAp49ZCBoJgixA5BV-e2YX0oGPlqRGnjjWoOvls0CJYfG7XIMBs54IkND5x5OwsN3UFz7c0b7RQ1w7LFB-Yx34Gc4uHVKavq5NC0BQv_Cz25--fsC4n8jK9uz31o1kT8eSFKkNZ028aTpHNsP4FvDjLMcO4B9TaMhFedxSdrnrWyuvksvXxkwWxCLggZhqsI-SFEJ2zRxRqbo6juhdC4ij2x1SIRVWwFdB0bWaDEtB0qeWxJ3l_khPo_Z
**
577oYlWGPwJyrjsHKGYl40YETD_Wn1mhZQTCySetdKZ9VBcn9nIk9XTCXroEdgRlxJWP2h-h8boctyJTIZ-XmlBgHX9frzjBsAtd517XDpfDDRMYxFL80IhC8AUuccTDv5dO-aAzr_px98Q8
*-QGrThe fp_code in the query returns a successful result. I ran ab on an
c1.medium and the echoprint server is also running on a c1.meduium both are
running Ubuntu 12.04—
Reply to this email directly or view it on GitHubhttps://github.com//issues/27#issuecomment-7945822.
from echoprint-server.
I'm running this using web.py.
Is there a sample apache config to run echoprint using the mod_wsgi for apache?
from echoprint-server.
I upgraded to Apache and the errors are still occurring.....
For reference my .htaccess looks like:
RewriteEngine on
RewriteBase /
RewriteCond %{REQUEST_URI} !^/icons
RewriteCond %{REQUEST_URI} !^/favicon.ico$
RewriteCond %{REQUEST_URI} !^(/.)+api.py/
RewriteRule ^(.)$ api.py/$1 [PT,QSA]
Apache site config looks like:
<VirtualHost *:8080>
ServerAdmin [email protected]
DocumentRoot /usr/local/echoprint/API
<Directory /usr/local/echoprint/API>
SetHandler wsgi-script
Options ExecCGI FollowSymLinks
</Directory>
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
CustomLog /var/log/apache2/access.log combined
ServerSignature Off
from echoprint-server.
I am also getting those errors on current version of the server, just running api.py
It seems the error happen randomly when there is more load and more queries are executed. For me it happens when a lot of /ingest queries come in.
The problem is critical, since soon after the first error, the server blocks and does not respond to anymore queries.
Is it possible that there is some issue with threads? Not sure how it is handled in the python server.
When TokyoTyrant is configured with more threads, the server runs a bit longer without blocking. Eventually with high load it always blocks, sometimes even without showing any TokyoTyrant errors, so maybe the real problem is in the python server.
Some example traces (soon before server blocks):
10.228.66.255:52295 - - [13/Sep/2012 13:19:27] "HTTP/1.1 POST /ingest" - 200 OK
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 239, in process
return self.handle()
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 230, in handle
return self._delegate(fn, self.fvars, args)
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 420, in _delegate
return handle_class(cls)
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 396, in handle_class
return tocall(*args)
File "/home/ubuntu/echoprint-server/API/api.py", line 55, in POST
fp.ingest(data, do_commit=True, local=False)
File "/home/ubuntu/echoprint-server/API/fp.py", line 586, in ingest
get_tyrant().multi_set(codes)
File "/home/ubuntu/echoprint-server/API/pytyrant.py", line 307, in multi_set
self.t.misc("putlist", opts, lst)
File "/home/ubuntu/echoprint-server/API/pytyrant.py", line 540, in misc
return list(self._misc(func, opts, args))
File "/home/ubuntu/echoprint-server/API/pytyrant.py", line 524, in _misc
socksuccess(self.sock)
File "/home/ubuntu/echoprint-server/API/pytyrant.py", line 172, in socksuccess
raise TyrantError(fail_code)
TyrantError: 56
10.228.66.255:52297 - - [13/Sep/2012 13:19:28] "HTTP/1.1 POST /ingest" - 500 Internal Server Error
10.228.66.255:52300 - - [13/Sep/2012 13:19:29] "HTTP/1.1 POST /ingest" - 200 OK
10.32.7.67:34282 - - [13/Sep/2012 15:12:18] "HTTP/1.1 POST /ingest" - 200 OK
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 239, in process
return self.handle()
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 230, in handle
return self._delegate(fn, self.fvars, args)
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 420, in _delegate
return handle_class(cls)
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 396, in handle_class
return tocall(*args)
File "/home/ubuntu/echoprint-server/API/api.py", line 55, in POST
fp.ingest(data, do_commit=True, local=False)
File "/home/ubuntu/echoprint-server/API/fp.py", line 586, in ingest
get_tyrant().multi_set(codes)
File "/home/ubuntu/echoprint-server/API/pytyrant.py", line 307, in multi_set
self.t.misc("putlist", opts, lst)
File "/home/ubuntu/echoprint-server/API/pytyrant.py", line 540, in misc
return list(self._misc(func, opts, args))
File "/home/ubuntu/echoprint-server/API/pytyrant.py", line 524, in _misc
socksuccess(self.sock)
File "/home/ubuntu/echoprint-server/API/pytyrant.py", line 172, in socksuccess
raise TyrantError(fail_code)
TyrantError: 51
10.32.7.67:34283 - - [13/Sep/2012 15:12:20] "HTTP/1.1 POST /ingest" - 500 Internal Server Error
from echoprint-server.
@iscra Turning off Keep-Alives made my situation a lot better. That said I still have problem about 2-3 per day at our current loads.
Now when this problem crops up, the Apache worker has eaten a ton of memory (700-1.5 GB). Once I restart apache a log statement come out for a request that happened hours early, much like a thread got hung.
from echoprint-server.
Related Issues (20)
- track_id with spaces can't be looked up in fp.metadata_for_track_id and fp.delete HOT 2
- Ingesting the same song twice won't return results at lookup HOT 1
- API Leak.. HOT 1
- problem with best_match_for_query HOT 7
- Problem Accuracy echoprint-server HOT 1
- How to delete ingested records from tokyo cabinet? HOT 1
- SOLR shows result = 1 but API shows no results... anyone see this before?
- Possible bug in HashQueryComponent
- Is it possible to embed server in IOS or Android app? HOT 1
- No matches on popular songs
- An exception while integrating Hashr.jar into a collection in Solr HOT 1
- Socket Error HOT 3
- Ingest live stream from remote server? HOT 1
- Only run the server in localhost
- Get Metadata from local echoprint-server HOT 1
- How to host echoprint server instance on heroku?
- what's reason aboat the socket.error?
- Where can I find the EchoPrint database ? http://echoprint.me/data/ was taken down. HOT 5
- Execute a backup
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from echoprint-server.