Browse not working, about openbay

[isohuntto]

Hey,

Looks like there is an error with .htaccess
Can you try another hosting or configure apache?

Yours truly,

Isohunt.to team.

Join us on facebook https://www.facebook.com/isohuntto, twitter
https://twitter.com/Isohuntto, Isohunt.to forum http://forum.isohunt.to/
!

On Fri, Dec 19, 2014 at 11:18 PM, dynamitedj [email protected]
wrote:

I've got the site and everything up and running however, when i click to
search or even Browse torrents i just get a shit happens error message, any
ideas what could have gone wrong? i've re-installed several times but still
getting the same problem.
[image: capture]
https://cloud.githubusercontent.com/assets/10245756/5508294/1569e01c-87a3-11e4-8433-4643f270207b.JPG

—
Reply to this email directly or view it on GitHub
#2.

[dynamitedj]

I'm the Root account for the cpanel server so its no issue looking at the Apache, but i've checked htaccess and i cannot find anything relating to /browse, there is also no browse folder for /src/www/ where it is trying to find, should it not be trying to call the browse from your database server instead of mine?

i can try alternative hosting however the secondary server does not have PDO/PDO-MYSQL instaled as it is for Dev Testing of Web Scripts.

[GreySyntax]

Try enabling the debug flag in protected/config/debug.php and see if you get any more info.

[dynamitedj]

I have bolded the Red Lines from the debug page, if you want the Direct Link i can provide that @GreySyntax

---

PHP notice

Array to string conversion

/home/dynamite/public_html/proxy/src/protected/models/LTorrent.php(423)

411 ->option('ranker', 'SPH04')
412 ->search('@tags "' . $obj->halfEscapeMatch($tag) . '"')
413 ->where('torrent_status', LTorrent::TORRENT_STATUS_GOOD)
414 ->order('weight()', 'DESC')
415 ->order('id', 'DESC')
416 ->limit($count);
417
418 $torrentsIds = array_merge($torrentsIds, Yii::app()->sphinx->cache(600)->createCommand($obj->build())->queryColumn());
419 }
420 Yii::app()->cache->set($key, $torrentsIds);
421 }
422 } catch (Exception $e) {
423 Yii::log('getLastTorrentIdsByCategories failed. Exception: ' . $e->getMessage() . PHP_EOL . 'Trace: ' . $e->getTrace(), CLogger::LEVEL_ERROR);
424
425 if (YII_DEBUG) {
426 throw $e;
427 }
428 }
429
430 return $torrentsIds;
431 }
432
433 public function getBBParsed($str = null)
434 {
435 if ($str === null) {
Stack Trace
#0
– /home/dynamite/public_html/proxy/src/protected/controllers/MainController.php(257): LTorrent::getLastTorrentIdsByCategories()
252 public function actionCategories()
253 {
254 $this->pageTitle = 'Browse Torrents | ' . Yii::app()->name . ' Torrent Search Engine';
255
256 $torrentsByTags = array();
257 $torrentsIds = LTorrent::getLastTorrentIdsByCategories();
258
259 if (empty($torrentsIds)) {
260 Yii::log('Empty last torrents ids', CLogger::LEVEL_WARNING);
261 } else {
262 $torrentModel = LTorrent::model();
#1
– /home/dynamite/public_html/proxy/src/protected/controllers/MainController.php(102): MainController->actionCategories()
097 {
098 if ($q || $iht) {
099 return $this->actionSearch($q, $iht, $status);
100 }
101
102 return $this->actionCategories();
103 }
104
105 protected function checkSearchRequest() {
106 $uri = Yii::app()->getRequest()->getRequestUri();
107 if (preg_match('#ihq=([^&]+)#', $uri, $matches)) {
#2
unknown(0): MainController->actionBrowse(null, null, 0)
#3

• /home/dynamite/public_html/proxy/src/protected/vendor/yiisoft/yii/framework/web/actions/CAction.php(108): ReflectionMethod->invokeArgs(MainController, array(null, null, 0))
  #4
• /home/dynamite/public_html/proxy/src/protected/vendor/yiisoft/yii/framework/web/actions/CInlineAction.php(47): CAction->runWithParamsInternal(MainController, ReflectionMethod, array("r" => "main/browse"))
  #5
• /home/dynamite/public_html/proxy/src/protected/vendor/yiisoft/yii/framework/web/CController.php(308): CInlineAction->runWithParams(array("r" => "main/browse"))
  #6
• /home/dynamite/public_html/proxy/src/protected/vendor/yiisoft/yii/framework/web/CController.php(286): CController->runAction(CInlineAction)
  #7
• /home/dynamite/public_html/proxy/src/protected/vendor/yiisoft/yii/framework/web/CController.php(265): CController->runActionWithFilters(CInlineAction, array())
  #8
• /home/dynamite/public_html/proxy/src/protected/vendor/yiisoft/yii/framework/web/CWebApplication.php(282): CController->run("browse")
  #9
• /home/dynamite/public_html/proxy/src/protected/vendor/yiisoft/yii/framework/web/CWebApplication.php(141): CWebApplication->runController("main/browse")
  #10
• /home/dynamite/public_html/proxy/src/protected/vendor/yiisoft/yii/framework/base/CApplication.php(180): CWebApplication->processRequest()
  #11
  – /home/dynamite/public_html/proxy/src/www/index.php(13): CApplication->run()
  08 $yii = DIR . '/../protected/vendor/yiisoft/yii/framework/' . (YII_DEBUG ? 'yii.php' : 'yiilite.php');
  09 require_once (DIR . '/../protected/vendor/autoload.php');
  10 require_once ($yii);
  11
  12 $config =(is_file('installer.php')) ? 'installer.php' : DIR . '/../protected/config/config.php';
  13 Yii::createWebApplication($config)->run();
  2014-12-19 18:18:56 Apache/2.2.29 (Unix) mod_ssl/2.2.29 OpenSSL/1.0.1e-fips mod_bwlimited/1.4 Yii Framework/1.1.15

[GreySyntax]

Looks like your sphinx config is incorrect, check the searchd listen section is correct

[isohuntto]

@dynamitedj was the issue fixed by recent commits?

[dynamitedj]

i'm waiting to find out, i've logged out of my server for now but will update tomorrow and let you know as i've been up 40 hours and i'm flagging so not with it, don't want to risk completely messing it up.

[dynamitedj]

Nope, i can only assume that with my server not having Sphinx installed and trying to use your Sphinx from the installer so i didn't have to fight with Cpanel.

[dynamitedj]

i'll have to install Sphinx and try to get it working with CPanel

[FREEZX]

I have the same issue. My hosting does not have sphinx, and i'm unable to browse and search.

[GreySyntax]

After getting sphinx to behave, the browse and search pages have started working for me. I think this may actually be a non-issue

[FREEZX]

The issue could be with the provided sphinx config.

[NewEraCracker]

I would be great if they had some sort of mechanism that allowed falling back to MySQL.
It shouldn't be hard to add an index to 'name' column and use it to search when Sphinx is not available.

This would put the script working, even in hosts without Sphinx.

[dynamitedj]

I've installed Sphinx but its still failing for me, wondering if it is a problem with Config now, i've manually changed to my Sphinx instead of the default Sphinx and still getting the same thing, wondering now...

[YasarChavez]

http://openbay.com.nu/ GOOD!

[dynamitedj]

@tuneniitoo what Sphinx are you running? are you running your own or are you running the default config? i'm on a dedicated server with full root access i have Sphinx installed locally and it is still not working as it should be

[dynamitedj]

Ok, so i've got Sphinx somewhat working, but now i'm getting errors galore, please remember this is a CPANEL server, i've altered the conf slighly to listen on 3312 and 3307:mysql41 now when i click browse i instantly get the 404 error rather than a delay, i'm also getting warnings and a fatal error on sphinx boot.

---- QUOTE ---
Sphinx 2.2.6-id64-release (r4843)
Copyright (c) 2001-2014, Andrew Aksyonoff
Copyright (c) 2008-2014, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/root/local/etc/sphinx.conf'...
WARNING: key 'charset_type' was permanently removed from Sphinx configuration. Refer to documentation for details.
WARNING: key 'enable_star' was permanently removed from Sphinx configuration. Refer to documentation for details.
WARNING: key 'max_matches' was permanently removed from Sphinx configuration. Refer to documentation for details.
WARNING: preopen_indexes=1 has no effect with seamless_rotate=0
listening on all interfaces, port=3312
listening on all interfaces, port=3307
WARNING: index 'opb_common': key 'path' not found - NOT SERVING
precaching index 'opbtorrents'
WARNING: index 'opbtorrents': preload: failed to open /var/lib/sphinx/opbtorrents2.sph: No such file or directory; NOT SERVING
FATAL: no valid indexes to serve

---- END QUOTE ----

Any ideas?

[FREEZX]

http://stackoverflow.com/questions/20010439/sphinx-search-error-not-indexing

[dynamitedj]

Tried that but i'm using the Conf that was in the download, unless i've missed something else.... any other ideas?

[src386]

Hi, same for me, 404 everywhere except for the "Recent torrents".
I get the feeling that the "official" sphinx.conf is problematic, at least for debian. I finally managed to run it, here are the steps :

1. edit /etc/sphinxsearch/sphinx.conf and paste the example configuration
2. In the source opb_dbconnect { section set your mysql parameters
3. In the index opb_common { section add the following lines :

source = opbtorrents
path = /var/lib/sphinxsearch/data/opb_common

4. In the index opbtorrents : opb_common { section add the following lines :

source = opbtorrents
path = /var/lib/sphinxsearch/data/opbtorrents2

5. In the searchd { section, add/modify these :

listen = 9312
listen = 9306:mysql41
log = /var/log/sphinxsearch/searchd.log
query_log = /var/log/sphinxsearch/query.log
pid_file = /var/run/sphinxsearch/searchd.pid
binlog_path = /var/lib/sphinxsearch/data

6. Execute these commands :

indexer -c /etc/sphinxsearch/sphinx.conf opb_common
indexer -c /etc/sphinxsearch/sphinx.conf opbtorrents

7. Run sphinxsearch !

service sphinxsearch start

Sphinx seems to be working now, but I'm still getting 404 errors ....

[src386]

On openbay/src/protected/runtime/application.log I get that strange error :

SQLSTATE[28000] [1045] Access denied for user 'root'@'localhost' (using password: NO)

This is not normal. I don't use the root account for mysql. Here is my config.php :

'connectionString' => 'mysql:host=localhost;port=3306;dbname=dbopenbay',
        'username' => 'openbay',
        'password' => 'MYSECRETPASSWORD!',
        'schemaCachingDuration' => '86400',
        'charset' => 'utf8',
        'enableProfiling' => true

Is it a bug ?

[booth-f]

I followed all the instructions and such but I still cant get the search to work, just try to search on there and you'll see what I mean
http://xalogen.koding.io/

[src386]

Is there a way to get openbay working ? I'm stuck on this 404 error.
No documentation, no help, no log...

[booth-f]

@src386 Just turn debug mode on in protected/config and then try browsing again. That should give you an error to work from.

[src386]

Thanks.
This is what I get :
CDbConnection failed to open the DB connection: SQLSTATE[28000] [1045] Access denied for user 'root'@'localhost' (using password: NO)

Yeah because I DON'T WANT TO USE ROOT !
And my config.php says 'username' => 'openbay',
Looks like openbay doesn't care about config.php...

[hutchy1990]

This is the error I am getting 👎

DbCommand failed to execute the SQL statement: SQLSTATE[42000]: Syntax error or access violation: 1064 unknown local index 'opbtorrents' in search request. The SQL statement executed was: SELECT count(*) FROM opbtorrents ORDER BY weight() DESC, id DESC LIMIT 0, 20

[no1453]

Just wanted to make the comment that if the example-sphinx.conf file is using 3306, then you can basically assume it's f-ed up. Mysql uses 3306. That number should definitely be 9306, for a default configuration. (EDIT: I might be wrong, still researching.) I'm struggling through the same issues.

@src386- Apparently on debian mysql defaults to not allowing web access. You have to enable it, and then give permissions to the connecting user (root or whoever you choose to use.)

see https://rtcamp.com/tutorials/mysql/remote-access/

@GreySyntax- Would you mind posting the sphinx.conf you used that seems to be working?

[aaruni96]

Browse giving me this error.

CDbCommand failed to execute the SQL statement: SQLSTATE[3D000]: Invalid catalog name: 1046 No database selected. The SQL statement executed was: SELECT id FROM opbtorrents WHERE MATCH('@tags \"Anime\"') AND torrent_status = 1 ORDER BY weight() DESC, id DESC LIMIT 0, 5 OPTION ranker=SPH04

[no1453]

@aaruni96 -- have you run "indexer --all" and "service sphinxsearch restart"? It looks like it can't find those databases (opbtorrent), which are created when you run "indexer --all".

EDIT: you also might want to check that you are consistently using the same name for your database, entered in the wizard, The same name should be in /etc/sphinxsearch/sphinx.conf. And you did create a database with that name in mysql before running the wizard?

[no1453]

Thanks to the info about turning on debug messages, I now have an error to work from.

CDbConnection failed to open the DB connection: SQLSTATE[HY000] [2054] Server sent charset unknown to the client. Please, report to the developers

[src386]

@no1453 : Thanks, but everything is local, there is not remote access.

[aaruni96]

@no1453 : The recent section reads from my database. The browse section doesn't.

And I'm using MariaDB. SphinxSE is bundled into MariaDB and all I did was activate SphinxSE. I did nothing else, because I didn't know anything else was required.

My database name is "torrents". I have no idea why the sql query is for "opbtorrent". The code must be bugged. Or "opbtorrent" must be a virtual database or something.

[no1453]

Yes, opbtorrent is created from your database when it's indexed. Not sure how SphinxSE does that. I'm getting the same behavior with "browse" and "recent".

You might try running "indexer --all" (if it exists in your path). I got useful info from the output.

[no1453]

@src386- If you're using "localhost" for the server address, you still might need to grant access, it's a loopback address, but it still acts like a remote connection, I believe. I got the same error you posted until I enabled remote access and granted access to my user. If you want to keep everything local, I believe you can enable access for only 127.0.0.1 as the IP. The link I posted has steps for that, I think (for granting access only to a certain ip.)

[src386]

@ no1453 : it does not work because the root user is not allowed and it's normal. I created a user "openbay" with all rights on "dbopenbay". I don't understand why openbay wants the root user.

[hutchy1990]

Anyone have any idea about this error it's doing my head in

DbCommand failed to execute the SQL statement: SQLSTATE[42000]: Syntax error or access violation: 1064 unknown local index 'opbtorrents' in search request. The SQL statement executed was: SELECT count(*) FROM opbtorrents ORDER BY weight() DESC, id DESC LIMIT 0, 20

[no1453]

@hutchy1990 - If the opbtorrents index is missing, I think it means you need to run "indexer --all" (as root). That will create the opbtorrents index (from your specified database).

[no1453]

Surprisingly, after about 20 hours of work, I have openbay working, using my own sphinx and local database. So if anyone wants to ask questions about my config, fire away.
Two points; I had to compile the newest version of sphinxsearch, and I had to remove the line " charset = utf-8" from /src/protected/config/config.php. (That isn't the exact line, but it's pretty much what it looked like)

[src386]

@no1453 : I admit I'm jealous :) congratuilations.
I'm using sphinx 2.2.1 (freebsd ports) and I commented the utf8 in config.php but I'm stuck on a SQL syntax error.
Could you post your config.php please ? at least these two sections :
'db' => array(
and
'sphinx' => array(

[no1453]

Here's the whole thing.

---

<?php

return array(
    'name' => 'NeoPirate',
    'theme' => 'oldpiratebay',
        'preload' => array('log'),
        'import' => array(
        'application.helpers.*',
    ),
    'basePath' => __DIR__ . DIRECTORY_SEPARATOR . '..',
    'language' => 'en',
    'sourceLanguage' => 'en',
    'import' => array(
        'application.components.*',
        'application.models.*',
      'ext.ESphinxQL.*'
    ),
    'components' => array(
        'db' => array(
            'connectionString' => 'mysql:host=MY_IP_FILL_IN_YOURS;port=17666;dbname=tordb',
            'username' => 'root',
            'password' => 'xxxxxxxx',
            'schemaCachingDuration' => '86400',
            'charset' => 'utf8',
            'enableProfiling' => true
        ),
        'cache' => array(
            'class' => 'system.caching.CFileCache',
        ),
        'sphinx' => array(
            'class' => 'system.db.CDbConnection',
            'connectionString' => 'mysql:host=MY_IP_FILL_IN_YOURS;port=9306',
            'queryCacheID' => 'cache'
        ),
        'request' => array(
            'class' => 'application.components.AHttpRequest'
        ),
        'clientScript' => array(
            'scriptMap' => array(
                'jquery.min.js' => '//ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js',
            ),
            'packages' => array(
                'base' => array(
                    'baseUrl' => '/css/',
                    'css' => array(
                        'opb.css',
                   ),
                ),
            ),
        ),
        'urlManager' => array(
            'urlFormat' => 'path',
            'showScriptName' => false,
            'caseSensitive' => false,
            'rules' => require(__DIR__ . '/rules.php')
        ),
        'errorHandler' => array(
            'errorAction' => 'main/error'
        ),
                'log' => array(
            'class' => 'CLogRouter',
            'routes' => array(
                array(
                    'class' => 'CFileLogRoute',
                    'levels' => 'error, warning'
                )
            )
        ),
                'session' => array(
            'sessionName' => 'MYSESSIONID'
        ),
        'format' => array(
            'class' => 'application.components.Formatter'
        ),
    ),
    'params' => array(
        'adminEmail' => '[email protected]',
        'sphinx' => array(
            'indexes' => array(
                'torrents' => 'opbtorrents',
            ),
        ),
    ),
);

[no1453]

A note about the above, I have mysql using port 17666, and sphinxsearch on 9306

Edit: somehow the charset = utf8 line snuck back in there, but it still works.

[src386]

I changed "localhost" to "127.0.0.1" in my config.php and remove some lines previously added by me.
No error this time but 0 resultats, I think I broke my sphinx indexes. I try to repair this.
If I can make it work, I'll write a wiki page for FreeBSD and Debian installation...
EDIT : It works !

[hutchy1990]

@no1453 I have ran indexer --all and it ran smooth opbtorrent was created maybe it's to do with I have sphinx installed on root and my website is a normal account created in cpanel

[compsjp]

i am also just getting 404 on any links. trying use everything default., web server is small. wont be able to do a own spinx. why is default broken?

[src386]

The documentation has flaws :

• Example sphinx.conf is invalid (missing 'paths' and bad listening port)
• Forget to mention that you have to grab a real dump of the tpb database (1,2Go) BEFORE running indexer.conf found here

It took days for me but now it works with FreeBSD.
I'm currently writing a wiki page. I'll see if I can make something for a debian/ubuntu host.

[compsjp]

yea..... i got it working!!!!!!

[no1453]

I used the csv from isohnt.to/open bay, had to break it into chunks to import because more than one hash was 0 (when they have to be unique, or else import fails.) The third chunk of 1 million lines seems to be the one it doesn't like, chunks 1,2,&4 imported fine, and now i'm trying the last 3.8 million line chunk. I'll probably have to edit the 3rd chunk in Excel to find and remove any lines with hash 0 (Excel can only open a little more than a million entries.) A bit of work, but i've learned a lot in doing it. So if your import is failing like mine did, at least here's some idea where to look for the problem.

[compsjp]

i am on ubuntu 12.04
and honestly it was all cause of i had to custom up .htaccess, and apache to allow

Hello, I tried to make it work as src386 said in his instruction, but when I'm executing
indexer -c /etc/sphinxsearch/sphinx.conf opb_common

I'm getting errors:

Sphinx 2.0.4-id64-release (r3135)
Copyright (c) 2001-2012, Andrew Aksyonoff
Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/sphinxsearch/sphinx.conf'...
WARNING: key 'path' is not multi-value; value in /etc/sphinxsearch/sphinx.conf line 57 will be ignored.
WARNING: key 'log' is not multi-value; value in /etc/sphinxsearch/sphinx.conf line 77 will be ignored.
WARNING: key 'pid_file' is not multi-value; value in /etc/sphinxsearch/sphinx.conf line 79 will be ignored.
WARNING: key 'binlog_path' is not multi-value; value in /etc/sphinxsearch/sphinx.conf line 80 will be ignored.
indexing index 'opb_common'...
FATAL: out of memory (unable to allocate 436994064 bytes)

Why is that?
My system is Ubuntu 14.04 x64, I installed Sphinx by official guide (http://sphinxsearch.com/docs/latest/installing-debian.html).
I'm having only 512mb memory. Am I really need more to make it work? Can I make this index on my PC and load it on server? Will it need that much memory every little update?

[src386]

I'm not familiar with Sphinx but you could try to allow more RAM in :

indexer {
mem_limit = 32M

I've tried to use 1Gb RAM server and it was still not enough memory. When I'm restarted spinx after indexing it just freezes at 78%:

precaching index 'opb_common'
read 734.2 of 940.8 MB, 78.0% done

Fixed by adding ondisk_attrs = 1 for index opb_common and index opbtorrents sections to cache on disk.

And IT FINALLY WORKS! No more 404 errors. Thank you very much for your instructions, especially src386.
btw, my mirror is at http://pbay.me/ , check out if you have any interest. I'll keep working on it and updating.

[src386]

@ JohnCock : I realized that Sphinx uses 2,5GB of memory, that's why your 512MB was not enough.
I am on a VM with 8GB so it's OK, but my "production" server has 1GB and I have the same issue : #110
I'll try your solution, thanks.

[hutchy1990]

I am getting this error when running indexer --all any ideas?

FATAL: failed to lock /var/lib/sphinx/opb_common.spl: Resource temporarily unavailable, will not index. Try --rotate option.

[hutchy1990]

I reboot the VPs and it fixed that error indexer --all ran i got this error

WARNING: min_infix_len is not supported yet with dict=keywords; using dict=crc

now I get the error when i search for torrents

CDbConnection failed to open the DB connection: SQLSTATE[HY000] [2002] Connection refused

[TPBT-OFFICIAL]

@hutchy1990 "FATAL: failed to lock /var/lib/sphinx/opb_common.spl: Resource temporarily unavailable, will not index. Try --rotate option."

That error is caused because sphinx is online so you can just do "service searchd stop" do "indexer --all" and then do "service searchd start"

That other error "WARNING: min_infix_len is not supported yet with dict=keywords; using dict=crc" is not a big issue its just a warning.

"CDbConnection failed to open the DB connection: SQLSTATE[HY000] [2002] Connection refused"
Im not sure what it could be could you please show your config file?

[hutchy1990]

vi etc/sphinx/sphinx.conf

source opb_dbconnect {
type = mysql
sql_host = 127.0.0.1
sql_user = raider_data
sql_pass = secure_pass
sql_db = raider_data
sql_port = 3306
sql_query_pre = SET NAMES utf8
}
index opb_common {
source = opbtorrents
path = /var/lib/sphinx/opb_common
docinfo = extern
charset_type = utf-8
charset_table = 0..9, A..Z->a..z, a..z, U+410..U+42F->U+430..U+44F, U+430..U+44F
#min_word_len = 2
enable_star = 1
#min_infix_len = 3
#min_prefix_len = 3
expand_keywords = 1
index_exact_words = 1
morphology = stem_enru
min_stemming_len = 4
dict = keywords
ngram_len = 1
ngram_chars = U+4E00..U+9FBB, U+3400..U+4DB5, U+20000..U+2A6D6, U+FA0E, U+FA0F, U+FA11, U+FA13, U+FA14, U+FA1F, U+FA21, U+FA23, U+FA24, U+FA27, U+FA28, U+FA29, U+3105..U+312C, U+31A0..U+31B7, U+3041, U+3043, U+3045, U+3047, U+3049, U+304B, U+304D, U+304F, U+3051, U+3053, U+3055, U+3057, U+3059, U+305B, U+305D, U+305F, U+3061, U+3063, U+3066, U+3068, U+306A..U+306F, U+3072, U+3075, U+3078, U+307B, U+307E..U+3083, U+3085, U+3087, U+3089..U+308E, U+3090..U+3093, U+30A1, U+30A3, U+30A5, U+30A7, U+30A9, U+30AD, U+30AF, U+30B3, U+30B5, U+30BB, U+30BD, U+30BF, U+30C1, U+30C3, U+30C4, U+30C6, U+30CA, U+30CB, U+30CD, U+30CE, U+30DE, U+30DF, U+30E1, U+30E2, U+30E3, U+30E5, U+30E7, U+30EE, U+30F0..U+30F3, U+30F5, U+30F6, U+31F0, U+31F1, U+31F2, U+31F3, U+31F4, U+31F5, U+31F6, U+31F7, U+31F8, U+31F9, U+31FA, U+31FB, U+31FC, U+31FD, U+31FE, U+31FF, U+AC00..U+D7A3, U+1100..U+1159, U+1161..U+11A2, U+11A8..U+11F9, U+A000..U+A48C, U+A492..U+A4C6
preopen = 1
}

source opbtorrents : opb_dbconnect {
sql_query_range = SELECT MIN(id), MAX(id) FROM torrents
sql_range_step = 100000
sql_query = SELECT tr.id AS id, tr.hash, tr.name, tr.tags, tr.category_id, UNIX_TIMESTAMP(tr.created_at) AS created_at, tr.size, tr.downloads_count, tr.seeders, tr.leechers, tr.torrent_status,
tr.visible_status,
tr.files_count AS files_count
FROM torrents AS tr
WHERE tr.id BETWEEN $start AND $end
GROUP BY tr.id
sql_field_string = name
sql_field_string = tags
sql_attr_uint = category_id
sql_attr_timestamp = created_at
sql_attr_bigint = size
sql_attr_uint = downloads_count
sql_attr_uint = seeders
sql_attr_uint = leechers
sql_attr_uint = torrent_status
sql_attr_uint = visible_status
sql_attr_uint = files_count
sql_attr_string = hash
}
index opbtorrents : opb_common {
source = opbtorrents
path = /var/lib/sphinx/opbtorrents2
min_infix_len = 3
dict = keywords
}
indexer {
mem_limit = 512M
}
searchd {
listen = 9312
listen = 9306:mysql41
log = /var/log/sphinx/searchd.log
read_timeout = 5
max_children = 0
pid_file = /var/run/sphinx/searchd.pid
max_matches = 100000
seamless_rotate = 0
preopen_indexes = 1
unlink_old = 1
workers = threads
binlog_path = /var/lib/sphinx
}

[hutchy1990]

new problem is this

CDbCommand failed to execute the SQL statement: SQLSTATE[42000]: Syntax error or access violation: 1064 index opbtorrents: invalid schema: Count() or @count is queried, but not available in the schema. The SQL statement executed was: SELECT count() FROM opbtorrents ORDER BY weight() DESC, id DESC LIMIT 0, 20 I have no luck with this sphinx search

[TheChiefCoC]

I got the it all working now :) the search is working and i can also see recent torrents , whats not working is the Browse page doesnt show the recent torrents, but all categories are working, Thanks a lot guys, I will be posting a full guide in detail of what exactly i did soon 👍 but i need a place where i can post as i am not familiar with Markdown :/

[TheChiefCoC]

Just a note that i noticed : All the torrents from the database found here : isohuntto/openbay-db-dump#5 have a problem in their time created. So all have 0 mins as there time created. This causes a problem in the Browse option as it shows only the Recent torrents that have been submitted in the last day.

[TheChiefCoC]

@hutchy1990 Try running this command as root :

indexer --all --rotate

[hutchy1990]

I tried it this is my error

CDbCommand failed to execute the SQL statement: SQLSTATE[42000]: Syntax error or access violation: 1064 index opbtorrents: invalid schema: Count() or @count is queried, but not available in the schema. The SQL statement executed was: SELECT count() FROM opbtorrents WHERE MATCH(' @tags "=Movies"') ORDER BY weight() DESC, id DESC LIMIT 0, 20

[src386]

@hutchy1990 : Please post your config.php (don't forget to hide the password) because I had a similar issue.

[TPBT-OFFICIAL]

@hutchy1990 i had the same error as you, its because of the sphinx version please save your config file uninstall sphinx and install 2.1.9 version, if you can't install that version (like me it gives me dependency errors) then install the BETA version of sphinx and try again :)

[hutchy1990]

source opb_dbconnect {
type = mysql
sql_host = 127.0.0.1
sql_user = raider_data
sql_pass = secure_pass
sql_db = raider_data
sql_port = 3306
sql_query_pre = SET NAMES utf8
}
index opb_common {
source = opbtorrents
path = /var/lib/sphinx/opb_common
docinfo = extern
charset_type = utf-8
charset_table = 0..9, A..Z->a..z, a..z, U+410..U+42F->U+430..U+44F, U+430..U+44F
#min_word_len = 2
enable_star = 1
#min_infix_len = 3
#min_prefix_len = 3
expand_keywords = 1
index_exact_words = 1
morphology = stem_enru
min_stemming_len = 4
dict = keywords
ngram_len = 1
ngram_chars = U+4E00..U+9FBB, U+3400..U+4DB5, U+20000..U+2A6D6, U+FA0E, U+FA0F, U+FA11, U+FA13, U+FA14, U+FA1F, U+FA21, U+FA23, U+FA24, U+FA27, U+FA28, U+FA29, U+3105..U+312C, U+31A0..U+31B7, U+3041, U+3043, U+3045, U+3047, U+3049, U+304B, U+304D, U+304F, U+3051, U+3053, U+3055, U+3057, U+3059, U+305B, U+305D, U+305F, U+3061, U+3063, U+3066, U+3068, U+306A..U+306F, U+3072, U+3075, U+3078, U+307B, U+307E..U+3083, U+3085, U+3087, U+3089..U+308E, U+3090..U+3093, U+30A1, U+30A3, U+30A5, U+30A7, U+30A9, U+30AD, U+30AF, U+30B3, U+30B5, U+30BB, U+30BD, U+30BF, U+30C1, U+30C3, U+30C4, U+30C6, U+30CA, U+30CB, U+30CD, U+30CE, U+30DE, U+30DF, U+30E1, U+30E2, U+30E3, U+30E5, U+30E7, U+30EE, U+30F0..U+30F3, U+30F5, U+30F6, U+31F0, U+31F1, U+31F2, U+31F3, U+31F4, U+31F5, U+31F6, U+31F7, U+31F8, U+31F9, U+31FA, U+31FB, U+31FC, U+31FD, U+31FE, U+31FF, U+AC00..U+D7A3, U+1100..U+1159, U+1161..U+11A2, U+11A8..U+11F9, U+A000..U+A48C, U+A492..U+A4C6
preopen = 1
}

source opbtorrents : opb_dbconnect {
sql_query_range = SELECT MIN(id), MAX(id) FROM torrents
sql_range_step = 100000
sql_query = SELECT tr.id AS id, tr.hash, tr.name, tr.tags, tr.category_id, UNIX_TIMESTAMP(tr.created_at) AS created_at, tr.size, tr.downloads_count, tr.seeders, tr.leechers, tr.torrent_status,
tr.visible_status,
tr.files_count AS files_count
FROM torrents AS tr
WHERE tr.id BETWEEN $start AND $end
GROUP BY tr.id
sql_field_string = name
sql_field_string = tags
sql_attr_uint = category_id
sql_attr_timestamp = created_at
sql_attr_bigint = size
sql_attr_uint = downloads_count
sql_attr_uint = seeders
sql_attr_uint = leechers
sql_attr_uint = torrent_status
sql_attr_uint = visible_status
sql_attr_uint = files_count
sql_attr_string = hash
}
index opbtorrents : opb_common {
source = opbtorrents
path = /var/lib/sphinx/opbtorrents2
min_infix_len = 3
dict = keywords
}
indexer {
mem_limit = 512M
}
searchd {
listen = 9312
listen = 9306:mysql41
log = /var/log/sphinx/searchd.log
read_timeout = 5
max_children = 0
pid_file = /var/run/sphinx/searchd.pid
max_matches = 100000
seamless_rotate = 0
preopen_indexes = 1
unlink_old = 1
workers = threads
binlog_path = /var/lib/sphinx
}

Here is my config :)

[TheChiefCoC]

My sphinx.conf file of SphinxSearch 2.1.9:
https://ghostbin.com/paste/tz7vx
Delete the comment in Line No. 6 . Its only for your info. I made several DBs but opbtorrents worked for me. So I recommend making a db by that name.

[TheChiefCoC]

Also guys i can't make a full guide yet as i am pretty busy. YOu can shoot me a PM or ask a specific question and i will try my best to help whenever i can 👍

[LeonardoCiaccio]

My error, any idea ? :

"CDbConnection failed to open the DB connection: SQLSTATE[HY000] [2002] No connection could be made because the target machine actively refused it."

[src386]

Check all these points :

• Your database is not empty (your have to import the isohunt dump)
• Sphinx works and the indexes are correctly built (it takes at least 15min)
• Your mysql user can connect throught 'user'@'localhost' (you can try with root)

So far it works for me with FreeBSD. Doesn't works on Debian for now (I think the sphinx package is too old)

[bernd-stromberg]

@src386 : TOP answer of the day !

👏

[...]Doesn't works on Debian for now (I think the sphinx package is too old)

Thx a lot, now it works for me. Step by step...

• my system

  Linux apoc 3.2.0-4-686-pae #1 SMP Debian 3.2.60-1+deb7u3 i686 GNU/Linux

• be sure of right settings in apache like

        <Directory /[pathto]/openbay-master/>
                RewriteEngine on
                Options Indexes FollowSymLinks MultiViews
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>

• [optional] first I made a change in

/[pathto]/openbay-master/src/www/themes/oldpiratebay/views/main/index.php

change line 9 "main/browse" to "main/recent"

• repositories
  For example sphinxsearch in this case is NOT working correctly ! More later...

• getting openbay

  https://github.com/isohuntto/openbay/archive/master.zip

• unpack in webspace

  unzip master.zip

• structure like this

|  |  |  `--openbay-master
|  |  |  |  `--conf
|  |  |  |  `--src
|  |  |  |  |  `--protected
|  |  |  |  |  |  `--components
|  |  |  |  |  |  `--config
|  |  |  |  |  |  `--controllers
|  |  |  |  |  |  `--data
|  |  |  |  |  |  `--extensions
|  |  |  |  |  |  `--installer
|  |  |  |  |  |  `--models
|  |  |  |  |  |  `--runtime
|  |  |  |  |  |  `--vendor
|  |  |  |  |  |  `--widgets
|  |  |  |  |  `--www
|  |  |  |  |  |  `--assets
|  |  |  |  |  |  `--css
|  |  |  |  |  |  `--img
|  |  |  |  |  |  `--js
|  |  |  |  |  |  `--themes

• log in to the site
  
• fill in
  • General
  • Database
    [your own]
  • Sphinx
    [your own]
  • Cache
  • Log
• after that "[pathto]/openbay-master/src/protected/config/config.php" starting line 18 looks like

    'components' => array(
        'db' => array(
            'connectionString' => 'mysql:host=127.0.0.1;port=3306;dbname=thepiratebay',
            'username' => 'thepiratebay',
            'password' => 'PASSWORD',
            'schemaCachingDuration' => '86400',
            'charset' => 'utf8',
            'enableProfiling' => true
        ),
        'cache' => array(
            'class' => 'system.caching.CFileCache',
        ),
        'sphinx' => array(
            'class' => 'system.db.CDbConnection',
            'connectionString' => 'mysql:host=127.0.0.1;port=9306',
            'queryCacheID' => 'cache'

• get the latest database (first went to the path you want to act, like "cd /opt/temporary/openbay/sandbox")

  mysqldump -u openpiratebay -p046b3SVJN9TlG5X58z4lMTNxI opb -h 46.28.69.238 --port=443 --single-transaction > opb.sql

• get actual spinxsearch (see above "repositories")

  wget http://sphinxsearch.com/files/sphinxsearch_2.2.6-release-1~wheezy_i386.deb

• install sphinxsearch (have a look to the dependencies. You can reach them via apt-get for example)

  apt-get install libodbc1
  apt-get install unixodbc
  dpkg -i sphinxsearch_2.2.6-release-1~wheezy_i386.deb

• configuration sphinxsearch

  • /etc/sphinxsearch/sphinx.conf

source opb_dbconnect {
    type = mysql
    sql_host = localhost
    sql_user = thepiratebay
    sql_pass = SECRET
    sql_db = thepiratebay 
    sql_port = 3306
    sql_query_pre = SET NAMES utf8
}
index opb_common {
    docinfo = extern
    #charset_type = utf-8
    charset_table = 0..9, A..Z->a..z, a..z, U+410..U+42F->U+430..U+44F, U+430..U+44F
    #min_word_len = 2
    #enable_star = 1
    #min_infix_len = 3
    #min_prefix_len = 3
    expand_keywords = 1
    index_exact_words = 1
    morphology = stem_enru
    min_stemming_len = 4
    dict = keywords
    ngram_len = 1
    ngram_chars = U+4E00..U+9FBB, U+3400..U+4DB5, U+20000..U+2A6D6, U+FA0E, U+FA0F, U+FA11, U+FA13, U+FA14, U+FA1F, U+FA21, U+FA23, U+FA24, U+FA27, U+FA28, U+FA29, U+3105..U+312C, U+31A0..U+31B7, U+3041, U+3043, U+3045, U+3047, U+3049, U+304B, U+304D, U+304F, U+3051, U+3053, U+3055, U+3057, U+3059, U+305B, U+305D, U+305F, U+3061, U+3063, U+3066, U+3068, U+306A..U+306F, U+3072, U+3075, U+3078, U+307B, U+307E..U+3083, U+3085, U+3087, U+3089..U+308E, U+3090..U+3093, U+30A1, U+30A3, U+30A5, U+30A7, U+30A9, U+30AD, U+30AF, U+30B3, U+30B5, U+30BB, U+30BD, U+30BF, U+30C1, U+30C3, U+30C4, U+30C6, U+30CA, U+30CB, U+30CD, U+30CE, U+30DE, U+30DF, U+30E1, U+30E2, U+30E3, U+30E5, U+30E7, U+30EE, U+30F0..U+30F3, U+30F5, U+30F6, U+31F0, U+31F1, U+31F2, U+31F3, U+31F4, U+31F5, U+31F6, U+31F7, U+31F8, U+31F9, U+31FA, U+31FB, U+31FC, U+31FD, U+31FE, U+31FF, U+AC00..U+D7A3, U+1100..U+1159, U+1161..U+11A2, U+11A8..U+11F9, U+A000..U+A48C, U+A492..U+A4C6
    preopen = 1
    source = opbtorrents
    path = /var/lib/sphinxsearch/data/opb_common
    ondisk_attrs = 1
}
source opbtorrents : opb_dbconnect {
    sql_query_range = SELECT MIN(id), MAX(id) FROM torrents
    sql_range_step = 100000
    sql_query = SELECT tr.id AS id, tr.hash, tr.name, tr.tags, tr.category_id, UNIX_TIMESTAMP(tr.created_at) AS created_at, tr.size, tr.downloads_count, tr.seeders, tr.leechers, tr.torrent_status, \
        tr.visible_status, \
        tr.files_count AS files_count \
        FROM `torrents` AS tr \
        WHERE tr.id BETWEEN $start AND $end \
        GROUP BY tr.id
        sql_field_string        = name
        sql_field_string        = tags
        sql_attr_uint           = category_id
        sql_attr_timestamp      = created_at
        sql_attr_bigint         = size
        sql_attr_uint           = downloads_count
        sql_attr_uint           = seeders
        sql_attr_uint           = leechers
        sql_attr_uint           = torrent_status
        sql_attr_uint           = visible_status
        sql_attr_uint           = files_count
        sql_attr_string         = hash
}
index opbtorrents : opb_common {
    source = opbtorrents
    path = /var/lib/sphinxsearch/data/opbtorrents2
    min_infix_len = 3
    dict = keywords
    ondisk_attrs = 1
}
indexer {
    mem_limit = 512M
}
searchd {
    listen = 9312
    listen = 9306:mysql41
    log = /var/log/sphinxsearch/searchd.log
    read_timeout = 5
    max_children = 0
    pid_file = /var/run/sphinxsearch/searchd.pid
    #max_matches = 100000
    #seamless_rotate = 0
    preopen_indexes = 1
    unlink_old = 1
    workers = threads
    binlog_path = /var/lib/sphinxsearch/data
    query_log = /var/log/sphinxsearch/query.log
}

• indexing new database

  • [optional] preparing the *.sql for better performance (opb.sql = 3,7GB)

  • insert at the begining

    sed -i '1i SET FOREIGN_KEY_CHECKS=0; SET AUTOCOMMIT=0; SET unique_checks=0; START TRANSACTION;' < opb.sql > opb_2.sql

  • insert to the end

    sed -i '$a SET unique_checks=1; SET FOREIGN_KEY_CHECKS=1; COMMIT; SET AUTOCOMMIT = 1;' < opb_2.sql > opb_3.sql

  • [optional] lets control the status of importing (requires "pv") and start building the database

    pv opb_3.sql | mysql -u thepiratebay --password=PASSWORD thepiratebay

  • [optional] if you access from remote : its a good idea to run the sql-import in the background

  • create a script (sqlimport.pl) with the line above and run

screen -mS import ./sqlimport.pl

• now lets start doing the first index

/etc/init.d/sphinxsearch stop
indexer -c /etc/sphinxsearch/sphinx.conf opb_common

Sphinx 2.2.6-release
Copyright (c) 2001-2012, Andrew Aksyonoff
Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/sphinxsearch/sphinx.conf'...
indexing index 'opb_common'...
collected 7800319 docs, 531.8 MB
sorted 184.8 Mhits, 100.0% done
total 7800319 docs, 531797231 bytes
total 662.013 sec, 803302 bytes/sec, 11782.71 docs/sec
total 45 reads, 34.140 sec, 27410.2 kb/call avg, 758.6 msec/call avg
total 5120 writes, 34.608 sec, 660.2 kb/call avg, 6.7 msec/call avg

• second one

indexer -c /etc/sphinxsearch/sphinx.conf opbtorrents

Sphinx 2.2.6-release
Copyright (c) 2001-2012, Andrew Aksyonoff
Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/sphinxsearch/sphinx.conf'...
indexing index 'opbtorrents'...
WARNING: min_infix_len is not supported yet with dict=keywords; using dict=crc
collected 7800319 docs, 531.8 MB
sorted 1404.2 Mhits, 100.0% done
total 7800319 docs, 531797231 bytes
total 2640.750 sec, 201381 bytes/sec, 2953.82 docs/sec
total 449 reads, 173.156 sec, 14293.2 kb/call avg, 385.6 msec/call avg
total 15786 writes, 113.702 sec, 898.2 kb/call avg, 7.2 msec/call avg

• finishing

/etc/init.d/sphinxsearch start

Starting sphinxsearch: Sphinx 2.0.4-release (r3135)
Copyright (c) 2001-2012, Andrew Aksyonoff
Copyright (c) 2008-2012, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/sphinxsearch/sphinx.conf'...
listening on all interfaces, port=9312
listening on all interfaces, port=9306
precaching index 'opb_common'
precaching index 'opbtorrents'
precached 2 indexes in 119.421 sec
sphinxsearch.

It looks like...

📈 ...THIS ONE !

...have fun !

[hayzamjs]

@bernd-stromberg, I'm gonna try your method, hope it works!

[TheChiefCoC]

@bernd-stromberg when i run

sed -i '1 i\SET FOREIGN_KEY_CHECKS=0; SET AUTOCOMMIT=0; SET unique_checks=0; START TRANSACTION;' < opb.sql > opb_2.sql

i get

sed: no input files

[hayzamjs]

Wait, @TheChiefCoC, you already have Openbay running right?

[TheChiefCoC]

@Fazelesswhite i do but i don't have Browse working, and also i am trying to install the new 3.5 GB sql as the previous one was of 400 MB 👅

[hayzamjs]

^ Oh...

[hayzamjs]

Can you join an IRC ? I have a couple of questions.. Join this channel : #hacking-lab at http://webchat.freenode.net

[TheChiefCoC]

i was thinking what better perfomance meant

EDIT: @bernd-stromberg there is slight error in your command

sed -i '1 i\SET FOREIGN_KEY_CHECKS=0; SET AUTOCOMMIT=0; SET unique_checks=0; START TRANSACTION;' < opb.sql > opb_2.sql

i had to remove the "<" which made the command :

sed -i '1 i\SET FOREIGN_KEY_CHECKS=0; SET AUTOCOMMIT=0; SET unique_checks=0; START TRANSACTION;' opb.sql > opb_2.sql

this made it work for me 😄

EDIT 2: now the files have no size , 0k is there size 😞

[bernd-stromberg]

Ups, the above works well in my case. Did you load the file opb.sql to the right location where you start the other commands ?

get the latest database (first went to the path you want to act, like "cd /opt/temporary/openbay/sandbox")

But the sed-commands are only optional for better performance. Try to run...

mysql -u thepiratebay --password=PASSWORD thepiratebay opb.sql

...directly and post the result.

• mysql [starts the SQL environment]
• -u thepiratebay [the user in mySQL]
• --password=PASSWORD [the PASSWORD for this user]
• thepiratebay [the database name]
• opb.sql [the file in the path of running THIS COMMAND]

Hope it helps...

[hayzamjs]

Hey buddy I get a "(500) Internal Server Error! The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@localhost and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
Apache/2.2.22 (Debian) Server at IP Port 80" and in the log it says :

[Mon Jan 05 19:20:44 2015] [alert] [client 10.254.253.116] /var/www/openbay/.htaccess: Invalid command 'RewriteEngine', perhaps misspelled or defined by a module not included in the server configuration. How can I fix this?

EDIT : I commented out the RewriteEngine line and it worked and is there any problems if it runs like this?

[bernd-stromberg]

@Fazelesswhite :
Do you have any questions ?

@TheChiefCoC :
There is NO slight error in my command...! Sorry 😎

To all :

FIRST get the *.zip from this side, study "README.md" and try to run your webserver ! Sometimes a search engine is very, very helpfull...! Include the four (!) datasets inside. DO NOT try inserting 18.000.000 sets until then.

• You can reach your entrance ?
  Like THIS 🏠
• Are you able to call "Recent Torrents"
  Like THIS ⌚
• If the "Search" for the keyword "Photo" doesn't work come here again...
  Doesn't work like THIS 🔍

cu

[hayzamjs]

I commented out the rewrite engine line in .htaccess is there going to be any problem?

[TheChiefCoC]

@Fazelesswhite run this command :

sudo a2enmod rewrite && sudo service apache2 restart

[hayzamjs]

I'm adding the source to the database now, after this what should I do ? run 'indexer --all?'

[bernd-stromberg]

First

/etc/init.d/sphinxsearch stop

if it's running.

👍

[hayzamjs]

Okay..

[hayzamjs]

I have one more question, what will I give the sphinx port and IP ?

[bernd-stromberg]

See above

• configuration sphinxsearch
• /etc/sphinxsearch/sphinx.conf

BUT FIRST : Whats the output of

dpkg -l | grep "sphinxsearch"

?!

[hayzamjs]

root@FaZe:/var/www/openbay/src/www/themes/oldpiratebay/views/layouts# dpkg -l | grep "sphinxsearch"
ii sphinxsearch 2.2.6-release-1~wheezy amd64 Fast standalone full-text SQL search engine
root@FaZe:/var/www/openbay/src/www/themes/oldpiratebay/views/layouts#

[bernd-stromberg]

Cooool ! 2.2.6 is very OK.

So on...

...you can copy/paste the above /etc/sphinxsearch/sphinx.conf. Fill in only your DB-host/user/pass/database...

• sql_host = localhost
• sql_user = thepiratebay
• sql_pass = SECRET
• sql_db = thepiratebay

[hayzamjs]

but I gave a different database name.. and what did you say ? so i'll keep the sphinx toggle at using theirs itself ?

[bernd-stromberg]

Fill in only YOUR DB-host/...

[hayzamjs]

Okay

[hayzamjs]

How much time will this 'adding source' part take?

[bernd-stromberg]

Do you use the *.sql from the package here [github.com] ? Seconds...

What definitely do you mean with "adding source" ? Insert the table I hope.

[hayzamjs]

Yeah.. I used this one : isohuntto/openbay-db-dump#9

[bernd-stromberg]

Inside of

/[pathto]/openbay-master/src/protected/data

only four datasets for testing ! 👍

[hayzamjs]

What? Its okay right? will it work??

[bernd-stromberg]

What definitely do you mean with "adding source" ?

[hayzamjs]

Adding the database to mysql

[bernd-stromberg]

OK.

Using "/[pathto]/openbay-master/src/protected/data/schema.mysql.sql" only costs seconds.

[TheChiefCoC]

Its all workin for me now :) Thanks a lot @bernd-stromberg
You can visit my bay at : http://tpb.ifilescloud.com