teamtnt / tntsearch Goto Github PK
View Code? Open in Web Editor NEWA fully featured full text search engine written in PHP
Home Page: https://tnt.studio/solving-the-search-problem-with-laravel-and-tntsearch
License: MIT License
A fully featured full text search engine written in PHP
Home Page: https://tnt.studio/solving-the-search-problem-with-laravel-and-tntsearch
License: MIT License
I'm having some issues with the ranking. Is there any way to tell the driver to prioritize results that contain a match in the title rather than in the body? Right now it seems both columns have equal weight and the items I'm looking for almost never show at the top.
Thanks!
hello
i have tried to index arabic words, but the search not works
is there any configuration for utf-8 coding?
It finds results when searched by ID of indexed record.
How to avoid that?
Here is how I created the index:
$indexer = $this->tnt->createIndex('artists.index');
$indexer->query('SELECT id, name FROM artists');
$indexer->run();
Retrieving search results:
$this->tnt->selectIndex("artists.index");
$results = $this->tnt->search($keyword, 5);
$artists = Artist::whereIn('id', $results['ids'])->get();
Hi, I'm trying to follow the steps at the readme but there is no method run() or query() at TNTSearch(); Please fulfill the requirements to build the index. Thanks!
Hi,
In case of multi server environment, the search can originate form any server ( www1, 2, 3..10 ).
It's not possible to have an index on every box.
How do we solve the issue of having storage_path on an external ( virtual ) server accessible to all ?
Thanks.
Hello! As I mentioned in one of my email lately, I'm very happy to use the package for the performance it gives my app and such but I've been having a couple of issues with it. I started implementing a search feature in Laravel app on a Mac. It worked fine, even with a few bugs, but it worked. I then switched to using a PC with Ubuntu 16.04 and since then the package stopped working. At first it said "PHP driver not found". I then installed php7.0-sqlite3. I have sqlite3 installed on the machine so trying to open the index files created from artisan works fine. I can do a "select * from wordlist" for example and I got some results. But it doesn't show anything in the application. I got empty results whenever I try.
What am I doing wrong?
Do you plan on adding Fuzzy Searching to this awesome project?
Problem 1
- Installation request for teamtnt/tntsearch ^0.8.9 -> satisfiable by teamtnt/tntsearch[v0.8.9].
- teamtnt/tntsearch v0.8.9 requires ext-pdo_sqlite * -> the requested PHP extension pdo_sqlite is missing from your system.
To enable extensions, verify that they are enabled in those .ini files:
- /etc/php/7.0/cli/php.ini
- /etc/php/7.0/cli/conf.d/10-mysqlnd.ini
- /etc/php/7.0/cli/conf.d/10-opcache.ini
- /etc/php/7.0/cli/conf.d/10-pdo.ini
- /etc/php/7.0/cli/conf.d/15-xml.ini
- /etc/php/7.0/cli/conf.d/20-calendar.ini
- /etc/php/7.0/cli/conf.d/20-ctype.ini
- /etc/php/7.0/cli/conf.d/20-curl.ini
- /etc/php/7.0/cli/conf.d/20-dom.ini
- /etc/php/7.0/cli/conf.d/20-exif.ini
- /etc/php/7.0/cli/conf.d/20-fileinfo.ini
- /etc/php/7.0/cli/conf.d/20-ftp.ini
- /etc/php/7.0/cli/conf.d/20-gettext.ini
- /etc/php/7.0/cli/conf.d/20-iconv.ini
- /etc/php/7.0/cli/conf.d/20-json.ini
- /etc/php/7.0/cli/conf.d/20-mbstring.ini
- /etc/php/7.0/cli/conf.d/20-mysqli.ini
- /etc/php/7.0/cli/conf.d/20-pdo_mysql.ini
- /etc/php/7.0/cli/conf.d/20-phar.ini
- /etc/php/7.0/cli/conf.d/20-posix.ini
- /etc/php/7.0/cli/conf.d/20-readline.ini
- /etc/php/7.0/cli/conf.d/20-shmop.ini
- /etc/php/7.0/cli/conf.d/20-simplexml.ini
- /etc/php/7.0/cli/conf.d/20-sockets.ini
- /etc/php/7.0/cli/conf.d/20-sysvmsg.ini
- /etc/php/7.0/cli/conf.d/20-sysvsem.ini
- /etc/php/7.0/cli/conf.d/20-sysvshm.ini
- /etc/php/7.0/cli/conf.d/20-tokenizer.ini
- /etc/php/7.0/cli/conf.d/20-wddx.ini
- /etc/php/7.0/cli/conf.d/20-xmlreader.ini
- /etc/php/7.0/cli/conf.d/20-xmlwriter.ini
- /etc/php/7.0/cli/conf.d/20-xsl.ini
You can also run php --ini
inside terminal to see which files are used by PHP in CLI mode.
I get this error when I try to install....
I checked my phpinfo and pdo_sqlite is enabled.
please I need help. Thank you
Hello
IndexNotFoundException in TNTSearch.php line 65:
Index ./storage/users.index does not exist
I'm sure I have users.index file in storage directory. Am I missing something?
I LOVE this package. Thanks very much for putting it together.
I am using the Laravel driver and search multiple models in turn. I would like to then rank the results from all models by a score.
I presume from my digging around that this currently isn't possible? Or am I wrong?
One again, thanks for this. So simple and affective.
Helllo,
I want to use this package in a Bengali website. Works fine with English characters. But doesn't index utf8 bin characters.
Is there anyway I can achieve it?
Regards
Use of undefined constant SQLITE3_INTEGER - assumed 'SQLITE3_INTEGER'
Impressive, good work.
I'm having issues getting the index to automatically update. I followed the instructions outlined here:
http://tnt.studio/blog/searching-for-bobby-fisher-with-laravel-5
I added all 4 of the static functions to my model (I'm assuming the boot method is in the same model?). When I insert a new record into my database (using the DB facade), nothing seems to get indexed. Search never finds these new records, unless I manually run: php artisan .index.
At this point, I'm not sure if it's a bug…or if I added the boot method to the wrong model…or if it's something else. Any help would be much appreciated!
I'm very happy with this package!
I've been struggling to get this installed and working. Every time I work through an issue, a new one pops up. Here is my latest error:
[PDOException]
SQLSTATE[08006] [7] invalid connection option "localhostdbname"
TNTSearchServiceProvider.php: I'm using the defaults (except for the driver).
public function register()
{
$this->app['tntsearch'] = $this->app->share(function ($app) {
$config = [
'driver' => 'pgsql',
'host' => env('DB_HOST', 'localhost'),
'database' => env('DB_DATABASE'),
'username' => env('DB_USERNAME'),
'password' => env('DB_PASSWORD'),
'storage' => storage_path(),
];
if (isset($app['config']['services']['tntsearch'])) {
$config = $app['config']['services']['tntsearch'];
}
$tnt = new TNTSearch;
$tnt->loadConfig($config);
#dd($tnt);
return $tnt;
});
}
When I dd($tnt)
and run php artisan index:users
, it displays the correct settings for my postgresql database.
I'm not sure why it's throwing this error. I've checked everywhere in the documentation, and don't see anywhere that indicates postgresql isn't supported.
Appreciate any help!
thanks for your answer.
I have table where ID is uuid type (string). This library doesn't work when ID type is string. Can you make it?
Hi,
I'm using TNTSearch with Scout.
I'm having a little issue with this:
$ php artisan scout:import App\\Models\\CarModel
sqlite> SELECT * FROM info WHERE key = 'total_documents';
total_documents|0
Instead
$ php artisan tntsearch:import App\\Models\\CarModel
sqlite> SELECT * FROM info WHERE key = 'total_documents';
total_documents|4496
I think the problem is with TNTIndexer's delete method, executed by the update method.
delete method doesn't check if the document exists before decrement total_documents.
public function delete($documentId)
{
$rows = $this->prepareAndExecuteStatement("SELECT * FROM doclist WHERE doc_id = :documentId;", [
['key' => ':documentId', 'value' => $documentId]
])->fetchAll(PDO::FETCH_ASSOC);
...
$total = $this->totalDocumentsInCollection() - 1;
$this->updateInfoTable('total_documents', $total);
}
tinker:
sqlite> SELECT * FROM info WHERE key = 'total_documents';
total_documents|4490
$ php artisan tinker
>>> use TeamTNT\TNTSearch\TNTSearch;
>>> $tnt = new TNTSearch;
>>> $tnt->selectIndex(storage_path("car_models.index"));
>>> $index = $tnt->getIndex();
>>> $index->delete(30557); <-- this document exists
sqlite> SELECT * FROM info WHERE key = 'total_documents';
total_documents|4489
>>> $index->delete(305512397); <-- this document doesn't exists
sqlite> SELECT * FROM info WHERE key = 'total_documents';
total_documents|4488
Possible fix
public function delete($documentId)
{
...
if (sizeof($rows) > 0) {
$total = $this->totalDocumentsInCollection() - 1;
$this->updateInfoTable('total_documents', $total);
}
}
Thanks
First I will like to say this is an amazing search extension! However, I having issues getting this to work on my digitalocean cloud server with laravel forge. The extension works perfectly on my local server (Homestead / vagrant) but onces I upload everything to the cloud, it shows no result.
here is var_dump of my local server Homestead:
And here is var_dump of my digitalocean server:
Here is a var_dump of my code section:
Somehow it seems not pass in the results from $tnt on my cloud side.. Please let me know if I'm doing anything wrong here. My guess is that this is a server configuration issue since it's working fine on my local server.
How would I go about using the facade with delete? I'm getting undefined method currently?
I use a php framework called Fat Free Framework and I intend to create an indexer for its own database storage called Jig. It is simple an active cursor implementation which stores data as JSON structures.
I stumbled across TNTSearch but found the documentation is lacking greatly. I've glossed over the code but I suspect it may take a few weeks to fully understand how to work with the code base.
Might anyone give me any hints on how I may tackle this project?
When I try to use the search method in multiple calls it does not return result after adding the value to the index.
For example:
public function test()
{
$this->loadConfig();
if (!file_exists(storage_path('/app/events.index'))) {
$this->tnt->createIndex('events.index');
}
$this->tnt->selectIndex('events.index');
$str = 'test';
$this->tnt->search($str); // returns no results as expected
$index = $this->tnt->getIndex();
$index->insert(['id' => 1, 'name' => $str]);
$this->tnt->search($str); // should return 'test' as a result from the index but returns 'ids' array as null
}
Could you store the indexes on Amazon S3? So multiple servers could access the same data.
Hi
thank you for this great package ...
I tested it for a sample table with 1 million row , this is the link for download the sample database :
http://www.sample-videos.com/sql/Sample-SQL-File-1000000rows.sql
I ran this query and it returned about 40000 rows but the package returned just 500 rows .
query : select * from user_details where first_name = 'david'
and this is my code :
$tnt->loadConfig($config);
$tnt->selectIndex("Full");
$res = $tnt->search('david', 1000000);
var_dump($res);
what is the problem ?
How to get the numbers of entry in spesefic index ?
For Example:
Indexing like this:
$indexer = TNTSearch::createIndex('book.index');
$indexer->query('SELECT id, title, category_id FROM books');
$indexer->run();
How I can search for title in specific category_id for example?
Hello! Thanks for your package. It's really great.
I have it setup in a Laravel 5.2 application. The main issue I have is with the Ajax search. Doing search with one letter is necessary for my app, just like Google. After setting the asYouType property, which by the way needs the initialization of the class otherwise it won't accept it, it returns results for one letter but the wrong way.
Say I have many posts that have the word yeah in their title, when I type 'y' it will just show on post. But 'yeah' returns all the posts. That's a big issue for me.
Please see what you can do for that. And again good job!
Hi, TNTSearch was working fine using the Facade, but I wanted to use fuzziness, so I tried the method in the docs:
use TeamTNT\TNTSearch\TNTSearch;
$tnt_search = new TNTSearch;
$tnt_search->selectIndex( 'notes.index' );
$tnt_search->fuzziness = true;
$search_results = $tnt_search->searchBoolean( $search, 1000 );
But I get an error about some info table not existing..?
at PDO->query('SELECT * FROM info WHERE key = 'stemmer'') in TNTSearch.php line 297
Testing
Hello!
I want when search keyword "[email protected]" only return result of "[email protected]" . Currently, It return results with "@example.com". How to configuration it.
IMHO, it'd be better to leave this package as simply framework agnostic, and not include the Laravel service provider, and create a separate package like teamtnt/tntsearch-laravel
which would require this package and additionally require illuminate/support
(or whatever required Laravel stuff) with the appropriate version constraints, etc.
Hi there,
How do I tell TNTSearch to ignore accents in its indexing and searching process?
María
becomes maria
Thank you.
Hi,
I have found an issue when using the search/searchBoolean method where the phrase being searched was returning no ids despite the phrase being present in my db.
To replicate, please insert the following rows into the articles db with tests:
id | title | article
11 | Eldred | What the foo
12 | Eldred | Oh my bar
Then add these assertions and run phpunit.
$res = $tnt->searchBoolean('eldred -bar');
$this->assertEquals([11], $res['ids']);
$res = $tnt->searchBoolean('Eldred -bar');
$this->assertEquals([11], $res['ids']);
The second assertion will fail. It appears linked to the phrase being used in addition to that phrase starting with a capital letter. I have managed to get around this issue by using strtolower.
Hi there, first of all, thank's for this awesome project.
Is there a way to search for dash keywords, like this: 70-200
. My app have a lot products like Canon 70-200, and it's not searching correctly.
I pretend to have something like this
$whereRaw = "MATCH(name) AGAINST('+jame* +walk*' IN BOOLEAN MODE)";
As I have now against MySQL with the following results:
James Walker
Jamenie Walk
Jameelah Walka
Jamelle Walkama
Jamei Walkde
Can TNTSearch do this?
Hey @nticaric, thanks for sharing this great package.
I have a big WordPress website that I want to improve the search of. How would I go about to integrate TNTSearch with the current website? Do I need a new DB for it or it would work fine with the current website?
Any direction on this would help. Thanks very much.
First, thanks a lot for creating this package! :)
I just noticed a small issue - Laravel already has a dd
function defined, but there is also one defined in this package and it takes precedence over Laravel's implementation. I would like to continue using Laravel's dd, since it uses the Symfony var_dumper which results in a much more readable output.
Is there any chance that the dd function supplied with this package could be renamed since it's not really part of any core functionality?
Firstly, thank you for a great plugin.
It's very helpful.
I have everything set up but having one issue with search.
I have 2 emails in the database.
[email protected]
[email protected]
If I search for mandy, I only get back [email protected].
Why doesn't mandy2 appear ?
I have fuzziness set to true.
I also have searchBoolean.
Thanks.
Hey,
What are the fields table and hitlist table for in the database, they remain empty.
Also is there a way to save a title in the database so the search results have a title.
And third and last, my info table is filled with 'TeamTNT\TNTSearch\Stemmer\PorterStemmer' what is this for?
I'm having some ranking issues. When I use multiple words in the search string, the result containing all of the terms is not at the top. When using only one word the results are ok. How does searching with multiple terms work?
If I copy-paste the title of the article in the search box I would expect it to be the top result but it seems that it's always outranked by all articles containing only one of the words.
How would you go about indexing and searching relationships from like a eloquent query?
Hello,
I have a list of ~10k airports with additional data in a table. For example, I have a dynamic search input and want to select an airport in Berlin, if I request "b", I get one result "Vernon B. C." (a city in the USA), "be" gets me only "Nossi-Be" (Madagascar), "ber"/"berl"/"berli" — no results, and only full city name "berlin" finds and returns 6 Berlin airports. Same goes for every other query. I saw TNTSearch working very differently on the demo page, any ideas on what I might be doing wrong?
I'm indexing city names like so:
$tnt = new TNTSearch;
// I get "SQLSTATE[HY000] [14] unable to open database file"
// error unless I load a configuration.
$tnt->loadConfig([
'driver' => config('tntsearch.driver'),
'host' => config('tntsearch.host'),
'database' => config('tntsearch.database'),
'username' => config('tntsearch.username'),
'password' => config('tntsearch.password'),
'storage' => config('tntsearch.storage')
]);
$indexer = $tnt->createIndex('airports.index');
$indexer->query('SELECT id, city_name_ru, city_name_en FROM airports;');
$indexer->run();
And here's my controller:
public function searchCityWithAnAirport(Request $request)
{
$tnt = new TNTSearch;
$tnt->loadConfig([
'driver' => config('tntsearch.driver'),
'host' => config('tntsearch.host'),
'database' => config('tntsearch.database'),
'username' => config('tntsearch.username'),
'password' => config('tntsearch.password'),
'storage' => config('tntsearch.storage')
]);
$tnt->selectIndex('airports.index');
$tnt->asYouType = true;
$result = $tnt->searchBoolean($request->input('query'), 10);
return $result;
}
In case you'll have time to test it yourself, here's a Laravel migration file, and here's a CSV file with data.
Is this a typo? or am I confused?
Shouldn't TeamTNT\TNTSearch\Support\Hihglighter
be TeamTNT\TNTSearch\Support\Highlighter
After build in your search into my user search I found a bug.
When searching for something and the unique identifier is 0
the algorythmus finds the data but the filter
https://github.com/teamtnt/tntsearch/blob/master/src/TNTSearch.php#L109
filters out the correct result. *Because key is 0 I guess.
Do you have any idea?
Work arround for me is remove the filter lines between L109 and L117.
Maybe the Collection map/filter does something wrong there?
Is there a way to search for a partial of a word not for the full word? For example to return "Apple" if I search for "App" or "ppl"?
I would like to have a way to use another stemmer class like the one:
https://github.com/wamania/php-stemmer
It's possible?
From what I saw is fixed in the code the use of Stemmers:
$class = 'TeamTNT\TNTSearch\Stemmer\' . ucfirst(strtolower($language)) . 'Stemmer';
ps:sorry about my English
Thank you
The old version has folders in small case, in the src
which is causing errors.
The updated version if this package fixed that, but it isn't the one on Packagist.
When will the updated version be available?
Thanks.
Hello, i have a question regarding tntsearch and german umlauts on laravel 5.3. As i see the german umlauts i.e. ä,ö,ü,ß are not indexed. In the search form, i get no result for this. Do you have any suggestions how to config that? Thank you
regards Klaus
If I understand correctly I need to re index after inserts\updated are made?
do I need to run this code on all the results or only on the updated results like
$indexer = $tnt->createIndex('name.index');
$indexer->query('SELECT id, article FROM articles where id = $last_inserted_article;');
$indexer->run();
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.