Geospatial Indexing about nitrite-java

[kennywk]

Sure, will see what I can do once I get back to my office.

…

Sent from my iPhone

On 14. Nov 2017, at 11:12, Anindya Chatterjee ***@***.***> wrote: Honestly, I don't have any experience with geospatial operations. Care to contribute? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[anidotnet]

Honestly, I don't have any experience with geospatial operations. Care to contribute?

[kennywk]

I'm trying to get the gradle environment working correctly... seems to be giving me problems. However i have noticed that your using enough of the right libraries to implement geospatial queries/indexing rather easily... would just need to know what you'd prefer with regards to placement. On Tue, Nov 14, 2017 at 11:18 AM, Kenneth Keith <[email protected]> wrote:

…

Sure, will see what I can do once I get back to my office. Sent from my iPhone On 14. Nov 2017, at 11:12, Anindya Chatterjee ***@***.***> wrote: Honestly, I don't have any experience with geospatial operations. Care to contribute? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFgDOOpz-TSHwAvZhYubdWudCYEHdJjeks5s2WengaJpZM4QdCwv> .

[anidotnet]

I did not understand what is the "placement" you are referring to.

By the way what is the error you are facing currently with gradle?

[kennywk]

Yes, it is a gradle thing, basically not wanting to retrieve/get the necessary dependencies. By placement i am referring to what classes/pattern/location you would prefer that i place the code. I can put it inside the cursor classes, but it seems as if the logic is done somewhere else. I'll continue looking into the code to get better familiar with the flow.

…

On Wed, Nov 15, 2017 at 3:13 AM, Anindya Chatterjee < ***@***.***> wrote: I did not understand what is the "placement" you are referring to. By the way it is the error you are facing currently with gradle? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFgDOKUdEugnyAcFg0wfJdYK5lHdmfUaks5s2kjCgaJpZM4QdCwv> .

[anidotnet]

For query, you need to create different filters like WithinFilter, NearFilter etc. Take a look into other filters likeAndFilter, EqualFilter etc. You need to create functions like within, near in Filters class also. For special treatment in query you need to modify general query logic in SearchService and IndexedSearchService class. For indexing, take a look into IndexingService class.

For pojos, you need to work on similar classes under objects package.

[kennywk]

Cool, i'll look into it.... Any recommendations for the dependency issues? I normally use maven and it provides the ability within netbeans to download the necessary dependencies.... with gradle i'm not sure.

…

On Wed, Nov 15, 2017 at 3:10 PM, Anindya Chatterjee < ***@***.***> wrote: For query, you need to create different filters like WithinFilter, NearFilter etc. Take a look into other filters likeAndFilter, EqualFilter etc. You need to create functions like within, near in Filters class also. For special treatment in query you need to modify general query logic in SearchService and IndexedSearchService class. For indexing, take a look into IndexingService class. For pojos, you need to work on similar classes under objects package. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFgDOPvka7Yukn4GKBHLjbi456gO0R_Oks5s2vDFgaJpZM4QdCwv> .

[kennywk]

Anindya, Unfortunately, until i can get gradle to download the necessary dependencies i will be unable to add contributions. I have an idea of what i need to do, but my lack of knowledge with gradle and the ability for the ide to get the necessary jars has kept me from doing being able to do anything. K On Wed, Nov 15, 2017 at 5:02 PM, Kenneth Keith <[email protected]> wrote:

…

Cool, i'll look into it.... Any recommendations for the dependency issues? I normally use maven and it provides the ability within netbeans to download the necessary dependencies.... with gradle i'm not sure. On Wed, Nov 15, 2017 at 3:10 PM, Anindya Chatterjee < ***@***.***> wrote: > For query, you need to create different filters like WithinFilter, > NearFilter etc. Take a look into other filters likeAndFilter, EqualFilter > etc. You need to create functions like within, near in Filters class > also. For special treatment in query you need to modify general query logic > in SearchService and IndexedSearchService class. For indexing, take a > look into IndexingService class. > > For pojos, you need to work on similar classes under objects package. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#34 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AFgDOPvka7Yukn4GKBHLjbi456gO0R_Oks5s2vDFgaJpZM4QdCwv> > . >

[anidotnet]

Can you please run this below command in the project root dir?

./gradlew clean assemble -x test --info --stacktrace

If it fails, check if you are behind proxy firewall or not. If you are, then provide the proxy information in the command line

./gradlew -Dhttp.proxyHost=<http proxy host> -Dhttp.proxyPort=<http proxy port> -Dhttps.proxyHost=<https proxy host> -Dhttps.proxyPort=<http proxy port> clean assemble -x test --info --stacktrace

If it still fails then let me know the error details here. If the build passes, then you should open your project in either IntelliJ or eclipse. If you face any problem in just opening the project in IDE, you can go through numerous resource on net on how to do that or let me know here.

[anidotnet]

Are you still pursuing this?

[kennywk]

Anindya, Yes, i've worked on it some and have a basic implementation minus a btree/STR indexing. Still looking at how to integrate the actual indexing method. The existing code is not so clear.

…

On Mon, Nov 20, 2017 at 12:59 PM, Anindya Chatterjee < ***@***.***> wrote: Are you still pursuing this? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFgDON0BwusurqlgN2OnjxE1XlgMe9t4ks5s4Wm2gaJpZM4QdCwv> .

[anidotnet]

Indices in nitrite are just maps stored in database for reverse lookup. These maps are basically MVMaps which implements B+ trees. The same MVMap also stores documents and objects in nitrite. So it is just more maps for indices. You should look at these 3 files to understand its inner working

• https://github.com/dizitart/nitrite-database/blob/master/nitrite/src/main/java/org/dizitart/no2/internals/IndexingService.java
• https://github.com/dizitart/nitrite-database/blob/master/nitrite/src/main/java/org/dizitart/no2/internals/IndexedSearchService.java
• https://github.com/dizitart/nitrite-database/blob/master/nitrite/src/main/java/org/dizitart/no2/internals/IndexMetaService.java

IndexingService is for indexing the collections. IndexedSearchService is used to search on indexed fields and IndexMetaService stores metadata for indices for easy lookup.

I don't know how indexing works for geospatial data also don't know if it is possible to implement geospatial indexes using maps. But if it helps you can also take a look at the full-text indexing logic here

• Nitrite's own NitriteTextIndexingService https://github.com/dizitart/nitrite-database/blob/master/nitrite/src/main/java/org/dizitart/no2/internals/NitriteTextIndexingService.java
• Lucene based text indexing https://github.com/dizitart/nitrite-database/blob/master/nitrite/src/test/java/org/dizitart/no2/services/LuceneService.java

Let me know if you need anything specific.

[kennywk]

There are multiple types of spatial indexes depending upon what you're trying to do. Some popular ones are R-Tree, B-Tree and QuadTree.... I will take a look when i get time and see how it can be implemented. My initial research shows that it could be implemented, however i don't have the time at the moment to do it. K

…

On Mon, Nov 20, 2017 at 4:12 PM, Anindya Chatterjee < ***@***.***> wrote: Indices in nitrite are just maps stored in database for reverse lookup. These maps are basically MVMaps which implements B+ trees. The same MVMap also stores documents and objects in nitrite. So it is just more maps for indices. You should look at these 3 files to understand its inner working - https://github.com/dizitart/nitrite-database/blob/master/ nitrite/src/main/java/org/dizitart/no2/internals/IndexingService.java <https://github.com/dizitart/nitrite-database/blob/master/nitrite/src/main/java/org/dizitart/no2/internals/IndexingService.java> - https://github.com/dizitart/nitrite-database/blob/master/ nitrite/src/main/java/org/dizitart/no2/internals/ IndexedSearchService.java <https://github.com/dizitart/nitrite-database/blob/master/nitrite/src/main/java/org/dizitart/no2/internals/IndexedSearchService.java> - https://github.com/dizitart/nitrite-database/blob/master/ nitrite/src/main/java/org/dizitart/no2/internals/IndexMetaService.java <https://github.com/dizitart/nitrite-database/blob/master/nitrite/src/main/java/org/dizitart/no2/internals/IndexMetaService.java> IndexingService is for indexing the collections. IndexedSearchService is used to search on indexed fields and IndexMetaService stores metadata for indices for easy lookup. I don't know how indexing works for geospatial data also don't know if it is possible to implement geospatial indexes using maps. But if it helps you can also take a look at the full-text indexing logic here - Nitrite's own NitriteTextIndexingService https://github.com/dizitart/ nitrite-database/blob/master/nitrite/src/main/java/org/ dizitart/no2/internals/NitriteTextIndexingService.java <https://github.com/dizitart/nitrite-database/blob/master/nitrite/src/main/java/org/dizitart/no2/internals/NitriteTextIndexingService.java> - Lucene based text indexing https://github.com/dizitart/ nitrite-database/blob/master/nitrite/src/test/java/org/ dizitart/no2/services/LuceneService.java <https://github.com/dizitart/nitrite-database/blob/master/nitrite/src/test/java/org/dizitart/no2/services/LuceneService.java> Let me know if you need anything specific. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#34 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFgDOMLyeI49yZHHz2fCY8Nu1Bhg_9KSks5s4ZYYgaJpZM4QdCwv> .

[anidotnet]

I hope things are going well for you. Just wanted to check if you have done any progress on this. Have you considered geohash for the implementation?

[kennywk]

Hey thanks for message. I’ve made some progress but have not had time to invest. Geohashing works pretty good for points but I’m not sure if it would work for geometries

…

Sent from my iPhone

On 10. Dec 2017, at 05:21, Anindya Chatterjee ***@***.***> wrote: I hope things are going well for you. Just wanted to check if you have done any progress on this. Have you considered geohash for the implementation? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[anidotnet]

Fair enough.

[anidotnet]

Spatial indexing support has been added to 4.0.0-SNAPSHOTS