kentico / xperience-by-kentico-lucene

Xperience by Kentico search integration with the latest 4.8 beta version of Lucene.NET

Xperience by Kentico Lucene

	Package
Lucene
Lucene Administration only
Lucene Live site only

Description

Xperience by Kentico search integration with the latest 4.8 beta version of Lucene.NET, enabling auto-indexing of content in Xperience based on application-local, code-defined search indexes and search results retrieval.

This repository contains projects for 3 NuGet packages, that is Kentico.Xperience.Lucene.Admin, Kentico.Xperience.Lucene.Core and Kentico.Xperience.Lucene.

Use Kentico.Xperience.Lucene. This is the recommended version for usage of Xperience by Kentico support for Lucene .NET NuGet package and unlike the other two contains full support for both admin ui and live site.

Use Kentico.Xperience.Lucene.Core in your Xperience by Kentico project for Lucene Search in your live website. Use Kentico.Xperience.Lucene.Admin in your Xperience by Kentico Administration project for Lucene Search Administration UI managment.

Screenshots

Library Version Matrix

Xperience Version	Library Version
>= 29.0.0	>= 6.1.x
>= 28.4.3, < 29.0.0	>= 5.x.x
>= 28.0.0	>= 3.x.x
>= 26.2.0, < 27.0.0	>= 2.x, < 3.x.x

Dependencies

• ASP.NET Core 6.0
• Xperience by Kentico

Package Installation

Add the package to your application using the .NET CLI

dotnet add package Kentico.Xperience.Lucene

Quick Start

This section is relevant for each Lucene NuGet. Steps number 3 and 4 are only relevant if you use NuGet package Kentico.Xperience.Lucene or Kentico.Xperience.Lucene.Admin.

1. Define a custom DefaultLuceneIndexingStrategy implementation to customize how content pages/content items are processed for the index.

2. Add this library to the application services, registering your custom DefaultLuceneIndexingStrategy and Lucene

   // Program.cs
   
   // Registers all services and uses default indexing behavior (no data is indexed)
   services.AddKenticoLucene();
   
   // or
   
   // Registers all services and enables custom indexing behavior
   services.AddKenticoLucene(builder =>
      builder
         .RegisterStrategy<GlobalSearchIndexingStrategy>("DefaultStrategy")
         .RegisterStrategy<ArticlesSearchIndexingStrategy>("ArticlesStrategy"));

3. Create an index in Xperience's Administration within the Search application added by this library.

4. Rebuild the index in Xperience's Administration within the Search application added by this library.

5. Use the ILuceneSearchService (via DI) to retrieve the index populated by your custom ILuceneIndexingStrategy.

6. Execute a search with a customized Lucene Query (like the MatchAllDocsQuery) using the ILuceneSearchService.

7. Display the results on your site with a Razor View 👍.

Full Instructions

View the Usage Guide for more detailed instructions.

You can view and start the DancingGoat example project. For advanced Strategy usage add your website url base to the WebCrawlerBaseUrl appsettings.json

{
    // Other settings ...
    // Our example runs on localhost with port: 58093
    "WebCrawlerBaseUrl": "http://localhost:58093/"
}

This is required for our Scraping example.

Projects

Project	Description
src/Kentico.Xperience.Lucene	Xperience by Kentico Lucene integration library
src/Kentico.Xperience.Lucene.Core	Xperience by Kentico Lucene integration Live site only library
src/Kentico.Xperience.Lucene.Admin	Xperience by Kentico Lucene integration Admin only library
examples/DancingGoat	Example project to showcase basic search functionality

Contributing

To see the guidelines for Contributing to Kentico open source software, please see Kentico's CONTRIBUTING.md for more information and follow the Kentico's CODE_OF_CONDUCT.

Instructions and technical details for contributing to this project can be found in Contributing Setup.

License

Distributed under the MIT License. See LICENSE.md for more information.

Support

This project has Full support by 7-day bug-fix policy.

See SUPPORT.md for more information.

Security

For any security issues see Kentico's SECURITY.md.

xperience-by-kentico-lucene's Issues

Support multiple sets of facets

Is your feature request related to a problem? Please describe.
The LuceneSearchResultModel only supports a single facet value, which means if an index includes multiple faceted values a user filters on, there isn't a good way to represent that in the result model class.

Example: Coffee Flavor and Coffee Type could be 2 completely separate facets that could be filtered on independently.

Describe the solution you'd like
If a search index includes multiple faceted properties then the search results make it possible to represent multiple facets being filtered for a search operation.

Describe alternatives you've considered
I could avoid using the LuceneSearchResultModel and create a custom model if the LuceneSearchResultModel is intended only for "simple" single facet scenarios.

Additional context
I think LuceneSearchResultModel.Facet should be combined with LuceneSearchResultModel.Facets as a Dictionary<string, LabelAndValue[]> where each entry in the dictionary is all the facet result values for each faceted property.

In the simple case, LuceneSearchResultModel.Facets would have 1 entry for the single faceted property.

Compatibility between Kentico.Xperience.Lucene 6.0.0 and XbyK 29.0.0

Describe the bug
I have updated all the nuget packages in my project:

• XbyK to 29.0.0
• Kentico.Xperience.Lucene to 6.0.0

After the update, I am experiencing a 500 Internal server error when I try I open the Development -> Search application in the Admin.

Also, when I attempt to delete a page using XbyK API, I am getting error that is referring to this Lucene integration.

So I am wondering whether the issue is on my end or on yours.

When I remove the Kentico.Xperience.Lucene from the project, things are working okay.

To Reproduce
Steps to reproduce the behavior:

1. Login to Admin
2. Navigate to Development -> Search
3. See error

Search index updates aren't triggered if included content type entries contain whitespace

Brief bug description

We noticed certain page types were not triggering search index updates when the page was updated. After investigating we noticed that there was whitespace after a line item in the included content types of the index path. We removed the additional whitespace from the line item and edited the page again and saw the index update as expected.

Repro steps

1. Create or edit a page search index at /admin/lucene/indexes
2. In the "Included ContentType Items" field include an entry with extra whitespace after the item (in our instance the type that failed to trigger an update was at the end of the list)
3. Save the path, save the index, and observe the last updated time of the index
4. Edit a page's content (in our instance, we are updating the url of the page)
5. Head back to the indexes and observe the last updated time

Expected behavior

After waiting a reasonable amount of time for the index to update, the index last updated time should be after the page last updated time and the page that was modified returns it's new data in the results

Test environment

• Platform/OS:
  • net6.0
  • Windows 10
• Browser
  • Chrome
• Version
  • Xperience by Kentico version 28.1.0
  • Kentico.Xperience.Lucene 4.1.0

Screenshots

The configuration we had that caused the issue, with client codename omitted.

LuceneIndex has hard-coded StandardAnalyzer

Motivation

When I want to build index for Kentico, I'd like to cover some specifics (e.g. diacritics, umlaut and so on) of desired language by picking adequate Analyzer and not rely on preset StandardAnalyzer.
Standard analyzer does not correctly work with diacritics (I'd like to find "über" either as "über" and "uber").

Proposed solution

I'd like to add arbitrary analyzers during service registration (probably via new method RegisterAnalyzer on ILuceneBuilder) and then pick appropriate analyzer along with indexing strategy in Kentico admin.
I expect to have possibility to obtain selected analyzer in code.

XbyK Lucene Search null point error

Unable to integrate Lucene search into XbK - admin UI throws an error.

• Installed fresh DancingGoat template website for version 28.2.0
• Restored the template database with kentico-xperience-dbmanager
• Run the website to test it, and everything worked well.
  ------Search module does not exist yet.
• Installed the nuget package Kentico.Xperience.Lucene 4.0.0
  When I run the project again (by using IIS Express from VS) I got the following error message:

• I quickly add the lucene code into Program.cs file
  

• Code from previous step solved the live site issue that I previous had and I was able to boot up website. I saw in event log that classes were created for Lucene Module, I was able to confirm the same in the database (saw tables) and I was able to see the 'Search' module on dashboard.
  

When I clicked to the module, I get the 500 error (can't open module at all, issue occur on every click), which is the reason why I'm reaching out here. You can find the errors attached
err_1.txt
err_2.txt

Feature: Update existing index configuration when possible

Motivation

We currently delete all related data for an index when it's updated and create new relationships from the submitted configuration.

While this works, it is noisy in Xperience's CI repository - adding 1 index path will delete all languages, channels, and paths for an index.

This also makes more difficult to use CD because removing the deleted objects will require a RestoreMode of Full which isn't compatible with most PROD environments after an initial release.

Proposed solution

We should track the identifiers of the related objects in the LuceneIndexConfigurationComponent and use that to update items that already exist, only deleting when appropriate.

Additional context

I saved a search index without making any changes to it and this is the result in the CI repository.

The paths that were deleted locally will need to also be deleted in non-local environments, except manually.

Allow Lucene when Admin is not published

So, my general expectation for Lucene is to be able to implement it similar to the way we did in Kentico 12 and below where, we can tell it we want to do screen scraping or structured content searches, define some parameters, define a search results page that uses the settings and viola, search happens. I had a couple problems when implementing the github version of Lucene, first, it targets .net core 6, however, we’re on .net core 8 and I couldn’t get it working at all. But I left the code in place, and discovered the admin problem when I later started to working on separating the Admin.

So, I’m thinking there is an admin component where we configure how the search will work, info regarding that configuration gets saved in the DB, then when running the site and using search, it would utilize the saved config info in the database and provide the search feature and search results.

Also, since Kentico is suggesting .net core 8, so should this plugin be I would think.