Skip to Content

SOLR Search Integration

Version 25.1 by savis on 2012/05/28 11:45
Warning
For security reasons, the document is displayed in restricted mode as it is not the current version. There may be differences and errors due to this.

Failed to execute the [velocity] macro. Cause: [The execution of the [velocity] script macro is not allowed in [dev:XWiki.DesignClassSheet]. Check the rights of its last author or the parameters if it's rendered from another script.]. Click on this message for details.

Project Description

 The idea of the project is to use Apache SOLR as the search engine for XWiki. XWiki is using Lucene as a core component for Wiki Search. Lucene is little hard to configure and doesn't support features like facet search, hit highlighting, customizing search relevancy using boost index out of the box. Solr stands out in its minimal configuration to implement the search engine.Few libraries and a couple of XML configuration files are sufficient to implement a well to do engine. Configuring multiple languages is easy in SOLR compared to Lucene.
          
Using SOLR, one can customize the indexing process by using required analyzers with selected tokenizers and set of filters on the dataset to generate highly customizable relevancy index. Through the front end, the user can select or configure the fields to be searched for and their weight which contributes to the document score. The SOLR component offers flexibility in the following ways.

  • Based on SOLR's schema and complementary information, the indexing process is customizable to only index and store as little information as needed. 
  • Through code customizability (exploiting the possibility of Groovy code in pages), the transformation of a user-query to a SOLR query should be
       adjustable, far beyond the simple text-parsing (enabling, for example, the prohibition of some spaces, or the conversion to multiple fields based on input
       parameters).
  • This component supports calibrating the search engine's parameter (such as the Dismax' parser coefficients, the analyzers usefulness, ...)
       by classical quantitative methods such as precision-and-recall, which any wiki master, or a collaborator, should be able to exploit and report.

Approach

 XWiki-platform-search module is split into api and solr implementation. We can have a single Wiki Search API exposed to xwiki and have multiple components with different implementations of SOLR like Embedded Solr server, a remote solr server hosted on a different source, indexing and querying the database instead of XWiki document objects.. etc..

Tentative Project Architecture

http://dl.dropbox.com/u/2027256/XWikiApplication%281%29.png
 http://dl.dropbox.com/u/2027256/XWikiApplication.png

Failed to execute the [velocity] macro. Cause: [The execution of the [velocity] script macro is not allowed in [dev:XWiki.DesignClassSheet]. Check the rights of its last author or the parameters if it's rendered from another script.]. Click on this message for details.

Solr APIs

The following are the APIs proposed for Solr which can be used by any one of the XWiki's scripting language. The design of the APIs are aimed to be generic enough that the backend could be lucene or solr.

getBackend

public String getBackend()

Returns:

Returns the name of the backend which is currently in use. It could be Lucene or Solr. 

rebuildIndex

public int rebuildIndex(com.xpn.xwiki.api.XWiki wiki, com.xpn.xwiki.api.Context context)

Starts the rebuild of the whole index

Parameters:

wiki -

context -

Returns:

Number of documents scheduled for indexing. -1 in case of errors

search

public searchResponse search(java.lang.String query,

                                                                java.util.List<String> languages,

                                                                com.xpn.xwiki.api.XWiki wiki)

Searches the configured Indexes using the specified query for documents in the given languages. 

With virtual wikis enabled in your xwiki installation this will deliver results from all virtuall wikis.

Parameters : 

query - query string given by the user

languages -  list of language codes to search in, may be null to search all languages. Language codes can be:

  • default for content having no specific language information
  • lower case 2- letter language codes like en, es , fr.

Returns:

a searchResponse instance containing the results (i.e) the response objects.

search

public searchResponse search(java.lang.String query,

                                                               java.lang.String virtualWikiNames 

                                                               java.util.List<String> languages,

                                                               com.xpn.xwiki.api.XWiki wiki)

Searches the configured Indexes using the specified query for documents in the given languages belonging to one of the given virtual wikis.

Parameters : 

query - query string given by the user

languages -  list of language codes to search in, may be null to search all languages. Language codes can be:

  • default for content having no specific language information
  • lower case 2- letter language codes like en, es , fr.

virtualWikiNames - Names of the virtual wikis to search in. May be null for global search.

Returns:

a searchResponse instance containing the results (i.e) the response objects.

searchResponse class - It has methods to manipulate the QueryResponse object. The QueryResponse Object is passed as a constructor parameter. These methods can be accessed by the scripting language.
getRelevantResult

private List<searchResult> getRelevantResult()

This method preprocess the results obtained from the QueryResponse object based on score, XWiki Rights Object. This method is not exposed to scripting language but called from getResults method.

Returns :

A list containing the searchResult instances. The searchResult class contains the index fields as member variables and the get methods to access them.

getTotalResults

 public java.util.List getTotalResults()


Returns:
all search results in one list. The list consists of instances of searchResults.
getResults
public java.util.List getResults(int beginIndex, int items)
Returns a list of search results. According to beginIndex and endIndex, only a subset of the results is returned. To get the first ten results, one would use beginIndex=1 and items=10.

Parameters:
beginIndex - 1-based index of first result to return.
items - number of items to return
Returns:
List of SearchResult instances starting at beginIndex and containing up to items elements.
getHitcount
public int getHitcount()

Returns:
total number of searchresults the user is allowed to view.
hasNext
public boolean hasNext(java.lang.String beginIndex,
                        java.lang.String items)

Parameters:
beginIndex -
items -
Returns:
true when there are more results than currently displayed.
hasPrevious
public boolean hasPrevious(java.lang.String beginIndex)

Parameters:
beginIndex -
Returns:
true when there is a page before the one currently displayed, that is, when beginIndex > 1
getNextIndex
public int getNextIndex(java.lang.String beginIndex,
                        java.lang.String items)

Parameters:
beginIndex -
items -
Returns:
the value to be used for the firstIndex URL parameter to build a link pointing to the next page of results
getPreviousIndex
public int getPreviousIndex(java.lang.String beginIndex,
                            java.lang.String items)

Parameters:
beginIndex -
items -
Returns:
the value to be used for the firstIndex URL parameter to build a link pointing to the previous page of results
getEndIndex
public int getEndIndex(java.lang.String beginIndex,
                       java.lang.String items)

Parameters:
beginIndex -
items -
Returns:
the index of the last displayed search result

About

About

Support

Platform

User Guide

Admin Guide

Developer Guide

Projects

XWiki

Extensions

Other

Contribute

Status

Practices

Under the Hood

Get Involved

Get Connected