Adds support for Fulltext Search engines like Sphinx and Solr to SilverStripe CMS
Installs: 250 330
Open Issues: 12
- php: ^8.1
- ext-curl: *
- monolog/monolog: ^3.2
- silverstripe/framework: ^5
- silverstripe/silverstripe-proxy-db: ^2
- silverstripe/solr-php-client: ^1
- symfony/process: ^6.2
- silverstripe/fulltextsearch-localsolr: Adds a ready-to-use local Solr server for initial development
- symbiote/silverstripe-queuedjobs: Add background execution of indexing tasks
This package is auto-updated.
Last update: 2023-06-01 05:31:34 UTC
Adds support for fulltext search engines like Sphinx and Solr to Silverstripe CMS. Compatible with PHP 7.2
Important notes when upgrading to fulltextsearch 3.7.0+
There are some significant changes from previous versions:
Draft content will no longer be automatically added to the search index. This new behaviour was previously an opt-in behaviour that was enabled by adding the following line to a search index:
$this->excludeVariantState([SearchVariantVersioned::class => Versioned::DRAFT]);
canView() check against an anonymous user (i.e. someone not logged in) and a
ShowInSearch check is now
performed by default against all records (DataObjects) before being added to the search index, and also before being
shown in search results. This may mean that some records that were previously being indexed and shown in search results
will no longer appear due to these additional checks.
These additional checks have been added with data security in mind, and it's assumed that records failing these checks probably should not be indexed in the first place.
Enable indexing of draft content:
You can index draft content with the following yml configuration:
SilverStripe\FullTextSearch\Search\Services\SearchableService: variant_state_draft_excluded: false
However, when set to false, it will still only index draft content when a DataObject is in a published state, not a
draft-only or modified state. This is because it will still fail the new anonymous user
canView() check in
SearchableService::isSearchable() and be automatically deleted from the index.
If you wish to also index draft content when a DataObject is in a draft-only or a modified state, then you'll need
to also configure
SearchableService::indexing_canview_exclude_classes. See below for instructions on how to do this.
Disabling the anonymous user canView() pre-index check
You can apply configuration to remove the new pre-index
canView() check from your DataObjects if it is not necessary,
or if it impedes expected functionality (e.g. for sites where users must authenticate to view any content). This will
also disable the check for descendants of the specified DataObjects. Ensure that your implementation of fulltextsearch
is correctly performing a
canView() check at query time before disabling the pre-index check, as this may result in
leakage of private data.
SilverStripe\FullTextSearch\Search\Services\SearchableService: indexing_canview_exclude_classes: - Some\Org\MyDataObject # This will disable the check for all pagetypes: - SilverStripe\CMS\Model\SiteTree
You can also use the
updateIsSearchable extension point on
SearchableService to modify the result of the method
canView() checks have run.
It is highly recommend you run a solr_reindex on your production site after upgrading from 3.6 or earlier to purge any old data that should no longer be in the search index.
These additional check can have an impact on the reindex performance due to additional queries for permission checks.
If your site also indexes content in files, such as pdf's or docx's, using the text-extraction
module which is fairly time-intensive, then the relative performance impact of the
canView() checks won't be as noticeable.
Details on filtering before adding content to the solr index
SolrReindexBase. Used when indexing all records during Solr reindex.
SearchUpdateProcessor. Used when indexing single records during
Details on filtering when extracting results from the solr index
SolrIndex. This will often be used in CWP implementations that use the
CwpSearchEngineclass, as well as most custom implementations that call
SearchForm. This will be used in solr implementations where a
/SearchFormurl is used to display search results.
- Some implementations will call
SearchableService::isViewable()twice. If this happens then the first call will be cached in memory so there is virtually no performance penalty calling it a second time.
- If your implementation is very custom and does not subclass nor make use of either
SearchForm, then it's recommended you update your implementation to call
- Silverstripe 4.0+
Note: For Silverstripe 3.x, please use the 2.x release line.
For pure Solr docs, check out the Solr 4.10.4 guide.
See the docs for configuration and setup, or for the quick version see the quick start guide.
For details of updates, bugfixes, and features, please see the changelog.
Get rid of includeSubclasses - isn't actually used in practice, makes the codebase uglier, and ClassHierarchy can be used at query time for most of the same use cases
Fix field referencing in queries. Should be able to do
$query->search('Text', 'Content'), not
$query->search('Text', SiteTree::class . '_Content')like you have to do now
Make sure that when field exists in multiple classes, searching against bare fields searches all of them
Allow searching against specific instances too
Make fields restrictable by class in an index - 'SiteTree#Content' to limit fields to a particular class, maybe 'Content->Summary' to allow calling a specific method on the field object to get the text
Allow following user relationships (Children.Foo for example)
Be clearer about what happens with relationships to stateful objects (e.g. Parent.Foo where Parent is versioned)
Improvements to SearchUpdater
Make it work properly when in-between objects (the A in A.B.Foo) update
Allow user logic to cause triggering reindex of documents when field is user generated
Add generic APIs for spell correction, file text extraction and snippet generation