[BUGFIX] Prevent exception caused by hash collisions in indexed_search
The indexed_search extension uses a truncated md5 hash for building a word index. The truncation greatly increases the probability for collisions. These collisions are the root cause for the problems that are described in the related issues. The best solution would be to either use full-length hashes or no hashes at all. As this is not possible without changing the database schema and rebuilding the whole word index, this patch prevents the exceptions caused by the hash collisions. This is done by removing all words with colliding hashes before the word index is updated. Note that this patch just makes it possible to index page contents that contain hash collisions (words with colliding hashes) again. The affected words are obviously not added to the word index and though won't yield results when searched for. Resolves: #101249 Related: #84541 Related: #90977 Related: #87138 Releases: main, 12.4, 11.5 Change-Id: I5bf7d562ee42f63a5eeb99381b252018439bfaab Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/74742 Tested-by:core-ci <typo3@b13.com> Reviewed-by:
Benni Mack <benni@typo3.org> Reviewed-by:
Stefan Bürk <stefan@buerk.tech> Tested-by:
Stefan Bürk <stefan@buerk.tech> Tested-by:
Benni Mack <benni@typo3.org>
Showing
- typo3/sysext/indexed_search/Classes/Indexer.php 28 additions, 2 deletionstypo3/sysext/indexed_search/Classes/Indexer.php
- typo3/sysext/indexed_search/Tests/Functional/Fixtures/Indexer/indexing_words_twice_first.csv 10 additions, 0 deletions...unctional/Fixtures/Indexer/indexing_words_twice_first.csv
- typo3/sysext/indexed_search/Tests/Functional/Fixtures/Indexer/indexing_words_twice_second.csv 10 additions, 0 deletions...nctional/Fixtures/Indexer/indexing_words_twice_second.csv
- typo3/sysext/indexed_search/Tests/Functional/Fixtures/Indexer/phash_collision.csv 13 additions, 0 deletions...rch/Tests/Functional/Fixtures/Indexer/phash_collision.csv
- typo3/sysext/indexed_search/Tests/Functional/IndexerTest.php 91 additions, 0 deletionstypo3/sysext/indexed_search/Tests/Functional/IndexerTest.php
Please register or sign in to comment