[FEATURE] Automatically resolve relations for records
What does it do? When creating a record object out of a database row (via the RecordFactory), the Record object now holds enriched values for fields where the content is "known" based on TCA / Schema API. This is especially relevant for records with possible relations, for field types such as "type=group", "type=select" with foreign_table set and "type=inline" as an example. The main purpose for this relation enriching is rendering of the backend page module and content in frontend, but it can be used for any kind of TCA-based record. It also works recursively and with circular dependencies thanks to the RecordIdentityMap. In order to avoid any problems with large amounts of data, an approach of "Lazy+Greedy Fetching" was chosen. How does this approach work under the hood? As an example, we load 10 content elements on a page (1 DB query) so we can render them. Step 1: Lazy Collections / RecordPropertyClosure RecordFactory filters out only the relevant fields and their values from a record's type. Now, the RecordFactory also checks for fields with their meaning and uses a different value for a field. Example: For a type=inline value, there was the number "5" as value (= 5 relations) available, now we know we need the relation records (as a collection) properly sorted resolved there as well. For this to work, the new RecordFieldTransformer creates LazyCollection objects or RecordPropertyClosure objects (for a 1:1/n:1 relation) which means that the DB query is not made (yet) but only called when the value is accessed for the first time ("lazy loading"). Step 2: Getting the related UID/Table Pairs The RecordIdentityMap now knows about the 10 Records from tt_content, as they have been created completely before handing it to the output rendering. There comes the fun part. As soon as the value (with a lazy closure) is accessed for the first time, the RelationResolver checks the RelationHandler to find the table / uids that we should resolve. In our case, we now know that our first content element has 5 relations to a DB table e.g. "tx_mycarousel_item" with UIDs 12,13,14,15,16. Thanks to the RelationHandler, we also have the proper sorting of these items. Step 3: A greedy database query to get the full DB rows So, for the first content element, we want the 5 complete, related DB rows. The RelationResolver now sends this query to the "GreedyDatabaseBackend" which uses a subquery to not only fetch the 5 DB rows, but ALL rows of this DB table that are on the same PID with 1 DB query (using subselects). It however only returns the 5 items, and keeps the other items in a runtime cache. At this point we have made 3 DB queries. Currently, we then do the language + workspace overlays. Step 4: The long way back The RelationResolver now has the full DB rows and sorts them. The RecordFieldTransformer builds Record / Collection objects out of it, checks if an object has been created already (via the IdentityMap) or creates new ones, utilizing again the Lazy approach from step 1 to ensure we only resolve the records when we need them. Responsibilities: - RecordFieldTransformer - knows what to do based on the Field Type - returns objects, never raw DB records - initializes the lazy collections / closure objects - RelationResolver - uses RelationHandler to resolve uids - knows and applies the sorting - GreedyDatabaseBackend - does overlays / enableFields The design decisions behind this approach: - We only build Record objects when they are requested explicitly - We distinguish the cardinality (1:1 / n:1 vs. 1:n) - We do overlays on a very end of the chain Kudos to Nikita Hovratov for creating the first draft of this approach here: https://review.typo3.org/c/Packages/TYPO3.CMS/+/83725 along with comprehensive tests. PS: In the mid-term, the RelationResolver could be based on the sys_refindex and minimize queries. Resolves: #103581 Related: #103783 Related: #104002 Releases: main Change-Id: I73d1f017c5f98115f7ad4ddd2634b7acf66d183c Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/85046 Tested-by:Oliver Bartsch <bo@cedev.de> Reviewed-by:
Nikita Hovratov <nikita.h@live.de> Reviewed-by:
Oliver Bartsch <bo@cedev.de> Tested-by:
core-ci <typo3@b13.com> Tested-by:
Nikita Hovratov <nikita.h@live.de> Tested-by:
Benjamin Franzke <ben@bnf.dev> Reviewed-by:
Benjamin Franzke <ben@bnf.dev>
Showing
- typo3/sysext/backend/Classes/Preview/FluidBasedContentPreviewRenderer.php 4 additions, 1 deletion...kend/Classes/Preview/FluidBasedContentPreviewRenderer.php
- typo3/sysext/backend/Classes/Preview/StandardContentPreviewRenderer.php 103 additions, 57 deletions...ackend/Classes/Preview/StandardContentPreviewRenderer.php
- typo3/sysext/backend/Classes/Utility/BackendUtility.php 1 addition, 0 deletionstypo3/sysext/backend/Classes/Utility/BackendUtility.php
- typo3/sysext/backend/Classes/View/Drawing/BackendLayoutRenderer.php 1 addition, 1 deletion...xt/backend/Classes/View/Drawing/BackendLayoutRenderer.php
- typo3/sysext/core/Classes/Collection/LazyRecordCollection.php 98 additions, 0 deletions...3/sysext/core/Classes/Collection/LazyRecordCollection.php
- typo3/sysext/core/Classes/Configuration/Tca/TcaPreparation.php 41 additions, 6 deletions.../sysext/core/Classes/Configuration/Tca/TcaPreparation.php
- typo3/sysext/core/Classes/DataHandling/DataHandler.php 1 addition, 2 deletionstypo3/sysext/core/Classes/DataHandling/DataHandler.php
- typo3/sysext/core/Classes/DataHandling/RecordFieldTransformer.php 248 additions, 0 deletions...sext/core/Classes/DataHandling/RecordFieldTransformer.php
- typo3/sysext/core/Classes/DataHandling/RelationResolver.php 148 additions, 0 deletionstypo3/sysext/core/Classes/DataHandling/RelationResolver.php
- typo3/sysext/core/Classes/Database/ReferenceIndex.php 1 addition, 1 deletiontypo3/sysext/core/Classes/Database/ReferenceIndex.php
- typo3/sysext/core/Classes/Database/RelationHandler.php 1 addition, 1 deletiontypo3/sysext/core/Classes/Database/RelationHandler.php
- typo3/sysext/core/Classes/Domain/Persistence/GreedyDatabaseBackend.php 163 additions, 0 deletions...core/Classes/Domain/Persistence/GreedyDatabaseBackend.php
- typo3/sysext/core/Classes/Domain/Record.php 19 additions, 4 deletionstypo3/sysext/core/Classes/Domain/Record.php
- typo3/sysext/core/Classes/Domain/RecordFactory.php 94 additions, 23 deletionstypo3/sysext/core/Classes/Domain/RecordFactory.php
- typo3/sysext/core/Classes/Domain/RecordPropertyClosure.php 35 additions, 0 deletionstypo3/sysext/core/Classes/Domain/RecordPropertyClosure.php
- typo3/sysext/core/Classes/Resource/Collection/LazyFileReferenceCollection.php 96 additions, 0 deletions...asses/Resource/Collection/LazyFileReferenceCollection.php
- typo3/sysext/core/Classes/Resource/Collection/LazyFolderCollection.php 96 additions, 0 deletions...core/Classes/Resource/Collection/LazyFolderCollection.php
- typo3/sysext/core/Classes/Schema/Field/DateTimeFieldType.php 5 additions, 0 deletionstypo3/sysext/core/Classes/Schema/Field/DateTimeFieldType.php
- typo3/sysext/core/Classes/Schema/FieldTypeFactory.php 5 additions, 4 deletionstypo3/sysext/core/Classes/Schema/FieldTypeFactory.php
- typo3/sysext/core/Classes/Schema/FlexFormSchema.php 18 additions, 0 deletionstypo3/sysext/core/Classes/Schema/FlexFormSchema.php
Please register or sign in to comment