Skip to content
Snippets Groups Projects
Select Git revision
0 result
Search by author
  • Any Author
0 authors
  1. Aug 27, 2024
    • Benni Mack's avatar
      [FEATURE] Automatically resolve relations for records · 43a284ca
      Benni Mack authored 
      What does it do? When creating a record
object out of a database row (via the RecordFactory),
the Record object now holds enriched values
for fields where the content is "known" based
on TCA / Schema API.

This is especially relevant for records with
possible relations, for field types such as
"type=group", "type=select" with foreign_table set
and "type=inline" as an example.

The main purpose for this relation enriching
is rendering of the backend page module and
content in frontend, but it can be used
for any kind of TCA-based record.

It also works recursively and with circular
dependencies thanks to the RecordIdentityMap.

In order to avoid any problems with large amounts
of data, an approach of "Lazy+Greedy Fetching"
was chosen.

How does this approach work under the hood?

As an example, we load 10 content elements on a
page (1 DB query) so we can render them.

Step 1: Lazy Collections / RecordPropertyClosure

RecordFactory filters out only the relevant
fields and their values from a record's type.
Now, the RecordFactory also checks for fields
with their meaning and uses a different value for
a field. Example: For a type=inline value, there
was the number "5" as value (= 5 relations) available,
now we know we need the relation records (as a collection)
properly sorted resolved there as well. For this to
work, the new RecordFieldTransformer creates
LazyCollection objects or RecordPropertyClosure objects
(for a 1:1/n:1 relation) which means that the DB query is
not made (yet) but only called when the value is accessed
for the first time ("lazy loading").

Step 2: Getting the related UID/Table Pairs

The RecordIdentityMap now knows about the
10 Records from tt_content, as they have been created
completely before handing it to the output rendering.
There comes the fun part. As soon as the value (with a
lazy closure) is accessed for the first time,
the RelationResolver checks the RelationHandler
to find the table / uids that we should resolve.

In our case, we now know that our first content element
has 5 relations to a DB table e.g. "tx_mycarousel_item"
with UIDs 12,13,14,15,16. Thanks to the RelationHandler,
we also have the proper sorting of these items.

Step 3: A greedy database query to get the full DB rows

So, for the first content element, we want the 5
complete, related DB rows. The RelationResolver
now sends this query to the "GreedyDatabaseBackend"
which uses a subquery to not only fetch
the 5 DB rows, but ALL rows of this DB table
that are on the same PID with 1 DB query (using subselects).

It however only returns the 5 items, and keeps the
other items in a runtime cache.

At this point we have made 3 DB queries.

Currently, we then do the language + workspace overlays.

Step 4: The long way back

The RelationResolver now has the full DB rows and sorts
them. The RecordFieldTransformer builds Record / Collection
objects out of it, checks if an object has been created
already (via the IdentityMap) or creates new ones,
utilizing again the Lazy approach from step 1 to ensure
we only resolve the records when we need them.

Responsibilities:

- RecordFieldTransformer
  - knows what to do based on the Field Type
  - returns objects, never raw DB records
  - initializes the lazy collections / closure objects
- RelationResolver
  - uses RelationHandler to resolve uids
  - knows and applies the sorting
- GreedyDatabaseBackend
  - does overlays / enableFields

The design decisions behind this approach:
- We only build Record objects when they are requested explicitly
- We distinguish the cardinality (1:1 / n:1 vs. 1:n)
- We do overlays on a very end of the chain

Kudos to Nikita Hovratov for creating the first draft
of this approach here: https://review.typo3.org/c/Packages/TYPO3.CMS/+/83725
along with comprehensive tests.

PS: In the mid-term, the RelationResolver could be
based on the sys_refindex and minimize queries.

Resolves: #103581
Related: #103783
Related: #104002
Releases: main
Change-Id: I73d1f017c5f98115f7ad4ddd2634b7acf66d183c
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/85046


Tested-by: default avatarOliver Bartsch <bo@cedev.de>
Reviewed-by: default avatarNikita Hovratov <nikita.h@live.de>
Reviewed-by: default avatarOliver Bartsch <bo@cedev.de>
Tested-by: default avatarcore-ci <typo3@b13.com>
Tested-by: default avatarNikita Hovratov <nikita.h@live.de>
Tested-by: default avatarBenjamin Franzke <ben@bnf.dev>
Reviewed-by: default avatarBenjamin Franzke <ben@bnf.dev>
      43a284ca
  3. Aug 26, 2024
  5. Aug 25, 2024
  7. Aug 24, 2024
  9. Aug 23, 2024
  11. Aug 22, 2024
  13. Aug 21, 2024
  15. Aug 20, 2024