From e48050ef217246f03fc47823dc9b19866ba73c12 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?R=C3=A9my=20DANIEL?= <dogawaf@no-log.org>
Date: Sun, 6 Aug 2023 11:52:19 +0200
Subject: [PATCH] [DOCS] Document known issues with ext:linkvalidator
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Linkvalidator extension has a couple of known issues,
especially with external link checking.
Those issues are now documented, so integrators can
acknowledge them and implement counter-measures.
Resolves: #101597
Releases: main, 12.4, 11.5
Change-Id: I00dc01243426fc56c21a4bd11815c3009e3dd3ad
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/80364
Tested-by: core-ci <typo3@b13.com>
Tested-by: Stefan B�rk <stefan@buerk.tech>
Reviewed-by: Stefan B�rk <stefan@buerk.tech>
---
.../Documentation/Configuration/Index.rst | 5 +-
.../linkvalidator/Documentation/Index.rst | 1 +
.../Documentation/Introduction/Index.rst | 2 +-
.../Documentation/KnownProblems/Index.rst | 49 +++++++++++++++++++
4 files changed, 55 insertions(+), 2 deletions(-)
create mode 100644 typo3/sysext/linkvalidator/Documentation/KnownProblems/Index.rst
diff --git a/typo3/sysext/linkvalidator/Documentation/Configuration/Index.rst b/typo3/sysext/linkvalidator/Documentation/Configuration/Index.rst
index 2f9d8fa0ce94..7e23dfd40fed 100644
--- a/typo3/sysext/linkvalidator/Documentation/Configuration/Index.rst
+++ b/typo3/sysext/linkvalidator/Documentation/Configuration/Index.rst
@@ -120,11 +120,14 @@ linktypes
file: Check links to files located in your local TYPO3 installation.
- external: Check links to external files.
+ external: Check links to external URLs.
This list may be extended by other extensions providing a
:ref:`custom linktype implementation <linktype-implementation>`.
+ .. warning::
+ External links can lead to some :ref:`known issues<usagePitfallsExternalLinks>`.
+
Default
db,file,external
diff --git a/typo3/sysext/linkvalidator/Documentation/Index.rst b/typo3/sysext/linkvalidator/Documentation/Index.rst
index 1391072c93e4..079ac53ebaf6 100644
--- a/typo3/sysext/linkvalidator/Documentation/Index.rst
+++ b/typo3/sysext/linkvalidator/Documentation/Index.rst
@@ -49,6 +49,7 @@ send status mails when broken links are detected.
Configuration/Index
Development/Index
LargeSites/Index
+ KnownProblems/Index
.. Meta Menu
diff --git a/typo3/sysext/linkvalidator/Documentation/Introduction/Index.rst b/typo3/sysext/linkvalidator/Documentation/Introduction/Index.rst
index a67ff19e8159..e5e9752b3b4c 100644
--- a/typo3/sysext/linkvalidator/Documentation/Introduction/Index.rst
+++ b/typo3/sysext/linkvalidator/Documentation/Introduction/Index.rst
@@ -32,7 +32,7 @@ It includes the following features:
- The LinkValidator can check all kinds of links. This includes internal
links to pages and content elements, file links to files in the local
- file system and external links to files somewhere else in the web.
+ file system and external links to resources somewhere else in the web.
- The LinkValidator checks a number of fields by default, for example
:sql:`header` and :sql:`bodytext` fields of content elements.
diff --git a/typo3/sysext/linkvalidator/Documentation/KnownProblems/Index.rst b/typo3/sysext/linkvalidator/Documentation/KnownProblems/Index.rst
new file mode 100644
index 000000000000..73637ad1eaad
--- /dev/null
+++ b/typo3/sysext/linkvalidator/Documentation/KnownProblems/Index.rst
@@ -0,0 +1,49 @@
+.. include:: /Includes.rst.txt
+
+.. _known-problems:
+
+==============
+Known problems
+==============
+
+
+.. _usagePitfallsExternalLinks:
+
+Problems with external links
+============================
+
+The most relevant known problems currently concern "external broken links".
+
+Be polite when crawling external sites: They should not be checked too often.
+The extension currently provides no functionality to limit such requests.
+
+There are (at least) 2 possible counter-measures:
+
+#. Turn off external link checking entirely
+ by removing `external` from Page TSconfig :ref:`linktypes`
+
+#. :ref:`Override the ExternalLinktype class <linktype-implementation>`
+ (in your own extension), to check only specific URLs or exclude specific
+ URLs or handle only specific error types as errors.
+
+
+.. _usagePitfallsFalsePositives:
+
+Falsely reported broken links
+=============================
+
+Linkvalidator may find false negatives when checking links. This can be
+annoying for editors since there is no way to declare them "ok" manually,
+they will be shown to editors over and over again.
+
+There are a couple of scenarios where linkvalidator may show a link as
+broken leading to a misleading result:
+
+* Automatic server-side external link checking may fail. There are many possible
+ reasons for false negatives in this area like server connectivity issues or
+ funny deny rules on the target system.
+* Sometimes links may not be broken as such, but are considered "broken" due to
+ HTTP status like 4xx.
+* The broken link information may be "stale", the link has been checked a while ago
+ and did lead to a negative result due to temporary a system outage or similar, is
+ now working again, but has not been rechecked yet.
--
GitLab