The method GeneralUtility::array_merge_recursive_overrule()
always works on a copy of the given array(s). This is highly
inefficient when it comes to really big arrays and recursion.

This patches moves the functionality into the class ArrayUtility
and changes the behaviour to use a reference to the
original array.
All calls in the core are adjusted accordingly.

Furthermore we deprecate the method in GeneralUtility and
preserve backward compatibility.

Resolves: #54251
Releases: 6.2
Change-Id: I5499905593c2124897de5998be985e546a3d05ee
Reviewed-on: https://review.typo3.org/25986
Reviewed-by: Michiel Roos
Tested-by: Michiel Roos
Reviewed-by: Wouter Wolters
Tested-by: Wouter Wolters
Reviewed-by: Ernesto Baschny
Tested-by: Ernesto Baschny

MySQL up to version 5.5 allows at maximum 64 characters for table- 
and fieldnames. Other DBMS allow less. Since MySQL is our primary
database and DBAL could handle specialties.

This patch changes the fields tablenames, fieldname and table_local
to varchar(64) which currently is considered save.

Resolves: #54726
Releases: 6.2
Change-Id: Ic67450a9326dceb413fe694a6233b8c092f70c79
Reviewed-on: https://review.typo3.org/26625
Reviewed-by: Steffen Ritter
Tested-by: Steffen Ritter

The backend and frontend fire several queries for referenced
files - files with metadata overlay as used in tt_content for
images and media field as well as in pages:media.
Analysing the queries for that some indexed should be added
to improve the query speed on large installations.

This patch adds indexed for the table/fieldname combination
as well es the uid of the record the file is attached to.

Resolves: #54725
Releases: 6.2
Change-Id: I229e27e382e8d0fdaf9bbc0b6729f90dac019e19
Reviewed-on: https://review.typo3.org/26626
Reviewed-by: Steffen Ritter
Tested-by: Steffen Ritter

Change-Id: I1934a6201876290b706078519e934e05c28352ab
Resolves: #54053
Releases: 6.2
Reviewed-on: https://review.typo3.org/25740
Reviewed-by: Markus Klein
Tested-by: Markus Klein
Reviewed-by: Jo Hasenau
Reviewed-by: Michiel Roos
Tested-by: Michiel Roos
Reviewed-by: Wouter Wolters
Tested-by: Wouter Wolters

Fix superfluous comparison against boolean in
DatabaseRecordList::getTable

Change-Id: Icdbf78935da4135a7c6b4902b97c87e7ce658eb8
Resolves: #54049
Releases: 6.2
Reviewed-on: https://review.typo3.org/25736
Reviewed-by: Markus Klein
Tested-by: Markus Klein
Reviewed-by: Wouter Wolters
Tested-by: Wouter Wolters

Two Core tables (fe_users and index_config) still use the
"fe_admin_fieldList" property, which has no effect on the
Core since the fe_adminLib.inc was moved out of it
(into the statictemplates extension).

Additionally a deprecation message is logged in method
ExtensionManagementUtility::addTCAcolumns() when the $addTofeInterface
argument is set to TRUE.

Resolves: #54613
Releases: 6.2
Change-Id: I7c9961c5e92411a78bd355519a0493eb695177e7
Reviewed-on: https://review.typo3.org/26567
Reviewed-by: Markus Klein
Tested-by: Markus Klein
Reviewed-by: Oliver Klee
Reviewed-by: Wouter Wolters
Tested-by: Wouter Wolters
Reviewed-by: Francois Suter
Tested-by: Francois Suter

In the IndexedSearchRepository on line 157 an undefined/unused 
property $this->resultsPerPage is used for the calculation 
of the current page the visitor is showing on search results.

Instead of using $this->resultsPerPage we have to use 
$this->numberOfResults which is defined and contains the expected setting.

Change-Id: I37c3a08c1049eb6166704b2b98ba071f03aad243
Resolves: #54808
Releases: 6.2, 6.1
Reviewed-on: https://review.typo3.org/26677
Reviewed-by: Anja Leichsenring
Tested-by: Anja Leichsenring
Reviewed-by: Markus Klein
Tested-by: Markus Klein

The PHP-Doc comments for the render-method in PageBrowsing 
Viewhelper on Indexed Search define a $details argument 
for that method with is then not part of the method arguments 
itself.

As it seems to be unused the argument definition should simply 
be removed from doc comments to dont mess up the defintions 
of the other arguments.

Right now this results in maximumNumberOfResultPages be defined 
as array and causes a fatal error like:
The argument "maximumNumberOfResultPages" was registered with 
type "array", but is of type "string" in view helper 
"TYPO3\CMS\IndexedSearch\ViewHelpers\PageBrowsingViewHelper"

Resolves: #54807
Releases: 6.2, 6.1
Change-Id: I58376880ef64a1e1088561fbdb3ff25f9c1fbdc2
Reviewed-on: https://review.typo3.org/26676
Reviewed-by: Anja Leichsenring
Tested-by: Anja Leichsenring

The method packageRequirementIsComposerPackage of
TYPO3\CMS\Core\Package ignores all non composer package
names and thus dependency resolving of non composer packages
doesn't work.

This patch adapts the behavior to ignore just the platform
package links described here
http://getcomposer.org/doc/02-libraries.md#platform-packages

Fixes: #54142
Releases: 6.2
Change-Id: Idb6a837eebf792c6cbf9394885edc17c9d392a9f
Reviewed-on: https://review.typo3.org/26619
Reviewed-by: Stefan Froemken
Tested-by: Stefan Froemken
Reviewed-by: Anja Leichsenring
Tested-by: Anja Leichsenring
Reviewed-by: Markus Klein
Tested-by: Markus Klein
Reviewed-by: Oliver Hader
Tested-by: Oliver Hader

Section "TYPO3 Security" refers to the TYPO3 Security Cookbook, which is
obsolete and has been replaced by the official TYPO3 Security Guide in
December 2011. This patch removes the reference to the Cookbook and
replaces it by a link to the latest version of the Security Guide on
docs.typo3.org.

Resolves: #54756
Releases: 6.2
Change-Id: I4067d40e854e7d4b4f77e2c12239e3e00616f5a7
Reviewed-on: https://review.typo3.org/26656
Reviewed-by: Michiel Roos
Tested-by: Michiel Roos
Reviewed-by: Wouter Wolters
Tested-by: Wouter Wolters

ReflectionService::convertParameterReflectionToArray() checks
functions already returning boolean again for boolean value.

Remove this overhead and improve readability.

Change-Id: I113f16cd73c713da5ec75b0e57eeee1d0e17308c
Resolves: #54021
Releases: 6.2
Reviewed-on: https://review.typo3.org/25717
Reviewed-by: Markus Klein
Tested-by: Markus Klein

Datahandler functional tests lack proper initialization of the
BE_USER mock used.

Set a 'uid' for the mock in order to get tests working again.

Resolves: #54741
Releases: 6.2
Change-Id: I20cccf8d3622b2c63d6b7c9f07c0e2d238bea7b9
Reviewed-on: https://review.typo3.org/26642
Reviewed-by: Oliver Hader
Reviewed-by: Markus Klein
Tested-by: Markus Klein

Currently only UPDATE queries pass the $allowNull parameter to the
fullQuoteStr() method in the DatabaseHandler. To make the behavior of
both methods consistent and to allow NULL values during creation of
new records by TCEmain the $allowNull parameter is also set to TRUE for
INSERT queries.

Resolves: #53662
Releases: 6.2, 6.1, 6.0
Change-Id: I066b9880a557b6c9058fc15f467631f1313300f9
Reviewed-on: https://review.typo3.org/25422
Reviewed-by: Philipp Gampe
Reviewed-by: Frans Saris
Tested-by: Frans Saris
Reviewed-by: Xavier Perseguers
Reviewed-by: Markus Klein
Tested-by: Markus Klein

When an extension (like rsaauth) has configuration options in the
extension manager, the description is not shown as a tooltip when hovering
over the extension name. Then it will only show 'Configure'. Which is not
so helpful.

The ConfigureExtensionViewHelper can be modified to not add a title
attribute on the link. This will ensure that the title attribute of the td
is shown.

Change-Id: Ie6de8033545856cbe4a3689366e05d5d2ad3dc7b
Resolves: #54689
Releases: 6.2
Reviewed-on: https://review.typo3.org/26615
Reviewed-by: Markus Klein
Tested-by: Markus Klein
Reviewed-by: Xavier Perseguers
Tested-by: Xavier Perseguers

\TYPO3\CMS\Frontend\ContentObject\ContentObjectRenderer::joinTSarrays is
replaced by the PHP native array_replace_recursive() (added in PHP 5.3.0)
which does exactly the same and is twice as fast.

Usage of $cObj->joinTSarrays() is deprecated.

Change-Id: I748270a192bccc89927e0d6b82a1f405959b2209
Resolves: #54520
Releases: 6.2
Reviewed-on: https://review.typo3.org/26500
Reviewed-by: Ernesto Baschny
Tested-by: Ernesto Baschny

GeneralUtility::isValidUrl() idna converts whole URI instead of
domain only.

The expensive idna_convert() is called from isValidUrl(). Instead of
feeding it just the domain part, the whole URI is converted.

When supplying just the domain part, a great speed gain can be seen.

On seriously malformed URLs, parse_url() may return FALSE and emits an
E_WARNING. So we check for that first.

PHP no longer supports the flags FILTER_FLAG_HOST_REQUIRED and
FILTER_FLAG_SCHEME_REQUIRED. A scheme is now required by default. [1]
Return FALSE if the URL does not start with a scheme.

A public GeneralUtility::idnaEncode() method uses a static idna_convert
instance and fetches converted strings from a string cache array
to avoid multiple checks on the same domain.

All manual idna_convert instances are replaced with
GeneralUtility::idnaEncode() calls.

Special characters are not allowed in the URL except in the domain
part [2]. So the test with special characters in the path was removed
from the GeneralUtilityTest class.

[1] http://www.php.net/manual/en/filter.filters.flags.php#107382
[2] http://tools.ietf.org/html/rfc3986#appendix-A

Change-Id: I7a0ab0a399d9d6cf68c824f413be6b6d621947c1
Resolves: #53862
Releases: 6.2, 6.1, 6.0
Reviewed-on: https://review.typo3.org/25636
Reviewed-by: Markus Klein
Reviewed-by: Wouter Wolters
Tested-by: Wouter Wolters
Reviewed-by: Michiel Roos
Tested-by: Michiel Roos
Tested-by: Markus Klein
Reviewed-by: Andreas Wolf
Reviewed-by: Jo Hasenau
Reviewed-by: Xavier Perseguers
Tested-by: Xavier Perseguers

Folder::getFiles implements the logic of creating file objects
itself, after retrieving the information from the driver.
Besides the fact that this is slow since all information for the
object are received from the filesystem directly instead of the
cache in the sys_file table the uid is not present in these
objects which finally results in the lack of metadata in these
file objects.

In addition to that ommiting the ResourceFactory several objects
for the same file might exists which may lead to inconsistent
behaviour and output on modifying the file.

As the Folder/File Objects only should be a convinience facade
in front of the ResourceStorage this change introduces a new
method their, implementing the new and improved logic.

At the same time the old functionality - which enforces manual
file object creation - has been deprecated and the filelist
module is adapted accordingly.

Releases: 6.2
Resolves: #53688
Change-Id: I3fb97d432d325bd...

The method FileRepository::getFileIndexRecordsForFolder
retrieves the Index Records for files in a given folder using a
LIKE-query. This basically checks if the identifier of the a
folder is part of the identifier of the file.

This concept has several drawbacks. First not all storages are
hierarchical, second this will fail if there is a different
directory seperator than / and finally this fetches records
for the folder recursively which may result in a huge amount of
data if for example the root folder of an storage is queried.

In addition the method resides in the FileRepository while it
retrieves IndexRecords - which is a different concern.

This change introduces the function within the
FileIndexRepository and uses the folder_hash (introduced in
https://review.typo3.org/23398) to query for the files.

The old method is deprecated now, calls the new method and
all core-usages of the old function have been rewritten to
use the new functionality.

Releases: 6.2
Resolves: #53687
Change-Id: Icabf2350192c93951f2277f3aa7468fe958bfabf
Reviewed-on: https://review.typo3.org/26357
Reviewed-by: Frans Saris
Tested-by: Frans Saris
Reviewed-by: Steffen Ritter
Tested-by: Steffen Ritter

Fix incorrect message for install tool login attempt warning.

Resolves: #54531
Releases: 6.2, 6.1
Change-Id: I40f34b85ffc554187baf1f2601c578cbaafc5718
Reviewed-on: https://review.typo3.org/26509
Reviewed-by: Markus Klein
Tested-by: Markus Klein

Rework the documentation module according to the
discussion with the TYPO3 Usability Team.

Resolves: #54067
Releases: 6.2
Change-Id: Ia59c9a1554080b71412b467f6621bc7270a81c72
Reviewed-on: https://review.typo3.org/25975
Reviewed-by: Markus Klein
Tested-by: Markus Klein
Reviewed-by: Xavier Perseguers
Tested-by: Xavier Perseguers

Change-Id: Ia6a172bcc88924db714005165d92added4bb4a1a
Reviewed-on: https://review.typo3.org/26493
Reviewed-by: TYPO3 Release Team
Tested-by: TYPO3 Release Team

Change-Id: I59ca5042eda854c6d3daf02b54c23bb3acd615ff
Reviewed-on: https://review.typo3.org/26492
Reviewed-by: TYPO3 Release Team
Tested-by: TYPO3 Release Team

Since the travis ci service changed the notification
policy to not notify authors of commits on failed
builds, we need to set up explicit notifications
so that failing builds do not stay undiscovered.

A first step will be to sent a notification to
the irc channel and the core list.

Change-Id: I6f5953f65aab04c526eae8aa34890e39169ab60f
Reviewed-on: https://review.typo3.org/26491
Tested-by: Ernesto Baschny
Reviewed-by: Helmut Hummel
Tested-by: Helmut Hummel

The following changes to system categories should be mentioned
in NEWS.md:

* activation by default on pages and tt_content tables
* new menu types for categories-based menus

Resolves: #54368
Releases: 6.2
Change-Id: I1ca95ea6354f52a68bd31cf284d6a103563190a4
Reviewed-on: https://review.typo3.org/26362
Reviewed-by: Helmut Hummel
Tested-by: Helmut Hummel

This reverts commit 2d6e8dea
This merge broke travis unit and functional tests.
Needs some work on these components to get in again.

[ci skip]

Change-Id: Ifa96df8c60472def99707431793de291043d23e4
Reviewed-on: https://review.typo3.org/26488
Reviewed-by: Helmut Hummel
Tested-by: Helmut Hummel

New fields were added to the tt_content table to be used when creating
categories-based menus (of pages or content elements).
Those fields are currently missing CSH.

Resolves: #54366
Releases: 6.2
Change-Id: I0fea98a7c3ecd29ffca6559314bd3f790845e3e7
Reviewed-on: https://review.typo3.org/26361
Reviewed-by: Francois Suter
Tested-by: Francois Suter

During installation of extensions the Extension Manager does
not take the Category API into account. The code to do so is
present in the Install Tool in the Database Compare Tool. It is
cumbersome to switch to the install tool to update the database
in order to use the category fields. The install tool and extension
manager currently need to know which components manipulate
the table definitions and this is bad coupling of components
which shouldn't know each other.

This fix replaces the individual calls to the components by two
signals and thus a generic approach.

Fixes: #53016
Releases: 6.2
Change-Id: Ibaea293b96fb1b8df1eacdcdd2f98acf74fb155b
Reviewed-on: https://review.typo3.org/24942
Reviewed-by: Stefan Neufeind
Tested-by: Stefan Neufeind
Reviewed-by: Fabien Udriot
Tested-by: Fabien Udriot
Reviewed-by: Thomas Maroschik
Tested-by: Thomas Maroschik

This is a follow up to 7efcf2a4 which marks the newly
introduced method as internal.
https://review.typo3.org/25851

Resolves: #54126
Releases: 6.2
Change-Id: Iaffa3cf96e7e43487f4249a70460f64fc3a066ea
Reviewed-on: https://review.typo3.org/26486
Reviewed-by: Markus Klein
Tested-by: Markus Klein

Where onclick and oncontextmenu behave the same avoid duplicating
JavaScript-code and simply call the click()-functionality.

Change-Id: Iaa0b96fd311ea1a8367ef474b483e6c92bb1bcff
Resolves: #54288
Releases: 6.2
Reviewed-on: https://review.typo3.org/26094
Reviewed-by: Frans Saris
Tested-by: Frans Saris
Reviewed-by: Markus Klein
Tested-by: Markus Klein

Avoid having to use the substr/strlen-magic.
Also strlen(PATH_site) can be statically cached.

Change-Id: I0ef942e331e2039e2ece9a55dd740db2a3896e2c
Resolves: #54126
Releases: 6.2
Reviewed-on: https://review.typo3.org/25851
Reviewed-by: Philipp Gampe
Tested-by: Philipp Gampe
Reviewed-by: Markus Klein
Tested-by: Markus Klein

Since #53655, reviewed at https://review.typo3.org/25481 the indexer
takes care of updating file objects and index records if a file or
folder is moved.

The unit tests have not been adapted to that change accordingly.
This changeset fixes the tests for LocalDriver and ResourceStorage.

Change-Id: Id17b01b8e47dd63750d1c11c2d4e24313b053695
Resolves: #54499
Releases: 6.2
Reviewed-on: https://review.typo3.org/26477
Reviewed-by: Stefan Neufeind
Tested-by: Stefan Neufeind
Reviewed-by: Markus Klein
Tested-by: Markus Klein

Under certain circumstances the Failsafe Package Manager
could interfer with the Update Package Manager in the Install Tool.

In order to reduce this friction the migration logic is integrated
into the migration step.

Resolves: #53886
Releases: 6.2
Change-Id: I0300b9c74736262b03f9f9b59a49576b7edf5b2f
Reviewed-on: https://review.typo3.org/25648
Reviewed-by: Markus Klein
Reviewed-by: Thomas Maroschik
Reviewed-by: Stefan Neufeind
Tested-by: Stefan Neufeind

Due to the nature of the Flow Package Manager, packages cannot
be activated and directly used during runtime. Before the Package
Manager it was possible to activate/deactivate extensions in
AdditionalConfiguration.php under certain custom conditions.

This patch introduces a new setting in $GLOBALS['TYPO3_CONF_VARS']
['EXT']['runtimeActivatedPackages'] = array('{packageKey}') that
gets initialized right after the package management initialization.

Resolves: #53015
Releases: 6.2
Change-Id: Id3b85a3feb00876d2a04a02e85450a4568eb5bff
Reviewed-on: https://review.typo3.org/24939
Reviewed-by: Thomas Maroschik
Reviewed-by: Stefan Neufeind
Tested-by: Stefan Neufeind
Tested-by: Alexander Stehlik
Tested-by: Frans Saris
Reviewed-by: Stefan Froemken
Tested-by: Stefan Froemken
Reviewed-by: Markus Klein
Tested-by: Markus Klein

The processChangedAndNewFiles() method in the Indexer class now uses
the $fileIndexEntry variable for retrieving a file object from the
resource factory instead of the invalid $data variable which is NULL.

Resolves: #54312
Releases: 6.2
Change-Id: I3b9c2ce99f7b4b7c575cc4055c02912c306ed789
Reviewed-on: https://review.typo3.org/26339
Reviewed-by: Steffen Ritter
Tested-by: Steffen Ritter
Reviewed-by: Markus Klein
Tested-by: Markus Klein

The system extension filemetadata adds access restriction fields
for selecting frontend user groups as known from tt_content or
pages. Behind the scenes of the TCEform this relation is stored
in a MM table which is incompatible to the usual access checks.

In addition a opposite relation is added to fe_groups which
queries all files to show in a selector.

This change removes the MM table and reconfigures the fe_groups
field in the same way as it is done for tt_content and pages.

Resolves: #54236
Resolves: #54237
Releases: 6.2
Change-Id: I8a05073dee9e57e48335e1fe2a3917313563ac7d
Reviewed-on: https://review.typo3.org/25987
Reviewed-by: Markus Klein
Reviewed-by: Stefan Froemken
Tested-by: Stefan Froemken
Reviewed-by: Frans Saris
Reviewed-by: Steffen Ritter
Tested-by: Steffen Ritter

The ResourceStorage does not properly make use of the Indexer.
As result the indexRecord is not properly updated after a
file change.

This patch cleans up the ResourceStorage so it doesn't update
the index properties itself but leaves that to the indexer.

Resolves: #53655
Releases: 6.2
Change-Id: I249505a1bc0b93f8b3ffb0e9cb2b7f10a9a9968e
Reviewed-on: https://review.typo3.org/25481
Reviewed-by: Stefan Neufeind
Reviewed-by: Steffen Ritter
Tested-by: Steffen Ritter

ElementBrowser calls Folder::getFiles() with wrong parameters.
Properly implement the file extensions filter.

Resolves: #51752
Releases: 6.2, 6.1, 6.0
Change-Id: I56468c79225e2d3baa5e5784571074532e2287ad
Reviewed-on: https://review.typo3.org/25359
Reviewed-by: Steffen Ritter
Tested-by: Steffen Ritter

If an editor has got no file mounts, an uncaught exception
is shown in the element browser.

Fix this by checking if there is a selected folder at all.

Resolves: #52969
Releases: 6.2
Change-Id: I5f9e8cc7994edd69f6db6ae1cc647ee31e4930c6
Reviewed-on: https://review.typo3.org/25357
Reviewed-by: Stefan Neufeind
Reviewed-by: Steffen Ritter
Tested-by: Steffen Ritter

A regular expression in FrontendLoginController
contains an unknown modifier. Fix it by replacing the
/ to # at the beginning and the end of the regular
expression.

Change-Id: Id4d3439c1cdbec691d977570bf76ba0c7bad493c
Resolves: #52059
Releases: 6.2, 6.1, 6.0
Reviewed-on: https://review.typo3.org/23881
Reviewed-by: Markus Klein
Tested-by: Markus Klein
Reviewed-by: Stefan Neufeind
Tested-by: Stefan Neufeind

This fixes a wrong parsing of \r\n characters for radio
button options.

Resolves: #53727
Releases: 6.2, 6.1, 6.0
Change-Id: I9a88be010a7dd982776bee4a98ba99d97fcc406b
Reviewed-on: https://review.typo3.org/25482
Reviewed-by: Stefan Neufeind
Tested-by: Stefan Neufeind