LTS Changelog

Cache: Image cache has been refactored, utilizing now simpler structure with less folder depth.

Core: A potential data loss on an edge case of node migration over several schema versions, including field name/type change, has been fixed.

Documentation: A missing query parameters documentation for the /search and /{{project}}/search has been added.

Core: A crash, caused by duplicated nodes in a GraphQL resulting set, has been fixed.

Core: The Vert.x Handlebars engine dependency has been patched, to replace the vulnerable Apache commons-text dependency, targeting CVE-2022-42889.

Core: A crash has been fixed on an attempt of (micro)node migration over non-adjacent (micro)schema versions.

Documentation: The check/repair API calls are better documented in regards of an async processing. The logging documentation is extended with format examples.

Elasticsearch: An Elasticsearch Java Client has been updated to the version 1.1.2, containing the corrections to the connection failure processing mechanism. The Elasticsearch activity idle checking has been improved to be more proof of the unstable connections.

Core: Improvement of an error message of the webroot field URL conflict, now containing correct UUID, language and webroot URLS of a conflicting content.

REST client: The configuration now allows adding default parameter provider for query parameters, that should be added to every request (unless overwritten).

Tests: The tests API has been extended to allow tests which do not reset the database between test cases.

GraphQL: In cases of long running GraphQL requests, some other GraphQL requests were queued and executed after the long running request, even if enough workers were still available. The behaviour has been changed, so that GraphQL requests will only be queued, if all workers are currently busy.

Cache: The permission cache has been fixed to always return copies of the actually cached entries.

A regression regarding OAuth user filtering has been fixed.

Minor improvements in authentication-related utilities Java API, allowing more control over authentication URL management.

Cache: A mechanism of reattaching the cached entity to the persistence context has been introduced, allowing implementations of smarter cache policies.

Cache: Because of a flawed cache invalidation strategy, the project stayed on an old 'latest' branch, even if a new one has been assigned, until the caches are invalidated, or a restart is triggered. This has been fixed.

Core: Creating a translation for the root node of a project always failed with a "Bad Request" error, which has been fixed.

Elasticsearch: A full access to the data backed by ES has been given to the admin users.

REST client: Improves error handling for WebSocket errors.

GraphQL: When an inexisting language has been requested for a node, the schema info was not loaded as well. This has been fixed.

Elasticsearch: A possibility of DDoSing the ES with a bulk request of too many items has been eliminated.

Auth: Synchronization of users, groups and roles from AuthServicePlugin implementations (like the keycloak plugin) has been improved for performance and stability.

REST: The endpoints for invoking a consistency check/repair now have a new query parameter async which allows to perform the action as a background job instead of synchronously.

Core: Updating and publishing a node in a single request has been optimized to only create a single new version of the content.

GraphQL: Fetching of micronode fields has been improved to allow batch loading.

REST: The documentation of the generic parameter fields has been fixed. Now fields works over the Language entities as well, the values are uuid,name,languageTag,nativeName.

GraphQL: Some of (micro)schema fields related queries rely on the target (micro)schema having at least one field, crashing in HTTP 500 otherwise. This has now been fixed.

Core: Unnecessary uniqueness checks for segment fields will now be avoided, when updating nodes.

Auth: The unnecessary logging of outdated/mismatched auth token has been removed.

GraphQL: The overall performance of GraphQL requests has been improved by caching the GraphQL schemas.

Core: Now it is not allowed to set a new password to an empty or invalid (e.g. spaces) string.

Core: Improve the internal Java API for granting and revoking permissions.

Core: A regression of creation of the named entities has been fixed.

Core: Improve the internal Java API of the list field creation, targeting better batch performance.

Core: Now caches treat NULL as a valid value, instead of a constant miss.

Core: The cache internal Java API has been presented a new constructor, allowing using the weight instead of entity number for the cache size calculations, as well as some new statistics getters.

Tests: Added set of tests of uniqueness of the named entities on a database level.

GraphQL: referencedBy field fetcher has been refactored for more optimal batch loading.

GraphQL: Performance optimizations of the Elasticsearch-based requests.

Core: Requests to branches failed with an internal error, if monitoring was disabled. This has been fixed.

Core: The new configuration option vertxOptions.orderedBlockingHandlers allows to change the behaviour of concurrent handling of "long running" requests (requests that modify data or GraphQL requests). If the option is set to true (default value) such requests will be run in order (per http verticle) and never concurrently. As a consequence, it may happen that e.g. GraphQL requests are queued for some time if some other requests are currently handled. If the option is set to false, blocking requests will only be queued, if all threads of the worker pool are currently busy.

GraphQL: Parent loading has been speeded up by using a batch data loading.

Search: Minor logging changes in the search engine, making it less annoying.

Core: More NPE occurrences during the massive concurrent publishing process have been fixes.

Core: When running in the massive concurrent publishing process, it is possible to run into a race condition when some field containers are already processed while being referenced by the edge, throwing an NPE. This has now been fixed.

Clustering: Calling cluster specific REST Endpoints on non-clustered instances could cause internal server errors. The behaviour has been changed so that a "Bad Request" error is returned containing a proper error message.

Java Rest Client: After logging in with the login() method, the login token was never refreshed, which caused it to expire after the configured token expiration time (per default 1 hour), even if the client was used to do requests. This has been changed now, so that the login token in the client will be refreshed on every request to mesh.

GraphQL: The metadata request does not crash anymore on the empty (micro)schemas. The new related filter, isEmpty, has been added as well.

GraphQL: Link resolving in graphql queries ignored the language tags in the links. This has been fixed.

Core: All named instances have been presented an own cache.

Plugins: Logging fixed upon false triggering of a warning of inexisting role/group connection.

Core: Uniqueness checks for webroot url field values will now only be done, if those values actually change. This will improve performance of e.g. schema migrations, where the webroot url field values are likely to not change.

Core: An internal API for efficient loading of list field values has been added.

Clustering: Due to an incorrect check, it was possible that migration jobs could complete without actually migrating anything, when run in a clustered environment with disabled coordination mode.

GraphQL: Link resolving in graphql queries could cause an internal error (due to a DuplicateKeyException), which has been fixed now.

GraphQL: Performance of queries that return large amounts of nodes with link resolving in fields or getting node fields has been improved.

Core: The name of a Mesh user is now forced to be unique on a database level, to prevent creating users with an already existing username. The duplicated username detection mechanism has also been improved.

Monitoring: Calling the /health/ready endpoint while restoring an OrientDB backup would block the request. If too many requests were blocked in this way, Mesh would be considered not live any more. This has been fixed now, during a restore, the endpoint /health/ready will now fail immediately while the liveness status will be maintained.

Monitoring: Failing calls to any /health/…​ endpoints will no longer log the whole stacktrace, since this does not contain useful information.

Core: Migration jobs have been made more robust: Migration jobs will now be aborted, in cases were the storage is no longer ready to be written to (e.g. write quorum is not reached, or the storage is read-only due to insufficient disk space available). A periodic check (interval can be configured via setting migrationTriggerInterval) will continue processing of aborted jobs.

Clustering: when running in CUD coordination mode, the requests to trigger job migration were not delegated to the current master. This has been fixed now.

Core: If a node has more that one INITIAL content for a language in a branch, this can now be repaired with the consistency repair tool.

Docker: Additional command line tools have been added to the Mesh Docker image: rsync, curl, iputils, busybox-extras, tar, nano.

Search: The search endpoints (like e.g. /api/v1/search/nodes) were incorrectly treated as not "read-only", which caused them to fail during a cluster topology change, if the setting cluster.topologyChangeReadOnly was set to true. This has been changed now, the real search endpoints are read-only now. It is important to note, that this does still not apply to the index maintenance endpoints /api/v1/search/sync, /api/v1/search/clear and /api/v1/search/status.

GraphQL: In rare cases, GraphQL statements could "hang" forever, which caused the corresponding worker thread to be blocked forever. This has been fixed now by introducing a configurable timeout.

GraphQL: The graphql library has been updated to version 20.0.

Core: The node migration process has been improved to reduce resources footprint on large workloads, by chunking the data. An additional configuration item content.batchSize has been added.

Core: Corner case of updating the webroot info might throw a false conflict exception, when the segment field value is reset for a schema. This has been fixed.

Core: The name of a Mesh user is now forced to be unique on a database level, to prevent creating users with an already existing username. The duplicated username detection mechanism has also been improved.

Monitoring: Calling the /health/ready endpoint while restoring an OrientDB backup would block the request. If too many requests were blocked in this way, Mesh would be considered not live any more. This has been fixed now, during a restore, the endpoint /health/ready will now fail immediately while the liveness status will be maintained.

Monitoring: Failing calls to any /health/…​ endpoints will no longer log the whole stacktrace, since this does not contain useful information.

Core: Migration jobs have been made more robust: Migration jobs will now be aborted, in cases were the storage is no longer ready to be written to (e.g. write quorum is not reached, or the storage is read-only due to insufficient disk space available). A periodic check (interval can be configured via setting migrationTriggerInterval) will continue processing of aborted jobs.

Clustering: when running in CUD coordination mode, the requests to trigger job migration were not delegated to the current master. This has been fixed now.

Core: If a node has more that one INITIAL content for a language in a branch, this can now be repaired with the consistency repair tool.

Docker: Additional command line tools have been added to the Mesh Docker image: rsync, curl, iputils, busybox-extras, tar, nano.

Search: The search endpoints (like e.g. /api/v1/search/nodes) were incorrectly treated as not "read-only", which caused them to fail during a cluster topology change, if the setting cluster.topologyChangeReadOnly was set to true. This has been changed now, the real search endpoints are read-only now. It is important to note, that this does still not apply to the index maintenance endpoints /api/v1/search/sync, /api/v1/search/clear and /api/v1/search/status.

GraphQL: In rare cases, GraphQL statements could "hang" forever, which caused the corresponding worker thread to be blocked forever. This has been fixed now by introducing a configurable timeout.

GraphQL: The graphql library has been updated to version 20.0.

Core: The node migration process has been improved to reduce resources footprint on large workloads, by chunking the data. An additional configuration item content.batchSize has been added.

Core: Corner case of updating the webroot info might throw a false conflict exception, when the segment field value is reset for a schema. This has been fixed.

Clustering: Liveness check now marks an instance as dead, if the event bus did not get its own cluster instance ping within the timeout period, even if the non-cluster ping comes in time.

Search: When using language specific index settings, nodes were incorrectly indexed in the language unspecific index when they were moved or tagged causing the node to be present in both the language specific and language unspecific index. This has been fixed.

Core: Having a binary non-segment field update might break the uniqueness of the segment field value, allowing creation of multiple nodes with the same segment/webroot value. This has now been fixed.

Logging: Remove superfluous error messages for HTTP status 4xx errors. These are logged by the logger handler as warnings.

Docker-Image: The base images have been updated to a newer JDK minor version (11.0.16).

Core: On Elasticsearch index resync the indices for additional languages were deleted, ignoring the actual ES language schema setup. This has been fixed.

Core: The included Mesh UI has been updated to version 1.3.6.

Core: The included Mesh UI has been updated to version 1.3.5.

Clustering: when running in CUD coordination mode, post requests of plugins endpoint were redirected to the master instance. This redirection has been removed since plugins can only modify the graphdb using the mesh rest api.

Core: when an initial admin password is provided with an environment variable, its value won’t be logged anymore.

Core: when a binary file was manually deleted from the file system and the same file was uploaded again through the rest api, the binary upload endpoint was returning a success response even though the binary wasn’t actually uploaded. The file will now be uploaded correctly.

Core: when starting a cluster with the initCluster flag, caches were incorrectly initialized. This could lead to data consistency errors and various validations errors and has been fixed now.

Core: when updating permissions of a role for a given object, only the update permissions for the role were checked. A new check has been added to verify that the user requesting the permissions change has read permission on the object.

Java Rest Client: The classes MeshWebrootResponse and MeshWebrootFieldResponse now extend Closeable and calling close() on implementations will close the underlying response. See Gentics Mesh Java Client for examples of how to properly close the responses.

Core: The included Mesh UI has been updated to version 1.3.4.

GraphQL: all requests to graphql v2 were failing if one of the schemas included an s3 binary field. This has been fixed.

Dependencies: The 'spring-security', 'gson', 'zip4j' libraries have been updated.

S3: A support of S3 fields has been added to the indexing engine.

GraphQL: When a node with the root webpath was retrieved in a specific branch, a permission error was returned, even if user had permission for it. This has been fixed.

GraphQL: The behaviour for getting a single node with version=draft using GraphQL has been changed. Before, if a user had only read published permissions and was requesting a draft node, the node content was returned in case the draft version matched the published version. Now a permission error will be returned without the actual content of the node.

This version contains OrientDB version 3.1.18.

Core: Possible errors during node migrations in branches have been fixed.

Java Rest Client: The new options maxRetries and retryDelayMs can be used to configure automatic retries for requests to Mesh which failed due to a network error. By default retries are disabled. A value of -1 for retryDelayMs will automatically calculate a delay such that all tries are performed in the timespan of the call timeout configured for the Ok HTTP client.

Plugins: Fixes plugin initialization such that plugins that failed to load will cause the liveness probe to return an unhealthy status.

Cluster: The plugin initialization now waits not only for the write quorum being reached, but for the node availability as well, during the startup.

Core: Schema field names are now checked for uniqueness regardless of case sensitivity. This behavior was already enforced in the UI, and is now enforced in the REST API. The check is not performed for existing schemas.

Core: When a project was deleted, its associated version purge jobs were not. This has been fixed.

Core: The node migration process has been improved to reduce resources footprint on large workloads, by chunking the data. An additional configuration item content.batchSize has been added.

Core: Updating the webroot info might throw a false conflict exception, when the segment field value is reset for a schema. This has been fixed.

Clustering: Liveness check now marks an instance as dead, if the event bus did not get its own cluster instance ping within the timeout period, even if the non-cluster ping comes in time.

Search: When using language specific index settings, nodes were incorrectly indexed in the language unspecific index when they were moved or tagged causing the node to be present in both the language specific and language unspecific index. This has been fixed.

Core: When using segment fields, creating a node in a new branch would always fail with a 409 conflict error due to some incorrect uniqueness check for the segment field value. This bug was introduced in version 1.6.39 and has been fixed now.

Core: Having a binary non-segment field update might break the uniqueness of the segment field value, allowing creation of multiple nodes with the same segment/webroot value. This has now been fixed.

Logging: Remove superfluous error messages for HTTP status 4xx errors. These are logged by the logger handler as warnings.

Docker-Image: The base images have been updated to a newer JDK minor version (11.0.16).

Core: The included Mesh UI has been updated to version 1.3.6.

Core: The included Mesh UI has been updated to version 1.3.5.

Clustering: when running in CUD coordination mode, post requests of plugins endpoint were redirected to the master instance. This redirection has been removed since plugins can only modify the graphdb using the mesh rest api.

Core: when an initial admin password is provided with an environment variable, its value won’t be logged anymore.

Core: when a binary file was manually deleted from the file system and the same file was uploaded again through the rest api, the binary upload endpoint was returning a success response even though the binary wasn’t actually uploaded. The file will now be uploaded correctly.

Core: when updating permissions of a role for a given object, only the update permissions for the role were checked. A new check has been added to verify that the user requesting the permissions change has read permission on the object.

Java Rest Client: The classes MeshWebrootResponse and MeshWebrootFieldResponse now extend Closeable and calling close() on implementations will close the underlying response. See Gentics Mesh Java Client for examples of how to properly close the responses.

Core: The included Mesh UI has been updated to version 1.3.4.

GraphQL: all requests to graphql v2 were failing if one of the schemas included an s3 binary field. This has been fixed.

Dependencies: The 'spring-security', 'gson', 'zip4j' libraries have been updated.

S3: A support of S3 fields has been added to the indexing engine.

GraphQL: When a node with the root webpath was retrieved in a specific branch, a permission error was returned, even if user had permission for it. This has been fixed.

GraphQL: The behaviour for getting a single node with version=draft using GraphQL has been changed. Before, if a user had only read published permissions and was requesting a draft node, the node content was returned in case the draft version matched the published version. Now a permission error will be returned without the actual content of the node.

This version contains OrientDB version 3.1.18.

Java Rest Client: The new options maxRetries and retryDelayMs can be used to configure automatic retries for requests to Mesh which failed due to a network error. By default retries are disabled. A value of -1 for retryDelayMs will automatically calculate a delay such that all tries are performed in the timespan of the call timeout configured for the Ok HTTP client.

Plugins: Fixes plugin initialization such that plugins that failed to load will cause the livenes probe to return an unhealthy status.

Cluster: The plugin initialization now waits not only for the write quorum being reached, but for the node availability as well, during the startup.

GraphQL: The behaviour for getting lists of nodes with version=draft using the REST API or GraphQL was inconsistent for nodes, for which the requesting user only had the read published permission. When using the REST API, those nodes were never returned, using GraphQL, those nodes were returned, if they were published and the draft version happened to be identical to the published version. The behaviour when using GraphQL has been changed to be identical to the REST API now.

Rest: The endpoint GET /api/v2/{project}/tagFamilies/{tagFamilyUuid}/tags/{tagUuid}/nodes did not correctly check for the read when called with version=draft, but returned all tagged nodes with read published permission. This has been fixed.

Rest: The endpoint GET /api/v2/{project}/navroot/{path} failed with a 404 error if called with version=published and at least one of the nodes in the node structure was not published. This has been fixed.

Core: If a link to a node was rendered from a url field, the path prefix of the used branch was ignored. This has been changed now, so that links to nodes will always contain the path prefix, regardless of whether the link is rendered from a url field or from segment fields.

Core: When the pathPrefix of a branch was changed, node could possibly still be access using the old pathPrefix due to an internal cache, which was not invalidated. The cache will now be cleared when a branch is modified.

GraphQL: The fields hostname, ssl and pathPrefix have been added to the type branch.

OrientDB: The included OrientDB version has been updated to version 3.1.18.

Search: When a schema contained a field of type "micronode" or "list" of "micronode" and no "allow", generating the microschema version hash as part of the search index name failed with a NullPointerException. This has been fixed, so that the microschema version hash will now be null in such cases (like if "allow" was set to an empty array).

Core: The default values for the periodic disk quota check have been changed to absolute values: the default for storage.diskQuota.warnThreshold is now 3G and the default for storage.diskQuota.readOnlyThreshold is 1G.

Server: Fixed defined Health-Check in docker container and added a startup-grace period of 60s.

Rest: Many reading request were implemented in a way that they would never be executed concurrently with other reading requests. So would e.g. a long running graphql request cause webroot requests executed at the same time to be queued until the graphql request was done. The behaviour has been changed, so that reading requests will only be queued, if the worker pool has been exhausted.

Core: A periodic check for the EventBus has been added, which can be configured via the configuration settings vertxOptions.eventBus.checkInterval, vertxOptions.eventBus.warnThreshold and vertxOptions.eventBus.errorThreshold. If enabled (which is the default), and the periodically sent ping event is not received within the given error threshold, the Mesh instance which be considered unhealthy (liveness check will fail).

Clustering: When starting an instance of Gentics Mesh with the command line parameter -resetAdminPassword to reset the password of the admin user, starting would fail if the write quorum of OrientDB was not reached. The startup behaviour has been changed now, so that resetting the password is delayed until the write quorum is reached.

Rest: When creating a new user and setting the admin flag to false, this failed if the request user did not have the admin flag set to true. The permission check has been fixed now, so that the admin flag is only required for the requesting user, if the new user should also have the admin flag set to true.

Core: When a project was renamed, deleted and then recreated with the same name, an internal cache was not cleared and caused subsequent errors in calls to the project, until the Mesh instance was restarted. This has been fixed now. The internal caches will be properly cleared.

Rest: The new endpoint DELETE /api/v1/admin/cache will clear all internal caches (cluster wide).

Core: The included Mesh UI has been updated to version 1.3.3.

Core: The included Mesh UI has been updated to version 1.3.2.

Rest: The endpoint /api/v1/admin/jobs now supports query parameters for filtering (status, type, branchName, branchUuid, schemaName, schemaUuid, microschemaName, microschemaUuid, fromVersion, toVersion).

Search: The periodic check for correctness of the indices in Elasticsearch created temporary indices, which did not start with the installation prefix, and thus were sometimes not cleaned up. The naming of those temporary indices has now been changed to also start with the installation prefix.

Security: The 'spring-security', 'jsoup', 'commons-io', 'guava', 'jackson-databind' libraries have been updated.

Core: The 'commons-codec' libary has been updated. This fixes possible internal errors when parsing uploaded files using tika.

Cluster: When starting a node with a new version of Gentics Mesh in a cluster, application of the changelog and setting the new Mesh version and database revision was delayed until the write quorum is reached, but not until the cluster topology lock was raised, which could cause a blocked transaction. This has been fixed.

Search: When the ElasticSearch instance is not available, getting version information from the endpoint /api/v1 failed with an internal server error. This has been fixed, calling the endpoint /api/v1 will not fail anymore, but will not contain the ElasticSearch version.

Rest: When a project with name "project" was deleted, calls to the endpoint /projects would cause 404 errors, until the Mesh instance was restarted. This has been fixed now.

Java Rest Client: If requests to mesh fail with I/O errors (including timeouts), the thrown exception will now also contain the request method and URL.

Server: Liveness-Check script live.sh will now fail if the mesh.live file has not been changed since 20s (previously 60s)

Docker-Image: The base image has been changed from JRE to JDK to include more tools for analyzing (e.g. creating heap-dumps). The base image has been updated to a newer JDK minor version (11.0.12)

AWS S3: Now the Cross-Origin Request Settings parameters of a S3 connection can be configured. By default the allowed headers/origins are set to *, the allowed methods are set to "GET", "PUT", "POST", "DELETE". Setting any of the CORS configuration to null prevents is to override the server’s default configuration. Setting all the CORS configuration to null or empty value prevents the CORS config to be changed at all.

Cluster: When doing a rolling update in a cluster, a OConcurrentModificationException could occur during the startup, which could cause subsequent errors. This has been fixed.

Core: A periodic check for available disk space has been added. If not enough disk space is available, Gentics Mesh will be set to read-only to avoid corruption of the storage due to a full disk. Additionally, the metrics mesh_storage_disk_total and mesh_storage_disk_usable have been added to monitoring.

Search: The indices for nodes of a schema, which contains fields of type "micronode" have a mapping for the fields in the allowed microschemas, but were not changed, if the microschemas were changed. This caused subsequent errors while indexing or searching. The naming of such indices have been changed to contain a hash over all allowed microschema versions and when a microschema is changed, a new index will be created now. This update will automatically drop and recreate all existing indices for schemas containing fields of type "micronode".

AWS S3: Now the Cross-Origin Request Settings parameters of a S3 connection can be configured. By default the allowed headers/origins are set to *, the allowed methods are set to "GET", "PUT", "POST", "DELETE". Setting any of the CORS configuration to null prevents is to override the server’s default configuration. Setting all the CORS configuration to null or empty value prevents the CORS config to be changed at all.

Server: added a health/writable monitoring and REST endpoint to check whether mesh is in a writable state (e.g. the topology lock is not hold).

Server: when a schema was updated with a new field with elastic search properties, these were ignored. This has been fixed now.

Search: If ElasticSearch indices, which were originally created by Mesh were dropped, they would probably be re-created with the default mapping when Mesh stored a document in the index. This could cause the index to be not completely filled and also the mapping to be incorrect, so that search queries would fail to find documents. In order to detect and repair such situations, a periodic check has been added to Mesh, which will check existence and correctness of all required ElasticSearch indices. Missing indices will be created and incorrect indices will be dropped and re-created. Afterwards a full sync for the affected indices will be triggered. The check period can be configured with the new configuration setting search.indexCheckInterval, which defaults to 60_000 milliseconds. Setting the interval to 0 will disable the periodic check (not recommended).

Search: The index maintenance endpoints /api/v1/search/sync and /api/v1/search/clear have been extended with the query parameter index, which can be used to restrict the synchronized/cleared indices by a regular expression.

Search: The index maintenance actions triggered via /api/v1/search/sync and /api/v1/search/clear will now be redirected to the current master instance, if the cluster coordinator is used.

Core: The included Mesh UI has been updated to version 1.3.1, containing the language-related improvements and minor bugfixes.

Search: When a full synchronization of the search indices was triggered, language specific indices were unnecessarily removed first. This has been changed, language specific indices will now be treated like all other indices during a full synchronization.

Core: When performing an update for role permissions, an event was published even if the update actually did not change anything. A check has been added to only publish the event if permissions are actually changed.

Core: When eventbus messages where published during execution of a job (e.g. a node migration), the events were not published and handled immediately, but were queued in memory and eventually published after the job execution. This caused high memory consumption during a node migration and also the elasticsearch indexing of migrated nodes to be delayed until after all nodes had been migrated. Execution of the jobs has been changed now, so that events will be published immediately to fix those issues.

Core: The overall memory consumption during a migration job has been reduced.

Clustering: Due to a bug in the vert.x eventbus system, elasticsearch sync operations were not only performed in the Mesh instance, were the causing data change actually happened, but on all Mesh instances, which caused unnecessary high load on the elasticsearch. This has been fixed with a workaround.

Core: When deleting old versions of nodes, the related binaries were not deleted from the disk. This has been fixed now. Credits to https://github.com/kdima001.

Core: When an OrientDB transaction repeatedly fails and runs into the retry limit, the original cause will now also be logged.

Server: When an OutOfMemory was caught or one of the plugins (permanently) failed to initialize, mesh was still considered to be live. This has been fixed now.

Clustering: When doing an offline update of Mesh in a cluster, the first instance of the cluster could not be started with the new Mesh version, because the new version should be written into OrientDB, which failed due to write quorum not reached. The initialization procedure has been changed now, so that executing the changelog and storing the Mesh version and DB revision will now only be started after the write quorum has been reached.

Server: Mesh will now create a file mesh.live and touch it every 10 seconds as long as the instance is considered live. If an OutOfMemory is caught or one of the plugins (permanently) fails to initialize, mesh is considered no longer live and the file will not be touched any more.

Server: The readiness probe will now also check the status of the local OrientDB storage. A mesh instance will only be considered "ready" if the local OrientDB storage is "ONLINE".

GraphQL: The wait parameter will now only wait for elasticsearch to be ready when a query option in nodes is actively used. Previously it would wait for elasticsearch even if it wasn’t needed.

Clustering: When recovering from a split-brain situation, the topology lock (which was raised, because nodes rejoined the cluster) was not always removed. This has been fixed, and some info log about the reason for the topology lock has been added.

Core: The included Mesh UI has been updated to version 1.3.0, containing the stability improvements, as well as the new feature of the node status (draft/updated/published/archived etc) filtering.

Clustering: The high level changelog system has been extended to allow for specific changes to be executed in cluster mode.

Core: A bug preventing the string schema field restrictions from being cleared, has been fixed.

Core: The included Hazelcast-Kubernetes library has been updated to version 1.5.6, containing a DNS resolving fix.

AWS S3: A bug preventing the new image data from being stored into an existing s3binary container, has been fixed. AWS S3: An inability of changing permissions for the schema with an S3 field has been fixed.

Core: Added support for storing/referencing the binary data at Amazon Web Services S3 container.

Clustering: The cluster.topologyChangeReadOnly setting and MESH_CLUSTER_TOPOLOGY_CHANGE_READONLY environment setting have been added. By enabling this flag, Mesh will be automatically set in readOnly mode, when the cluster topology changes, which will let reading requests pass and writing requests fail (immediately), instead of blocking all requests, which happens when the cluster.topologyLockTimeout is set to a positive value.

Plugins: If plugin initialization fails due to topology changes (e.g. the OrientDB needs to be synchronized from another node), the plugin will be set to status FAILED_RETRY and initialization will be retried as soon as OrientDB becomes available again.

Clustering: The timeout for waiting to join the cluster (which includes the time for synchronizing the graphdb storage), which was hardcoded to 500_000 milliseconds, can now be configured with the setting storage.clusterJoinTimeout or the environment variable MESH_GRAPH_CLUSTER_JOIN_TIMEOUT.

OrientDB: The included OrientDB version has been updated to version 3.1.11.

Plugins: An issue in the plugin environment has been fixed which prevents plugin REST handler starting over 0.0.0.0 host IP address, when the OS disallows it.

Upload: More user-friendly message + HTTP 400 error on an attempt to upload an empty file.

OrientDB: The included OrientDB version has been updated to version 3.1.10.

Upload: The document processing library tika has been updated to version 1.26 to fix the PDF parser failing on files with some Adobe fonts.

Core: Added support for looking up the node field content by webroot path and a field name.

Rest: Added new endpoint for webroot path field lookup.

OrientDB: The included OrientDB version has been updated to version 3.1.6.

OrientDB: A bug in the startup order has been fixed which prevented opening of databases via OrientDB Studio.

Core: Binary upload processing failed for binaries with brackets in the metadata key. This has been fixed now.

GraphQL: Queries, that take longer than the configured threshold will be logged now. See GraphQL Slow Query Log for details.

Jobs: A race condition within the job processing mechanism has been fixed. In some cases newly created jobs would not be automatically invoked. This has been fixed now.

Search: Fixed potential NullPointerException that could occur during search index synchronization.

Clustering: The cluster.topologyChangeReadOnly setting and MESH_CLUSTER_TOPOLOGY_CHANGE_READONLY environment setting have been added. By enabling this flag, Mesh will be automatically set in readOnly mode, when the cluster topology changes, which will let reading requests pass and writing requests fail (immediately), instead of blocking all requests, which happens when the cluster.topologyLockTimeout is set to a positive value.

Plugins: If plugin initialization fails due to topology changes (e.g. the OrientDB needs to be synchronized from another node), the plugin will be set to status FAILED_RETRY and initialization will be retried as soon as OrientDB becomes available again.

Clustering: The timeout for waiting to join the cluster (which includes the time for synchronizing the graphdb storage), which was hardcoded to 500_000 milliseconds, can now be configured with the setting storage.clusterJoinTimeout or the environment variable MESH_GRAPH_CLUSTER_JOIN_TIMEOUT.

GraphQL: The breadcrumb field now accepts the lang parameter which will be used for additional fallback languages.

Upload: A potential error within the PDF parsing code has been patched.

OrientDB: The included OrientDB version has been updated to version 3.1.4.

Clustering: The coordination feature will now be automatically disabled whenever clustering is disabled.

Core: The link renderer now tries to use url fields to render the link if it cannot be constructed using segment fields.

OrientDB: The included OrientDB version has been updated to version 3.1.3.

Clustering: Internal caches will now be cleared when cluster topology changes occur.

Backup: The ?consistencyCheck query parameter was added to the /api/v2/admin/backup endpoint. When set to true it will run the consistency check before invoking the backup. The call will fail when inconsistencies were detected.

Clustering: An additional check has been added which will prevent nodes from joining a cluster which contains an outdated database. The -initCluster flag needs to be used for a single instance to migrate the cluster. This is done to prevent concurrency issues during changelog execution and cluster formation.

Tests: The path handling for the MeshContainer test container class has been updated. Container data will now be placed in the target/mesh-containers folder.

Additional log output and checks have been added to the auth plugin mapping code. This fix also addresses the Null keys are not supported error which was thrown when handling mappings which provide no group and role names. #1138

OrientDB: The included OrientDB version has been updated to version 3.0.34.

Rest: The webroot cache error handling has been improved.

Search: The memory footprint of the differential sync mechanism has been reduced. The sync operations will now be split into buckets. The size of the buckets can be influenced via the search.syncBatchSize setting.

Core: Fixed a bug that which prevented to update a node reference of a user if the user already had one set. #1114

Core: It is now possible to disable extracting of metadata or content from binaries on a per-field basis. See schema field types documentation for more information.

Plugins: The default plugin timeout has been increased from 15 seconds to two minutes.

Monitoring: The readiness probe will now also check for plugin status. Failed plugin deployments will let the readiness probe fail.

Monitoring: The readiness probe will now no longer fail when a plugin is not registered. Instead it will only fail when a plugin deployment has failed.

Monitoring: The liveness probe will now check for plugin status. Failed plugin deployments will let the liveness probe fail.

Clustering: The webroot handler no longer uses the cluster-wide write lock.

Logging: Failing readiness checks are now logged using the WARN level.

GraphQL: The wait parameter will now only wait for elasticsearch to be ready when a query option in nodes is actively used. Previously it would wait for elasticsearch even if it wasn’t needed.

Clustering: When recovering from a split-brain situation, the topology lock (which was raised, because nodes rejoined the cluster) was not always removed. This has been fixed, and some info log about the reason for the topology lock has been added.

Core: The included Mesh UI has been updated to version 1.3.0, containing the stability improvements, as well as the new feature of the node status (draft/updated/published/archived etc) filtering.

Core: A bug preventing the string schema field restrictions from being cleared, has been fixed.

Core: The included Hazelcast-Kubernetes library has been updated to version 1.5.6, containing a DNS resolving fix.

Clustering: The cluster.topologyChangeReadOnly setting and MESH_CLUSTER_TOPOLOGY_CHANGE_READONLY environment setting have been added. By enabling this flag, Mesh will be automatically set in readOnly mode, when the cluster topology changes, which will let reading requests pass and writing requests fail (immediately), instead of blocking all requests, which happens when the cluster.topologyLockTimeout is set to a positive value.

Plugins: If plugin initialization fails due to topology changes (e.g. the OrientDB needs to be synchronized from another node), the plugin will be set to status FAILED_RETRY and initialization will be retried as soon as OrientDB becomes available again.

Clustering: The timeout for waiting to join the cluster (which includes the time for synchronizing the graphdb storage), which was hardcoded to 500_000 milliseconds, can now be configured with the setting storage.clusterJoinTimeout or the environment variable MESH_GRAPH_CLUSTER_JOIN_TIMEOUT.

OrientDB: The included OrientDB version has been updated to version 3.1.11.

Plugins: An issue in the plugin environment has been fixed which prevents plugin REST handler starting over 0.0.0.0 host IP address, when the OS disallows it.

OrientDB: The included OrientDB version has been updated to version 3.1.10.

Upload: The document processing library tika has been updated to version 1.26 to fix the PDF parser failing on files with some Adobe fonts.

OrientDB: The included OrientDB version has been updated to version 3.1.6.

OrientDB: A bug in the startup order has been fixed which prevented opening of databases via OrientDB Studio.

Core: Binary upload processing failed for binaries with brackets in the metadata key. This has been fixed now.

GraphQL: Queries, that take longer than the configured threshold will be logged now. See GraphQL Slow Query Log for details.

Jobs: A race condition within the job processing mechanism has been fixed. In some cases newly created jobs would not be automatically invoked. This has been fixed now.

Search: Fixed potential NullPointerException that could occur during search index synchronization.

OrientDB: The included OrientDB version has been updated to version 3.1.4.

Clustering: The coordination feature will now be automatically disabled whenever clustering is disabled.

Clustering: Internal caches will now be cleared when cluster topology changes occur.

Backup: The ?consistencyCheck query parameter was added to the /api/v2/admin/backup endpoint. When set to true it will run the consistency check before invoking the backup. The call will fail when inconsistencies were detected.

Clustering: An additional check has been added which will prevent nodes from joining a cluster which contains an outdated database. The -initCluster flag needs to be used for a single instance to migrate the cluster. This is done to prevent concurrency issues during changelog execution and cluster formation.

OrientDB: The included OrientDB version has been updated to version 3.0.34.

Rest: The webroot cache error handling has been improved.

Core: It is now possible to disable extracting of metadata or content from binaries on a per-field basis. See schema field types documentation for more information.

Monitoring: The readiness probe will now no longer fail when a plugin is not registered. Instead it will only fail when a plugin deployment has failed.

Monitoring: The liveness probe will now check for plugin status. Failed plugin deployments will let the liveness probe fail.

Clustering: The webroot handler no longer uses the cluster-wide write lock.

Logging: Failing readiness checks are now logged using the WARN level.

Core: The included Hazelcast-Kubernetes library has been updated to version 1.5.6, containing a DNS resolving fix.

Clustering: The cluster.topologyChangeReadOnly setting and MESH_CLUSTER_TOPOLOGY_CHANGE_READONLY environment setting have been added. By enabling this flag, Mesh will be automatically set in readOnly mode, when the cluster topology changes, which will let reading requests pass and writing requests fail (immediately), instead of blocking all requests, which happens when the cluster.topologyLockTimeout is set to a positive value.

Plugins: If plugin initialization fails due to topology changes (e.g. the OrientDB needs to be synchronized from another node), the plugin will be set to status FAILED_RETRY and initialization will be retried as soon as OrientDB becomes available again.

Clustering: The timeout for waiting to join the cluster (which includes the time for synchronizing the graphdb storage), which was hardcoded to 500_000 milliseconds, can now be configured with the setting storage.clusterJoinTimeout or the environment variable MESH_GRAPH_CLUSTER_JOIN_TIMEOUT.

OrientDB: The included OrientDB version has been updated to version 3.1.11.

Plugins: An issue in the plugin environment has been fixed which prevents plugin REST handler starting over 0.0.0.0 host IP address, when the OS disallows it.

OrientDB: The included OrientDB version has been updated to version 3.1.10.

Upload: The document processing library tika has been updated to version 1.26 to fix the PDF parser failing on files with some Adobe fonts.

OrientDB: The included OrientDB version has been updated to version 3.1.6.

OrientDB: A bug in the startup order has been fixed which prevented opening of databases via OrientDB Studio.

Core: Binary upload processing failed for binaries with brackets in the metadata key. This has been fixed now.

GraphQL: Queries, that take longer than the configured threshold will be logged now. See GraphQL Slow Query Log for details.

Jobs: A race condition within the job processing mechanism has been fixed. In some cases newly created jobs would not be automatically invoked. This has been fixed now.

OrientDB: The included OrientDB version has been updated to version 3.1.4.

Clustering: The coordination feature will now be automatically disabled whenever clustering is disabled.

Clustering: Internal caches will now be cleared when cluster topology changes occur.

Backup: The ?consistencyCheck query parameter was added to the /api/v2/admin/backup endpoint. When set to true it will run the consistency check before invoking the backup. The call will fail when inconsistencies were detected.

Clustering: An additional check has been added which will prevent nodes from joining a cluster which contains an outdated database. The -initCluster flag needs to be used for a single instance to migrate the cluster. This is done to prevent concurrency issues during changelog execution and cluster formation.

OrientDB: The included OrientDB version has been updated to version 3.0.34.

Rest: The webroot cache error handling has been improved.

Monitoring: The readiness probe will now no longer fail when a plugin is not registered. Instead it will only fail when a plugin deployment has failed.

Monitoring: The liveness probe will now check for plugin status. Failed plugin deployments will let the liveness probe fail.

Clustering: The webroot handler no longer uses the cluster-wide write lock.

Logging: Failing readiness checks are now logged using the WARN level.

Core: Fixed a bug that which prevented to update a node reference of a user if the user already had one set. #1114

Plugins: The default plugin timeout has been increased from 15 seconds to two minutes.

Monitoring: The readiness probe will now also check for plugin status. Failed plugin deployments will let the readiness probe fail.

Clustering: The cluster coordinator will now detect changes caused by the OAuth token mapping and will redirect the request if necessary. Thus this change allows the use of the CUD mode when using request coordination in combination with authentication plugins.

Clustering: When using the cluster coordinator, forwarded requests will now contain the X-Mesh-Forwarded-From header.

Plugins: A potential resource leak could cause thread exhaustion when using new REST Clients for each request. This has been fixed now.

Plugins: Fixed an error which was triggered when trying to un-deploy a not yet deployed plugin.

Plugins: Fixed a bug in the plugin registration process in which plugin deployments would fail due to timeouts in clustered environments.

Plugins: The plugin registration process was changed. Plugins will now no longer be directly registered when deployed. Instead the plugins will remain in a pre-registration status util Gentics Mesh is able to deploy them. This is especially useful when running plugins in clustered mode. Plugins will only be registered once the write quorum has been reached. Additionally the plugin deployment process will now utilize a lock in clustered mode to prevent concurrent deployments of plugins. The plugin status was added to the plugin endpoint response.

Clustering: The write lock was removed from the DELETE /api/v2/admin/plugins/:id and POST /api/v2/admin/plugins endpoints to prevent potential deadlocks.