Release v6.10.0 (2024-04-30)

Published 2024-04-30 14:57:12Z

• fylr_checksums.txt

• fylr_v6.10.0_darwin_amd64.tar.gz

• fylr_v6.10.0_darwin_arm64.tar.gz

• fylr_v6.10.0_linux_amd64.tar.gz

• fylr_v6.10.0_windows_amd64.zip

Important

• This release brings a forced Re-index in blocking mode for the first time. The instance will go into a re-indexing state and thus not be accessible for users until the re-index is done. This can take anything between a few minutes and multiple hours (depending on the size and performance of your installation).

• Sorting and searching of languages with special characters has been greatly improved.

• /api/schema: Make loading more robust, skip inconsistencies in fields. If a field has no matching column in the table, loading would fail and crash fylr at startup. This code skips the faulty field (without fixing the root problem).

• Rights management: Changed the ASSET_SHOW and ASSET_DOWNLOAD rights when no versions are set. Before this was allowing all versions, now it is allowing none.

Server

New

• Support for Microsoft Azure Blob Storage in storage manager. This works like our S3 support, just for Microsoft Blob Storage. This works for new storage items only, a migration from storage A to B is not yet included in fylr.

• /api/pool/<id>/stat?include_subpools=1|0: New endpoint to collection file stats (like size in storage) for pools.

• /api/objecttype/<id>/stat?include_subpools=1|0: New endpoint to collection file stats for pools.

• fylr.yml: This new setting can be used to add a RemoteAddr info to all request logs: fylr.logger.addRemoteAddr

• /api/schema?format=svg|png: Download the data model in SVG or PNG format. This feature was still missing from easydb 5 and is now supported in fylr, too.

• Custom Data Type: Added support for geo_json. The new column type geo_json wasn't supported for custom data types. This patch adds the support for _standard.geo, and geo_json mapping (supports search and export).

• Datatype geo_json: Support all geojson types, not just FeatureCollection. The geo_json data type now accepts all types on top level as supported by Geo JSON.

• Transitions: Supports _owner (of affected objects) as email recipient.

• PUT /api/eas?collection=<id>: Supports file upload directly into a collection. This uses the same code paths as WebDAV and Hotfolder.

• POST /api/collection: Extended by collection.create_object.plugin and collection.create_object.plugin_config.

• Plugins for Collection: For the collection plugin, config is managed by the fylr server. Plugins can employ a new collection_upload section in their manifest.yml to hook into this feature. This also changes the behaviour in db_pre_save callback a little: Now, when using the Hotfolder upload, such automatically generated objects (like a linked image object), plugins are called. Supports api_user_access_token in WebDAV and Hotfolder access paths. This adds a new event type COLLECTION_UPLOAD to the WebDAV, Hotfolder and /api/eas?collection=<id> endpoints. These events include information about the file uploaded and the object(s) created. It is also written for each collection upload_log entry received from the collection upload plugin.

• Custom Data Type Updater: Add plugin_user[_access_token]. This patch adds a plugin_user & plugin_user_access_token to the callback towards the custom data type update. This can be configured in the manifest.yml of the plugin. See fylr-plugin-example for an example.

• /inspect/system/command: This new page shows currently running commands for backup & restore started by fylr. It's the place where such running processes can be stopped manually. This also works in multi binary setups.

• Backup manager: A new format pg_dump has been added. This uses the pg_dump utility for backups. This backup can be used for a full restore of fylr. Tables not included are file_queue, index_queue, fylr_backend, oauth2_token, object_cache, webdav_lock, command_track. However, the backup manager's main purpose is still for support & development at the moment.

• Produce config: Expose --background in recipe browserthumbs.

Improvements

• Rights management: Changed the ASSET_SHOW and ASSET_DOWNLOAD rights when no versions are set. Before this was allowing all versions, now it is allowing none. If your instance is affected by this, you need to manually enable all versions for the respective rights.

• All pages & emails have been refined and visually improved. Now, emails use a dark theme on readers in dark mode.

• New object loader code to optimise and streamline object loading. A new fylr.debug.logTimings has been introduced alongside with this to output a concise overview of timing information of API requests to the console. This knob is also available as a base config, accessible in the frontend.

• /inspect/files has a nicer header and improved action features to sync & load files.

• Many improvements in fylr backup and fylr restore. This makes the migration from easydb 5 easier and more reliable. Deleted objects which are still in the database are now ignored by the backup. Checkout fylr backup -h and fylr restore -h for an overview of the current set of features. We removed --user, --base and --file-api-append-to-url-query in restore and added --file-api-access-token.

• Improved file extension detection for unusual/special content types like x-ms-bmp.

• /api/user/session: Add database_locales. This enabled the frontend to display dates in database language formatting.

• /api/export: Remove unset values in filename templates. If a filename template defines a replacer like %object.title% which is unset in the actual data, we now replace this with an empty string instead of outputting the literal replacer. We decided its better to use the original filename than something like SID-EAS_ID as fallback for filename template which return "". This can happen if a template says %object.title% and the object has no title set.

• Custom Tag Colors: Base config now supports custom tag colors for frontends to use.

• Plugin Callbacks: Add api_user to callback info. With this, a plugin can more easily access which user is running the callback.

• File worker: Produce PNG files for .ai and .svg with no full version.

• IIIF tiles: Transparent sources are now rendering to a white instead of a black background for zoomer tiles. This fixes color flipping from white to black when zooming in.

• /api/export: Clean-up custom version calculation, add contain as custom_size. Fixes the custom_size height which was not calculated properly.

• /api/search: Improved sorting and searching for languages with special characters like Danish and Arabic. For simple text fields we now use the first database language (base config) to sort on. Before this was unspecified using a general icu_collation_keyword. The icu_collation_keyword is not configurable enough for us (no char_filter), so that we cannot properly sort Arabic. This patch refactors the index to not use a "fields" mapping of Elastic but rather send distinct "properties" where we can prepare the collation_keywords already in fylr.

• /api/search: Properly sort facets by language terms. This was a long standing problem in our facet implementation which now has been fixed. Special characters should now sort correctly in the filter tree depending on the language they are stored in.

• Base Config Languages: Removed custom indexer configuration. This simplifies base configuration of languages. It takes away the possibility to use built-in or custom analyzers from Elastic. It also removes the "variant" support for the icu collation keyword analyzer. So, e.g. it is no more possible to configure "phonebook" sorting for de-DE.

• SAML: Log into our logger, fixes console output. With the logger re-direction to our console, the SAML error message should now also appear in /inspect/system/console.

• Cache expiration: This has been improved and is less racy now. Also supports multi binary deployments better.

• fylr convert: Rename --quality to --jpegquality. This was never really supported for anything else but JPEG, so rename the parameter and added some additional checks. Also, expose --jpegquality in recipe browserthumbs.

• Better email checks Self-Registration & User Management: The mail address checks in these places have been improved.

• Export Custom Versions: Use white background for lost transparent backgrounds in JPEG targets and filled space when --rotate is used.

• DELETE /api/xmlmapping/mapping/<id>: Introduce a 202 process to warn about collections which still use that mapping.

Fixed

• Make sure fylr does not endlessly create new file entries for plugin ZIP files if a storage location isn't available.

• Compression has been turned off for backend file delivery (storage type file). This speeds up file production (like zoomer tiles). A new X-Fylr-Compression response header is set to show if a response was compressed or not.

• GPS coordinates with a negative longitude or latitude have not been correctly recognised by fylr. You need to re-sync affected files to fix the detection.

• /api/db: Fixed tag removal in group mode. If multiple tags of the same group where to be removed in group mode the api would complain with error MultipleChoicesInTaggroup which is technically correct but did not take into account that the user wants to remove the tags. So, in that case we can accept multiple tags of the same group.

• New event added to support frontend: ASSET_DOWNLOAD_CONFIRMATION_MESSAGE. This fixes download with confirmation form where this ASSET_DOWNLOAD_CONFIRMATION_MESSAGE event type is used. This event type was missing in fylr.

• api/db: Run local transitions in pools and objecttypes. Due to not fully loaded objects, the transitions configured for pools did not run.

• File worker: Fix rendering tiled watermark. The implementation for "tile" watermark settings (in pools) was missing.

• File delivery: when writing metadata, set a 24h timeout. Before this was 3m which wasn't enough for bigger files over slower connections.

• Indexer: Fixed missing re-index cases in hierarchies when parents changed.

• Import: Fixed mapping into double columns.

• Group emails in transitions fixed: Collection emails which are meant for a whole group would fail due to a malformed SQL query. This resulted in emails which cannot be sent.

• Plugin: Don't use custom mappings of disabled plugins. The server did not acknowledge the fact that a plugin is disabled and still read its custom_types configuration.

• Rights management: Check permission for historic objects by checking the latest object's permissions. This was an oversight in fylr's implementation of the rights management in which we wrongly checked the permissions of the old version instead of always applying the permissions of the latest version even to older versions.

• /api/search: The index of pool._level was 1 too high, so this might have affected API users when searching for pool._level.

• Multi binary deployment: WebSocket, Cache and other things where not working reliably in multi binary setups (like 2 pods running the same fylr instance in a Kubernetes setup). For that we changed the WebSocket broadcast implementation.

• /api/transition: Fixed sending back property sticky. sticky was stored and loaded and used, but not sent back over the API.

• Metadata mapping: Removed File:BitDepth from profile as it makes no sense, it's neither "Bit Rate" nor "Frame Rate".

• Workflows: Improved grouping of emails for recipient _pool_contact. Before all objects in a batch were sent to all pool contacts found in that batch. Now the object lists are split according to the pool contact.

Frontend

New

• Hotfolder Plugins: Added support in the frontend to display options for the new hotfolder collection plugins. This configuration will be displayed in the settings of the collections that have the hotfolder functionality activated.

• Hotfolder Collections: Now hotfolder collections use the new API included in this release to load assets. The server now manages the loading and creation of records.

• Custom Tag Color: Added support for custom tag colors, which can be added in the base config and selected in the tag settings.

• Pool Statistics: Added a new tab in the pool management panel to display asset statistics by pools. These statistics can show the content of the selected pool or include child pools.

• Objecttype Statistics: Added a new tab in the objecttype manager to display statistics of assets that exist in the selected objecttype.

• Filter Manager on Main Lists: The filter manager has been added to the main lists; previously, it was only available in Main Search, so it could not be used for secondary object types.

• Geo Standard: Added support for Geo Standard in custom data types.

• Backup Formats: Added a new "pg_dump" option in the backup manager.

Improvements

• CSV Importer RPUT: Improved the behavior of image loading via RPUT using the CSV importer.

• Copy Record Tool: Improved the functionality of copying a record in the main search. The copied object is now displayed correctly in the search.

• CSV Importer: Added the function to search for parent linked objects using fields other than the ID.

• Admin Messages: General improvements have been made to the Admin messages panel.

• User Settings: Adjustments have been made to the user settings panel. Some fields are no longer mandatory to save the settings.

• Pool Selector: If a pool is not available for selection but needs to be displayed to not break the hierarchy, this pool will not be selectable but will allow viewing child pools that are.

• Filter Manager: Improved the filter manager to show a notice when no filters are available. Previously, only an empty panel was displayed.

• Asset Browser: Improved the info panel in the asset browser to display which version is being shown in the version panel.

• FullScreen: Improved version management in the asset browser when viewed in full screen. Bugs have been corrected where the version displayed was not the most suitable for the viewport size.

• Transitions: Now the owner can be selected as the recipient of an email transitions.

• Arabic and Danish Sorting: Improved the frontend handling of these languages when used to sort elements.

• Zoomer: Improvements have been made to the zoomer tool to zoom in the asset browser. These improvements include better visualization of images with transparency and optimizations when using very large images as a source.

• Password Field: Improvements have been made to the password fields regarding the show and hide password button.

• Request Data: Optimized JSON delivery in requests made by the frontend, reducing the size of requests in certain scenarios such as in large data models.

• DEV Panel: Improvements have been made to the DEV panel on the frontend, including the addition of a selector to choose the type of event poller.

• Shared Collection: Added new icons to show which collections are shared.

• JSON Importer Error Handler: Improved the error messages generated by the JSON importer.

• Full Screen Detail: Corrections have been made to the full screen detail panel.

• Show History on Editor: Added the possibility to view the history change of an object while in the sidebar editor. Previously, this was only possible if we were in the detail of the object.

Fixed

• Base Config: Fixed how JSON fields are rendered for plugin_type class types.

• Datamodel SVG Export: The generated SVG now opens in a new window.

• Search Input: Corrected the double-click on query elements; they can now be modified correctly by double-clicking without mistakenly triggering a search.

• Hierarchies on CSV Importer: Corrected the search for parent linked objects by system_global_object_id.

• Date Inputs: Date inputs now correctly use the format specified in the settings.

• Field Facets Sort: Made corrections in filter sorting. Filters created by custom data types are now sorted correctly.

• Hide Label On Masks: Corrected a bug where the "Hide field label" option was not displayed correctly in nested fields.

• CSS: Numerous corrections have been made to the app's CSS.

• EAS Images: Now the blob data from images is released from memory once the image is rendered on the frontend. This solves memory leak problems caused by the way images are managed on the frontend.

• Table View: Corrected an error where rows that should not open in Table View after conducting a search were opening.

• Base Type Sorting: Corrected the language selected for sorting base types; now the language configured for the frontend will be chosen.

• Data Sorting: Corrected the language used to sort data in the application.

• Logo Customization: Corrected a bug where the field to configure the logo in the base config was not displayed correctly.

• Detail Sidebar: Fixes the detail sidebar trying to load not ready objects when we just created one but was not yet indexed. Now the detail can wait for indexing.

• Pool Selector Wildcard: Corrected the use of * to search for pools in the pool selector.

• OAuth Refresh Token: Corrected the refresh token behavior on the frontend.

• Polyhierarchy Filters: Corrected the use of filters for polyhierarchy linked objects. Previously, children filters were not displayed correctly.

• Date Columns: Now the date columns in records will correctly use the language configured as the database language, previously the frontend language was used to format dates in these fields.

• Base Config Upload: Errors in loading base configs from older versions have been corrected.

• Color Profiles on Exports: Corrected a failure when trying to export an asset without specifying a color profile.