Release v6.9.0 (2024-02-28)

Published 2024-02-28 14:15:46Z

Server

Important

  • Config change in fylr.yml: The config fylr.services.webapp.tls.letsEncrypt.forwardHttpAddr has been changed to fylr.services.webapp.tls.forwardHttpAddr. You must correct this in your fylr.yml. We recommend to add the new option before the update.

  • A re-index is recommended after the update. We changed the standard text analyzer for fields (see below). Also, if you work with remotely stored files and experience black previews in collections, a re-index will fix this problem. And searching for an object owner was broken and needs a re-index to be fixed.

  • If you are using the standard exec server wait group configuration provided by us in the fylr.default.yml, check out the changes below. The wait groups have been renamed and as such, if you use the wait groups names in your own fylr.yml, you must check your configuration and adopt this to the new fylr.default.yml. If your configuration does not have all the wait groups defined, which are referenced by execserver: services:, fylr will not start. The default wait groups are now slow medium fast. Also the new fylr.default.yml might put more pressure on your system, as we now allow more queue runners to be active.

New

  • The OAUTH2 Password Flow now also supports collection, email and ldap login methods. Before only easydb was supported.

  • New system right frontend_features[asset_variants], used by our frontend to allow accessing asset variants.

  • New system right frontend_features[collection.share_anonymous], used by our frontend to show anonymous share option.

  • New system right frontend_features[changelog_with_user], used by our frontend to show changelog information including the user's name.

  • LDAP/SAML: Added a base config option to set the update column for user. When a user is mapped from ldap or saml we used to use the reference column to find the user in the database to check if he is new or not. Now, the new configuration allows to find the user by email or login in addition to reference.

  • /api/collection: Add timestamps _created_at and _updated_at to the API.

  • /api/message: Add client ids filter. With the new property client_ids messages of type "server" can be filtered by the oauth2 client id, the user used to login.

  • Server: Add http to https redirect support for manual configured SSL certificates. This patch brings a forwarding web server listening on http redirecting to https for manually configured certificates (if you are not using Let's Encrypt). The config fylr.services.webapp.tls.letsEncrypt.forwardHttpAddr has been changed to fylr.services.webapp.tls.forwardHttpAddr.

  • /inspect/system/location: Show disk usage statistics.

  • Support rw2 RAW file formats.

Improved

  • The hotfolder ignores files with 0 bytes. This makes the hotfolder more reliable when used with Mac OS.

  • Also, add a base configuration option to set the wait time in seconds before a file is considered for import by the hotfolder it its file size has not changed since the last check.

  • /api/db_info: When replying the allowed masks, we returned all masks which are available to the user in all pools. This patch changes this behaviour and checks the masks only for the pool the object tested is in, or the pool if explicitly requested in an url parameter. This fixes surprising "no rights" messages while using the group editor.

  • Improved built-in backup: Skip backup of events, chunk insert commands at 10MB. This should make our backup (use for debug purposes only) more reliable.

  • Faster re-index: During re-index we no longer store OBJECT_INDEX events. Such events are stored only if indexing happens regularly, outside an re-index.

  • Indexer: Prevent looping event indexing when disk is full. Since fylr indexes events and events are written when an index error happens, it is possible that we create a loop starting with an indexing error written to db and the following indexing event. If that again results in an indexing error, the writing never stops. Break the loop by looking at the error message ("disk usage exceeded") and in case of indexing type "event", stop the error messaging.

  • /api/search/parse: Do not execute subsearches with search: false. Before,we would execute sub searches although we only wanted to check for errors.

  • /api/user: Allow API maintenance of ldap & sso groups ("automatic"). This patch changes the way we deal with "_automatic_auth" in /api/user. Previously these groups were automatically maintained by the fylr server as ldap users login. Since we need to also set them in a restore, we need to change the api to allow setting these automatic groups. Now, _auth_automatic info can be written as usual. It is now also required to re-sent the _auth_automatic info in order to keep the user group relation active.

  • /api/user: Output _groups for users, this was previously omitted but available in easydb 5. Needed for proper backups of the users in a fylr database.

  • fylr restore: Rename --link-versions to --upload-versions, improved help text for this option.

  • fylr restore: Add --rename-versions. With --rename-versions version names can be changed during uploads. The right names as well as uploaded version names are changed. This is useful to migrate easydb 5 to fylr which use a video.1920p whereas fylr uses a video.1080p version.

  • Do compress .csv and .svg file deliveries.

  • /api/eas?mapping: Compile XML from in database JSON for faster mapping.

  • /eas/api?format=long: Sort all tags by alphabet, so that we have a stable order in the JSON as well as in the XML. format=long uses the original duplicate tag name now (not the indexed name used for de-duplication as stored in the metadata map in the database).

  • /inspect/files/file: Display the interim XML used for metadata mapping.

  • fylr metadata read: When reading the metadata from exiftool this patch uses the xml format instead of exiftool's json format. With that, we are able to map keys which are repeated in the XML to JSON (by adding "-").

  • fylr metadata read: Removed -f xml. Using the XML format from metadata is no longer needed for fylr.

  • /api/db: Disallow multiple tag choices, support group mode. This patch supports group mode for adding tags which are in a tag group of type choice. In this case, existing tags of the same group are automatically removed in the group mode. Also, a new error MultipleChoicesInTaggroup is included which disallows to send multiple tags of the same choice tag group.

  • Login page: Add link to the register page.

  • /inspect/files: Refactored action menu. Also add total count to the search result. Added a Location menu to filter files by location they are stored in. Faster queuing of copy / move jobs. Allow to only copy / move files without locally producing versions.

  • fylr restore: Use priority background for file uploads. By using a lower prio for file production for fylr restore uploaded files, fylr stays interactive during the processing of these files. E.g. it is possible to upload & configure new plugins during a migration.

  • fylr backup / restore: Improved --include-password. Now, it is stored in the backup if users contain password hashes to be used during restore. If not a restore with --include-password fails.

  • /inspect/migration: Include password is added as restore option.

  • JSON, XML export of poly hierarchies: Include _parents in export. Before the parent information was not output at all since we do not include _path in exports. With this patch, the data rendered into _parents is less than before as it is not climbing into all parents, but only the immediate once. In that matter it now behaves like when writing the data where also only the immediate parents can be set.

  • Set Password Page: Improvements for easier usage.

  • fylr restore: Added --log-network (-n) to log network. Before -v would also turn on network logging which could be quite a lot hiding relevant information.

  • /api/search: New tokenizer to split column type text at _ and .. The standard icu_tokenizer does not split at _ and . which is not ideal for strings like filenames. This patch changes the provided icu tokenizer to a custom one, using a regular expression based used in easydb 5.

  • /api/search: Support include_fields _path, and exclude_fields <file>.technical_metadata and <file>.versions.

  • fylr.default.yml: Change wait group settings for smoother processing. Rethought wait groups for exec server tasks. Introduce slow, medium and fast. In the wait group slow, we encode videos and office docs, generally stuff which takes a lot of CPU. The medium group is for image conversion only and the fast group is used for plugins and such. Renaming the wait groups might be incompatible with existing fylr.yml overlays!. Also the queue runners have been increased, we now have 10 parallel high and 18 parallel runners.

  • /api/db, /api/search: Improved object loading with reverse. If the loaded mask do not require reverse objects to be loaded, skip loading them. Before this patch, we unconditionally loaded all reverse objects (even dove deep) which could take a long time for bigger hierarchies. This speeds up cases of text view loading 120x (12s -> 100ms) on our test systems.

  • File worker: If jobs need re-queueing due to a slow exec server, do not set the corresponding file into state error. This was a confusing design choice, as that error was only temporary and easily recovered from.

  • /inspect/objects: When checking the indexed objects, this page renders now a lot faster by omitting the not indexed history objects.

  • File previews: Use highest resolution thumbnail from RAW formats. Before we only used a medium size.

  • /inspect: User select shows more information and groups users.

  • Logins with <system.root>/password as password work now for all user types including ldap and sso.

Fixed

  • /api/session: Properly add the method & time & oauth client id to the session.authenticated object. Before this was only returning unknown as we did not save the info in oauth2_token, but we do know.

  • /api/search: Make _owner.user._id searchable. This requires a re-index.

  • /api/search/parse: Fixed fylr panic for non root users when the query includes a sub search.

  • /api/eas: The file api output used value and print (as exiftool does), but never merged the exiftool output (delivered as keys ending in Num for value) into one item. Instead the output kept the Num which was meant to be an internally used storage format only. This patch fixes the long standing problem and now merged the Num values properly into the actual values delivering different output for value and print where applicable.

  • /api/eas/rput?leave: Properly store technical metadata. If metadata is sent to eas/rput using the request body, we store the metadata which is provided but did not parse the short form "_technical_metadata" out of this. This resulted in a non working zoomer if assets where migrated using eas/rput?leave with generate_metadata unset or set to false.

  • /api/search: This patch is an improvement for simple case like "lea?let". The previous code was splitting this into 2 tokens "lea" und "let" which is wrong as the term stored is "leaflet".

  • /api/search: Fixed standard index settings to work with more than 1 million records.

  • Black previews of remote files in a collection: The cached version of files would include an empty hash for remotely stored versions. That would lead to an invalid HMAC signature and to black preview pics in collections. This requires a full re-index of affected instances.

  • /api/collection/(splice|remove): Fixed not requiring UNLINK right for system.root users when collection objects where to be removed. Also fixed "splice" with empty object list (for all users).

  • /api/event: Fixed format=csv with csv_explode and null values in event.info. If csv explode (of "info") contained null values, the csv output would panic. Also, no longer JSON encode strings in output.

  • Support event indexing for Opensearch 2.12.0.

  • Support metadata of files with huge JSON numbers which overflow Go's float64.

Frontend

Improved

  • Search: button "Top Level Only" replaces the "Flat Hierarchy" option

  • CSV importer: status panel moved to its own tab

  • Detail: linked objects in dropdown are dynamically loaded when missing, starting with max. 100 items

Fixed

  • Search: fixed bug triggering search on when typing after a query tag was deleted

  • Common: multi-input fields (e.g. in datamodel editor) had an issue when application language was changed

  • Group edit: bug fixed when object was already open in editor sidebar

  • Text view: fix improper rendering

  • Collections: fix "compare view"

  • Detail: fix reset of hierarchy lists on selection

  • History: fix display of boolean values, were always shown as deleted

  • Rights: asset version names in rights like in other places

PreviousRelease v6.9.1 (2024-03-06)NextRelease v6.8.5 (2024-02-02)

Last updated