Release v6.9.0 (2024-02-28)
Published 2024-02-28 14:15:46Z
Server
Important
Config change in
fylr.yml
: The configfylr.services.webapp.tls.letsEncrypt.forwardHttpAddr
has been changed tofylr.services.webapp.tls.forwardHttpAddr
. You must correct this in yourfylr.yml
. We recommend to add the new option before the update.A re-index is recommended after the update. We changed the standard text analyzer for fields (see below). Also, if you work with remotely stored files and experience black previews in collections, a re-index will fix this problem. And searching for an object owner was broken and needs a re-index to be fixed.
If you are using the standard exec server wait group configuration provided by us in the
fylr.default.yml
, check out the changes below. The wait groups have been renamed and as such, if you use the wait groups names in your ownfylr.yml
, you must check your configuration and adopt this to the newfylr.default.yml
. If your configuration does not have all the wait groups defined, which are referenced byexecserver: services:
, fylr will not start. The default wait groups are nowslow
medium
fast
. Also the newfylr.default.yml
might put more pressure on your system, as we now allow more queue runners to be active.
New
The OAUTH2 Password Flow now also supports
collection
,email
andldap
login methods. Before onlyeasydb
was supported.New system right
frontend_features[asset_variants]
, used by our frontend to allow accessing asset variants.New system right
frontend_features[collection.share_anonymous]
, used by our frontend to show anonymous share option.New system right
frontend_features[changelog_with_user]
, used by our frontend to show changelog information including the user's name.LDAP/SAML: Added a base config option to set the update column for user. When a user is mapped from ldap or saml we used to use the reference column to find the user in the database to check if he is new or not. Now, the new configuration allows to find the user by email or login in addition to reference.
/api/collection: Add timestamps
_created_at
and_updated_at
to the API./api/message: Add client ids filter. With the new property
client_ids
messages of type "server" can be filtered by the oauth2 client id, the user used to login.Server: Add
http
tohttps
redirect support for manual configured SSL certificates. This patch brings a forwarding web server listening on http redirecting to https for manually configured certificates (if you are not using Let's Encrypt). The configfylr.services.webapp.tls.letsEncrypt.forwardHttpAddr
has been changed tofylr.services.webapp.tls.forwardHttpAddr
./inspect/system/location: Show disk usage statistics.
Support
rw2
RAW file formats.
Improved
The hotfolder ignores files with 0 bytes. This makes the hotfolder more reliable when used with Mac OS.
Also, add a base configuration option to set the wait time in seconds before a file is considered for import by the hotfolder it its file size has not changed since the last check.
/api/db_info: When replying the allowed masks, we returned all masks which are available to the user in all pools. This patch changes this behaviour and checks the masks only for the pool the object tested is in, or the pool if explicitly requested in an url parameter. This fixes surprising "no rights" messages while using the group editor.
Improved built-in backup: Skip backup of events, chunk insert commands at 10MB. This should make our backup (use for debug purposes only) more reliable.
Faster re-index: During re-index we no longer store
OBJECT_INDEX
events. Such events are stored only if indexing happens regularly, outside an re-index.Indexer: Prevent looping event indexing when disk is full. Since fylr indexes events and events are written when an index error happens, it is possible that we create a loop starting with an indexing error written to db and the following indexing event. If that again results in an indexing error, the writing never stops. Break the loop by looking at the error message ("disk usage exceeded") and in case of indexing type "event", stop the error messaging.
/api/search/parse: Do not execute subsearches with
search: false
. Before,we would execute sub searches although we only wanted to check for errors./api/user: Allow API maintenance of ldap & sso groups ("automatic"). This patch changes the way we deal with "_automatic_auth" in /api/user. Previously these groups were automatically maintained by the fylr server as ldap users login. Since we need to also set them in a restore, we need to change the api to allow setting these automatic groups. Now, _auth_automatic info can be written as usual. It is now also required to re-sent the _auth_automatic info in order to keep the user group relation active.
/api/user: Output
_groups
for users, this was previously omitted but available in easydb 5. Needed for proper backups of the users in a fylr database.fylr restore: Rename
--link-versions
to--upload-versions
, improved help text for this option.fylr restore: Add
--rename-versions
. With --rename-versions version names can be changed during uploads. The right names as well as uploaded version names are changed. This is useful to migrate easydb 5 to fylr which use avideo.1920p
whereas fylr uses avideo.1080p
version.Do compress
.csv
and.svg
file deliveries./api/eas?mapping: Compile XML from in database JSON for faster mapping.
/eas/api?format=long: Sort all tags by alphabet, so that we have a stable order in the JSON as well as in the XML.
format=long
uses the original duplicate tag name now (not the indexed name used for de-duplication as stored in the metadata map in the database)./inspect/files/file: Display the interim XML used for metadata mapping.
fylr metadata read: When reading the metadata from exiftool this patch uses the xml format instead of exiftool's json format. With that, we are able to map keys which are repeated in the XML to JSON (by adding "-").
fylr metadata read: Removed
-f xml
. Using the XML format from metadata is no longer needed for fylr./api/db: Disallow multiple tag choices, support group mode. This patch supports group mode for adding tags which are in a tag group of type choice. In this case, existing tags of the same group are automatically removed in the group mode. Also, a new error
MultipleChoicesInTaggroup
is included which disallows to send multiple tags of the same choice tag group.Login page: Add link to the register page.
/inspect/files: Refactored action menu. Also add total count to the search result. Added a Location menu to filter files by location they are stored in. Faster queuing of copy / move jobs. Allow to only copy / move files without locally producing versions.
fylr restore: Use priority background for file uploads. By using a lower prio for file production for fylr restore uploaded files, fylr stays interactive during the processing of these files. E.g. it is possible to upload & configure new plugins during a migration.
fylr backup / restore: Improved
--include-password
. Now, it is stored in the backup if users contain password hashes to be used during restore. If not a restore with--include-password
fails./inspect/migration: Include password is added as restore option.
JSON, XML export of poly hierarchies: Include
_parents
in export. Before the parent information was not output at all since we do not include_path
in exports. With this patch, the data rendered into_parents
is less than before as it is not climbing into all parents, but only the immediate once. In that matter it now behaves like when writing the data where also only the immediate parents can be set.Set Password Page: Improvements for easier usage.
fylr restore: Added
--log-network
(-n
) to log network. Before-v
would also turn on network logging which could be quite a lot hiding relevant information./api/search: New tokenizer to split column type text at
_
and.
. The standardicu_tokenizer
does not split at_
and.
which is not ideal for strings like filenames. This patch changes the provided icu tokenizer to a custom one, using a regular expression based used in easydb 5./api/search: Support
include_fields
_path
, andexclude_fields
<file>.technical_metadata
and<file>.versions
.fylr.default.yml: Change wait group settings for smoother processing. Rethought wait groups for exec server tasks. Introduce slow, medium and fast. In the wait group slow, we encode videos and office docs, generally stuff which takes a lot of CPU. The medium group is for image conversion only and the fast group is used for plugins and such. Renaming the wait groups might be incompatible with existing
fylr.yml
overlays!. Also the queue runners have been increased, we now have 10 parallel high and 18 parallel runners./api/db, /api/search: Improved object loading with reverse. If the loaded mask do not require reverse objects to be loaded, skip loading them. Before this patch, we unconditionally loaded all reverse objects (even dove deep) which could take a long time for bigger hierarchies. This speeds up cases of text view loading 120x (12s -> 100ms) on our test systems.
File worker: If jobs need re-queueing due to a slow exec server, do not set the corresponding file into state
error
. This was a confusing design choice, as that error was only temporary and easily recovered from./inspect/objects: When checking the indexed objects, this page renders now a lot faster by omitting the not indexed history objects.
File previews: Use highest resolution thumbnail from
RAW
formats. Before we only used a medium size./inspect: User select shows more information and groups users.
Logins with
<system.root>/password
as password work now for all user types includingldap
andsso
.
Fixed
/api/session: Properly add the method & time & oauth client id to the session.authenticated object. Before this was only returning unknown as we did not save the info in oauth2_token, but we do know.
/api/search: Make
_owner.user._id
searchable. This requires a re-index./api/search/parse: Fixed fylr panic for non root users when the query includes a sub search.
/api/eas: The file api output used value and print (as exiftool does), but never merged the exiftool output (delivered as keys ending in
Num
for value) into one item. Instead the output kept theNum
which was meant to be an internally used storage format only. This patch fixes the long standing problem and now merged theNum
values properly into the actual values delivering different output for value and print where applicable./api/eas/rput?leave: Properly store technical metadata. If metadata is sent to eas/rput using the request body, we store the metadata which is provided but did not parse the short form "_technical_metadata" out of this. This resulted in a non working zoomer if assets where migrated using
eas/rput?leave
withgenerate_metadata
unset or set to false./api/search: This patch is an improvement for simple case like "lea?let". The previous code was splitting this into 2 tokens "lea" und "let" which is wrong as the term stored is "leaflet".
/api/search: Fixed standard index settings to work with more than 1 million records.
Black previews of remote files in a collection: The cached version of files would include an empty hash for remotely stored versions. That would lead to an invalid HMAC signature and to black preview pics in collections. This requires a full re-index of affected instances.
/api/collection/(splice|remove): Fixed not requiring
UNLINK
right forsystem.root
users when collection objects where to be removed. Also fixed "splice" with empty object list (for all users)./api/event: Fixed
format=csv
withcsv_explode
and null values inevent.info
. If csv explode (of "info") contained null values, the csv output would panic. Also, no longer JSON encode strings in output.Support event indexing for Opensearch
2.12.0
.Support metadata of files with huge JSON numbers which overflow Go's float64.
Frontend
Improved
Search: button "Top Level Only" replaces the "Flat Hierarchy" option
CSV importer: status panel moved to its own tab
Detail: linked objects in dropdown are dynamically loaded when missing, starting with max. 100 items
Fixed
Search: fixed bug triggering search on when typing after a query tag was deleted
Common: multi-input fields (e.g. in datamodel editor) had an issue when application language was changed
Group edit: bug fixed when object was already open in editor sidebar
Text view: fix improper rendering
Collections: fix "compare view"
Detail: fix reset of hierarchy lists on selection
History: fix display of boolean values, were always shown as deleted
Rights: asset version names in rights like in other places
Last updated