Create payloads (fylr backup)
This page describes the process of creating a local backup of JSON payloads from a source instance
Last updated
This page describes the process of creating a local backup of JSON payloads from a source instance
Last updated
The fylr backup
command performs GET
requests to the API of the source instance, until all relevant base types and records are retrieved and stored in local files.
A basic backup command looks like this:
--server
, --login
and --password
refer to the source server.
--purge
deletes a backup in --dir
in case there is already one. All existing files in this directory are deleted!
The backup over the API only stores the data in local files. The files are only referenced by URLs. Only during restoring, the files are loaded from the source instance. Until the restore is finished, the source instance still needs to be reachable.
Not all events will be migrated from easydb5 to fylr, because they have no purpose in fylr. Also, some events need to be changed.
For a complete overview, see
This is a complete overview of the command line parameters (run fylr backup -h
):
--dir
Target folder for the backup files. If it does not exist, it will be created. If it exists, and --purge
is used, an existing folder is deleted and created new.
this parameter is mandatory!
type: string
--server
API Url of the source instance. The Url must include the API base endpoint, for fylr this is <source url>/api/v1
.
this parameter is mandatory!
type: string
--login
Username of the user in the source instance. It should be a user with root rights or sufficient read rights.
this parameter is mandatory!
type: string
--password
Password of the user in the source instance.
this parameter is mandatory!
type: string
--purge
Defines the mode of the backup (purge or continue).
If this is true
, the complete backup starts from the beginning, and an existing backup folder with the same name is purged.
The parameters --purge
and --continue
are mutually exclusive. Not both can be true
.
If the target folder already exists, this parameter (or --continue
) must be set, otherwise the backup will fail.
type: bool
default: false
--continue
Defines the mode of the backup (purge or continue).
If this is true
, the backup continues from the last point in the manifest.json
file, if a previous backup run was interrupted.
The parameters --purge
and --continue
are mutually exclusive. Not both can be true
.
If the target folder already exists, this parameter (or --purge
) must be set, otherwise the backup will fail.
type: bool
default: false
--size
Deprecated! This parameter is removed in fylr in version v6.18.0.
The target size for objects in payloads. The effective size of the generated payloads is limited by --chunk
.
type: int
minimum: 1
maximum: 1000
default: 100
--chunk
Deprecated! This parameter is removed in fylr in version v6.18.0.
The requested size for objects from the source instance. Can be used to control the size of the responses. It can be lowered if the requests cause timeouts or network problems.
This parameter was renamed to --chunk-size
type: int
minimum: 1
maximum: 1000
default: 100
--chunk-size
The requested size for objects from the source instance. Can be used to control the size of the responses. It can be lowered if the requests cause timeouts or network problems.
type: int
minimum: 1
maximum: 1000
default: 100
--limit
Deprecated! This parameter is removed in fylr in version v6.18.0.
Set this to a number bigger than 0
to limit the number of objects of each objecttype. This can be used to test or debug, and can be combined with --include
to only backup a small sample of the source instance.
This parameter was renamed to --max-count
type: int
minimum: 0
default: 0
--max-count
Set this to a number bigger than 0
to limit the number of objects of each objecttype. This can be used to test or debug, and can be combined with --include
to only backup a small sample of the source instance.
type: int
minimum: 0
default: 0
--max-parallel
Maximum numbers of parallel workers.
This creates more load on the source system.
type: int
minimum: 0
default: 1
--compression
GZIP compression level.
If 0
is selected, there is no compression and the payloads are stored as .json
files. If a value bigger than 0
is selected, the payloads are stored as compressed .json.gz
files.
type: int
minimum: 0
maximum: 9
default: 0
--all-versions
Set to true so that all history versions of the records are requested.
Not to be confused with asset versions (see parameters for fylr restore
)!
type: bool
default: false
--include
If this is a valid non empty regex string, only objecttypes are backupped where the internal objecttype name matches the regex.
type: string
--include-events
Comma separated list of event types.
Specific event types, which are irrelevant for the migration, are never in the backup, even if they are specifically requested.
For now, this includes OBJECT_INDEX
, SESSION_INVALID
, FRONTEND_ERROR
and COLLECTION_OWNER_RIGHTS_ERROR
, but this might be extended in the future.
type: string
default: ""
--retry-max-count
If a request fails with one of the following HTTP status codes that indicate network problems, the request is repeated for a maximum number of times: 502
: Bad Gateway, 503
: Service Unavailable, 504
: Gateway Timeout
type: int
default: 10
--retry-sleep-between
Defines the waiting time in seconds between repeated failed requests.
type: int
default: 30
--client-id
If the source instance uses OAuth2 for user authentication, this is the configured Client ID.
type: string
--client-secret
If the source instance uses OAuth2 for user authentication, this is the configured Client Secret. Can be kept empty if the instance is public.
type: string
--client-token-url
If the source instance uses OAuth2 for user authentication, this is the OAuth2 callback endpoint of the instance. For fylr this is <source url>/api/oauth2/token
.
type: string
--insecure
Set to true
to skip the certificate check for the connection to the source instance.
Only use this option if you can trust the remote server!
type: bool
default: false
--verbose
Set to true
to log debugging info.
type: bool
default: false
--log-network
Set to true
to log the requests and responses.
type: bool
default: false
--log
If this is a valid file path, the log output is written to this file. If this is empty (default), the log output is written to stdout
instead.
type: string
--pretty
Set to true
to save the data in a prettified JSON.
type: bool
default: false
Events of the following types are either not migrated at all, or they will automatically be changed to be imported into fylr.
All event types which are not mentioned here, will be migrated without any type changes.
The event info block will be migrated without any changes. IDs which relate to objects, files, etc., and which are not part of the event info block, are automatically replaced during the restore process.
API_CALL
Events with this type can not be migrated to fylr at all!
API_PROGRESS
Events with this type can not be migrated to fylr at all!
COLLECTION_OWNER_RIGHTS_ERROR
Events with this type can not be migrated to fylr at all!
CUSTOM_TYPE_UPDATER_START
The event type will be changed from CUSTOM_TYPE_UPDATER_START
to CUSTOM_DATA_TYPE_UPDATER
in fylr.
CUSTOM_TYPE_UPDATER_RESPONSE
The event type will be changed from CUSTOM_TYPE_UPDATER_RESPONSE
to CUSTOM_DATA_TYPE_UPDATER
in fylr.
CUSTOM_TYPE_UPDATER_RESPONSE_ERROR
The event type will be changed from CUSTOM_TYPE_UPDATER_RESPONSE_ERROR
to CUSTOM_DATA_TYPE_UPDATER_ERROR
in fylr.
CUSTOM_TYPE_UPDATER_RESPONSE_FAILED
The event type will be changed from CUSTOM_TYPE_UPDATER_RESPONSE_FAILED
to CUSTOM_DATA_TYPE_UPDATER_ERROR
in fylr.
SCHEMA_COMMIT
The event type will be changed from SCHEMA_COMMIT
to DATAMODEL_COMMIT
in fylr.
EXPORT_ASSET
Events with this type can not be migrated to fylr at all!
EXPORT_OBJECT
Events with this type can not be migrated to fylr at all!
ASSET_EXPORT_TRANSPORT_DOWNLOAD
The event type will be changed from ASSET_EXPORT_TRANSPORT_DOWNLOAD
to EXPORT_TRANSPORT
in fylr.
EXPORT_STOPPED
Events with this type can not be migrated to fylr at all!
DOWNLOAD_EXPORT
Events with this type can not be migrated to fylr at all!
ASSET_DOWNLOAD
The event type will be changed from ASSET_DOWNLOAD
to FILE_DOWNLOAD
in fylr.
The basetype of this event will be changed from asset
to file
in fylr.
ASSET_EXPORT_DOWNLOAD
The event type will be changed from ASSET_EXPORT_DOWNLOAD
to FILE_DOWNLOAD
in fylr.
The basetype of this event will be changed from asset
to file
in fylr.
SUGGEST_INDEX_DONE
Events with this type can not be migrated to fylr at all!
SUGGEST_INDEX_FAILED
Events with this type can not be migrated to fylr at all!
JANITOR_SESSION_PURGE
The event type will be changed from JANITOR_SESSION_PURGE
to JANITOR
in fylr.
MAPPING_DELETE
The basetype of this event will be set to mapping
in fylr. In easydb5, there was no basetype for this event type.
MAPPING_INSERT
The basetype of this event will be set to mapping
in fylr. In easydb5, there was no basetype for this event type.
MAPPING_UPDATE
The basetype of this event will be set to mapping
in fylr. In easydb5, there was no basetype for this event type.
RESOURCE_NOT_AVAILABLE
Events with this type can not be migrated to fylr at all!
SERVER_START
Events with this type can not be migrated to fylr at all!
SERVER_SHUTDOWN
Events with this type can not be migrated to fylr at all!
SESSION_INVALID
Events with this type can not be migrated to fylr at all!
SUGGEST_INDEX_DONE
Events with this type can not be migrated to fylr at all!
SUGGEST_INDEX_FAILED
Events with this type can not be migrated to fylr at all!
USER_CREATED
The event type will be changed from USER_CREATED
to USER_INSERT
in fylr.
The basetype of this event will be set to user
in fylr. In easydb5, there was no basetype for this event type.
USER_LOGIN
The basetype of this event will be set to user
in fylr. In easydb5, there was no basetype for this event type.
USER_LOGOUT
The basetype of this event will be set to user
in fylr. In easydb5, there was no basetype for this event type.
LOGIN_FAILED
Events with this type can not be migrated to fylr at all!
USER_ACCEPTED_MESSAGE
Events with this type can not be migrated to fylr at all!
WEBHOOK_OK
The event type will be changed from WEBHOOK_OK
to WEBHOOK
in fylr.