> For the complete documentation index, see [llms.txt](https://docs.fylr.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.fylr.io/for-system-administrators/migration/backup.md).

# Create payloads (fylr backup)

This page describes the process of creating a local backup of JSON payloads from a source instance

The `fylr backup` command performs `GET` requests to the API of the source instance, until all relevant base types and records are retrieved and stored in local files.

A basic backup command looks like this:

```
fylr backup \
  --server '<source-url>/api/v1/' \
  --login 'root' \
  --password '<cleartext>' \
  --dir '<instance folder>/' \
  --compression 9 \
  --purge
```

`--server`, `--login` and `--password` refer to the source server.

The base configuration is copied as it is, the SAML settings included. From fylr **6.35.0** neither `fylr backup` nor `fylr restore` fetches the identity provider's metadata, so an identity provider that cannot be reached from where the backup runs does not stop it.

{% hint style="warning" %}
`--purge` deletes a backup in `--dir` in case there is already one. All existing files in this directory are deleted!
{% endhint %}

{% hint style="warning" %}
By default the backup over the API only stores the data in local files; the files themselves are only referenced by URL and are loaded from the source instance during the restore, so until the restore is finished the source instance still needs to be reachable.

Use [`--include-files`](#include-files) to pack the file bytes into the backup instead, making it self-contained.
{% endhint %}

{% hint style="warning" %}
Not all events will be migrated from easydb5 to fylr, because they have no purpose in fylr. Also, some events need to be changed.

For a complete overview, see [below](/for-system-administrators/migration/limitations.md#migration-of-events)
{% endhint %}

## Parameters

This is a complete overview of the command line parameters (run `fylr backup -h`):

```
Usage: fylr backup [flags]

Backup ez5 or FYLR via API

Flags:

-h, --help                      Show context-sensitive help.
-v, --verbose                   Set to true, to show additional info.
-n, --log-network               Set to true, to log all network traffic.
    --server=STRING             Source url (overwrites URL of source instance from config)
-l, --login=STRING              If --server is set, use as login. Make sure to use the system root user to connect if used together with --purge.
-p, --password=STRING           If --server is set, use as password
    --client-id=""              If --server is set, use as OAUTH2 client ID
    --client-secret=""          If --server is set, use as OAUTH2 client secret
    --client-token-url=""       If --server is set, use as OAUTH2 token url. If not set, this is derived from --server
    --insecure                  Set to true, to not verify the server's certificate chain and host name
    --log=STRING                Set output to logfile
    --purge                     For backup: set to true, to purge the target directory. For restore: set to true, to purge the target and copy the datamodel. The current password of the user used for the login will be set for the system root user.
    --continue                  Set to true, to continue.
    --verify                    If set, verify payloads of an existing backup (same what --continue does at the beginning).
    --chunk-size=100            chunk size for fetching/pushing data.
    --include-events=""         Comma separated list of event types. Use "-" to skip backup/restoring of events. Empty string will backup/restore all known event types.
    --max-parallel=1            Maximum numbers of parallel workers. 0 uses the number of available CPUs. This creates more load on the source / target system. Defaults to "1" (only one parallel process).
-d, --dir=STRING                Target directory.
    --compression=0             0: no compression, 1: speed, 9: best.
    --all-versions              Set to true, to request all versions of an object.
    --include-deleted           By default soft-deleted objects are skipped. Set this to include them in the backup, so that links pointing at a deleted object can be restored faithfully (paired with restore's --include-deleted-linked).
    --include=""                Filter regexp to include objecttypes.
    --maximum-count=0           Limit records. Set to 0 for unlimited.
    --retry-max-count=10        Number of retries for failed requests with network problems.
    --retry-sleep-between=30    Wait time in seconds between retries for failed requests.
    --pretty                    Output pretty JSON.
    --file-url-expire-days=365  How many days the file urls stored in the backup stay valid. The source signs every file url it renders for the backup, and the restore hands those urls to the target, which fetches the bytes from them. The signature is what authorizes that fetch, so the backup is only restorable while it holds. The horizon is baked in when the backup is taken and cannot be extended afterwards — a backup whose urls expired has to be taken again. Set 0 to store unsigned urls, which only restore when the source grants the target access to the files by other means.
    --include-files=""          How to store file bytes in the backup. Empty (default): store only the file URLs, the restore fetches the bytes from the source (which must stay reachable). "original": pack each non-leave file's original bytes into the backup directory (files/<id-floored-to-1e6>/<id-floored-to-1e3>/<id>-<filename>, the filename sanitized like an export filename and shortened in the middle to keep <id>-<filename> within 100 bytes; the restore uploads the file under its original name), so the backup is self-contained and byte-for-byte and the source need not stay reachable; renditions (preview versions) are not packed, the restore regenerates them, so such a backup cannot be restored with --upload-versions. "with-versions": also pack the renditions, so that a restore with --upload-versions uploads them from the backup (byte-identical) instead of regenerating them. Files kept on the remote (leave) are never packed; their upstream URL is preserved.
```

### `--dir`

Target folder for the backup files. If it does not exist, it will be created. If it exists, and `--purge` is used, an existing folder is deleted and created new.

* this parameter is **mandatory**!
* type: `string`

### `--server`

API Url of the source instance. The Url must include the API base endpoint, for fylr this is `<source url>/api/v1`.

* this parameter is **mandatory**!
* type: `string`

### `--login`

Username of the user in the source instance. It should be a user with root rights or sufficient read rights.

* this parameter is **mandatory**!
* type: `string`

### `--password`

Password of the user in the source instance.

* this parameter is **mandatory**!
* type: `string`

### `--purge`

Defines the mode of the backup (purge or continue).

If this is `true`, the complete backup starts from the beginning, and an existing backup folder with the same name is purged.

{% hint style="warning" %}
The parameters `--purge` and `--continue` are mutually exclusive. Not both can be `true`.

If the target folder already exists, this parameter (or `--continue`) must be set, otherwise the backup will fail.
{% endhint %}

* type: `bool`
* default: `false`

### `--continue`

Defines the mode of the backup (purge or continue).

If this is `true`, the backup continues from the last point in the `manifest.json` file, if a previous backup run was interrupted.

{% hint style="warning" %}
The parameters `--purge` and `--continue` are mutually exclusive. Not both can be `true`.

If the target folder already exists, this parameter (or `--purge`) must be set, otherwise the backup will fail.
{% endhint %}

* type: `bool`
* default: `false`

### `--chunk-size`

The requested size for objects from the source instance. Can be used to control the size of the responses. It can be lowered if the requests cause timeouts or network problems.

* type: `int`
* minimum: `1`
* maximum: `1000`
* default: `100`

### `--maximum-count`

Set this to a number bigger than `0` to limit the number of objects of each objecttype. This can be used to test or debug, and can be combined with `--include` to only backup a small sample of the source instance.

* type: `int`
* minimum: `0`
* default: `0`

### `--max-parallel`

Maximum numbers of parallel workers.

{% hint style="info" %}
`0` uses the number of available CPUs.

Defaults to `1` (only one parallel process).
{% endhint %}

{% hint style="warning" %}
This creates more load on the source system.
{% endhint %}

* type: `int`
* minimum: `0`
* default: `1`

### `--compression`

GZIP compression level.

If `0` is selected, there is no compression and the payloads are stored as `.json` files. If a value bigger than `0` is selected, the payloads are stored as compressed `.json.gz` files.

* type: `int`
* minimum: `0`
* maximum: `9`
* default: `0`

### `--all-versions`

Set to true so that all history versions of the records are requested.

{% hint style="warning" %}
Not to be confused with asset versions (see parameters for `fylr restore`)!
{% endhint %}

* type: `bool`
* default: `false`

### `--include`

If this is a valid non empty regex string, only objecttypes are backupped where the internal objecttype name matches the regex.

* type: `string`

### `--include-deleted`

{% hint style="info" %}
This parameter is available in fylr from version **6.33.0**.
{% endhint %}

By default, linked objects that are soft-deleted (their latest version carries `_latest_version_deleted_at`) are skipped from the backup. Set this parameter to `true` to include them as top-level entries in the payload, in their soft-deleted state.

Links pointing at a soft-deleted target are always written to the backup as a `lookup:_id` wrapper carrying `_latest_version_deleted_at` next to `_system_object_id` and `_allow_defer`.

For a faithful round-trip (soft-deleted targets restored as they are, with their incoming links intact) use this together with `fylr restore --include-deleted-linked`. If the backup was done without `--include-deleted` and restored with `--include-deleted-linked`, the wrappers are kept but their lookups defer, so the links are stored as `_purged_or_deferred`.

{% hint style="info" %}
Use this paired with the `fylr restore` parameter `--include-deleted-linked`
{% endhint %}

* type: `bool`
* default: `false`

### `--file-url-expire-days`

{% hint style="info" %}
This parameter is available in fylr from version **6.35.0**.
{% endhint %}

A backup that stores file **URLs** (the default — see [`--include-files`](#include-files)) relies on those URLs still being served when the restore runs: the target fetches the bytes from them without a session of its own, and what authorizes that fetch is the **signature** the source puts in the URL.

That signature normally lasts `system.file_url_expire.default_days` — one day by default — which is far shorter than the useful life of a backup. `fylr backup` therefore asks the source for a horizon of its own, **365 days by default**. The `system.file_url_expire.max_days` cap does not apply to a caller holding the `system.root` system right, so this needs no configuration change on the source instance.

Set `0` to store unsigned URLs. A restore can then only fetch them if the source grants the target access to the files some other way.

{% hint style="warning" %}
The horizon is fixed when the backup runs and **cannot be extended afterwards**. Once it has passed, the source no longer serves the URLs and [`fylr restore`](/for-system-administrators/migration/restore.md) refuses the backup before it touches the target. The remedy is a new backup — or one taken with [`--include-files`](#include-files), whose bytes never expire.
{% endhint %}

Files whose bytes are packed into the backup by `--include-files` carry no URL and are unaffected. So are the files fylr stores for a user picture, a pool watermark, custom data on a group or objecttype, and base config values: those are never signed, and the backup packs their bytes regardless of this setting.

* type: `int` (days)
* default: `365`

### `--include-files`

{% hint style="info" %}
This parameter is available in fylr from version **6.34.0**.
{% endhint %}

By default a backup only stores the file **URLs**, and the restore fetches the bytes from the source instance (which therefore has to stay reachable until the restore finishes). Set `--include-files` to pack the file **bytes** into the backup itself, under a `files/` directory, so the restore uploads them from the local backup and no longer depends on the source instance. It takes a mode:

* **empty** (default): store only the file URLs (the behaviour described above).
* **`original`**: pack each file's **original** bytes, but not its renditions (preview versions). On restore the target regenerates the renditions itself, so an `--include-files=original` backup **cannot** be restored with [`--upload-versions`](/for-system-administrators/migration/restore.md#upload-versions) — that option uploads the source's renditions, but they are not in the backup and the source may be gone, so the restore rejects the combination.
* **`with-versions`**: pack the originals **and** their renditions, so that a restore with [`--upload-versions`](/for-system-administrators/migration/restore.md#upload-versions) uploads the renditions from the backup byte-identical instead of regenerating them. Without `--upload-versions` the target still regenerates them (exactly as for a URL-only backup) — the flag stays explicit. This is the basis for a fully self-contained, byte-for-byte backup.

Files stored *leave on remote* are never packed: the backup keeps their real upstream URL (it requests `files_real_url=1` on the `/db` list, which needs the *root* system right), and the restore re-creates them as references to that upstream. Their renditions are ordinary local files, so with `with-versions` a leave original's renditions are packed and uploaded from the backup too.

From version **6.35.1**, a packed file is named `<id>-<filename>` with its original filename sanitized like an export filename: `/ \ : * ? " < > |`, `#`, `%` and control characters become `_`, so the path works wherever it ends up in a URL. A name that would make `<id>-<filename>` longer than 100 bytes loses its middle to `...`, keeping its start, its end and its extension, so the backup's paths also stay within the limits of Windows. A rendition is named `<id>-<original>-<version>.<extension>` next to its original. The name is only for orientation: the restore finds each file by the `path` the payload stores for it and uploads it under its original filename. A backup made by an earlier version, whose packed names may still contain `#`, `?` or `%`, restores from 6.35.1 on as it is.

The backup form under `/inspect/migration` offers this as an **Include Files** selector, and its backup viewer browses the nested `files/` directories.

* type: `string` (one of `""`, `original`, `with-versions`)
* default: `""`

### `--include-events`

Comma separated list of event types.

{% hint style="info" %}
Use `--include-events=-` to skip backup of events.

If this parameter is unset, all events are backupped
{% endhint %}

{% hint style="warning" %}
Specific event types, which are irrelevant for the migration, are never in the backup, even if they are specifically requested.

For now, this includes `OBJECT_INDEX`, `SESSION_INVALID`, `FRONTEND_ERROR` and `COLLECTION_OWNER_RIGHTS_ERROR`, but this might be extended in the future.
{% endhint %}

* type: `string`
* default: `""`

### `--retry-max-count`

If a request fails with one of the following HTTP status codes that indicate network problems, the request is repeated for a maximum number of times: `502`: Bad Gateway, `503`: Service Unavailable, `504`: Gateway Timeout

* type: `int`
* default: `10`

### `--retry-sleep-between`

Defines the waiting time in seconds between repeated failed requests.

* type: `int`
* default: `30`

### `--client-id`

If the source instance uses OAuth2 for user authentication, this is the configured Client ID.

* type: `string`

### `--client-secret`

If the source instance uses OAuth2 for user authentication, this is the configured Client Secret. Can be kept empty if the instance is public.

* type: `string`

### `--client-token-url`

If the source instance uses OAuth2 for user authentication, this is the OAuth2 callback endpoint of the instance. For fylr this is `<source url>/api/oauth2/token`.

* type: `string`

### `--insecure`

Set to `true` to skip the certificate check for the connection to the source instance.

{% hint style="warning" %}
Only use this option if you can trust the remote server!
{% endhint %}

* type: `bool`
* default: `false`

### `--verbose`

Set to `true` to log debugging info.

* type: `bool`
* default: `false`

### `--log-network`

Set to `true` to log the requests and responses.

* type: `bool`
* default: `false`

### `--log`

If this is a valid file path, the log output is written to this file. If this is empty (default), the log output is written to `stdout` instead.

* type: `string`

### `--pretty`

Set to `true` to save the data in a prettified JSON.

* type: `bool`
* default: `false`