# Migration with map to local storage This method can save time and does not need a separate server. easydb5 is only needed to extract metadata, but not to extract asset files, which are used from storage directly as is. The following example migrates all from to . (IP address 1.2.3.4) ## Checks and Requirements The server should have enough **RAM** available (at least 8 GB free when easydb is running, more is recommended during image preview generation).
Is there enough free storage? * for doubling the indexes and SQL-DB of easydb (fylr will have its own) * for doubling the preview images of easydb (fylr is recommended to generate its own) * 20+ GB for fylr container versions
A DNS subdomain for fylr Have a DNS entry, in our example *fylr.example.com*, pointing to the same IP address as the easydb (in our example *easydb.example.com*).
Prevent name collisions Check indexes will not collide ***by name*** between eaydb5 and fylr: ```bash docker exec easydb-server curl http://easydb-elasticsearch:9200/_cat/indices ```

... good if they do not start with "fylr".

Allow passwords to be transferred In standard configuration, easydb does not output account passwords for migration, as a security feature. To make a full migration, you have to change that setting temporarily, at least for the step "*extract metadata"* ("backup"). The setting in easydb5 is: ```yaml server: api: user: include_password: true ``` ... e.g. in `easydb-server.yml` . Restart container `easydb-server` to make active.\ For more information see
## 1. fylr installation To save resources like RAM, we use easydb's infrastructure
1.a use docker compose for fylr installation ```bash apt-get install docker-compose-plugin mkdir /srv/fylr ; cd /srv/fylr mkdir -p config/fylr assets backups migration postgres sqlbackups chown 1000 assets backups migration chown 999 postgres sqlbackups curl https://raw.githubusercontent.com/programmfabrik/fylr-gitbook/main/_assets/fylr.yml -o config/fylr/fylr.yml curl https://raw.githubusercontent.com/programmfabrik/fylr-gitbook/main/_assets/maintain -o maintain curl https://raw.githubusercontent.com/programmfabrik/fylr-gitbook/main/_assets/docker-compose.yml -o docker-compose.yml chmod a+x maintain vi docker-compose.yml # see below for content docker compose up -d; docker compose logs -f ``` Stop outputting log messages with `Ctrl`-`c` if seen enough
1.b edit /srv/fylr/docker-compose.yml * Turn the whole Opensearch section into just comments * Change the network to `easydb_default` * Change fylr port * Check the volume paths, left of the `:`,\ so .e.g. `/srv/easydb/eas/lib/assets/orig`. * The example below has two volumes for two easydb partitions. Your easydb may have more partitions. Create one fylr volume per easydb partition. ... so that you have something like this example: 
services:
  # opensearch:
  # [...not shown here: more opensearch as comments ...]
  
  postgresql:
    image: postgres:18
    container_name: postgresql
    restart: always
    shm_size: 1g
    environment:
      POSTGRES_DB: 'fylr'
      POSTGRES_USER: 'fylr'
      POSTGRES_PASSWORD: 'fylr'
      PGDATA: /var/lib/postgresql/data/pgdata
    volumes:
      - "/srv/fylr/postgres:/var/lib/postgresql/data"
      - "/srv/fylr/sqlbackups:/mnt"
    command: >
      -c work_mem=64MB
      -c maintenance_work_mem=32MB
      -c max_wal_size=512MB
      -c max_connections=100
    networks:
      - easydb_default
    logging:
      driver: "journald"
      
  fylr:
    image: docker.fylr.io/fylr/fylr:latest
    hostname: fylr.localhost
    container_name: fylr
    restart: always
    ports:
      - "127.0.0.1:91:91"
    networks:
      - easydb_default
    volumes:
      - "/srv/easydb/eas/lib/assets/orig:/mnt/orig_early:ro"
      - "/srv/easydb/eas/lib/assets/dest:/mnt/dest_early:ro"
      - "/srv/fylr/config/fylr:/fylr/config"
      - "/srv/fylr/assets:/srv"
      - "/srv/fylr/backups:/fylr/files/backups"     # /inspect/system/backups/ and /backupmanager
      - "/srv/fylr/migration:/fylr/files/migration" # /inspect/migration/
    logging:
      driver: "journald"

networks:
  easydb_default:
    external: true
1.c Adjust /srv/fylr/config/fylr/fylr.yml ```yaml fylr+: allowpurge: true externalURL: "https://fylr.example.com" [...] elastic+: addresses: - "http://easydb-elasticsearch:9200" [...] services+: webapp+: addr: ":91" tls: ``` ... and of course unique `encryptionKey` and `signingSecret` .
1.d Start postgres and fylr ```bash docker compose up -d; docker compose logs -f ``` Stop outputting log messages with `Ctrl`-`c` if you have seen enough.
1.e Allow purging fylr in the frontend See the screenshot:
## 2. Apache and https certificate for fylr
2.a Assuming certbot with LetsEncrypt is OK for you, do... Add a minimal VirtualHost for the fylr to your Apache configuration: ``` ServerName fylr.example.com ``` ... replace the IP Address 1.2.3.4 and of course the domain name. Install and use certbot: *(unless you have another method to obtain a https-certificate for fylr)* ``` apache2ctl graceful apt install snapd snap install --classic certbot certbot --apache # in the shown choice: select the fylr domain ``` After certbot improved your Apache configuration, add the configuration to show fylr: ``` ServerName fylr.example.com RewriteEngine on RewriteCond %{SERVER_NAME} =fylr.example.com RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent] ServerName fylr.example.com ProxyPreserveHost On ProxyPass / http://127.0.0.1:91/ ProxyPassReverse / http://127.0.0.1:91/ SSLCertificateFile /etc/letsencrypt/live/fylr.example.com/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/fylr.example.com/privkey.pem Include /etc/letsencrypt/options-ssl-apache.conf ``` Make sure that you have the correct number of virtual hosts: 4 (easydb 80, easydb port 443, fylr 80, fylr 443) across all Apache config files. Certbot creates an additional config file and double VirtualHosts (e.g. a fifth one) can cause hard to find errors. Make sure to now also use explicit **IP address** (not `*`) and **ServerName** in the easydb configuration for apache: ``` ServerName easydb.example.com [...] ServerName easydb.example.com ```
2.b Log into your fylr () as root with password admin. 2.c Change the password of root to a secure one. 2.d Check that fylr is at least Version 6.26.0
## 3. Extract metadata from easydb5 3.a. Surf to \*\* 3.b. expand the Paragraph **`Create backup`** (by clicking the triangle)
3.c. Fill in at least the following values: * `URL of server` : Fill in your equivalent of `https://easydb.example.com` * `Login`: `root` * `Password`: password of easydb's root account * `OAuth2`: uncheck this box, it is only needed to extract from fylr * `Max Parallel`: To not slow your easydb down, choose a number that is half or less of the available CPU cores. * `Purge`: you can leave this on, it does not affect easydb or fylr. (It was added to overwrite backup files, but currently it creates a new backup anyway)
Click **`Backup`** . This can take from one minute to several hours depending on your data. See [here](/for-system-administrators/migration/inspect.md) if you want more information about this process. ## 4. Inject metadata into fylr 4.a. Surf to \*\* (log in as root) 4.b. expand the Paragraph **`Restore backup`** (by clicking the triangle)
4.c. Fill in at least the following values: * `Backup` : choose the backup that you created above * `URL` : Fill in your equivalent of `https://fylr.example.com` * `Login`: `root` * `Password`: password of fylr's root account * `File Mode`: choose `Use files from source - rput_leave (bulk)` * `File Version`: use the default `original` * `Copy file preview versions`: Enable this box. * `Include Password`: Can be turned off for test runs. When turned on, passwords are included. But for that, the above backup has to be made with a less secure easydb configuration active. See * `Include Events`: Turn on if you want to transfer the events that were recorded in easydb. Considered not needed unless you know you want it. * `OAuth2`: This box has to be enabled. * `OAuth2 Client Id`: leave the default fylr-web-frontend * `Max Parallel`: Choose less than the number of CPU cores. * `Purge or Continue`: `Purge` This will overwrite fylr's contents with easydb, which is the whole point.\ `Continue` is useful if your previous attempt aborted with a timeout or network error and should be continued.
Click **`Restore`** . This can take from a few minutes to many hours depending on your data. * It will continue if you close your browser. * You can come back to it via **https\://**fylr.example.com**/inspect/migration** * And also directly via **https\://**fylr.example.com**/inspect/migration/**mymigrationname See [here](/for-system-administrators/migration/inspect.md#restore) if you want more information about this process. When the restore/inject is done, a reindex will be done for fylr, so it will take a while until you can see the data in the webfrontend . ## 5. Teach fylr where to find easydb assets on disk
5.a Collect the "Remote URL Prefixes" of all easydb partitions Look into **https\://**fylr.example.com**/inspect/files/** (log in as root) * Click on an ID number of a *version* file (the `Version` column has `small` or `full` or others but *not* `ORIGINAL`). * note the content of field `Remote URL`, it might contain e.g. `https://easydb.example.com/eas/partitions-inline/2/0/1270/1270/4839d32e5c8ecca1` * Click on an ID number of an *original* file (`Version` column *has* `ORIGINAL`). * Also for this asset note the content of field `Remote URL`, it might contain e.g. `https://easydb.example.com/eas/partitions-inline/1/0/1270/1270/acda0f0f5982bb64` * Next you need to cut off the last parts of the Remote URLs in your notes, so that only the URL prefix remains, which is what you need. The prefix ends before the single zero. So in our example the prefixes are: ``` https://easydb.example.com/eas/partitions-inline/2/ (for Versions and) https://easydb.example.com/eas/partitions-inline/1/ (for Originals) ``` * Count the number of partitions in **https\://**easydb.example.com**/servermanager**.\ For that you have to choose easydb Asset Server (EAS) at the top:
* Continue searching through different files as above until you have the URL prefix for each partition (but often there are just two).
5.b Configure the locations Surf to **https\://**fylr.example.com**/locationmanager** (log in as root) Create the following two: * Fylr location `easydb originals` * **Read Only**: enable this * **Directory** (in container) `/mnt/orig_early` * **Remote URL Prefix** example: Use here one of the prefixes you collected in 5.a.\ \&#xNAN;*(Do not confuse this with the **Prefix** field. Use **Remote URL Prefix**.)* * Fylr location `easydb versions` * **Directory** (in container) `/mnt/dest_early` * **Remote URL Prefix**: As above, use one of the collected Remote URL Prefixes.\ Example: `https://easydb.example.com/eas/partitions-inline/2/` * If you have enough free storage space to double all preview versions, then set this location to `Read Only`. Then none of them will be deleted. Otherwise set it as Default Location for `versions` . Then easydb previews will be deleted as they are replaced with fylr previews. `Read Only` is safer, especially if you still want to use easydb, and thus recommended. * In case your easydb has more than two partitions, you may have to add more fylr locations.
5.c Let fylr use the asset files of easydb5 directly from disk, without asking easydb * Go to **https\://**fylr.example.com**/inspect/files/** * Select `Location`=`remote` and click Search, so that a search is performed. * Choose Action `Map to local storage` and `Search result`, not `Selected`. Click the button `Action` at the right. * Now the easydb is not used by fylr any more. (But elasticsearch is still used by fylr)
**This was the central step.** fylr is now independent from easydb and easydb can be turned off. (after testing fylr of course. And Elasticsearch is still used by fylr) ## 6. Adjust previews to fylr standards As previews from easydb are different from fylr previews, it is recommended to replace easydb previews with fylr previews. This will be done in the background while you can already work with fylr. Although this whole step is optional, you should do it if you encounter problems. When seeking support from the developer you will likely be asked to do this step to come to a clean coherent state.
6.a Produce fylr previews * Surf to **https\://**fylr.example.com**/inspect/files/** (login as root) * In the drop down menu `Version` choose `original` and click the `Search` button. * In the drop down menu `Action` choose `produce versions` . * To the right select the round button below `Search result` (not below `Selected`). * Click the `Action` button.
6.b Check whether all previews have been replaced * Is the File queue empty at **https\://**fylr.example.com**/inspect/system/queues/?queue=file** ?\ At the top it would show something like:\ \&#xNAN;*` There are 18 parallel and 10 parallel high priority only file workers active. The queue`` `` `**`contains 0 total entries`**`.`* * Surf to **https\://**fylr.example.com**/inspect/files/** (login as root) * In the drop down menu `location` choose `easydb versions` and click the `Search` button. * The Search shows zero results when all have been replaced.
6.c Remove easydb preview versions to regain storage (optional) * At first remove the location `easydb versions` in the location manager. But do not remove originals! * Next, remove fylr's access to them (e.g. remove it from `/srv/fylr/docker-compose.yml` and recreate the container). But do not remove originals. * Check that the fylr webfrontend still shows previews, at . * Then, when all is still working, you can delete the easydb previews to free storage capacity.\ \ **Warning**: Removing the preview versions of easydb should only be done if the easydb is not needed any more.\ \ **Warning** 2: But the easydb ***originals*** are used by fylr, so **do never remove them**!
## 7. Remove easydb These steps are optional but recommended, as easydb lifetime and support will end before fylr's. Also check that you have uploaded a fylr license and tested/configured optional features that you need, e.g. Single Sign On, email sending, schema-sync via objectstore, hotfolder, etc..
7.a Replace easydb Indexer Adjust `docker-compose.yml` to now feature Opensearch: ``` services: opensearch: [...not shown here: more opensearch details...] ``` Make sure fylr and Opensearch are in the same docker-network. Change `fylr.yml` to use Opensearch: ``` fylr+: [...] elastic+: addresses: - "http://opensearch:9200" ``` Start Opensearch and prepare its directory: ``` cd /srv/fylr mkdir indexer chown 1000 indexer docker compose up -d opensearch ; docker logs -f opensearch ``` Stop outputting log messages with `Ctrl`-`c` if you have seen enough. Restart fylr: ``` docker compose restart fylr ; docker logs -f --tail 0 fylr ``` Stop outputting log messages with `Ctrl`-`c` if you have seen enough. Create the Indexes in Opensearch: *(might take a while!)* Surf to **https\://**fylr.example.com**/inspect/** - System - `Reindex (Blocking)`
7.b Remove Apache See the [default fylr installation](/for-system-administrators/installation/linux-docker-compose.md#installation) and adjust `docker-compose.yml` and `fylr.yml`. for default ports and certificate. ``` systemctl stop apache2 systemctl disable apache2 systemctl mask apache2 cd /srv/fylr /srv/fylr/maintain fylr-recreate ; docker-compose logs -f fylr ``` Is certbot now also not needed any more? Then consider: ``` systemctl mask certbot.timer ```
**7.c Change fylr domain** In case you want to change fylr's domain (e.g. to the former domain of easydb), see [here](/for-system-administrators/configuration/dns-domains.md#changes-to-the-main-domain). **7.d Stopping easydb services** Consider stopping all easydb services that are not needed any more (check whether you still use the indexer). You may want to stop regular maintenance tasks for easydb, typically in `/etc/cron.d/easydb*`. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.fylr.io/help/tutorials/for-system-administrators/migrate-map-to-local-storage.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.