DOCUMENTATION
  • Welcome
  • Releases
    • 2025
      • Release v6.20.2 (2025-05-07)
      • Release v6.20.1 (2025-05-06)
      • Release v6.20.0 (2025-04-30)
      • Release v6.19.2 (2025-04-11)
      • Release v6.19.1 (2025-03-31)
      • Release v6.19.0 (2025-03-27)
      • Release v6.18.2 (2025-03-11)
      • Release v6.18.1 (2025-03-07)
      • Release v6.18.0 (2025-02-26)
      • Release v6.17.3 (2025-02-14)
      • Release v6.17.2 (2025-02-07)
      • Release v6.17.1 (2025-02-06)
      • Release v6.17.0 (2025-01-30)
    • 2024
      • Release v6.16.0 (2024-12-12)
      • Release v6.15.0 (2024-11-27)
      • Release v6.14.2 (2024-11-05)
      • Release v6.14.1 (2024-11-01)
      • Release v6.14.0 (2024-10-31)
      • Release v6.13.3 (2024-10-16)
      • Release v6.13.2 (2024-10-10)
      • Release v6.13.1 (2024-10-02)
      • Release v6.13.0 (2024-09-25)
      • Release v6.12.2 (2024-09-18)
      • Release v6.12.1 (2024-08-01)
      • Release v6.12.0 (2024-07-25)
      • Release v6.11.5 (2024-07-09)
      • Release v6.11.4 (2024-07-05)
      • Release v6.11.3 (2024-07-03)
      • Release v6.11.2 (2024-06-21)
      • Release v6.11.1 (2024-06-14)
      • Release v6.11.0 (2024-06-05)
      • Release v6.10.2 (2024-05-15)
      • Release v6.10.1 (2024-05-08)
      • Release v6.10.0 (2024-04-30)
      • Release v6.9.3 (2024-03-19)
      • Release v6.9.2 (2024-03-15)
      • Release v6.9.1 (2024-03-06)
      • Release v6.9.0 (2024-02-28)
      • Release v6.8.5 (2024-02-02)
      • Release v6.8.4 (2024-02-01)
      • Release v6.8.3 (2024-01-12)
      • Release v6.8.2 (2024-01-05)
    • 2023
      • Release v6.8.1 (2023-12-22)
      • Release v6.8.0 (2023-12-14)
      • Release v6.7.4 (2023-11-15)
      • Release v6.7.3 (2023-11-14)
      • Release v6.7.2 (2023-11-03)
      • Release v6.7.1 (2023-10-17)
      • Release v6.7.0 (2023-10-13)
      • Release v6.6.4 (2023-09-29)
      • Release v6.6.3 (2023-09-28)
      • Release 6.6.2 (2023-09-14)
      • Release v6.6.1 (2023-08-10)
      • Release v6.6.0 (2023-08-03)
      • Release v6.5.1 (2023-06-23)
      • Release v6.5.0 (2023-06-22)
      • Release v6.4.0 (2023-05-31)
      • Release v6.3.1 (2023-04-28)
      • Release v6.3.0 (2023-04-05)
      • Release v6.2.5 (2023-03-16)
      • Release v6.2.4 (2023-02-01)
      • Release v6.2.3 (2023-01-12)
      • Release v6.2.2 (2023-01-12)
      • Release v6.2.1 (2023-01-05)
    • 2022
      • fylr first Production Ready Release 🎉 (2022-12-22)
  • License
  • Help
    • FAQs
    • Tutorials
      • For Users
      • For Administrators
        • Exporting & Importing Hierarchical Lists
        • Regenerating preview images
        • Search Text in images or office files
      • For System Administrators
        • How to setup and use IIIF
        • External access: Sharing collections with anonymous users
    • Glossary
  • FOR USERS
    • Getting Started
    • Asset / Records Management
      • Creating Records
      • Editing Records
        • Input Fields
        • Group Editor
      • Deleting Records
    • Quick Access
      • Collections (& Presentations)
      • Saved Searches (& Lists)
    • Lists
    • Plugins
      • Plugin Overview
  • FOR ADMINISTRATORS
    • Permissions
      • User
      • Groups
      • Object Types
      • Pools
      • Tags & Workflows
      • Presets
    • Tools
      • CSV Importer
        • General Information
        • Options
        • Examples
          • All Data Types
          • Lists
          • Hierarchies
          • Files
      • JSON Importer
        • Step-by-Step Tutorial
          • Write Import Manifest
          • Create Basetype Payloads
          • Create Object Payloads
          • Collection Payloads
          • Optional: Update links between Objects
          • Start Import
      • Permissions Download & Upload
    • Base Configuration
      • General
      • Access
      • User Management
      • Languages
      • Email
      • Export & Deep Links
      • Workflow Webhooks
      • Publications
      • File Worker
        • Preview Configuration
        • Location Defaults
        • Custom .icc Color Profiles
      • Objectstore
      • Services
      • License Management
      • Development
      • Plugins
    • Plugin Manager
    • Location Manager
    • Messages
    • Events
    • Backup Manager
    • Additional Features
      • IIIF
      • Connector
      • Wordpress
      • Zooniverse
      • Protocols
        • OAI/PMH
  • FOR SYSTEM ADMINISTRATORS
    • Installation
      • Linux
        • multiple fylrs in one Linux
        • proxy and fylr
      • Windows
      • Kubernetes
    • Configuration
      • fylr.example.yml
      • fylr.default.yml
      • performance tuning
      • pre-load frontend config
      • Load Custom Plugins
      • HTTP and HTTPS
      • DNS Domains
    • Backups & Restore
    • Migration Tool
      • Create payloads (fylr backup)
      • Insert payloads (fylr restore)
      • Best Practice
      • Using the fylr inspect page
    • Integration
      • Authentication
      • Hotfolder
    • Symptom & Solution
      • Log messages that can be ignored
      • too many clients are connected
      • too many nested clauses
      • context canceled
      • ContainerConfig error
      • Purge objects
    • PostgreSQL versions
  • Tutorials
    • Project Workflow
    • Hotfolder & File System Connect
      • Preparations Before Usage
      • Setting Up An Upload Collection
      • Importing Files
    • PDF Creator
    • Extracting File Metadata Later On
    • Overlay Resource
    • Authentication
      • LDAP
      • SAML
    • Data Model Sync
    • Purge a fylr instance
    • typo3 plugin
    • Use fylr in Google docs via CI HUB
  • FOR DEVELOPERS
    • API
      • OAuth2
      • Endpoints
        • /api/collection
        • /api/config
        • /api/db_info
        • /api/db
        • /api/eas
        • /api/event
        • /api/export
        • /api/group
        • /api/l10n
        • /api/mask
        • /api/message
        • /api/oaipmh
        • /api/objects
        • /api/objecttype
        • /api/plugin
        • /api/pool
        • /api/publish
        • /api/right
        • /api/schema
        • /api/search
        • /api/settings
        • /api/suggest
        • /api/system
        • /api/tags
        • /api/transitions
        • /api/user
        • /api/webdav
        • /api/xmlmapping
        • /api/task
    • System Data Types
      • pool
      • file
      • user
      • group
      • pool
      • collection
      • message
      • publish
      • event
    • User Data Types
      • text, text_oneline
      • string
      • text_l10n, text_l10n_oneline
      • boolean
      • number
      • integer.2
      • double
      • date, datetime
      • daterange
      • geojson
    • Custom Data
    • Emails
    • Export
    • Exec server
    • File versions
    • WebDAV
    • Plugin
    • Collection Pin Code
    • easydb 5
    • Localization
    • Access private Repositories
Powered by GitBook
On this page
  • Fields
  • Pre-fill fields from manifest.json
  • Payload list
  • Buttons
  • Handling problems with Mixed Content
  • CORS Plugin
  1. FOR ADMINISTRATORS
  2. Tools

JSON Importer

The JSON importer can be used to bulk import data in JSON format

PreviousFilesNextStep-by-Step Tutorial

Last updated 12 months ago

The JSON importer can be used to bulk import data in JSON format.

A step by step tutorial on how to generate JSON Payloads can be found .

The importer can be found under "Tools > JSON Importer".

The configuration for the JSON importer can be uploaded in a manifest file (.json). The manifest contains payloads and other information which will preset the importer.

Please note that the server where the manifest.json and the payloads are stored, needs to be accesible from the frontend

Fields

Option
Description

URL manifest.json

This is a URL to get the manifest.json by clicking the Load button next to the input. This file can contain the following settings, alternatively, they must be entered in the JSON Importer form

Source

Name of the source instance. Can be freely selected. This information is not migrated, it is only used for identification purposes (optional)

Payloads base URI

Base URI for all the payloads in the list. It will be prepended to each payload. The default value will be the same base URL of the manifest. If the payloads are not stored in the same folder as the manifest (or on another server), this is needed to build absolute paths from the payload file names. This value needs to be the relative path to the payload folder

Batch size

Quantity of objects pushed to the server per request

File upload type

  • Direct: frontend uploads files to EAS using put

  • URL (remote put): EAS loads file from remote url using rput

  • Ignore: no files are uploaded

Skip HEAD Request

Enable this, to skip the HEAD request that is performed by fylr before files are loaded. This is only effective, if remote put is selected

Metadata mapping

Metadata mapping used for files (optional)

Ignore file upload errors

When checked, all errors of file uploads will be ignored and the import process will continue. All errors will be available in the logs

Ignore unique constraint errors

When checked, all UNIQUE constraint database errors from the server will be ignored

File replace url

The location for each file URL will be replaced by the value configured here. If a file URL in the payload starts with http://localhost, this part will be replaced by this URL (optional)

Pre-fill fields from manifest.json

Some of these fields can be pre-filled by values in the manifest.json file:

Option
Key
Type
Description

Source

"source"

String

Payloads base URI

"payload_base_uri"

String

Batch size

"batch_size"

Integer

min: 1, max: 1000

File upload type

"eas_type"

String

  • direct: Direct

  • url: URL (remote put)

  • ignore: Ignore

Skip HEAD Request

"skip_head_request"

Boolean

Ignore file upload errors

"eas_skip_errors"

Boolean

File replace url

"eas_replace_url"

String

Valid URL

Metadata mapping

"mapping"

Integer

ID of the metadata mapping

Payload list

This list show all available payloads. It is possible to see the content by clicking them.

Header
Description

Status

Current status of the payload. Pending / Success / Error

URL

The url of the payload

Starting batch

Number of the starting batch. As default the first one is selected, so all batches are processed

Batches

Quantity of batches (they will depend on the quantity of objects and the selected batch size)

Objects

Quantity of objects

Objecttype

Objecttype of the payload

Actions

Opens the payload in a new tab

Enabled

Checkbox to enable or skip the payload (Ctrl+Click to select/unselect all in the selected page, or click in the button below to select/unselect in all pages)

Buttons

Function
Description

Load

(Re)Load the manifest file from the URL

Download logs in CSV format.

Download the logs in CSV format

Clear the logs

Clear the logs of the last run

See the last report generated.

See the report of the last run

Prepare

By clicking some validations are performed before the actual import can be started

Start

It starts the import of all the enabled payloads


Handling problems with Mixed Content

Since the frontend loads the JSON files as a client, it is possible that the browser you are using will block the request to the other server.

Mixed Content Blocking is implemented in many browsers and is a security feature:

  • this might also be implemented in other browsers in some way

If the manifest and the payload files can not be loaded, check the developer tab of your browser and look for an error message that looks like any of these (there my be variations):

... has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

or

This request has been blocked; the content must be served over HTTPS.

If this is the case, your browser blocks the request. Depending on the browser, there are ways to work around this, and allow loading the files anyways.

CORS Plugin

Caution: Mixed Content Blocking is a security feature. Only disable it if you trust the other server. This might be a security risk!

For different browsers, there are plugins available which allow to disable Mixed Content Blocking, for example:

With many of the available plugins it is possible to toggle Mixed Content Blocking. Please check the availability of plugins for your browser.

Google Chrome:

Mozilla Firefox:

Microsoft Edge:

Google Chrome:

Mozilla Firefox:

"No More Mixed Messages About HTTPS"
"Mixed content blocking in Firefox"
"Learn how Microsoft Edge handles mixed content downloads"
"Allow CORS: Access-Control-Allow-Origin"
"CORS Everywhere"
here
json importer in the frontend