ai metadata plugin

How to configure and use the ai-metadata plugin

The ai metadata plugin sends images in a fylr instance to a LLM like ChatGPT to write the generated texts into user defined fields in fylr objects.

Plugin description

The ai-metadata plugin enables the integration of LLMs compliant with the Chat Completions API by OpenAI into fylr. Custom prompts can be configured and mapped to fylr Object Types through metadata mappings. The generated output is written into objects during their creation or during the execution of a (scheduled) background task.

Datamodel requirements

To successfully use the ai-metadata plugin to describe your instances content, your Object Type needs to have

• a file field: the source image for the LLM to describe

• text fields or lists: (multilingual) text fields to map the LLM output to

• also supported: dates, boolean fields

Installation

This plugin is a paid feature. The plugin is only available after a valid license has been purchased.

Please contact us to obtain the license.

Move to your fylr instances Plugin Manager to install the ai-metadata plugin by finding the -Icons in the bottom of the plugin list and clicking the plus symbol.

While installing enabling the plugin is the default behavior. You can enable or disable the plugin anytime, active configuration will be preserved until enabled again.

An ongoing ZIP installation of the ai-metadata plugin

Install the plugin by release URL to always get the latest updates, after contacting support you will receive an installation URL instead of a ZIP.

For the setup of this plugin root permissions are suggested.

Configuration

After installing and activating the plugin, select it in the plugin list.

Configure the plugin in the "General Tab" by adding an API Key

Setting	Description
Extended protocol	Additional debug context in FILE_METADATA events
API Type	OPEN_AI / AZURE: choose Azure if you want to connect to your own LLM URL.
Base URL	OPEN_AI: Leave empty if using API Type OPEN_AI (default fallback) AZURE / others: example: https://{{ressource}}.openai.azure.com Could be any Endpoint compliant with the Chat Completions API
API Key (Required)	Enter your API key. Without API Key, requests will be denied.
API Version (only with API Type AZURE)	Use the current version of your LLM (Azure-REST-API-Version, a date string e.g. 2026-02-01 oder 2026-01-01-preview )
Model (Deployment Name)	Free entry: input your deployment name. Required with API Type AZURE.
Model (List)	Select from the list of OpenAI models
Used image size	Controls the resolution of images sent to the LLM. The default of 600px is sufficient for most use cases.
Instructions for the AI	Describe the AI’s general style, role, and behavior here. These instructions apply to all answers, for example tone, perspective, or fixed phrasing rules. Do not enter the actual question here. Note: does currently not work with API Type AZURE

It's currently not possible to save more than one LLM configuration.

Creating the prompts

Inside the plugin managers config tab, create one or more named prompts by clicking .

The available inputs are mandatory.

Setting	Description	Options
Name	Name your prompt. This name will be used later to refer to this particular prompt.	-
Prompt	Your prompt to the LLM	-
Type	Output type	Single line text Single line text (multilingual) List text List text (multilingual) Boolean Date (ISO 8601)

Choosing the Output Type

The output type matters when mapping the prompt to the object type later, make sure you select the option that fits your needs and matches your object types fields.

If it doesn't, edit your datamodel accordingly and continue afterwards.

• Single line text

  • maps toSingle Line & Multi Line Text, Simple Text (String)

• Single line text (multilingual)

  • maps to Single Line & Multiline Text (multilingual)

• List text

  • maps to Single Line & Multi Line Text, Simple Text (String) in a nested field.

• List text (multilingual)

  • maps to Single Line & Multiline Text (multilingual) in a nested field.

• Boolean

  • maps to Boolean field

• Date (ISO 8601)

  • maps to Date, Date + Time

Examples

Example prompts created for the ai-metadata plugin. The prompts display the usage of the different output type options while showcasing prompts in various complexities.

Multilingual fields: Make sure to include the languages of your expected output in your prompt.

Each named list entry in the Plugin Managers configuration tab of the ai-metadata plugin becomes available in the Metadata Mapping section of fylr. (see in sidebar > Administration)

As a start, you can upload the displayed configuration and modify as needed.

1KB

fylr-ai-metadata-plugin-example-config.json

Open

Creating the Metadata Mapping

Create a new IMPORT Mapping by finding the -Icons in the bottom of the available Mappings and clicking the plus symbol, selecting IMPORT.

Option	Description
Object Type	Select an Object Type to apply the mapping to
Name	Provide a descriptive name for your mapping

If the plugin is enabled and a prompt is configured in the previous step, you should now see the section “ChatGPT Metadata” listing your created prompts.

Per default all Object Types are listed, in the below example, only the fields for the selected Object Type are shown because the object type limits the mapping to be used on OT "ChatGPT Object"

To map the available fields to the created prompts, drag and drop the individual fields onto your created mapping. Right click a mapping to remove.

Example of a created Metadata Mapping in fylr using drag and drop.

Right-click on a mapping to remove it. Duplicate Mappings are allowed.

Reload the frontend to make the mapping available for upload dialogs and record creation.

Usage while creating records

When creating a new record, select the metadata mapping that includes your prompt configuration.

The mapping can be used

• during upload via the plus icon

When clicking Next, you will see the Metadata Mapping being applied with a progress indicator.

Metadata mapping status progress

In the next view, see the prompt results arranged into your fields. After saving, the records will be accessible just like any other record.

Upload collections can be configured to apply a custom metadata mapping to be applied to all newly added records

If not satisfied with the results, after updating your prompts with closer alignment to your requirements, restart the import process or continue reading to apply the mapping with a background task.

---

Usage with Background Tasks

To run the ai-metadata plugin in background tasks, follow these steps:

Create and Configure the Task

1. Open the Background Tasks section in the header bar of your fylr instance.

background tasks manager located in the header bar of fylr

1. Create a new task by using the -Icon and selecting the module Metadata

2. Define the schedule:

• Manual time (default: now)

• Scheduled time (future execution)

Task Parameters

Configure the mapping

Example mapping configuration for using the ai-metadata plugin in a background task

Parameter	Description
Object Type	Apply only to the specified object type.
Metadata Mapping	Select the metadata mapping to be applied.
Pool	Used when creating linked records with pool management during metadata mapping.
Mask	Define which mask to use for the task.
Field for Files	If multiple file fields exist, specify which one to use as source for yourn prompt.

Creating a search result to apply the mapping to

For the task to have records to apply the mapping to, configure a search that finds the records you want to be filled by the LLM (selection of records / the entire search result).

If individual records are selected, only those will be mapped.

If an entire search result is selected using "Select All", the same search will be reused for the next scheduled run of the background task.

Warning: An empty search will result in the background-task sending ALL records to the LLM. This may or may not result in a high usage of tokens.

Example configuration of a search query showing results to be mapped by the ai-metadata plugin using the bg-tasks.

If the found records have no values in their fields yet, the Override Values checkbox might not be required.

After the created tasks next scheduled job has finished (now or later, depending on the amount of records to be mapped), confirm the changes made in the tasks log and the record itself.

Customizable variables in prompts

Create customizable variables used in your prompts to be filled later during usage.

Configuration of a variable:

• Name (string):

  • the name of your variable, will be visible later during usage. No empty space characters allowed.

  • Use the variable in your prompts by referencing it using the this syntax: %variables.NAME%

• Description (string): will be used as a label to your variables input

• Changeable by user (boolean): users can set the value of the variable during upload or while setting up a background task

Simple usage examples

A customisable prompt to be created during usage

1. A variable prompt to be only defined during usage, not in the plugins prompt configuration:

1. Create a placeholder prompt using only the variables content as the full prompt to be send to the LLM

1. Create a metadata mapping that uses the the placeholder prompt to connect your variable is input for your placeholder prompt to your object types fields.

2. When selecting the metadata mapping that's using the placeholder prompt, notice the cog wheel. In the input behind the cog wheel provide a value for the variable (here: the entire prompt!)

An example dialogue when creating a new record. A metadata mapping is selected, thus the parameter input is enabled (see cog-wheel icon)

Providing the variable value during creation of a new record.

A variable to control the length of a LLM response

Setup:

Example Setup of a variable controlling the length of a LLM response

Reference this variable in another prompt:

Usage of the amout_of_words variable in a LLM prompt

Now during configuration of a prompt a user can control the amount of words used in the LLMs response.

Those are examples with the goal to showcase the setup of the variable system in the ai-metadata plugin.

The LLMs responses are only as good as the prompts provided.