About
Use drop folders to automate or streamline content ingestion into the Kaltura platform. Drop folders are monitored continuously and automatically ingest new content based on your configuration.
What you can do with drop folders:
- Automate or partially automate an ongoing content ingestion workflow.
- Support bulk ingestion of content to Kaltura, either when you set up an account and want to migrate a large amount of existing content, or for use on an ongoing basis.
- Work with content providers and operational teams who do not have access to the Kaltura Management Console (KMC) or any other content management application.
Kaltura’s drop folder service is designed to support a variety of integration requirements and business needs and has flexible configuration options.
Drop folder locations
Hosted on Kaltura cloud
Kaltura sets up and manages the drop folder. Publishers get access via FTP/SFTP or Aspera and are responsible for uploading content. This option is recommended for publishers looking for a fully managed solution, and for publishers whose workflow and security needs are better met by this option.
Hosted on-premise by publisher
Kaltura defines the drop folder configuration, including its location and access details. The publisher sets up the folder on their premises or on any FTP/SFTP-accessible location. A Kaltura drop folder 'watcher' monitors the folder and downloads content to the Kaltura cloud once it's ready.
This setup allows publishers to retain control over their infrastructure while using Kaltura's ingestion service. It's recommended for publishers who already have a designated network location or who need to meet specific internal security policies.
Configuration options
KMC administrators can view and delete media files in the account drop folder on the Drop Folders Control page. The drop folder input package may be a single source file or a complete media package including multiple flavors and metadata xml. You can create a new entry based on the input package or add it to an existing entry.
Drop folder configuration options:
Transcoding profile
Each drop folder is linked to a transcoding profile. This determines whether uploaded content is treated as a single media file (to be transcoded by Kaltura) or as a complete media package with multiple flavors (created by a local transcoder) A media package is added from the drop folder to a single media entry as one unit.
File name patterns
By default, drop folders accept all file name patterns. To limit processing to certain files (whitelist), you can define specific patterns the drop folder should handle. This is useful when only files with a specific type or naming structure should be ingested.
To exclude certain files (blacklist), you can define patterns for the drop folder to ignore. This is helpful for skipping temporary files created by transfer clients, which often use consistent name patterns or extensions.
Ingestion types
Content ingestion from a drop folder can be triggered in two ways:
Media files: Place media files in the drop folder. Kaltura automatically uploads them, and you can edit metadata later in the KMC. This method may require extra configuration in the drop folder profile.
Kaltura MRSS-formatted XML: Place an XML file that includes metadata and references to media files. The XML can include multiple entries, with media files either in the drop folder or at remote locations. Ingestion begins automatically once all referenced files are available, and metadata is populated from the XML.
To learn more, please refer to What is bulk upload and FTP content ingestion?
If a transcoding profile is defined in the XML, it takes precedence over the one attached to the drop folder settings.
Ingestion from media files - matching policies
When a drop folder is set to ingest content from media files, one of the following matching policies should be selected:
Create an entry from media
When media files are dropped into the folder, Kaltura automatically creates new entries - ideal for users who don’t use the KMC for uploads. Metadata can be edited later in the KMC.
- Drop a media file into the drop folder. A new entry in Kaltura is automatically created. This triggers an automatic conversion in Kaltura according to the transcoding profile settings attached to the drop folder. The title of the new entry is based on the file name.
- Edit the new entry metadata in the KMC as needed. A process to automatically replace the media cannot be triggered from a drop folder configured to support this policy.
Match media to an existing entry
To update an existing entry, include a matching Reference ID in the media file name. This supports workflows where metadata and media are handled separately.
- Prepare a new entry in the KMC.
- Set metadata for the new entry and set its Reference ID in the Edit Entry's Metadata tab. Reference ID example: MyVideoClip123
- Drop a media file in the drop folder and include the Reference ID in the filename. For example MyVideoClip123.mp4
If the drop folder contains MyVideoClip123.mp4, Kaltura matches it to an entry with MyVideoClip123 as the Reference ID and replaces the media. Transcoding is triggered automatically. If no match is found, the file stays in the folder until an entry with that Reference ID is created.
When the matched entry is already associated with media, a process is triggered that automatically replaces the media.
Match media when entry exists or create a new entry
If no matching Reference ID is found, Kaltura creates a new entry and adds the file to it - useful for workflows needing automatic uploads, post-upload metadata editing, and media replacement via drop folders.
When a local transcoder is enabled, multiple bitrate files can be uploaded as a single package. Ingestion starts only once all files are present in the drop folder.
Ingestion from Media Files - matching a regular expression
When a drop folder is set to match media files to an existing media entry, the file name must include the Reference ID that is used as a matching key. By default, the file name itself is treated as a Reference ID.
Examples: (the Reference ID is red):
MyVideoClip123.mp4
When an account is configured to support the use of a local transcoder and ingestion of multiple bit rate media files to a single media entry, file naming convention must be defined to also include the system name of a specific flavor in the transcoding profile that this file should be matched to.
In the following examples, all media files are added to a single entry and each file is matched to the appropriate transcoding flavor, according to its system_name definition in the transcoding profile (the Reference ID is red, the transcoding flavor system name is blue):
MyVideoClip123_HD.flv
MyVideoClip123_Basic-Small.flv
MyVideoClip123_Basic-Large.flv
Drop folder file management policies
For drop folders hosted on-premise, publishers can define a file deletion policy. This policy determines if files are automatically deleted by the Kaltura drop folder service after successful ingestion or manually by the publisher via the KMC.
When selecting automatic deletion, publishers must specify the number of days before files are removed, limited to successfully processed files only.
For drop folders hosted in Kaltura, files are automatically deleted shortly after successful processing. Manual deletion can be performed via the KMC or Kaltura API, though direct manual deletion from the folder is not recommended due to potential synchronization issues.
In both cases, any change to a file in the drop folder resets its handling, treating it as a new file.
Drop folder access settings
When a drop folder is managed by Kaltura and is hosted on the Kaltura cloud, Kaltura’s technical support team sets the drop folder location and provides full access and authentication information to the publisher.
When a drop folder is hosted and managed at the publisher's premises, the publisher must provide the following information, per preferred transfer protocol, to Kaltura, to set up the drop folder configuration:
| Protocol | Access Details | Authentication Details |
|---|---|---|
| FTP | Host, Port, Storage path | Username, Password |
| SFTP | Host, Port, Storage path | Username & Password, or Username & SSH private/public keys (by default keys will be generated by Kaltura) |
Drop folders control panel
This section applies only after the Kaltura Drop Folder service is set up.
In the KMC Content menu, KMC administrators can use a control panel/table to monitor and troubleshoot media files in drop folders. Access to the Drop Folders control panel can be granted to specific administrators through the administrator's user role definition.
The Drop Folders control panel includes the current processing status of each file in the drop folder, and an option to filter the file display for a specific drop folder, status, and creation date range. For files that completed their processing, a link to the entry editing window is available as well.
An additional option to manually delete files from the drop folder can be granted separately only to authorized administrators.
Drop folders table
The drop folders files table in the Drop Folder Control page lists all media and/or XML files currently available in the account’s drop folders. The current details of each file are available within the table. In the filter bar on the left side of the page, you can filter the files by:
- Search criteria
- Drop folder
- File creation date
- Drop folder File Processing status
If you need to delete files from the drop folder, click Delete from the 3 dots menu.
Drop folder files statuses
The following statuses describe the different stages and possible error cases that may apply to a specific file in the drop folder:
- Parsed from XML – A reference to the media file was parsed from the XML. The media file is expected but is not yet available in the drop folder.
- Uploading – File is currently uploading to the drop folder.
- Pending – File upload is completed - waiting for drop folder processing to start.
- Waiting for Related Files (Formerly “Matched”) – Processing of this file should continue when all related media files are available in the drop folder (i.e. other files referenced in the same XML or other flavors expected for the same entry).
- Waiting for Matched Entry (Formerly “Not Matched”) – Processing of this file should continue when all related files are available in the drop folder (for example, XML file, other media files referenced in the same XML, or other flavors expected for the same entry).
- Processing - Drop folder engine is currently processing this file.
- Downloading - The file is currently importing from the drop folder to Kaltura. (applicable only in drop folders hosted on-premise by customers)
- Done – Drop folder processing was completed for this file.
- Error – Drop folder engine processing failed.
- Download Failed - Failed to import the file from the drop folder to Kaltura (applicable only in drop folders hosted on-premises by customers).
- Delete Failed - Failed to delete the file from the drop folder.
Unsupported file name characters
When uploading to a drop folder, it's important to note that there are unsupported characters in file names that should be avoided when uploading files. Below you can find a full list of all unsupported characters:
| Character | Name |
|---|---|
| \ | backslash |
| / | forward slash |
| : | colon |
| ; | semi-colon |
| * | asterisk |
| ? | question mark |
| " | double quotes |
| < | left angle bracket |
| > | right angle bracket |
| | | pipe |
| % | percent |
| , | comma |
| # | pound or number |
| $ | dollar |
| ! | exclamation |
| + | plus |
| { | left bracket |
| } | right bracket |
| & | ampersand |
| [ | left square bracket |
| ] | right square bracket |
| • | bullet |
| ' | apostrophe |
| ` | backtick |
For more information on the XML Custom data upload see the following:
http://www.kaltura.com/api_v3/xsdDoc/?type=dropFolderXmlBulkUpload.dropFolderXml
The fileSize element is no longer required. Following is a sample XML file without the fileSize element, see sample.xml
