Overview of Physical Setup

The physical deployment of the Kaltura platform is flexible and should be tailored according to the size and applicative needs of the specific site deployment.  For optimal resource utilization as well as security reasons, Kaltura platform deployment should be separated to a front-end layer and a back-end layer.

Sample Deployment Architecture

The following diagram illustrates a sample deployment architecture:

CategoryComponent NameDescription

Load BalancersLVS based, running on Linux server
APIWeb ServersKaltura API and access to shared content
APIThumbnail ServersDedicated API servers for generating thumbnails from images or videos
APIInternal API ServersInternal API servers for Kaltura batch servers
DataDatabase masterPrimary MySQL server for write operations
DataDatabase slaveMySQL servers for read-only operations
DataSphinx serversSearch engine for text-base search
DataMemcache serversCaches database query results
DataShared storageStores all Kaltura media assets, accessed by all relevant servers over a distributed file system
DataAnalytics databaseRuns analytics ETL software and a data warehouse database
BatchBatch serversKaltura dedicated batch servers
BatchDocument conversion serversWindows based server that performs document conversion tasks
BatchEncoding serversKaltura generic Linux-based encoding server

FMS serversServers for webcam recording

FTP serversFTP server for pushing logs from CDN

Admin serverMonitoring and application administration

Virtualization

All server components listed in this section can be set on a physical machine or run on top of a virtualization infrastructure. Kaltura provides a standard VMware image to be deployed and run on a VMware ESXi Hypervisor setup. In this case dedicated hardware should be VMWare ESXi compatible.

Standard Setup Sizes

The following sections describe Kaltura’s recommendation for required hardware to be set for initial setup of an instance of the Kaltura platform. Specific site assessment can be made by Kaltura’s engineering and IT teams upon request.

Small - Single Server Setup

Single server setup may be suitable in the following cases, all assuming low traffic and CPU utilization:

  • Small deployment of a Kaltura Community Edition instance.
  • Platform deployment dedicated for providing limited usage video capabilities to a CMS/LMS platform while using Kaltura CMS/LMS extensions
  • Platform deployment used for development or platform evaluation purposes.

In this setup, all required components are deployed on one server. Document conversion module and workflow cannot be utilized in this topology.

Medium - Distributed Site Setup

This physical setup is suitable as an initial setup for small to medium-size deployments of the Kaltura platform, that are expected to serve medium to high traffic/streaming volume and medium to high number of video uploads and transcoding operations. With growth in service utilization this initial setup can be scaled up as needed, to add the required server type (e.g. additional transcoding servers, additional database servers for replications, additional front end server etc).

Server Recommended Hardware Specs. Operable Components 
 Load Balancing Server/sSite Admin responsibility Site Admin responsibility 
Platform Shared Storage

Can reside on one of the platform servers. Storage size to be set according to the expected volume of media assets.

On average - 3X size of the expected video source files (assuming source should be stored as well).

To be accessible by the Kaltura platform servers via NFS.

Storage
2 Front-End Servers64bit based Linux OS with  4-8 GB RAM and 1 quad core CPU

Kaltura Web Services Module, on top of an apache server

Kaltura Video Recording Module  (if applicable)

Connected to Shared Storage (NFS)

Back-End Batch Server64bit based Linux OS with at least 4GB RAM and 1 quad core CPU

Batch Jobs Module

Connected to Shared Storage (NFS)

Back-End Transcoding Server64bit based Linux OS with at least 8GB RAM and 2 quad core CPU

Transcoding Module

Connected to Shared Storage (NFS)

Back-End DB Server64bit based Linux OS with at least 4GB RAM and 1 quad core CPU

Operational Database

Sphinx – full text search engine

Connected to Shared Storage (NFS)

Back-End DWH Server64bit based Linux OS with at least 8GB RAM and 1 quad core CPU

Video Analytics Module

Connected to Shared Storage (NFS)

Site Admin Server

Linux based OS.

No minimal specifications on hardware

Site Admin Module 

Document Conversion Server (optional)Windows server 2008 With at least 2GB RAM. 1 CPUKaltura Document Conversion Module (if applicable)

Large - Full Distributed Site Setup

This physical setup may be suitable as an initial setup for large-size deployments of a Kaltura platform, that are expected to serve high traffic/streaming volume and high number of video uploads and transcoding needs. With any growth in service utilization this initial setup can be scaled up as needed, to add the required server type (for example, additional transcoding servers, additional database replication server, additional front end server and others).

This setup may include the following servers:

Server Recommended Hardware Specs. Operable Components 
 Load Balancing Server/sSite Admin responsibility Site Admin responsibility 
Few Front-End ServersSeveral 64bit based Linux OS with at least 8GB RAM and 1 quad core CPU

Kaltura Web Services Module, on top of an apache server.

Kaltura Video Recording Module  (if applicable)

Connected to Shared Storage (NFS)

Few Back-End Batch ServersSeveral 64bit based Linux OS with at least 4GB RAM and 1 quad core CPU

Batch Jobs Module

Connected to Shared Storage (NFS)

Few Back-End Transcoding ServersSeveral 64bit based Linux OS with at least 8GB RAM and 2 quad core CPU

Transcoding Module

Connected to Shared Storage (NFS)

Few Back-End DB Server/s  (master + replications)64bit based Linux OS with at least 4GB RAM and 1 quad core CPU

Operational Database

Connected to Shared Storage (NFS)

Back-End DWH Server/s64bit based Linux OS with at least 16 GB RAM and 1 quad core CPU

Video Analytics Module

Connected to Shared Storage (NFS)

Site Admin ServerLinux based OS. No minimal specifications on hardware

Site Admin Module 

Connected to Shared Storage (NFS)

Document Conversion Server/s (optional)Windows server 2008. With at least 2GB RAM. 1 CPUKaltura Document Conversion Module (if applicable) 
Shared Storage Dedicated Server

64bit based Linux OS with at least 4GB RAM and 1 quad core CPU

On average - 3X size of the expected video source files (assuming source should be stored as well)

Storage size to be set according to the expected volume of media assets

Shared Storage

Multiple Data-Centers Site Setup

For enabling full redundancy and better performance for end users, utilization of multiple data centers may be needed. The geographic location of the data center should be chosen according to the geo- location distribution of the end users.

The synchronization of multiple data centers should be performed on both database and storage. For database synchronization a native MySQL Master-Master replication can be utilized.

For media files synchronization, a batch process can copy media items from one datacenter to the other datacenter, in order to maintain full online redundancy. The following diagram illustrates the architecture of a multi-datacenter setup of the Kaltura Platform.

Self-Streaming Considerations

In most cases, when planning for a new deployment of the Kaltura online video platform, a use of external CDN functionality is recommended. When this is not possible and/or not required, it is recommended to add additional front-end hardware to be dedicated for playback streaming.

Kaltura recommends the following addition to site setup in such cases:

 Server Recommended Hardware Specs. Operable Components
Streaming proxy servers2 servers with  64bit based Linux OS, at least 8GB RAM and 1 quad core CPU Media streaming (relevant for self-streaming deployments) 

Kaltura eCDN Plus is the newest addition to Kaltura’s wide range of deployment options – including on-premise, hybrid, and full SaaS deployment. It is based on the robust Kaltura API and leverages streaming technology by Wowza Media Systems as well. Kaltura eCDN Plus is tightly integrated with Kaltura’s market-leading video products, such as the company’s CorporateTube video portal, and integrations with social business platforms (SharePoint, IBM Connections, and Jive), as well as Kaltura’s products for the education market (including integrations with Blackboard, Sakai, Moodle, Desire2Learn, and Canvas) providing a seamless user experience, and centralized management and control.

Additional Scalability Considerations

 Factor Estimated Sizing Information
Storage Resources 

As a general rule of thumb, in order to have multiple flavors generated for each video entry, including mobile flavors, - storage of 3X the size of the expected video source files should be reserved (assuming source should be stored as well).

Example of storage calculation: 

Original video clip at backup storage:

Bit rate = 8000 kbps - good quality camera recording

Storage Size = ~60 Mb for a one minute duration

 

Normal Web Quality Flavor (transcoded files)

Bit rate = 600 kbps

Storage Size = 4-8 MB for a one minute duration = ~ 7.5% of Original

 

High Web Quality Flavor (transcoded files)

Bit rate = 1200 kbps

Storage Size = 8-16 MB for a one minute duration = ~ 15% of Original

Transcoding Resources

Each CPU allows for 360-720 video hours of transcoding per month

One CPU can handle one transcoding task at a time. 

Playback Streaming

The actual streaming volume can be deduced from the single clip storage size and site usage information. 

if CDN is in use - most streaming volume is handled by CDN provider (excluding cases of long tail media type) 

Video Recording Streaming (if applicable)

Each FMS server can hold hundreds of concurrent recording connections

30 days * 24 hours = 720 streaming hours a month per connection 

Web ServicesOne front-end server can support ~75,000 daily unique users (API calls and playback streaming to CDN when CDN is in use) 
Operational Database2 quad core DB Machines running in a master/slave topology can hold tens of millions of media items 
Video Analytics1 quad core DB Machine for storing 6 months of historical data for tens of millions of media items 

The following aspects should be taken into account as a basis for a scale-up decision:

 Scale up Reason Scale up Activity
Back-end server/s CPU’s are repeatedly overloaded due to increase in number of transcoding operationsMove transcoding to a dedicated  server or add a dedicated transcoding server 
Apache/web services are repeatedly overloadedAdd a dedicated front end server (and load balancing hardware if needed)
System is repeatedly slow due to db related operations, with no change after performance tuningMove DB to a dedicated server or add a db server 
System is intolerable slow during daily ETL processes or reporting executionsMove ETL or DWH& ETL to a dedicated server 
Back end server/s CPU is overloaded (no increase in number of transcoding operations)Add a dedicated back end server for batch processes
In This Article