Many organizations have a monitoring system in place to ensure maximum up-time and alert administrators to issues immediately. Commonly used tools include WhatsUp Gold, ServerDensity, and SolarWinds. Below is an overview of some areas where the Ensemble Video Platform can be configured with monitoring to ensure maximum up-time and reliability.
Basic Server Monitoring
It is recommended that you monitor some basic items on each server that is part of the Ensemble Video Platform.
- % Processor Time – This is the % of time the CPU is running and performing tasks. Values consistently above 85%-90% might indicate a performance bottleneck.
- Processor Queue Length – Indicates the number of threads waiting for CPU access. This indicates the the CPU is busy servicing other requests and there is a backlog or delay processing the newer requests.
If these counters are indicating a bottleneck, then adding more CPU resources (if this is a VM) might reduce these issues. Or, separating tasks onto different VMs (i.e. split Ensemble and the SQL Service onto their own VM) would eliminate contention for resources on busier systems.
- Available MB – Systems should have more than 50% of memory available, on average.
- Pages/Sec – If this number is consistently high (>500) and you have low available RAM this indicates a potential bottleneck.
If these counters indicate a potential issue, then adding more RAM to the VM should help. You can also consider separating services onto their own VM (Ensemble and SQL on their own server) to eliminate contention for resources.
- Average Disk Queue Length – This is the average number of read/write requests on a disk. Usually a value >2 for extended periods of time indicate a bottleneck at the disk.
You will need to engage your System Engineer to develop a plan to deal with this potential bottleneck. Given that most servers are VMs, they tie into SANs and NAS devices that are beyond the control of the VM administrator.
You will need to monitor network performance as well. This will require assistance from your system engineer due to many factors that determine your network speed. Link speed, internet speed, web filters, packet shapers, etc. can all impact what speed you can expect from your network.
The Ensemble Service is made up of a web application, API and a windows service. Below are some tips on monitoring the system to ensure up-time.
Ensemble Web App
You can monitor the URL https://your.ensembleservice.edu/api/info to ensure the IIS sever is responding appropriately. This screen should return a simple XML format, but this will indicate that the IIS service is up and running and has access to the database.
Ensemble Video Platform utilizes a local Windows Service that manages and monitors things like:
- Watch/Ingest Directories
- Live Recordings
- Ensemble License
- Ensemble Studio Recording Manager
- Process user Uploads
- Trim Videos
You will need to monitor the Ensemble Video Service. If the service fails, you will get license errors on the Web UI and users will be unable to add new content to the system, scheduled recordings won’t start, watch and ingest directories won’t process, etc.
Windows Service Name: Ensemble Service
Process Name (Task Manager): EnsembleVideoLiveStreamingService(32 bit)
You can monitor for the status of Ensemble Service and ensure that the EnsembleVideoLiveStreamingService(32 bit) is running as a process on the server.
The Redis Service runs on the Ensemble Server. This process provides session state storage for users while they are using the web application. This service is critical to the operation of the Ensemble application.
Windows Service Name: Redis
Process Name (Task Manager): Redis-Server
The Wowza Service provides the VOD playback as well as Live Stream viewing for Ensemble Video. There are a couple of items to monitor for Wowza Service:
The Wowza service uses a number of different ports for streaming. Below is a list of ports and URLs you can monitor to ensure the different streaming protocols are enabled and working. These ports should be accessible from both inside and outside your network (if you are allowing external viewing of videos).
HLS Playback (443)
RTMP Send/Receive (1935)
RTSP Send/Receive (554)
Wowza Windows Service
You will need to monitor for Wowza service to ensure that it is running.
Windows Service Name: Wowza Streaming Engine 4.x.x
Process Name (Task Manager): Java™ Platform SE Binary
**Note: Wowza Process Name may differ depending on your version and update level of the Java platform you are running on that server**
The Squeeze service is rather simple to monitor. The service itself as well as disk space are the main areas to monitor. CPU usage will vary depending on transcoding load, however if the server is consistently maxed out on CPU, you may want to add more resources. Sorenson recommends a 2:1 ration of RAM:CPU. (i.e. 8 core VM should have at least 16GB RAM).
Windows Service Name: Squeeze Server
Process Name (Task Manager): Squeeze Server Service