Release Notes 2.5.1 (March 4th 2016)

Viakoo Release Notes 2.5.1

(March 4th, 2016)

Viakoo Release 2.5.1 is an update for the Viakoo Service focused on throttling down server offline alerts and a server Priority control to adjust this behavior.

Server-Offline Throttling

Background

For the Viakoo Service to detect when a Server has gone down, the Communications Agent (CA) relies on certain network protocols to interrogate each recording server. There are three separate measures used to detect whether a Recording Server is having trouble. The primary method is to contact the Reader Agent (RA) running on that server. If the RA is able to respond then we know that the associated Recording Server is still functional. If we fail to get a response from the RA, then we check two other measures: responsiveness to PING and HTTP requests.

PING

A PING request on the associated IP address of the server indicates that the server is electrically still alive and connected to the network. This test is unfortunately unreliable because PING does not guarantee a response even when things are okay, so a failed PING does not necessarily mean that the server is disconnected. The CA will repeat the PING test several times to try and improve confidence in this measure when attempting to verify that a server is offline.

HTTP Test

The installation of the RA creates a basic listener on a different HTTP port. If this port is still active, it indicates to the CA that the OS on that server is still operational, even if the RA is having trouble responding for whatever reason. The simplicity of this extra listener means it is unlikely to crash even in situations where the RA is having trouble. If both the listener and the RA are unresponsive, this is a strong indicator that the Server’s OS is having trouble or is hung. There are a small percentage of cases where this test fails due to network connectivity issues, or the HTTP listener crashing while other services continue to run, and the server may still be functioning fine.

Condition

Ticket Category

RA response

HTTP response

PING response

RA is responding normally (Server is clearly OKAY)

n/a

TRUE

(any value)

(any value)

Server OS is okay but something is affecting RA*

HOST_OFFLINE_PINGABLE

FALSE

TRUE

(any value)

Network between CA and Server is okay and Server is electrically alive but Server OS may have crashed.

HOST_OFFLINE_UNREACHABLE

FALSE

FALSE

TRUE

Server maybe down or disconnected from CA**

HOST_OFFLINE_UNPINGABLE

FALSE

FALSE

FALSE

 

In the condition where the server appears to be up and running but the RA is not responding, it can sometimes be correlated with issues detected before this event. For example, if available freespace on the Recording Server’s system partition (C:\) is seen to be low prior to the RA not responding, then the RA issue is most probably associated with the free space problem. Other examples include issues that arise from PING_HANG, or PENDING_REBOOT events opening on that server.

Alerting on Server (Host) offline

There are two scenarios we have identified from speaking with customers where server offline alerts are being sent when it would be better to not be notified:

  1. False Alarms - As explained above, there are times when these networking protocols aren’t completely reliable when identifying that the server is actually down. Though the service will attempt several retries of HTTP and PING, with the variety of network topologies and other factors there are bound to be instances where the server will appear down when it is still operating normally.
  2. Server Recovers Automatically - For many situations, servers and the necessary services can recover automatically within a short period of time without requiring human intervention.  This is distinct from a false alarm as the server was truly offline, but the user experience is very similar because by the time someone gets to check on the problem they find everything running smoothly.

In light of these two scenarios some customers have asked that they not be alerted until it is clear the server is having trouble AND that it is not going to recover on its own. Therefore, in Release 2.5.1, we’ve introduced both a throttling mechanism for server tickets and the ability for users to leverage a server’s priority to have some control over this new mechanism. For all “NORMAL” priority servers this new mechanism will open an Elevated ticket when the server first reports as offline, which will not send out an alert. Then after 40 minutes, if the server still hasn’t recovered, it will change to a Failure ticket and send out an alert. We also understand that there are many of you who would still like to receive an alert as soon as Viakoo sees that a server is not reporting, and this is where the priority comes into play. Setting a server’s priority to “CRITICAL” will cause the Viakoo service to open a Failure ticket and generate an alert immediately upon detecting the server as offline (i.e., equivalent to prior behavior).

Viewing and Setting a Server’s Priority

A server’s priority shows up in the top-level summary of a server on the “Details” tab as well as a column in the Server table for a site.

From a Server’s Details tab, you can change the priority by clicking on the “Edit” button in the upper-righthand corner.

After you change the Priority and/or Type, you then need to click “Save” to have the changes take affect.

At the site-level, you can change one or more server’s priority by entering into edit-mode for the associated server table in the Site Details tab. To go into edit-mode, click on the “Start Edit” button.

Edit-mode on a server table allows you to change one or more servers’ Type and/or Priority by allowing you to change those values individually by selecting the pulldown for each server’s field.

Server Priority can be changed in bulk for one or more servers at a site by selecting multiple servers, setting priority for all the selected servers, and then clicking the “Save” button.

 



More information

If you have any questions, comments, bug reports, or suggestions, please reach out to us through the live-chat feature or contact us at feedback@viakoo.com.

We love hearing from you!

-Team Viakoo

1 (855) 585-3400

Have more questions? Submit a request

Comments

Powered by Zendesk