Overview

The RM statistics service enables backend applications to provide statistical data about their runtime performance (System Statistics) as well as statistical data for the connected devices (Device Statistics). This could be, for example: total number of users connected to a specified port; average humidity level; price level changes, devices connected, traffic for each device, usage of a specific device or a group of devices, etc. Different applications may have different needs about the form of the data. For example, some may need to report about their peak and lowest values, others may provide information about the average value of the produced data, third may need counting information. In addition, the applications providing statistics may be spread on multiple hosts, with systems on different hosts providing related statistics (for example applications running on different hosts may provide statistics about the connections made to their hosts).

The RM statistics service is the central point where statistical data are stored and can be accessed. Through it the user can obtain a list of all statistics, or a combination of related statistics, obtained from different hosts or a single host of interest. The statistics service stores the statistical information permanently in the system database, so it can be retrieved at any point of time or alternatively information may be stored temporary in the memory.

Statistics Service Features

Statistic Properties

Each statistic contains the following basic properties:

• Group - a name (user friendly, shown into UI) that is used as a facility for grouping different statistics into groups. This could be used to declare that some similar statistics shall be shown under same group. Also this name could provide high level common description for all the statistics from a group. For instance all OS related statistics could be in group called "OS" (or "OS Statistics" depending on the way you want to be represented into the UIs). The statistic MUST always specify a group (i.e. The Group could NOT be null).
  
  
• Name - the name (user friendly, shown into UI) of certain variable/quantity that we want to have statistic data for. For instance it could be "CPU Usage", "Free Memory" or "Net traffic". The pair (group, name) identifies the variable quantity that a statistic represents. The statistic MUST always specify a name (i.e. The Name could NOT be null).
  
  
• Source - a source is represented by two related properties type and name (e.g. Source Type and Source Name). A statistic may support single source (i.e. singleton source) or could support more thann one sources.

For instance source could be: (type: "System", name: "System"), (type: "mprm.osgi.device", name="testdevice") or (type: "Bundle", name: "com.foo"). The sources could be compound/composite. If we have source that is related to a device and a bundle, we could represent the source as a "bundle on a device": (type: "device/bundle", name: "WER334/com.foo" - this represents bundle: com.foo of device: WER334).

For instance a Control Unit (CU) may be present with type: "factory.type" and name="core1" if on RM or type:"device/factory.type" and name="WER334/core1" if on Device:WER334

Types of Statistics Reports

Every statistic could have one ore more reports. A report represent an aspect of the quantity, e.g. minimum, maximum, average.

The report names could represent a view/report the statistics service stores the statistical information a quantity. The StatistcProvider just have to name an report name and this one will be used for calculating its info. The Statistics Service allows registering of different com.prosyst.mprm.backend.statistics.spi.ReportProvider.

A statistic describes the behavior of a variable during a defined period of time. The statistical method used for representing the variable's behavior can be different depending on the application's needs. The RM statistics service allows bundles to provide the following types of statistical reports:

• Average value for a defined period - The statistical report shows the average for all values taken by the observed variable during a period of time. This type of statistic is useful when needing information about a variable that changes often and we are not interested in each single value occupied but in the average level for the period. This could be, for example, average temperature, humidity, currency exchange rates, etc.
  
  
• Sum of all values taken during the period - The statistical data is calculated by adding all values taken by the variable. This form can be used when needing counting information. For example: total number of users that have used a service.
  
  
• Maximum value - This statistic shows the peak value of the observed variable for a defined period. It is useful when we are interested in the changes of the variable during the period. For example: highest scores from a game.
  
  
• Minimum value - This statistic shows the lowest value of the observed variable for a defined period.
  
  
• Last value - The information shows the latest value occupied by the variable.

A bundle can provide more than one statistical report per variable/statistic and also provide statistics about more than one variable. A single variable/value/quantity may be included in statistics with different report types (Min/Max/Average, etc.), this report listing shall be for instance into the report statistic property part. For example, the information about the users connected on a defined port can include average number of users, minimum and maximum number of users for a period, and the currently connected users.

Current value: Statistics API supports query for the current value of the quantity behind a statistic. So the user could pool/retrieve this current value as long as the statistics provider supports current value retrieval.

Statistics Providers

Statistics could be provided using two methods:

• com.prosyst.mprm.backend.statistics.spi.StatisticProvider - an SPI that provides statistic information - group, name, report names as well as issues on request statistic reports, source lists and so on. Statistics Providers listen for statistic events and push the statistic data into the Statistics Service which is responsible to collect it and to generate statistics on demand. com.prosyst.mprm.backend.statistics.spi.StatisicProvider-s must be registered on the backend as OSGi services. It is the method the common integrator/service provider is expected to use.
• com.prosyst.mprm.admin.statistics.Statistic - alternative way for providing statistics is not to put measurements into Statistics Service (the way the StatisicProvider does) but to be retrieved on demand from the Statistics service. For instance when statistic history/data is kept into different database it may be more optimal to be retrieved from there on demand, instead of putting (and duplicating it) into the Statistics Service. This pattern of providing of statistics is by simply registering com.prosyst.mprm.admin.statistics.Statistic service as an OSGi service.

Persistence

Statistics Providers that register com.prosyst.mprm.backend.statistics.spi.StatisticProvider service could specify which database shall be used for keeping their records. Out of the box the RM supports two types of databases - In Memory and RDBMS. While the former one loses its data when RM is restarted (and has to consider the statistics size and count in order to not use too much memory), the later stores info into a persistent storage - RDBMS, it is configured with. Therefore the statistic providers may use persistence if needed.

The databases that could be used by these providers could vary because the RM allows registering of Database providers which could then be used by statistic providers for storing data. For instance one could implement a Database for a specific NoSQL database.

The persistence of the statistics provided via com.prosyst.mprm.admin.statistics.Statistic, is not in the scope of this document. It is up to the providers' implementation.

Compression

Since the statistic data could grow up in time and the statistics types (they are bound to different - potentially millions of sources) the user has to take care about the storage volume. Generally there are two options:

• to drop the oldest statistics when a threshold is reached.
• to compress the oldest data - for instance the user could keep the last 100 records of chunks of 5 sec, the next 100 records of chinks of 30 sec, the next 100 records of chunks of 10 min, the next ... , the next 100 record of chunks of 1 month.

The RM Statistics Service implements both approaches for keeping the history relatively small. It provides compression allowing to have "more" data on a relatively small cost and drop the oldest data if the event compressed data becomes too big. The RM Statistics Service keeps periods of sizes: minute, hour, day, month or year (i.e. calendar centric intervals) and allows the StatisticProvider services to specify how many periods from any size shall be used.

Generally the minimum count for the period sizes are: 60, 23, 30, 11, 0 (correspondingly) . However in some occasions the statistic provider could choose to not use the minute periods and to have only hour-granular statistic. It is possible to use less periods by having for instance periods of 5 minutes.

If the StatisticProvider wants to keep more (let say minute) intervals it could specify e.g. 120 minute intervals. Then queries for the last 2 hours will contain minute-granular statistics, while the queries for the last 3 hours - hour-granularity.

Utilities

Generally the providers could be PUSH and PОLL/Periodic (the statistics are being pulled at a defined interval in order to put the statistics into the statistics service):

• Push Provider - it is responsible to push data into the statistic service when it decides to that (e.g. Collecting user log-ins). This provider is proper for an event statistics. An implementation of such provider could use com.prosyst.mprm.backend.statistics.utils.BasicStatisticProvider for an easy implementation.
  
  
• Pоll Provider - a provider that is periodically (at some interval) asked for values (e.g. CPU usage). This provider is proper for statistics that have no events for changes. An implementation of such provider could use com.prosyst.mprm.backend.statistics.utils.BasicPeriodicStatisticProvider.

User Interface

The statistical data generated by applications can be browsed through console. They are displayed when you select Dashboard -> Statistics. See Statistics from User's Guide for details.

Statistics API

The Statistics API allows custom backend bundles to provide statistical data to the system, and other applications (backend bundles or non-RM applications) to retrieve statistical information from the statistics service.