Measurement Data

Introduction

Measurement data represents the essential core content of Sedrix – and your valuable working capital as a user. In order to manage, represent and analyze etc. this data, there is a certain amount of necessary effort “in and around” this core. The majority of this handbook describes this. This chapter sets out the essentials surrounding it, that is, the relevant measurement data:

• Measurement data properties in Sedrix

• The import of measurement data into Sedrix, measured by sensors or measuring devices such as measuring systems (like the MCC):

  • automatisch (standard case)

  • manually

• Calculation and analysis options, filtering data

• Representation of measurement data in Sedrix; ± general configuration options that are available for this purpose, e.g. with regard to

  • the number of decimal places

  • representing status or error codes

  • selecting units etc.

General Information About Measurement Data

When monitoring time-dependent processes, a single date must be described with at least two values:

• the actual measured value

• a time stamp,which identifies the time at which the value was measured.

In addition, in Sedrix each measured value carries a status and error code, by which a certain (error) state can be indicated.

Display Measurement Data

Measurement data can be displayed in the measurement point view of a measurement point by the means of so-called evaluations, e.g. as diagrams or tables. For this purpose, the measuring device which supplies the data to be displayed must first be linked to the measuring point. This is done by creating a so-called assignment. The steps required for this are described in detail in the sections linked above.

Automatically Upload Measurement Data

The core element of Sedrix is that measuring devices/measuring systems are capable of automatically uploading their measurement data to “their” Sedrix. Depending on the device type, this is either done via FTP or via a WebSocket connection.

This requires the devices to be configured so that they know “their” Sedrix. For more details, refer to the descriptions under Add Devices for the various device types. Please also refer to the operating instructions for the respective device.

For devices that upload their data via FTP, this includes the corresponding access data. Please consult your system administrator.

Manually Importing Measurement Data

Sedrix also makes it possible to manually import and export the measuring data from certain device types. In the subsections below, you will find the file formats required for importing data.

Note on licensing: This functionality requires separate licensing

Very important: When creating data files, adhere exactly to the respective format as specified below in order to avoid problems during processing. This applies in particular to notating decimal numbers: Unless otherwise specified, please use a point (period) as the decimal separator, and do not add a point as thousands separators (i.e. this example is incorrect: 1.234,5 for the number one thousand two hundred and thirty four point five. This should be written instead as 1234.5).

Warning: Special care should be taken when using programs like Excel to manually create CSV files containing measurement data! Make sure that numbers are also saved in the format described above (depending on the language settings of the computer / the settings of the program this may not be the case!), and ideally check in a text editor before uploading!

Import Manually Measured Tachymeter Data

Importing manually measured tachymeter data, presented as a CSV file in the “Sedrix Tachymeter CSV” format (see below), can be done as described below (from version 2.11):

1. Open the device overview in the desired project and set up a “generic tachymeter” device there (if this has not already been done). The folder to which the data is to be uploaded should be entered here (relative to the data directory of the project). This is known as the import folder.

2. Upload the CSV file with an FTP client (e.g. FileZilla), from your computer to your Sedrix into the import folder specified under step 1.

The file will then be automatically processed by Sedrix, and the data can be displayed in a measuring point just like the data from other devices. See Display Measurement Data for the necessary steps.

“Sedrix Tachymeter CSV” Format

The necessary format of a CSV file used for this purpose is described in the following example file:

Station;Punktgruppe 1;;;;;;; Measurement time;2018-04-29T18:34:00+02:00;;;;;;; Start;2018-04-29T18:34:00+02:00;;;;;;; End;2018-04-29T18:34:00+02:00;;;;;;; Pressure [mBar];950.3;;;;;;; Dry temperature [°C];14;;;;;;; Wet temperature [°C];20.4;;;;;;; Tachymeter length inclination [gon];0;;;;;;; Tachymeter cross inclination [gon];0;;;;;;; Coordinate system; CH1903+ ;;;;;;; Messpunkt;Measurement time;E;Unit;N;Unit;H;Unit;Durchgänge MTh_5;2018-04-29T18:34:00+02:00;17.00297406;m;6.978666506;m;1.328789513;m;1 MTh_6;2018-04-29T18:34:00+02:00;16.29822192;m;0.552248402;m;1.302087357;m;1 101;2018-04-29T18:34:00+02:00;8.585497723;m;6.511685539;m;1.007654771;m;1 102;2018-04-29T18:34:00+02:00;8.790657113;m;6.791447731;m;1.006665711;m;1 103;2018-04-29T18:34:00+02:00;8.774723639;m;7.112170485;m;1.005923748;m;1 104;2018-04-29T18:34:00+02:00;9.333302668;m;6.266344943;m;1.008357046;m;1 105;2018-04-29T18:34:00+02:00;9.280260556;m;6.621149192;m;1.006806306;m;1 106;2018-04-29T18:34:00+02:00;9.521870116;m;6.88304942;m;1.004014369;m;1 

Further details:
Such a CSV file describes the coordinates for one or more geo points of a station or device measuring point for a certain measuring time (measuring period). In this example:

• The header defines the device measuring point Punktgruppe 1 for the measurement period 2018-04-29T18:34:00+02:00, followed by meteorological data (among other things) from the measurement time, along with the coordinate system being used.

• Important: The coordinate system given in the CSV file must be the same as the coordinate system specified when setting up the generic tachymeter!

• The data section defines the measurement results (coordinates) for the geo points MTh_5, MTh_6 and 101 to 106.

Import Manually Measured Logger Data

Importing manually measured logger data, presented as a CSV file, can be done by following these steps:

1. Open the device overview in the desired project and add a “generic data logger” device (if this has not already be done). The folder to which the data is to be uploaded should also be given (relative to the data directory of the project). This folder is called the import folder.

2. Upload the CSV file with an FTP client (e. g. FileZilla), from your computer to your Sedrix and into the import folder specified under step 1.

The file will then be automatically processed by Sedrix, and the data can be displayed in a measuring point just like the data from other devices. See Display Measurement Data for the necessary steps.

Formats

“Sedrix Logger CSV” Format

1. Column: Time stamp plus time zone.
   Example: “Date (UTC-5)”

2. Column: Channel name.
   Example: “Temperature”

3. Column: Unit.
   Example: “Unit”

Columns 2 + 3 can be re-used for further channels, if desired.

Examples for two channels:

Date (UTC-5);CA1;Unit;Asentamiento CA1;Unit; 31.01.2018 16:00;1,3123;bar;0,00;cm; 03.02.2018 08:43;1,3215;bar;-9,41;cm; 07.02.2018 08:37;1,3265;bar;-14,48;cm; 09.02.2018 08:40;1,3284;bar;-16,47;cm; 28.03.2018 09:54;2,4042;bar;-16,47;cm; 30.03.2018 10:09;2,4120;bar;-24,43;cm; 04.04.2018 10:43;2,4189;bar;-31,49;cm; 06.04.2018 09:30;2,4228;bar;-35,47;cm; 11.04.2018 09:30;2,4303;bar;-43,07;cm; 13.04.2018 10:40;2,4303;bar;-43,07;cm; 18.04.2018 11:00;2,4354;bar;-48,32;cm; 20.04.2018 14:10;2,4303;bar;-4

Filtering Measurement Data

From version 2.14, the Filter function can be used to used to filter time series of measured data from devices in the tachymeter and logger modules, before they are analyzed (in particular with regard to alerting), processed (the calculation of function blocks), or displayed (evaluations).
The currently used filters average measured data according to certain predetermined criteria, so-called “Moving Average” filters. The current measured value is determined by averaging a given number of past measured values, whereby further criteria can be applied to these values. This number of past measured values effectively forms a window – the “averaging window”. The number represents its (maximum) “width”, which can still be limited by the other criteria mentioned. See the description of the specific filters below.
In order to calculate a specific measured value, you can imagine that this averaging window is positioned over the measured value, and “releases” or selects a certain number of measured values of the past time series for averaging. In the course of the calculation for the whole time series, this window is incrementally shifted from measured value to measured value, starting at the oldest and moving towards the most recent. The mean value for each current measured value is recalculated.

Note: It follows from the process sketched out above that filtering is not possible for a certain number of the oldest measured values within the time series, because there are not enough older values available (the error code 910 is given for the measured values concerned – also see the Notes below). When interpreting the results, therefore, a certain “running-in time” must be taken into account, which depends in particular on the selected window width and the measuring cycle with which the time series was taken.

Available Filter Methods:

Simple Moving Average – SMA

The current measured value is determined by averaging a number of past measured values. When configuring this filter, a “maximum measured value age” relative to the current measured value time stamp must be specified, along with the window width.

This means that the new value to be calculated for the current point in time by this filter process results as a simple arithmetic mean from the previous values in the averaging window, whereby a measured value in the window only enters into the averaging if its time stamp (“age”) is not older than the specified maximum age, relative to the time stamp of the current measuring point.

When averaging new measured values, this particularly prevents the inclusion of irrelevant, too-old values (which may still lie solely in the given averaging window) in the case of (correspondingly long) pauses in measurement.

Average Variance

The averaging is carried out initially as for SMA, but in the end the measured values within the averaging window are subjected to a further check: only measured values which are within a specified limit, defined as a multiple of the current standard deviation and are found around the current window average, are used in calculating the final mean.
This can be used to suppress outlying values.
In this averaging process, the “age criterium” described for SMA can also be applied to measured values within the averaging window.

Set Filtering for a Channel

The filtering process can be defined separately for each measuring point channel. This is done in the measuring point assignments; the filter settings are therefore sub-properties of assignments. These properties are described more fully in the next subsection.

Filter Properties

• Filter
  Dropdown menu to select a type of filter
  Default: “No filter”. The additional properties described below are only displayed if a specific filter type has been selected.

• Window width
  Text field to specify the number of past measured values. This determines the width of the averaging window(metaphorically speaking).
  Constraints: This integer must be greater than 2 (otherwise there would be nothing to average). The larger the value chosen, the greater the computational effort.
  Default: 3.

• Filter width in hours
  Text field to specify a time span as the maximum age of measured values within the averaging window, relative to the currently calculated measured value.
  Constraints: This time span must be greater than 0 and smaller than 7 * 24h (Logger) or 14 * 24h (Tachymeter).
  Default: 6 hours.

• Multiple of standard deviation – only for the “Average Variance Filter”:
  Text field to specify multiple of the standard deviation.
  Constraints: This factor must be greater than 0.
  Default: 1.

• It is recommended that you note the selected filter type and (if applicable, its properties such as window width and filter width in hours) in the name of the assignment. On the one hand, this helps to easily distinguish filtered channels from unfiltered ones (e.g. for selection as an input channel in a function block). On the other hand, this means that the legend of a diagram shows that a curve belongs to filtered data.
  Similarly it is recommended that you name to output channels of function blocks accordingly, if they are filtered by input channels.

• The filter width in hours must be made large enough so that for the measuring frequency of the channels being filtered, enough measured values are within the window!
  A measured value that has fewer than the required number of values within the window is indicated by the error code 910 “Not enough data for filtering” (shown if the filtered channel is displayed in a table and that table has been set up so as to display error codes).

• Important: If the data of a filtered channel is displayed simultaneously in a table with that of other channels (e. g. unfiltered channels), the error code 900 “No data” is shown for measured values with the error code 910, since these measurements are not taken into account during the normalization of the channels (= on a single time grid). This means that they are considered as non-existent and then interpreted as “No data”.

Internal Calculation and Analysis Options

The raw data supplied by measuring devices can be processed internally in various ways:

• Vibration measurement: FFT etc.

• Tachymeter and data logger: Function blocks, see Function Blocks

• …

Status and Error Codes

Measured values can, in a real world, be faulty. This is not, however, a question of any measurement inaccuracies (this is outside the scope of responsibility of a data portal such as Sedrix), but rather because of measurements that are missing or incomplete due to technical reasons. In addition to the actual value and its time stamp, each measured value in Sedrix therefore also carries an error code – a numerical value – as a further property that describes the cause of the error in more detail. Normally (ideally) the value of the error code is 0, i.e. the associated measured value itself is (“ok”) error-free (technically; apart from any measurement inaccuracies, see above).
An error code can also be used to indicate that a value could not be calculated during processing / analysis in Sedrix (e.g. due to a faulty configuration of a function block), or that a value – according to previously defined criteria in a permitted manner – was determined by interpolation (e.g. for the representation of the data). In the latter case, it is not a errorcode in the actual sense, but a status code, which only has an informative purpose ( internally status codes are managed/stored by Sedrix however in the very same way like error codes, and also displayed in the user interface).

Error classes
Status and error codes are divided into the following error classes according to the severity of the states or errors they indicate:

• Codes < 1000: Status codes (no errors).
  Examples:

  • Codes 9, 8, 10: Value has been interpolated

  • Code 900: No data

  • Code 905: Value cannot be calculated

  • Code 910: Not enough data for filtering

• Codes >= 1000 but less 2000: Light errors
  Example:

  • Code 1000: Measured value available, but uncertain

• Codes >= 2000 but less than 3000: Medium errors
  Examples:

  • Code 2000: Measured value available, but very likely invalid

• Codes >= 3000: Severe errors (no measured value present)
  Examples:

  • Code 3001: Reference measurement not found

  • Code 3220: Invalid measured value

Display of status and error codes
Status and error codes can be displayed in table evaluations if required. These evaluations have a settings option in the form of a dropdown menu “Display error codes” “ (in the Edit mode of the measuring point, see Measuring Point View) along with the entries.

• “Show error codes (only error codes > 10)”

• “In separate column (all)”

• “In separate column (only error codes > 10)”

• “No display”

If one of the options for displaying error codes is selected here, any status or error codes that may have occurred are displayed in a small box (if specified in a separate column).
The color of the box provides a quick indication of the severity of the error:

• Light blue: Status codes and light errors

• Orange: medium errors

• Red: Serious errors

Position your mouse over an error code box to get a short textual description of the error as a tooltip.