The Data Reducer Agent is a component of EQuIS Live that can be used to reduce the amount of data points of an input series using the Douglas Peucker algorithm.
Reducing Data
The Data Reducer Agent can be run as a scheduled EQuIS Information Agent (EIA), at the desired frequency, to automatically load data as often as needed.
Setup
To use the Data Reducer Agent to produce a reduced data series, follow these steps:
1.Create a new series in the DT_LOGGER_SERIES table.
a.An existing LOGGER_ID can be used, or create a new logger in DT_LOGGER.
b.Make sure to execute the equis.get_ids stored procedure to get a new LOGGER_SERIES_ID, or add the series (DT_LOGGER_SERIES row) in EQuIS Professional.
2.Set SERIES_NAME, SERIES_DESC, and SERIES_UNIT as desired.
3.Set DERIVATION equal to 1.
4.Set SERIES_FUNCTION to EarthSoft.Live.Reports.DataReducerAgent.
5.Set SERIES_FUNCTION_INFO to a JSON (JavaScript Object Notation) string that contains the following:
a.sourceSeriesId – The LOGGER_SERIES_ID value that represent the input data for the reduced series; the input series may come directly from a data logger, or it may be another reduced or derived data series.
b.window – Break the input series data into 'windows' of this size, and pass each window to the Douglas Peucker algorithm to reduce the data; must be one of the following:
i.mi : One value every minute (i.e., minutely).
ii.hh : One value every hour (i.e., hourly).
iii.dd : One value every day (i.e., daily).
iv.wk : One value every week (i.e., weekly).
v.mm : One value every month (i.e., monthly).
vi.yy : One value every year (i.e., annual).
c.epsilon – The epsilon (i.e., granularity) value used by the Douglas Peucker algorithm. Larger epsilon values reduce the data more.
Example:
o{sourceSeriesId:1234,window:"dd",epsilon:1.23}
Using the input series with LOGGER_SERIES_ID = 1234, break the data into daily windows (starting at midnight of each day), and reduce the data for each day using the Douglas Peucker algorithm, and an epsilon (i.e., granularity) of 1.23.
6.Set STATUS_FLAG to 'A' to activate the series.
A reduced data series may be used as a normal series in other components of EQuIS Live (charts, agents, etc.).
Pseudocode
For each series with SERIES_FUNCTION = EarthSoft.Live.Reports.DataReducerAgent
{
Using SERIES_FUNCTION_INFO, e.g., { "sourceSeriesId": 1234, "window": "dd", "epsilon": 1.23 }
{
For each window, e.g. “dd” (1 day), starting where we left off last time, till the end of the source series's data
{
Add data points from the source series to the reduced series, using the Douglas Peucker algorithm and epsilon, to get an approximation of the source series. Using the example, and 3 points (P1, P2, and P3), add points P1 and P3 to the reduced series, but do not add point P2 if it is close enough to points P1 and P3 (i.e., if the perpendicular distance of point P2 from a line made from points P1 and P3 (the length of the red line in the image below) is less than 1.23 (epsilon)).
There is a more detailed explanation of how the Douglas Peucker algorithm works (with more than 3 points) at sourceforge.
}
}
}