Lake Ecosystem Research New Zealand
Automated collection of environmental monitoring data is becoming increasingly commonplace. High-frequency datasets provide uniqueinsights into ecosystem dynamics, however, the quantity of data collected is often at computational limits for existing software applications. Accurate and comprehensive quality assurance and quality control are vital for the effective and appropriate use of high-frequency datasets. LERNZ has developed standalone, open-source suite of software including B3, designed for the QA/QC ofhigh-frequency monitoring data. It is a more efficient alternative to environmental data processing in Microsoft ExcelTM, with an intuitive interface and lower 'barrier-to-entry' than more technical software packages such as MATLABTM. A range of QA/QC tools are currently included in B3. Detailed site and sensor metadata are stored, including calibration points which can be used to correct for sensor drift. A range of sensor-specific limits can be set and a variety of methods are provided for rapid error/outlier detection. Subsets of data can be transformed or deleted by selection within a visual plot. Selected data ranges can be mathematically transformed, and sensor drift between calibration points can be quickly and easily back-corrected. Furthermore, B3 retains a full log of editing changes including user-input reasons for any changes, and modified data can be reverted to raw data at any time.
As well as B3, LERNZ has also developed Data Standardizer, an application which takes raw field data from high frequency buoys and standardizes in a common matrix format. Data Standardizer then cleans the data by removing any empty or waste lines and by bringing the header into a standard format. A range of formats are then available for export, including .txt for B3, .wtr, .wnd and.sal for Lake Analyser and the general format .csv. It also can aggregate data to a preferred time step, and outputs a Meta data file that can be directly read in by B3 to provide all the site data necessary, as well as the sensor metadata including calibration details, brand, type etc.
Data Standardizer User Guide
B3 Users Guide