Argo pressure correction audits
Auditing the Argo GDAC archive to improve standardisation of pressure
correction and of meta data,
and to measure and reduce pressure errors
Introduction
Pressure data from different types and generations of Argo floats requires different treatments
to make correct use of surface pressure readings. The Argo community has worked to understand
and agree on what treatments are required by which floats, and also on standard elements in the
metadata allowing third parties to carry out or verify the surface pressure corrections.
Since August 2010 we have run assessments of the global archive to determine how well the
metadata identifies float type (as it relates to pressure correction), and the extent to
which the agreed pressure correction is being applied. We have used this process to provide
input to efforts to standardise the metadata and the treatments, and to provide feedback to
PIs and national DACs so they can modify files where required.
During this period there has been steady refinement of metadata standards, and improvement in
implementation of metadata and pressure correction. This has lead to bias reduction and improved
accuracy of the pressure dataset, which in turn potentially improves global climate signal
estimates.
Method
All R and D files except those grey-listed are exaimined. Key fields in meta and tech files are
scanned for standard strings. To increase the likelihood
of identifying standard strings we have ignored Case and the placement of spaces. Tables of
equivalent strings have been created to assist in some cases. Arguably even fields such as
PROJECT_NAME and PI_NAME should have standard entries or codes, to assist matching and
aggregation of statistics.
It seemed the critical piece of metadata for determining pressure correction is the
Surface Pressure Offset name ("SPO name") in the TECHNICAL_PARAMETER_NAME field. By constructing
a table prescribing the relationship between this name and the type of pressure correction we
can uniquely identify the required correction by reading just this one metadata field. This is
desirable because it should be simple and foolproof, but also because other existing fields, such
as PLATFORM_MODEL, SENSOR_MODEL, SENSOR_SERIAL_NUMBER, are mostly poorly populated or very far
from standardised. However, we already have exceptions where the float type (SOLO) is also needed
in order to determine the correction, and also where one SPO name is associated with two possible
treatments.
Defects which we have sought to eliminate include:
The value associated with the SPO name is also decoded. It is regarded as being valid if it is
present and lies between -100 and +100. The series of SPO values for each float is pre-processed
in the recommended manner for R and D files, omitting however the recommended manual assessment for
delayed-mode. Where the pressure correction method is able to be determined we apply that
correction to the contents of the PRES field. The result is compared with the existing apparent
pressure correction, deduced from the difference between PRES and PRES_ADJUSTED.
Missing ADJUSTED data
Many profiles have some missing data in the PRES_ADJUSTED or PRES_ADJUSTED_QC field (where
there are corresponding values for PRES and PRES_QC). This is much more common in
R-files but also occurs in D-files. This is different to cases where pressure data has been
assessed and flagged as bad. It is obviously serious as the user community is being deprived
of this data. This condition is highlighted on audit plots and listings, and also quantified
in the audit Pressure Correction summary table. Obviously a knowledgable user could apply the
pressure correction for themselves but first they need to correctly identify the correction
appropriate to that float.
There are many variations to this fault. Technically, ADJUSTED fields only have to be populated
in R-files if an adjustment to the raw data is required [but if populated for ANY parameter they
are then required to be populated for ALL parameters]. However, since users will prefer to
access ADJUSTED fields it may be warranted to populate these even if just with a copy of the
raw data? Some Apex floats have missing PRES_ADJUSTED fields - which is not acceptable because
this data requires correction,
The actual algorithm to detect this condition is (as of Nov 2011):
if PRES data and (empty PRES_ADJUSTED or empty PRES_ADJUSTED_QC or
( all PRES_ADJUSTED==Fillvalue and any PRES_ADJUSTED_QC<4 ))
The final pair of conditions is to allow ADJUSTED fields filled with Fillvalue if and only if
the QC values indicate the data has been intentionally flagged as bad. A potential fault with
this
test is if the data values have been overwritten with Fillvalue where QC=3 [I don't know
whether or not this is accepted practice.]
A great many cases arise because PRES_ADJUSTED_QC appears to be empty when decoded with
Matlab str2num function, due to the presence on a value (such as 0) which does not translate
to a numeral. In future, such cases will be listed and reported to the DACs.
A less common fault is where the difference between PRES and PRES_ADJUSTED varies
throughout the profile. (SOLO floats may have the deepest pressure value adjusted to correct
the average for that bin. Obviously we do not regard that as an error.) Previously we just
reported the floats with
this error but we now identify the individual profiles so that it can be more easily traced and
corrected.
Comparison of our correction and that performed by the DAC
The correction performed by the DAC can differ from our estimated correction for several
reasons:
The level of agreement is quantified by counting the number of profiles where:
Where the DAC determines for a given float that any discrepancies are due solely to more
correct judgements applied during DM processing, we record the float details in a list. That
float is then excluded from discrepancy statistics and the float's plot is identified with a
distinctive colour. This has so far been applied for a number of floats managed by John Gilson.
For float types where two possible pressure correction treatments are accepted, we test against
both treatments and notify where the secondary correction is used, or where neither correction
matches that performed by the DAC.
TNPD
Assessment of Truncated Negative Pressure Drift (TNPD) Apex floats is best done by visual
inspection by a knowledgable operator. However an algorithm is used which attempts to identify
the TNPD condition despite spikes and other bad SP values. It cannot always be correct, and
certainly cannot match an expert's assessment of the point of onset of significant negative
drift. However the resulting bulk statistics do indicate the level of skill and compliance
with the TNPD detection and notification methods.
Different levels of TNPD are recorded by adjusting values in PRES_ADJUSTED_ERROR,
PRE_ADJUSTED_QC
and strings in SCIENTIFIC_CALIB_COMMENTS. These strings are sometimes intended to notify TNPD
but do not precisely match the standard string. Such cases of "imperfect comments" are being
steadily corrected throughout the archive.
Reporting
Each audit is reported through a collection of webpages. These include tables analysing the
metadata, with statistics for each DAC, and listings showing occurences of a range of conditions
for each float, with a link to a surface pressure correction plot for each float. Tables also
quantify the pressure correction and TNPD detection and reporting for each DAC. The main page of
each audit also carries a commentary of significant changes or findings in that audit.
|