The Rich Inventory

From NGDCWiki

Jump to: navigation, search

The Rich Inventory is a granule metadata management tool developed by the Enterprise Data Systems Group at NGDC with support from CLASS and the NOAA Observing System Architecture Projects. The principle concept of the system is to extract existing metadata from granule headers, store it in a database, and make it avialble as part of a data discovery system.

Contents

Rich Inventory Architecture

The Rich Inventory System is made up of four components that are connected by web services and open source tools. The Calculator produces data for the rich inventory that is packaged in Rich Inventory Markup Language (RIML) and provided to the Loader using a web service. The data is loaded into the Database using Hibernate, an open source object-relational mapping tool. The Monitor uses SQL to check for alarm conditions that are then sent to the archive or the data provider using a notification service.

Image:RichInventoryArchitecture.png

Rich Inventory Implementations

The Rich Inventory has been implemented for a number of datasets at NGDC. These include NOAA Observing Systems that are being monitored as part of NOSA, Ocean Observations monitored by OSMC, Mesonet Observing Systems that are part of MADIS, and Level 3 Satellite Products produced by NESDIS and archived by CLASS.

Rich Inventory Reports

Rich Inventory and ISO Metadata

The Rich Inventory is designed to take advantage of the fact that many granule metadata elements remain constant over time from granule to granule. These periods of constant behavior are called segments in the Rich Inventory. In the ISO 19115 metadata standard a similar concept exists. In that case these periods are called temporal extents and they are part of the more general ISO extent object. The similarity between these two concepts creates a strong connection between the Rich Inventory and ISO metadata.

Rich Inventory Displays

Description Data Info Id Search Display Link
RIML Template Sets RI IDB Templates
View Table Descriptions and Histoggrams RIV30 Tables
Parameter Definition 102834 11 11 SST Definition
Parameter Definition 102771 11 11 Age of Most Recent Observation Definition
Attribute Value List 102834 20 20 SST Algorithm Attribute Values
Search for RIV30 Granule Type 102834 2 2 RIV30 Granule Type Search
NOSA specific Granule Type search 102834 24 203 RIV30 NOSA Granule Type Search
OSMC specific Granule Type search 102834 25 204 RIV30 OSMC Granule Type Search
Granule Type Overview 102834 2 3,4,5,7 OSMC Observations
Granule Type List (Informix) 102863 2 2 Granule Types
Granule List (RIV30) 102834 8 8 OSMC Observation Granules
Granule List (ri_annotations) 102771 9 8 DMSP Granules
Granule List (ri_annotations) 102771 5 17 14km SST Granules
Most Recent Granule Type List (RIV30) 102834 26 205 Most Recent RIV30 Granule Type
Granule RIML (RIV30) 102834 8 60,61 SST Granule RIML
Granule RIML (ri_annotations) 102771 8 50,51 DMSP Granule RIML
Granule Segment List (ri_annotations) 102771 5 3 DMSP Granule segment List
Most Recent Segment (RIV30) 102834 2 30 SST Recent Segment
Complete Segment List (RIV30) 102834 2 9 SST Complete Segment
Segment Report (RIV30) 102834 3 24,25 SST Report
Segment Report (ri_annotations) 102771 3 24,25 Age of Most Recent Observation Report
Recent Granules RSS Feed (RIV30) 102834 9 54 Recent Granules
RIPERF Granule Type search 102859 2 2 RIPERF Granule Type search

Rich Inventory Views

Database views are important for making it easier to retrieve data from the Rich Inventory.

RIV30.GRANULE_TYPE_SUMMARY_VIEW

create or replace view RIV30.GRANULE_TYPE_SUMMARY_VIEW as
select  RIV30.granule_type.name GRANULE_TYPE_NAME
,       RIV30.GRANULE_TYPE.ID GRANULE_TYPE_ID
,       RIV30.GRANULE_TYPE.METADATA_ID GRANULE_TYPE_METADATA_ID
,       RIV30.GRANULE_TYPE.DESCRIPTION GRANULE_TYPE_DESCRIPTION
,       RIV30.GRANULE_TYPE.LAST_UPDATE_DATE GRANULE_TYPE_UPDATE
,       RIV30.DATA_TYPE.ID DATA_TYPE_ID
,       RIV30.data_type.parameter_id PARAMETER_ID
,       RIV30.PARAMETER.name PARAMETER_NAME
,       RIV30.PARAMETER.standard_name PARAMETER_STANDARD_NAME
,       RIV30.PARAMETER.long_name PARAMETER_LONG_NAME
,       RIV30.PARAMETER.DESCRIPTION PARAMETER_DESCRIPTION
,       RIV30.PARAMETER.UDUNITS PARAMETER_UNITS
,       RIV30.PARAMETER.PRECISION PARAMETER_PRECISION
,       RIV30.PARAMETER.LAST_UPDATE_DATE PARAMETER_UPDATE
,       RIV30.ATTRIBUTE.ID ATTRIBUTE_ID
,       RIV30.ATTRIBUTE.NAME ATTRIBUTE_NAME
,       RIV30.ATTRIBUTE.DESCRIPTION ATTRIBUTE_DESCRIPTION
,       RIV30.ATTRIBUTE.LAST_UPDATE_DATE ATTRIBUTE_UPDATE
,       RIV30.ATTRIBUTE_VALUE.ID ATTRIBUTE_VALUE_ID
,       RIV30.ATTRIBUTE_VALUE.VALUE ATTRIBUTE_VALUE
,       RIV30.ATTRIBUTE_VALUE.DESCRIPTION ATTRIBUTE_VALUE_DESCRIPTION
,       RIV30.ATTRIBUTE_VALUE.LAST_UPDATE_DATE ATTRIBUTE_VALUE_UPDATE
from    RIV30.data_type
,       RIV30.granule_type
,       RIV30.parameter
,       RIV30.ATTRIBUTE
,       RIV30.ATTRIBUTE_VALUE
,       RIV30.DATA_TYPE_ATTR_VALUES
where   RIV30.data_type.granule_type_id=RIV30.granule_type.id
and     RIV30.data_type.parameter_id = RIV30.parameter.id
and     RIV30.data_type_attr_values.data_type_id = RIV30.data_type.id 
and     RIV30.data_type_attr_values.attribute_value_id = RIV30.attribute_value.id 
and     RIV30.attribute_value.attribute_id = RIV30.attribute.id
;


Rich Inventory Use Cases

Merge Parameters
Remove Granules and Segments
Segment Checker

Rich Inventory and RSS

RIV30 Updates during last 12 hours

Personal tools