Section 13. CDBS Control Relations and Tables
------------------------------------------------

13.1 CDBS Design
~~~~~~~~~~~~~~~~

During 1996 CDBS underwent a major redesign. The new simplified
structure was put in place in February 1997. The new design increases
the level of automation in the delivery process and permits more
detailed tracking. It also accommodates the new instruments, STIS and
NICMOS, with their new file structures and naming conventions. Each
instrument now has two relations, known as the file-level relation and
the row-level relation. These have names like ``acs\_file`` and ``acs\_row``.
The relations are linked by having matching values for file name and
expansion number.

**Table 13-1. File-Level Relation**

+-------------------------------+------------+
| Field                         |Format      |
+===============================+============+
| file\_name                    |varchar(50) |
+-------------------------------+------------+
| expansion\_number             |integer     |
+-------------------------------+------------+
| delivery\_number              |Integer     |
+-------------------------------+------------+
| reference\_file\_type         |varchar(50) |
+-------------------------------+------------+
| useafter\_date                |datetime    |
+-------------------------------+------------+
| opus\_flag                    |char(1)     |
+-------------------------------+------------+
| comment                       |text        |
+-------------------------------+------------+
| general\_availability\_date   |datetime    |
+-------------------------------+------------+
| opus\_load\_date              |datetime    |
+-------------------------------+------------+
| archive\_date                 |datetime    |
+-------------------------------+------------+
| reject\_flag                  |char(1)     |
+-------------------------------+------------+
| reject\_delivery\_number      |integer     |
+-------------------------------+------------+
| reject\_by\_file\_name        |varchar(50) |
+-------------------------------+------------+
| reject\_by\_expansion\_number |integer     |
+-------------------------------+------------+
| comparison\_file\_name        |varchar(50) |
+-------------------------------+------------+

**Table 13-2. Row-Level Relation**

+--------------------------+----------------+
| Field                    |Format          |
+==========================+================+
| file\_name               |varchar(50)     |
+--------------------------+----------------+
| pedigree                 |varchar(50)     |
+--------------------------+----------------+
| observation\_begin\_date |datetime        |
+--------------------------+----------------+
| observation\_end\_date   |atetime         |
+--------------------------+----------------+
| observation\_mode        |multiple fields |
+--------------------------+----------------+
| equivalence\_class       |multiple fields |
+--------------------------+----------------+
| expansion\_number        |integer         |
+--------------------------+----------------+
| comment                  |text            |
+--------------------------+----------------+

Whenever a calibration image is installed, a row is added to the
file-level relation and to the ro-level relation.

When a table is
installed, a row is still added to the file-level, but a number of
row-level entries may be made, one for each instrument mode covered. 

When
a delivery is made, the ``general\_availability\_date`` is set. The files
are delivered to OPUS and in parallel to DADS; this is done automatically
by the CDBS Pipeline. OPUS receives two more
files, an SQL command file and a catalog. The SQL commands will be run
to install in OPUS a version of the CDBS update. The catalog lists the
files to be added to the set of files maintained by OPUS to service the
pipeline. The catalog also lists any files that may be deleted because
they are superseded by the new delivery. When OPUS has updated its data
base, an e-mail message is sent to a predefined CDBS directory. The
message is detected automatically and causes the ``opus\_load\_date`` to be
set. Similarly, a message from DADS automatically sets the
``archive\_date``.

13.2 Load Files
~~~~~~~~~~~~~~~

A special file known as the load file (generated with the ``mkload`` task)
accompanies every delivered calibration file. It is largely derived from
the header of the file and therefore has matching values of all the
configuration keywords and ``useafter\_date``. The values of ``opus\_flag`` and
``level\_of\_change`` must be set by the deliverer. The name of the
comparison file is automatically generated from what seems to be the
best match but may be altered by the person delivering the files. A
file-level comment must be supplied, and row-level comments may be
supplied.

When ``mkload`` runs, it also checks to see if any of the keyword values 
going into the ``.lod`` file contain one of the "wildcard sentinel" values.  
This is done directly in the ``mkload`` code itself, not afterwards.  The 
wildcards are listed in a file used by the ``mkload`` code and are also in 
the ``###``.rule files where ``###`` is the instrument name. The wildcard 
sentinel list is currently:

::

  ../../inc/config.h:#define  WILDCARD_LIST   
    "-1|any|-1.|abcdall|single_amp|fullframe_4amp|fullframe_2amp|
    custom_subarrays|chip1_sub_nocorners|chip2_sub_nocorners|quad_corner_subarrays"

If any one of those values is found in a keyword, and the keyword is NOT one of these:

::

  change_level|pedigree|observation_begin_date|observation_end_date|expansion_number|comment

then the load file is marked as needing expansion and the code will automatically put an entry into the
   ``.lod`` file for each expanded keyword value, as given in the appropriate ``###``.rule file.


**Table 13-3. Load File**

+------------------------+------------+
| Field                  |Format      |
+========================+============+
| Header                 |Header      |
+------------------------+------------+
| file\_name             |varchar(50) |
+------------------------+------------+
| instrument             |varchar(50) |
+------------------------+------------+
| reference\_file\_type  |varchar(50) |
+------------------------+------------+
| useafter\_date         |datetime    |
+------------------------+------------+
| comparison\_file\_name |varchar(50) |
+------------------------+------------+
| opus\_flag             |char(1)     |
+------------------------+------------+
| comment (file-level)   |text        |
+------------------------+------------+

**Table 13-3. Load File**

+--------------------------+------------+
| Field                    |Format      |
+==========================+============+
| Detail                   |Detail      |
+--------------------------+------------+
| level\_of\_change        |varchar(30) |
+--------------------------+------------+
| pedigree                 |varchar(50) |
+--------------------------+------------+
| observation\_begin\_date |datetime    |
+--------------------------+------------+
| observation\_end\_date   |datetime    |
+--------------------------+------------+
| observation\_mode        |multiple    |
+--------------------------+------------+
| comment (row-level)      |text        |
+--------------------------+------------+

The load file is used to generate the SQL commands that load the CDBS
relations and is then discarded. It may be regenerated from the data
base in order to edit the comments if further information becomes
available. This is done without extracting or in any way changing the
calibration files themselves.


13.3 Synphot Relations
~~~~~~~~~~~~~~~~~~~~~~

The relations to support pysynphot/Synphot are almost identical to the file and
row relations given above. Pysynphoti/Synphot is used by all instruments and is
treated like another instrument. There are ``synphot\_file`` and
``synphot\_row`` relations.

**Table 13-4. Synphot\_File Relation**

+-----------------------------+------------+
| Field                       |Format      |
+=============================+============+
| file\_name                  |varchar(50) |
+-----------------------------+------------+
| expansion\_number           |int         |
+-----------------------------+------------+
| delivery\_number            |int         |
+-----------------------------+------------+
| reference\_file\_type       |varchar(50) |
+-----------------------------+------------+
| useafter\_date              |datetime    |
+-----------------------------+------------+
| opus\_flag                  |char        |
+-----------------------------+------------+
| comment                     |text        |
+-----------------------------+------------+
| general\_availability\_date |datetime    |
+-----------------------------+------------+
| opus\_load\_date            |datetime    |
+-----------------------------+------------+
| archive\_date               |datetime    |
+-----------------------------+------------+
| reject\_flag                |char        |
+-----------------------------+------------+
| reject\_delivery\_number    |int         |
+-----------------------------+------------+
| reject\_by\_file\_name      |varchar(50) |
+-----------------------------+------------+
| reject\_by\_expansion       |number      |
+-----------------------------+------------+
| comparison\_file\_name      |int         |
+-----------------------------+------------+

**Table 13-5. Synphot\_Row Relation**

+------------------------------+------------+
| Field                        |Format      |
+==============================+============+
| file\_name                   |varchar(50) |
+------------------------------+------------+
| expansion\_number            |int         |
+------------------------------+------------+
| delivery\_number             |int         |
+------------------------------+------------+
| compname                     |varchar(50) |
+------------------------------+------------+
| equivalence\_class\_severe   |int         |
+------------------------------+------------+
| equivalence\_class\_moderate |int         |
+------------------------------+------------+
| pedigree                     |varchar(50) |
+------------------------------+------------+
| observation\_begin\_date     |datetime    |
+------------------------------+------------+
| observation\_end\_date       |datetime    |
+------------------------------+------------+
| comment                      |text        |
+------------------------------+------------+