@@ -33,17 +33,19 @@ Attributes are entities that are specified during the planning of a holography o
Below is the list of defined attributes in this version of HDS:
-**Version**: The HDS is versioned and begins at version 1.0. The versions are incremental. *Unit: [n.a.]*
-**RCU list**: An array that contains the list of RCUs that are planned to be used during the holography observation. *Unit: [n.a.]*
-**mode**: The RCU mode that will be set during the holography observation. *Unit: [n.a.]*
-**SAS id**: An array that contains the SAS ids that were created for all observation runs of the holography observation. *Unit: [n.a.]*
-**Target station name**: Name of the LOFAR target station. *Unit: [n.a.]*
-**Target station position**: Position in cartesian coordinates. *Unit: [m, m, m]*
-**Source name**: Name of the source observed during the holography observation. *Unit: [n.a.]*
-**Source position**: Celestial coordinates (RA, DEC) of the source. *Unit: [rad, rad]*
-**Observation time**: Planned start and end time-stamps of the entire holography observation in MJD. *Unit: [s]*
-**Rotation matrix**: Rotation matrix that allows to convert (RA, DEC) coordinates to (l, m) coordinates and vice versa. *Unit: [n.a.]*
-**Antenna field position**: Cartesian coordinates with reference to IRTF of the centre of each antenna field that will be used in the holography observation. *Unit: [m, m, m]*
-**HDS_version**: The HDS is versioned and begins at version 1.0. The versions are incremental. *Unit: [n.a.]*
-**RCU_list**: An array that contains the list of RCUs that are planned to be used during the holography observation. *Unit: [n.a.]*
-**Mode**: The RCU mode that will be set during the holography observation. *Unit: [n.a.]*
-**SAS_ids**: An array that contains the SAS ids that were created for all observation runs of the holography observation. *Unit: [n.a.]*
-**Target_station_name**: Name of the LOFAR target station. *Unit: [n.a.]*
-**Target_station_position**: Position in cartesian coordinates. *Unit: [m, m, m]*
-**Source_name**: Name of the source observed during the holography observation. *Unit: [n.a.]*
-**Source_position**: Celestial coordinates (RA, DEC) of the source. *Unit: [rad, rad]*
-**Observation_time**: Planned start and end time-stamps of the entire holography observation in MJD. *Unit: [s]*
-**Rotation_matrix**: Rotation matrix that allows to convert (RA, DEC) coordinates to (l, m) coordinates and vice versa. *Unit: [n.a.]*
-**Antenna_field_position**: Cartesian coordinates with reference to IRTF of the centre of each antenna field that will be used in the holography observation. *Unit: [m, m, m]*
Additionally if the HDS has been used to specify the holography observation these fields will also be present:
-**Specified reference station**: List of reference stations that were planned for the holography observation. *Unit: [n.a.]*
-**Specified frequency**: List of frequencies that were planned to be used for the holography observation. *Unit: [Hz]*
-**Specified RA DEC**: List of the planned target station's beamlet pointings for the holography observation. *Unit: [rad, rad]*
...
...
@@ -54,18 +56,30 @@ Data tables serve as the work horses for the cross-correlations in an HDS. They
Below is the list of defined data tables in this version of HDS:
-**Referencestation**: The reference station data table contains the list of all reference stations that were actually used in the holography observation. This list can contain less stations than the **Specified reference station** list. For a given reference station the index of it in this list serves as one of the three indices of **data**. *Unit: [n.a.]
-**Reference_station**: The reference station data table contains the list of all reference stations that were actually used in the holography observation. This list can contain less stations than the **Specified reference station** list. For a given reference station the index of it in this list serves as one of the three indices of **data**. *Unit: [n.a.]
-**frequency**: List of frequencies that were actually used for the holography observation. This list can contain less elements than the **Specified frequency** list. For a given frequency the index of it in this list serves as one of the three indices of **data**. *Unit [Hz]*
-**RA DEC:** List of (rightascension, declination) target station beam pointings for a given (frequency, beamlet) tuple. This list allows it to translate between (frequency, beamlet) and (RA, DEC). *Unit: [rad, rad]*
-**data**: This data table contains the cross-correlations *XX*, *XY*, *YX* and *YY* for a given frequency, beamlet# and reference station. *Unit: [(n.a, n.a.), (n.a, n.a), (n.a, n.a), (n.a, n.a)]*\
It also contains the time-stamp as MJD when the data was measured. *Unit: [s]*
Additionally, a table containing the cross correlations and the right ascension and declination of each beam are stored in a group structure that will be described in the next sub section.
### Groups in the HDS
The right ascension, declination and the crosscorrelation are stored in the HDS in hierarchical groups structure. This facilitate the access to a specific set of sample given the reference station name,
the frequency of the sample and the beamlet number.
The notation ```%(station_name)``` is used to refer to a string containing a station name. The same notation is also used to refer to the string representation of the frequency in Hz (```%(frequency)```) and the beamlet number (```%(beamlet)```).
And finally it contains the values *l* and *m* which were calculated from **RA DEC** and **Rotation matrix**. *Unit: [m, m]*
The structure of the groups is the following:
```
/CROSSCORRELATION
/%(station_name)
/%(frequency)
%(beamlet) This data table contains the cross-correlations *XX*, *XY*, *YX*, *YY*, l, m, the time stamp of the sample, and if or not the sample has been flagged for a given frequency, beamlet# and reference station. *Unit: [(n.a, n.a.), (n.a, n.a), (n.a, n.a), (n.a, n.a), (n.a), (n.a), (s), (True|False)]*\
/RA_DEC
/%(frequency)
%(beamlet) This datatable contains the (rightascension, declination, epoch) of the target station beam pointings. This list allows it to translate between (frequency, beamlet) and (RA, DEC). *Unit: [rad, rad]*
```
## Cross-correlation data in HDS
Values in **data** are stored in a pseudo-structure. A simple representation of a single element in **data** can be given in Python language:
Values in the cross-correlation tables are stored in a pseudo-structure. A simple representation of a single element in cross-correlation table can be given in Python language:
```
data_element = dict(
"XX" = c_XX,
...
...
@@ -74,22 +88,13 @@ Values in **data** are stored in a pseudo-structure. A simple representation of
"YY" = c_YY,
"t" = t_MJD,
"l" = l,
"m" = m)
```
Here the indices "XX", "XY", "YX", "YY" allow accessing the cross-correlation values, "t" allows to access the MJD of the observation and "l", "m" allow to access the coordinates l and m.
Each element in **data** can be accessed through the three indices *(Reference station index, frequency index, beamlet#)*. The indices can be determined as follows, again this is Python code:
```
# Works only in Python! Other languages have to have a *convenience function* implemented!
frequency_index = hds.frequency.index(frequency)
# Works only in Python! Other languages have to have a *convenience function* implemented!
Here the indices "XX", "XY", "YX", "YY" allow accessing the cross-correlation values, "t" allows to access the MJD of the observation and "l", "m" allow to access the coordinates l and m and finally flag to access to the flag value.
The way how data is accessed may look awkward and that is in fact true. But at the moment the limited capability of the HDF5 file format forces us to jump through this hoop. Later versions of HDS will have this limitation lifted and will very likely not use the HDF5 file format any more.
As described before each cross-correlation table can be accessed by address. Meaning, that the observation sample at a given frequency `%(frequency)`, for a reference station `%(station_name)` and a beamlet `%(beamlet)` can be directly accessed
as `/%(station_name)/%(frequency)/%(beamlet)` (Ex. `/RS409C/114843750.0/0`).
## Layout of the HDS in HDF5 files
...
...
@@ -110,9 +115,9 @@ The way how data is accessed may look awkward and that is in fact true. But at
/Target_station_name (string), name of the target station
/Target_station_position (double[3]), position of the centre of the station in cartesian coordinates
/Specified_reference_station (1-d array, string), entries are reference station names
/Specified_frequency (1-d array, double), entries are the frequencies of all observations that were specified
/Specified_RA_DEC (2-d array, compound[double, double, string[10]]), indices are frequency index, beamlet#
/Specified_reference_station (1-d array, string), entries are reference station names [optional]
/Specified_frequency (1-d array, double), entries are the frequencies of all observations that were specified [optional]
/Specified_RA_DEC (2-d array, compound[double, double, string[10]]), indices are frequency index, beamlet# [optional]
# Groups
/Reference_station (1-d array, string[8]), entries are reference station names
