CRIS/SIS: Level 1.1 Data Retrieval Libraries

Last Modified Friday, 09-Jan-1998 14:42:06 PST.

This is a description of Level 1.1 data retrieval from the Level 1.1 archives for both CRIS and SIS. It will cover both the c libraries and the corresponding routines in IDL. Differences between CRIS and SIS and c and IDL will be noted as needed.

Note on use in c:

The c getData (and utilities) libraries are locally available at
	/home/idunn1/rgr/prototype/lib/
and are called
	libCrisGetData.a
	libSisGetData.a
	libUtilities.a
and the associated header files
	crisGetData.h
	sisGetData.h
	utilities.h
are at
	/home/idunn1/rgr/prototype/src/include/
Also contained in the headers in this directory and its sub-directories are the definitions of the CRIS and SIS data structures.

The actual code is kept in

	/home/idunn1/rgr/prototype/src/lib/
For download and remote installation of these libraries, see the get data lib transfer and setup notes. Also, mostly for the purposes of porting the libraries, here is the documentation on the low level c routines used in the libraries.

Finally, here is a sample Makefile which is used to compile test code which is used in some of the code examples which follow.

Note on use in IDL:

The IDL routines will be available to you in IDL if you have your "+/home/idunn1/rgr/prototype/idl" as part of your IDL_PATH environment variable before starting your IDL session. The actual code resides at
	/home/idunn1/rgr/prototype/idl/lib/
Definitions for all the IDL structs for the CRIS/SIS data are in
	/home/idunn1/rgr/prototype/idl/include/

General Use Information:

First, all data structs are indexed by time tags in atomic seconds since the beginning of 1996 (which is also referred to as spacecraft time), and is here associated with the variable "secondsSince1996."

All data structs also have a time interval in which they are valid. Most of the data structs have a constant periododicity, such as an event cycle's 256 second period or a summary's 1 second period. A few data structs, such as browse parameter structs and command table structs do not have a constant periododicity, but instead have an explicit end time stamp associated with the data. In the descriptions here, "endSecond" will refer either to ((secondsSince1996 - 1 sec) + period) for those structs with a constant periododicity, or to the ending time stamp in non-periodic data structs.

A general rule we decided upon when returning data structs is that we always return an array of structs (even when you think you are only asking for an individual struct). The last struct in this array will always be a zeroed struct. It is easy to spot because it will be the only struct in the array which has a time stamp equal to 0.

There are 2 reasons for this. First, it is an easy way in IDL and c to deal with the case of no data being found. In this case, you will get an array of length 1, whose only element is the terminal zeroed structure. Second, there is the possibility due to quirks in the spacecraft clock and/or other anomolies that more than one struct will be tagged as valid during a particular interval which would usually just have a single valid struct. All such structs will be returned to the user who must decide what to do with them.

In order to make it easy to shift from archive to archive in cases where different tests may be being performed on archiving data, the user needs to set the environment variable L1_CRIS_DATA_BASE_DIRECTORY to access CRIS archives and L1_SIS_DATA_BASE_DIRECTORY to access SIS archives. For example,

setenv L1_SIS_DATA_BASE_DIRECTORY /home/idunn1/rgr/prototype/data/sis
c only: the argument "autoMemoryFreeFlag" in the routines below is required for handling the issue of memory management. When a large array is allocated in c, it should be deallocated when the array is no longer needed. If autoMemoryFreeFlag is set (meaning autoMemoryFreeFlag = 1), the routine will automatically deallocate the memory for the array it last passed back to the user before allocating memory for the new array it will now return. This means that anything pointing to the old array will no longer be valid. If the user wishes to keep data from a previous call available past a subsequent call to the same routine, they should pass autoMemoryFreeFlag = 0. The pointer returned to the user will not be tracked by the subroutine, and the user will assume the responsibility for freeing the allocated memory when they have finished with the data.

Also note that there are many utility routines available for dealing with times and dates in the c utilities library and the IDL utilities library. E.g., determing the start secondsSince1996 for a particular day will not be difficult.


THE ROUTINES:

Getting Individual Data Structs

For retrieving an individual (usually) structure where the input parameter secondsSince1996 is greater than or equal the time tag on the data struct, and is less than endSecond (as described above), use these routines:

c:

The c routines return a pointer to an array of structs of the appropriate type, the last of which has all fields zeroed. Sample code.
CRIS:
struct BrowseCris *getBrowseParamStruct_c(secondsSince1996, autoMemoryFreeFlag);
struct L1CrisCommandEcho *getCmdEchoStruct_c(secondsSince1996,
							autoMemoryFreeFlag);
struct L1CrisCommandTable *getCmdTableStruct_c(secondsSince1996,
							autoMemoryFreeFlag);
struct L1CrisDiagnosticEvent *getDiagnosticEventStruct_c(secondsSince1996,
							autoMemoryFreeFlag);
struct L1CrisEventCycle *getEventCycleStruct_c(secondsSince1996,
							autoMemoryFreeFlag);
struct L1CrisHighPriorityRate *getHiRateStruct_c(secondsSince1996,
							autoMemoryFreeFlag);
struct L1CrisHskp *getHskpStruct_c(secondsSince1996, autoMemoryFreeFlag);
struct L1CrisLowPriorityRate *getLoRateStruct_c(secondsSince1996,
							autoMemoryFreeFlag);
struct L1CrisSubset *getSubsetStruct_c(secondsSince1996, autoMemoryFreeFlag);
struct L1CrisSummary *getSummaryStruct_c(secondsSince1996, autoMemoryFreeFlag);
SIS:
struct BrowseSis *getBrowseParamStruct_s(secondsSince1996, autoMemoryFreeFlag);
struct L1SisCommandEcho *getCmdEchoStruct_s(secondsSince1996,
							autoMemoryFreeFlag);
struct L1SisCommandTable *getCmdTableStruct_s(secondsSince1996,
							autoMemoryFreeFlag);
struct L1SisDacOffset0 *getDacOffset0Struct_s(secondsSince1996,
							autoMemoryFreeFlag);
struct L1SisDacOffset1 *getDacOffset1Struct_s(secondsSince1996,
							autoMemoryFreeFlag);
struct L1SisDiagnosticEvent *getDiagnosticEventStruct_s(secondsSince1996,
							autoMemoryFreeFlag);
struct L1SisEventCycle *getEventCycleStruct_s(secondsSince1996,
							autoMemoryFreeFlag);
struct L1SisHighPriorityRate *getHiRateStruct_s(secondsSince1996,
							autoMemoryFreeFlag);
struct L1SisHskp *getHskpStruct_s(secondsSince1996, autoMemoryFreeFlag);
struct L1SisLowPriorityRate *getLoRateStruct_s(secondsSince1996,
							autoMemoryFreeFlag);
struct L1SisSubset *getSubsetStruct_s(secondsSince1996, autoMemoryFreeFlag);
struct L1SisSummary *getSummaryStruct_s(secondsSince1996, autoMemoryFreeFlag);

IDL:

The IDL routines return an array of structs of the appropriate type, the last of which has all fields zeroed. Sample code.
CRIS:
structs = get_browse_param_struct_c (secondsSince1996)
structs = get_cmd_echo_struct_c (secondsSince1996)
structs = get_cmd_table_struct_c (secondsSince1996)
structs = get_diagnostic_event_struct_c (secondsSince1996)
structs = get_event_cycle_c (secondsSince1996)
structs = get_hi_rate_struct_c (secondsSince1996)
structs = get_hskp_struct_c (secondsSince1996)
structs = get_lo_rate_struct_c (secondsSince1996)
structs = get_subset_struct_c (secondsSince1996)
structs = get_summary_struct_c (secondsSince1996)
SIS:
structs = get_browse_param_struct_s (secondsSince1996)
structs = get_cmd_echo_struct_s (secondsSince1996)
structs = get_cmd_table_struct_s (secondsSince1996)
structs = get_dac_offset0_struct_s (secondsSince1996)
structs = get_dac_offset1_struct_s (secondsSince1996)
structs = get_diagnostic_event_struct_s (secondsSince1996)
structs = get_event_cycle_s (secondsSince1996)
structs = get_hi_rate_struct_s (secondsSince1996)
structs = get_hskp_struct_s (secondsSince1996)
structs = get_lo_rate_struct_s (secondsSince1996)
structs = get_subset_struct_s (secondsSince1996)
structs = get_summary_struct_s (secondsSince1996)


Getting Events

Most likely, the end users will be more interested in the getEvents routines described below than the getEventCycle routines. The getEvents routines return an array of event structures uncompressed from the event cycle pointed to by secondsSince1996.

c:

The c routines return a pointer to an array of ce_crisEvt structs, the last of which has all fields zeroed. Sample code.
CRIS:
struct ce_crisEvt * getEvents_c (secondsSince1996);
SIS:
struct ce_sisEvt * getEvents_s (secondsSince1996);

IDL:

The IDL routines return an array of ce_crisEvt structs, the last of which has all fields zeroed. Sample code.
CRIS:
structs = get_events_c (secondsSince1996)
SIS:
structs = get_events_s (secondsSince1996)

Important note: Because uncompressed event structures will be too large if arbitrary start and stop times are used, this is the only direct way to get events. I.e., there are no getGroupEvents or getDaysEvents routines. The sample code in this section shows how to step through a day's worth of events using the getEvents routines.


Getting Groups of Data Structures Between Start and Stop Times

Usually, the user will wish to retrieve data over an extended time interval rather than a single data structure. The following routines return all data structs between the two times secondsSince1996Start and secondsSince1996Stop. These routines should be used with some care because they can be asked to return arrays of arbitrary size, which could in some cases, seriously hamper the performance of the machine the code is running on. Also, be aware that a data struct which is valid at secondsSince1996Stop will not be returned unless it is the same data struct which is valid at secondsSince1996Start. See note below for more details.

There is no getGroupEvents because the event arrays will be too large. To access events, the Getting Events section above.

c:

The c routines return a pointer to an array of structs of the appropriate type, the last of which has all fields zeroed. Sample code.
CRIS:
struct BrowseCris *getGroupBrowseParams_c(secondsSince1996Start,
				secondsSince1996Stop, autoMemoryFreeFlag);
struct L1CrisCommandEcho *getGroupCmdEchoes_c(secondsSince1996Start,
				secondsSince1996Stop, autoMemoryFreeFlag);
struct L1CrisCommandTable *getGroupCmdTables_c(secondsSince1996Start,
				secondsSince1996Stop, autoMemoryFreeFlag);
struct L1CrisDiagnosticEvent *getGroupDiagnosticEvents_c(secondsSince1996Start,
				secondsSince1996Stop, autoMemoryFreeFlag);
struct L1CrisHighPriorityRate *getGroupHiRates_c(secondsSince1996Start,
				secondsSince1996Stop, autoMemoryFreeFlag);
struct L1CrisHskp *getGroupHskps_c(secondsSince1996Start,
				secondsSince1996Stop, autoMemoryFreeFlag);
struct L1CrisLowPriorityRate *getGroupLoRates_c(secondsSince1996Start,
				secondsSince1996Stop, autoMemoryFreeFlag);
struct L1CrisSubset *getGroupSubsets_c(secondsSince1996Start,
				secondsSince1996Stop, autoMemoryFreeFlag);
struct L1CrisSummary *getGroupSummarys_c(secondsSince1996Start,
				secondsSince1996Stop, autoMemoryFreeFlag);
SIS:
struct BrowseSis *getGroupBrowseParams_s(secondsSince1996Start,
                                secondsSince1996Stop, autoMemoryFreeFlag);
struct L1SisCommandEcho *getGroupCmdEchoes_s(secondsSince1996Start,
                                secondsSince1996Stop, autoMemoryFreeFlag);
struct L1SisCommandTable *getGroupCmdTables_s(secondsSince1996Start,
                                secondsSince1996Stop, autoMemoryFreeFlag);
struct L1SisDacOffset0 *getGroupDacOffest0s_s(secondsSince1996Start,
                                secondsSince1996Stop, autoMemoryFreeFlag);
struct L1SisDacOffset1 *getGroupDacOffest1s_s(secondsSince1996Start,
                                secondsSince1996Stop, autoMemoryFreeFlag);
struct L1SisDiagnosticEvent *getGroupDiagnosticEvents_s(secondsSince1996Start,
                                secondsSince1996Stop, autoMemoryFreeFlag);
struct L1SisHighPriorityRate *getGroupHiRates_s(secondsSince1996Start,
                                secondsSince1996Stop, autoMemoryFreeFlag);
struct L1SisHskp *getGroupHskps_s(secondsSince1996Start,
                                secondsSince1996Stop, autoMemoryFreeFlag);
struct L1SisLowPriorityRate *getGroupLoRates_s(secondsSince1996Start,
                                secondsSince1996Stop, autoMemoryFreeFlag);
struct L1SisSubset *getGroupSubsets_s(secondsSince1996Start,
                                secondsSince1996Stop, autoMemoryFreeFlag);
struct L1SisSummary *getGroupSummarys_s(secondsSince1996Start,
                                secondsSince1996Stop, autoMemoryFreeFlag);

IDL:

The IDL routines return an array of structs of the appropriate type, the last of which has all fields zeroed. Sample code.
CRIS:
structs = get_group_browse_params_c(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_cmd_echoes_c(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_cmd_tables_c(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_diagnostic_events_c(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_hi_rates_c(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_hskps_c(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_lo_rates_c(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_subsets_c(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_summaries_c(secondsSince1996Start, secondsSince1996Stop)
SIS:
structs = get_group_browse_params_s(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_cmd_echoes_s(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_cmd_tables_s(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_dac_offset0s_s(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_dac_offset1s_s(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_diagnostic_events_s(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_hi_rates_s(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_hskps_s(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_lo_rates_s(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_subsets_s(secondsSince1996Start, secondsSince1996Stop)
structs = get_group_summaries_s(secondsSince1996Star t, secondsSince1996Stop)

IMPORTANT NOTE: (both c and IDL)
The structs array will always contain the data struct which was valid (see notes above for definition of "valid") at secondsSince1996Start if such a struct exists, no matter what value secondsSince1996Stop has. Then, if there are more structs time stamped before secondsSince1996Stop, they will be included as well. If the last struct is valid at secondsSince1996Stop, it WILL NOT be included in the structs array. The reason for this is to prevent seemingly contiguous time interval requests from getting overlapping data. In order to be sure you are getting all of the data as you process contiguous time intervals, a subsequent call to a get_group routine should have a secondsSince1996Start value EQUAL to the last call's secondsSince1996Stop value. But this does not remove all ambiguity, because if the time interval secondsSince1996Start to secondsSince1996Stop is smaller than the time a single data struct is valid, then such a call will return the struct valid at secondsSince1996Start, and the subsequent call to the next contiguous time interval could also return said data struct. Obviously, this is a silly case which can easily be avoided by being sure your time interval secondsSince1996Start to secondsSince1996Stop is longer than a single cycle.


Getting a Day's Worth of Data Structs

Finally, there are also routines to retrieve a full day's worth of data structures at a time. These routines will return an array of structures covering the entire day (terminated in a zeroed struct as usual.

Again, there is no get_days_events routine because of the size of the array which would be returned.

c:

The c routines return a pointer to an array of structs of the appropriate type, the last of which has all fields zeroed. Sample code.
CRIS:
struct BrowseCris *getDaysBrowseParams_c(int day1996, int autoFreeMemoryFlag);
struct L1CrisCommandEcho *getDaysCmdEchoes_c(int day1996, int autoFreeMemoryFlag);
struct L1CrisCommandTable *getDaysCmdTables_c(int day1996, int autoFreeMemoryFlag);
struct L1CrisDiagnosticEvent *getDaysDiagnosticEvents_c(int day1996, int autoFreeMemoryFlag);
struct L1CrisHighPriorityRate *getDaysHiRates_c(int day1996, int autoFreeMemoryFlag);
struct L1CrisHskp *getDaysHskps_c(int day1996, int autoFreeMemoryFlag);
struct L1CrisLowPriorityRate *getDaysLoRates_c(int day1996, int autoFreeMemoryFlag);
struct L1CrisSubset *getDaysSubsets_c(int day1996, int autoFreeMemoryFlag);
struct L1CrisSummary *getDaysSummarys_c(int day1996, int autoFreeMemoryFlag);
SIS:
struct BrowseSis *getDaysBrowseParams_s(int day1996, int autoFreeMemoryFlag);
struct L1SisCommandEcho *getDaysCmdEchoes_s(int day1996, int autoFreeMemoryFlag);
struct L1SisCommandTable *getDaysCmdTables_s(int day1996, int autoFreeMemoryFlag);
struct L1SisDacOffset0 *getDaysDacOffset0s_s(int day1996, int autoFreeMemoryFlag);
struct L1SisDacOffset1 *getDaysDacOffset1s_s(int day1996, int autoFreeMemoryFlag);
struct L1SisDiagnosticEvent *getDaysDiagnosticEvents_s(int day1996, int autoFreeMemoryFlag);
struct L1SisHighPriorityRate *getDaysHiRates_s(int day1996, int autoFreeMemoryFlag);
struct L1SisHskp *getDaysHskps_s(int day1996, int autoFreeMemoryFlag);
struct L1SisLowPriorityRate *getDaysLoRates_s(int day1996, int autoFreeMemoryFlag);
struct L1SisSubset *getDaysSubsets_s(int day1996, int autoFreeMemoryFlag);
struct L1SisSummary *getDaysSummarys_s(int day1996, int autoFreeMemoryFlag);

IDL:

The IDL routines return an array of structs of the appropriate type, the last of which has all fields zeroed. Sample code.
CRIS:
structs = get_days_browse_params_c(aceTimeStructForDay)
structs = get_days_cmd_echoes_c(aceTimeStructForDay)
structs = get_days_cmd_tables_c(aceTimeStructForDay)
structs = get_days_diagnostic_events_c(aceTimeStructForDay)
structs = get_days_hi_rates_c(aceTimeStructForDay)
structs = get_days_hskps_c(aceTimeStructForDay)
structs = get_days_lo_rates_c(aceTimeStructForDay)
structs = get_days_subsets_c(aceTimeStructForDay)
structs = get_days_summaries_c(aceTimeStructForDay)
SIS:
structs = get_days_browse_params_s(aceTimeStructForDay)
structs = get_days_cmd_echoes_s(aceTimeStructForDay)
structs = get_days_cmd_tables_s(aceTimeStructForDay)
structs = get_days_dac_offset0s_s(aceTimeStructForDay)
structs = get_days_dac_offset1s_s(aceTimeStructForDay)
structs = get_days_diagnostic_events_s(aceTimeStructForDay)
structs = get_days_hi_rates_s(aceTimeStructForDay)
structs = get_days_hskps_s(aceTimeStructForDay)
structs = get_days_lo_rates_s(aceTimeStructForDay)
structs = get_days_subsets_s(aceTimeStructForDay)
structs = get_days_summaries_s(aceTimeStructForDay)

The input aceTimeStructForDay is an {AceTime} struct defined as

data_struct = {AceTime,          $
                       year: 0L, $
                      month: 0L, $
                        day: 0L, $
                       hour: 0L, $
                        min: 0L, $
                        sec: 0L}
and can be set as is shown in the Sample code.
bruce