GSFIT Data Format: Difference between revisions

From EOVSA Wiki
Jump to navigation Jump to search
 
(5 intermediate revisions by the same user not shown)
Line 111: Line 111:


==Restoring a GSFIT log file as a pixel-based fit list==
==Restoring a GSFIT log file as a pixel-based fit list==
The '''''gsfit_log2list.pro''''' may be used to restore a fit list from a GSFIT *.log file in its raw format using the following calling sequence:
The '''''gsfit_log2list''''' function may be used to restore a fit list from a GSFIT ''*.log'' file in its raw format using the following calling sequence:
<pre>
<pre>
IDL> fitlist=gsfit_log2list('gsfitcp.log')
IDL> fitlist=gsfit_log2list('gsfitcp.log')
Line 127: Line 127:
FITLIST        STRUCT    = -> <Anonymous> Array[409600]
FITLIST        STRUCT    = -> <Anonymous> Array[409600]
</pre>
</pre>
GSFITCP and GSFIVIEW applications use internally the '''''gsfit_log2list.pro''''' to import a previously saved fit list for the various purposes. However, one may use tis function to restore and concatenate two or more fit lists stored in separate files.
GSFITCP and GSFIVIEW applications use internally the '''''gsfit_log2list.pro''''' to import a previously saved fit list, to add additional fit solutions to it in another session, or to convert the list to a parameter map structure for visualization and further analysis. However, one may use this function from the IDL console, or programmatically, to restore and concatenate two or more fit lists stored in separate GSFIT ''*.log' files.
 
==Restoring a GSFIT log file as a parameter maps structure==
==Restoring a GSFIT log file as a parameter maps structure==
The '''''gsfit_log2map''''' function may be used to restore a fit list from a GSFIT *.log file and convert it to a parameter maps structure, using the following calling sequence:
<pre>
<pre>
IDL> maps=gsfit_log2map(file)
IDL> maps=gsfit_log2map(file)
Line 141: Line 142:
% RESTORE: Restored variable: XDR.
% RESTORE: Restored variable: XDR.
% GSFIT_LOG2LIST: Restoring 409599 additional records
% GSFIT_LOG2LIST: Restoring 409599 additional records
</pre>
</pre>
However, the '''''gsfit_log2map''''' function, instead of a file path, may also accept as argument an already restore pixel-based fit list solution  restored by the '''''gsfit_log2fit''''' function.

Latest revision as of 23:37, 11 May 2019

This section describes the input and output data formats used by the GSFIT, GSFITCP, and GSFITVIEW applications, which are part of the GSFIT SSW IDL package developed for fitting the spatially-resolved solar flare radio spectra obtained with the EOVSA instrument. A tutorial about reduction of EOVSA data may be found here.

GSFIT Input Data Format

GSFIT expects input files to contain a multidimensional array of traditional SSW IDL map structures that must have a least two first mandatory dimensions, the number of frequencies and number of time frames, and an optional third dimension, the number of polarizations, if more than one are provided. The STOKES I demo EOVSA map cube shipped with the GSFIT package is described below.

IDL> help, EOMAPS
EOMAPS          STRUCT    = -> <Anonymous> Array[30, 10]
IDL> help,eomaps,/str
** Structure <214d9c00>, 26 tags, length=91472, data length=91449, refs=1:
   DATA            FLOAT     Array[151, 151]
   XC              DOUBLE           249.22902
   YC              DOUBLE          -224.62914
   DX              DOUBLE           2.0000000
   DY              DOUBLE           2.0000000
   TIME            STRING    ' 4-Sep-2017 20:46:36.500'
   ID              STRING    'EOVSA  3.42 GHz'
   DUR             FLOAT           4.00000
   XUNITS          STRING    'arcsec'
   YUNITS          STRING    'arcsec'
   ROLL_ANGLE      DOUBLE          0.00000000
   ROLL_CENTER     DOUBLE    Array[2]
   SOHO            BYTE         0
   L0              DOUBLE          0.00000000
   B0              DOUBLE           7.2364274
   RSUN            DOUBLE           951.09152
   RMS             FLOAT      3.17954e+006
   SNR             FLOAT           97.6565
   SCALEFAC        FLOAT           1.18274
   RMSXRAN         FLOAT     Array[2]
   RMSYRAN         FLOAT     Array[2]
   DATAUNITS       STRING    'K'
   RMSUNITS        STRING    'K'
   FREQ            FLOAT           3.42000
   FREQUNIT        STRING    'GHz'
   STOKES          STRING    'XX'

NOTE: The name of the variable stored in a data file may be arbitrarily, since GSFIT would accept as a valid input data the first restored array of map structure that, in addition to a standard SSW IDL map structure, has at least the following required tags: FREQ, FREQUNIT, STOKES, RMS

GSFIT Parameter Maps Structure

This is an IDL structure that stores a final product of the GSFIT package. This output data structure is described below.

IDL> help, maps
** Structure <1e0e3430>, 28 tags, length=189883336, data length=189666533, refs=1:
   FITMAPS         STRUCT    -> <Anonymous> Array[30, 100]       ;[nfreq x ntimes] GSFIT solution maps
   DATAMAPS        STRUCT    -> <Anonymous> Array[30, 100]       ;[nfreq x ntimes] input data maps use to compute the GSFIT solutions
   ERRMAPS         STRUCT    -> <Anonymous> Array[30, 100]       ;[nfreq x ntimes] input error maps used to weight the input data 
   N_NTH           STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of estimated nonthermal electron densities
   ERRN_NTH        STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of 1-sigma fit errors of the estimated nonthermal electron densities
   B               STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of estimated absolute magnetic field strength
   ERRB            STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of 1-sigma fit errors of the estimated 
   THETA           STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of estimated LOS angle of the magnetic field vector 
   ERRTHETA        STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of 1-sigma fit errors of the estimated LOS angle of the magnetic field vector 
   N_TH            STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of estimated thermal electron densities
   ERRN_TH         STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of 1-sigma fit errors of the estimated thermal electron densities
   DELTA           STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of estimated distribution over energy power law slope
   ERRDELTA        STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of 1-sigma fit errors of the estimated distribution over energy power law slope
   E_MAX           STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of estimated maximum energy cutoff of nonthermal electrons
   ERRE_MAX        STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of 1-sigma fit errors of the estimated maximum energy cutoff of nonthermal electrons
   T_E             STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of estimated plasma temperature
   ERRT_E          STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of 1-sigma fit errors of the estimated plasma temperature
   CHISQR          STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of fit CHISQR
   RESIDUAL        STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of fit Residuals 
   PEAKFLUX        STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of estimated peak flux of the fit solution
   ERRPEAKFLUX     STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of 1-sigma fit errors of the estimated peak flux of the fit solution 
   PEAKFREQ        STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of estimated peak frequency of the fit solution
   ERRPEAKFREQ     STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of 1-sigma fit errors of the estimated peak frequency of the fit solution
   WB              STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps estimated magnetic field energy
   ERRWB           STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of 1-sigma fit errors of the estimated magnetic field energy
   WNTH            STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps estimated nonthermal electron energy
   ERRWNTH         STRUCT    -> <Anonymous> Array[100]           ;[ntimes] maps of 1-sigma fit errors of the estimated nonthermal electron energy 
   HEADER          STRUCT    -> <Anonymous> Array[1]             ;structure containing information about the GSFIT solutions from which these maps were created

GSFIT FitList Format

This is the internal raw data format used by GSFIT and GSFITCP to store in memory or log on the disk the pixel-based fit solutions in the sequential order they ae produced at run-time. Each individual pixel-based fit solution structure described below contains the position and temporal information that, together to a header fit task structure, are used to build the fit parameter maps structure described above..

IDL> help,fitlist,/str
** Structure <21fed180>, 24 tags, length=504, data length=504, refs=2:
   X               LONG                67     ;pixel X index
   Y               LONG                42     ;pixel Y index
   T               LONG                 0     ;pixel time frame index
   FMIN            INT              0
   FMAX            INT             23
   DATA            FLOAT     Array[30]
   ERRDATA         FLOAT     Array[30]
   SPECFIT         FLOAT     Array[30]
   N_NTH           DOUBLE          0.00000000
   ERRN_NTH        DOUBLE          0.00000000
   B               DOUBLE          0.00000000
   ERRB            DOUBLE          0.00000000
   THETA           DOUBLE          0.00000000
   ERRTHETA        DOUBLE          0.00000000
   N_TH            DOUBLE          0.00000000
   ERRN_TH         DOUBLE          0.00000000
   DELTA           DOUBLE          0.00000000
   ERRDELTA        DOUBLE          0.00000000
   E_MAX           DOUBLE          0.00000000
   ERRE_MAX        DOUBLE          0.00000000
   T_E             DOUBLE          0.00000000
   ERRT_E          DOUBLE          0.00000000
   CHISQR          DOUBLE          0.00000000
   RESIDUAL        DOUBLE          0.00000000

GSFIT Log File Format

GSFIT and GSFITCP applications save the list of spectral fit solutions in binary "*.log" files that are formatted as a header section followed by identical records logged in the order in which they are computed at runtime.

Although such log files may appear to be traditional IDL *.sav files, attempting to restore them via an IDL command line using the native RESTORE procedure would only recover the first record logged in the file. To recover the full information stored in such files, one may use one of the two support routines that are part of the GSFIT package,

Restoring a GSFIT log file as a pixel-based fit list

The gsfit_log2list function may be used to restore a fit list from a GSFIT *.log file in its raw format using the following calling sequence:

IDL> fitlist=gsfit_log2list('gsfitcp.log')
% Compiled module: MULTI_RESTORE.
% RESTORE: Portable (XDR) SAVE/RESTORE file.
% RESTORE: Save file written by gnita@baozi.hpcnet.campus.njit.edu, Fri Mar  1 18:15:09 2019.
% RESTORE: IDL version 8.5 (linux, x86_64).
% RESTORE: Restored variable: HEADER.
% RESTORE: Restored variable: TEMPLATE.
% RESTORE: Restored variable: HINT.
% RESTORE: Restored variable: KEY.
% RESTORE: Restored variable: XDR.
% GSFIT_LOG2LIST: Restoring 409599 additional records
IDL> help, fitlist
FITLIST         STRUCT    = -> <Anonymous> Array[409600]

GSFITCP and GSFIVIEW applications use internally the gsfit_log2list.pro to import a previously saved fit list, to add additional fit solutions to it in another session, or to convert the list to a parameter map structure for visualization and further analysis. However, one may use this function from the IDL console, or programmatically, to restore and concatenate two or more fit lists stored in separate GSFIT *.log' files.

Restoring a GSFIT log file as a parameter maps structure

The gsfit_log2map function may be used to restore a fit list from a GSFIT *.log file and convert it to a parameter maps structure, using the following calling sequence:

IDL> maps=gsfit_log2map(file)
% RESTORE: Portable (XDR) SAVE/RESTORE file.
% RESTORE: Save file written by gnita@baozi.hpcnet.campus.njit.edu, Fri Mar  1 18:15:09 2019.
% RESTORE: IDL version 8.5 (linux, x86_64).
% RESTORE: Restored variable: HEADER.
% RESTORE: Restored variable: TEMPLATE.
% RESTORE: Restored variable: HINT.
% RESTORE: Restored variable: KEY.
% RESTORE: Restored variable: XDR.
% GSFIT_LOG2LIST: Restoring 409599 additional records

However, the gsfit_log2map function, instead of a file path, may also accept as argument an already restore pixel-based fit list solution restored by the gsfit_log2fit function.