EOVSA Data Analysis Tutorial 2022 - Revision history

Sshaik: /* 3. Software requirements and installation */

2022-07-07T21:56:24Z

3. Software requirements and installation

← Older revision		Revision as of 21:56, 7 July 2022
Line 37:		Line 37:
	=3. Software requirements and installation=		=3. Software requirements and installation=
	EOVSA data processing and analysis are developed over two packages:		EOVSA data processing and analysis are developed over two packages:
	* '''[https://github.com/suncasa/suncasa SunCASA]''' A Python wrapper around [https://casa.nrao.edu/ CASA (the Common Astronomy Software Applications package)] for synthesis imaging and visualizing solar spectral imaging data. CASA is one of the leading software tools for "supporting the data post-processing needs of the next generation of radio astronomical telescopes such as ALMA and VLA", an international effort led by the [https://public.nrao.edu/ National Radio Astronomy Observatory]. The current version of CASA uses Python (3.6, 3.10*) interface. More information about CASA can be found on [https://casa.nrao.edu/ NRAO's CASA website ]. Note, CASA is available ONLY on UNIX-BASED PLATFORMS (and therefore, so is SunCASA).		* '''[https://github.com/suncasa/suncasa SunCASA]''' A Python wrapper around [https://casa.nrao.edu/ CASA (the Common Astronomy Software Applications package)] for synthesis imaging and visualizing solar spectral imaging data. CASA is one of the leading software tools for "supporting the data post-processing needs of the next generation of radio astronomical telescopes such as ALMA and VLA", an international effort led by the [https://public.nrao.edu/ National Radio Astronomy Observatory]. The current version of CASA uses Python (3.6) interface. More information about CASA can be found on [https://casa.nrao.edu/ NRAO's CASA website ]. Note, CASA is available ONLY on UNIX-BASED PLATFORMS (and therefore, so is SunCASA).

	* '''[https://github.com/Gelu-Nita/GSFIT GSFIT]''' A IDL-widget (GUI)-based spectral fitting package called GSFIT, which provides a user-friendly display of EOVSA image cubes and an interface to fast-fitting codes (via platform-dependent shared-object libraries).		* '''[https://github.com/Gelu-Nita/GSFIT GSFIT]''' A IDL-widget (GUI)-based spectral fitting package called GSFIT, which provides a user-friendly display of EOVSA image cubes and an interface to fast-fitting codes (via platform-dependent shared-object libraries).

Sshaik: /* Other useful links */

2022-07-07T21:56:02Z

Sshaik: /* 12. Installation of SunCASA (optional) */

2022-07-07T21:18:23Z

12. Installation of SunCASA (optional)

← Older revision		Revision as of 21:18, 7 July 2022
Line 193:		Line 193:
	[[file:GSFIT_20220507.PNG\|thumb\|right\|400px\|Figure 9: Image and the corresponding spectrum on GSFIT GUI.]]		[[file:GSFIT_20220507.PNG\|thumb\|right\|400px\|Figure 9: Image and the corresponding spectrum on GSFIT GUI.]]

	=12. Installation of SunCASA (optional)=		=11. Installation of SunCASA (optional)=
	If you want to install SunCASA on your local machine follow this section. If not, run SunCASA directly on the Colab Workspace.		If you want to install SunCASA on your local machine follow this section. If not, run SunCASA directly on the Colab Workspace.

Sshaik: /* 11. Spectral Fitting with GSFIT (optional) */

2022-07-07T21:18:15Z

11. Spectral Fitting with GSFIT (optional)

← Older revision		Revision as of 21:18, 7 July 2022
Line 188:		Line 188:
	This section is for interested users who wish to generate FITS files with full control of all parameters being used for synthesis imaging. Please follow the [https://colab.research.google.com/drive/19NQb6Emb9HvKX4QHq9ZYCP3RM6nT7sDL#scrollTo=cLdDVptBGG-X EOVSA Workspace] for an example on downloading image fits files.		This section is for interested users who wish to generate FITS files with full control of all parameters being used for synthesis imaging. Please follow the [https://colab.research.google.com/drive/19NQb6Emb9HvKX4QHq9ZYCP3RM6nT7sDL#scrollTo=cLdDVptBGG-X EOVSA Workspace] for an example on downloading image fits files.

	=11. Spectral Fitting with GSFIT (optional)=		=10. Spectral Fitting with GSFIT (optional)=
	The .fits images obtained in the [https://colab.research.google.com/drive/19NQb6Emb9HvKX4QHq9ZYCP3RM6nT7sDL#scrollTo=cLdDVptBGG-X EOVSA Workspace] can be read as an input to the GSFIT package. The screenshot of that is shown in Figure 9. Refer [http://ovsa.njit.edu/wiki/index.php/GSFIT_Help this page] for a detailed description of further GSFIT analysis.		The .fits images obtained in the [https://colab.research.google.com/drive/19NQb6Emb9HvKX4QHq9ZYCP3RM6nT7sDL#scrollTo=cLdDVptBGG-X EOVSA Workspace] can be read as an input to the GSFIT package. The screenshot of that is shown in Figure 9. Refer [http://ovsa.njit.edu/wiki/index.php/GSFIT_Help this page] for a detailed description of further GSFIT analysis.

Sshaik: /* 10. Plotting Images in SSWIDL */

2022-07-07T21:17:59Z

10. Plotting Images in SSWIDL

@@ Line 187: / Line 187: @@
 =9. Downloading fits files to your local system=
 This section is for interested users who wish to generate FITS files with full control of all parameters being used for synthesis imaging. Please follow the [https://colab.research.google.com/drive/19NQb6Emb9HvKX4QHq9ZYCP3RM6nT7sDL#scrollTo=cLdDVptBGG-X EOVSA Workspace] for an example on downloading image fits files.
 =11. Spectral Fitting with GSFIT (optional)=

Sshaik: /* 9. Downloading fits files to your local system */

2022-07-07T21:17:32Z

9. Downloading fits files to your local system

← Older revision		Revision as of 21:17, 7 July 2022
Line 186:		Line 186:

	=9. Downloading fits files to your local system=		=9. Downloading fits files to your local system=
	This section is for interested users who wish to generate FITS files with full control on all parameters being used for synthesis imaging. Please follow the [https://colab.research.google.com/drive/19NQb6Emb9HvKX4QHq9ZYCP3RM6nT7sDL#scrollTo=cLdDVptBGG-X EOVSA Workspace] for an example on downloading image fits files. ~~Please~~		This section is for interested users who wish to generate FITS files with full control of all parameters being used for synthesis imaging. Please follow the [https://colab.research.google.com/drive/19NQb6Emb9HvKX4QHq9ZYCP3RM6nT7sDL#scrollTo=cLdDVptBGG-X EOVSA Workspace] for an example on downloading image fits files.

	=10. Plotting Images in SSWIDL=		=10. Plotting Images in SSWIDL=

Sshaik: /* 1. Introduction to EOVSA and Datasets */

2022-07-07T20:10:10Z

1. Introduction to EOVSA and Datasets

← Older revision		Revision as of 20:10, 7 July 2022
Line 1:		Line 1:
	=1. Introduction to EOVSA and Datasets=		=1. Introduction to EOVSA and Datasets=
	[[file:Eovsa1.png\|center\|500px\|EOVSA]]		[[file:Eovsa1.png\|center\|500px\|EOVSA]]
	EOVSA (Expanded Owens Valley Solar Array) is a solar-dedicated radio interferometer operated by the New Jersey Institute of Technology. EOVSA observes the full disk of the Sun at all times when the Sun is >10 degrees above the local horizon, which is season dependent and ranges from 7-12 hours duration centered on 20 UT. Like any radio interferometer, the fundamental measurement for imaging is the correlated amplitude and phase between each pair of antennas, which is called a “complex visibility.” EOVSA’s 13 antennas form 78 such visibilities at any frequency and instant of time, i.e. 78 measurements of the spatial Fourier transform of the solar brightness distribution. EOVSA records these visibilities at *451 science frequency channels each second, in four polarization products (XX, YY, XY, YX), as well as additional total flux measurements from each individual antenna. These data are then processed through a pipeline processing system whose data flow is shown in the block diagram (Figure 1). One of the outputs of the pipeline is a visibility database in a widely used open-standard format called a CASA measurement set (or “ms”; CASA is the Common Astronomy Software Applications package used by many modern interferometer arrays).		EOVSA (Expanded Owens Valley Solar Array) is a solar-dedicated radio interferometer operated by the New Jersey Institute of Technology. EOVSA observes the full disk of the Sun at all times when the Sun is >10 degrees above the local horizon, which is season dependent and ranges from 7-12 hours duration centered on 20 UT. Like any radio interferometer, the fundamental measurement for imaging is the correlated amplitude and phase between each pair of antennas, which is called a “complex visibility.” EOVSA’s 13 antennas form 78 such visibilities at any frequency and instant of time, i.e. 78 measurements of the spatial Fourier transform of the solar brightness distribution. EOVSA records these visibilities at 451 science frequency channels each second, in four polarization products (XX, YY, XY, YX), as well as additional total flux measurements from each individual antenna. These data are then processed through a pipeline processing system whose data flow is shown in the block diagram (Figure 1). One of the outputs of the pipeline is a visibility database in a widely used open-standard format called a CASA measurement set (or “ms”; CASA is the Common Astronomy Software Applications package used by many modern interferometer arrays).

	EOVSA delivers the radio interferometry data on the following three levels:		EOVSA delivers the radio interferometry data on the following three levels:

Dgary: /* Level 1 (calibrated visibility) Data */

2022-07-07T14:27:29Z

Level 1 (calibrated visibility) Data

← Older revision		Revision as of 14:27, 7 July 2022
Line 23:		Line 23:

	===Level 1 (calibrated visibility) Data===		===Level 1 (calibrated visibility) Data===
	The images you see in the browser are generated by an automated pipeline that also produces calibrated CASA ms datasets, but at 1-min integration. If a 1-min cadence is sufficient for your purposes, you may wish to download the calibrated data and perform your own imaging. The 1-min cadence calibrated data are at [http://www.ovsa.njit.edu/fits/UDBms_slfcaled/ this link].		The images you see in the browser are generated by an automated pipeline that also produces calibrated CASA ms datasets, but at 1-min integration. If a 1-min cadence is sufficient for your purposes, you may wish to download the calibrated data and perform your own imaging. The 1-min cadence calibrated data are at [http://www.ovsa.njit.edu/fits/UDBms_slfcaled/ this link]. You should be aware that a model disk has been subtracted from the visibilities in these files, which is what you want if you are interested in imaging active regions or flares. It is possible to add the model back, but this is not available (yet) in standard software, so please contact someone in the NJIT solar radio group for help with that.

	====Other useful links====		===Other useful links===

Dgary: /* Level 0 (raw visibility data) */

2022-07-07T14:23:33Z

Level 0 (raw visibility data)

← Older revision		Revision as of 14:23, 7 July 2022
Line 19:		Line 19:
	EOVSA data can also be browsed through [http://sprg.ssl.berkeley.edu/~tohban/browser/ RHESSI Browser]. Check the "EOVSA Radio Data" box on the data selection area (top-left corner). Then select year/month/date to view the overall EOVSA dynamic spectrum. Note if a time is selected at early UTC hours (e.g., 0-3 UT), it will show the EOVSA dynamic spectrum from the previous day. Also, note that EOVSA data were not commissioned for spectroscopic imaging prior to April 2017.		EOVSA data can also be browsed through [http://sprg.ssl.berkeley.edu/~tohban/browser/ RHESSI Browser]. Check the "EOVSA Radio Data" box on the data selection area (top-left corner). Then select year/month/date to view the overall EOVSA dynamic spectrum. Note if a time is selected at early UTC hours (e.g., 0-3 UT), it will show the EOVSA dynamic spectrum from the previous day. Also, note that EOVSA data were not commissioned for spectroscopic imaging prior to April 2017.

	===Level 0 (raw visibility ~~data~~)===		===Level 0 (raw visibility) Data===
	Once you identify the flare time, you can find the full-resolution (1-s cadence) uncalibrated visibility files (in Miriad format) at [http://www.ovsa.njit.edu/fits/IDB/ this link]. Each data file is usually 10 minutes in duration. The name convention is YYYYMMDD (folder name) /IDBYYYYMMDDHHMMSS (file name), where the time in the file name indicates the start time of the visibility data. However, to be useful for imaging, these files must be first calibrated and then converted to a CASA measurement set. At present, this can only be done with software at the OVRO site although we are working on software to allow this processing to be done elsewhere. For now, please request this processing to be done for you by emailing Dale Gary, Bin Chen, Sijie Yu, or others you may know in the solar radio group.		Once you identify the flare time, you can find the full-resolution (1-s cadence) uncalibrated visibility files (in Miriad format) at [http://www.ovsa.njit.edu/fits/IDB/ this link]. Each data file is usually 10 minutes in duration. The name convention is YYYYMMDD (folder name) /IDBYYYYMMDDHHMMSS (file name), where the time in the file name indicates the start time of the visibility data. However, to be useful for imaging, these files must be first calibrated and then converted to a CASA measurement set. At present, this can only be done with software at the OVRO site although we are working on software to allow this processing to be done elsewhere. For now, please request this processing to be done for you by emailing Dale Gary, Bin Chen, Sijie Yu, or others you may know in the solar radio group.

			===Level 1 (calibrated visibility) Data===
			The images you see in the browser are generated by an automated pipeline that also produces calibrated CASA ms datasets, but at 1-min integration. If a 1-min cadence is sufficient for your purposes, you may wish to download the calibrated data and perform your own imaging. The 1-min cadence calibrated data are at [http://www.ovsa.njit.edu/fits/UDBms_slfcaled/ this link].

	====Other useful links====		====Other useful links====

Dgary: /* Level 0 (raw visibility data) */

2022-07-07T14:18:33Z

Level 0 (raw visibility data)

← Older revision		Revision as of 14:18, 7 July 2022
Line 19:		Line 19:
	EOVSA data can also be browsed through [http://sprg.ssl.berkeley.edu/~tohban/browser/ RHESSI Browser]. Check the "EOVSA Radio Data" box on the data selection area (top-left corner). Then select year/month/date to view the overall EOVSA dynamic spectrum. Note if a time is selected at early UTC hours (e.g., 0-3 UT), it will show the EOVSA dynamic spectrum from the previous day. Also, note that EOVSA data were not commissioned for spectroscopic imaging prior to April 2017.		EOVSA data can also be browsed through [http://sprg.ssl.berkeley.edu/~tohban/browser/ RHESSI Browser]. Check the "EOVSA Radio Data" box on the data selection area (top-left corner). Then select year/month/date to view the overall EOVSA dynamic spectrum. Note if a time is selected at early UTC hours (e.g., 0-3 UT), it will show the EOVSA dynamic spectrum from the previous day. Also, note that EOVSA data were not commissioned for spectroscopic imaging prior to April 2017.

	====Level 0 (raw visibility data)====		===Level 0 (raw visibility data)===
	Once you identify the flare time, you can find the full-resolution (1-s cadence) uncalibrated visibility files (in Miriad format) at [http://www.ovsa.njit.edu/fits/IDB/ this link]. Each data file is usually 10 minutes in duration. The name convention is YYYYMMDD (folder name) /IDBYYYYMMDDHHMMSS (file name), where the time in the file name indicates the start time of the visibility data. However, to be useful for imaging, these files must be first calibrated and then converted to a CASA measurement set. At present, this can only be done with software at the OVRO site although we are working on software to allow this processing to be done elsewhere. For now, please request this processing to be done for you by emailing Dale Gary, Bin Chen, Sijie Yu, or others you may know in the solar radio group.		Once you identify the flare time, you can find the full-resolution (1-s cadence) uncalibrated visibility files (in Miriad format) at [http://www.ovsa.njit.edu/fits/IDB/ this link]. Each data file is usually 10 minutes in duration. The name convention is YYYYMMDD (folder name) /IDBYYYYMMDDHHMMSS (file name), where the time in the file name indicates the start time of the visibility data. However, to be useful for imaging, these files must be first calibrated and then converted to a CASA measurement set. At present, this can only be done with software at the OVRO site although we are working on software to allow this processing to be done elsewhere. For now, please request this processing to be done for you by emailing Dale Gary, Bin Chen, Sijie Yu, or others you may know in the solar radio group.

← Older revision		Revision as of 21:56, 7 July 2022
Line 34:		Line 34:
	* [http://www.ovsa.njit.edu/fits/UDBms_slfcaled/ EOVSA 1-min averaged CASA ms database]		* [http://www.ovsa.njit.edu/fits/UDBms_slfcaled/ EOVSA 1-min averaged CASA ms database]
	* [https://github.com/suncasa/suncasa-src SunCASA on Github] and [https://pypi.org/project/suncasa/ PyPI]		* [https://github.com/suncasa/suncasa-src SunCASA on Github] and [https://pypi.org/project/suncasa/ PyPI]
	* Flare pipeline?

	=3. Software requirements and installation=		=3. Software requirements and installation=