http://www.ovsa.njit.edu/wiki/api.php?action=feedcontributions&user=Dgary&feedformat=atomEOVSA Wiki - User contributions [en]2019-10-15T04:41:25ZUser contributionsMediaWiki 1.23.14http://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-11T20:10:00Z<p>Dgary: /* Comparing Data and Model */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine <br />
described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.2624GHz', '1.5874GHz', '2.5625GHz', '2.8875GHz', '3.2125GHz',<br />
'3.5374GHz', '3.8624GHz', '4.1875GHz', '4.5125GHz', '4.8375GHz',<br />
'5.1625GHz', '5.4875GHz', '5.8125GHz', '6.1375GHz', '6.4625GHz',<br />
'6.7875GHz', '7.1125GHz', '7.4375GHz', '7.7625GHz', '8.0875GHz',<br />
'8.4124GHz', '8.7375GHz', '9.0625GHz', '9.3875GHz', '9.7125GHz',<br />
'10.037GHz', '10.361GHz', '10.687GHz', '11.011GHz', '11.337GHz',<br />
'11.662GHz', '11.986GHz', '12.312GHz', '12.636GHz', '12.962GHz',<br />
'13.287GHz', '13.611GHz', '13.937GHz', '14.261GHz', '14.587GHz',<br />
'14.912GHz', '15.236GHz', '15.562GHz', '15.886GHz', '16.212GHz',<br />
'16.536GHz', '16.862GHz', '17.187GHz', '17.511GHz', '17.836GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop over spectal window number calls diskmodel() to create .fits and .im files for each band. The diskmodel() routine assumes that the EOVSA bands are 325 MHz wide, and CASA requires that the reference frequency reffreq correspond to the center of the band.<br />
<br />
==Inserting the Model into the Observation Measurement Set==<br />
<br />
Once the model images are complete, they are inserted into the measurement set using the CASA 'ft' (Fourier Transform) task. The loop below accomplishes this for all 50 spw.<br />
<br />
import glob<br />
files = glob.glob('disk*.im')<br />
<br />
for i in range(50):<br />
# Find spw in filename (assumes filename 'disk<spw>_*.im')<br />
spw = int(files[i].split('disk')[1].split('_')[0])<br />
ft(vis=vis, spw=str(spw), field='', model=str(files[i]), nterms=1, <br />
reffreq="''", complist="''", incremental=False, usescratch=True)<br />
<br />
This will result in filling the MODEL column of the dataset for each spw with the Fourier transform of the corresponding disk image.<br />
<br />
==Comparing Data and Model==<br />
<br />
To check how well the disk models fit the data, it is useful to compare the data and model using plotms. The following commands will display the result in Figure 6. [[file:4band-comparison.png|thumb|right|300px|'''Fig. 6:''' Final comparison of MODEL data column (cyan) to the original DATA column (blue) for spectral windows 16, 26, 36, and 41.]]<br />
<br />
spws = ['16', '26', '36', '41']<br />
for i in range(4):<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
gridrows=4, gridcols=1, ydatacolumn='data', rowindex=i, plotindex=2*i,<br />
plotrange=[10,1000,0,400000], clearplots=[True,False,False,False][i], spw=spws[i],<br />
customsymbol=True, symbolcolor='0000ff' , xsharedaxis=True)<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
timerange='14:00:00~23:00:00',<br />
gridrows=4, gridcols=1, ydatacolumn='model', rowindex=i, plotindex=2*i+1,<br />
plotrange=[10,1000,0,400000], clearplots=False, spw=spws[i],<br />
customsymbol=True, symbolcolor='00aaff' , xsharedaxis=True)<br />
<br />
Note that some amplitude scaling issues are apparent, so we should try amplitude selfcal to correct the data.<br />
<br />
The source code of the following procedures can be found in the [https://github.com/suncasa/suncasa/blob/master/eovsa/eovsa_diskmodel.py suncasa] package.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-10T20:34:38Z<p>Dgary: /* Inserting the Model into the Observation Measurement Set */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine <br />
described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.2624GHz', '1.5874GHz', '2.5625GHz', '2.8875GHz', '3.2125GHz',<br />
'3.5374GHz', '3.8624GHz', '4.1875GHz', '4.5125GHz', '4.8375GHz',<br />
'5.1625GHz', '5.4875GHz', '5.8125GHz', '6.1375GHz', '6.4625GHz',<br />
'6.7875GHz', '7.1125GHz', '7.4375GHz', '7.7625GHz', '8.0875GHz',<br />
'8.4124GHz', '8.7375GHz', '9.0625GHz', '9.3875GHz', '9.7125GHz',<br />
'10.037GHz', '10.361GHz', '10.687GHz', '11.011GHz', '11.337GHz',<br />
'11.662GHz', '11.986GHz', '12.312GHz', '12.636GHz', '12.962GHz',<br />
'13.287GHz', '13.611GHz', '13.937GHz', '14.261GHz', '14.587GHz',<br />
'14.912GHz', '15.236GHz', '15.562GHz', '15.886GHz', '16.212GHz',<br />
'16.536GHz', '16.862GHz', '17.187GHz', '17.511GHz', '17.836GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop over spectal window number calls diskmodel() to create .fits and .im files for each band. The diskmodel() routine assumes that the EOVSA bands are 325 MHz wide, and CASA requires that the reference frequency reffreq correspond to the center of the band.<br />
<br />
==Inserting the Model into the Observation Measurement Set==<br />
<br />
Once the model images are complete, they are inserted into the measurement set using the CASA 'ft' (Fourier Transform) task. The loop below accomplishes this for all 50 spw.<br />
<br />
import glob<br />
files = glob.glob('disk*.im')<br />
<br />
for i in range(50):<br />
# Find spw in filename (assumes filename 'disk<spw>_*.im')<br />
spw = int(files[i].split('disk')[1].split('_')[0])<br />
ft(vis=vis, spw=str(spw), field='', model=str(files[i]), nterms=1, <br />
reffreq="''", complist="''", incremental=False, usescratch=True)<br />
<br />
This will result in filling the MODEL column of the dataset for each spw with the Fourier transform of the corresponding disk image.<br />
<br />
==Comparing Data and Model==<br />
<br />
To check how well the disk models fit the data, it is useful to compare the data and model using plotms. The following commands will display the result in Figure 6. [[file:4band-comparison.png|thumb|right|300px|'''Fig. 6:''' Final comparison of MODEL data column (cyan) to the original DATA column (blue) for spectral windows 16, 26, 36, and 41.]]<br />
<br />
spws = ['16', '26', '36', '41']<br />
for i in range(4):<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
gridrows=4, gridcols=1, ydatacolumn='data', rowindex=i, plotindex=2*i,<br />
plotrange=[10,1000,0,400000], clearplots=[True,False,False,False][i], spw=spws[i],<br />
customsymbol=True, symbolcolor='0000ff' , xsharedaxis=True)<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
gridrows=4, gridcols=1, ydatacolumn='model', rowindex=i, plotindex=2*i+1,<br />
plotrange=[10,1000,0,400000], clearplots=False, spw=spws[i],<br />
customsymbol=True, symbolcolor='00aaff' , xsharedaxis=True)<br />
<br />
Note that some amplitude scaling issues are apparent, so we should try amplitude selfcal to correct the data.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-10T20:31:55Z<p>Dgary: /* Inserting the Model into the Observation Measurement Set */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine <br />
described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.2624GHz', '1.5874GHz', '2.5625GHz', '2.8875GHz', '3.2125GHz',<br />
'3.5374GHz', '3.8624GHz', '4.1875GHz', '4.5125GHz', '4.8375GHz',<br />
'5.1625GHz', '5.4875GHz', '5.8125GHz', '6.1375GHz', '6.4625GHz',<br />
'6.7875GHz', '7.1125GHz', '7.4375GHz', '7.7625GHz', '8.0875GHz',<br />
'8.4124GHz', '8.7375GHz', '9.0625GHz', '9.3875GHz', '9.7125GHz',<br />
'10.037GHz', '10.361GHz', '10.687GHz', '11.011GHz', '11.337GHz',<br />
'11.662GHz', '11.986GHz', '12.312GHz', '12.636GHz', '12.962GHz',<br />
'13.287GHz', '13.611GHz', '13.937GHz', '14.261GHz', '14.587GHz',<br />
'14.912GHz', '15.236GHz', '15.562GHz', '15.886GHz', '16.212GHz',<br />
'16.536GHz', '16.862GHz', '17.187GHz', '17.511GHz', '17.836GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop over spectal window number calls diskmodel() to create .fits and .im files for each band. The diskmodel() routine assumes that the EOVSA bands are 325 MHz wide, and CASA requires that the reference frequency reffreq correspond to the center of the band.<br />
<br />
==Inserting the Model into the Observation Measurement Set==<br />
<br />
Once the model images are complete, they are inserted into the measurement set using the CASA 'ft' (Fourier Transform) task. The loop below accomplishes this for all 50 spw.<br />
<br />
import glob<br />
files = glob.glob('disk*.im')<br />
<br />
for i in range(50):<br />
# Find spw in filename (assumes filename 'disk<spw>_*.im')<br />
spw = int(files[i].split('disk')[1].split('_')[0])<br />
ft(vis=vis, spw=str(spw), field='', model=str(files[i]), nterms=1, <br />
reffreq=\'\', complist=\'\', incremental=False, usescratch=False)<br />
<br />
This will result in filling the MODEL column of the dataset for each spw with the Fourier transform of the corresponding disk image.<br />
<br />
==Comparing Data and Model==<br />
<br />
To check how well the disk models fit the data, it is useful to compare the data and model using plotms. The following commands will display the result in Figure 6. [[file:4band-comparison.png|thumb|right|300px|'''Fig. 6:''' Final comparison of MODEL data column (cyan) to the original DATA column (blue) for spectral windows 16, 26, 36, and 41.]]<br />
<br />
spws = ['16', '26', '36', '41']<br />
for i in range(4):<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
gridrows=4, gridcols=1, ydatacolumn='data', rowindex=i, plotindex=2*i,<br />
plotrange=[10,1000,0,400000], clearplots=[True,False,False,False][i], spw=spws[i],<br />
customsymbol=True, symbolcolor='0000ff' , xsharedaxis=True)<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
gridrows=4, gridcols=1, ydatacolumn='model', rowindex=i, plotindex=2*i+1,<br />
plotrange=[10,1000,0,400000], clearplots=False, spw=spws[i],<br />
customsymbol=True, symbolcolor='00aaff' , xsharedaxis=True)<br />
<br />
Note that some amplitude scaling issues are apparent, so we should try amplitude selfcal to correct the data.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/File:4band-comparison.pngFile:4band-comparison.png2019-09-10T12:41:07Z<p>Dgary: </p>
<hr />
<div></div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-10T12:40:44Z<p>Dgary: /* Comparing Data and Model */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine <br />
described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.2624GHz', '1.5874GHz', '2.5625GHz', '2.8875GHz', '3.2125GHz',<br />
'3.5374GHz', '3.8624GHz', '4.1875GHz', '4.5125GHz', '4.8375GHz',<br />
'5.1625GHz', '5.4875GHz', '5.8125GHz', '6.1375GHz', '6.4625GHz',<br />
'6.7875GHz', '7.1125GHz', '7.4375GHz', '7.7625GHz', '8.0875GHz',<br />
'8.4124GHz', '8.7375GHz', '9.0625GHz', '9.3875GHz', '9.7125GHz',<br />
'10.037GHz', '10.361GHz', '10.687GHz', '11.011GHz', '11.337GHz',<br />
'11.662GHz', '11.986GHz', '12.312GHz', '12.636GHz', '12.962GHz',<br />
'13.287GHz', '13.611GHz', '13.937GHz', '14.261GHz', '14.587GHz',<br />
'14.912GHz', '15.236GHz', '15.562GHz', '15.886GHz', '16.212GHz',<br />
'16.536GHz', '16.862GHz', '17.187GHz', '17.511GHz', '17.836GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop over spectal window number calls diskmodel() to create .fits and .im files for each band. The diskmodel() routine assumes that the EOVSA bands are 325 MHz wide, and CASA requires that the reference frequency reffreq correspond to the center of the band.<br />
<br />
==Inserting the Model into the Observation Measurement Set==<br />
<br />
Once the model images are complete, they are inserted into the measurement set using the CASA 'ft' (Fourier Transform) task. The loop below accomplishes this for all 50 spw.<br />
<br />
import glob<br />
files = glob.glob('disk*.im')<br />
<br />
for i in range(50):<br />
# Find spw in filename (assumes filename 'disk<spw>_*.im')<br />
spw = int(files[i].split('disk')[1].split('_')[0])<br />
ft(vis=vis, spw=str(spw), field='', model=str(files[i]), nterms=1, <br />
reffreq='', complist='', incremental=False, usescratch=False)<br />
<br />
This will result in filling the MODEL column of the dataset for each spw with the Fourier transform of the corresponding disk image.<br />
<br />
==Comparing Data and Model==<br />
<br />
To check how well the disk models fit the data, it is useful to compare the data and model using plotms. The following commands will display the result in Figure 6. [[file:4band-comparison.png|thumb|right|300px|'''Fig. 6:''' Final comparison of MODEL data column (cyan) to the original DATA column (blue) for spectral windows 16, 26, 36, and 41.]]<br />
<br />
spws = ['16', '26', '36', '41']<br />
for i in range(4):<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
gridrows=4, gridcols=1, ydatacolumn='data', rowindex=i, plotindex=2*i,<br />
plotrange=[10,1000,0,400000], clearplots=[True,False,False,False][i], spw=spws[i],<br />
customsymbol=True, symbolcolor='0000ff' , xsharedaxis=True)<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
gridrows=4, gridcols=1, ydatacolumn='model', rowindex=i, plotindex=2*i+1,<br />
plotrange=[10,1000,0,400000], clearplots=False, spw=spws[i],<br />
customsymbol=True, symbolcolor='00aaff' , xsharedaxis=True)<br />
<br />
Note that some amplitude scaling issues are apparent, so we should try amplitude selfcal to correct the data.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-10T12:39:20Z<p>Dgary: /* fit_vs_freq() */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine <br />
described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.2624GHz', '1.5874GHz', '2.5625GHz', '2.8875GHz', '3.2125GHz',<br />
'3.5374GHz', '3.8624GHz', '4.1875GHz', '4.5125GHz', '4.8375GHz',<br />
'5.1625GHz', '5.4875GHz', '5.8125GHz', '6.1375GHz', '6.4625GHz',<br />
'6.7875GHz', '7.1125GHz', '7.4375GHz', '7.7625GHz', '8.0875GHz',<br />
'8.4124GHz', '8.7375GHz', '9.0625GHz', '9.3875GHz', '9.7125GHz',<br />
'10.037GHz', '10.361GHz', '10.687GHz', '11.011GHz', '11.337GHz',<br />
'11.662GHz', '11.986GHz', '12.312GHz', '12.636GHz', '12.962GHz',<br />
'13.287GHz', '13.611GHz', '13.937GHz', '14.261GHz', '14.587GHz',<br />
'14.912GHz', '15.236GHz', '15.562GHz', '15.886GHz', '16.212GHz',<br />
'16.536GHz', '16.862GHz', '17.187GHz', '17.511GHz', '17.836GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop over spectal window number calls diskmodel() to create .fits and .im files for each band. The diskmodel() routine assumes that the EOVSA bands are 325 MHz wide, and CASA requires that the reference frequency reffreq correspond to the center of the band.<br />
<br />
==Inserting the Model into the Observation Measurement Set==<br />
<br />
Once the model images are complete, they are inserted into the measurement set using the CASA 'ft' (Fourier Transform) task. The loop below accomplishes this for all 50 spw.<br />
<br />
import glob<br />
files = glob.glob('disk*.im')<br />
<br />
for i in range(50):<br />
# Find spw in filename (assumes filename 'disk<spw>_*.im')<br />
spw = int(files[i].split('disk')[1].split('_')[0])<br />
ft(vis=vis, spw=str(spw), field='', model=str(files[i]), nterms=1, <br />
reffreq='', complist='', incremental=False, usescratch=False)<br />
<br />
This will result in filling the MODEL column of the dataset for each spw with the Fourier transform of the corresponding disk image.<br />
<br />
==Comparing Data and Model==<br />
<br />
To check how well the disk models fit the data, it is useful to compare the data and model using plotms. The following commands will display the result in Figure 5. [[file:'4band-comparison.png'] Caption]]<br />
<br />
spws = ['16', '26', '36', '41']<br />
for i in range(4):<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
gridrows=4, gridcols=1, ydatacolumn='data', rowindex=i, plotindex=2*i,<br />
plotrange=[10,1000,0,400000], clearplots=[True,False,False,False][i], spw=spws[i],<br />
customsymbol=True, symbolcolor='0000ff' , xsharedaxis=True)<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
gridrows=4, gridcols=1, ydatacolumn='model', rowindex=i, plotindex=2*i+1,<br />
plotrange=[10,1000,0,400000], clearplots=False, spw=spws[i],<br />
customsymbol=True, symbolcolor='00aaff' , xsharedaxis=True)<br />
<br />
Note that some amplitude scaling issues are apparent, so we should try amplitude selfcal to correct the data.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-10T12:38:33Z<p>Dgary: /* Comparing Data and Model */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.2624GHz', '1.5874GHz', '2.5625GHz', '2.8875GHz', '3.2125GHz',<br />
'3.5374GHz', '3.8624GHz', '4.1875GHz', '4.5125GHz', '4.8375GHz',<br />
'5.1625GHz', '5.4875GHz', '5.8125GHz', '6.1375GHz', '6.4625GHz',<br />
'6.7875GHz', '7.1125GHz', '7.4375GHz', '7.7625GHz', '8.0875GHz',<br />
'8.4124GHz', '8.7375GHz', '9.0625GHz', '9.3875GHz', '9.7125GHz',<br />
'10.037GHz', '10.361GHz', '10.687GHz', '11.011GHz', '11.337GHz',<br />
'11.662GHz', '11.986GHz', '12.312GHz', '12.636GHz', '12.962GHz',<br />
'13.287GHz', '13.611GHz', '13.937GHz', '14.261GHz', '14.587GHz',<br />
'14.912GHz', '15.236GHz', '15.562GHz', '15.886GHz', '16.212GHz',<br />
'16.536GHz', '16.862GHz', '17.187GHz', '17.511GHz', '17.836GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop over spectal window number calls diskmodel() to create .fits and .im files for each band. The diskmodel() routine assumes that the EOVSA bands are 325 MHz wide, and CASA requires that the reference frequency reffreq correspond to the center of the band.<br />
<br />
==Inserting the Model into the Observation Measurement Set==<br />
<br />
Once the model images are complete, they are inserted into the measurement set using the CASA 'ft' (Fourier Transform) task. The loop below accomplishes this for all 50 spw.<br />
<br />
import glob<br />
files = glob.glob('disk*.im')<br />
<br />
for i in range(50):<br />
# Find spw in filename (assumes filename 'disk<spw>_*.im')<br />
spw = int(files[i].split('disk')[1].split('_')[0])<br />
ft(vis=vis, spw=str(spw), field='', model=str(files[i]), nterms=1, <br />
reffreq='', complist='', incremental=False, usescratch=False)<br />
<br />
This will result in filling the MODEL column of the dataset for each spw with the Fourier transform of the corresponding disk image.<br />
<br />
==Comparing Data and Model==<br />
<br />
To check how well the disk models fit the data, it is useful to compare the data and model using plotms. The following commands will display the result in Figure 5. [[file:'4band-comparison.png'] Caption]]<br />
<br />
spws = ['16', '26', '36', '41']<br />
for i in range(4):<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
gridrows=4, gridcols=1, ydatacolumn='data', rowindex=i, plotindex=2*i,<br />
plotrange=[10,1000,0,400000], clearplots=[True,False,False,False][i], spw=spws[i],<br />
customsymbol=True, symbolcolor='0000ff' , xsharedaxis=True)<br />
plotms(vis=vis, xaxis='uvwave', yaxis='amp', antenna='0~3', correlation='XX',<br />
gridrows=4, gridcols=1, ydatacolumn='model', rowindex=i, plotindex=2*i+1,<br />
plotrange=[10,1000,0,400000], clearplots=False, spw=spws[i],<br />
customsymbol=True, symbolcolor='00aaff' , xsharedaxis=True)<br />
<br />
Note that some amplitude scaling issues are apparent, so we should try amplitude selfcal to correct the data.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-10T12:00:18Z<p>Dgary: /* Inserting the Model into the Observation Measurement Set */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.2624GHz', '1.5874GHz', '2.5625GHz', '2.8875GHz', '3.2125GHz',<br />
'3.5374GHz', '3.8624GHz', '4.1875GHz', '4.5125GHz', '4.8375GHz',<br />
'5.1625GHz', '5.4875GHz', '5.8125GHz', '6.1375GHz', '6.4625GHz',<br />
'6.7875GHz', '7.1125GHz', '7.4375GHz', '7.7625GHz', '8.0875GHz',<br />
'8.4124GHz', '8.7375GHz', '9.0625GHz', '9.3875GHz', '9.7125GHz',<br />
'10.037GHz', '10.361GHz', '10.687GHz', '11.011GHz', '11.337GHz',<br />
'11.662GHz', '11.986GHz', '12.312GHz', '12.636GHz', '12.962GHz',<br />
'13.287GHz', '13.611GHz', '13.937GHz', '14.261GHz', '14.587GHz',<br />
'14.912GHz', '15.236GHz', '15.562GHz', '15.886GHz', '16.212GHz',<br />
'16.536GHz', '16.862GHz', '17.187GHz', '17.511GHz', '17.836GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop over spectal window number calls diskmodel() to create .fits and .im files for each band. The diskmodel() routine assumes that the EOVSA bands are 325 MHz wide, and CASA requires that the reference frequency reffreq correspond to the center of the band.<br />
<br />
==Inserting the Model into the Observation Measurement Set==<br />
<br />
Once the model images are complete, they are inserted into the measurement set using the CASA 'ft' (Fourier Transform) task. The loop below accomplishes this for all 50 spw.<br />
<br />
import glob<br />
files = glob.glob('disk*.im')<br />
<br />
for i in range(50):<br />
# Find spw in filename (assumes filename 'disk<spw>_*.im')<br />
spw = int(files[i].split('disk')[1].split('_')[0])<br />
ft(vis=vis, spw=str(spw), field='', model=str(files[i]), nterms=1, <br />
reffreq='', complist='', incremental=False, usescratch=False)<br />
<br />
This will result in filling the MODEL column of the dataset for each spw with the Fourier transform of the corresponding disk image.<br />
<br />
==Comparing Data and Model==<br />
<br />
To check how well the disk models fit the data, it is useful to compare the data and model using plotms.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-10T11:47:33Z<p>Dgary: /* Using the Results to Create CASA Disk Model Images */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.2624GHz', '1.5874GHz', '2.5625GHz', '2.8875GHz', '3.2125GHz',<br />
'3.5374GHz', '3.8624GHz', '4.1875GHz', '4.5125GHz', '4.8375GHz',<br />
'5.1625GHz', '5.4875GHz', '5.8125GHz', '6.1375GHz', '6.4625GHz',<br />
'6.7875GHz', '7.1125GHz', '7.4375GHz', '7.7625GHz', '8.0875GHz',<br />
'8.4124GHz', '8.7375GHz', '9.0625GHz', '9.3875GHz', '9.7125GHz',<br />
'10.037GHz', '10.361GHz', '10.687GHz', '11.011GHz', '11.337GHz',<br />
'11.662GHz', '11.986GHz', '12.312GHz', '12.636GHz', '12.962GHz',<br />
'13.287GHz', '13.611GHz', '13.937GHz', '14.261GHz', '14.587GHz',<br />
'14.912GHz', '15.236GHz', '15.562GHz', '15.886GHz', '16.212GHz',<br />
'16.536GHz', '16.862GHz', '17.187GHz', '17.511GHz', '17.836GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop over spectal window number calls diskmodel() to create .fits and .im files for each band. The diskmodel() routine assumes that the EOVSA bands are 325 MHz wide, and CASA requires that the reference frequency reffreq correspond to the center of the band.<br />
<br />
==Inserting the Model into the Observation Measurement Set==<br />
<br />
Once the model images are complete, they are inserted into the measurement set using the CASA 'ft' (Fourier Transform) task. The loop below accomplishes this for all 50 spw.<br />
<br />
for spw in range(50):</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-09T17:43:41Z<p>Dgary: /* Final Adopted Result of Fitting */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.2624GHz', '1.5874GHz', '2.5625GHz', '2.8875GHz', '3.2125GHz',<br />
'3.5374GHz', '3.8624GHz', '4.1875GHz', '4.5125GHz', '4.8375GHz',<br />
'5.1625GHz', '5.4875GHz', '5.8125GHz', '6.1375GHz', '6.4625GHz',<br />
'6.7875GHz', '7.1125GHz', '7.4375GHz', '7.7625GHz', '8.0875GHz',<br />
'8.4124GHz', '8.7375GHz', '9.0625GHz', '9.3875GHz', '9.7125GHz',<br />
'10.037GHz', '10.361GHz', '10.687GHz', '11.011GHz', '11.337GHz',<br />
'11.662GHz', '11.986GHz', '12.312GHz', '12.636GHz', '12.962GHz',<br />
'13.287GHz', '13.611GHz', '13.937GHz', '14.261GHz', '14.587GHz',<br />
'14.912GHz', '15.236GHz', '15.562GHz', '15.886GHz', '16.212GHz',<br />
'16.536GHz', '16.862GHz', '17.187GHz', '17.511GHz', '17.836GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop over spectal window number calls diskmodel() to create .fits and .im files for each band.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-09T02:05:22Z<p>Dgary: /* Using the Results to Create CASA Disk Model Images */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.0999GHz', '1.4249GHz', '2.4GHz', '2.725GHz', '3.05GHz',<br />
'3.3749GHz', '3.6999GHz', '4.025GHz', '4.35GHz', '4.675GHz',<br />
'5.0000GHz', '5.3250GHz', '5.65GHz', '5.975GHz', '6.3GHz',<br />
'6.625GHz', '6.9500GHz', '7.275GHz', '7.6GHz', '7.925GHz',<br />
'8.2499GHz', '8.5750GHz', '8.9GHz', '9.225GHz', '9.55GHz',<br />
'9.875GHz', '10.199GHz', '10.525GHz', '10.849GHz', '11.175GHz',<br />
'11.5GHz', '11.824GHz', '12.15GHz', '12.474GHz', '12.8GHz',<br />
'13.125GHz', '13.449GHz', '13.775GHz', '14.099GHz', '14.425GHz',<br />
'14.75GHz', '15.074GHz', '15.4GHz', '15.724GHz', '16.05GHz',<br />
'16.374GHz', '16.7GHz', '17.025GHz', '17.349GHz', '17.674GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop over spectal window number calls diskmodel() to create .fits and .im files for each band.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-09T02:04:31Z<p>Dgary: /* Using the Results to Create CASA Disk Model Images */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.0999GHz', '1.4249GHz', '2.4GHz', '2.725GHz', '3.05GHz',<br />
'3.3749GHz', '3.6999GHz', '4.025GHz', '4.35GHz', '4.675GHz',<br />
'5.0000GHz', '5.3250GHz', '5.65GHz', '5.975GHz', '6.3GHz',<br />
'6.625GHz', '6.9500GHz', '7.275GHz', '7.6GHz', '7.925GHz',<br />
'8.2499GHz', '8.5750GHz', '8.9GHz', '9.225GHz', '9.55GHz',<br />
'9.875GHz', '10.199GHz', '10.525GHz', '10.849GHz', '11.175GHz',<br />
'11.5GHz', '11.824GHz', '12.15GHz', '12.474GHz', '12.8GHz',<br />
'13.125GHz', '13.449GHz', '13.775GHz', '14.099GHz', '14.425GHz',<br />
'14.75GHz', '15.074GHz', '15.4GHz', '15.724GHz', '16.05GHz',<br />
'16.374GHz', '16.7GHz', '17.025GHz', '17.349GHz', '17.674GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop repeatedly calls diskmodel() to create .fits and .im files for one band.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-09T02:04:14Z<p>Dgary: /* Final Adopted Result of Fitting */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.0999GHz', '1.4249GHz', '2.4GHz', '2.725GHz', '3.05GHz',<br />
'3.3749GHz', '3.6999GHz', '4.025GHz', '4.35GHz', '4.675GHz',<br />
'5.0000GHz', '5.3250GHz', '5.65GHz', '5.975GHz', '6.3GHz',<br />
'6.625GHz', '6.9500GHz', '7.275GHz', '7.6GHz', '7.925GHz',<br />
'8.2499GHz', '8.5750GHz', '8.9GHz', '9.225GHz', '9.55GHz',<br />
'9.875GHz', '10.199GHz', '10.525GHz', '10.849GHz', '11.175GHz',<br />
'11.5GHz', '11.824GHz', '12.15GHz', '12.474GHz', '12.8GHz',<br />
'13.125GHz', '13.449GHz', '13.775GHz', '14.099GHz', '14.425GHz',<br />
'14.75GHz', '15.074GHz', '15.4GHz', '15.724GHz', '16.05GHz',<br />
'16.374GHz', '16.7GHz', '17.025GHz', '17.349GHz', '17.674GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])<br />
<br />
==Using the Results to Create CASA Disk Model Images==<br />
<br />
CASA starts with properly formatted images from which to create uv models. I wrote a routine to create such properly formatted images in both FITS and CASA (.im) format. To make a frequency-dependent model, one must have such an image for each desired band (spw), which for EOVSA means 50 images. It is quite fast and easy to use the above three parameter arrays to create disk models, using the disk modeling routine diskmodel() that I wrote. The full call to create all 50 images is:<br />
<br />
for spw in range(50):<br />
diskmodel(outname='disk'+str(spw)+'_',direction='J2000 10h40m53.785s 08d20m37.60407s',<br />
reffreq=frq[spw],flux=fdens[spw],eqradius=dsize[spw],polradius=dsize[spw])<br />
print spw<br />
<br />
where the loop repeatedly calls diskmodel to create .fits and .im files for one band.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-09T01:44:39Z<p>Dgary: </p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Final Adopted Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.0999GHz', '1.4249GHz', '2.4GHz', '2.725GHz', '3.05GHz',<br />
'3.3749GHz', '3.6999GHz', '4.025GHz', '4.35GHz', '4.675GHz',<br />
'5.0000GHz', '5.3250GHz', '5.65GHz', '5.975GHz', '6.3GHz',<br />
'6.625GHz', '6.9500GHz', '7.275GHz', '7.6GHz', '7.925GHz',<br />
'8.2499GHz', '8.5750GHz', '8.9GHz', '9.225GHz', '9.55GHz',<br />
'9.875GHz', '10.199GHz', '10.525GHz', '10.849GHz', '11.175GHz',<br />
'11.5GHz', '11.824GHz', '12.15GHz', '12.474GHz', '12.8GHz',<br />
'13.125GHz', '13.449GHz', '13.775GHz', '14.099GHz', '14.425GHz',<br />
'14.75GHz', '15.074GHz', '15.4GHz', '15.724GHz', '16.05GHz',<br />
'16.374GHz', '16.7GHz', '17.025GHz', '17.349GHz', '17.674GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-09T01:43:18Z<p>Dgary: /* fit_vs_freq() */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. <br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]<br />
<br />
==Result of Fitting==<br />
In order to go further, a smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-dependent source model, described below. The following three input arrays, based on these fitting results, are used for creating the source model.<br />
* The list of spectral window starting frequencies, formatted for CASA:<br />
frq = np.array(['1.0999GHz', '1.4249GHz', '2.4GHz', '2.725GHz', '3.05GHz',<br />
'3.3749GHz', '3.6999GHz', '4.025GHz', '4.35GHz', '4.675GHz',<br />
'5.0000GHz', '5.3250GHz', '5.65GHz', '5.975GHz', '6.3GHz',<br />
'6.625GHz', '6.9500GHz', '7.275GHz', '7.6GHz', '7.925GHz',<br />
'8.2499GHz', '8.5750GHz', '8.9GHz', '9.225GHz', '9.55GHz',<br />
'9.875GHz', '10.199GHz', '10.525GHz', '10.849GHz', '11.175GHz',<br />
'11.5GHz', '11.824GHz', '12.15GHz', '12.474GHz', '12.8GHz',<br />
'13.125GHz', '13.449GHz', '13.775GHz', '14.099GHz', '14.425GHz',<br />
'14.75GHz', '15.074GHz', '15.4GHz', '15.724GHz', '16.05GHz',<br />
'16.374GHz', '16.7GHz', '17.025GHz', '17.349GHz', '17.674GHz'],<br />
dtype='|S9')<br />
<br />
* The flux density, in units of Jy, which are basically the RSTN sfu values doubled:<br />
fdens = np.array([ 891282, 954570, 1173229, 1245433, 1373730, 1506802,<br />
1613253, 1702751, 1800721, 1946756, 2096020, 2243951,<br />
2367362, 2525968, 2699795, 2861604, 3054829, 3220450,<br />
3404182, 3602625, 3794312, 3962926, 4164667, 4360683,<br />
4575677, 4767210, 4972824, 5211717, 5444632, 5648266,<br />
5926634, 6144249, 6339863, 6598018, 6802707, 7016012,<br />
7258929, 7454951, 7742816, 7948976, 8203206, 8411834,<br />
8656720, 8908130, 9087766, 9410760, 9571365, 9827078,<br />
10023598, 8896671])<br />
<br />
* The solar disk size (assumed circular), formatted for CASA:<br />
dsize = np.array(['1228.0arcsec', '1194.0arcsec', '1165.0arcsec', '1139.0arcsec', '1117.0arcsec', <br />
'1097.0arcsec', '1080.0arcsec', '1065.0arcsec', '1053.0arcsec', '1042.0arcsec', '1033.0arcsec', <br />
'1025.0arcsec', '1018.0arcsec', '1012.0arcsec',<br />
'1008.0arcsec', '1003.0arcsec', '1000.0arcsec', '997.0arcsec', '994.0arcsec', '992.0arcsec',<br />
'990.0arcsec', '988.0arcsec', '986.0arcsec', '985.0arcsec', '983.0arcsec', '982.0arcsec',<br />
'980.0arcsec', '979.0arcsec', '978.0arcsec', '976.0arcsec', '975.0arcsec', '974.0arcsec',<br />
'972.0arcsec', '971.0arcsec', '970.0arcsec', '969.0arcsec', '968.0arcsec', '967.0arcsec',<br />
'966.0arcsec', '965.0arcsec', '964.0arcsec', '964.0arcsec', '963.0arcsec', '962.0arcsec',<br />
'962.0arcsec', '961.0arcsec', '960.0arcsec', '959.0arcsec', '957.0arcsec', '956.0arcsec'])</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-09T01:37:32Z<p>Dgary: /* fit_vs_freq() */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine described below (this is 2 x RSTN)<br />
One should perhaps not take the summary plot results at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Nevertheless, the results as a whole are plausible as shown in Figure 5. This plot is the result of one such run, where the uv fit range (internal to the fit_vs_freq() routine) is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. A smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-depended source model<br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-09T01:35:16Z<p>Dgary: /* Disk Fitting Python Code */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
====read_ms()====<br />
The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
====fit_diskmodel====<br />
The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
====fit_vs_freq()====<br />
The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. The syntax is a very simple one-line call:<br />
result = fit_vs_freq(out)<br />
The result dictionary has the following keys:<br />
'band' The spectral window band number from 0-49<br />
'fghz' The frequency in GHz corresponding to the start of the band<br />
'eqradius' The equatorial radius for each band, in arcsec<br />
'polradius' The polar radius for each band, in arcsec<br />
'radius' The radius determined from the fit to all baselines<br />
'disk_flux' The flux density one should use in a later routine described below (this is 2 x RSTN)<br />
One should perhaps not take the summary at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Figure 5 shows the result of one such run, where the uv fit range is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. A smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-depended source model<br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/File:20190901_Solar_Disk_Size.pngFile:20190901 Solar Disk Size.png2019-09-09T01:26:43Z<p>Dgary: </p>
<hr />
<div></div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-09T01:26:27Z<p>Dgary: /* Disk Fitting Python Code */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
* The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
* The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|300px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|300px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40. The true reason for the discrepancy in fits between Figures 3 and 4 is still being investigated. One idea is that there is a relatively sharp inner limb and a more gradual fall-off at greater heights. In this case, the longer baselines may resolve out this gradual decline and respond to the smaller, sharper limb, while the shorter baselines see the broader outer limb just fine.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|300px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
* The final routine, fit_vs_freq(), simply automates the running of fit_diskmodel() for all bands, plots the results, and returns the results in a nice summary dictionary. One should perhaps not take the summary at face value in the light of Figures 3 and 4, however, since clearly different answers can be obtained depending on the uv range used for the fitting. Figure 5 shows the result of one such run, where the uv fit range is scaled outward in an algorithmic way according to<br />
uvfitrange = np.array([10,150])+np.array([1,18])*i<br />
where i is the band number. A smooth fit (5th-order polynomial) to the red points has been done in order to create a frequency-depended source model<br />
[[file:20190901_Solar_Disk_Size.png|thumb|right|300px|'''Fig. 5:''' Result of running fit_diskmodel() for all 50 spectral windows. There is a clear and plausible drop in solar radius with frequency.]]</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/File:20190901_band10_diskfit2.pngFile:20190901 band10 diskfit2.png2019-09-09T01:12:51Z<p>Dgary: </p>
<hr />
<div></div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/File:20190901_band10_diskfit1.pngFile:20190901 band10 diskfit1.png2019-09-09T01:12:29Z<p>Dgary: </p>
<hr />
<div></div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-09T01:12:09Z<p>Dgary: /* Disk Fitting Python Code */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
* The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
* The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. One can see that the fit is excellent all the way out to 1000 wavelengths, and for this high frequency (14.7 GHz), the three fits (to equator, pole, and all points) all agree closely on the solar radius (964", 961", and 962", respectively). The photospheric disk is only 951" on this date, so these sizes are about 10" larger than the photosphere at this high frequency. The flux scale factor is also nominal, meaning that the RSTN flux density fits well to the data. [[file:20190901_band40_diskfit.png|thumb|right|500px|'''Fig. 2:''' Upper panel, fit of an Airy disk to baselines with more-or-less equatorial orientation. Middle panel, fit to baselines with more-or-less polar orientation. Lower panel, fit to all baselines.]]<br />
<br />
However, at lower frequencies things are a little more tricky. The plot in Figure 3 shows the spw 10 (5 GHz) fit for uv range 20-250 wavelengths. For that range, which covers the bulk of the flux, all of the disk sizes have increased quite a lot. In addition, the equatorial disk seems to be significantly larger than the polar disk, with an equatorial radius of 1045" and a polar radius of 1026". But also note how bad the fit becomes at longer uv distances.<br />
[[file:20190901_band10_diskfit1.png|thumb|right|500px|'''Fig. 3:''' Same as Figure 2, but for 5 GHz (spw 10), and fitting only the shorter baselines.]]<br />
<br />
When a larger uv range is used, as shown in Figure 4 (uvrange 200-900 wavelengths), a smaller disk size is suggested, around 990". This is still considerably larger than at spw 40.<br />
[[file:20190901_band10_diskfit2.png|thumb|right|500px|'''Fig. 4:''' Same as Figure 3, but fitting a wider range of baselines.]]<br />
<br />
<br />
<br />
I also investigated whether there is a dependence on the location of the minima with the angle of the baseline (which would indicate a non-circular disk). There does seem to be some evidence for that, with a rather surprisingly large variation of order 10%. Still, the sizes I am getting for the equatorial direction are something like 2100", which is pretty big, so it might make sense that the polar direction is much smaller. One would expect these variations to decrease at higher frequency, where the disk is less affected by the corona. In fact, this does seem to be the case. Figure 2 shows the same date as Fig. 1, but at higher frequencies where a single radius (968") fits all of the nulls up to at least the first 15! Note that this is still 17" larger than the photospheric value of 951" on that date. [[file:Disk_vis_high_freq.png|thumb|right|500px|'''Fig. 2:''' Example all-day visibilities for baselines 1-* at frequencies around 10 GHz or higher, showing the periodic nulls due to the solar disk. The orange lines are the predicted position of the nulls for a disk of 968".]]</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/File:20190901_band40_diskfit.pngFile:20190901 band40 diskfit.png2019-09-09T00:50:12Z<p>Dgary: </p>
<hr />
<div></div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-09T00:49:51Z<p>Dgary: /* Estimating disk axes and flux density from the data */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. The value of <math>z</math> is related to the diameter of the solar disk by the relation<br />
<br />
<math>z = \pi D_{{\rm rad}} s_\lambda = \pi^2 D^{''}/(180\times3600) s_\lambda</math>, <br />
<br />
where <math>D_{{\rm rad}}</math> is the solar diameter in radians, <math>s_\lambda</math> is the uv distance in wavelengths, and the second relation is just the conversion to the solar diameter in arcsec, <math>D^{''}</math>. Note that the relevant disk size is not the photospheric radius, but rather the radius at radio frequencies, which varies with frequency and may even be different in the equatorial and polar axes. It may generally be expected to grow larger at lower frequencies, and grow more oblate near the equator as the coronal flux density becomes non-negligible.<br />
<br />
===Disk Fitting Python Code===<br />
A set of three routines have been written in the Python module disk_fitting.py for the purposes of fitting the data in an all-day UDBms for the frequency-dependent disk shape and flux density. The input data are expected to be fully calibrated for amplitude and phase by the standard EOVSA pipeline. <br />
* The routine read_ms() reads the UDBms visibilities and returns the relevant data in a convenient dictionary format that is used by the subsequent fitting routines. This routine must be used in the CASA (or SunCASA) environment. Here is an example:<br />
import disk_fitting as df<br />
vis = '/data1/eovsa/fits/UDBms/201908/UDB20190825.ms'<br />
out = df.read_ms(vis)<br />
It takes several minutes to read an entire day's file.<br />
* The routine fit_diskmodel() uses the output from read_ms() and fits a disk model for one band (or spectral window, spw, in CASA parlance) at a time. A nice summary plot of the data and fit is generated if doplot=True. One thing that the routine needs is the full Sun flux density spectrum derived from the RSTN fluxes collected each day by NOAA. The example below shows how to get that information and use it to fit a disk to the spw 40 data using fit_diskmodel():<br />
import rstn<br />
from util import Time<br />
frq, flux = rstn.rd_rstnflux(t=Time('2019-09-01'))<br />
rstn_flux = rstn.rstn2ant(frq, flux, out['fghz']*1000, t=Time('2019-09-01'))<br />
fit_diskmodel(out, 40, rstn_flux, uvfitrange=[20,800], angle_tolerance=np.pi/2)<br />
The resulting plot is shown in Figure 2, where the uv range used for fitting (20-800 wavelengths) is shown by the blue-highlighted data points. [[file:20190901_band40_diskfit.png|thumb|right|500px|'''Fig. 2:''' ]]<br />
<br />
<br />
I also investigated whether there is a dependence on the location of the minima with the angle of the baseline (which would indicate a non-circular disk). There does seem to be some evidence for that, with a rather surprisingly large variation of order 10%. Still, the sizes I am getting for the equatorial direction are something like 2100", which is pretty big, so it might make sense that the polar direction is much smaller. One would expect these variations to decrease at higher frequency, where the disk is less affected by the corona. In fact, this does seem to be the case. Figure 2 shows the same date as Fig. 1, but at higher frequencies where a single radius (968") fits all of the nulls up to at least the first 15! Note that this is still 17" larger than the photospheric value of 951" on that date. [[file:Disk_vis_high_freq.png|thumb|right|500px|'''Fig. 2:''' Example all-day visibilities for baselines 1-* at frequencies around 10 GHz or higher, showing the periodic nulls due to the solar disk. The orange lines are the predicted position of the nulls for a disk of 968".]]</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-08T23:31:02Z<p>Dgary: /* Estimating disk axes and flux density from the data */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The visibilities due to the solar disk are the same as the Fourier Transform of a circular aperture, which is an Airy pattern (not an Airy function, which apparently is a different mathematical function). The power distribution vs. radius in an Airy disk is <math>P = P_0 \bigg({2J_1(z) \over z}\bigg)^2</math>, where <math>J_1(z)</math> is the first-order Bessel function and <math>P_0</math> is the peak intensity. However, in the uv-plane we are looking not at the power, but rather the electric field pattern itself, hence we are interested in the square root of this function, i.e.<br />
<math>I = I_0 \bigg({2J_1(z) \over z}\bigg)</math>. Note that the first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
which must correspond to the nearly periodic dips (nulls) in Figure 1. From which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = 206265 {m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. In some comparisons with the data, I do find that there is a systematic trend for the diameter to get smaller for larger ''m''. One possibility that I explored is whether a fuzzy disk has different minima than a sharply defined disk, but this does not seem to be the case.<br />
<br />
I also investigated whether there is a dependence on the location of the minima with the angle of the baseline (which would indicate a non-circular disk). There does seem to be some evidence for that, with a rather surprisingly large variation of order 10%. Still, the sizes I am getting for the equatorial direction are something like 2100", which is pretty big, so it might make sense that the polar direction is much smaller. One would expect these variations to decrease at higher frequency, where the disk is less affected by the corona. In fact, this does seem to be the case. Figure 2 shows the same date as Fig. 1, but at higher frequencies where a single radius (968") fits all of the nulls up to at least the first 15! Note that this is still 17" larger than the photospheric value of 951" on that date. [[file:Disk_vis_high_freq.png|thumb|right|500px|'''Fig. 2:''' Example all-day visibilities for baselines 1-* at frequencies around 10 GHz or higher, showing the periodic nulls due to the solar disk. The orange lines are the predicted position of the nulls for a disk of 968".]]</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-06T19:15:53Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, which allows one to create a disk with unequal axes (generally larger in the equatorial axis and smaller in the polar axis). However, before it can be used one must have a good idea of the disk size and flux density of the disk.<br />
<br />
== Estimating disk axes and flux density from the data ==<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. As it turns out, EOVSA visibilities provide excellent measures of the disk shape and flux density if we can make use of it. After a lot of effort, I believe we have an understanding of how to interpret the EOVSA visibilities in terms of the disk shape and flux density, and therefore how to fit the data to derive estimates for the three key parameters: equatorial radius, polar radius, and flux density. In practice, the flux density seems to fit quite well with the flux density imposed by the total power/auto-correlation calibration, so it is likely that we will do better just using the nominal flux density imposed from RSTN. Figure 1 shows an example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
from which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = 206265 {m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. In some comparisons with the data, I do find that there is a systematic trend for the diameter to get smaller for larger ''m''. One possibility that I explored is whether a fuzzy disk has different minima than a sharply defined disk, but this does not seem to be the case.<br />
<br />
I also investigated whether there is a dependence on the location of the minima with the angle of the baseline (which would indicate a non-circular disk). There does seem to be some evidence for that, with a rather surprisingly large variation of order 10%. Still, the sizes I am getting for the equatorial direction are something like 2100", which is pretty big, so it might make sense that the polar direction is much smaller. One would expect these variations to decrease at higher frequency, where the disk is less affected by the corona. In fact, this does seem to be the case. Figure 2 shows the same date as Fig. 1, but at higher frequencies where a single radius (968") fits all of the nulls up to at least the first 15! Note that this is still 17" larger than the photospheric value of 951" on that date. [[file:Disk_vis_high_freq.png|thumb|right|500px|'''Fig. 2:''' Example all-day visibilities for baselines 1-* at frequencies around 10 GHz or higher, showing the periodic nulls due to the solar disk. The orange lines are the predicted position of the nulls for a disk of 968".]]</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-04T03:56:56Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, but it remains too simple and will be embellished over time to create a more sophisticated disk.<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. One suggestion is to use the zeros, or nulls, evident in the data. Figure 1 shows and example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
from which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = 206265 {m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. In some comparisons with the data, I do find that there is a systematic trend for the diameter to get smaller for larger ''m''. One possibility that I explored is whether a fuzzy disk has different minima than a sharply defined disk, but this does not seem to be the case.<br />
<br />
I also investigated whether there is a dependence on the location of the minima with the angle of the baseline (which would indicate a non-circular disk). There does seem to be some evidence for that, with a rather surprisingly large variation of order 10%. Still, the sizes I am getting for the equatorial direction are something like 2100", which is pretty big, so it might make sense that the polar direction is much smaller. One would expect these variations to decrease at higher frequency, where the disk is less affected by the corona. In fact, this does seem to be the case. Figure 2 shows the same date as Fig. 1, but at higher frequencies where a single radius (968") fits all of the nulls up to at least the first 15! Note that this is still 17" larger than the photospheric value of 951" on that date. [[file:Disk_vis_high_freq.png|thumb|right|500px|'''Fig. 2:''' Example all-day visibilities for baselines 1-* at frequencies around 10 GHz or higher, showing the periodic nulls due to the solar disk. The orange lines are the predicted position of the nulls for a disk of 968".]]</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-04T03:56:13Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, but it remains too simple and will be embellished over time to create a more sophisticated disk.<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. One suggestion is to use the zeros, or nulls, evident in the data. Figure 1 shows and example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
from which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = 206265 {m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. In some comparisons with the data, I do find that there is a systematic trend for the diameter to get smaller for larger ''m''. One possibility that I explored is whether a fuzzy disk has different minima than a sharply defined disk, but this does not seem to be the case.<br />
<br />
I also investigated whether there is a dependence on the location of the minima with the angle of the baseline (which would indicate a non-circular disk). There does seem to be some evidence for that, with a rather surprisingly large variation of order 10%. Still, the sizes I am getting for the equatorial direction are something like 2100", which is pretty big, so it might make sense that the polar direction is much smaller. One would expect these variations to decrease at higher frequency, where the disk is less affected by the corona. In fact, this does seem to be the case. Figure 2 shows the same data as Fig. 1, but at higher frequencies where a single radius (968") fits all of the nulls up to at least the first 15! Note that this is still 17" larger than the photospheric value of 951" on that date. [[file:Disk_vis_high_freq.png|thumb|right|500px|'''Fig. 2:''' Example all-day visibilities for baselines 1-* at frequencies around 10 GHz or higher, showing the periodic nulls due to the solar disk. The orange lines are the predicted position of the nulls for a disk of 968".]]</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-03T21:25:05Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, but it remains too simple and will be embellished over time to create a more sophisticated disk.<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. One suggestion is to use the zeros, or nulls, evident in the data. Figure 1 shows and example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
from which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = 206265 {m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. In some comparisons with the data, I do find that there is a systematic trend for the diameter to get smaller for larger ''m''. One possibility that I explored is whether a fuzzy disk has different minima than a sharply defined disk, but this does not seem to be the case.<br />
<br />
I also investigated whether there is a dependence on the location of the minima with the angle of the baseline (which would indicate a non-circular disk). There does seem to be some evidence for that, with a rather surprisingly large variation of order 10%. Still, the sizes I am getting for the equatorial direction are something like 2100", which is pretty big, so it might make sense that the polar direction is much smaller. One would expect these variations to decrease at higher frequency, where the disk is less affected by the corona. In fact, this does seem to be the case. Figure 2 shows the same data as Fig. 1, but at higher frequencies where a single radius fits all of the nulls up to at least the first 15! [[file:Disk_vis_high_freq.png|thumb|right|500px|'''Fig. 2:''' Example all-day visibilities for baselines 1-* at frequencies around 10 GHz or higher, showing the periodic nulls due to the solar disk. The orange lines are the predicted position of the nulls for a disk of 968".]]</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/File:Disk_vis_high_freq.pngFile:Disk vis high freq.png2019-09-03T21:21:46Z<p>Dgary: </p>
<hr />
<div></div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-03T20:34:34Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, but it remains too simple and will be embellished over time to create a more sophisticated disk.<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. One suggestion is to use the zeros, or nulls, evident in the data. Figure 1 shows and example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
from which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = 206265 {m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. In some comparisons with the data, I do find that there is a systematic trend for the diameter to get smaller for larger ''m''. One possibility that I explored is whether a fuzzy disk has different minima than a sharply defined disk, but this does not seem to be the case.<br />
<br />
I also investigated whether there is a dependence on the location of the minima with the angle of the baseline (which would indicate a non-circular disk). There does seem to be some evidence for that, with a rather surprisingly large variation of order 10%. Still, the sizes I am getting for the equatorial direction are something like 2100", which is pretty big, so it might make sense that the polar direction is much smaller. One would expect these variations to decrease at higher frequency, where the disk is less affected by the corona.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-03T19:59:25Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, but it remains too simple and will be embellished over time to create a more sophisticated disk.<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. One suggestion is to use the zeros, or nulls, evident in the data. Figure 1 shows and example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]<br />
<br />
The first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
from which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = 206265 {m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. In some comparisons with the data, I do find that there is a systematic trend for the diameter to get smaller for larger ''m''. One possibility that I explored is whether a fuzzy disk has different minima than a sharply defined disk, but this does not seem to be the case.<br />
<br />
I also investigated whether there is a dependence on the location of the minima with the angle of the baseline (which would indicate a non-circular disk). There does seem to be some evidence for that, with a rather surprisingly large variation of order 10%.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-03T18:46:32Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, but it remains too simple and will be embellished over time to create a more sophisticated disk.<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. One suggestion is to use the zeros, or nulls, evident in the data. Figure 1 shows and example of some data for 2019 Sep 01. [[file:Disk_vis.png|thumb|right|500px|'''Fig. 1:''' Example all-day visibilities for baselines 1-*, showing the periodic nulls due to the solar disk.]]. <br />
<br />
The first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
from which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = 206265 {m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. In some comparisons with the data, I do find that there is a systematic trend for the diameter to get smaller for larger ''m''. One possibility that I explored is whether a fuzzy disk has different minima than a sharply defined disk, but this does not seem to be the case.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/File:Disk_vis.pngFile:Disk vis.png2019-09-03T18:42:36Z<p>Dgary: </p>
<hr />
<div></div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-03T18:36:53Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, but it remains too simple and will be embellished over time to create a more sophisticated disk.<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. One suggestion is to use the zeros, or nulls, evident in the data. The first 15 zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
from which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = 206265 {m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. In some comparisons with the data, I do find that there is a systematic trend for the diameter to get smaller for larger ''m''. One possibility that I explored is whether a fuzzy disk has different minima than a sharply defined disk, but this does not seem to be the case.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-03T18:36:06Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, but it remains too simple and will be embellished over time to create a more sophisticated disk.<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. One suggestion is to use the zeros, or nulls, evident in the data. The zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
from which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = 206265 {m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. In some comparisons with the data, I do find that there is a systematic trend for the diameter to get smaller for larger ''m''. One possibility that I explored is whether a fuzzy disk has different minima than a sharply defined disk, but this does not seem to be the case.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-03T18:35:24Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, but it remains too simple and will be embellished over time to create a more sophisticated disk.<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. One suggestion is to use the zeros, or nulls, evident in the data. The zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
from which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = {206265 m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. In some comparisons with the data, I do find that there is a systematic trend for the diameter to get smaller for larger ''m''. One possibility that I explored is whether a fuzzy disk has different minima than a sharply defined disk, but this does not seem to be the case.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-09-03T18:10:19Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, but it remains too simple and will be embellished over time to create a more sophisticated disk.<br />
<br />
One can use the data themselves to estimate the size of the disk needed to fit the data. One suggestion is to use the zeros, or nulls, evident in the data. The zeros of the airy disk are listed as:<br />
m = array([ 1.21967, 2.23313, 3.23831, 4.24106,<br />
5.24276, 6.24392, 7.24476, 8.24539,<br />
9.24589, 10.24629, 11.24662, 12.24689,<br />
13.24713, 14.24733, 15.24750])<br />
from which the diameter of the solar disk, in arcsec, is found from the location of the ''i''th minimum, in uv wavelengths <math>s_{\lambda,i}</math> as<br />
<br />
<math>D^{''} = {206265 m_i \over \pi s_{\lambda,i}}</math>.<br />
<br />
So the different minima will give different estimates and the final size would have to be some sort of mean of them. One possibility to explore is whether a fuzzy disk has different minima than a sharply defined disk.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-08-28T18:24:36Z<p>Dgary: /* Creating Disk Models in CASA */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =<br />
<br />
The first step in simulating a source is to create a "sky model," which is an image and associated descriptive header information to represent a source in the sky. CASA's [https://casaguides.nrao.edu/index.php/Simulation_Guide_Component_Lists_(CASA_5.4) componentlist] routines provide a recipe for creating different image components, including gaussian sources, point sources, disks, and so on. I created a python script diskmodel.py that eases the work of creating a disk model, but it remains too simple and will be embellished over time to create a more sophisticated disk.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-08-28T16:54:47Z<p>Dgary: </p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models in CASA =</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-08-28T16:54:09Z<p>Dgary: /* Purpose */</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provide limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models =</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Full_Disk_SimulationsFull Disk Simulations2019-08-28T16:53:58Z<p>Dgary: Created page with "= Purpose = EOVSA's 13 antennas provided limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We h..."</p>
<hr />
<div>= Purpose =<br />
EOVSA's 13 antennas provided limited uv coverage for imaging the solar disk, but the baselines do contain considerable flux, especially on shorter baselines. We have good evidence that weak features of the quiet Sun (e.g. filament channels, prominences, and weak plage areas) do show up in the data, but the variations in brightness due to the poorly sampled disk often swamp them. The correct approach to imaging such features is to model the quiet solar disk and remove it from the uv data. then clean the residual flux that contains these weaker features, and finally add the disk model back. Self calibration also requires a good source model that includes the disk.<br />
<br />
The purpose of this page is to describe the nuts and bolts of that procedure and document the performance.<br />
<br />
'''NB: Since we are just getting started with this procedure, this page will initially present a number of tests. Ultimately it should become a definitive description of the procedure without a lot of false starts.'''<br />
<br />
= Creating Disk Models =</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Expanded_Owens_Valley_Solar_ArrayExpanded Owens Valley Solar Array2019-08-28T16:45:29Z<p>Dgary: /* Using EOVSA Data */</p>
<hr />
<div>[[File:Eovsa1.png|border|text-top|800px]]<br />
<br />
<big>[http://ovsa.njit.edu/ EOVSA] (Expanded Owens Valley Solar Array) is a new solar-dedicated radio interferometer operated by the New Jersey Institute of Technology. This wiki serves as the site for EOVSA documentation. </big><br />
<br />
== EOVSA Documentation ==<br />
<br />
* <big>General</big><br />
** [[Downconversion and Frequency Tuning]]<br />
** [[Dealing with Radio Frequency Interference]]<br />
** [[Switching between 200 MHz and 300 MHz Correlator]]<br />
<br />
* <big>Computer-Network</big><br />
** [[Computing Systems]]<br />
** [[Network]]<br />
<br />
* <big>Control System</big><br />
** [[27-m Antenna Commands]]<br />
** [[Schedule Commands]]<br />
** [[Control Commands]]<br />
** [[Attenuation and Level Control]]<br />
<br />
* <big>Hardware</big><br />
** [[Hardware Overview]]<br />
** [[2.1-m Antennas]]<br />
** [[27-m Antennas]]<br />
<br />
* <big>System Software</big><br />
** [[Calibration Database]]<br />
** [[Stateframe Database]]<br />
** [[Create CASA measurement sets]]<br />
<br />
* <big>Calibration</big><br />
**[[Calibration Overview]]<br />
**[[Pointing Calibration]]<br />
**[[Total Power Calibration]]<br />
**[[System Gain Calibration]]<br />
**[[Antenna Position]] (Baseline Calibration)<br />
**[[Reference Gain Calibration]]<br />
**[[Daily Gain Calibration]]<br />
**[[Delay Calibration]]<br />
**[[Bandpass Calibration]]<br />
**[[Polarization Calibration]]<br />
**[[Calibrator Survey]]<br />
**[[Practical Calibration Tutorial]]<br />
<br />
* <big>[[Starburst]]</big><br />
<br />
== Using EOVSA Data ==<br />
* <big>Analysis Software</big><br />
** [https://github.com/suncasa/suncasa SunCASA] A wrapper around [https://casa.nrao.edu/ CASA (the Common Astronomy Software Applications package)] for synthesis imaging and visualizing solar spectral imaging data. CASA is one of the leading software tool for "supporting the data post-processing needs of the next generation of radio astronomical telescopes such as ALMA and VLA", an international effort led by the [https://public.nrao.edu/ National Radio Astronomy Observatory]. The current version of CASA uses Python (2.7) interface. More information about CASA can be found on [https://casa.nrao.edu/ NRAO's CASA website ]. Note, CASA is available ONLY on UNIX-BASED PLATFORMS (and therefore, so is SunCASA). <br />
** [https://github.com/Gelu-Nita/GSFIT GSFIT] A IDL-widget(GUI)-based spectral fitting package called gsfit, which provides a user-friendly display of EOVSA image cubes and an interface to fast fitting codes (via platform-dependent shared-object libraries). <br />
** [[Spectrogram Software]]<br />
* <big>Data Analysis Guides</big><br />
** <big>[[EOVSA Data Analysis Tutorial]]</big> at [http://rhessi18.umn.edu/ RHESSI XVIII Workshop]<br />
** [[Self-Calibrating Flare Data]] Example script and guides for self-calibrating EOVSA flare data (to be completed)<br />
<!-- ** [[Imaging]] --><br />
<!-- ** [[Flare Imaging]] --><br />
<br />
* <big>EOVSA Modeling Guide</big><br />
**[[GX Simulator]]<br />
<br />
* Other helpful links<br />
** [https://casaguides.nrao.edu CASA Guides]<br />
** [http://www.lmsal.com/solarsoft/ SolarSoft IDL]<br />
** [http://www.atnf.csiro.au/computing/software/miriad/userguide/userhtml.html Miriad Guides]<br />
** [https://sites.google.com/site/fgscodes/ Fast Gyrosynchrotron Codes (Alexey Kuznetsov's website)]<br />
** [[Basic GitHub Tutorial]]<br />
<br />
<!--* <big>[[EOVSA Imaging Workshop]]</big>--><br />
* <big>[[Full Disk Simulations]]</big><br />
* <big>[[Analyzing Pre-2017 Data]]</big><br />
<br />
== System Software ==<br />
<br />
* LabVIEW software<br />
* Python code [https://github.com/dgary50/eovsa Github repository]<br />
<br />
== Observing Log ==<br />
[[2016 November]]<br />
<br />
[[2016 December]]<br />
<br />
[[2017 January]]<br />
<br />
[[2017 February]]<br />
<br />
[[2017 March]]<br />
<br />
[[2017 April]]<br />
<br />
[[2017 May]]<br />
<br />
[[2017 June]]<br />
<br />
[[2017 July]]<br />
<br />
[[2017 August]]<br />
<br />
[[2017 September]]<br />
<br />
[[2017 October]]<br />
<br />
[[2017 November]]<br />
<br />
[[2017 December]]<br />
<br />
[[2018 January]]<br />
<br />
[[2018 February]]<br />
<br />
[[2018 March]]<br />
<br />
[[2018 April]]<br />
<br />
[[2018 July]]<br />
<br />
[[2019 March]]<br />
<br />
[[2019 April]]<br />
<br />
[[2019 May]]<br />
<br />
[[2019 June]]<br />
<br />
[[2019 July]]<br />
<br />
== Tohbans ==<br />
<br />
[[Trouble Shooting Guide]]<br />
<br />
[[Tohban Records]]<br />
<br />
== EOVSA Flare List ==<br />
<br />
See [https://docs.google.com/spreadsheets/d/1P8jHuDRF93dMflU6RMQcsJqVepD9vFkPkofV8Imj4xA/edit?usp=sharing this link] for a list of EOVSA flares as a Google Spreadsheet. <br />
<br />
[http://ovsa.njit.edu/jay/rd_db.php Another link] available at the EOVSA website. <br />
<br />
<!--<br />
<br />
{| class="wikitable" style="text-align: center; width: 1200px; height: 200px;"<br />
|-<br />
| '''Date''' || '''RHESSI Time''' || '''RHESSI Lightcurve''' || '''XSP Data''' || '''RHESSI Coverage''' || '''RHESSI Energy Range (keV)''' || '''EOVSA Time''' || '''EOVSA Image''' || '''GOES Class''' || '''Baseline Plot'''<br />
|-<br />
| Jun 3, 2017 || 21:43:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/03/hsi_20170603_212240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170603213511.png XSP] || Yes || 6 - 12 || 21:42:00 || || B2.0 || [[TBD | Error]]<br />
|-<br />
| " || 19:29:19 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/03/hsi_20170603_181420_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170603173511.png XSP] || Yes || 12 - 25 || 19:24:00 || || C2.5 || [[:File:UDB20170603173518.png | Plot]]<br />
|-<br />
| Jun 2, 2017 || 19:55:54 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/02/hsi_20170602_184040_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170602173511.png XSP] || Yes || 6 - 12 || 19:50:00 || || B7.0 || [[:File:UDB20170602173518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/02/hsi_20170602_170640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170602173511.png XSP] || No || || 17:55:00 || || C8.0 || [[:File:UDB20170602173518.png | Plot]]<br />
|-<br />
| Jun 1, 2017 || 20:27:40 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/01/hsi_20170601_190640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170601173511.png XSP] || Yes || 6 - 12 || 20:25:00 || || C1.1 || [[:File:UDB20170601173518.png | Plot]]<br />
|-<br />
| " || 18:04:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/01/hsi_20170601_173240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170601173511.png XSP] || Yes || 6 - 12 || 17:55:00 || || C1.4 || [[:File:UDB20170601173518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/01/hsi_20170601_173240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170601173511.png XSP] || No || || 17:37:00 || || C1.0 || [[:File:UDB20170601173518.png | Plot]]<br />
|-<br />
| May 31, 2017 || 23:14:45 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/31/hsi_20170531_224100_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170531213511.png XSP] || Yes || 12 - 25 || 23:12:00 || || C2.0 || [[TBD | Error]]<br />
|-<br />
| " || TBD || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/31/hsi_20170531_210700_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170531213511.png XSP] || Yes || || 22:20:00 || || B9.8 || [[TBD | Error]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/31/hsi_20170531_193240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170531205112.png XSP] || No || || 20:54:00 || || C1.3 || [[TBD | Error]]<br />
|-<br />
| May 28, 2017 || 19:28:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/28/hsi_20170528_191440_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170528173511.png XSP] || Yes || 25 - 50 || 19:27:00 || || C3.2 || [[:File:UDB20170528173520.png | Plot]]<br />
|-<br />
| May 6, 2017 || 20:07:24 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/06/hsi_20170506_194540_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170506180511.png XSP] || Yes || 6 - 12 || 20:06:49 || || B3.5 || [[:File:UDB20170506180518.png | Plot]]<br />
|-<br />
| April 21, 2017 || 17:27:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/21/hsi_20170421_170640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170421141111.png XSP] || Yes || 6 - 12 || 17:26:51 || || B4.1 || [[:File:UDB20170421141120.png | Plot]]<br />
|-<br />
| April 17, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/17/hsi_20170417_233740_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170417232011.png XSP] || No || || 23:45:51 || || B2.0 || [[:File:UDB20170417232020.png | Plot]]<br />
|-<br />
| " || 21:40:04 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/17/hsi_20170417_202920_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170417213511.png XSP] || Yes || 6 - 12 || 21:38:51 || || B7.5 || [[:File:UDB20170417213520.png | Plot]]<br />
|-<br />
| April 10, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/10/hsi_20170410_171800_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170410173511.png XSP] || No || || 18:26:51 || || B4.8 || [[:File:UDB20170410173520.png | Plot]]<br />
|-<br />
| April 8, 2017 || 23:58:40 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/08/hsi_20170408_225240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170408231011.png XSP] || Yes || 6 - 12 || 23:53:53 || || B2.3 || [[:File:UDB20170408231022.png | Plot]]<br />
|-<br />
| April 7, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/07/hsi_20170407_183600_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170407191511.png XSP] || No || || 19:47:52 || || C4.3 || [[:File:UDB20170407191521.png | Plot]]<br />
|-<br />
| " || 00:23:45 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/06/hsi_20170406_234500_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170406233011.png XSP] || Yes || 12 - 25 || 00:23:52 || || C2.7 || [[:File:UDB20170406233021.png | Plot]]<br />
|-<br />
| April 5, 2017 || 17:38:20 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/05/hsi_20170405_162020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170405173511.png XSP] || Yes || 6 - 12 || 17:38:51 || || B9.1 || [[:File:UDB20170405173520.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/05/hsi_20170405_144600_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170405142811.png XSP] || No || || 15:52:50 || || B5.9 || [[:File:UDB20170405142819.png | Plot]]<br />
|-<br />
| April 4, 2017 || 23:42:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/04/hsi_20170404_230340_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170404231511.png XSP] || Yes || 25 - 50 || 23:40:51 || [[:File:Eovsa_flare_20170404T233951.png | Image]] [http://www.ovsa.njit.edu/movies.html Movies] || C4.9 || [[:File:UDB20170404231520.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/04/hsi_20170404_182100_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170404191511.png XSP] || No || || 19:42:52 || || B9.4 || [[:File:UDB20170404191521.png | Plot]]<br />
|-<br />
| " || 00:39:35 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/03/hsi_20170403_233020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170403231511.png XSP] || Yes || 12 - 25 || 00:37:49 || || C1.4 || [[:File:UDB20170404003935.png | Plot]]<br />
|-<br />
| April 3, 2017 || 23:51:49 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/03/hsi_20170403_233020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170403231511.png XSP] || Yes || 12 - 25 || 23:44:49 || || C1.7 || [[:File:UDB20170403231518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/03/hsi_20170403_184740_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170403191511.png XSP] || No || || 20:11:53 || || C5.0 || [[:File:UDB20170403191522.png | Plot]]<br />
|-<br />
| " || 01:00:44 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_235700_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402213511.png XSP] || Yes || 25 - 50 || 01:00:52 || || M1.2 || [[:File:UDB20170402223521.png | Plot]]<br />
|-<br />
| April 2, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_191420_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402191511.png XSP] || No || || 20:31:49 || || M5.7 || [[:File:UDB20170402191518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_174020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402173511.png XSP] || No || || 18:48:52 || || M2.1 || [[:File:UDB20170402173521.png | Plot]]<br />
|-<br />
| " || 16:25:30 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_160600_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402143211.png XSP] || Yes || 12 - 25 || 16:24:50 || 6 sfu || C3.0 ||[[:File:UDB20170402143219.png | Plot]]<br />
|-<br />
| April 1, 2017 || 23:08:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_224940_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401213511.png XSP] || Yes || 12 - 25 || 23:01:51 || 49 sfu || C5.1 || [[:File:UDB20170401213520.png | Plot]]<br />
|-<br />
| " || 21:48:35 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_211520_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401213511.png XSP] || Yes || 50 - 100 || 21:46:51 || 286 sfu || M4.4 || [[:File:UDB20170401213520.png | Plot]]<br />
|-<br />
| " || 19:57:23 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_194120_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401191511.png XSP] || Yes || 12 - 25 || 19:42:50 || 15 sfu || C3.7 || [[:File:UDB20170401191519.png | Plot]]<br />
|-<br />
| " || 15:37:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_145840_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401143743.png XSP] || Yes || 12 - 25 || 15:55:21 || 2.5 sfu || B8.1 || [[:File:UDB20170401143750.png | Plot]]<br />
|-<br />
| March 31, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/03/31/hsi_20170331_214220_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170331213511.png XSP] || No || || 22:57:49 || || C1.1 || [[:File:UDB20170331213518.png | Plot]]<br />
|-<br />
| March 29, 2017 || 23:27:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/03/29/hsi_20170329_223640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170329213511.png XSP] || Yes || 6 - 12 || 23:40:50 || || B6.5 || [[:File:UDB20170329213519.png | Plot]]<br />
|-<br />
| March 25, 2017 || 23:34:44 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/03/25/hsi_20170325_225100_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170325213511.png XSP] || Yes || 6 - 12 || 23:44:50 || || B2.3 || [[:File:UDB20170327213520.png | Plot]]<br />
|-<br />
| February 23, 2017 || 20:52:00 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/02/23/hsi_20170223_202800_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170223191510.png XSP] || Yes || 6 - 12 || 20:52:50 || || C1.3 || [[:File:UDB20170223191519.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/02/23/hsi_20170223_185340_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170223191510.png XSP] || No || || 20:25:50 || || B5.8 || [[:File:UDB20170223191519.png | Plot]]<br />
|-<br />
| February 21, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/02/21/hsi_20170221_225520_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170221220653.png XSP] || No || || 22:51:10 || || B3.0 || [[:File:UDB20170221220739.png | Plot]]<br />
|-<br />
| January 25, 2017 || 21:27:14 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/01/25/hsi_20170125_204400_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170125200011.png XSP] || Yes || 12 - 25 || 21:26:57 || || B8.5 || [[:File:UDB20170125200025.png | Plot]]<br />
|}<br />
--><br />
<br />
== EOVSA Publications ==<br />
Here is a (partial) list of publications that utilizes EOVSA data.<br />
<br />
; 2018<br />
: [https://ui.adsabs.harvard.edu/#abs/2018ApJ...864...63P/abstract Polito et al. (2018), ApJ, 864, 63] Broad Non-Gaussian Fe XXIV Line Profiles in the Impulsive Phase of the 2017 September 10 X8.3-class Flare Observed by Hinode/EIS<br />
: [https://ui.adsabs.harvard.edu/#abs/2018ApJ...863...83G/abstract Gary et al. (2018), ApJ, 863, 83] Microwave and Hard X-Ray Observations of the 2017 September 10 Solar Limb Flare<br />
: [https://ui.adsabs.harvard.edu/#abs/2018ApJ...852...32K/abstract Kuroda et al. (2018), ApJ, 852, 32] Three-dimensional Forward-fit Modeling of the Hard X-ray and the Microwave Emissions of the 2015 June 22 M6.5 flare<br />
<br />
== VLA Flare List ==<br />
See [http://www.ovsa.njit.edu/wiki/index.php/VLA_Data_Survey#List_of_Jansky_VLA_Solar_Observations this link] for a list of VLA flare observations.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Analyzing_Pre-2017_DataAnalyzing Pre-2017 Data2019-07-25T19:25:20Z<p>Dgary: Created page with "= Analyzing Pre-2017 Data = Prior to April 2017, the EOVSA correlator was not fully working and so imaging is not possible. However, EOVSA still offers some unique total powe..."</p>
<hr />
<div>= Analyzing Pre-2017 Data =<br />
Prior to April 2017, the EOVSA correlator was not fully working and so imaging is not possible. However, EOVSA still offers some unique total power measurements and even some limited baseline data, albeit without proper calibration. This page describes how to proceed with that analysis on the Pipeline machine.<br />
<br />
== Accessing Analysis Software ==<br />
As a matter of keeping the software up to date, at some point the current software could no longer work with older data. This was solved on the Pipeline machine by creating a Python virtual environment containing the older software. To activate a terminal session on Pipeline that is using the appropriate Python environment, issue the following command:<br />
source ~/.virtualenvss/astropy-dev/bin/activate.csh<br />
which will result in a new prompt:<br />
[astropy-dev] pipeline:~><br />
You can now use the commands on the remainder of this page, some of which will NOT work in the standard environment.<br />
<br />
To exit this special environment, issue this command:<br />
deactivate<br />
<br />
== Total Power Analysis ==<br />
The total power can be read from IDB files using the dump_tsys module, which has the rd_miriad_tsys() routine. The call is of this form<br />
out = dump_tsys.rd_miriad_tsys(trange)<br />
which will result in</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Expanded_Owens_Valley_Solar_ArrayExpanded Owens Valley Solar Array2019-07-25T19:11:23Z<p>Dgary: /* Using EOVSA Data */</p>
<hr />
<div>[[File:Eovsa1.png|border|text-top|800px]]<br />
<br />
<big>[http://ovsa.njit.edu/ EOVSA] (Expanded Owens Valley Solar Array) is a new solar-dedicated radio interferometer operated by the New Jersey Institute of Technology. This wiki serves as the site for EOVSA documentation. </big><br />
<br />
== EOVSA Documentation ==<br />
<br />
* <big>General</big><br />
** [[Downconversion and Frequency Tuning]]<br />
** [[Dealing with Radio Frequency Interference]]<br />
** [[Switching between 200 MHz and 300 MHz Correlator]]<br />
<br />
* <big>Computer-Network</big><br />
** [[Computing Systems]]<br />
** [[Network]]<br />
<br />
* <big>Control System</big><br />
** [[27-m Antenna Commands]]<br />
** [[Schedule Commands]]<br />
** [[Control Commands]]<br />
** [[Attenuation and Level Control]]<br />
<br />
* <big>Hardware</big><br />
** [[Hardware Overview]]<br />
** [[2.1-m Antennas]]<br />
** [[27-m Antennas]]<br />
<br />
* <big>System Software</big><br />
** [[Calibration Database]]<br />
** [[Stateframe Database]]<br />
** [[Create CASA measurement sets]]<br />
<br />
* <big>Calibration</big><br />
**[[Calibration Overview]]<br />
**[[Pointing Calibration]]<br />
**[[Total Power Calibration]]<br />
**[[System Gain Calibration]]<br />
**[[Antenna Position]] (Baseline Calibration)<br />
**[[Reference Gain Calibration]]<br />
**[[Daily Gain Calibration]]<br />
**[[Delay Calibration]]<br />
**[[Bandpass Calibration]]<br />
**[[Polarization Calibration]]<br />
**[[Calibrator Survey]]<br />
**[[Practical Calibration Tutorial]]<br />
<br />
* <big>[[Starburst]]</big><br />
<br />
== Using EOVSA Data ==<br />
* <big>Analysis Software</big><br />
** [https://github.com/suncasa/suncasa SunCASA] A wrapper around [https://casa.nrao.edu/ CASA (the Common Astronomy Software Applications package)] for synthesis imaging and visualizing solar spectral imaging data. CASA is one of the leading software tool for "supporting the data post-processing needs of the next generation of radio astronomical telescopes such as ALMA and VLA", an international effort led by the [https://public.nrao.edu/ National Radio Astronomy Observatory]. The current version of CASA uses Python (2.7) interface. More information about CASA can be found on [https://casa.nrao.edu/ NRAO's CASA website ]. Note, CASA is available ONLY on UNIX-BASED PLATFORMS (and therefore, so is SunCASA). <br />
** [https://github.com/Gelu-Nita/GSFIT GSFIT] A IDL-widget(GUI)-based spectral fitting package called gsfit, which provides a user-friendly display of EOVSA image cubes and an interface to fast fitting codes (via platform-dependent shared-object libraries). <br />
** [[Spectrogram Software]]<br />
* <big>Data Analysis Guides</big><br />
** <big>[[EOVSA Data Analysis Tutorial]]</big> at [http://rhessi18.umn.edu/ RHESSI XVIII Workshop]<br />
** [[Self-Calibrating Flare Data]] Example script and guides for self-calibrating EOVSA flare data (to be completed)<br />
<!-- ** [[Imaging]] --><br />
<!-- ** [[Flare Imaging]] --><br />
<br />
* <big>EOVSA Modeling Guide</big><br />
**[[GX Simulator]]<br />
<br />
* Other helpful links<br />
** [https://casaguides.nrao.edu CASA Guides]<br />
** [http://www.lmsal.com/solarsoft/ SolarSoft IDL]<br />
** [http://www.atnf.csiro.au/computing/software/miriad/userguide/userhtml.html Miriad Guides]<br />
** [https://sites.google.com/site/fgscodes/ Fast Gyrosynchrotron Codes (Alexey Kuznetsov's website)]<br />
** [[Basic GitHub Tutorial]]<br />
<br />
<!--* <big>[[EOVSA Imaging Workshop]]</big>--><br />
* <big>[[Analyzing Pre-2017 Data]]</big><br />
<br />
== System Software ==<br />
<br />
* LabVIEW software<br />
* Python code [https://github.com/dgary50/eovsa Github repository]<br />
<br />
== Observing Log ==<br />
[[2016 November]]<br />
<br />
[[2016 December]]<br />
<br />
[[2017 January]]<br />
<br />
[[2017 February]]<br />
<br />
[[2017 March]]<br />
<br />
[[2017 April]]<br />
<br />
[[2017 May]]<br />
<br />
[[2017 June]]<br />
<br />
[[2017 July]]<br />
<br />
[[2017 August]]<br />
<br />
[[2017 September]]<br />
<br />
[[2017 October]]<br />
<br />
[[2017 November]]<br />
<br />
[[2017 December]]<br />
<br />
[[2018 January]]<br />
<br />
[[2018 February]]<br />
<br />
[[2018 March]]<br />
<br />
[[2018 April]]<br />
<br />
[[2018 July]]<br />
<br />
[[2019 March]]<br />
<br />
[[2019 April]]<br />
<br />
[[2019 May]]<br />
<br />
[[2019 June]]<br />
<br />
[[2019 July]]<br />
<br />
== Tohbans ==<br />
<br />
[[Trouble Shooting Guide]]<br />
<br />
[[Tohban Records]]<br />
<br />
== EOVSA Flare List ==<br />
<br />
See [https://docs.google.com/spreadsheets/d/1P8jHuDRF93dMflU6RMQcsJqVepD9vFkPkofV8Imj4xA/edit?usp=sharing this link] for a list of EOVSA flares as a Google Spreadsheet. <br />
<br />
[http://ovsa.njit.edu/jay/rd_db.php Another link] available at the EOVSA website. <br />
<br />
<!--<br />
<br />
{| class="wikitable" style="text-align: center; width: 1200px; height: 200px;"<br />
|-<br />
| '''Date''' || '''RHESSI Time''' || '''RHESSI Lightcurve''' || '''XSP Data''' || '''RHESSI Coverage''' || '''RHESSI Energy Range (keV)''' || '''EOVSA Time''' || '''EOVSA Image''' || '''GOES Class''' || '''Baseline Plot'''<br />
|-<br />
| Jun 3, 2017 || 21:43:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/03/hsi_20170603_212240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170603213511.png XSP] || Yes || 6 - 12 || 21:42:00 || || B2.0 || [[TBD | Error]]<br />
|-<br />
| " || 19:29:19 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/03/hsi_20170603_181420_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170603173511.png XSP] || Yes || 12 - 25 || 19:24:00 || || C2.5 || [[:File:UDB20170603173518.png | Plot]]<br />
|-<br />
| Jun 2, 2017 || 19:55:54 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/02/hsi_20170602_184040_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170602173511.png XSP] || Yes || 6 - 12 || 19:50:00 || || B7.0 || [[:File:UDB20170602173518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/02/hsi_20170602_170640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170602173511.png XSP] || No || || 17:55:00 || || C8.0 || [[:File:UDB20170602173518.png | Plot]]<br />
|-<br />
| Jun 1, 2017 || 20:27:40 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/01/hsi_20170601_190640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170601173511.png XSP] || Yes || 6 - 12 || 20:25:00 || || C1.1 || [[:File:UDB20170601173518.png | Plot]]<br />
|-<br />
| " || 18:04:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/01/hsi_20170601_173240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170601173511.png XSP] || Yes || 6 - 12 || 17:55:00 || || C1.4 || [[:File:UDB20170601173518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/01/hsi_20170601_173240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170601173511.png XSP] || No || || 17:37:00 || || C1.0 || [[:File:UDB20170601173518.png | Plot]]<br />
|-<br />
| May 31, 2017 || 23:14:45 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/31/hsi_20170531_224100_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170531213511.png XSP] || Yes || 12 - 25 || 23:12:00 || || C2.0 || [[TBD | Error]]<br />
|-<br />
| " || TBD || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/31/hsi_20170531_210700_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170531213511.png XSP] || Yes || || 22:20:00 || || B9.8 || [[TBD | Error]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/31/hsi_20170531_193240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170531205112.png XSP] || No || || 20:54:00 || || C1.3 || [[TBD | Error]]<br />
|-<br />
| May 28, 2017 || 19:28:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/28/hsi_20170528_191440_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170528173511.png XSP] || Yes || 25 - 50 || 19:27:00 || || C3.2 || [[:File:UDB20170528173520.png | Plot]]<br />
|-<br />
| May 6, 2017 || 20:07:24 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/06/hsi_20170506_194540_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170506180511.png XSP] || Yes || 6 - 12 || 20:06:49 || || B3.5 || [[:File:UDB20170506180518.png | Plot]]<br />
|-<br />
| April 21, 2017 || 17:27:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/21/hsi_20170421_170640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170421141111.png XSP] || Yes || 6 - 12 || 17:26:51 || || B4.1 || [[:File:UDB20170421141120.png | Plot]]<br />
|-<br />
| April 17, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/17/hsi_20170417_233740_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170417232011.png XSP] || No || || 23:45:51 || || B2.0 || [[:File:UDB20170417232020.png | Plot]]<br />
|-<br />
| " || 21:40:04 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/17/hsi_20170417_202920_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170417213511.png XSP] || Yes || 6 - 12 || 21:38:51 || || B7.5 || [[:File:UDB20170417213520.png | Plot]]<br />
|-<br />
| April 10, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/10/hsi_20170410_171800_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170410173511.png XSP] || No || || 18:26:51 || || B4.8 || [[:File:UDB20170410173520.png | Plot]]<br />
|-<br />
| April 8, 2017 || 23:58:40 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/08/hsi_20170408_225240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170408231011.png XSP] || Yes || 6 - 12 || 23:53:53 || || B2.3 || [[:File:UDB20170408231022.png | Plot]]<br />
|-<br />
| April 7, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/07/hsi_20170407_183600_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170407191511.png XSP] || No || || 19:47:52 || || C4.3 || [[:File:UDB20170407191521.png | Plot]]<br />
|-<br />
| " || 00:23:45 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/06/hsi_20170406_234500_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170406233011.png XSP] || Yes || 12 - 25 || 00:23:52 || || C2.7 || [[:File:UDB20170406233021.png | Plot]]<br />
|-<br />
| April 5, 2017 || 17:38:20 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/05/hsi_20170405_162020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170405173511.png XSP] || Yes || 6 - 12 || 17:38:51 || || B9.1 || [[:File:UDB20170405173520.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/05/hsi_20170405_144600_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170405142811.png XSP] || No || || 15:52:50 || || B5.9 || [[:File:UDB20170405142819.png | Plot]]<br />
|-<br />
| April 4, 2017 || 23:42:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/04/hsi_20170404_230340_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170404231511.png XSP] || Yes || 25 - 50 || 23:40:51 || [[:File:Eovsa_flare_20170404T233951.png | Image]] [http://www.ovsa.njit.edu/movies.html Movies] || C4.9 || [[:File:UDB20170404231520.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/04/hsi_20170404_182100_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170404191511.png XSP] || No || || 19:42:52 || || B9.4 || [[:File:UDB20170404191521.png | Plot]]<br />
|-<br />
| " || 00:39:35 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/03/hsi_20170403_233020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170403231511.png XSP] || Yes || 12 - 25 || 00:37:49 || || C1.4 || [[:File:UDB20170404003935.png | Plot]]<br />
|-<br />
| April 3, 2017 || 23:51:49 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/03/hsi_20170403_233020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170403231511.png XSP] || Yes || 12 - 25 || 23:44:49 || || C1.7 || [[:File:UDB20170403231518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/03/hsi_20170403_184740_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170403191511.png XSP] || No || || 20:11:53 || || C5.0 || [[:File:UDB20170403191522.png | Plot]]<br />
|-<br />
| " || 01:00:44 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_235700_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402213511.png XSP] || Yes || 25 - 50 || 01:00:52 || || M1.2 || [[:File:UDB20170402223521.png | Plot]]<br />
|-<br />
| April 2, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_191420_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402191511.png XSP] || No || || 20:31:49 || || M5.7 || [[:File:UDB20170402191518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_174020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402173511.png XSP] || No || || 18:48:52 || || M2.1 || [[:File:UDB20170402173521.png | Plot]]<br />
|-<br />
| " || 16:25:30 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_160600_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402143211.png XSP] || Yes || 12 - 25 || 16:24:50 || 6 sfu || C3.0 ||[[:File:UDB20170402143219.png | Plot]]<br />
|-<br />
| April 1, 2017 || 23:08:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_224940_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401213511.png XSP] || Yes || 12 - 25 || 23:01:51 || 49 sfu || C5.1 || [[:File:UDB20170401213520.png | Plot]]<br />
|-<br />
| " || 21:48:35 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_211520_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401213511.png XSP] || Yes || 50 - 100 || 21:46:51 || 286 sfu || M4.4 || [[:File:UDB20170401213520.png | Plot]]<br />
|-<br />
| " || 19:57:23 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_194120_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401191511.png XSP] || Yes || 12 - 25 || 19:42:50 || 15 sfu || C3.7 || [[:File:UDB20170401191519.png | Plot]]<br />
|-<br />
| " || 15:37:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_145840_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401143743.png XSP] || Yes || 12 - 25 || 15:55:21 || 2.5 sfu || B8.1 || [[:File:UDB20170401143750.png | Plot]]<br />
|-<br />
| March 31, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/03/31/hsi_20170331_214220_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170331213511.png XSP] || No || || 22:57:49 || || C1.1 || [[:File:UDB20170331213518.png | Plot]]<br />
|-<br />
| March 29, 2017 || 23:27:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/03/29/hsi_20170329_223640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170329213511.png XSP] || Yes || 6 - 12 || 23:40:50 || || B6.5 || [[:File:UDB20170329213519.png | Plot]]<br />
|-<br />
| March 25, 2017 || 23:34:44 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/03/25/hsi_20170325_225100_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170325213511.png XSP] || Yes || 6 - 12 || 23:44:50 || || B2.3 || [[:File:UDB20170327213520.png | Plot]]<br />
|-<br />
| February 23, 2017 || 20:52:00 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/02/23/hsi_20170223_202800_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170223191510.png XSP] || Yes || 6 - 12 || 20:52:50 || || C1.3 || [[:File:UDB20170223191519.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/02/23/hsi_20170223_185340_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170223191510.png XSP] || No || || 20:25:50 || || B5.8 || [[:File:UDB20170223191519.png | Plot]]<br />
|-<br />
| February 21, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/02/21/hsi_20170221_225520_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170221220653.png XSP] || No || || 22:51:10 || || B3.0 || [[:File:UDB20170221220739.png | Plot]]<br />
|-<br />
| January 25, 2017 || 21:27:14 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/01/25/hsi_20170125_204400_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170125200011.png XSP] || Yes || 12 - 25 || 21:26:57 || || B8.5 || [[:File:UDB20170125200025.png | Plot]]<br />
|}<br />
--><br />
<br />
== EOVSA Publications ==<br />
Here is a (partial) list of publications that utilizes EOVSA data.<br />
<br />
; 2018<br />
: [https://ui.adsabs.harvard.edu/#abs/2018ApJ...864...63P/abstract Polito et al. (2018), ApJ, 864, 63] Broad Non-Gaussian Fe XXIV Line Profiles in the Impulsive Phase of the 2017 September 10 X8.3-class Flare Observed by Hinode/EIS<br />
: [https://ui.adsabs.harvard.edu/#abs/2018ApJ...863...83G/abstract Gary et al. (2018), ApJ, 863, 83] Microwave and Hard X-Ray Observations of the 2017 September 10 Solar Limb Flare<br />
: [https://ui.adsabs.harvard.edu/#abs/2018ApJ...852...32K/abstract Kuroda et al. (2018), ApJ, 852, 32] Three-dimensional Forward-fit Modeling of the Hard X-ray and the Microwave Emissions of the 2015 June 22 M6.5 flare<br />
<br />
== VLA Flare List ==<br />
See [http://www.ovsa.njit.edu/wiki/index.php/VLA_Data_Survey#List_of_Jansky_VLA_Solar_Observations this link] for a list of VLA flare observations.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/2019_July2019 July2019-07-23T13:00:22Z<p>Dgary: /* July 02 */</p>
<hr />
<div>[http://www.ovsa.njit.edu/wiki/index.php/Expanded_Owens_Valley_Solar_Array#Observing_Log Back to Observing Log]<br />
<br />
== July 02 ==<br />
'''12:14 UT''' Today I set all of the antennas (1-13) back to their old pointing coefficients from before I did the pointing measurements. For whatever reason, the update to the pointing measurements was incorrect and nothing I did could make the updated pointing coefficients work. I really do not understand why the update should fail. I suppose I need to go through the algorithm in detail and see if there is a bug somewhere. I did leave the 27-m (ant14) pointing coefficients updated, although I do not have a good way to check how good they are.<br />
<br />
== July 23 ==<br />
'''12:54 UT''' Yesterday I attempted to reset the DCM master table, but I messed up the command and the result was a table that was very much mis-set. Probably yesterday's data are not much good. This morning I figured out what went wrong, and reset it to correct values in time for the morning high-frequency receiver calibration. I am trying to determine the EQ coefficients and come up with a scheme for setting them quickly and automatically, and to allow them to vary with antenna and band.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Computing_SystemsComputing Systems2019-07-16T13:02:18Z<p>Dgary: /* Helios */</p>
<hr />
<div>== Overview of EOVSA Computing Systems ==<br />
The computing infrastructure for EOVSA has been developed largely according to the initial plan, which calls for multiple computers dedicated to specific tasks. This document describes the computing hardware specifications and capabilities, role of each computer in the system, and interdependences among them. A schematic summary is shown in Figure 1.<br />
[[File:Figure 1.png|none|Figure 1]]<br />
<br />
== Specifications of Each System ==<br />
In this section we detail the system specifications, for reference purposes. The systems are listed alphabetically by name.<br />
=== acc.solar.pvt (192.168.24.10) ===<br />
The Array Control Computer (ACC) is a National Instruments (NI) PXIe-1071 4-slot chassis with three units installed: NI PXIe-8133 real-time controller, NI PXI-6682H timing module, and NI PXI-8431 RS485 module. The relevant features of acc are:<br />
* Real-Time PharLap operating system running Labview RT<br />
* Quad-core i7-820QM, 1.73 GHz processor with 4 GB RAM<br />
* High-bandwidth PXI express embedded controller<br />
* Two 10/100/1000BASE-TX (gigabit) Ethernet – one port is connected to the LAN, and the other is dedicated to the Hitite synthesizer.<br />
* 4 hi-speed USB, GPIB, RS-232 serial, 120 GB HDD<br />
* Rackmount 4U Chassis<br />
=== dpp.solar.pvt (192.168.24.100) ===<br />
The Data Packaging Processor (DPP) is a Silicon Mechanics Rackform nServ A331.v3 computer with 32 cores. The main features of dpp are:<br />
* Two Opteron 6276 CPUs (2.3 GHz, 16-core, G34, 16 MB L3 Cache)<br />
* 64 GB RAM, 1600 MHz<br />
* Intel 82576 dual-port Gigabit ethernet controller<br />
* Two 2-TB Seagate Constellation SATA HDD<br />
* Myricom 10G-PCIE2-8B-2S dual-port 10-GB Ethernet controller with SFP+ interface<br />
* Ubuntu 18.04 LTS (long-term support) 64-bit operating system (updated 22 October 2018)<br />
* Rackmount 1U Chassis<br />
<br />
=== helios.solar.pvt (192.168.24.103) ===<br />
The Scheduling and Fault System computer is a Dell Precision T3600 computer. The main features of helios are:<br />
* Intel Xeon E5-1603 CPU (4-core, 2.8 GHz, 10 MB L3 Cache)<br />
* 8 GB RAM, 1600 MHz<br />
* Two 2-TB SATA HDD<br />
* 512 MB NVidia Quadro NVS 310 dual-monitor graphics adapter<br />
* Dell Ultrasharp U2312H 23-inch monitor<br />
* 16xDVD+/-RW optical DVD drive<br />
* Integrated gigabit Ethernet controller<br />
* Ubuntu 18.04 LTS (long-term support) 64-bit operating system (updated 2018 Sep 18)<br />
* Free-standing Tower Chassis<br />
<br />
=== ovsa.njit.edu (192.100.16.206) ===<br />
The Gateway/Web Server is an older machine (ordered 1/12/2011). The DELL service tag number is FHDD8P1. The main features of ovsa are:<br />
* Intel Xeon W3505 (2.53 GHz, dual-core, 8 MB L3 Cache)<br />
* 4 GB RAM, 1333 MHz<br />
* 250 GB SATA HDD (operating system) + 250 GB data disk + 500 GB archive disk<br />
* 512 MB Nvidia Quadro FX580 dual-monitor graphics adapter<br />
* Dell U2211H 21.5-inch monitor<br />
* 16X DVD+/-RW optical DVD drive<br />
* Broadcom NetXtreme 57xx Gigabit Ethernet controller<br />
* Ubuntu 16.04 LTS (long-term support) 32-bit operating system (updated 2018 Sep 18)<br />
* Free-standing Tower Chassis<br />
<br />
=== pipeline.solar.pvt (192.168.24.104) ===<br />
The Pipeline Computer is a Silicon Mechanics Storform iServ R513.v4 computer with 20 cores. Pipeline is the most powerful of the EOVSA computers, and is meant to handle the real-time creation of data products. The main features of pipeline are:<br />
* Two Intel Xeon E5-2660v2 CPUs (2.2 GHz, 10-core, 25 MB L3 Cache)<br />
* 128 GB RAM, 1600 MHz<br />
* Integrated dual-port Gigabit Ethernet controller<br />
* LP PCIe 3.0 x16 SAS/SATA controller<br />
* Twelve 4-TB Seagate Constellation SATA HDD configured as RAID 6 with hot spare (36 TB usable)<br />
* Ubuntu 18.04 LTS (long-term support) Server Edition 64-bit operating system (updated 2018 Sep 18)<br />
* Rackmount 2U Chassis<br />
<br />
=== roach{n}.solar.pvt (192.168.24.12{n}) ===<br />
The eight ROACH2 boards each have a Power-PC CPU, with hostnames are roach1.solar.pvt, roach2.solar.pvt, … roach8.solar.pvt. The receive their operating systems via NFS netboot from the Gateway/Web Server computer ovsa.njit.edu. The CPUs on the roaches are mainly for interacting with the on-board FPGAs, which are programmed to run the correlator design.<br />
=== tawa.solar.pvt (192.168.24.105) ===<br />
The Analysis Computer is essentially a clone of the DPP Computer, a Silicon Mechanics Rackform nServ A331.v4 computer with 32 cores. The main features of tawa are:<br />
* Two Opteron 6276 CPUs (2.3 GHz, 16-core, G34, 16 MB L3 Cache)<br />
* 64 GB RAM, 1600 MHz<br />
* Intel 82576 dual-port Gigabit ethernet controller<br />
* Two 2-TB Seagate Constellation SATA HDD, in a RAID1 configuration.<br />
* DVD+/-RW optical DVD drive<br />
* Ubuntu 12.04 LTS (long-term support) Server Edition 64-bit operating system<br />
* Rackmount 1U Chassis<br />
=== sqlserver.solar.pvt (192.168.24.106) ===<br />
The RDBMS computer is a Dell PowerEdge R520. It records the main monitor database, which can be queried using standard SQL queries. The computer was installed on 9/25/2014, and runs the Windows Server 2008 R2 Standard operating system. The main features of sqlserver are:<br />
* Intel Xeon E5-2420 CPU (1.9 GHz, 6-core, 15MB Cache)<br />
* 32 GB RAM, 1600 MHz<br />
* Broadcom 5720 Quad-port Gigabit Ethernet controller<br />
* Four 4-TB SAS Western Digital WD-4001FYYG HDD, in a RAID5 configuration (12 TB usable)<br />
* iDRAC remote management card (IP address 192.168.24.108)<br />
* Read-only optical DVD drive <br />
* SQL Server 2012 Developer software<br />
=== win1.solar.pvt (192.168.24.101) ===<br />
The Windows PC is a clone of the Gateway/Web Server, purchased around the same time (ordered 5/19/2011), but runs the Windows 7 operating system, mainly for the purpose of running Windows-only utilities including monitoring the ACC and the antennas. The DELL service tag number is C303GQ1. The main features of win1 are:<br />
* Intel Xeon W3530 (2.8 GHz, dual-core, 8 MB L3 Cache)<br />
* 4 GB RAM, 1333 MHz<br />
* Two 500 GB SATA HDD<br />
* 512 MB Nvidia Quadro FX580 dual-monitor graphics adapter<br />
* Dell U2211H 21.5-inch monitor<br />
* 16X DVD+/-RW optical DVD drive<br />
* Broadcom NetXtreme 57xx Gigabit Ethernet controller<br />
* Windows 7, 32-bit operating system<br />
* Free-standing Tower Chassis<br />
<br />
== Function/Purpose of Computers ==<br />
In the cases of the acc, dpp, pipeline, and sqlserver, the system host name is suggestive of the main purpose of the computer, while helios and tawa are the names of mythological Sun gods. This section gives a somewhat detailed description of the function of each system.<br />
=== ovsa.njit.edu ===<br />
This is the web server and gateway computer, which is the only one that is on the Wide-Area-Network (WAN). It has no other function in the overall system except to permit outside users to connect to the private network (LAN) by tunneling. To gain access to machines on the private network, it is necessary to log in to ovsa via ssh, at the same time declaring a tunnel that specifically opens a relevant port through to the desired machine on the LAN. The protocol is to issue a command like:<br /><br />
ssh -L <local port>:<host>.solar.pvt:<desired port> <user>@ovsa.njit.edu<br />
where <host> is the host name of the machine to tunnel to, on the solar.pvt LAN, <desired port> is the port you wish to reach, and <user> is the name of a user account on ovsa.njit.edu that you will use to log in and create the tunnel. The <local port> is the port you will connect to from your local machine. Once you have issued the above command and logged in, you must open a second connection to localhost:<local port>. As a concrete example, say I wish to log in to helios as user sched via ssh. I would issue the command<br /><br />
ssh –L 22:helios.solar.pvt:22 dgary@ovsa.njit.edu<br />
where 22 is the usual ssh port. I would then (in a second window) issue the command<br /><br />
ssh sched@localhost<br />
which defaults to port 22 since I am using ssh. Another example is to set up for a VNC connection to the dpp. For that, I would issue the command<br /><br />
ssh –L 5902:dpp.solar.pvt:5900 dgary@ovsa.njit.edu<br />
and then open a VNC connection to localhost:2 (VNC defaults to adding 5900 to the port number, hence this would connect via port 5902). For users of the Windows operating system, I suggest the use of the excellent MobaXterm (http://mobaxterm.mobatek.net/), which allows such tunnels to be set up and saved, then executed as a one-click operation.<br />
<br />
=== acc.solar.pvt ===<br />
As its name implies, this is the Array Control Computer (ACC), which runs the supervisory LabVIEW code to communicate with the cRIO computer systems in each antenna, and to assemble and serve the 1-s stateframe. It provides a dedicated TCP/IP port (6341) from which each subsystem can connect and read stateframes of various “age” from the history buffer, from 0-9 seconds old. Stateframe 0 is the incomplete one being filled for the current second. It also supplies ports for the ACC to receive stateframe information from the schedule (port 6340) and the DPP (port 6344) for adding to the stateframe. In addition to talking to the cRIOs, it also controls the Hittite LO system, the subarray switches in the LO Distribution Module (LODM), and the downconverter modules (DCMs).<br />
=== helios.solar.pvt ===<br />
This machine has several functions. It is the computer that runs the schedule (for definiteness, only one schedule is allowed to run at a time), and as such it also must control the ROACHes at several time cadences. It initializes them on startup, it sends information about the frequency sequence once per scan, and it sends integer delays once per second. It also creates the scan_header data file to the ACC, which is read by the DPP at the start of each scan. And it creates and sends schedule information, including the exact delays, to the ACC for inclusion in the stateframe. The schedule is meant to run the same general sequence of commands every day automatically, so that unless some special configuration of the system is needed (such as for non-regular calibrations or system tests) it will continue to run for multiple days without intervention. It also must support all scheduling of calibration observations, but at least for now this is expected to be done semi-manually. The schedule may also eventually play a role in the monitor RDBMS, but this has not been fully defined as yet.<br /><br />
Another important function of helios is to run the fault system supervisory program. As of this writing, the fault system has not been implemented, but when functional it will examine the contents of the stateframe and create a parallel array of flags indicating problems, which various systems can examine and decide whether to alert the operator or take some corrective action.<br /><br />
Finally, helios also runs a version of the operator display (called sf_display for stateframe display), which can also run additional copies as desired on other machines. It currently works fine on external machines running either Linux or Windows, when they are properly set up for tunneling through ovsa.njit.edu.<br /><br />
It is anticipated that helios will have only a single user account, called sched.<br />
<br />
=== dpp.solar.pvt ===<br />
As the name implies, this is the Data Packaging Processor (DPP), which receives the raw 10-GBe UDP data packets from the ROACHes, processes them, and outputs the “interim database” Miriad files. It has a total of 4 TB of hard disk, which is sufficient for about 1 month of interim data under full EOVSA operation. If the interim data are to be kept, they will have to be transferred to the pipeline machine, and eventually to NJIT. The DPP sends its subsystem information to the ACC, which adds it to the stateframe. The DPP also reads the stateframe as well as the scan_header file and various calibration files residing on the ACC, in order to do the correct processing of the data into Miriad files. Once the interim data have been produced (each file containing roughly 2 minutes of data), control should pass to the Pipeline machine (pipeline.solar.pvt) for further processing into archival data bases as well as real-time data products. It will be necessary to use nfs to mount the dpp.solar.pvt disks on pipeline.solar.pvt. We should avoid nfs-mounting pipeline disks on dpp, however, so that the data-taking is not compromised when pipeline is down for some reason.<br /><br />
It is anticipated that dpp will have only a single user account, called user.<br />
<br />
=== sqlserver.solar.pvt ===<br />
This is the RDBMS computer responsible for recording and serving the monitor database (scan header and stateframe information). Its programming allows it to adapt seamlessly to multiple versions of the stateframe. Explicit methods of accessing, storing and querying data from Python are documented in Python_Access_to_Database.pdf. The main purpose of the database is to provide historical information about the state of the system for engineering purposes, and tracking down problems in the system. It is anticipated that certain summary web pages will be created that perform standard queries to provide an overview of the state of the system. <br />
=== pipeline.solar.pvt ===<br />
This is the Pipeline computer responsible for real-time processing of the interim data, which has several purposes: (1) To provide a continuous indication of the quality of the data and the state of solar activity by permitting display of light-curves, spectra, and images. These data-quality indicators should be put directly onto the web for public access, as well as for use as an operator console. (2) To generate the metadata and near-real-time data products, including ultimately coronal magnetic field maps and other parameters. These data products will go into a searchable database accessible via Dominic Zarro’s system. (3) To process certain daily calibration observations in an automatic way so that the results are available to the DPP for applying to the next scan’s interim data. (4) To process and/or reprocess the interim database into final archival databases, which will form the standard uv data to be used by scientists to create and analyze their own (non-real-time) data products. (5) In the case that the archival database calibration is somehow better than that available for the real-time processing, off-line reprocessing and recreation of the real-time data products will be done for archival purposes.<br /><br />
The 32 TB of RAID disk storage that are available on pipeline.solar.pvt should provide enough space for at least a year of interim data, and several years of archival data, depending on the as yet unknown data volume of the metadata and data products. A second copy of the archival data will be kept at NJIT, and possibly other sites.<br /><br />
Pipeline currently has only one user account, called user. It may have additional accounts for a few individuals who are responsible for developing pipeline’s software, such as Jim McTiernan and Stephen White.<br />
<br />
=== tawa.solar.pvt ===<br />
This is a general-purpose analysis machine, meant to provide for analysis capability that is off the critical data path. It should have access to pipeline’s disks via nfs, and should be capable of doing all of the Pipeline analysis as well as other tasks. <br /><br />
Tawa, as a general-purpose machine can have multiple user accounts. It may ultimately have a general guest account so that outside users can process limited jobs locally without having the full suite of software on their own computer.<br />
<br />
== Interdependencies of the EOVSA Computers ==<br />
In order for the various computers in the system to do their jobs effectively, there are certain interdependencies that are built in to the infrastructure. It is important to clarify these and make sure that they are as limited as possible in order to allow the system to function when non-critical infrastructure is down for maintenance or repair.<br /><br />
[[File:Figure 2.png|none]]<br />
The above table summarizes the interdependencies of the computers in the EOVSA system. The critical computers for control of the system are acc, helios and dpp. Basic control and data-taking are not possible without these three machines being operational. Still important, but not critical, computers (i.e. not single-point failures) are the cRIOs in the antennas, the antenna controllers themselves, and the ROACH boards. In general, the system should be capable of operating without one or more of these systems. Note that losing a ROACH board means that the two antennas input to that board are lost, and additionally all frequency channels (1/8th of each 500-MHz band) handled by that board’s X-engine are also lost. In terms of impact, then, an individual ROACH board is more critical than an individual cRIO or antenna controller. Next in importance is pipeline, which has a role not only in real-time data products and interim-to-archival processing, but also in certain near-real-time calibration processing. Although the interim database can be created without pipeline, the quality of the interim database may be compromised. Finally, tawa is off the critical path and should not generally affect any aspect of data-taking or data-processing. The gateway/web server, ovsa, could be critical in providing access to the private network from outside, although in case of ovsa failure tawa could be pressed into service by merely plugging it into the WAN and possibly making some changes in the DHCP/firewall settings at the site. It may be worthwhile to put into place preparations (and advance testing) to make this option as quick and easy as possible.<br /><br />
Certain disks should be accessible via nfs from multiple machines, but care should be taken with nfs to avoid loss of a non-critical machine causing problems with a critical one. Therefore, unless otherwise required, the cross-mounting should be limited to pipeline having r/o access to dpp, and tawa having r/o access to dpp and pipeline.<br />
<br />
== Troubleshooting Tips ==<br />
We have had problems when replacing computers or motherboards, but using existing boot disks from the old machine. The new machine does not configure its network, due to a file that is on the existing boot disk. Here is a statement of the problem:<br />
<pre><br />
On ubuntu, /etc/udev/rules.d/70-persistent-net.rules maps mac addresses to ethernet interface names. <br />
It will be created if it doesn't exist or appended to if it does exist. If it gets appended to then <br />
you end up with ethN where N is the number of ethernet interfaces on the machine from whence the disk <br />
came. Renaming the file to have a ".old" extension and rebooting will cause it to be recreated <br />
starting with eth0. </pre><br />
<br />
All that is needed is to rename this file so that a file of that name does not exist, then reboot. The system will automatically create the file correctly, and all will be well.<br />
<br />
== Rebooting the Computers ==<br />
For all of the Linux computers running Ubuntu, the command to reboot is<br />
<pre><br />
sudo reboot now<br />
</pre><br />
which will immediately start the reboot process. After a few seconds, your remote connection will be interrupted and you will have to wait for the boot process to complete (up to 5 minutes) before reconnecting.<br />
<br />
If multiple machines are to be rebooted, the computers should be rebooted in the following order: ''Helios'', ''DPP'', ''Pipeline'', ''Tawa'', ''Ovsa''. This ensures that the remote-mounted disk connections will be reestablished. <br />
<br />
When some systems are rebooted, they do not necessarily restart all required software (although they should, and the problems should be researched and fixed). Currently, the following exceptions are needed:<br />
<br />
=== Helios ===<br />
Whenever Helios is rebooted, its IDL system has to be manually loaded. Because it is the server of IDL on all machines, it is extremely important to restart it. To restart it, type the following at a terminal command prompt:<br />
<pre><br />
sudo /etc/init.d/sys5_idl_lmgrd start<br />
</pre><br />
Also, the sched dropbox server does not automatically restart. To restart it, type the following at a terminal command prompt:<br />
<pre><br />
python /home/sched/Downloads/dropbox.py start<br />
</pre><br />
After a reboot, the schedule has been seen to act strangely (time not updating smoothly). This was traced to the chronyd (time) service not starting correctly. To check it, type<br />
chronyc sourcestats<br />
which should return something like this:<br />
210 Number of sources = 6<br />
Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev<br />
==============================================================================<br />
ntp1.solar.pvt 4 3 6 +1.491 45.171 +12us 6424ns<br />
ntp1.cm.pvt 0 0 0 +0.000 2000.000 +0ns 4000ms<br />
chilipepper.canonical.com 4 3 7 +474.809 6872.249 -4302us 1104us<br />
pugot.canonical.com 4 3 7 -88.119 12753.910 -8834us 2163us<br />
golem.canonical.com 4 4 7 -453.965 9604.132 -14ms 1207us<br />
alphyn.canonical.com 4 3 7 +18.198 24.140 -2128us 3812ns<br />
If not, then start it with the command<br />
sudo systemctl restart chrony.service<br />
and reissue the command<br />
chronyc sourcestats<br />
to check that it is running correctly.<br />
<br />
Finally, after a reboot the schedule and sf_display programs have to be restarted. To do that, simply log on via VNC as usual, and then click once on their icons in the left taskbar (hover over them with the mouse if you are unsure which is which).<br />
<br />
To permit connections via vnc, first check if the x11vnc server is running by <br />
ps -ef | grep -v grep | grep /usr/local/bin/x11vnc<br />
which should display a line that indicates that it is running (it will not be, on a new reboot). If not, type<br />
x11go<br />
to start it.<br />
<br />
Definition of x11go is<br />
alias x11go="x11vnc -shared -display :0 -forever -clip 1920x1200+0+0 -bg -o /var/log/x11vnc.log -usepw -rfbport 20000 -auth /home/sched/.Xauthority"<br />
<br />
=== DPP ===<br />
First, check that the 10 Gb ethernet interfaces are present, by typing the command<br />
<pre><br />
ifconfig<br />
</pre><br />
which should show a list of interfaces including enp5s0 and enp7s0. If these are NOT present, type<br />
<pre><br />
sudo modprobe myri10ge<br />
</pre><br />
to start the device driver. Then check again that these interfaces are present.<br />
<br />
When the DPP is rebooted, its interrupt priority needs to be reconfigured. This is done via the following, typed at a terminal command prompt:<br />
sudo /home/user/test_svn/shell_scripts/SMP_AFFINITY_20181023_test.sh<br />
If the dppxmp program was running at the time of the reboot, then after the reboot it will also be necessary to delete the lock file:<br />
rm /home/user/test_svn/Miriad/dpp/DPPlock.txt<br />
or just issue the equivalent alias<br />
rmlock<br />
<br />
==== NB: ====<br />
'''Note also that if the DPP reboots it is likely necessary to mount its disk on Pipeline, otherwise the pipeline task will fail.''' To check, log in to Pipeline and type '''df'''. The following shows its output when the DPP data1 disk is NOT present:<br />
pipeline:~> df<br />
Filesystem 1K-blocks Used Available Use% Mounted on<br />
udev 65964708 0 65964708 0% /dev<br />
tmpfs 13198988 2096 13196892 1% /run<br />
/dev/sda3 100676016 66240196 29298668 70% /<br />
tmpfs 65994932 0 65994932 0% /dev/shm<br />
tmpfs 5120 0 5120 0% /run/lock<br />
tmpfs 65994932 0 65994932 0% /sys/fs/cgroup<br />
/dev/loop0 13312 13312 0 100% /snap/gnome-characters/124<br />
/dev/loop3 2304 2304 0 100% /snap/gnome-calculator/238<br />
/dev/loop4 14848 14848 0 100% /snap/gnome-logs/43<br />
/dev/loop6 3840 3840 0 100% /snap/gnome-system-monitor/57<br />
/dev/loop1 90112 90112 0 100% /snap/core/5328<br />
/dev/loop2 144384 144384 0 100% /snap/gnome-3-26-1604/70<br />
/dev/loop5 43264 43264 0 100% /snap/gtk-common-themes/701<br />
/dev/sda1 463844 205889 229488 48% /boot<br />
/dev/sdb1 35051283456 25353337932 9697945524 73% /data1<br />
cgmfs 100 0 100 0% /run/cgmanager/fs<br />
192.168.24.103:/common 1914515456 302341120 1514899456 17% /common<br />
tmpfs 13198984 28 13198956 1% /run/user/111<br />
tmpfs 13198984 32 13198952 1% /run/user/1000<br />
/dev/loop7 89984 89984 0 100% /snap/core/5548<br />
/dev/loop8 14976 14976 0 100% /snap/gnome-logs/45<br />
/dev/loop9 89984 89984 0 100% /snap/core/5662<br />
/dev/loop10 144128 144128 0 100% /snap/gnome-3-26-1604/74<br />
<br />
If so, type '''sudo mount -a''' on Pipeline, then the '''df''' command will show the same thing, but with the additional line<br />
pipeline:~> df<br />
Filesystem 1K-blocks Used Available Use% Mounted on<br />
.<br />
.<br />
. <br />
192.168.24.100:/data1 1922728960 569949184 1255088128 32% /dppdata1<br />
.<br />
.<br />
.<br />
where /dppdata1 is the DPP data1 disk.<br />
<br />
=== Correlator (ROACH boards) ===<br />
If the number of packets coming from the correlator, as seen by the command (the part before the $ is the dpp prompt)<br />
user@dpp:~$ python dpp_eth_mon.py<br />
is considerably less than 155000 on each interface, the ROACH boards may need to be rebooted. Note that if the number of packets is close, like 148000 or so, it may be that the above SMP_AFFINITY_20160511.sh script needs to be run. In any case, if the correlator needs to be rebooted, do the following commands in ipython:<br />
import roach as r<br />
ro = ['roach' + str(i+1) for i in range(8)]<br />
r.reload(ro)<br />
This will result in quite a bit of output as the 8 ROACH boards reboot (takes several minutes). Near the end of the process, all boards should show success, the sync value should be 1, and the mcount should be within 2-3 of the predicted value. After that, you can close the ipython session and recheck the presence of the packets, which should now be 155000 or so.<br />
<br />
=== LNA14 ===<br />
The LNA14 computer is the Beaglebone embedded system computer in the 27-m receiver AUX box. It controls the power to the low-noise amplifiers (LNAs) in the frontend. If that is rebooted, some software has to be started manually, using these commands (the first kills any running python tasks):<br />
<pre><br />
killAll python<br />
python ccat_bitbang_chips-master/boards/ccat_bias_board/bbServer.py -p 50002 &<br />
</pre><br />
<br />
=== ACC ===<br />
For now (2018 Jan 20) we cannot let the ACC reboot by itself, but rather we need to start it from the Win computer. See the instructions [http://www.ovsa.njit.edu/wiki/index.php/Trouble_Shooting_Guide#ACC_Restart HERE.]<br />
<br />
== Backing up the Computers ==<br />
At present, we have not pursued a rigorous backup protocol for the computers on the site. However, we need to pay more attention to this, and we can use this link: [https://help.ubuntu.com/community/BackupYourSystem] to develop a protocol. More will be written here as that is developed.<br />
--[[User:Dgary|Dgary]] ([[User talk:Dgary|talk]]) 19:39, 30 November 2016 (UTC)<br />
<br />
Here is a useful link explaining how to backup ubuntu system with rsync [https://wiki.archlinux.org/index.php/full_system_backup_with_rsync].<br />
For this moment we use the following command to backup ovsa.njit.edu machine.<br />
<pre># rsync -aAXv --exclude={"/common/*","/archive/*","/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} / /path/to/backup/folder</pre><br />
If any error shows up, we need to find out what wasn't copied. <br />
<pre># rsync -anq --exclude={"/common/*","/archive/*","/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} / /path/to/backup/folder</pre><br />
Add the parameter -n to compare source and destination directories without transfering.<br />
<br />
Here is another useful link: [https://blog.sleeplessbeastie.eu/2012/05/08/how-to-mount-software-raid1-member-using-mdadm/] on how to mount a raid-1-member disk without its mirror. This is not really related to backups, but was needed when tawa was down and we needed access to its boot disk.<br />
<br />
<br />
==Steps to setup Helios==<br />
<br />
===Setup the Python environment===<br />
Install anaconda2 to /common/anaconda2. See [https://www.anaconda.com/download/#linux anaconda website]. Once having anaconda installed, add the installation path to .bashrc.<br />
<pre><br />
export PATH="/common/anaconda2/bin:$PATH" <br />
</pre><br />
<br />
To link the eovsapy package to /common/python/current, a symbolic link did not work. Instead two changes were made:<br />
mkdir /common/python/current<br />
sudo mount --bind /home/sched/Dropbox/PythonCode/pycode /current<br />
Then add the path to .bashrc.<br />
<pre><br />
export PYTHONPATH="/common/python/current/:$PYTHONPATH"<br />
</pre><br />
<br />
To make this bind mount visible on nfs, add crossmnt to the /etc/exportfs file:<br />
/common 192.168.24.90/24(rw,sync,no_root_squash,no_subtree_check,crossmnt)<br />
<br />
===Install required packages with conda===<br />
Install as many requirements as possible with conda, then use pip. See [https://www.anaconda.com/blog/developer-blog/using-pip-in-a-conda-environment/ Using Pip in a Conda Environment ] for a reference.<br />
<pre><br />
conda install -c anaconda pip<br />
</pre><br />
<br />
===Install the rest of the required packages with pip===<br />
Use pip to install '''aipy'', '''corr''', and '''construct'''' to helios. <br />
<pre><br />
pip install --target=/common/python/packages/helios aipy<br />
pip install --target=/common/python/packages/helios corr<br />
pip install --target=/common/python/packages/helios --upgrade construct==2.5.0<br />
</pre><br />
If you got into an error similar to "gcc: error trying to exec 'cc1plus': execvp: No such file or directory", restall 'build-essential'. Otherwise, skip this step.<br />
<pre><br />
sudo apt-get update<br />
sudo apt-get install --reinstall build-essential<br />
</pre><br />
Add the path to .bashrc<br />
<pre><br />
export PYTHONPATH="/common/python/packages/helios/:$PYTHONPATH"<br />
</pre><br />
<br />
===Setup NFS (Network File System)===<br />
This step is to make the folder /common accessiable from other machines.<br />
Define the exported directory in /etc/exports,<br />
<pre><br />
/common 192.168.24.90/24(rw,sync,no_root_squash,no_subtree_check,crossmnt)<br />
/common 192.100.16.206/0(rw,sync,no_root_squash,no_subtree_check,crossmnt)<br />
</pre><br />
To mount the "pycode" permanently on "current" across the reboots, add an entry to /etc/fstab<br />
<pre><br />
/home/sched/Dropbox/PythonCode/pycode /common/python/current none defaults,bind 0 0<br />
</pre><br />
<br />
<br />
<br />
<br />
==Steps to setup pipeline==<br />
<br />
===Setup the Python environment===<br />
Assume that we have install anaconda2 to /common/anaconda2. Add the installation path to .bashrc.<br />
<pre><br />
export PATH="/common/anaconda2/bin:$PATH" <br />
</pre><br />
<br />
Then add the eovsapy path to .bashrc.<br />
<pre><br />
export PYTHONPATH="/common/python/current/:$PYTHONPATH"<br />
</pre><br />
<br />
===Install the rest of the required packages with pip===<br />
<pre><br />
pip install --target=/common/python/packages/pipeline aipy<br />
pip install --target=/common/python/packages/pipeline sunpy==0.7.9<br />
pip install --target=/common/python/packages/pipeline pyodbc<br />
</pre><br />
<br />
Add the path to .bashrc<br />
<pre><br />
export PYTHONPATH="/common/python/packages/pipeline:$PYTHONPATH"<br />
</pre><br />
<br />
<br />
Add the path to ~/.casa/init.py<br />
<pre><br />
sys.path.append('/common/python/packages/pipeline')<br />
</pre><br />
<br />
===Update the dependencies package in CASA===<br />
CASA ships with a few thrid party packages (NumPy, SciPy, Matplotlib).<br />
However, some essential packages in EOVSAPY and SUNCASA, e.g., aipy and SunPy, require dependencies package with a higher version. <br />
<br />
<pre><br />
numpy>=1.14.0. ## was 1.11.3, required by aipy<br />
dateutil>=2.5.0 ## was 1.5.0, required by SunPy<br />
</pre><br />
<br />
This step is to update those package to appropriate versions. In CASA prompt,<br />
<pre><br />
try:<br />
from pip import main as pipmain<br />
except:<br />
from pip._internal import main as pipmain<br />
pipmain(['install', '--target=/common/casa/casa-release-5.4.1-31.el6/lib/python2.7/site-packages', '--upgrade', 'numpy==1.14.3'])<br />
pipmain(['install', '--target=/common/casa/casa-release-5.4.1-31.el6/lib/python2.7/site-packages', '--upgrade', 'python-dateutil==2.7.5'])<br />
</pre></div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/2019_July2019 July2019-07-02T12:19:11Z<p>Dgary: Created page with "[http://www.ovsa.njit.edu/wiki/index.php/Expanded_Owens_Valley_Solar_Array#Observing_Log Back to Observing Log] == July 02 == '''12:14 UT''' Today I set all of the antennas (..."</p>
<hr />
<div>[http://www.ovsa.njit.edu/wiki/index.php/Expanded_Owens_Valley_Solar_Array#Observing_Log Back to Observing Log]<br />
<br />
== July 02 ==<br />
'''12:14 UT''' Today I set all of the antennas (1-13) back to their old pointing coefficients from before I did the pointing measurements. For whatever reason, the update to the pointing measurements was incorrect and nothing I did could make the updated pointing coefficients work. I really do not understand why the update should fail. I suppose I need to go through the algorithm in detail and see if there is a bug somewhere. I did leave the 27-m (ant14) pointing coefficients updated, although I do not have a good way to check how good they are.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/Expanded_Owens_Valley_Solar_ArrayExpanded Owens Valley Solar Array2019-07-02T12:13:49Z<p>Dgary: /* Observing Log */</p>
<hr />
<div>[[File:Eovsa1.png|border|text-top|800px]]<br />
<br />
<big>[http://ovsa.njit.edu/ EOVSA] (Expanded Owens Valley Solar Array) is a new solar-dedicated radio interferometer operated by the New Jersey Institute of Technology. This wiki serves as the site for EOVSA documentation. </big><br />
<br />
== EOVSA Documentation ==<br />
<br />
* <big>General</big><br />
** [[Downconversion and Frequency Tuning]]<br />
** [[Dealing with Radio Frequency Interference]]<br />
** [[Switching between 200 MHz and 300 MHz Correlator]]<br />
<br />
* <big>Computer-Network</big><br />
** [[Computing Systems]]<br />
** [[Network]]<br />
<br />
* <big>Control System</big><br />
** [[27-m Antenna Commands]]<br />
** [[Schedule Commands]]<br />
** [[Control Commands]]<br />
** [[Attenuation and Level Control]]<br />
<br />
* <big>Hardware</big><br />
** [[Hardware Overview]]<br />
** [[2.1-m Antennas]]<br />
** [[27-m Antennas]]<br />
<br />
* <big>System Software</big><br />
** [[Calibration Database]]<br />
** [[Stateframe Database]]<br />
** [[Create CASA measurement sets]]<br />
<br />
* <big>Calibration</big><br />
**[[Calibration Overview]]<br />
**[[Pointing Calibration]]<br />
**[[Total Power Calibration]]<br />
**[[System Gain Calibration]]<br />
**[[Antenna Position]] (Baseline Calibration)<br />
**[[Reference Gain Calibration]]<br />
**[[Daily Gain Calibration]]<br />
**[[Delay Calibration]]<br />
**[[Bandpass Calibration]]<br />
**[[Polarization Calibration]]<br />
**[[Calibrator Survey]]<br />
<br />
* <big>[[Starburst]]</big><br />
<br />
== Using EOVSA Data ==<br />
* <big>Analysis Software</big><br />
** [https://github.com/suncasa/suncasa SunCASA] A wrapper around [https://casa.nrao.edu/ CASA (the Common Astronomy Software Applications package)] for synthesis imaging and visualizing solar spectral imaging data. CASA is one of the leading software tool for "supporting the data post-processing needs of the next generation of radio astronomical telescopes such as ALMA and VLA", an international effort led by the [https://public.nrao.edu/ National Radio Astronomy Observatory]. The current version of CASA uses Python (2.7) interface. More information about CASA can be found on [https://casa.nrao.edu/ NRAO's CASA website ]. Note, CASA is available ONLY on UNIX-BASED PLATFORMS (and therefore, so is SunCASA). <br />
** [https://github.com/Gelu-Nita/GSFIT GSFIT] A IDL-widget(GUI)-based spectral fitting package called gsfit, which provides a user-friendly display of EOVSA image cubes and an interface to fast fitting codes (via platform-dependent shared-object libraries). <br />
** [[Spectrogram Software]]<br />
* <big>Data Analysis Guides</big><br />
** <big>[[EOVSA Data Analysis Tutorial]]</big> at [http://rhessi18.umn.edu/ RHESSI XVIII Workshop]<br />
** [[Self-Calibrating Flare Data]] Example script and guides for self-calibrating EOVSA flare data (to be completed)<br />
<!-- ** [[Imaging]] --><br />
<!-- ** [[Flare Imaging]] --><br />
<br />
* <big>EOVSA Modeling Guide</big><br />
**[[GX Simulator]]<br />
<br />
* Other helpful links<br />
** [https://casaguides.nrao.edu CASA Guides]<br />
** [http://www.lmsal.com/solarsoft/ SolarSoft IDL]<br />
** [http://www.atnf.csiro.au/computing/software/miriad/userguide/userhtml.html Miriad Guides]<br />
** [https://sites.google.com/site/fgscodes/ Fast Gyrosynchrotron Codes (Alexey Kuznetsov's website)]<br />
** [[Basic GitHub Tutorial]]<br />
<br />
<!--* <big>[[EOVSA Imaging Workshop]]</big>--><br />
<br />
== System Software ==<br />
<br />
* LabVIEW software<br />
* Python code [https://github.com/dgary50/eovsa Github repository]<br />
<br />
== Observing Log ==<br />
[[2016 November]]<br />
<br />
[[2016 December]]<br />
<br />
[[2017 January]]<br />
<br />
[[2017 February]]<br />
<br />
[[2017 March]]<br />
<br />
[[2017 April]]<br />
<br />
[[2017 May]]<br />
<br />
[[2017 June]]<br />
<br />
[[2017 July]]<br />
<br />
[[2017 August]]<br />
<br />
[[2017 September]]<br />
<br />
[[2017 October]]<br />
<br />
[[2017 November]]<br />
<br />
[[2017 December]]<br />
<br />
[[2018 January]]<br />
<br />
[[2018 February]]<br />
<br />
[[2018 March]]<br />
<br />
[[2018 April]]<br />
<br />
[[2018 July]]<br />
<br />
[[2019 March]]<br />
<br />
[[2019 April]]<br />
<br />
[[2019 May]]<br />
<br />
[[2019 June]]<br />
<br />
[[2019 July]]<br />
<br />
== Tohbans ==<br />
<br />
[[Trouble Shooting Guide]]<br />
<br />
[[Tohban Records]]<br />
<br />
== EOVSA Flare List ==<br />
<br />
See [https://docs.google.com/spreadsheets/d/1P8jHuDRF93dMflU6RMQcsJqVepD9vFkPkofV8Imj4xA/edit?usp=sharing this link] for a list of EOVSA flares as a Google Spreadsheet. <br />
<br />
[http://ovsa.njit.edu/jay/rd_db.php Another link] available at the EOVSA website. <br />
<br />
<!--<br />
<br />
{| class="wikitable" style="text-align: center; width: 1200px; height: 200px;"<br />
|-<br />
| '''Date''' || '''RHESSI Time''' || '''RHESSI Lightcurve''' || '''XSP Data''' || '''RHESSI Coverage''' || '''RHESSI Energy Range (keV)''' || '''EOVSA Time''' || '''EOVSA Image''' || '''GOES Class''' || '''Baseline Plot'''<br />
|-<br />
| Jun 3, 2017 || 21:43:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/03/hsi_20170603_212240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170603213511.png XSP] || Yes || 6 - 12 || 21:42:00 || || B2.0 || [[TBD | Error]]<br />
|-<br />
| " || 19:29:19 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/03/hsi_20170603_181420_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170603173511.png XSP] || Yes || 12 - 25 || 19:24:00 || || C2.5 || [[:File:UDB20170603173518.png | Plot]]<br />
|-<br />
| Jun 2, 2017 || 19:55:54 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/02/hsi_20170602_184040_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170602173511.png XSP] || Yes || 6 - 12 || 19:50:00 || || B7.0 || [[:File:UDB20170602173518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/02/hsi_20170602_170640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170602173511.png XSP] || No || || 17:55:00 || || C8.0 || [[:File:UDB20170602173518.png | Plot]]<br />
|-<br />
| Jun 1, 2017 || 20:27:40 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/01/hsi_20170601_190640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170601173511.png XSP] || Yes || 6 - 12 || 20:25:00 || || C1.1 || [[:File:UDB20170601173518.png | Plot]]<br />
|-<br />
| " || 18:04:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/01/hsi_20170601_173240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170601173511.png XSP] || Yes || 6 - 12 || 17:55:00 || || C1.4 || [[:File:UDB20170601173518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/06/01/hsi_20170601_173240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170601173511.png XSP] || No || || 17:37:00 || || C1.0 || [[:File:UDB20170601173518.png | Plot]]<br />
|-<br />
| May 31, 2017 || 23:14:45 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/31/hsi_20170531_224100_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170531213511.png XSP] || Yes || 12 - 25 || 23:12:00 || || C2.0 || [[TBD | Error]]<br />
|-<br />
| " || TBD || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/31/hsi_20170531_210700_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170531213511.png XSP] || Yes || || 22:20:00 || || B9.8 || [[TBD | Error]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/31/hsi_20170531_193240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170531205112.png XSP] || No || || 20:54:00 || || C1.3 || [[TBD | Error]]<br />
|-<br />
| May 28, 2017 || 19:28:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/28/hsi_20170528_191440_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170528173511.png XSP] || Yes || 25 - 50 || 19:27:00 || || C3.2 || [[:File:UDB20170528173520.png | Plot]]<br />
|-<br />
| May 6, 2017 || 20:07:24 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/05/06/hsi_20170506_194540_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170506180511.png XSP] || Yes || 6 - 12 || 20:06:49 || || B3.5 || [[:File:UDB20170506180518.png | Plot]]<br />
|-<br />
| April 21, 2017 || 17:27:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/21/hsi_20170421_170640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170421141111.png XSP] || Yes || 6 - 12 || 17:26:51 || || B4.1 || [[:File:UDB20170421141120.png | Plot]]<br />
|-<br />
| April 17, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/17/hsi_20170417_233740_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170417232011.png XSP] || No || || 23:45:51 || || B2.0 || [[:File:UDB20170417232020.png | Plot]]<br />
|-<br />
| " || 21:40:04 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/17/hsi_20170417_202920_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170417213511.png XSP] || Yes || 6 - 12 || 21:38:51 || || B7.5 || [[:File:UDB20170417213520.png | Plot]]<br />
|-<br />
| April 10, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/10/hsi_20170410_171800_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170410173511.png XSP] || No || || 18:26:51 || || B4.8 || [[:File:UDB20170410173520.png | Plot]]<br />
|-<br />
| April 8, 2017 || 23:58:40 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/08/hsi_20170408_225240_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170408231011.png XSP] || Yes || 6 - 12 || 23:53:53 || || B2.3 || [[:File:UDB20170408231022.png | Plot]]<br />
|-<br />
| April 7, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/07/hsi_20170407_183600_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170407191511.png XSP] || No || || 19:47:52 || || C4.3 || [[:File:UDB20170407191521.png | Plot]]<br />
|-<br />
| " || 00:23:45 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/06/hsi_20170406_234500_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170406233011.png XSP] || Yes || 12 - 25 || 00:23:52 || || C2.7 || [[:File:UDB20170406233021.png | Plot]]<br />
|-<br />
| April 5, 2017 || 17:38:20 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/05/hsi_20170405_162020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170405173511.png XSP] || Yes || 6 - 12 || 17:38:51 || || B9.1 || [[:File:UDB20170405173520.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/05/hsi_20170405_144600_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170405142811.png XSP] || No || || 15:52:50 || || B5.9 || [[:File:UDB20170405142819.png | Plot]]<br />
|-<br />
| April 4, 2017 || 23:42:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/04/hsi_20170404_230340_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170404231511.png XSP] || Yes || 25 - 50 || 23:40:51 || [[:File:Eovsa_flare_20170404T233951.png | Image]] [http://www.ovsa.njit.edu/movies.html Movies] || C4.9 || [[:File:UDB20170404231520.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/04/hsi_20170404_182100_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170404191511.png XSP] || No || || 19:42:52 || || B9.4 || [[:File:UDB20170404191521.png | Plot]]<br />
|-<br />
| " || 00:39:35 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/03/hsi_20170403_233020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170403231511.png XSP] || Yes || 12 - 25 || 00:37:49 || || C1.4 || [[:File:UDB20170404003935.png | Plot]]<br />
|-<br />
| April 3, 2017 || 23:51:49 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/03/hsi_20170403_233020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170403231511.png XSP] || Yes || 12 - 25 || 23:44:49 || || C1.7 || [[:File:UDB20170403231518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/03/hsi_20170403_184740_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170403191511.png XSP] || No || || 20:11:53 || || C5.0 || [[:File:UDB20170403191522.png | Plot]]<br />
|-<br />
| " || 01:00:44 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_235700_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402213511.png XSP] || Yes || 25 - 50 || 01:00:52 || || M1.2 || [[:File:UDB20170402223521.png | Plot]]<br />
|-<br />
| April 2, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_191420_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402191511.png XSP] || No || || 20:31:49 || || M5.7 || [[:File:UDB20170402191518.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_174020_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402173511.png XSP] || No || || 18:48:52 || || M2.1 || [[:File:UDB20170402173521.png | Plot]]<br />
|-<br />
| " || 16:25:30 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/02/hsi_20170402_160600_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170402143211.png XSP] || Yes || 12 - 25 || 16:24:50 || 6 sfu || C3.0 ||[[:File:UDB20170402143219.png | Plot]]<br />
|-<br />
| April 1, 2017 || 23:08:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_224940_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401213511.png XSP] || Yes || 12 - 25 || 23:01:51 || 49 sfu || C5.1 || [[:File:UDB20170401213520.png | Plot]]<br />
|-<br />
| " || 21:48:35 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_211520_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401213511.png XSP] || Yes || 50 - 100 || 21:46:51 || 286 sfu || M4.4 || [[:File:UDB20170401213520.png | Plot]]<br />
|-<br />
| " || 19:57:23 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_194120_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401191511.png XSP] || Yes || 12 - 25 || 19:42:50 || 15 sfu || C3.7 || [[:File:UDB20170401191519.png | Plot]]<br />
|-<br />
| " || 15:37:25 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/04/01/hsi_20170401_145840_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170401143743.png XSP] || Yes || 12 - 25 || 15:55:21 || 2.5 sfu || B8.1 || [[:File:UDB20170401143750.png | Plot]]<br />
|-<br />
| March 31, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/03/31/hsi_20170331_214220_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170331213511.png XSP] || No || || 22:57:49 || || C1.1 || [[:File:UDB20170331213518.png | Plot]]<br />
|-<br />
| March 29, 2017 || 23:27:10 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/03/29/hsi_20170329_223640_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170329213511.png XSP] || Yes || 6 - 12 || 23:40:50 || || B6.5 || [[:File:UDB20170329213519.png | Plot]]<br />
|-<br />
| March 25, 2017 || 23:34:44 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/03/25/hsi_20170325_225100_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170325213511.png XSP] || Yes || 6 - 12 || 23:44:50 || || B2.3 || [[:File:UDB20170327213520.png | Plot]]<br />
|-<br />
| February 23, 2017 || 20:52:00 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/02/23/hsi_20170223_202800_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170223191510.png XSP] || Yes || 6 - 12 || 20:52:50 || || C1.3 || [[:File:UDB20170223191519.png | Plot]]<br />
|-<br />
| " || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/02/23/hsi_20170223_185340_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170223191510.png XSP] || No || || 20:25:50 || || B5.8 || [[:File:UDB20170223191519.png | Plot]]<br />
|-<br />
| February 21, 2017 || - || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/02/21/hsi_20170221_225520_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170221220653.png XSP] || No || || 22:51:10 || || B3.0 || [[:File:UDB20170221220739.png | Plot]]<br />
|-<br />
| January 25, 2017 || 21:27:14 || [http://hessi.ssl.berkeley.edu/hessidata/metadata/2017/01/25/hsi_20170125_204400_corrected_rate.png Lightcurve] || [http://ovsa.njit.edu/flaremon/XSP20170125200011.png XSP] || Yes || 12 - 25 || 21:26:57 || || B8.5 || [[:File:UDB20170125200025.png | Plot]]<br />
|}<br />
--><br />
<br />
== EOVSA Publications ==<br />
Here is a (partial) list of publications that utilizes EOVSA data.<br />
<br />
; 2018<br />
: [https://ui.adsabs.harvard.edu/#abs/2018ApJ...864...63P/abstract Polito et al. (2018), ApJ, 864, 63] Broad Non-Gaussian Fe XXIV Line Profiles in the Impulsive Phase of the 2017 September 10 X8.3-class Flare Observed by Hinode/EIS<br />
: [https://ui.adsabs.harvard.edu/#abs/2018ApJ...863...83G/abstract Gary et al. (2018), ApJ, 863, 83] Microwave and Hard X-Ray Observations of the 2017 September 10 Solar Limb Flare<br />
: [https://ui.adsabs.harvard.edu/#abs/2018ApJ...852...32K/abstract Kuroda et al. (2018), ApJ, 852, 32] Three-dimensional Forward-fit Modeling of the Hard X-ray and the Microwave Emissions of the 2015 June 22 M6.5 flare<br />
<br />
== VLA Flare List ==<br />
See [http://www.ovsa.njit.edu/wiki/index.php/VLA_Data_Survey#List_of_Jansky_VLA_Solar_Observations this link] for a list of VLA flare observations.</div>Dgaryhttp://www.ovsa.njit.edu/wiki/index.php/2019_June2019 June2019-06-29T18:05:37Z<p>Dgary: /* June 19 */</p>
<hr />
<div>[http://www.ovsa.njit.edu/wiki/index.php/Expanded_Owens_Valley_Solar_Array#Observing_Log Back to Observing Log]<br />
<br />
== June 01 ==<br />
'''12:18 UT''' Just a note to say the data archive disk is getting full, so we need to stage some data off. One option is to move data prior to 2017 off. Here are the data usages by year:<br />
* 2014 (0.3 TB)<br />
* 2015 (1 TB)<br />
* 2016 (9.5 TB)<br />
* 2017 (6 TB)<br />
* 2018 (5.4 TB)<br />
* 2019 (5.4 TB)<br />
Note that 2019 is less than half over, so we are looking at more than 10 TB/year (actually should be more than 20 TB/year by my calculation of 60 GB/day).<br />
<br />
== June 04 ==<br />
'''15:05 UT''' More on the storage issue. Looking at tape backup, one can purchase a HP Ultrium tape drive for about $1600, with 1.5/3 TB cartridges costing $25 each. I also find an 80 GB Stonefly rackmount storage system costs about $7000 (with 8 x 10 TB drives). The latter could store up to 3.5 years of data at our current rate (or 2 years in addition to our current database). We really need to have one unit at OVRO and a second one at NJIT. We could solve our current problem, then, by spending $2600 on tape backup (for long-term storage) plus $365/year going forward, and $14,000 for dual-capacity storage. Total for the next two years: $17,500. This is certainly a good investment! In two years, there may be other storage options.<br />
<br />
== June 19 ==<br />
'''23:09 UT''' Just a note to say that I (--[[User:Dgary|Dgary]] ([[User talk:Dgary|talk]]) 23:11, 19 June 2019 (UTC)) am at OVRO this week and will be messing with the system. Today is a good example--several calibrations, correlator restart, setting of delays, etc., have pretty much messed up today's data. I am chasing a long-standing problem with low signal levels at high frequencies, and will also be working on improving total power calibration.<br />
<br />
== June 29 ==<br />
'''17:52 UT''' As predicted, the system was in disarray for about a week as I did various tests and a long series of pointing measurements. The update to pointing went awry, however, when I had antenna 4 out of the subarray, not realizing that the software does not work well with missing antennas when updating the pointing. This morning I finally realized what was wrong, and have put ant4 back in the array. I am doing another SOLPNTCAL after updating the pointing correctly, I hope.<br />
<br />
'''18:04 UT''' Yes, the pointing is now very good for this moment. Later I will check the other SOLPNTCALs for the day and see how well they do at other times.</div>Dgary