11:03 UT The data from 19:00 UT yesterday until now are no good. Yesterday Kjell replaced the 800 MHz clock derived from the signal generator with one from a new Valon 5009 Synthesizer. Although the data seemed to be okay, I could not see any coherence in the phasecals or refcals. That prevented me from getting a delay correction. This morning I checked the delay using a communication satellite and discovered that two antennas, 8 and 14, had extremely high delays, about 300 ns! I rebooted the correlator this morning and rechecked, and now they are fine. I have seen this strange behavior once before, so I can only surmise that occasionally on a reboot of the correlator we can get really wacky delays on some digitizers (or ROACH channels--maybe not the digitizer itself). This suggests that we really need to make a quick observation of a satellite after a reboot in order to do a sanity check on the delays.
12:56 UT The delays have been updated this morning prior to today's calibrations. For some reason, ant13 has poor (but not zero) phase coherence as of mid-day yesterday. I just tried stowing to be sure it was not off in motor counts, but the stow looked nominal. I see no other indications of a problem.
15:14 UT I reset all of the gains just prior to the first calibrator scan, since I was not happy with the levels. Based on an earlier snapshot of a geosat, it looks like Ant13 receiver is not working at all, but I will try it again in a half hour or so to see if setting the levels helps.
22:00 UT Kjell was converting the new synthesizer for the correlator clock to its permanent location in the rack today, but ran into a lot of problems. Eventually he got it installed and working, so I had to reboot the correlator as a result.
02:40 UT The delays were reset after the correlator reboot.
12:02 UT It is getting frustrating that the Ant A control system dies each night around 2:00 UT. I have been trying to do an overnight calibration (X-Y delay phase calibration) and for the past two nights it died each time. The symptom is that the control system shows occasional 1-s outages most of the day, and then suddenly starts doing it a lot more (for a couple of hours) before it just stops. To combat this, I am trying to restart it every four hours (at 1, 5, 9, 13, 17, and 21 h), so we will see if this helps. Meanwhile, I could not even check the delays after the above reset because the Ant A receiver was not in the correct state for subsequent observations. Hopefully the first calibrator scan of today will be good.
15:02 UT Oh oh. I have been taking SKYCALTEST scans for about a year, and today I find out that they are not being done correctly! They were supposed to be taken as for a solar scan, but on the position of a calibrator. What I learned today is that while I did change the frequency sequence, it was done "manually" so that the setting of the DCM attenuation was not correct for the solar sequence. Instead, it was switching the DCM attenuation for the pcal_hi sequence. I am now doing a test that should result in a correct SKYCALTEST scan.
13:09 UT This is not really an observing entry, but I wanted to record some findings on my investigation of total power calibration. When I get the SKYCALTEST observation correct, and compare with both raw observations of SOLPNTCAL and the fitted results of the off Sun background from SOLPNTCAL, I find the following patterns:
- The SOLPNTCAL observations are done with some FEMATTN inserted, but are scaled to 0 dB (not sure where in the code this is done)
- The scaled SOLPNTCAL result, when reliable, is systematically higher than the SKYCAL background over frequency indexes 70-170 or so. This might be due to 5 degrees not being far enough from Sun center to reach background.
- The scaled SOLPNTCAL result is systematically lower than the SKYCAL background over higher frequencies, roughly indexes 170-380 or so, although the amount is quite small for most antennas.
- The scaled SOLPNTCAL result is again systematically higher than the SKYCAL background over frequency indexes above 380, which might cause the anomalously high flux densities we often see in the total power spectra.
- The SOLPNTCAL fits are often very poor at frequency indexes below 70, which is what the SKYCAL measurement is designed to fix. But it may be that the SKYCAL background should be used for all SOLPNTCAL fits, and for the off Sun value for the purposes of total power calibration, since the background from the fits seem unreliable.
- It is clear that some measure of goodness of fit should be used to affect an uncertainty estimate (error bar) for the total power calibration.