Contents:
If you are reducing LOFAR data, you should at least read the LOFAR imaging cookbook. The NDPPP chapter contains also information about the "quality" tools that are included in the AOFlagger package. A first tutorial on the GUI can be found here: RFI gui tutorial, and an example to optimize the default strategy here: Strategy optimization example. More documentation might follow on this website, and of course scanning through this FAQ might solve some of your questions. See the "How does RFI console recognise RFI" faq item for some links to scientific papers about the flagger.
Yes, it does. As far as I know, it has already been tested and shown good results on LOFAR, WSRT, VLA and GMRT data. The only requirement is that the data is stored in the Aips++ / casa Measurement Set format (in my personal opinion, this is one of the best formats around for radio observations). If you have results for other telescopes, I'd love to hear about it.
Note though, that the settings have been optimized for LOFAR, and some of the settings might need to be changed to get good results. See the "I get bad results, how te solve?" question for more info.
Strong sources and different telescope properties can cause the default settings to fail in certain cases. The gui can help you a lot when trying to solve problems. Be sure to read the strategy optimization example. If you think rficonsole/rfigui is doing a bad job, I would suggest to experiment with the GUI to optimize the following settings:
I have never seen an observation in which optimizing the settings did not work in the end, so don't give up. Make sure the phase centre does not change during the observation (see the question on multiple fields). You can always mail me and hand over a small part of your observation to let me take a look at the problem.
That is well possible, even after using rficonsole. RFI console will only flag RFI that is (1) detectable and (2) not both broadband and continuous. Especially broadband in situ RFI (RFI generated by hardware near the telescope) can not be removed by flagging and might cause calibration failures and/or sidelobes while imaging. Other techniques should be used in such cases.
It will treat them as separate baselines, which will be fine. If there is some problem, the best way is to split them into separate measurement sets. LOFAR never stores multiple bands in one measurement set by default, but WSRT does.
I have not tested this, but it will probably treat the entire observation as a whole, and look at the concatenated baselines (i.e., time sort the data and assume the data should look the same). Therefore, if you sweep between a strong calibrator source and another field, rficonsole might flag your calibrator completely! Therefore, always split the measurement set when the phase centre changes. LOFAR will never store multiple sources in one measurement set, so for LOFAR this is not an issue.
Here are some hints on memory usage of rficonsole:
<THREAD COUNT> x 4 x <BASELINE SIZE>
bytes of memory in direct mode and <THREAD COUNT> x 2.5 x <BASELINE SIZE>
bytes in indirect mode. The size of a single baseline can be calculated like 8 (bytes/complex float) x 4 (polarizations) x 256 (channels) x 6000 (time steps)
(thus 50 MB per baseline in this case, 0.5 GB in total, a rather typical LOFAR example). Hence, if the system really runs out of memory / starts swapping, decrease the number of threads as explained in the command line help of rficonsole and make sure to use the indirect reader.This probably means that reading the data takes a lot of time and the computing threads are awaiting new data. Reasons for slow IO can be:
Generally, the indirect reading mode is much faster than the direct reading mode.
The full message is:
terminate called after throwing an instance of 'casa::TableInvOper'
what(): Invalid Table operation: FLAG is readonly (use the ROxxxColumn class)Abort
This means you are trying to flag raw LOFAR data in which the LofarStMan storage manager is used for the flag column. This storage manager is read only, and rficonsole can not update these flags. The LOFAR imaging cookbook will tell you on changes, but the current way to solve this on the LOFAR cluster is:
makeFLAGwritable <MyObservation.MS>
This will take a few seconds to rewrite the flag column, after which you can run rficonsole.
[update 2012-03: the GUI now uses an internal plotter, and this answer is probably no longer relevant]. Plotting requires several programs:
If plotting is not working, check the terminal; RFI gui will output errors there.
The mode is called the indirect reading mode, and is described on the following page: reading mode.
The latest version is the one installed on the LOFAR cluster. This version is build daily from the Astron Subversion repository and should be the preferred version at all times, if you have access to it. If not, you can use the packaged version from http://www.astro.rug.nl/rfi-software. If the version there is older than two months, mail me so that I package the latest version -- it is in heavy development and many bugs and fixes are made within a month.
Newer versions can not always read strategies which are created by older tools. Sometimes, the changes are small and can be easily solved, but sometimes it is better to recreate the strategies. You can try changing the version of the strategy file with a text editor and load it with rficonsole (rficonsole -strategy old_strategy.rfis). It will probably reply with an exception about which action is causing the incompatibility. By comparing that part of the strategy with a newer strategy, you might be able to solve the problem.
Since October 2010, RFI console uses the LOFAR logger, which in turn uses LogCplus. Standard logging configuration can be set up in a configuration file. The parameters -nolog
and -v
can be used to alter logging strategy. The standard unix program tee
can be used to save the output to file while at the same time monitoring the standard output.
It estimates the astronomical signal and searches for statistical abnormalities that look like lines in the time/frequency plots. After initial detection, it applies a morphological algorithm to find likely contaminated samples. It has several tricks to enhance the speed. The following refereed papers describe parts of the detection technique in detail:
I love to have feedback in order to update this FAQ and get a grib on common problems. You can reach me on my e-mail address, <my-last-name>@astro.rug.nl
, and my last name is offringa.