Enthusiastic flagging?

Hi,

I was wondering if LINC is possibly flagging excessive amounts of data. We've processed a few hundred observations between 2018 and 2023 with LINC and find the following flagging fractions for the LOFAR HBA calibrator and target runs.

Calibrator (red initial and yellow final):

Target (red intial; green prep; blue concat; yellow final)

Previously for e.g. LoTSS-DR2 which consisted of data between 2014 and 2020 we saw typically around 9% of the data flagged (see blue line in the following plot which is the average (dark) and individual pointing (semi transparent) flagging fraction as a function of freq).

It could also be that the amount of RFI or bad data impacting the observations has severely increased in recently. Or that previously we didnt flag enough data. However just thought to mention it because its a lot of data.

Not sure if to help investigate the json file that records the flagging fractions could be a little finer (presently e.g. for calibrator it is just initial and final). This might help see if some particular steps are responsible.

Thanks,

Tim