diff --git a/applications/lofar2/doc/prestudy/station2_sdp_icd.txt b/applications/lofar2/doc/prestudy/station2_sdp_icd.txt index 679e720ba482e655210ac304238e9b524fc933c2..d51c651115fe9e44cc9973881fde89befc1524f2 100755 --- a/applications/lofar2/doc/prestudy/station2_sdp_icd.txt +++ b/applications/lofar2/doc/prestudy/station2_sdp_icd.txt @@ -498,7 +498,267 @@ Do not support more than one block per packet, because for SST and BST the paylo Statistics packet fields format: See ICD SC-SDP in Polarion +The ToD in SC and the ToD in SDPTR shall be within +-10 ms. This value is a compromise between using a measurable value, a feasible value with NTP and not adding too much timing margin between SC and SDP. + + +Direct: + +Per memory mapped (MM) point in the register map of a PN on a UniBoard2: + +/unb[0:7]/pn[0:3]/mm/peripheral/field + + +Composite: + +Per functional point per antenna band, where antenna band is Low Band (= SDP - LBAS) or High Band (= SDP - HBAS) + +Monitor and control of SDP + +The monitoring and control of the SDP Translator and of the SDP Firmware is done via the OPC-UA interface of the SDP Translator. The SDP Translator is central to entire SDP. + + +The SDP Firmware is distirubuted + +List of products with M&C in SDP + +Path Product name + +/sdptr SDP Translator +/band[0:1]/bsp SDPFW Board Support Package +/band[0:1]/general SDPFW +/band[0:1]/ait SDPFW ADC Input and Timing +/band[0:1]/fsub SDPFW Subband Filterbank +/band[0:1]/bf SDPFW Beamformer +/band[0:1]/ri SDPFW Ring +/band[0:1]/xc SDPFW Subband Correlator +/band[0:1]/tbuf SDPFW Transient Buffer +/band[0:1]/tdet SDPFW Transient Detector + + +List of control points per antenna band and per product in SDP + +Path Point +/sdptr /sdp_info +/band[0:1]/bsp /pn_image +/band[0:1]/general /sdp_info + /data_flags + /signal_input_range +/band[0:1]/ait /processing_enable + /si_sample_delay + /si_waveform_generator +/band[0:1]/fsub /pfir_coefficients + /subband_weights + /sst_select + /sst_offload_destination + /sst_offload_enable +/band[0:1]/bf /subband_select + /weights_xx + /weights_yy + /bst_offload_destination + /bst_offload_enable + /beamlet_offload_destination + /beamlet_offload_enable +/band[0:1]/ri/ +/band[0:1]/xc/ +/band[0:1]/tbuf/ +/band[0:1]/tdet/ + + +List of monitor points per antenna band and per product in SDP + +Path Point +/sdptr /tr_info + /tr_timing + /tr_network +/band[0:1]/bsp /pn_info + /pps_timing + /mc_network +/band[0:1]/general /sdp_info +/band[0:1]/ait /sdp_status + /si_timing + /si_mean + /si_sigma + /si_data_buffer + /si_histogram +/band[0:1]/fsub - If sst_offload_enable = off, then the SST can be read via direct monitoring. +/band[0:1]/bf - If bst_offload_enable = off, then the BST can be read via direct monitoring. +/band[0:1]/ri/ +/band[0:1]/xc/ +/band[0:1]/tbuf/ +/band[0:1]/tdet/ + + + +For the SDP Firmware the + +/sdptr/ + Control: + - sdp_info: + . station_id + Monitor: + - tr_info: + . software name and version + . temperature ? (or via EC2 ?) + - tr_timing: + . NTP status + . PPS status : Locked, not locked, unknown + - Not locked = NTP time at SDPTR does not coincide within +- 1 ms with PPS top of second at SDPFW + - Locked = NTP time at SDPTR is aligned within +- 1 ms with the PPS top of second at SDPFW → based on the offset_cnt information from the PPSH in one of the PN in the SDPHW / SDPFW + - Unknown when the PPS information in SDPFW is not stable or not available + - tr_network: + . SC-SDPTR link status at SDPTR : nof tx pkts, nof rx pkts, nof crc errors, nof retransmissions, nof timeouts + . SDPTR-SDPFW link status at SDPTR : nof tx pkts, nof rx pkts, nof crc errors, nof retransmissions, nof timeouts + + +/band[0:1]/bsp/ + Control for all allocated PN + - pn_image : design name + + Monitor per allocated PN + - pn_info : unb_hw_version, gn_id, rn_id, fw_image (design_name with date, time and revision stamp, design note), f_adc, fsub_type + - pn_regmap : mmap + - pps_timing : + . STF-SDPFW PPS status at SDPFW: + 'not active' = measured PPS count >> f_adc = 200M and clipped at 2**28 - 1 + 'asynchronous' = measured PPS count != expected count -1, +0, or +1 + 'active' = measured PPS count = expected count -1, +0, or +1 + 'active and stable' → measured PPS count has been equal to expected count -1, +0, or +1 since last time it was monitored. + - eth1g : + . SDPTR-SDPFW link status at SDPFW : nof tx pkts, nof rx pkts, nof crc errors, nof retransmissions, nof timeouts + +band[0:1]/general + Control + - sdp_info: observation_id, nyquist_zone_index + - data_flags : beam repositioning flag, calibrated subbands flag + - signal_input_range : R_ant = (O_si, N_si) → this will allocate N_pn = N_si / S_pn PN starting with PN with rn_id = O_si / S_pn + + Both the offset and the amount are a multiple of S_pn = 12 signals inputs, because there are S_pn = 12 signal inputs per FPGA processing node (PN) in the SDP Hardware and because the PN are interconnected in a fixed sequence. + + Monitor per allocated PN + - sdp_info : f_adc, fsub_type + - eth10g + +/band[0:1]/ait/ (AIT = adc input and timing) + Control: + - processing_enable : Start, stop + - si_sample_delay, Sample delay buffer : (unit16)ait_sample_delay[R_ant] + - si_waveform_generator (WG) : start at BSN, stop, freq[R_ant], ampl[R_ant], phs[R_ant] + + Monitor: + - sdp_status : 'stopped', 'running' or running and stable' → based on read BSN from BSN source being equal and incrementing correctly at all PN + - si_timing : (enum)ait_timing[R_ant] + 'not active' = no data + 'active' = there is data + 'active and stable and not aligned' = there is a stable flow of data since last time it was monitored, but the recovered LMFC is not aligned with the local FPGA_SYSREF + 'aligned' = there is a stable flow of data and it is aligned + 'aligned and stable' = there is a stable flow of data and it is aligned since the last time it was monitored + + Monitor on request: + - si_mean : (float)ait_mean[R_ant] + - si_sigma : (float)ait_sigma[R_ant] + - si_data_buffer : (sint16)ait_data_buffer[R_ant][N] (N = 1k samples) + - si_histogram : (uint32)ait_histogram[R_ant][N] (N = 1k bins) + +/band[0:1]/fsub/ (Fsub = subband filterbank) + Control: + - pfir_coefficients: FIR filter coefficients: (sint16)pfir_coefficients[N_taps*N_fft] + - subband_weights : (cint16)subband_weights[R_ant][N_sub] + - sst_input_select : (bool)sst_input_select, select calibrated or not calibrated for SST + - sst_offload_destination : SST UDP offload header + . Destination : (tuple)sst_offload_destination = (MAC address, IPv4 address, UDP port) + - sst_offload_enable : SST UDP offload : (bool)sst_offload_enable = on, off + + Monitor: None, if sst_offload = off, then the SST can be read via direct monitoring. + +/band[0:1]/bf/ (BF = beamformer) + Control: + - Subband select per beam: (uint16)beamlet_subband_select[N_beamlets] + - Beamformer weights for X polarization beams: (cint16)beamlet_weights_xx[N_ant][N_beamlets] + - Beamformer weights for Y polarization beams: (cint16)beamlet_weights_yy[N_ant][N_beamlets] + - BST UDP offload header + . Destination : (tuple)bst_offload_destination[N_beamsets] = (MAC address, IPv4 address, UDP port) + - BST UDP offload : (bool)bst_offload[N_beamsets] = on, off + + Monitor: None, if bst_offload = off, then the BST can be read via direct monitoring. + +/band[0:1]/ri/ (RI = ring) + +/band[0:1]/xc/ (XC = subband correlator) + +/band[0:1]/tbuf/ (Tbuf = Transient Buffer) + +/band[0:1]/tdet/ (Tdet = Transient Detector) + + +De SDP Translator is valt ook onder beheer EC2. + + + +################################################################################################### +# L3 ICD 11423 SDPTR - SDPFW + +ToD +The SDP maintains timing using the PPS and the sample clock. Hence SDP knows about seconds, but it does not know the time of day (ToD) since 1970 at the PPS. After power up SDP needs to get the ToD information from SC to be able to timestamp SDP output data. Both SC and SDP know the PPS grid. The SC has to provide the ToD information at PPS in one PPS interval and then SDP will apply that ToD in the next PPS interval. The ToD at the PPS is defined at the block sequence number (BSN) grid, using initial BSN and a BSN offset [Does SC care about BSN. IMHO that's SDP internal.] [AD-4]. The SC only has to provide the ToD once, because after that SDP can maintain the ToD by counting PPS pulses and sample clock periods. The SDP provides a monitoring point that reports the time since last PPS. [Do you really mean time since last PPS? What's the purpose? Assuming time distribution is done properly within the station, then all subsystem will receive the PPS at the same time. AFAIK, SC doesn't need to align with the PPS. Maybe you're referring to the ToD when the last PPS occurred? That could indeed be useful to discover inconsistencies between SC and SDP.] The SC can read this monitoring point to check whether is own PPS is aligned to the PPS in SDP. + +PPS monitoring +The monitoring of the status of the PPS in the SDP Firmware reveals: + + whether the PPS period is correct and stable. This allows SC to monitor whether processing by SDP is possible. If the period is wrong or not stable, then this indicates an issue between the STF and SDP. The stable interval is defined by the time since between monitoring requests by SC of the PPS period. [We have to be careful here, not to make the same mistakes as in APERTIF, where the misalignment detection generated a lot of false positives. What problem are we trying to solve here? Only when we've answered that question, it's time to ask ourselves what is the best solution?] + the number of sample clock cycles since last PPS, upon reception of the monitoring message. This allows SC to monitor the synchronisation between SC and SD + +BSN timing grid +The ToD information defines the BSN timing grid and shall consist of [AD-3]: [Again, isn't this all SDP internals? AFAIC, SC provides the ToD, only. Whatever SDP does internally with BSN is none of SC's business.] + + an initial BSN that counts subband periods T_sub, + a BSN offset in ADC sample periods T_adc relative to the initial PPS + + +SDPTR requirements: + + SC-SDP synchronisation. The Time of Day (ToD) in SC and the PPS in SDP shall be synchronous within a margin of ±100 ms, to ensure that real-time [There is no real-time communication.] M&C communication that starts in a certain PPS interval can also complete in that PPS interval. + + PPS monitoring. SC shall monitor the status of the PPS in the SDP Firmware, [Like I said before, this is IMHO the wrong (APERTIF) approach. Do we really want to go that route again?] as listed in LOFAR2-8852 - PPS monitoring. + + Time of Day (ToD) information at PPS. SC shall provide the initial ToD information to SDP within the pulse per second (PPS) interval that precedes the PPS that corresponds to that initial ToD. For subsequent PPS intervals the SDP maintains the ToD information by itself. + + Block Sequence Number (BSN) timing grid. SC shall provide the ToD information as defined in LOFAR2-8853 - BSN timing grid. + + Block Sequence Number (BSN) monitoring. The SC shall monitor the BSN monitoring points in the SDP Firmware to monitor whether the processing in SDP is synchronous. + + SDP processing start and restart.The SC shall start the SDP processing by providing the ToD information to SDP. The SDP processing can be started or restarted at any PPS (i.e. no need to wait for an even second like in LOFAR1). + +SDPFW requirements: + + SC-SDP synchronisation. The Timeof Day (ToD) in SC and the PPS in SDP shall be synchronous within a margin of ±100 ms, to ensure that real-time M&C communication that starts in a certain PPS interval can also complete in that PPS interval. + + PPS monitor. The SDP Firmware shall report the status of the PPS upon request, as listed in LOFAR2-8852 - PPS monitoring. + + Time of Day (ToD) information at PPS. SDP Firmware shall apply the received ToD information at the next pulse per second (PPS). For subsequent PPS intervals the SDP Firmware shall maintain the ToD information by counting sample clock cycles. + + Block Sequence Number (BSN) timing grid. [I'm not sure why this should be in the ICD. Like I said, BSN is IMHO SDP internal.] SDP shall apply the ToD information as defined in LOFAR2-8853 - BSN timing grid. + + Block Sequence Number (BSN) monitor. The SDP Firmware shall provide monitoring points that show the status of the timing within a processing node (PN) and the status of the timing alignment between PN within SDP. + + SDP processing start and restart.The SDP shall start the SDP processing at the next PPS, when it has received the ToD information from SC. The SDP processing can be started or restarted at any PPS. + + +I.011211.SDPTR.SDPFW d.7, Data - Synchronous control using time activated control +In LOFAR1LOFAR1: RSP driver moet 2 x een setting sturen naar het RSP board, vanwege het per seconde swappen. Dit moeten we voorkomen in LOFAR2. +Time activated control is control that takes effect in the SDP Firmware at a scheduled instant. The scheduled instant uses the block sequence number (BSN) grid [AD-4], so it is not restricted to the seconds grid. Time activated control implies that the SDP Firmware uses a dual page register to store the control data. One page can accept new control data, while the control data in the other page is being used. At a trigger the pages swap or shift, dependent on whether the register consist of RAM or logic. In LOFAR1 the dual page registers swap at the PPS, independent of whether the RSP driver had updated the setting. This causes that the RSP driver has to set the same setting twice in two successive PPS intervals, to avoid that the page swap of a RAM register results in using old page values. For LOFAR2.0 this is avoided by letting the SC control when the register in SDP should swap, instead of at any PPS. Therefore the time activated control in LOFAR2.0 uses a trigger that is scheduled by SC [I don't think I agree with that approach. Unless I completely misunderstand the issue here, but why should SC have to ensure SDP's internal consistency?!?! This definitely needs to be discussed.] instead of by the PPS. The scheduled instant for the trigger is typically in the future. If the scheduled instant is in the past, then SDP will not do the page swap. If the scheduled instant is at BSN = 0 then SDP will do the page swap immediately. The trigger can be rescheduled by SC as long as its time instant has not yet past. SC has to provide a trigger instant per page swap, otherwise the pages do not swap. Hence SC has full control over when and how often time activated control is applied. +Linked Work Items has parent: LOFAR2-8812 - Station Control (L3-SC) - SDP Translator (L4-SDPTR) +[Create Link] List of time-activated control pointsThe time activated control is used for controlling: + + starting of the subband correlator, + starting the waveform generators + +SC requirements: + Time activated control. For time activated control the SC shall provide the control data and a timestamp. The timestamp schedules an instant at the BSN grid. The SDP Firmware shall activate the control at that timestamp. The SC shall use time-activated control for the control points listed in LOFAR2-8854 - List of time-activated control points. + +SDP requirements: + + Time activated control. For time activated control the SC shall provide the control data and a timestamp. The timestamp schedules an instant at the BSN grid. The SDP Firmware shall activate the control at that timestamp. The SDP shall use time-activated control for the control points listed in LOFAR2-8854 - List of time-activated control points. + ################################################################################################### # L2 ICD 11207 RCU2S-SDP