From 41236949bdaf529a4dd92581d0a714ece2158404 Mon Sep 17 00:00:00 2001 From: Eric Kooistra <kooistra@astron.nl> Date: Wed, 12 Feb 2020 14:59:56 +0100 Subject: [PATCH] Wrote SDP design document on timing based on station2_sdp_firmware_design.txt. Text snippets that were not yet used are kept in station2_tmp.txt. Hence station2_sdp_firmware_design.txt is now obsolete. --- .../doc/prestudy/station2_sdp_timing.txt | 53 ++++++++++--------- .../lofar2/doc/prestudy/station2_tmp.txt | 37 +++++++++++++ 2 files changed, 66 insertions(+), 24 deletions(-) create mode 100644 applications/lofar2/doc/prestudy/station2_tmp.txt diff --git a/applications/lofar2/doc/prestudy/station2_sdp_timing.txt b/applications/lofar2/doc/prestudy/station2_sdp_timing.txt index 060a82f773..55318c326c 100644 --- a/applications/lofar2/doc/prestudy/station2_sdp_timing.txt +++ b/applications/lofar2/doc/prestudy/station2_sdp_timing.txt @@ -95,13 +95,13 @@ and any M&C upon the data, because: The actual resolution T_sub of the Station BSN in LOFAR2 depends on the ADC sample frequency and on the subband filterbank: - N_blk T_adc T_sub T_sub_i - 1024 * 5 ns = 5120 ns = 25600 [0.2 ns] for critical sampled filterbank at 200 MHz - 1024 * 6.4 ns = 6553.6 ns = 32768 [0.2 ns] for critical sampled filterbank at 160 MHz - 864 * 5 ns = 4320 ns = 21600 [0.2 ns] for oversampled filterbank R_os = 32/27 = 1.185 at 200 MHz - 864 * 6.4 ns = 5529.6 ns = 27648 [0.2 ns] for oversampled filterbank R_os = 32/27 = 1.185 at 160 MHz - 800 * 5 ns = 4000 ns = 20000 [0.2 ns] for oversampled filterbank R_os = 32/25 = 1.28 at 200 MHz - 800 * 6.4 ns = 5120 ns = 25600 [0.2 ns] for oversampled filterbank R_os = 32/25 = 1.28 at 160 MHz + N_blk T_adc T_sub + 1024 * 5 ns = 5120 ns for critical sampled filterbank at 200 MHz + 1024 * 6.25 ns = 6400 ns for critical sampled filterbank at 160 MHz + 864 * 5 ns = 4320 ns for oversampled filterbank R_os = 32/27 = 1.185 at 200 MHz + 864 * 6.25 ns = 5400 ns for oversampled filterbank R_os = 32/27 = 1.185 at 160 MHz + 800 * 5 ns = 4000 ns for oversampled filterbank R_os = 32/25 = 1.28 at 200 MHz + 800 * 6.25 ns = 5000 ns for oversampled filterbank R_os = 32/25 = 1.28 at 160 MHz In LOFAR2 the timestamp should be independent of: @@ -110,35 +110,39 @@ In LOFAR2 the timestamp should be independent of: If T_sub was fixed then T_sub could be used as timestamp resolution (like in APERTIF). However T_sub depends on the type of subband filterbank with a resolution of T_adc. If T_adc was fixed then T_adc could be used -as timestamp resolution. However T_adc depends on the sample clock rate. Therefore the timestamp resolution -needs to be as fine as the greatest common time resolution of T_adc = 5 ns and T_adc = 6.4 ns, which is 0.2 ns. -A 64 bit timestamp with 0.2 ns resolution can count 2**64 / (365.25 * 24 * 3600 / 0.2e-9) = 116 years. Hence -for t_epoch = 1970 this is until 2086, which is sufficient for the lifetime of LOFAR2.0. Internally in SDP +as timestamp resolution. However T_adc depends on the sample clock rate. At most the timestamp resolution +needs to be as fine as the greatest common time resolution of T_adc = 5 ns and T_adc = 6.25 ns, which is +0.25 ns. However typical R_os = p/q have p is a power of 2 and p <= 64, because N_FFT = 1024 = 2**10. Hence +N_FFT / p >= 16, so typical subband periods can always be expressed as a multiple of 16* 6.25 = 100 ns or +16 * 5 = 80 ns. Therefore the minimum required time stamp resolution is gcd(100, 80) = 20 ns. For human +readability choose 1 ns as timestamp resolution +A 64 bit unsigned timestamp with 1 ns resolution can count 2**64 / (365.25 * 24 * 3600 / 1e-9) = 584 years. +Hence for t_epoch = 1970 this is more than enough for the lifetime of LOFAR2.0. Internally in SDP firmware use the BSN to count T_sub. Externally at the SDP interface use timestamp values with a resolution -of 0.2 ns such that they are: +of 1 ns such that they are: * integer values, and * independent of the sample period. -The actual timestamp in fractional seconds of 0.2 ns follows from: +The actual timestamp in units of 1 ns follows from: - timestamp = Station BSN * T_sub_i * 0.2 [ns]. + timestamp = Station BSN * T_sub [ns]. -The BSN and T_sub_i can be specified as: +The BSN and T_sub can be specified as: -- single 64 bit integer timestamp value of BSN * T_sub_i [0.2 ns] -- two separate fields with an incrementing BSN and resolution given by T_sub_i [0.2 ns] +- single 64 bit unsigned integer timestamp value of BSN * T_sub [ns] +- two separate fields with an incrementing BSN and resolution given by T_sub [ns] -To cover 116 years for a BSN with smallest T_sub = 4000 ns for R_os = 32/25 = 1.28 requires: +To cover 100 years for a BSN with smallest T_sub = 4000 ns for R_os = 32/25 = 1.28 requires: - log2( 116 * (365.25 * 24 * 3600 / 4000e-9) ) = 49.7, so 50 bits + log2( 100 * (365.25 * 24 * 3600 / 4000e-9) ) = 49.7, so 50 bits Therefore allocate 64b in a packet header to send the BSN information. The BSN and timestamp are direcly -related via T_sub_i, but the advantage of providing the BSN separately is that it increments by 1 for +related via T_sub, but the advantage of providing the BSN separately is that it increments by 1 for each block period T_sub, so it can be used as block index. -The range of T_sub is 4000 ns - 5120 ns, so the range of T_sub_i is 20000 - 25600. These T_sub_i values -can be covered in a 16 bit number. Alternatively T_sub_i can be derived from the four possible +The range of T_sub is 4000 ns - 5120 ns. These T_sub values +can be covered in a 16 bit number. Alternatively T_sub can be derived from the four possible combinations of f_adc = 200M or 160M and R_os = 1 or 32/25, that can be represented with 2 bits. @@ -148,7 +152,8 @@ combinations of f_adc = 200M or 160M and R_os = 1 or 32/25, that can be represen ******************************************************************************* Together the initial BSN, counting blocks and the order of the data within a block uniquely define the timing -of the data in a Station. However counting blocks is not sufficient to maintain the data timing, because: +of the data in a Station. However counting blocks is not sufficient to maintain the data timing, because +there can be gaps due to: - The data flow at the SDP input may be stopped and restarted, - at the external interfaces of the FPGAs in SDP it is possible that blocks of data get lost. @@ -172,7 +177,7 @@ FPGA in the SDP firmware has a BSN source, that all run synchronously within a S Stations, because they have been started by the external PPS from the Timing Distributor (TD). The sync is a periodic signal with period larger than the maximum latency of the data within the SDP, to ensure that a sync at any FPGA refers to the same time instant. If somewhere in an FPGA the sync comes along, then -the BSN can be recreated by directly using the BSN that was held at the sync by the BSN source and start +the BSN can be recreated by directly using the BSN that was held at the sync by the local BSN source and start continue counting blocks from there. diff --git a/applications/lofar2/doc/prestudy/station2_tmp.txt b/applications/lofar2/doc/prestudy/station2_tmp.txt new file mode 100644 index 0000000000..ca0bc98aa3 --- /dev/null +++ b/applications/lofar2/doc/prestudy/station2_tmp.txt @@ -0,0 +1,37 @@ +Remaining texts that have not been used yet in official SDP detailed design documents + +******************************************************************************* +* From station2_sdp_timing +******************************************************************************* + +- It is not necessary to represent fine group delays of digital filters or analogue electronics and + cables in the BSN, because these delays are all accounted for after calibration. + . Course group delays and cable delay differences can be compensated for in steps to T_adc via the signal + input buffer of every ADC input in SDP. + . Fine group delay differences within a Station can be calibrated via the subband calibration weights. +- Group delay differences between Stations need to be calibrated at CEP, and can be compensated at CEP or + at Station via the input buffers and the subband calibration weights. + + + +Key ideas: +- Use Ethernet CRC and DP CRC to ensure detection of packet errors and to ensure error free blocks + within FPGA firmware + + + RCU2 Subband Ring + PFB + + data data + data ------> BSN --------> Move, -------> Packet + PPSH ------> source sync DSP sync encoding + BSN .........> BSN ring + + Ring BF, XC + data data data + Packet --------> Validate --> Validate --> BSN --------> Move, --------> Packet + decoding sync CRC BSN aligner sync DSP sync encoding + ring BSN .......................................................> BSN output + + + -- GitLab