From 3a2f002c9d2111671e994a64e32cdfef01fa7bfb Mon Sep 17 00:00:00 2001
From: Eric Kooistra <kooistra@astron.nl>
Date: Wed, 21 Jun 2023 13:49:24 +0200
Subject: [PATCH] Move transient detection notes to separate txt file. Added
 TBuf development planning.

---
 .../station2_sdp_transient_buffer.txt         | 294 ++++++++++++++++--
 1 file changed, 262 insertions(+), 32 deletions(-)

diff --git a/applications/lofar2/doc/prestudy/station2_sdp_transient_buffer.txt b/applications/lofar2/doc/prestudy/station2_sdp_transient_buffer.txt
index e82291dea2..b46f13b09e 100644
--- a/applications/lofar2/doc/prestudy/station2_sdp_transient_buffer.txt
+++ b/applications/lofar2/doc/prestudy/station2_sdp_transient_buffer.txt
@@ -12,7 +12,7 @@ Detailed design: Transient Buffer (TBuf) function for LIFT project
 8) Crossbar
 10) Planning
 11) Transient detection (TDet) Design
-12) Cosmic ray
+12) Development planning
 
 
 References:
@@ -444,7 +444,7 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
               dump_nof_pages = 0
 
           # Set packets that will be dumped by SDPTR
-          reg_dump_start_page_RW = dump_start_page % (page_max + 1)
+          reg_dump_start_page_RW = dump_start_page % nof_pages_in_buffer
           reg_dump_nof_pages_RW = dump_nof_pages
 
     . reg_memory_read_nof_packets_R
@@ -656,7 +656,7 @@ Het is gewoon een json-bestandje dat je naast een databestand met alleen complex
   mux right, then apply drv_copi with wr_not_rd and burstbegin.
 
 
-10) Planning
+9) Design planning
 
 - ICD STAT-CEP --> tbuf packet format
 - ICD SC-SDP --> OPC-UA CP and MP
@@ -687,33 +687,263 @@ The selection between recording all or half of the antennas per FPGA has the fol
 - requires decision making at higher software, configuration or user control levels, to decide which selection to use
 
 
-11) Transient detection (TDet) Design
 
-- no self triggering yet for MVP
-- Pulse detection messages contain event info and timestamp, which is still
-  useful for ligthning science even without dumping buffer.
-
-- will use Hilbert transform of real input and > 30MHz BPF
-  https://nl.mathworks.com/help/signal/ug/single-sideband-modulation-via-the-hilbert-transform.html
-  For the FIR Hilbert transformer we will use an odd length filter which is
-  computationally more efficient than an even length filter. Albeit even
-  length filters enjoy smaller passband errors. The savings in odd length
-  filters is a result that these filters have several of the coefficients that
-  are zero. Also, using an odd length filter will require a shift by an
-  integer time delay, as opposed to a fractional time delay that is required
-  by an even length filter. For an odd length filter, the magnitude response
-  of a Hilbert Transformer is zero for w=0 and w=Ï€. For even length filers the
-  magnitude response doesn't have to be 0 at π, therefore they have increased
-  bandwidths. So for odd length filters the useful bandwidth is limited to
-  0 < w < π.
-
-- https://en.wikipedia.org/wiki/Analytic_signal --> Smith, J.O. "Analytic Signals and Hilbert Transform Filters", in Mathematics of the Discrete Fourier Transform (DFT) with Audio Applications, Second Edition
-- https://nl.mathworks.com/help/dsp/ug/envelope-detection.html
-
-- forced trigger message every 30 s, for logging of default radio environment --> send to LCU2
-  radio triggered message once per hour
-
-
-12) Cosmic ray
-* Katie Mulrey (RU, Cosimc Ray group)
-* Stijn Buitink (VUB, Cosmic Ray group)
+10) Development planning
+
+[1] FW design decisions, https://support.astron.nl/confluence/display/L2M/L4+SDPFW+Decision%3A+Transient+buffer+raw+data
+[2] FW detailed design, https://support.astron.nl/confluence/display/L2M/L5+SDPFW+Design+Document%3A+Transient+buffer+raw+data
+[3] ICD SC-SDP, https://support.astron.nl/confluence/display/L2M/L2+STAT+Decision%3A+SC+-+SDP+OPC-UA+interface
+[4] ICD SDPTR-SDPFW, https://support.astron.nl/confluence/display/L2M/L3+SDP+Decision%3A+SDPTR+-+SDPFW+register+map+interface
+[5] ICD STAT-CEP, https://plm.astron.nl/polarion/#/project/LOFAR2System/wiki/L1%20Interface%20Control%20Documents/STAT%20to%20CEP%20ICD
+[6] https://support.astron.nl/confluence/display/L2M/L3+SDP+Testing+Notebook%3A+Transient+buffer
+
+a) SDPTR
+
+Read TBuf documentation
+  done: when FW design and ICDs are clear
+
+#######################################
+# CP and MP
+#######################################
+
+Add direct access CP and MP for TBuf
+ddr4 memory interface
+signal_input
+recording
+output
+ring
+
+Add composite CP and MP for dump bit rate
+  - convert bps to inter packet gap and unit test, see section in [4]
+
+Add composite CP and MP for dump page range
+  - convert requested dump interval to actual dump interval and unit test, see section in [4]
+
+Access CP and MP of TBuf for one node on HW
+  done: when all CP and MP can be accessed via OPC-UA and SDPTR using sdp_rw.py
+
+#######################################
+# tbuf_dump script
+#######################################
+
+Create tbuf_dump script for recording on one node
+    similar as *_stream.py scripts for statistics and beamlets
+    follow dynamic behavior template for recording and dumping in ICD [4]
+    extend simpel stub for recording a fixed interval when recording is enabled
+    Done: when tbuf_dump script can setup, recording and freeze in simulation
+
+Extend tbuf_dump script with dumping on one node
+  - follow dynamic behavior template for recording and dumping in ICD [4]
+  - support different dump intervals
+  - extend simpel stub for actual dumping interval and MP counters
+  done: when tbuf_dump script can dump intervals in simulation
+
+Verify tbuf_dump script with one node in simulation
+  - verify MP counters
+  done: when tbuf_dump script and stub are part of SDPTR SW CICD
+
+Verify tbuf_dump M&C with one node on HW
+  - verify MP counters
+  done: when tbuf_dump script can do a record, freeze and dump.
+
+Support tbuf packet decoding in stream_reader.py
+  - no need for stream_reader stub in simulation (?), because stream_reader has been verified already on HW for beamlets
+  done: when stream_reader.py can unpack tbuf packets
+
+Verify tbuf_dump stream header with one node on HW
+  done: when tbuf_dump script can verify the dumped packet headers
+
+Verify tbuf_dump stream data with one node on HW
+  done: when tbuf_dump script can verify the dumped packet data (based on expected amplitude level)
+
+Extend tbuf_dump script for multiple nodes in simulation
+  - setup CP for tbuf ring lane
+  - support dumping from a list of nodes
+  done: when tbuf_dump script with multiple SDPFW node stubs works in SDPTR SW CICD
+
+Verify tbuf_dump script for multiple nodes with ring on HW
+  done: when tbuf_dump script can do a record, freeze and dump and read all dumped packet headers
+
+#######################################
+# tbuf in CICD
+#######################################
+
+Maintain tbuf_dump SDPTR SW CICD test in simulation
+  done: when tbuf in SDPTR SW CICD test in simulation still runs ok after an update
+
+Setup unb2c HW platform for SDP CICD test on HW
+  done: when SDP CICD test can run with at least one unb2c every weekend
+
+Add tbuf_dump script to SDP CICD test on HW
+  done: when tbuf_dump test is part of SDP CICD test on unb2c
+
+Maintain tbuf_dump SDP CICD test on HW
+  done: when tbuf in SDP CICD test on HW still runs ok after an update
+
+
+b) SDPFW
+
+Review TBuf documentation
+  done: when FW design and ICDs are clear
+
+#######################################
+# tbuf coding and initial synthesis
+#######################################
+
+Code sdp_tbuf_registers.vhd for REG_TBUF
+  - sdp_tbuf_registers.vhd --> sdp.peripheral.yaml
+  done: when HDL code compiles
+
+Prepare tbuf firmware design revision for one node
+  - node_sdp_transient_buffer.vhd with MM connected for:
+    . sdp_tbuf_registers.vhd
+  - lofar2_unb2c_sdp_station_tbuf_one.vhd -->
+    . lofar2_unb2c_sdp_station.yaml
+    . qsys_lofar2_unb2c_sdp_station.qsys
+    . mmm_lofar2_unb2c_sdp_station.vhd
+  done: when VHDL code compiles and generated MMAP is ok
+
+Add tbuf remaining MM ports for one node
+  - node_sdp_transient_buffer.vhd with MM connected for:
+    . dp_rsn_source.vhd
+    . dp_bsn_monitor_v2.vhd
+    . dp_strobe_total_count.vhd
+    . io_ddr.vhd
+    . sdp_tbuf_output.vhd skeleton only with:
+      - dp_offload_tx_v3.vhd : app header
+      - dp_offload_tx_v3.vhd : network header
+      - mms_dp_xonoff
+  - lofar2_unb2c_sdp_station_tbuf_one.vhd -->
+    . lofar2_unb2c_sdp_station.yaml
+    . qsys_lofar2_unb2c_sdp_station.qsys
+    . mmm_lofar2_unb2c_sdp_station.vhd
+  done: when VHDL code compiles and generated MMAP is ok
+
+Code tbuf firmware for record all
+  - sdp_tbuf_pkg.vhd
+  - sdp_tbuf_arbiter.vhd
+  - sdp_tbuf_writer.vhd
+  - sdp_tbuf_reader.vhd
+  - sdp_tbuf_output.vhd
+  - node_sdp_transient_buffer.vhd
+  done: when VHDL code is complete and compiles (no verification yet)
+
+Try synthesize tbuf design revision for one node
+  done:
+  . when synthesis of lofar2_unb2c_sdp_station_tbuf_one.vhd yields expected resource usage and meets timing
+  . report resource usage in [6]
+
+Access CP and MP for DDR4 in tbuf design on HW
+  . access the CP and MP via OPC-UA and SDPTR using sdp_rw.py
+  . read MP for DDR4 to check that it is available using sdp_rw.py
+  done: when all CP and MP can be accessed via OPC-UA and SDPTR and the DDR4 interface is calibrated
+
+#######################################
+# tbuf verification in simulation
+#######################################
+
+Prepare tbuf test bench on one node
+  - tb_sdp_tbuf_pkg.vhd
+  - tb_lofar2_unb2c_sdp_station_tbuf_one.vhd
+  done: when VHDL code compiles
+
+Verify tbuf recording in simulation
+  done: when sdp_tbuf_writer in design can record to DDR4
+
+Verify tbuf reading in simulation
+  done: when sdp_tbuf_reader in design can read from DDR4
+
+Verify tbuf recording and dumping (headers) in simulation
+  - p_verify_dump_header
+  done: when tbuf loop recording and dumping works in simulation and the headers are ok
+
+Verify tbuf recording and dumping (payloads) in simulation
+  - p_verify_dump_data
+  done: when tbuf loop recording and dumping works in simulation and the payloads are ok
+
+Synthesize tbuf design revision for one node
+  done: when synthesis of lofar2_unb2c_sdp_station_tbuf_one.vhd yields expected resource usage and meets timing
+  . report resource usage in [6]
+
+Reconsider dp_repack_data to 504b instead of 512b
+  . see TODO in dp_repack_data section in [2]
+  . necessary if there are synthesis issues
+  . requires update of FW design document
+
+Verify tbuf MP total strobe counters in simulation
+  done: when MP total strobe counters are correct
+
+Verify tbuf with small inter packet gap in simulation
+  done: when no FIFO overflow occurs and MP report dropped packets
+
+Verify tbuf output in combination with beamlet output in simulation
+  - see TBuf output in combination with beamlet output section in [2]
+  - see TODO in packet transport flow control section in [2]
+  done: when no FIFO overflow occurs and MP report dropped packets
+
+#######################################
+# tbuf verification on HW
+#######################################
+
+Access CP and MP of tbuf design revision for one node on HW
+  done: when all CP and MP can be accessed via OPC-UA and SDPTR
+
+Verify tbuf output on HW
+  - use tbuf_dump script
+  done: when tbuf dumps the expected range of packets, report results in [6]
+
+Verify tbuf output in combination with beamlet output on HW
+  - use beamlet_stream.py script to enable beamlet output
+  - use tbuf_dump script
+  done: when tbuf dumps the expected range of packets, report results in [6]
+
+
+#######################################
+# tbuf with ring
+#######################################
+
+Prepare tbuf firmware design revision with ring
+  - connect ring_lane
+  - lofar2_unb2c_sdp_station_tbuf_ring.vhd -->
+    . lofar2_unb2c_sdp_station.yaml
+    . qsys_lofar2_unb2c_sdp_station.qsys
+    . mmm_lofar2_unb2c_sdp_station.vhd
+  done: when VHDL code compiles and generated MMAP is ok
+
+Synthesize tbuf design revision with ring
+  done:
+  . when synthesis of lofar2_unb2c_sdp_station_tbuf_ring.vhd yields expected resource usage and meets timing
+  . report resource usage in [6]
+
+Verify tbuf recording and dumping via ring in simulation
+  - see TBuf with ring transport section in [2]
+  done: when tbuf loop recording and dumping via ring works in simulation
+
+Change c_err_bi = 6 for all ring interfaces
+  - see TODO in ring interface section in [2]
+  done: when VHDL regression tests still run ok.
+
+Change RX_select in sdp_station.vhd
+  - see TODO in ring interface section in [2]
+  done: when VHDL regression tests still run ok.
+
+Verify tbuf recording and dumping via ring on HW
+  done: when tbuf loop recording and dumping works on HW, report results in [6]
+
+
+c) Support record all or half of the antenna inputs
+
+#######################################
+# tbuf support record all or half
+#######################################
+
+Extend tbuf firmware with record half support
+  done: when VHDL code is complete and compiles
+
+Verify tbuf record all or half in simulation
+  done: when tbuf loop recording and dumping works ok for alternatingly all or half
+
+Extend tbuf_dump script with support for record all or half
+
+Verify tbuf record all or half on hardware
+
-- 
GitLab