Skip to content
Snippets Groups Projects
Commit 20fe6def authored by Eric Kooistra's avatar Eric Kooistra
Browse files

Updates after competing ICD and FW design.

parent 43bb282f
No related branches found
No related tags found
No related merge requests found
Pipeline #49894 passed
......@@ -12,6 +12,7 @@ Detailed design: Transient Buffer (TBuf) function for LIFT project
8) Crossbar
10) Planning
11) Transient detection (TDet) Design
12) Cosmic ray
References:
......@@ -92,7 +93,7 @@ Design decision 16GByte DDR4 na L2SDP-854, 850
- Buffer lengte versus nof antennes
- Self trigger
The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW yet. I was n
The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW yet.
3) TBB (Transient Buffer Board) LOFAR1
- From 2.3 in [4]
......@@ -145,11 +146,11 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
- timing
. Use sample sequence number (RSN) or mem_bsn:
- RSN increments by c_tbuf_nof_samples_per_page = 8176
- mem_bsn increments by 1 per page, so per block of c_tbuf_nof_samples_per_page = 8176.
- RSN increments by c_sdp_tbuf_nof_samples_per_block = 8176
- mem_bsn increments by 1 per page, so per block of c_sdp_tbuf_nof_samples_per_block = 8176.
. RSN counts sample periods (5 ns) since t_epoch = 1970, can fit
2**64 / (365.25 * 24 * 3600 / 5e-9) > 2922 years
. TBuf uses sop and eop to mark c_tbuf_nof_samples_per_page = 8176
. TBuf uses sop and eop to mark c_sdp_tbuf_nof_samples_per_block = 8176
. TBuf does not need sync ?
. Start RSN or mem BSN at same time as SDP BSN by FPGA_processing_enable_RW.
......@@ -207,10 +208,10 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
that DDR4-I can wrap without a gap or extend to DDR-II without a gap.
- RSN = 64b = 8B
- 14b packed data
. c_tbuf_nof_samples_per_page = 8K - 8 - 8 = 8192 - 16 = 8176 B / 14b = 4672
. c_sdp_tbuf_nof_samples_per_block = 8K - 8 - 8 = 8192 - 16 = 8176 B / 14b = 4672
. 4672 * 5 ns = 23.36 us per page, so ~42.8 pages / ms
- CRC = 64b = 8B, use 64b CRC to match 64b words, no need to reduce stored
CRC to less bits, because c_tbuf_nof_samples_per_page fits preferrably a multiple
CRC to less bits, because c_sdp_tbuf_nof_samples_per_block fits preferrably a multiple
of 64b words.
- Memory storage overhead is 16 / 8192 = 0.2%
......@@ -270,10 +271,8 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
. SDPFW: no need for MM interface:
use nof_clk_per_sync from dp_bsn_source also for dp_rsn_source
start RSN source when dp_bsn_source is (re)started, derive RSN from BSN
. c_tbuf_nof_samples_per_block =
c_tbuf_nof_samples_per_page =
c_tbuf_nof_samples_per_packet = 2000
. nof_block_per_sync = nof_clk_per_sync / c_tbuf_nof_samples_per_packet
. c_sdp_tbuf_nof_samples_per_block = 2000
. nof_block_per_sync = nof_clk_per_sync / c_sdp_tbuf_nof_samples_per_block
= 100k for sync interval 1 s at FPGA_sdp_info_f_adc_R = 200 MHz
= 80k for sync interval 1 s at FPGA_sdp_info_f_adc_R = 160 MHz
......@@ -306,31 +305,28 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
* Dumping:
. FPGA_tbuf_dump_inter_packet_time_RW
Wait time in seconds between packets send to CEP. SDPTR translates float time into
T_adc units.
T_adc units --> reg_dump_inter_packet_time_RW
. FPGA_tbuf_dump_start_timestamp_RW[pn]
FPGA_tbuf_dump_start_timestamp_R[pn] = reg_dump_start_rsn_RW * T_adc
FPGA_tbuf_dump_start_timestamp_R[pn] = dump_start_rsn * T_adc in SDPTR
Timestamp of first packet that will be dumped. Equals FPGA_tbuf_dump_start_timestamp_RW
if it exists in the buffer.
. FPGA_tbuf_dump_end_timestamp_RW[pn]
FPGA_tbuf_dump_end_timestamp_R[pn]
FPGA_tbuf_dump_end_timestamp_R[pn] = reg_dump_end_rsn_RW * T_adc
Timestamp of last packet that will be dumped. Equals FPGA_tbuf_dump_end_timestamp_RW
if it exists in the buffer.
. FPGA_tbuf_dump_nof_packets_R[pn] = reg_dump_nof_pages_RW
Amount of packets that will be dumped for range that is requested via FPGA_tbuf_dump_start_timestamp_RW
and FPGA_tbuf_dump_end_timestamp_RW. Forced to 0 if requested tange is not in buffer.
Each packet contains c_tbuf_nof_samples_per_packet = 2000 raw samples. This corresponds
to c_tbuf_nof_samples_per_packet * T_adc = 2000 * 5 ns = 10 us or 2000 * 6.25 ns = 12.5 us
of raw sample data per packet. Hence it takes about 100 packets to transport 1 ms of
raw sample data for one antenna input, and about 100000 packets to transport 1 s.
Amount of packets that will be dumped per antenna input for range that is requested via
FPGA_tbuf_dump_start_timestamp_RW and FPGA_tbuf_dump_end_timestamp_RW. Forced to 0 if
requested range is not in buffer.
. FPGA_tbuf_dump_enable_RW[pn][ai]
FPGA_tbuf_dump_enable_R[pn][ai] = value of FPGA_tbuf_dump_enable_RW in SDPFW
True = start / keep dumping packets for the requested ai, dumping stops when all
packets have been dumped, it is not necessary to explicitely stop the dumping.
- when busy then FPGA_tbuf_memory_read_nof_packets_R increments until it reaches value of
FPGA_tbuf_dump_nof_packets_R
- when finished then FPGA_tbuf_memory_remaining_nof_packets_R = 0
A_en * FPGA_tbuf_dump_nof_packets_R
- when finished then FPGA_tbuf_memory_read_nof_packets_R == FPGA_tbuf_dump_nof_packets_R
False = RW: stop dumping packets immediately, R: dumping has stopped
SDPFW:
- loops over one or multiple ai
......@@ -338,23 +334,6 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
- takes care that only one global pn is selected at a time to avoid 10GbE link overload
- loops global pn, ai via FPGA_tbuf_dump_enable_RW
. FPGA_tbuf_memory_remaining_nof_packets_R[pn][ai]
Number of packets that still need to be dumped,
starting at reg_dump_nof_pages_RW and then decrementing down to 0, when
FPGA_tbuf_dump_enable_RW is True. If it does not reach 0 then there is something wrong in SDPFW.
FPGA_tbuf_memory_read_nof_packets_R[pn][ai]
Number of packets that have been read so far
FPGA_tbuf_memory_dropped_nof_packets_R[pn][ai]
Number of packets that have been read and had a CRC error,
count per ai, because each ai has own CRC
FPGA_tbuf_memory_dumped_nof_packets_R[pn][ai]
Number of packets that have been dumped, packets with a read error are not passed on for dump
= FPGA_tbuf_memory_read_nof_packets_R - FPGA_tbuf_memory_dropped_nof_packets_R,
Not: FPGA_tbuf_clear_total_counts_RW[pn] --> clear all TBuf total counts in pn,
total count not needed if MP is read after every dump
Not: FPGA_tbuf_dump_enable_RW[pn] -- antenna index
Not: Fixed buffer sizes, so no need to allocate pages per ai.
......@@ -369,12 +348,9 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
Enable output on the pn at end of ring that has interface to CEP.
* 10GbE output monitor:
. FPGA_tbuf_output_clear_counts_RW[pn]
Use dp_strobe_total_count with MM clear register rather than in_clr = revt(reg_output_enable_RW) and
rather then dp_bsn_monitor with sync = revt(reg_output_enable_RW).
. FPGA_tbuf_output_nof_packets_R[pn]
Number of dump packets that have been output while FPGA_tbuf_output_enable_RW = True,
reset to 0 when FPGA_tbuf_output_clear_counts_RW = True event
. FPGA_tbuf_output_nof_packets_R[pn]
Number of dump packets that have been output while FPGA_tbuf_output_enable_RW = True,
reset to 0 when FPGA_tbuf_clear_counts_RW = True event
* Memory DDR4:
With streaming use of io_ddr then the dvr_wr_flush_en = '0' (so ctlr_wr_flush_en = 0 in MP), because
......@@ -392,14 +368,14 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
ctlr_rst_out_i & ctlr_wr_flush_en & ctlr_tech_miso.waitrequest_n & ctlr_tech_miso.done, c_mem_reg_dat_w);
Use ddr_ prefix for DDR4 in slot I, in case in future DDR4 slot II is also used, then use e.g. ddr_ii for that module.
. FPGA_ddr_calibrated_R = ctlr_tech_miso.done = cal_ok && !cal_fail -->
. FPGA_tbuf_ddr_calibrated_R = ctlr_tech_miso.done = cal_ok && !cal_fail -->
True is ddr4 memory is available,
False is ddr4 calibration failed or ddr4 not present.
. FPGA_ddr_wr_fifo_full_R = wr_fifo_full_reg, True if write FIFO to ddr4 memory got full since last MP read, else False. Should remain False.
. FPGA_ddr_rd_fifo_full_R = rd_fifo_full_reg, True if read FIFO from ddr4 memory got full since last MP read, else False. Should remain False.
. FPGA_ddr_wr_fifo_usedw_R = ctlr_wr_fifo_usedw, current fill level of write FIFO to ddr4 memory in number of 512b words, should be 0 when not recording
. FPGA_ddr_rd_fifo_usedw_R = ctlr_rd_fifo_usedw, current fill level of read FIFO from ddr4 memory in number of 512b words, should be 0 when not dumping
. FPGA_tbuf_ddr_wr_fifo_full_R = wr_fifo_full_reg, True if write FIFO to ddr4 memory got full since last MP read, else False. Should remain False.
. FPGA_tbuf_ddr_rd_fifo_full_R = rd_fifo_full_reg, True if read FIFO from ddr4 memory got full since last MP read, else False. Should remain False.
. FPGA_tbuf_ddr_wr_fifo_usedw_R = ctlr_wr_fifo_usedw, current fill level of write FIFO to ddr4 memory in number of 512b words, should be 0 when not recording
. FPGA_tbuf_ddr_rd_fifo_usedw_R = ctlr_rd_fifo_usedw, current fill level of read FIFO from ddr4 memory in number of 512b words, should be 0 when not dumping
There is no MP for drv_miso.done, could show hanging io_ddr_driver. All other fields in drv_miso are unused ('X').
......@@ -411,7 +387,7 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
- ctlr_tech_miso.waitrequest_n ? could show hanging io_ddr_driver or refresh cycles
* Memory buffer:
REG_TBUF_RAW new in node_sdp_transient_buffer_raw.vhd
REG_TBUF new in node_sdp_transient_buffer_raw.vhd
. reg_record_all_RW = FPGA_tbuf_reg_record_all_RW
True = record all antenna inputs,
False = record only half of the antenna inputs, the once that have even index.
......@@ -420,7 +396,7 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
False = pn is frozen
Constants, signals:
. c_tbuf_nof_samples_per_page = c_tbuf_nof_samples_per_packet = 2000
. c_sdp_tbuf_nof_samples_per_block = 2000
. nof_ddr_words_per_page = 657 or 329, depends on reg_record_enable_RW
SDPTR: ddr_nof_pages = floor(ddr_gigabytes * 1024**3 / (nof_ddr_words_per_page * ctrl_nof_bytes_per_ddr_word)
. Moet SDPTR dit weten, heeft SDPTR dit nodig? --> Nee want SPDTR hoeft alleen pages/blocks/packets
......@@ -443,7 +419,6 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
RSN of block at reg_recorded_last_page_R
. reg_dump_start_page_RW --> index of start page to dump, same for all ai in pn
reg_dump_start_rsn_RW --> RSN of reg_dump_start_page_RW
reg_dump_nof_pages_RW
Number of pages = packets to dump, starting at reg_dump_start_page_RW,
Use nof_pages instead of index of last_page, to be able to set 0 pages, same for all ai in pn
......@@ -454,49 +429,49 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
dump_end_rsn = FPGA_tbuf_dump_end_timestamp_RW / T_adc
# Determine start page that includes start RSN
offset_page = floor((dump_start_rsn - reg_recorded_first_rsn_R) / c_tbuf_nof_samples_per_page)
offset_page = floor((dump_start_rsn - reg_recorded_first_rsn_R) / c_sdp_tbuf_nof_samples_per_block)
if offset_page < 0:
offset_page = 0
dump_start_page = reg_recorded_first_page_R + offset_page
# Determine end page that includes end RSN
offset_page = ceil((dump_end_rsn - reg_recorded_last_rsn_R) / c_tbuf_nof_samples_per_page)
offset_page = ceil((dump_end_rsn - reg_recorded_last_rsn_R) / c_sdp_tbuf_nof_samples_per_block)
if offset_page > 0:
offset_page = 0
dump_end_page = reg_recorded_last_page_R + offset_page
dump_nof_pages = dump_end_page - dump_start_page + 1
if dump_nof_pages > 0:
if dump_nof_pages < 0:
dump_nof_pages = 0
# Set packets that will be dumped by SDPTR
reg_dump_start_page_RW = dump_start_page
reg_dump_start_page_RW = dump_start_page % (page_max + 1)
reg_dump_nof_pages_RW = dump_nof_pages
. reg_memory_read_nof_packets_R[ai]
. reg_memory_read_nof_packets_R
Number of packets that have been read so far
reg_memory_read_nof_crc_errors_R[ai]
reg_memory_read_nof_crc_errors_R
Number of packets that have been read and were dropped because they had a CRC error,
count per ai, because each ai has own CRC
reg_memory_read_nof_rsn_errors_R[ai]
reg_memory_read_nof_rsn_errors_R
Number of packets that have been read and were dropped because they had a RSN error,
count per ai, because RSN is reread for each ai
#reg_memory_remaining_nof_packets_R[ai]
# Number of packets that still need to be read,
# = reg_dump_nof_pages_RW - reg_memory_read_nof_packets_R --> Daarom niet nodig
reg_memory_dumped_nof_packets_R[ai]
reg_memory_dumped_nof_packets_R
Number of packets that have been read correctly and were passed on to ring
= reg_memory_read_nof_packets_R - reg_memory_read_nof_crc_errors_R - reg_memory_read_nof_rsn_errors_R
FPGA_tbuf_memory_clear_counts_RW[pn]
Use dp_strobe_total_count with MM clear register, rather than in_clr = revt(reg_dump_enable_RW)
to clear all reg_memory_* counters.
. FPGA_tbuf_memory_read_nof_packets_R[pn][ai] = reg_memory_read_nof_packets_R
. FPGA_tbuf_memory_dropped_nof_packets_R[pn][ai] = reg_memory_read_nof_crc_errors_R + reg_memory_read_nof_rsn_errors_R
. FPGA_tbuf_memory_remaining_nof_packets_R[pn][ai] = reg_memory_remaining_nof_packets_R
. FPGA_tbuf_memory_dumped_nof_packets_R[pn][ai] = reg_memory_dumped_nof_packets_R
FPGA_tbuf_clear_counts_RW[pn]
Use dp_strobe_total_count with MM clear register, rather than in_clr = revt(reg_dump_enables_RW)
to clear all counters.
Typically clear counters before enabling a new dump.
Use central counters in dp_strobe_total_count. Use one counter for all ai,
because counting per ai is more complex to use and not needed.
. FPGA_tbuf_memory_read_nof_packets_R[pn] = reg_memory_read_nof_packets_R
. FPGA_tbuf_memory_read_nof_crc_errors_R[pn] = reg_memory_read_nof_crc_errors_R
. FPGA_tbuf_memory_read_nof_rsn_errors_R[pn] = reg_memory_read_nof_rsn_errors_R
. FPGA_tbuf_memory_dumped_nof_packets_R[pn] = reg_memory_dumped_nof_packets_R
. FPGA_tbuf_output_nof_packets_R[pn] = reg_output_nof_packets_R
. reg_dump_enable_RW[ai]
. reg_dump_enables_RW[ai]
True = start / keep dumping packets for the requested ai,
False = stop dumping packets immediately
When True maintain reg_memory counters, when False clear reg_memory counters to 0.
......@@ -553,8 +528,8 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
read FPGA_tbuf_recorded_first_timestamp_R[pn] to know recorded time interval.
* Clear counts:
. write FPGA_tbuf_output_clear_counts_RW = True to clear FPGA_tbuf_output_nof_packets_R
. write FPGA_tbuf_memory_clear_counts_RW = True to clear FPGA_tbuf_memory_*_R counts
. write FPGA_tbuf_clear_counts_RW = True to clear FPGA_tbuf_output_nof_packets_R
and FPGA_tbuf_memory_*_R counts
* Prepare dump:
. write FPGA_tbuf_dump_start_timestamp_RW to request start time of dump interval
......@@ -563,37 +538,40 @@ The CP FPGA_beamlet_output_nof_beamlets_RW is not supported in SDPTR and SDPFW y
. read FPGA_tbuf_dump_end_timestamp_R to check actual end time of dump interval
. read FPGA_tbuf_dump_nof_packets_R to know how many packets will be dumped per ai
* Do dump:
* Start dump:
for pn in range(N_pn):
. write FPGA_tbuf_dump_enable_RW = True to start dumping from pn
. read FPGA_tbuf_dump_enable_R to wait until dumping has started in pn,
for enabled ai in range(A_pn):
. read FPGA_tbuf_memory_read_nof_packets_R[ai] == FPGA_tbuf_dump_nof_packets_R and
read FPGA_tbuf_memory_remaining_nof_packets_R[ai] == 0 to wait until dumping has
finished for all ai. If it does not finish, then timeout break because there is
something wrong in SDPFW.
for enabled ai in range(A_pn):
. read FPGA_tbuf_memory_dropped_nof_packets_R[ai] to monitor nof dropped packets due to memory errors
. read FPGA_tbuf_memory_dumped_nof_packets_R[ai] to monitor nof packets that were put on ring
. read FPGA_tbuf_memory_read_nof_packets_R == A_en * FPGA_tbuf_dump_nof_packets_R
to wait until dumping has finished for all enabled ai. If it does not finish,
then timeout break because there is something wrong in SDPFW.
. read FPGA_tbuf_memory_dumped_nof_packets_R to monitor nof packets that were
put on to the ring.
In case of dump timeout:
. write FPGA_tbuf_dump_enable_RW = False to force stop dumping from pn
* Monitor dump results:
. read packet counts (or use for ai loop instead of sum):
N_remaining = sum(FPGA_tbuf_memory_remaining_nof_packets_R[pn][ai])
N_read = sum(FPGA_tbuf_memory_read_nof_packets_R[pn][ai])
N_dropped = sum(FPGA_tbuf_memory_dropped_nof_packets_R[pn][ai])
N_dumped = sum(FPGA_tbuf_memory_dumped_nof_packets_R[pn][ai])
N_lost = N_read - N_dropped - N_dumped
N_output = FPGA_tbuf_output_nof_packets_R
. Expected results:
. N_remaining == 0, else something went wrong in SDPFW
. N_dumped = N_read - N_dropped, else something went wrong in SDPFW
. N_dropped == 0, else packets got lost due to DDR4 memory access errors
. N_lost == 0, else packets got lost on the ring --> check ring MP for details
. N_read == FPGA_tbuf_dump_nof_packets_R * nof active ai, else something went wrong in SDPFW
. N_output == N_read - N_dropped - N_lost, else something went wrong in SDPFW
for pn in range(N_pn):
. read packet counts:
N_read[pn] = FPGA_tbuf_memory_read_nof_packets_R[pn]
N_crc_err[pn] = FPGA_tbuf_memory_read_nof_crc_errors_R[pn]
N_rsn_err[pn] = FPGA_tbuf_memory_read_nof_rsn_errors_R[pn]
N_dumped[pn] = FPGA_tbuf_memory_dumped_nof_packets_R[pn]
N_dropped[pn] = N_read[pn] - N_dumped[pn]
N_lost[last pn] = N_output - sum(N_dumped[pn])
N_output[pn] = FPGA_tbuf_output_nof_packets_R[pn]
. Expected results:
. N_crc_err == 0, else something went wrong in SDPFW at DDR4 interface
. N_rsn_err == 0, else something went wrong in SDPFW at DDR4 interface or with RSN
. N_dropped == 0, else packets were dropped due to DDR4 interface errors or RSN timing errors
. N_lost == 0, else packets got lost on the ring --> check ring MP for details
. N_read == FPGA_tbuf_dump_nof_packets_R * A_en, else something went wrong in SDPFW
. N_output == N_read - N_dropped - N_lost, else something went wrong in SDPFW
* Disable dump:
for pn in range(N_pn):
. write FPGA_tbuf_dump_enable_RW = False to stop dumping
7) TBuf ICD STAT/SDP-CEP
......@@ -701,9 +679,19 @@ Het is gewoon een json-bestandje dat je naast een databestand met alleen complex
- Verify TBuf within SDP on HW
- Integration effort with SC
The selection between recording all or half of the antennas per FPGA has the following impact:
- shows up as if-then-else selector function at various places in the code, also in the read out because the memory page size depends on the selection
- requires conditioning, to ensure that the selection does not cause the implementation to hang if it changes at any moment
- requires verification of both selections
- requires decision making at higher software, configuration or user control levels, to decide which selection to use
11) Transient detection (TDet) Design
- no self triggering yet for MVP
- Pulse detection messages contain event info and timestamp, which is still
useful for ligthning science even without dumping buffer.
- will use Hilbert transform of real input and > 30MHz BPF
https://nl.mathworks.com/help/signal/ug/single-sideband-modulation-via-the-hilbert-transform.html
......@@ -726,8 +714,6 @@ Het is gewoon een json-bestandje dat je naast een databestand met alleen complex
radio triggered message once per hour
Hoi Reinier, kun je deze voor L2SDP-18 voor mij reviewen: https://git.astron.nl/desp/hdl/-/merge_requests/324
en ook dit doc al een keer doorlezen. Het is nog niet af dat doe ik in L2SDP-942, maar misschien heb je al een opmerking:
https://support.astron.nl/confluence/pages/viewpage.action?spaceKey=L2M&title=L5+SDPFW+Design+Document%3A+Transient+buffer+raw+data
12) Cosmic ray
* Katie Mulrey (RU, Cosimc Ray group)
* Stijn Buitink (VUB, Cosmic Ray group)
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment