Skip to content
Snippets Groups Projects
Commit dc033d5a authored by John Romein's avatar John Romein
Browse files

Added remark on negative input values that cannot be conjugated properly.

parent 0729a1ec
No related branches found
No related tags found
No related merge requests found
......@@ -52,6 +52,8 @@ typedef Sample Samples[NR_CHANNELS][NR_SAMPLES_PER_CHANNEL / NR_TIMES_PER_BLOCK]
typedef Visibility Visibilities[NR_CHANNELS][NR_BASELINES][NR_POLARIZATIONS][NR_POLARIZATIONS];
```
Note that in 4-bit and 8-bit mode, the input samples may not contain -8 or -128
respectively, as these values cannot be conjugated properly.
The input data type (`Samples`) is a weird format, but this seemed to be the only
format that yields good performance (tensor cores are very unforgiving).
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment