[ChuckMcM]

In parallel :-) Divide that number by the number of telescopes in operation (in this case 12) and you're fetching 433 GB/sec on a telescope. Imagine that is I/Q data coming from the antenna. Now you're only looking for neutral hydrogen which has a much narrower spectrum, so you process that data with a band pass decimating filter, that takes you down to a few GB/sec. Then take the redshift into account and process everything that is in your 'target range' (a red shift of .26) plus or minus a bit and now your down to perhaps megabytes per second per telescope.

[autocorr]

The example of being able to reduce the data rate by only configuring the system to look in a narrow frequency range in the spectra is perceptive! Although unfortunately you cannot divide the data rate down by the number of antennas by processing each antenna in parallel. Interferometers such as this must correlate the signals between each unique pair of antennas and in approximate real time (1 sec to 1 min range). The data rate comes from sending the signal of each individual antenna by fiber or waveguide to a special FPGA/ASIC super computer called a correlator that computes the correlated signal between each unique pair of antennas. So it all goes into the hopper at once.

Now what you're saying can actually be done, and is done on very long baseline facilities like the VLBA or Event Horizon Telescope (EHN). As an example, the EHN uses hydrogen atomic clocks to essentially time stamp the signal from a single antenna into a 50 TB disk pack, where each of the antennas are off by themselves from Hawaii, Spain, to the South Pole. They then bring all the disk packs to MIT where they then do the correlation from the time stamped data streams. This process is actually very complex however and it requires more total processing in the end. The higher computation comes from having to repeatedly process the data searching for subtle time offsets and antenna position offsets to find the correlation (this is how they use interferometers to measure those 1 cm/year continental drifts).

[discodave]

If it's per unique pair of telescopes then will the data rate scale O(n^2) to the number of dishes? If that is the case then holy balls the final SKA will produce a lot of data.

[autocorr]

It's pretty crazy! And yes, the number of measurements scales as N(N-1)/2, so O(N^2). The funny part about this was, the SKA has been in design for >15 or so years at this point, and they had to predict that computational hardware would be good enough to make it work when they actually start building it! The hardware in 2000 wouldn't have been anywhere close to capable of powering the SKA's correlator. This is actually also a game that they had to play with the Large Synoptic Survey Telescope (LSST)[1], breaking ground now, with the relatively pedestrian 20 TB a night for archiving.

[1] https://lsst.org