PDD - Latency
By Section | Whole Paper | PDF (233 pages, ~6.62mb)
8 Data Handling-
8.1 System Elements
- 8.1.1 Software Management
- 8.1.2 System Engineering
- 8.1.3 Development Environment
- 8.1.4 Analysis Framework
- 8.1.5 Database
- 8.1.6 Visualization
- 8.1.7 Development Interfaces
- 8.1.8 Integration at Pole
- 8.1.9 Hardware
- 8.1.10 Data Distribution
- 8.2 Offline Data Flow
-
8.3 Data Model
- 8.3.1 High Multiplicity
- 8.3.2 Upgoing Tracks
- 8.3.3 Cascades/Taus
- 8.3.4 GRB Downgoing Muons
- 8.3.5 Icetop
- 8.3.6 Supernova
- 8.3.7 Prescaled Raw Data
- 8.3.8 Monitor
- 8.3.9 Calibration
- 8.3.10 Full-Sky Summary Histograms
- 8.3.11 Unfiltered Raw
- 8.4 Data Sample Organization
- 8.5 Latency
- 8.6 Schedule
- 8.7 Summary
8.5 Latency
Latency is defined as the time between when an event is triggered and when it arrives on an analyzer's disk. Data from the main analysis tags (high multiplicity, upgoing track, cascade/tau) should have a 1-2 week latency, which includes possible further processing/filtering in the north. Monitor data should have a latency of less than 24 hrs. Supernova data should have as close to zero latency as possible. One way to implement a very short latency time is to use the Iridium system (if it remains available). Data volumes for supernova triggers are small enough to make this feasible. IceCube can then be a full-fledged participant in SNEWS, the Supernova Early Warning System [48].
Possible uses of 24/7 low-bandwidth coverage beyond SNEWS include IceCube monitor data. This would have the clear advantage of allowing us to keep close track of detector performance, including during times when the winter-overs would be sleeping. This coverage would also be very handy for small system maintenance tasks and for sending alarms to to the winterovers. Obviously, the South Pole becomes much less remote with this kind of coverage.
After the first six-string year, the full raw data has a latency of at least a year, corresponding to the movement of tapes from the Pole to the north. For IceCube design purposes, the latency of this data sample should be considered infinite since the resources needed to use it are not in the current budget.
Another latency issue arises due to calibration constants (e.g., geometry) which will be needed by the Pole filters. We envision three categories of constants:
- initial constants: Within 2 days of deployment, crude constants should be available for the operation of the string. An example of this would be OM locations from drill logs.
- on-line constants: Within a week or two of deployment, the results of in situ light sources and downgoing muons need to be included for stable data taking.
- offline constants: As the analysis requires, constants need to be included in the database. Multiple sets can be supported with the use of version numbers.
