2005 AMANDA Point Source Analysis

Data Sample
Simulation
Event Selection
Methodology
Sensitivity, Discovery Potential, & Systematics
Unblinding Proposal
Results
Questions and Answers

NASA/ESA Hubble

Methodology

The binned point source search method is very simple and makes use of a single statistic, namely "How many events are within bin radius 'b'" and a background estimation to make a statement about the existence of a source at any particular position in the sky. It is reasonable to think the use of additional information must enhance our ability to search for point sources.

Examples of additional information include:

Events outside the search bin, which would be lost by the binned search
Distribution of events within the search bin
Energy estimation
Event-by-event track resolution estimation

Benefits of the unbinned method include better sensitivity and discovery potential. Other benefits include simultaneous estimation of the signal spectral index (discussed later) and source position confidence contours. Drawbacks stem from the increased complexity relative to the binned search. The unbinned analysis requires more effort, including a fundamental understanding of the underlying statistical methods.

The following is an illustration of a potential E^-2 source generated with flux ~2.5*10^-7 Gev^-1 cm ^-2 s^-1 with our standard AMANDA simulation chain against the background of randomized 2005 data. The source produces 6 events which survive 2005 filtering and final cuts. Plot labels indicate the Nch value for each event.

Unbinned Approach	Binned Approach

The signal events are higher energy and clustered near the source location. Utilizing this additional information in a likehood approach, the 3.3σ fluctuation becomes a discovery.

The Likelihood Function

The unbinned method described as follows differs from the method described by Till Neunhoffer¹. Rather than using track-by-track resolution estimates, the resolution is estimated from Monte-Carlo and parameterized by a 2-D Gaussian. Using the track-by-track resolution does not yield significant benefit over this simplifying model². The signal PDF, for a given potential source location x_o, may now be written as a function of the angular distance of each event at x_i from x_o:

The background PDF B_i is flat. Both the signal and background PDF are normalized. The likelihood function can now be written in terms of the signal PDF S_i, background PDF B_i, and a free parameter s describing the number of signal events in the declination band. (+- 5 degrees from source location)

The logarithm of the likelihood function describing all events in the chosen declination band may now be written as:

The free parameter s is then measured by minimizing the -Log(likelihood) with respect to s. The logarithm of the likelihood ratio λ of the signal hypothesis (s = s_best) to the background hypothesis (s = 0) is then used in statistical tests.

Incorporating Event Energy

The energy distribution of a hypothetical E^-2 source is drastically different from that of the atmospheric neutrino background. If high energy events are observed, such events are not very compatable with atmospheric neutrino background and enhance discoverability. Conversely, if high energy events are not observed, the experiment is able to reject the signal hypothesis more strongly. An energy-correlated observable is required, and the logical choice for AMANDA is the number of hit channels (Nch).

The Nch distribution for an E^-2 source is significantly different from the atmospheric neutrino background. To incorporate energy information into the likelihood function, Nch distributions for atmospheric neutrinos and point sources of spectral index 1.5 < γ < 4.0 are tabulated and the signal and background functions are adjusted accordingly:

The spectral index γ becomes a free parameter and is adjusted along with the signal strength s during the minimization of -Log(likelihood). This allows estimation of the signal spectral index along with signal strength, and is discussed in detail below.

Evaluating Significance

To calculate significance, the distribution of the logarithm of the likelihood ratio for events randomized in right ascention is obtained over many (~1M) trials. The logarithm of the likelihood ratio of the potential source region is calculated and compared to the distribution of randomized data, yielding p-value and significance.

For 3σ it is easy to directly obtain the corresponding LLH. For 5σ, it is possible to fit an exponential to the LLH distribution, but it is not unreasonable computationally to perform 10⁸ random trials if a source warrants. To compute discovery potential, sources with 0-50 events are simulated and compared to randomized distributions determine the number of simulations eclipsing the 3σ and 5σ level. This data is combined with poisson weighting to determine the discovery fraction for a given signal strength, since the number of true events produced by a source of a given strength is poissonian.

For a sample consisting of an unknown mixture of potential sources and signal (i.e. unblinded AMANDA data), it is preferable to exclude a region near the area under consideration when calculating the background LLH distribution to avoid contamination by potential signal events.

Evaluating Limits

The problem of calculating limits is more complicated since the convenience of poisson counting statistics no longer applies. Feldman-Cousins confidence bands, which are precalculated in the poisson case, must be calculated for each declination band. Confidence band construction only requires an observable which increases corresponding to increases in signal flux. For the poisson case, this observable is just 'the number of observed events'. The significance estimate ζ described by Till¹ is such an observable.

The analysis could just as well use 'sign(s)*LLH' instead since the significance estimate is just a monotonic function of LLH. Calculation of confidence bands in accomplished in four steps:

Obtain distributions of ζ for simulated signals with 0-50 true events
For some mean signal strength μ_s, Combine the distributions with poisson weighting with mean μ_s to determine the significance distribution for μ_s
Integrate the distribution using Feldman-Cousins ordering until desired coverage is reached.
Increase coverage where necessary to ensure monotonicity

The final step is neccessary to guarantee stated coverage, since distributions of ζ are not necessarily smooth.

Limits are obtained by calculating the significance estimate for the source in question and and computing the upper (and lower) limit from the band upper and lower boundary.

Measuring Spectral Index

Since signal spectral index γ is a free parameter in the likelihood formulation, it is possible to estimate γ from the minimization.

Error bars indicate 1σ uncertainty in spectral index, over a large number of trials. The above test shows good spectral index estimation with as few as 5-8 signal events. For any potential source, it is possible to construct 2-D confidence intervals (μ_s on one axis, spectral index γ on the other). Of course, spectral index estimation would improve significantly with a better event energy estimator.

The following is in illustrates spectral index separation using the Nch observable. The Nch values of 10 random data events and 10 events each drawn from E^-2 and E^-2.5 simulation are plotted:

And again for 50 events each:

The E^-2 signal clearly has higher Nch values than data or E^-2.5. A Nch >= 100 cut is applied, and the number of events passing the cut for data, E^-2, and E^-2.5 are counted and compared for 1000 trials. For 10 events:

N(E^-2) > N(E^-2.5)	71.5%
N(E^-2) < N(E^-2.5)	10.7%
N(E^-2) = N(E^-2.5)	17.8%
N(E^-2) < N(Data)	4.0%

For 50 events:

N(E^-2) > N(E^-2.5)	97.5%
N(E^-2) < N(E^-2.5)	0.7%
N(E^-2) = N(E^-2.5)	1.8%
N(E^-2) < N(Data)	0.0%

Likelihood methods would tend to perform better than a simple cut, so this illustration is a lower limit on the determination of spectral index using Nch information.

References

[1] T. Neunhoffer and L. Kopke. Searching for localized cosmic particle sources with an unbinned maximum likelihood approach. Nucl. Instr. Meth. Phys. Res.A. - 2006.- v.558, N.2. - p.561-568.
[2] J. Braun. Talk given at PSU Analysis Meeting 6/22/2006