South Pole Weekly Report, January 4, 2009

Preparing DOMs for transport.: Photo by K. Andeen

The current IceCube population stands at 52. Drilling of the 12th hole of the season has been started at location four. The week started with the drilling of hole 5 (the 10th hole of the season) on Sunday, December 28 at approximately 10:30 am, finishing at 6:30 pm on December 29 and ended with the start of drilling of the 12th hole of the season on the morning of Sunday, January 4. In between these holes, a drill maintenance day took place on December 30 followed by a two day New Year break December 31-January 1, 2009. Drilling continued after the New Year break with the start of the 11th hole (string 11) at 8:30 am on January 2 and completion by 11:30 pm on January 3. We are currently drilling hole number 12.

A few problems were overcome during the week. Two more hose leaks required hose replacement. One of these leaks resulted in the drill bottoming out on the ice face before the problem was discovered and corrected. A load spike on the hose resulted from an incorrect configuration of the drill and cable pendant and the computer controls. This has now been corrected in the drive software so the possibility of this incorrect configuration that resulted in reel movement has been eliminated. A firn hole has finally been drilled at hole 37, after about 10 aborted attempts. This location is about 29 meters from the original location in the direction of the station. An incorrect speed setting on the firn drill caused the drill to end up on its side in one of the holes. After several attempts to back the drill out, the firn drill was retrieved by pumping water down the hole. Firn drilling is otherwise going well and we are currently about three holes ahead of the main drill and should be four holes ahead with the completion of hole 2, which we are currently drilling. The crew is well rested after the New Year break and they are eager to get to work on the remaining holes for the season.

IceTop tank freeze process is ongoing as shown. Tank 27A: Photo by L. Shulman

This week featured zero reportable injuries. This marks 63 plus days on the ice without a reportable injury. Two incident reports were initiated during this week. The first issue involved a section of the down-hole hose failing behind the crimp. No one was injured by the leaking water but it took approximately two hours for the hose to be replaced. The serial numbers of the hose are being looked at to determine any correlation with previously failed sections. The drill team continues to monitor the condition of the hose during taping and un-taping activities to help determine sections of the hose that need to be replaced. Posters giving examples of nonconforming sections of the hose are present in both TOSs. The second issue involved the melting of a fan used on the enclosure heaters in the snowmobiles. The thermostat on the heater became stuck, causing the heater to become very hot and melt the plastic cooling fan. Drillers immediately noticed smoke coming from the hood of the snowmobile and quickly put out the small flame. Although this is the first failure of these heaters, the use of alternative heaters is being investigated. The station also had a casualty drill this week that was interrupted when an accident occurred during the drill. Although the majority of IceCube had little involvement in the drill, winterovers Camille and Erik served as first responders.

SPATS pinger deployment

It has been an excellent week for deployment at South Pole. String 5 was deployed and tied off on December 30 giving the drillers and deployers time to work on equipment maintenance and prepare for the January sequence of string installations. String 11 was installed and tied off on January 4th, just in time to see the early season deployers off, and a new group take their place next week.

The cable engineering cycle of winch prep, cable drag, patch cable and testing continues with each deployment. We are currently completing some inventory numbers for cable materials and keeping track of what will be needed for further deployment seasons.

This week in DOM-Testing we have completed SPATS 16, 17, and 18, which correspond to Strings 4, 10, and 3 respectively. The only SPAT with any problems was SPAT 16 that had two questionable DOMs about which we have not yet come to a decision. For Strings 10 and 3 we provided 60 DOMs and held back 4 spares each. For String 4 we held back only one spare and provided 61 DOMs for the string, as one DOM was known to have connectivity problems (this DOM retested with no problem but we were not sure that it would connect properly during deployment). For the SPATs this week we used all 112 DOMs that had been shipped without palettes. This required hand-loading all 64 DOMs of SPAT 17 and 48 DOMs of SPAT 18 on to the sleds. Fortunately, though this proved to be "very good exercise", with help it went quickly. We also tallied all of the remaining DOMs after the deployment of the 10th string of the season and discovered that we have enough DOMs for 12 (actually 12.5) more strings, which is good news. Our buffer currently contains exactly 151 DOMs.

Monitoring of the 38-tank freeze-in continues with minor adjustments as required. RPSC has started a routine tank-cleaning schedule that will clear each tank once weekly. Major tasks accomplished this week include a 40-station snow depth survey of the previously installed IceTop stations, and sending the IceTop Water Filter to Christchurch for refurbishment.

Most of the week was dominated by OS upgrades. All went well except for the IceTop hubs, which have been postponed to next week. Three more In-ice hubs will be brought online over the next two weeks in preparation for the additional strings planned this season. There have been no hardware issues in the ICL and all systems continue to function well.

Driller T. Stezelberger on his way to the ICL with his DOM adjustment tool.

The SPATS acoustic pinger was successfully deployed for the third time this season in hole 5 on December 29 between 9:00 pm and 11:00 pm, it went down to -500m, stopping at 5 of the 9 SPATS instrumented levels. The pinger was set to transmit pulses at a 10Hz rate, the sensors recorded 18s of data every 4 minutes. The first look at the pinger data of this season shows that the data is of high quality.

Seven strings have now been turned on. No recent problems have been discovered. The biggest problems were with string 36, which had 4 "ultra-high current" DOMs and an additional 3 marginally-high current DOMs all in the top half of the string (i.e. 7 serious DOM issues on string 36). String 28 had one ultra-high current DOM . This discovery has led to multiple investigations and mitigation efforts:

It was discovered that a drain-wire ground cable for the DOM Connectivity Monitor had been broken for some time (this happened around the deployment of string 28, the fourth string of the season, although it is not clear exactly when). This was fixed in time for the deployment of string 6 (the 9th string of the season). One purpose of this drain-wire is to prevent the buildup of charge on the cable
Per Olof Hulth et. al. speculated that high winds could have led to charge build-up on the In-ice cable during deployment. He obtained the meteorological data for all IceCube deployments, and found that for this season so far, the worst winds occurred during the deployment of string 36 (many DOM issues), string 28 (one ultra-high current DOM) and string 12 (next in line to be turned on). It is possible that ESD discharges would then have damaged DOMs, especially in the Local Coincidence (LC) circuitry where there is not as much ESD protection as there is in the power and communications circuitry.

Many skiers from all over the world.: Photo by M. Krasberg

There are two ultra-high current DOMs in IC40, both from two years ago, "Ringkobben" and "Hundgrundet". The currents for these DOMs have been periodically checked (and the DOMs turned back off). One thing that has been known for a long time, is that sometimes high currents inexplicably go away (this can occur after freeze-in). It was decided to turn on one pair of these DOMs for a sustained period and determine how usable they were. We turned on "Ringkobben" (and its partner) and determined that they worked well except for Ringkobbben's LC. Turning on the HV for the pair can blow the DOR firmware fuse, which is not good. Within 24 hours of turning "Ringkobben" on, the high currents had completely gone away and the LC problem for this DOM had disappeared. The DOM was allowed to cool back down, and the high current problem did not return when power was restored. "Hundgrundet" was then turned on and in this case, within 24 hours, the overcurrent was reduced by about one-half. This is good because the current is now low enough that DOR firmware fuses will no longer blow when the HV is turned on (this DOM now has the symptoms of the "standard" marginally-high current broken LC DOMs which are already part of IC40). The ability of DOMs to heal themselves is far from understood. The plan is to slowly investigate the other ultra-high current DOMs in the near future.

IceCube DOM 'Jack Lightbody' named for the recently retired NSF IceCube Program Officer.: Photo by M. Krasberg

Mark Krasberg gave a tour of the IceCube drill camp and ICL simultaneously to three groups of "tourist" skiers (11 people in all), with help from several drillers. Some of the skiers appeared to be in dire need of beer (they are not allowed to buy it at the station), and were not happy about repeatedly being told that each IceCube hole has enough melted ice to fill 2.4 million beer bottles.

A different type of milestone was met recently with the retirement of NSF Program Officer Jack Lightbody last Friday. His support and steady guidance since 2001 will be missed by the project. The crew at the South Pole dedicated a DOM to be deployed this season in his name.