Status history, last updated 11-8-11

The T1 connection to the upper labs has been problematic all year. We believe the OVL-summit link antenna needs replacing, and have ordered a new unit which is stored at OVL until next spring. Right now the link works well as long as temps remain low, but sometimes cuts out in the afternoons or late mornings.

This fall we moved the Barcroft Starband antenna to a more stable location, and we hope that will solve some of the problems we have been having that cause us to re-aim the antenna. We also switched the telular telephone units to ATT GS technology and re-aimed the antennas toward Bishop. The old CDMA phones were aimed at Tonopah. Currently we have phone and satellite internet service at both sites, as well as T1 service at Barcroft. The T1 link went down at Crooked Creek when the power was shut off for the winter.

Electronics detail (password needed)
Facilities virtual tour

Older Notices:

9-1-10 New "critical loads" power system installed at Barcroft. Operates communications backbone and energy lab monitoring equipment. Will operate Starband and weather station when wires are run up to computer closet.

Radio signal booster installed at Summit (antenna A uplink) and at CC pole (antanna uplink). Two downlinks already have boosters (OVL downlink and Summit downlink). The dome radio remains to be modified - needs new housing and uplink radio booster.

6-29-10 Starband Antenna destroyed winter storm, will replace asap. T1 connection working, but slow. Replaced in August

10-20-09 New Starband Modem installed at Barcroft, technician re-pointed antenna. New IP numbers assigned.

10-16-09 Replaced 6 AGM batteries at Summit Lab with new ones of same type. Old batteries were failing, as evidenced by voltage drop at night and corrosion around terminals and beneath battery.

8/10/09 Barcroft T1 connection repaired, functioning normally. Starband continues to function normally. We have ordered some new internet switches and other devices. Once these arrive the currently functioning units will be kept as backups.

7/30/09 The lightning event destroyed or damaged at least nine devices, including internet switches, media converters, and digital pressure gauges. There is also damage to the dome power supply, either in the transformers or in the buried line. JTS is working on a write-up of the event.

7-29-09 Lightning visited the area in a big way last Thursday (more details), and we now are troubleshooting various problems: the T1 connection to Barcroft is down, even though the dome Wi-Lan radio is working. Electric power is also off at the observatory dome - there seems to be short or some other problem. The Starband satellite connection is working at Barcroft, as is the T1 to Crooked Creek. We also just installed a new, higher bandwidth (2 Mb/s download, 500 Kb/s upload, with 5 static IP addresses) "Hughes Net" satellite internet service at Crooked Creek - details on how to access this later. The Crooked Creek Starband has been disconnected.

7/20/09 we installed a new "Hughes Net" satellite internet service at Crooked Creek, replacing the old "Starband" unit. It seems to be working well, with higher bandwidth (500 KB uplink and 1.5 MB downlink). We still have to reprogram the CC weather station for the new service: for now the CC weather station is off-line. We plan a similar replacement for Barcroft in the next month or so.

10-17-08 All systems working! The only problems now seem to be associated with bandwidth. T1 just isn't enough anymore with the way people are now using the internet. Note: we installed a new double-WAN Cisco 1811 router at Barcroft which should automatically swtch internet connectivity to the starband if the T1 fails. Accessing devices at Barcroft through this router will now require use of local IP addresses, a special port number and special programming in the router unit. Other units not using the new router can keep their old IP addresses.

6-3-08 All systems working! The problem below was caused by a short (broken insulator) along the pole line to Crooked Creek. Once the fuse was replaced, all is in order! However, until the insulator is repaired the line is vulnerable to outages and power surges.

1-17-08 T1 functioning to summit, dome, Barcroft closet and CC pole. However, starting a week or two ago, we lost contact to CCshop and the CC starband, at about the same time. The cause for this is unknown, but may have to do with the power supply which is shared by the two devices. The CCpole has a separate circuit on a different power leg.

4-3-07 T1 functioning to summit and dome, but battery power is lost at Barcroft (batteries discharged and burst). CC on satellite only.

12-8-06 Current status: T1 functioning to summit and Barcroft, running on solar/batteries. No AC power at Barcroft but diesel generator is in place. Crooked Creek internet on satellite only. Phones OK at both sites.

Fall 2006: redid T1 system at summit: moved Wi-Lan radio inside building and put all antenna wires in conduit. Beefed up grounding system, added metal rod to NW corner of building. Added battery/solar panel power supply for powering dome radio and switch/media converter in dome.

8-20-06 A series of lightning storms took out the internet links at the summit and at Crooked Creek. We do not plan an immediate replacement until we can beef up the lightning protection at each site. Hopefully the T1 link will be restored sometime in September. The starband satellite links are woking at Barcroft and Crooked Creek.

7-25-06 Lightning damages main router at OVL - internet connection and email out for about 24 h.

7-18-06 lightning storm(s) damage T1 internet connectivity at Crooked Creek and summit. Backup starband remains functional at both sites.

7-5-06 Full strength connection between summit and OVL restored itself - with hindsight I realiize that water must have gotten in to the connectors, which are bare after the emergency repairs made last winter (I didn't have time to seal them then). Will repair next summit trip.

7-1-06 Connection to Crooked Creek partially restored: summit radio link restored but weak. Intermittent problems with CCshop radio bridge - perhaps power supply is bad? Will check Monday July 3. No problem w/ power supply.

1-06 There is a serious power outage at Barcroft, with no visible cause. We are bringing in consultants with test equipment to isolate and diagnose the problem. We believe that power went out at barcroft on new years day; but sometime after December 20 at the earliest. Crooked Creek has power

6-30-06 Connection to Crooked Creek via summit was failing on the afternoon of Wednesday June 28, and went down completely later in the evening. Link remains down. We will try to get to the summit to make repairs asap. Barcroft still down (power out) but starband backup system is working.

Old info:

Wi-Lan Internet Bridge Radios - all checked out; OK on 5-13-05 and again on 11-15-05. Link went down 1:30 am 11-29-05 - we are planning a summit trip to repair. Trip keeps getting delayed by bad weather, but is still a high priority. Fixed 2-8.

• OVL "root" internet bridge radio - OK working normally through antenna B
• Summit "repeater" internet bridge radio - went down 1-3-05. Service restored 1-7-05 Summit connection lost 8pm 8/14/05 during severe storm. Still out 8/15/05. Repaired power supply, replaced radio unit 8-22-05. Icing 11-29-05. Damage lost link. Repaired 2-8-06.
• Observatory dome"leaf" internet bridge radio - went down 12-30-04. Service restored 1-7-05. Down 1-11-05. intermittent 1-20-05, Service restored 1-26-05 at about 6pm. Intermittent problems 7-25-05, went down entirely 8-5-05. Repaired with new power supply and replaced terminal connector 8-9-05.
• Crooked Creek pole "repeater" internet bridge radio - went down ?. As of 1/7/05 there is a normal link through antenna A. B link intermittent and probably faulty. Replaced antenna - link now strong. 5-26-05
• Crooked Creek shop "leaf" internet bridge radio - a strong link through antenna A; 5-26-05

Other LAN connections

• Observatory dome; switch went down for about 5 hours on 5-13-05, back up by 1500; replacement switch 9-05
• Barcroft Pace Lab; switch went down for about 5 hours on 5-13-05, back up by 1500; connection lost 6-19-05. Connection intermittent 6-27-05; OK after 8/9/05 repairs; replaced switch 10-05
• Crooked Creek shop switch; on 5-26-05
• Crooked Creek wireless; on 5-25-05
• Twin-WAN router - recovered T1 link 1-26-05 - starband link out but restored by 2-8-05 - went down for about 5 hours on 5-13-05, back up by 1500, off since 8/05.

Starband and Hughes Net Satellite Connections

• BAR Satellite connection - Starband down as of 1/1/05; Still down 1/10/05; Recovered 1-24-05; OK 5-13-05 reliable since then. New Starband modem, antenna repointed October 2009.
• Crooked Creek Satellite Connection - down as of November 2004 - host computer problems satellite modem probably OK, still down 11-05 need tech support to get running again. CC unit replaced with Hughes Net system summer 2009.

IP units (old info)

• summit weather station - OK 1-7-05, OK 5-13-05; damaged in storm of 8-14-05; repaired 9/13/05 except wind instruments repaired 9-27-05, wind direction seems off by 90 degrees (i.e. a west wind shows on the output as a south wind).
• barcroft weather station - OK 1-7-05; went offline 1-11-0;. Restored 1-26-05; Went down for about 5 hours on 5-13-05, back up by 1500 ; went offline 6-19-05 2100; went offline October 21, 2005
• barcroft webcam - no contact since December; restored 4-29-05; went down lightning?? 5-5-05; OK
• barcroft PV monitoring computer (via starband) - no contact since December. Restored 4-29-05 using T1 connection
• barcroft wireless access point down in late september; awaiting replacement unit; installed replacement 10-05 OK
• crooked creek weather station on line 9/13/05; analog instruments stopped recording in mid October 2005.
• crooked creek wireless access point OK; sometimes needs to be rebooted (POE may be too tenous over too long a reach)
• crooked creek US Array seismic installation OK

Notes: all connections below Dome Radio were out for 5-6 hours on 5-13-05; were restored without any action on our part. Since late June dome power is only working on one of the two split single phase circuits; repairs are in the planning stages.