UPDATE: EECS Network and Service Outage 12/22/16

We ran into a number of difficulties on the electrical side which delayed restoration of services, but basic networking (wired and wireless) were brought back at approximately 12PM.

At this time we are experiencing some slowness in the LDAP system, and we have also encountered trouble in bringing the NetApp file servers online. As a result the department website and file storage, as well as other services which rely on them, are still offline.

EECS Full Network and Service Outage, Thurs. 12/22/16 7:30AM – 1:00PM

Published 2:43 p.m. Nov 01, 2016

Scheduled Maintenance

Services Affected: Active Directory, DHCP, DNS, FTP Server, Home Directory Storage, IRIS Website, Mailing Lists, Project Storage, Repo Service, Unix Login Server, Wiki Hosting, Windows Terminal Server, Wired Networking, and Wireless Networking

Begins 7:30 a.m. Dec 22, 2016

We will have a full system shutdown on Thursday, December 22nd at 7:30 AM to perform necessary fire and emergency power testing in our server rooms. The entire EECS network and all hosted resources will be offline during this maintenance.

This will affect networking in the following locations:

Wired and wireless networks offline:

  • Soda Hall
  • Cory Hall
  • Sutardja Dai Hall
  • Jacobs Hall
  • BWRC

Wireless networks offline, wired remains online:

  • Blum Hall
  • Calvin Lab

Hosted resources such as the EECS website, file servers, etc. will be offline and globally inaccessible.

The facilities maintenance window is scheduled from 7:30 AM to 1 PM, but we anticipate the network and critical services should be back by 11AM if all goes well.

The intent is to perform these tests annually going forward; however we will learn from this year’s procedure and aim to keep more critical services online in the future.

If you have any questions about this maintenance, please contact help@eecs.berkeley.edu.

Fileserver maintenance, Nov 24 2016, 10am

Originally Published 11:56 a.m. Oct 26, 2016 on IRIS News

Scheduled Maintenance

Services Affected: Home Directory Storage, IRIS Website, Project Storage, Repo Service, and Unix Login Server

Begins 10:00 a.m. Nov 24, 2016

On the morning of Thursday, November 24 (Thanksgiving Day), we will be performing some disruptive maintenance on the department fileserver:

  • Recable to new network datacenter switches.
  • Upgrade cluster switches firmware.
  • Upgrade NetApp OS.

Department fileservice will be unavailable during the maintenance window, approximately 10am until 1pm. Many services that rely on access to department fileservice (home directories, webservice, repo service, etc), will either be offline or will fail to work during this window. All NFS and CIFS mounts will be affected.

We expect most, if not all, clients to recover automatically once the fileservers are back online. This news item will be updated to indicate when the maintenance is completed.

Systems still experiencing mount issues after the maintenance window, would need to remount or reboot.

COMPLETION UPDATE:

FILESERVER MAINTENANCE COMPLETED
Pathma Venasithamby at 12:37 p.m. Nov 24, 2016
Incident Resolved
Scheduled Maintenance Resolved
Systems on new datacenter network switches. All firmware updates completed. Netapp OS upgraded on all nodes. If there are any NFS or CIFS issues please reconnect and followup with helpdesk if there are outstanding issues.

Network outage at department border

Staff are investigating a network outage affecting all EECS network hosts. The outage began at about 6:25am September 22.

At this time, it’s unclear whether the outage is due to EECS network equipment, or upstream with campus networking.

 

UPDATE: Cause was due to a loss of default routes normally advertised by campus BGP. The issue was isolated and the campus NOC restored the routes shortly before 10AM.

See also https://iris.eecs.berkeley.edu/news/16634-eecs-internet-outage-92216

 

IRIS Website Scheduled Maintenance, July 12 2015, 7am-11am

The primary IRIS website, including applications such as the account request forms and network registration forms, will be offline Sunday July 12, 2015 from 7am-11am.  IST will be upgrading our back-end Oracle database during this time.

Questions can be sent to help@eecs.berkeley.edu.

 Update: July 12, 11:15am

The maintenance seems to be running behind schedule.  We are monitoring the status of the Oracle server and will restore our services as soon as it comes back online.

Network Outage Resolved

As of about 9:45 PM, production services were externally available, and internal building networking came online during the next few hours. As of 1:30 AM we have restored service to all locations except a small handful, which currently include BWRC and Blum Hall. We will be making use of tomorrow’s scheduled contingency maintenance period to fully restore all services in all locations.

Network Outage Extended to 10PM

Due to unforeseen complexities, we are extending our maintenance window. We currently expect production services to be externally reachable by 10PM, with internal network service possibly taking longer. Updates will be posted as they are available.

We apologize for the trouble, and hope that it doesn’t stop you from enjoying what’s left of the holiday season.