Alerts Page

Multiple Outages and Degraded Service Reported Across Campus

Status: Resolved

***Updates***

02.27.2024

  • The cluster has been stable since 02.22.2024 at 6:00 p.m., so we are moving from "major incident" to "normal operations."

02.26.2024

  • 8:30 a.m. - Primary systems such as Banner and websites have remained stable over the weekend. Secondary systems will continue to be restored.


02.23.2024

  • 3:30 p.m. - Critical datacenter services have remained stable throughout the day. Additional core services have also been restored. Less critical services will remain offline through the weekend until a more thorough analysis of the situation can be completed.
  • 9:00 a.m. - Services continue to be restored and the performance of priority systems like Banner and websites continue to be monitored.

02.22.2024

  • 2:30 p.m. - Some services have been restored but may be unstable. Repair work is continuing.
  • 11:00 a.m. - Repairs to restore service continue but myWestern and other wwu.edu websites are currently down. Banner users are advised to avoid running any complex process due to current system instability, however, it's okay to perform data entry, general queries, and reports.
  • 9:00 a.m. - Instability in datacenter have caused system issues to resurface.

02.21.2024

  • 8:00 p.m. - Banner is back up. Final patches and various cleanup and restore tasks are being performed
  • 8:00 a.m. - Alert reinstated. An email has been sent to WWU students and employees. A major system stability issue in the campus datacenters is causing many campus applications to periodically go offline. Banner was down overnight Tuesday and remains down first thing Wednesday morning. The nature of the outages is intermittent and unpredictable. Any system hosted on campus servers could become unavailable at any time, for a length of time from 20 minutes to overnight.  Cloud-based systems like Canvas and Microsoft 365 are not impacted, except when they rely on an on-campus system to function.  Information Technology Services is working with Microsoft support engineers to investigate and resolve the stability issues, but Microsoft believes it may take another week to identify the root cause of the issue. Esign is giving a 503 Service Unavailable.

02.16.2024

  • 4:42 p.m. - Migration of the services has completed, and services should be restored.
  • 8:20 a.m. - Issues have cropped up again and services are being migrated to another node to address stability. There may be some brief performance impacts as the moves take place. EIS is working with the vendor to discuss the analysis on the potential root causes of these service impacts.
     

*** Original Message ***
ITS has identified and is currently investigating outages and degraded service across platforms hosted in the campus datacenters. Work is being done to restore service. 

Services restored:

  • Argos
  • UISAP
  • Library EZProxy
  • Drupal
  • Rightfax
  • Universal password reset
  • Banner job submission
  • Facilities Online Management
  • Alertus
  • Some library systems, such as Libloan.
  • Student printing and EVPrint.
  • Banner as a whole was running slowly. 
  • Banner clean was addresses potentially offline.
  • The data warehouse did not rebuild. A manual run of data warehouse took place.

Banner & Related Systems Down

Status: Resolved

**UPDATES**
2/5/24 1:00 pm - Banner production and test services were restored to functionality last evening, 2/5/24, at 5:00 pm.

**ORIGINAL POST**

2.5.24 - 8:45 am - Enterprise Infrastructure Services and Database Administrators are investigating the root cause of a Banner outage this morning. Folks are unable to log in, seeing an "invalid credentials" error. Services that rely on Banner, such as DegreeWorks, are also impacted.

Microsoft Teams Intermittent Service Impacts

Status: Resolved

**UPDATES**

1.26.2024 4:30 p.m. - Microsoft has improved connectivity to Teams by applying optimizations and mitigations to their services. There may still be some minor impacts.

**ORIGINAL POST*

1.26.2024 9:20 a.m. - Microsoft is having a large-scale service impact with Microsoft Teams which may impact your ability to log into or send messages within Teams. Another symptom is seeing an "Oops" Error message on the main Teams window. Microsoft is currently investigating the scope and source of the impact and is working to resolve the connectivity problems.

If you experience problems with Teams, please be aware that this issue is international in scope and will be fixed by Microsoft without the need for action on your computer or mobile device.

Windows 11 Upgrades Beginning January 22

Status: Resolved

Windows 11 will begin downloading to university-owned computers the week of Jan. 22 (most computer labs and classroom computers were upgraded last summer). Installation will begin during nightly maintenance windows. Computers that cannot support Windows 11 or those that have other client health issues (e.g., hard drives with inadequate space) will not be included in this upgrade. Please see article in the January 12, 2024, issue of Western Today for more information.

Wireless and VPN Unavailable, Tonight from 9:00-9:30 p.m.

Status: Resolved

To troubleshoot some wireless connection issues, maintenance will be done tonight, Thursday, January 18, 2024, from 9:00-9:30 p.m. During that time, users will not be able to login (authenticate) to the VPN or wireless networks. Clients that are already connected should not be affected.

Contract Management (CMM) Unavailable

Status: Resolved

Friday, 1/5, 7:00am - CMM Restored. The Contracts Team will be reviewing and reaching out to anyone where an Approval may have stalled. If anyone attempted to Approve a contract yesterday via email link after around 10:30am, that Approval did not get logged and they will likely need to re-Approve.

5:00pm - CMM found a larger issue than initially suspected. It will likely be down through the night; another update will be posted tomorrow morning.

2:00pm - The CMM is currently experiencing an unexpected downtime. Our vendor is working on resolving this issue and it is anticipated access will be available again by end of day today (Jan 4, 2024).  

Access to WWUwireless-Guest Temporarily Restricted to Address Network Slowness

Status: Active

UPDATES:

2/13
Updates to the infrastructure continue to be put in place to address the bandwidth constraints the campus is experiencing during peak hours. If this week's efforts are successful, the guest wireless network may be restored as soon as next week. Thank you for your continued patience.

1/11

Today at 10:00am access to the internet on the WWUwireless-Guest network will be removed to temporarily address the bandwidth congestion that has been present the past couple days.

 

  • If you are connected to the WWUwireless-Guest, please disconnect and connect to the WWUwireless-Secure network.
  • Devices that have been connected to the WWUwireless-guest network because they do not support the wireless secure network (e.g., signboards, smart TVs) will not be impacted and should remain on WWUwireless-Guest.
  •  Guests needing access to the Wi-Fi may reach out to the ATUS Help Desk to be issued a guest account for the purpose of connecting to WWUWireless-Secure. A budget code will not be required for accounts created to address Wi-Fi access needs during this temporary restriction.

1/10— Due to a high volume of traffic returning to campus for winter quarter, many people are experiencing slow wireless connections, and occasionally not being able to connect at all. These difficulties may extend to use of the VPN as well. If you live on campus or do not have another location from which to access the internet, you can head to a computer lab on campus. Haggard Hall 101 and 112 are open 24/7. Wireless may be particularly slow during peak hours, roughly 12-3pm.


**ORIGINAL ALERT**

1/04 — The annual updating of security certificates on our wireless network was completed on January 4, 2023. You may need to "forget" the WWUwireless-Secure network on your device before you can reconnect to the wireless network. More wireless information can be found at Wireless & Networking | ATUS | Western Washington University (wwu.edu).

Wireless Unavailable, 01.04.2024, from 6:30-7:00 a.m.

Status: Resolved

To update wireless security certificates, wireless access will be unavailable on Thursday, January 4, for a few minutes between 6:30-7:00 a.m. After the update is complete, you may need to "forget" the WWUwireless-Secure network on your device before you can reconnect to it.

VPN and Wireless Unavailable, 12/21/2023

Status: Resolved

**UPDATES**

12/21/23 10:00 AM

VPN and wireless network maintenance has concluded. If you have difficulty reconnecting to the wireless network, please forget the network on your device and reconnect to it. 


**ORIGINAL ALERT**

To perform an update to the university’s wireless network, service to VPN and WWUwireless-Secure and other university wireless networks* will be unavailable on Thursday, December 21, beginning at 8:00 a.m. The networks are scheduled to be unavailable until 10:00 a.m. but could be available sooner.

After the update is complete, you may need to "forget" the WWUwireless-Secure network on your device before you can reconnect to it. Additional info: Connecting to Western’s Wireless Network article

Lab Printer Web Release Intermittently Down

Status: Resolved

**UPDATES**

11/12/2023 4:00 P.M.

This issue is believed to be resolved following a configuration change.

**ORIGINAL POST**

11/8/2023 12:25 P.M.

We have identified an issue where the web release portal for releasing print jobs from labs may put users in a loop of asking for username and password despite the correct values being entered. This appears to happen on a device-by-device basis, with computers in the same area potentially working or not. If you are on a computer that cannot access the web release you my print your document from the EVPrint platform. Instructions on how to use EVPrint can be found here. Despite the article stating it is used to print from personal devices you may use it on a lab computer.