October 2024

Planned Non-Service Impacting Network Upgrades - Vancouver YVR1 Datacentre

October 26, 2024 1:00AM PDT

Planned Non-Service Impacting Network Upgrades - Vancouver YVR1 DatacentrePlanned Maintenance

Description

Please be advised that we will be performing non-service impacting network maintenance from October 25th at 23:00 to October 26th at 01:00 PT. This will only affect services hosted in our Vancouver (YVR1) Datacentre. Services originating from our other datacentres will not be affected. We have scheduled a 2-hour window, but the process should take less than 20 minutes. We are upgrading capacity and redundancy to one of our upstream transit providers. We will have to drop BGP sessions with the provider and bring them back up after the upgrade is completed. During this period, you may notice increased latency and suboptimal routing for a brief period of time (around 5-10 minutes) as traffic re-routes. We do not anticipate any additional downtime as a result of these processes. However, customers are advised to take precautions against potential longer interruptions due to the sensitive nature of the work being done.

Components

Cloud Desktop, Bare Metal Servers, Colocation, HyperSpeed Internet, Tech Futures Portal

Datacentres

YVR1 - Vancouver, YVR0 - Vancouver, HyperSpeed Internet Burnaby, HyperSpeed Internet Coquitlam, HyperSpeed Internet New Westminster

Schedule

October 25, 2024 11:00PM - October 26, 2024 1:00AM PDT
October 26, 2024 6:00AM - October 26, 2024 8:00AM UTC

---

October 25, 2024 11:00PM PDT
October 26, 2024 6:00AM UTC

[Update] Scheduled maintenance is starting.

October 26, 2024 1:00AM PDT
October 26, 2024 8:00AM UTC

[Update] Scheduled maintenance is complete.

Partial Disruption to Voice Calls

October 11, 2024 8:54PM PDT

Partial Disruption to Voice CallsDegraded Performance

Incident Status

Degraded Performance

Components

Voice

Datacentres

YVR1 - Vancouver, YYZ0 - Toronto, YYC0 - Calgary

---

October 10, 2024 12:35PM PDT
October 10, 2024 7:35PM UTC

[Identified] We are currently monitoring upstream carrier issues that is affecting some inbound/outbound calling. Will provide further updates as we receive them from the carriers.

October 10, 2024 2:13PM PDT
October 10, 2024 9:13PM UTC

[Identified] The upstream carrier has reported they are still working towards a resolution, but have not provided an ETR at this time. We will continue to monitor closely and report any updates as we receive them.

October 10, 2024 3:19PM PDT
October 10, 2024 10:19PM UTC

[Monitoring] The upstream carrier has reported the issue was cleared at approximately 3:08 PT. We will continue to monitor closely for the next 24 hours.

October 11, 2024 8:54PM PDT
October 12, 2024 3:54AM UTC

[Resolved] Resolved

September 2024

YYZ0 Emergency Maintenance

August 31, 2024 7:05PM PDT

YYZ0 Emergency MaintenanceDegraded Performance

Incident Status

Degraded Performance

Components

Colocation

Datacentres

YYZ0 - Toronto

---

August 28, 2024 1:15AM PDT
August 28, 2024 8:15AM UTC

[Investigating] We are currently performing emergency maintenance on YYZ0-CORE-RT1. Customers with redundancy to RT2 will not be affected.

August 28, 2024 1:58AM PDT
August 28, 2024 8:58AM UTC

[Identified] High RE CPU has been determined to be the cause. We are working with the vendor to apply recommended software upgrades.

August 28, 2024 3:09AM PDT
August 28, 2024 10:09AM UTC

[Identified] An issue has occurred while doing the upgrade, which unfortunately will take additional time to resolve. We are working towards a resolution as quickly as possible.

August 28, 2024 3:54AM PDT
August 28, 2024 10:54AM UTC

[Identified] We have re-engaged vendor support on the new issue encountered. Working with the vendor on a resolution.

August 28, 2024 4:05AM PDT
August 28, 2024 11:05AM UTC

[Identified] The vendor support team advised the remaining recovery steps cannot be performed remotely. We are dispatching local resources to the site now to complete the recovery steps.

August 28, 2024 4:45AM PDT
August 28, 2024 11:45AM UTC

[Identified] Local resources are on-site now, starting recovery operations as directed by vendor support team.

August 28, 2024 6:06AM PDT
August 28, 2024 1:06PM UTC

[Monitoring] We have completed recovery and seeing systems restore to normal through RT1. We will continue to monitor closely.

August 31, 2024 7:05PM PDT
September 1, 2024 2:05AM UTC

[Resolved] Resolved

July 2024

Planned Non-Service Impacting Network Maintenance - Vancouver YVR1 Datacentre

July 28, 2024 4:00AM PDT

Planned Non-Service Impacting Network Maintenance - Vancouver YVR1 DatacentrePlanned Maintenance

Description

Please be advised that we will be performing non-service impacting network maintenance from Saturday, July 27th at 22:00 to July 28th at 04:00 PT. This is the final maintenance to fully decommission our old core router. All customer traffic has already been migrated to the new core routers, per the prior maintenance windows; therefore, we do not expect any impact to traffic. We do not anticipate any additional downtime as a result of these processes. However, customers are advised to take precautions against potential longer interruptions due to the sensitive nature of the work being done. If you have any questions or concerns, do not hesitate to reach out to our support team

Components

Cloud Servers, Bare Metal Servers, Colocation

Datacentres

YVR1 - Vancouver

Schedule

July 27, 2024 10:00PM - July 28, 2024 4:00AM PDT
July 28, 2024 5:00AM - July 28, 2024 11:00AM UTC

---

July 27, 2024 10:00PM PDT
July 28, 2024 5:00AM UTC

[Update] Scheduled maintenance is starting.

July 28, 2024 4:00AM PDT
July 28, 2024 11:00AM UTC

[Update] Scheduled maintenance is complete.

March 2024

Investigating Internet Outage - Coquitlam/New Westminster

March 12, 2024 1:49PM PDT

Investigating Internet Outage - Coquitlam/New WestminsterPartial Service Disruption

Incident Status

Partial Service Disruption

Components

HyperSpeed Internet

Datacentres

HyperSpeed Internet Coquitlam, HyperSpeed Internet New Westminster

---

March 12, 2024 1:07PM PDT
March 12, 2024 8:07PM UTC

[Investigating] We are currently investigating an outage affecting internet customers in Coquitlam and New Westminster. We will post updates as soon as we have them.

March 12, 2024 1:49PM PDT
March 12, 2024 8:49PM UTC

[Resolved] Service is now restored. Root cause was a fibre transport outage, we had to migrate traffic over a different link.

November 2023

Planned Network Maintenance - Service Impacting

November 25, 2023 1:00AM PST

Planned Network Maintenance - Service ImpactingPlanned Maintenance

Description

Friday, November 17th from 11:00PM to November 18th at 1:00 AM we will be performing a core fibre circuit upgrade. During this maintenance period, your connectivity will be intermittent for short periods of 1-10 minutes, for the related services listed only. HyperSpeed Internet Burnaby EVPL Layer 2 Circuits (to YVR1) We do not anticipate any other downtime as a result of these processes. However, customers are advised to take precautions against longer interruptions due to the sensitive nature of the work being done.

Components

HyperSpeed Internet

Datacentres

HyperSpeed Internet Burnaby

Schedule

November 24, 2023 11:00PM - November 25, 2023 1:00AM PST
November 25, 2023 7:00AM - November 25, 2023 9:00AM UTC

---

November 24, 2023 11:00PM PST
November 25, 2023 7:00AM UTC

[Update] Scheduled maintenance is starting.

November 25, 2023 1:00AM PST
November 25, 2023 9:00AM UTC

[Update] Scheduled maintenance is complete.

Intermittent Connectivity - Coquitlam Internet

November 2, 2023 11:48PM PDT

Intermittent Connectivity - Coquitlam InternetPartial Service Disruption

Incident Status

Partial Service Disruption

Components

HyperSpeed Internet

Datacentres

HyperSpeed Internet Coquitlam

---

November 2, 2023 6:11PM PDT
November 3, 2023 1:11AM UTC

[Investigating] We are currently monitoring intermittent connectivity issues affecting internet users in Coquitlam. We are actively working on resolving the issue, the root cause at this time is still under investigation.

November 2, 2023 11:48PM PDT
November 3, 2023 6:48AM UTC

[Resolved] As of 11:48PM service was fully restored. Root cause was a faulty fibre transceiver, which was replaced.

September 2023

Monitoring Partial Outage Affecting Toronto YYZ0 Datacentre

September 25, 2023 5:46PM PDT

Monitoring Partial Outage Affecting Toronto YYZ0 DatacentrePartial Service Disruption

Incident Status

Partial Service Disruption

Components

Colocation

Datacentres

YYZ0 - Toronto

---

September 18, 2023 5:50AM PDT
September 18, 2023 12:50PM UTC

[Investigating] We are monitoring a connectivity outage at our Toronto YYZ0 Datacentre, more information to follow as we know more.

September 18, 2023 6:57AM PDT
September 18, 2023 1:57PM UTC

[Monitoring] A temporary fix has been implemented and we are seeing connectivity restored. We will continue to monitor and work towards a permanent solution.

September 25, 2023 5:46PM PDT
September 26, 2023 12:46AM UTC

[Resolved] Marking resolved. Affected customers who did not have VRRP enabled gateways were contacted. If you did not receive a notification from us, please reach out to support.

August 2023

Planned Non-Service Impacting Upstream Maintenance - Vancouver

August 19, 2023 5:00AM PDT

Planned Non-Service Impacting Upstream Maintenance - VancouverPlanned Maintenance

Description

August 19 between 1 AM and 5 AM one of our tier 1 upstream providers will be performing network maintenance on their network. They have informed us this impacts our connectivity to them, we will be therefore re-routing traffic during their maintenance window. During the maintenance period, you may experience high latency, re-convergence and/or suboptimal routing which may vary during this window. We do not anticipate any other downtime as a result of these processes. However, customers are advised to take precautions against longer interruptions due to the sensitive nature of the work being done.

Components

Bare Metal Servers, Colocation

Datacentres

YVR1 - Vancouver

Schedule

August 19, 2023 1:00AM - August 19, 2023 5:00AM PDT
August 19, 2023 8:00AM - August 19, 2023 12:00PM UTC

---

August 19, 2023 1:00AM PDT
August 19, 2023 8:00AM UTC

[Update] Scheduled maintenance starting.

August 19, 2023 5:00AM PDT
August 19, 2023 12:00PM UTC

[Update] Scheduled maintenance complete.

Planned Non-Service Impacting Maintenance - Vancouver

August 3, 2023 9:00PM PDT

Planned Non-Service Impacting Maintenance - Vancouver Planned Maintenance

Description

Tomorrow August 3rd from 18:00 to 21:00 we will be performing a PDU swap in one of our core networking cabinets in our Vancouver YVR0 Datacentre. All equipment has dual power supplies and no service impact is expected. We do not anticipate any downtime as a result of these processes. However, customers are advised to take precautions against longer interruptions due to the sensitive nature of the work being done.

Components

Cloud Servers, Bare Metal Servers, Colocation, HyperSpeed Internet

Datacentres

YVR1 - Vancouver, YVR0 - Vancouver

Schedule

August 3, 2023 6:00PM - August 3, 2023 9:00PM PDT
August 4, 2023 1:00AM - August 4, 2023 4:00AM UTC

---

August 3, 2023 6:00PM PDT
August 4, 2023 1:00AM UTC

[Update] Scheduled maintenance is starting.

August 3, 2023 9:00PM PDT
August 4, 2023 4:00AM UTC

[Update] Scheduled maintenance is complete.

July 2023

Investigating Service Disruption at YVR1

July 16, 2023 11:22PM PDT

Investigating Service Disruption at YVR1Partial Service Disruption

Incident Status

Partial Service Disruption

Components

Voice, NeuralBeam, Sites, Cortex, Cloud Desktop, Cloud Servers, Bare Metal Servers, Colocation, Tech Futures Portal

Datacentres

YVR1 - Vancouver

---

July 16, 2023 8:15PM PDT
July 17, 2023 3:15AM UTC

[Investigating] We are investigating reports of a service disruption affecting some customers at YVR1 Datacentre, we are dispatching technicians to site and investigating remotely. Next update in 1 hour or sooner.

July 16, 2023 9:16PM PDT
July 17, 2023 4:16AM UTC

[Investigating] Technicians on-site, next update in 1 hour sooner.

July 16, 2023 10:22PM PDT
July 17, 2023 5:22AM UTC

[Investigating] We have narrowed down the service disruption is only affecting a subset of customers, continuing to investigate root cause and find a solution. Next update in 1 hour or sooner.

July 16, 2023 11:22PM PDT
July 17, 2023 6:22AM UTC

[Resolved] Incident resolved. If you are still experiencing any issues please reach out to our support team. We will post a full RFO once we have concluded our investigation.

May 2023

Planned Emergency Network Maintenance - DSL/Cable

May 28, 2023 3:00AM PDT

Planned Emergency Network Maintenance - DSL/CablePlanned Maintenance

Description

Saturday, May 27 to Sunday May 28 from 6:00PM to 3:00 AM one of our fibre carriers is performing an emergency maintenance to relocate a core fibre path. During this maintenance period, your internet connection may be intermittent. This only affects customer connected via DSL or Cable, direct fibre customers are not affected. We do not anticipate any other downtime as a result of these processes. However, customers are advised to take precautions against longer interruptions due to the sensitive nature of the work being done. If you do not have connectivity after the maintenance is complete, please restart your modem/router and/or reach out to our support team.

Components

Cable/DSL Internet

Datacentres

External

Schedule

May 27, 2023 6:00PM - May 28, 2023 3:00AM PDT
May 28, 2023 1:00AM - May 28, 2023 10:00AM UTC

---

May 27, 2023 6:00PM PDT
May 28, 2023 1:00AM UTC

[Update] Planned emergency maintenance is starting.

May 28, 2023 3:00AM PDT
May 28, 2023 10:00AM UTC

[Update] Maintenance is complete.

Partial Cable Internet Outage

May 16, 2023 1:33PM PDT

Partial Cable Internet OutageOperational

Incident Status

Operational

Components

Cable/DSL Internet

Datacentres

External

---

May 16, 2023 12:38PM PDT
May 16, 2023 7:38PM UTC

[Identified] We are monitoring a partial outage for cable internet customers that is causing intermittent connectivity. The upstream carrier has informed us a fibre cut is the root cause, we are waiting on an ETA for repair.

May 16, 2023 1:33PM PDT
May 16, 2023 8:33PM UTC

[Resolved] We are seeing connectivity has been restored, we will continue to closely monitor. If you do not have connectivity still, please try rebooting your modem/router or open a support ticket.

Calgary YYC0 Planned Host Maintenance

May 2, 2023 6:43PM PDT

Calgary YYC0 Planned Host MaintenancePlanned Maintenance

Description

Please be advised we will be performing service impacting maintenance Tuesday May 2nd from 10:00 to 13:00 PT. This will affect only services hosted in our Calgary (YYC0) facility, other services originating in our other facilities or those with high availability and failover will not be affected. We are preparing for core router/switch upgrades and will be moving one of our VM hosts at the same time. This may result in a brief interruptions or intermittent connectivity for 10 to 20 minutes during the maintenance window. We do not anticipate any other downtime as a result of these processes. However, customers are advised to take precautions against longer interruptions due to the sensitive nature of the work being done.

Components

Cloud Servers

Datacentres

YYC0 - Calgary

Schedule

May 2, 2023 10:00AM - May 2, 2023 1:00PM PDT
May 2, 2023 5:00PM - May 2, 2023 8:00PM UTC

---

May 2, 2023 10:46AM PDT
May 2, 2023 5:46PM UTC

[Update] Planned maintenance is starting now.

May 2, 2023 1:19PM PDT
May 2, 2023 8:19PM UTC

[Update] Maintenance window will be extending another 2 hours minimum to replace a failed disk in one of the arrays.

May 2, 2023 3:50PM PDT
May 2, 2023 10:50PM UTC

[Update] Array is rebuilding, once we have confirmed it is in healthy state host will return to production.

May 2, 2023 6:43PM PDT
May 3, 2023 1:43AM UTC

[Update] Maintenance is completed and affected host is back in production.

April 2023

YYZ0 Network Distruption

April 27, 2023 3:48AM PDT

YYZ0 Network DistruptionOperational

Incident Status

Operational

Components

Colocation

Datacentres

YYZ0 - Toronto

---

April 27, 2023 3:48AM PDT
April 27, 2023 10:48AM UTC

[Resolved] We have identified and resolved the root cause of a network disruption at our Toronto YYZ0 Datacentre. Root cause was a partial software failure that has been recovered. Our planned new routers for additional redundancy from recent maintenance windows have not yet been fully put into production, but would mitigate such issues in the future. Our sincere apologies for the disruption. We understand how critical it is for your services to be available at all times, and the burden of trust placed upon us as your infrastructure service provider. Any form of downtime is regrettable, and we are committed to continuously improving our infrastructure.

March 2023

Planned Fibre Maintenance - Hyperspeed Residential Coquitlam

March 3, 2023 6:00AM PST

Planned Fibre Maintenance - Hyperspeed Residential CoquitlamPlanned Maintenance

Description

This Friday, March 3rd from 12:00 AM to 6:00 AM the fibre connecting your building is being moved on the street to a new location. During this maintenance period, your internet connection may be intermittent or out completely. Please refer to the email notification or login to our portal for specific building affected.

Components

HyperSpeed Internet

Datacentres

HyperSpeed Internet Coquitlam

Schedule

March 3, 2023 12:00AM - March 3, 2023 6:00AM PST
March 3, 2023 8:00AM - March 3, 2023 2:00PM UTC

---

March 3, 2023 12:00AM PST
March 3, 2023 8:00AM UTC

[Update] Scheduled maintenance is starting.

March 3, 2023 6:00AM PST
March 3, 2023 2:00PM UTC

[Update] Scheduled maintenance is complete.

December 2022

Monitoring Inbound Call Failures From a Subset of Numbers From Bell Network

December 23, 2022 4:31PM PST

Monitoring Inbound Call Failures From a Subset of Numbers From Bell NetworkOperational

Incident Status

Operational

Components

Voice

Datacentres

YVR1 - Vancouver, YYZ0 - Toronto, YYC0 - Calgary

---

December 23, 2022 9:09AM PST
December 23, 2022 5:09PM UTC

[Monitoring] We have been made aware of intermittent call completion problems for calls originating from Bell Mobility. Currently this is only affecting some calls that originate from Bell Mobility, other carriers are completing calls normally. We will provide updates as more information becomes available from the carrier in question.

December 23, 2022 4:31PM PST
December 24, 2022 12:31AM UTC

[Resolved] We have received an update that Bell corrected the issue with the circuits and all calls should be working across the board.

Planned Emergency Network Maintenance - Burnaby

December 2, 2022 6:00AM PST

Planned Emergency Network Maintenance - BurnabyPlanned Maintenance

Description

December 2nd between 12:00AM to 6:00 AM PT one of our carrier partners will be performing emergency maintenance on the fibre circuit that connects your building. Please refer to the email sent out for more details. They do not expect the maintenance to last more than 1 hour during the planned window. Your connectivity may be intermittent during that period.

Components

HyperSpeed Internet

Datacentres

HyperSpeed Internet Burnaby

Schedule

December 2, 2022 12:00AM - December 2, 2022 6:00AM PST
December 2, 2022 8:00AM - December 2, 2022 2:00PM UTC

---

December 2, 2022 12:00AM PST
December 2, 2022 8:00AM UTC

[Update] Scheduled emergency maintenance is starting.

December 2, 2022 6:00AM PST
December 2, 2022 2:00PM UTC

[Update] Scheduled emergency maintenance is complete.

November 2022

Planned Network Maintenance - Burnaby

November 17, 2022 6:00AM PST

Planned Network Maintenance - BurnabyPlanned Maintenance

Description

November 17th between 12:00AM to 6:00 AM PT one of our carrier partners will be performing maintenance on the fibre circuit that connects your building. Please refer to the email sent out for more details. They do not expect the maintenance to last more than 2 hours during the planned window. Your connectivity may be intermittent during that period.

Components

HyperSpeed Internet

Datacentres

HyperSpeed Internet Burnaby

Schedule

November 17, 2022 12:00AM - November 17, 2022 6:00AM PST
November 17, 2022 8:00AM - November 17, 2022 2:00PM UTC

---

November 17, 2022 12:00AM PST
November 17, 2022 8:00AM UTC

[Update] Scheduled maintenance is starting.

November 17, 2022 6:00AM PST
November 17, 2022 2:00PM UTC

[Update] Scheduled maintenance is complete.

September 2022

[External] IoT SIM Emergency Network Maintenance

September 9, 2022 4:00AM PDT

[External] IoT SIM Emergency Network MaintenancePlanned Maintenance

Description

One of our external partners has advised they will be performing emergency network maintenance between Sep 8, 9PM to Sep 9 4AM PT. Minimal impact should be expected to IoT SIM service.

Components

IoT SIM

Datacentres

External

Schedule

September 8, 2022 9:00PM - September 9, 2022 4:00AM PDT
September 9, 2022 4:00AM - September 9, 2022 11:00AM UTC

---

September 8, 2022 9:00PM PDT
September 9, 2022 4:00AM UTC

[Update] Scheduled emergency maintenance is starting.

September 9, 2022 4:00AM PDT
September 9, 2022 11:00AM UTC

[Update] Scheduled emergency maintenance is complete.