Active Incident

Updated a few seconds ago

Incident Status

Degraded Performance

Components

OpenStack

Locations

STO1 - Stockholm South, STO2 - Stockholm West, STO3 - Stockholm North



April 24, 2024 10:20 CEST
April 24, 2024 08:20 UTC
[Investigating] We are aware of issues with newly created instances experiencing network issues and are currently investigating it closer. In some cases, a possible workaround might be to simply restart the newly created instances that are experiencing issues. If this does not resolve your issues then feel free to reach out to us with your instance ID at https://support.elastx.se

April 24, 2024 14:02 CEST
April 24, 2024 12:02 UTC
[Investigating] We are currently working together with our supplier to locate the cause of the issue.

April 24, 2024 15:03 CEST
April 24, 2024 13:03 UTC
[Investigating] Investigation is underway together with our supplier. The next status update will be posted at around 17:00 CEST

April 24, 2024 16:59 CEST
April 24, 2024 14:59 UTC
[Investigating] The investigation together with the supplier is ongoing. The next update will be around 20:00 CEST.

April 24, 2024 20:37 CEST
April 24, 2024 18:37 UTC
[Investigating] The investigation together with the supplier is ongoing, we are now waiting for a plan on how to proceed. The next update will be around 23:00 CEST.

April 24, 2024 22:21 CEST
April 24, 2024 20:21 UTC
[Investigating] We are still waiting for a plan on how to proceed from our provider. Since the impact is quite low ,no more actions will be taken today and the next update will be around 10:00 tomorrow.

April 25, 2024 11:17 CEST
April 25, 2024 09:17 UTC
[Investigating] No news, we are still waiting for an update from our vendor. Next update will be at 14:00.

April 25, 2024 16:50 CEST
April 25, 2024 14:50 UTC
[Investigating] Our supplier support is still processing the given information about the issue and we are waiting for their feedback. Next update will be at 21:00.

April 25, 2024 21:09 CEST
April 25, 2024 19:09 UTC
[Investigating] We have received feedback from the supplier and are planning for actions later tonight. As part of these actions we will upgrade STO1 and STO2 network infrastructure during an emergency maintenance window starting 22:00.

April 26, 2024 04:54 CEST
April 26, 2024 02:54 UTC
[Investigating] We have have not carried out the announced emergency maintenance fully as our tests have shown that the issue remains. Hence we are not going to risk degrading service further. Our supplier is notified and we will resume in the morning or as soon as we get more feedback. Next update around 10:00.

April 26, 2024 10:02 CEST
April 26, 2024 08:02 UTC
[Investigating] We are awaiting further analysis from our supplier. Next update will be around 13:00 CEST.

April 26, 2024 13:06 CEST
April 26, 2024 11:06 UTC
[Investigating] We are awaiting more information from our supplier. The next update will be before 17:00 CEST.

April 26, 2024 16:26 CEST
April 26, 2024 14:26 UTC
[Investigating] Investigation together with our supplier is ongoing. The next update will be on Monday unless the situation changes over the weekend.

April 28, 2024 21:04 CEST
April 28, 2024 19:04 UTC
[Investigating] We upgraded the remaining switches in sto1 today. This did not resolve the issue but it will be easier to find the root cause. There are still delays in creating new networks in sto2 and sto3 but all other tasks should be operational. Currently it is not possible to perform any network changes in sto1. All existing network configurations should be operational.

April 29, 2024 13:55 CEST
April 29, 2024 11:55 UTC
[Investigating] Investigation together with supplier is still ongoing. Next update around 16:30 CEST.

April 29, 2024 16:27 CEST
April 29, 2024 14:27 UTC
[Investigating] We are still investigating together with our supplier. Next update will be around 10:00 tomorrow unless the situation changes.

April 30, 2024 09:07 CEST
April 30, 2024 07:07 UTC
[Identified] We have had some success in resolving the issue. Still service is degraded, however it seems like new resources can be created in all AZs (with some delays) now. We are waiting for a patch from the supplier which we will test and roll out. Please contact us if you want to create new instances and experience problems. Next update 13.00 today.

April 30, 2024 13:08 CEST
April 30, 2024 11:08 UTC
[Identified] No news. The next update will be around 16:30 today.

April 30, 2024 17:04 CEST
April 30, 2024 15:04 UTC
[Identified] The supplier is still working on fix for this problem. The current status is: - New networks will take some time before they are deployed and fully operational. - New instances on existing networks will for the most time work instantly but in some cases it can be a delay before it is connected to the network. - All other network operations like security group, floating ip and router changes should be instant. The next update will be around 13:30 tomorrow.

May 1, 2024 13:14 CEST
May 1, 2024 11:14 UTC
[Identified] We are still waiting for a fix. The status of the network is the same. The next update will be around 18:00.

May 1, 2024 19:18 CEST
May 1, 2024 17:18 UTC
[Identified] The vendor is still working on a fix. The next update will be around 10:00 tomorrow.

May 2, 2024 10:07 CEST
May 2, 2024 08:07 UTC
[Identified] The vendor developer team is working on a fix and we expect an update during the night. The next update will be around 10:00 tomorrow.

May 3, 2024 13:32 CEST
May 3, 2024 11:32 UTC
[Identified] We have been informed that we will not get a fix this week, at the earliest on Monday. The next update will be around 10:00 Monday.

May 6, 2024 13:16 CEST
May 6, 2024 11:16 UTC
[Identified] We have been informed that we get a fix this week, at the earliest on Tuesday. The service seems stable but slow when creating new resources. Please contact us (support@elastx.se) if you experience issues. The next update will be around 10:00 Tuesday.

May 7, 2024 10:37 CEST
May 7, 2024 08:37 UTC
[Identified] We have been informed that we get a fix this week, at the earliest on Tuesday. The service is stable but slow when creating new resources. Please contact us (support@elastx.se) if you experience issues. The next update will be around 10:00 Wednesday.

May 8, 2024 13:43 CEST
May 8, 2024 11:43 UTC
[Identified] We have received a fix and are implementing this. We will work on implementing the fix today and monitor the situation over the weekend. The service is stable but perhaps slow when creating new resources. Please contact us (support@elastx.se) if you experience issues.

Scheduled Maintenance

Schedule

May 22, 2024 09:00 - 17:00 CEST
May 22, 2024 07:00 - 15:00 UTC

Components

OpenStack

Locations

STO1 - Stockholm South, STO2 - Stockholm West, STO3 - Stockholm North

Description

We are upgrading the HSM cluster that is the backend storage for Barbican in Openstack. This service is used to store secrets, mainly legacy encrypted volume keys, and SSL certificate keys when using Octavia L7 proxies. We do not expect any disturbance but there is an increased risk during this service window. There should be no impact to any already provisioned resources.

1

Active Incidents

0

Active Maintenances

1

Upcoming Maintenances

Cloud Connect




Operational

Cloud VPN




Operational

DBaaS




Operational

Mail Relay




Operational

Kubernetes




Operational

OpenStack




Degraded Performance

Virtuozzo PaaS




Operational

External Services

History (Last 7 days)

Description

System maintenance and upgrades of the Elastx identity provider will be performed. During the maintenance window, the ability for users enrolled in the "Elastx Identity" tech preview (https://docs.elastx.cloud/docs/tech-previews/idp-ops/) to login to the OpenStack web console (Horizon) and DBaaS may be affected. The maintenance should not affect any other users, systems or customer workloads.


Components

DBaaS, OpenStack


Locations

STO1 - Stockholm South, STO2 - Stockholm West, STO3 - Stockholm North


Schedule

May 8, 2024 13:00 - May 8, 2024 17:00 CEST
May 8, 2024 11:00 - May 8, 2024 15:00 UTC



May 8, 2024 13:00 CEST
May 8, 2024 11:00 UTC
[Update] Scheduled maintenance is starting.

May 8, 2024 14:38 CEST
May 8, 2024 12:38 UTC
[Update] Scheduled maintenance is ending.
Virtuozzo-PaaS upgradePlanned Maintenance

Description

We're excited to announce the upgrade of our Virtuozzo-PaaS to its latest version! This release brings numerous enhancements, bug fixes, and new features to improve your experience. Some of the key highlights include the ability to exclude Nodes from DNS during Sequential Actions, as well as support for Debian 12 and Ubuntu 23. For more detailed information about all the changes, please refer to the release notes provided below: * Release Notes 8.4: https://www.virtuozzo.com/application-platform-docs/release-notes-84/ * Release Notes 8.6: https://www.virtuozzo.com/application-platform-docs/release-notes-86/ While we strive to minimize any disruptions, please note that there's an increased risk of environment disturbances during the maintenance window. However, no downtime is expected. During this period, the management dashboard will be inaccessible. Additionally, API requests, CLI, and SSH Gateways may experience periodic unavailability as we upgrade corresponding infrastructure components. If you have any questions or require assistance, please don't hesitate to contact our support team at https://support.elastx.se or support@elastx.se Thank you for your understanding and cooperation during this upgrade process!


Components

Virtuozzo PaaS


Locations

STO1 - Stockholm South, STO2 - Stockholm West, STO3 - Stockholm North


Schedule

May 7, 2024 22:00 - May 8, 2024 06:00 CEST
May 7, 2024 20:00 - May 8, 2024 04:00 UTC



May 7, 2024 22:00 CEST
May 7, 2024 20:00 UTC
[Update] Scheduled maintenance is starting.

May 8, 2024 06:00 CEST
May 8, 2024 04:00 UTC
[Update] Scheduled maintenance is complete.