Active Incident

Updated a few seconds ago

STO1 v1-volume issueDegraded Performance

Incident Status

Degraded Performance

Components

DBaaS, Kubernetes, OpenStack

Locations

STO1 - Stockholm South


February 12, 2025 18:57 CET
February 12, 2025 17:57 UTC
[Investigating] We are currently seeing nightly disturbances in the API for v1 volumes in the STO1 availability zone. This occurs every night between 19:00 and 02:00 CET. Impact is inability to manage v1 volumes in STO1 during this time, impacted tasks is creation, deletion and attach/detach of volumes. There is NO impact on performance or availability of already attached volumes. We are actively working with our storage vendor to correct this issue.

March 5, 2025 09:53 CET
March 5, 2025 08:53 UTC
[Identified] We have made progress with our storage-vendor in mitigating the effects of the issue, reducing the time-window and impact it is causing. Still we have no performance-impact but still (very) limited issues with volume-tasks (creation/deletion/attach&detach).

1

Active Incidents

0

Active Maintenances

0

Upcoming Maintenances

Cloud Connect




Operational

Cloud VPN




Operational

DBaaS




Operational

Mail Relay




Operational

Kubernetes




Operational

OpenStack




Operational

Virtuozzo PaaS




Operational

Application Monitoring




Operational

External Services

GitHub

Let's Encrypt

Slack

Zendesk

History (Last 7 days)

DBaaS disruptionPartial Service Disruption

Incident Status

Partial Service Disruption


Components

DBaaS


Locations

STO1 - Stockholm South, STO2 - Stockholm West, STO3 - Stockholm North



June 9, 2025 08:09 CEST
June 9, 2025 06:09 UTC
[Investigating] We have a problem with the controlplane in DBaas currenty which affects customer datastores.

June 9, 2025 09:44 CEST
June 9, 2025 07:44 UTC
[Identified] We have identified the issue causing disruptions in DBaaS and are working on deploying a fix.

June 9, 2025 10:53 CEST
June 9, 2025 08:53 UTC
[Identified] Fix has been deployed for the underlying issue. Work is still being done to make bring individual monitoring to specific datastores online again. Some datastores might also report being in read only mode after the incident and we are working through them manually.

June 9, 2025 11:55 CEST
June 9, 2025 09:55 UTC
[Resolved] The DBaaS issue has now been resolved and the service should have returned to a normal state again.

June 9, 2025 15:57 CEST
June 9, 2025 13:57 UTC
[Investigating] We are unfortunately reopening the issue. The control plane is unstable and we are continuing the monitoring.

June 10, 2025 14:38 CEST
June 10, 2025 12:38 UTC
[Resolved] The service has now returned to normal operational state.
API-disturbance in OpenstackPartial Service Disruption

Incident Status

Partial Service Disruption


Components

OpenStack


Locations

STO1 - Stockholm South, STO2 - Stockholm West, STO3 - Stockholm North



June 10, 2025 11:05 CEST
June 10, 2025 09:05 UTC
[Identified] We are currently monitoring a partial service-disruption with API-calls in Openstack primarily affecting (but not limited to) the swift-service. We have a solution ready and will shortly implement it to resolve the issue, this status-message is primarily sent for transparency as the disturbance currently is very limited. When fix is implemented 11:15 some timeouts could be detected as the only impact.

June 10, 2025 11:39 CEST
June 10, 2025 09:39 UTC
[Resolved] Fix as been implemented and services are again stable, all API-calls are fully functional and our monitoring looks good. Incident now closed and marked as resolved.
OpenStack Hardware Maintenance STO3Planned Maintenance

Description

NOTE: We will get in contact directly with affected customers later this week with more detailed information on relevant OpenStack instances. As part of our continuous work to renew and expand our hardware fleet we need to drain and remove the oldest generation of OpenStack v1 shared hypervisors from our STO3 availability zone. OpenStack instances on the affected hypervisors will be shutdown, migrated to another hypervisor and then restarted with an estimated downtime of 5-10 minute per instance. Note that this work will also affect some DBaaS datastores and Kubernetes PCaaS clusters running on shared v1 flavors.


Components

DBaaS, Kubernetes, OpenStack


Locations

STO3 - Stockholm North


Schedule

June 4, 2025 23:00 - June 5, 2025 06:00 CEST
June 4, 2025 21:00 - June 5, 2025 04:00 UTC


June 4, 2025 23:00 CEST
June 4, 2025 21:00 UTC
[Update] Scheduled maintenance is starting.

June 5, 2025 06:00 CEST
June 5, 2025 04:00 UTC
[Update] Scheduled maintenance is complete.
Intermittent network-issuesPartial Service Disruption

Incident Status

Partial Service Disruption


Components

OpenStack


Locations

STO1 - Stockholm South, STO2 - Stockholm West, STO3 - Stockholm North



June 3, 2025 11:47 CEST
June 3, 2025 09:47 UTC
[Identified] This morning we noticed intermittent issues with a network-related service (unrelated to tonights maintenance). Issue has been identified and fix implemented; any impact noticed should look like short network interruptions or delays during 30-60 second windows. Fix was pushed to our A-Zs during different intervals to reduce impact which caused the impact to be resolved in different times depending on which AZ the impacted server or node was located in. For now everything should be stable, we are monitoring some impacted nodes and environments. For any ongoing issues or impacts please alert our support immidiately.

June 4, 2025 16:04 CEST
June 4, 2025 14:04 UTC
[Resolved] We have not seen any further issues since yesterday and will close the incident as things are stable.
OpenStack Hardware Maintenance STO2Planned Maintenance

Description

NOTE: We will get in contact directly with affected customers later this week with more detailed information on relevant OpenStack instances. As part of our continuous work to renew and expand our hardware fleet we need to drain and remove the oldest generation of OpenStack v1 shared hypervisors from our STO2 availability zone. OpenStack instances on the affected hypervisors will be shutdown, migrated to another hypervisor and then restarted with an estimated downtime of 5-10 minute per instance. Note that this work will also affect some DBaaS datastores and Kubernetes PCaaS clusters running on shared v1 flavors.


Components

DBaaS, Kubernetes, OpenStack


Locations

STO2 - Stockholm West


Schedule

June 3, 2025 23:00 - June 4, 2025 06:00 CEST
June 3, 2025 21:00 - June 4, 2025 04:00 UTC


June 3, 2025 23:00 CEST
June 3, 2025 21:00 UTC
[Update] Scheduled maintenance is starting.

June 4, 2025 06:00 CEST
June 4, 2025 04:00 UTC
[Update] Scheduled maintenance is complete.