Ninefold
All Systems Operational
Management Portal Operational
Asia Pacific 1 Region Operational
Asia Pacific 2 Region Operational
DNS ? Operational
Knowledge Base ? Operational
ninefold.com Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Scheduled Maintenance
Ninefold will be undertaking scheduled platform maintenance:
Saturday September 5 8pm - Sunday September 6, 5am (AEST).

**Please note this is the maintenance work which was postponed on August 22nd**

We will be upgrading our cloud orchestration layer to the latest version, the upgrade will further improve the stability and performance of Ninefold.

A full notice regarding this maintenance has been sent to customers and is available here:
http://info.ninefold.com/webmail/36692/347913869/9dd5c1fede4ad1fa10df3860bbf9450f
Posted on Aug 27, 11:59 AEST
Past Incidents
Aug 31, 2015

No incidents reported today.

Aug 30, 2015

No incidents reported.

Aug 29, 2015

No incidents reported.

Aug 28, 2015
Completed - All Virtual Servers on the affected host are now running after repairs were completed successfully.

We do not anticipate any further issues. If you experience an issue please create a ticket in the Ninefold Portal.
Aug 28, 22:54 AEST
Update - Our Vendor engineer is on-site and performing work on the impacted virtualisation host.
Aug 28, 22:02 AEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Aug 28, 21:00 AEST
Scheduled - Emergency maintenance is required to repair hardware on a single virtualisation host in Asia Pacific 1. The expected window is 28/08/2015 21:00 AEST - 29/08/2015 00:30 AEST.

Virtual servers on this host will be unavailable during this period.

All impacted customers have an open ticket in the Ninefold portal and have received email correspondence about this emergency maintenance window. If you have not received either, you are not impacted.

Please check the status page for updates during maintenance.
Aug 28, 20:30 AEST
Resolved - All affected virtual machines are back online. Our team + vendor engineers are continuing monitor. This incident will now be closed however customer tickets will remain open so that we can advise next steps in the coming hours. In addition, any emergency maintenance will also be posted via this status page.
Aug 28, 17:49 AEST
Identified - Important Update - Our team have been able to bring the affected host back online (Servers are booting up) however there is still an ongoing hardware fault.

Our checks indicate there was no data loss.

As a precaution the Ninefold team are taking backups of the Root volume of all affected servers.

We are awaiting the delivery of the replacement parts and will need to bring the severs down for the installation.

More information on this maintenance window shortly.
Aug 28, 16:59 AEST
Update - All affected customers now have a ticket in the Ninefold portal for tracking purposes. Vendor and Ninefold Engineers onsite for Diagnostics. Next update in 30 mins.
Aug 28, 16:08 AEST
Update - Engineers have identified that there has been a failure relating to the storage system on 1x physical host in Asia Pacific 1. Our support team will be logging individual tickets for each affected customer over the coming minutes to identify the impacted VM's and discuss options moving forward. No resolution time has been determined at this stage.
Aug 28, 15:18 AEST
Update - Engineers are currently engaged with our hardware vendor, No restoration time has yet been determined.
Aug 28, 14:47 AEST
Update - We have identified that 1 Physical host has gone offline affecting a subset of customers in APAC-1, Our engineers are currently working to restore this as a matter of urgency.
Aug 28, 14:22 AEST
Investigating - We are currently investigating a small subset of customers in APAC1, Their servers may have just become unresponsive. More updates to come shortly
Aug 28, 14:16 AEST
Aug 27, 2015

No incidents reported.

Aug 26, 2015

No incidents reported.

Aug 25, 2015

No incidents reported.

Aug 24, 2015
Resolved - We have now resolved this issue, no customer servers/network were affected during this time.
Aug 24, 05:50 AEST
Investigating - We are currently investigating an issue with the Ninefold Portal, currently users who log in will not be able to view their resources. This is not affecting the servers themselves and is localised to the portal functionality. Further updates to come.
Aug 24, 05:40 AEST
Aug 23, 2015

No incidents reported.

Aug 22, 2015
Completed - Maintenance Window Aborted - Due to an issue encountered during the software upgrade stage (Stage 1), we have made the decision to postpone this upgrade whilst investigations continue.

No customer servers or networks were impacted during this time and testing has been performed by our team to ensure the platform is functioning correctly.

We will be in touch in the coming weeks regarding a re-scheduled window to complete this upgrade.

Regards,
Ninefold Operations Team
Aug 22, 23:19 AEST
Update - Stage 1 (Software upgrade) is still in progress, Engineers are currently working with our Vendor to finalise this process. This will mean that Stages 2, 3 and 4 will begin later than originally scheduled. A further update will be provided shortly as to when Stage 2 will commence.
Aug 22, 21:29 AEST
Update - Scheduled maintenance has commenced.

You will be unable to view, create or modify Ninefold services during this maintenance window.
You can raise and manage tickets via https://portal.ninefold.com at any time during the window.

Maintenance window plan:
Stage 1 In Progress (8PM-9:30PM): Ninefold software upgrade (no impact to customer services)
Stage 2 (estimated 9:30PM-12:30AM): Upgrade network services in APAC2 (APAC2 services unavailable for two 20 minute periods during this window)
Stage 3: (estimated 12:30AM-4:30AM): Upgrade network services in APAC1 (APAC1 services unavailable for two 20 minute periods during this window)
Stage 4: (estimated 4:30AM-5AM) Verify and close maintenance window.
Aug 22, 20:02 AEST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Aug 22, 20:00 AEST
Scheduled - Ninefold will be undertaking scheduled platform maintenance:
Saturday August 22 8pm - Sunday August 23, 5am (AEST).

We will be upgrading our cloud orchestration layer to the latest version, the upgrade will further improve the stability and performance of Ninefold.

A full notice regarding this maintenance has been sent to customers and is available here: http://info.ninefold.com/webmail/36692/336303497/6dce59321bdb6a5df8e75a640e4b0691
Aug 10, 11:12 AEST
Aug 21, 2015

No incidents reported.

Aug 20, 2015

No incidents reported.

Aug 19, 2015

No incidents reported.

Aug 18, 2015
Resolved - Asia Pacific 1 network connectivity has now normalised. Please contact our support team if you continue to see any further issues.
Aug 18, 16:24 AEST
Identified - Our team have just taken further action to normalise the APAC 1 internet connection. Early testing is positive, we are continuing to monitor and will provide further updates shortly.
Aug 18, 16:12 AEST
Update - Our engineers are currently working with our upstream provider to resolve this issue. At this time customers may still experience service degradation. Further updates to come soon.
Aug 18, 15:39 AEST
Investigating - We are currently investigating reports of connectivity issues in Asia Pacific 1, we have isolated the issue to an increase in outbound traffic which our engineers are analysing. Further details to come shortly.
Aug 18, 14:40 AEST
Aug 17, 2015

No incidents reported.