Network Events

Future / Scheduled events

There are no future events.

Currently affecting the system

There are no events currently affecting the system.

Historic events

CityLink Maintenance

Date: 15 Apr 18 12:00 - 16 Apr 18 12:00
Affected services: CityLink

CityLink have advised a planned outage in relation to maintenance on one of their core switches.
This work concerns Wellington CityLink Ethernet services only - FibreLINK/dark fibre services are not affected.

Outage Details: Monday, 16th April 2018
Work Start Time: 00:10am
Work End Time: 01:00am
Outage Duration: It is expected that there will be an outage of up to 15 minutes

Impacted Services: All Wellington Ethernet Services.

UFB Handover Upgrade

Date: 11 Apr 18 12:00 - 12 Apr 18 12:00
Affected services:

At midnight Wednesday the 11th/Thursday the 12th we will be upgrading our UFB handover with Chorus to allow for future capacity. Wellington based UFB connections will experience a short outage (this should be well under one minute).
If you find your UFB connection is not working on Thursday morning please first try power cycling your firewall and if you still do not have a connection please do not hesitate to call us.

Core Switching Infrastructure Upgrades

Date: 15 Mar 18 12:00 - 16 Mar 18 12:00
Affected services: Hosted

Update 2: This Thursday the 15th of March will be the final upgrade.

Update: This Thursday the 8th of March will be the final upgrade.

Beginning this Thursday the 8th of February and the following 3 Thursdays, at 11:00 PM we will be upgrading our Cisco core network switching infrastructure to stay ahead of customer demand.
Each one of our four core switches (one per Thursday) will need both switching supervisors upgraded. This process will result in a loss of connectivity for all connections (only to the switch being upgraded) for up to 5 minutes.

If you do experience any sustained loss of connectivity to your equipment over this time please do not hesitate to call us direct (Dion Hallam/021 755 873 or Mike Lee/021 755 872)

CityLink Handover Upgrade

Date: 05 Mar 18 12:00 - 06 Mar 18 16:00
Affected services: CityLink

On Tuesday the 6th at approximately 6:30 AM we will be migrating our CityLink customer handover connection to an upgraded circuit. This is a simple change in our equipment and most CityLink connections will not notice any interruption at all.
We know from past experience that some customer firewalls (ZyXEL in particular) may need to be rebooted/power cycled to pick up the new connection. If you find you have no internet access on Tuesday morning please first try power cycling your firewall.

Edge Router Issue

Date: 06 Dec 17 07:54 - 06 Dec 17 09:10
Affected services: DSL

Update: At approximately 4:14 this morning our Cisco edge router to which our ADSL, VDSL, HSNS and a select few CityLink customers connect to became unresponsive for ~2 minutes. Again at 4:28 it became unresponsive, this time for ~12 minutes.
The unit showed nothing untoward in its internal logs and its syslog output. The unit had been up for 5 ?? years so we power cycled the unit at 5:23 and it was back online at 5:26.
We updated one of our hot standby routers with a current configuration, and when the existing unit became unresponsive again at 7:29 we cut over to the replacement unit which we had online by 7:39.
We were then noticing packet loss on the replacement unit and troubleshooting eventually led us to a failing memory module. We down powered the unit at 8:57, replaced both memory modules and the unit was back online at 9:10 with no packet loss either to or through the unit.
We will continue close monitoring of this but are confident there will be no further issues.
We thank you all for your patience and understanding.


We are currently experiencing poor connectivity through our Edge Router servicing ADSL, VDSL and HSNS connections.
We are actively troubleshooting this.

Room 1 UPS Replacements

Date: 25 Nov 17 07:00 - 26 Nov 17 21:00
Affected services:

Update: We will now be replacing the UPS's next weekend (the 25th and 26th).


This weekend (the 18th and 19th) we will be replacing both UPS's and batteries for our Data Centre Room 1.
We will be replacing the first UPS & battery string on Saturday and the second UPS & battery string on the Sunday.
We will be running our entire facility on an industrial 1MW backup power generator for the period of the changeovers, and our own generators and UPS's will still be in play, so we do not envisage an interruption or any other concerns regarding power supply.
We will have our industrial electricians running the project and the UPS suppliers technical team will also be on site.
If there are any last minute concerns we will be postponing the project to next weekend (the 25th and 26th).
If you do experience any suspected loss of power or connectivity to your equipment over the weekend please do not hesitate to call us direct (Dion Hallam/021 755 873 or Mike Lee/021 755 872)

Finally, we apologies for the short notice but have been waiting on confirmation from all the involved parties.

Co-Location Gateway Migration

Date: 20 Oct 17 06:00 - 20 Oct 17 12:00
Affected services: Hosted

On Friday the 20th at 6:00 AM we will be migrating our Co-Location customer gateways to new hardware. The process should be seamless but some customer firewalls might experience an outage if their ARP tables do not update efficiently.
If you have any connectivity issues after 6 AM please do not hesitate to call us on 04 474 0770 or follow your standard support process.

Firewall upgrades

Date: 07 Apr 17 04:00 - 07 Apr 17 09:00
Affected services: E-Mail, Hosted

This Friday morning between 4:00 and 5:30 we will be replacing our core firewalls that protect our website, email and DNS servers. We envisage that this will cause a disruption of 1-2 minutes, during which time web, email and DNS hosted with us will be unavailable.

CityLink Planned Outage Notification

Date: 19 Mar 17 18:00 - 20 Mar 17 09:00
Affected services: CityLink

CityLink have advised the following:
CityLink will be undertaking Wellington network upgrades during our regularly scheduled maintenance window (1st and 3rd Sunday of each month) as per our standard Terms and Conditions. This window will be between the hours of the Sunday 18:00 to the Monday 06:00.

This planned outage is in relation to the core switch being upgraded, while network configuration changes are carried out to optimise network performance. Note that this work concerns Wellington CityLink Ethernet services only - FibreLINK/dark fibre services are not affected.

Please do not hesitate to contact us on 0800 469 783 on Monday morning if you arrive at work and have no internet connectivity.

Data Centre Room One CityLink Switch Upgrade

Date: 07 Mar 17 12:32 - 10 Mar 17 09:00
Affected services: CityLink

This Friday morning between 6:00 AM and 7:00 AM we will be repatching all Data Centre Room One CityLink connections to a new CityLink switch. This will result in an outage of up to 10 seconds as each CityLink circuit is physically re-patched.

Spark Wholesale Fibre Damage

Date: 03 Feb 17 16:32 - 04 Feb 17 00:00
Affected services:

Spark Wholesale have advised of a damaged Fibre between Huntly and Hamilton. As we have multiple domestic upstream providers we are not aware of any impact to us or our customers.

Partial CityLink outage in Wellington

Date: 20 Jan 17 08:17 - 20 Jan 17 17:00
Affected services: CityLink

CityLink update as at 9:15 - Field teams have located a damaged fibre cable. Teams are working on restoring this - current ETR is 17:00 20/01/17

CityLink is currently aware of and investigating an issue causing loss of connectivity for some customers, there is no ETA for resolution at this time.

Wellington UFB Upgrade

Date: 26 May 16 05:30 - 26 May 16 12:00
Affected services:

This Thursday morning between 5:30 and 6:00 we will be migrating our Wellington based UFB connections handover link from our existing Cisco edge router to our new Juniper edge router.
You will experience an outage of up to 5 minutes while the handover link is re-patched to the new Juniper edge router and the new routing changes take effect within our core network.

If you have no internet connectivity on Thursday morning please first try turning your firewall/router off and on again.
If you still have no internet connectivity please do not hesitate to call us on 04 474 0770 or 0800 469 873 and select option 2 for support.

CityLink Outage

Date: 23 May 16 08:34 - 23 May 16 10:06
Affected services: CityLink

CityLink had advised that various buildings in the CBD are currently without power. If you have power but no internet it is probable that an "upstream" building (a building that your CityLink connection relies on) is currently without power, resulting in the loss of your internet.

10:06: CityLink have isolated and replaced some equipment that failed after power was restored. All CityLink connections should now be restored.

Wellington UFB Upgrade

Date: 13 May 16 05:30 - 13 May 16 12:00
Affected services:

This Friday morning between 5:30 and 6:00 we will be migrating our Wellington based UFB connections handover link from our existing Cisco edge router to our new Juniper edge router.
You will experience an outage of up to 5 minutes while the handover link is re-patched to the new Juniper edge router and the new routing changes take effect within our core network.

If you have no internet connectivity on Friday morning please first try turning your firewall/router off and on again.
If you still have no internet connectivity please do not hesitate to call us on 04 474 0770 or 0800 469 873 and select option 2 for support.

Wellington UFB Upgrade

Date: 22 Apr 16 05:30 - 26 Apr 16 12:00
Affected services:

Update: We were unable to stand the connection up on the new router. We have rolled back to the old router and will investigate and reschedule for 5:30 AM on Tuesday the 26th.

On Friday morning we will be migrating our Wellington based UFB connections handover link from our existing Cisco edge router to our new Juniper edge router.
You will experience an outage of up to 5 minutes while the handover link is re-patched to the new Juniper edge router and the new routing changes take effect within our core network.

Edge Routers Upgrade

Date: 31 Mar 16 15:25 - 15 Apr 16 12:00
Affected services: Cable, CityLink, DSL, Wireless

Over the next two weeks we will be migrating all of our customer handover links from our existing Cisco edge routers to a new Juniper edge router.
We will be migrating these handover links between 3:00 AM and 6:00 AM Monday to Friday mornings. You will experience an outage of up to 5 minutes while the handover links are re-patched to the new Juniper edge router and the new routing changes to take effect within our core network.

If you have no internet connectivity on any morning please first try turning your firewall/router off and on again.
If you still have no internet connectivity please do not hesitate to call us on 04 474 0770 or 0800 469 873 and select option 2 for support.

Please note, this does not include any co-location connections within our two data centres as these connections are directly connected
to our core network and do not come in through our edge routers.

Kind regards,
The team @ Xtreme.

UPS Capacity Upgrade

Date: 10 Feb 16 18:00 - 10 Feb 16 21:00
Affected services:

This evening at 18:00 we will be adding additional modules to increase the capacity of the UPS's in our second Data Centre facility.
This process should not have any impact on any services.

CityLink Auckland Issues

Date: 20 Jan 16 13:50 - 20 Jan 16 14:30
Affected services: CityLink

CityLink are aware of widespread issues on their Auckland network causing complete loss of connectivity. They are currently investigating.

CityLink Auckland Issues

Date: 21 Dec 15 14:20 - 21 Dec 15 16:40
Affected services: CityLink

At 14:20 this afternoon CityLink's Auckland network became very unstable disrupting most connections.

CityLink are investigating, we have no updates as yet.

This was resolved at 16:40, no further information has been provided.

Chorus xDSL issues

Date: 02 Dec 15 07:00 - 03 Dec 15 10:00
Affected services: DSL

Chorus have advised the following:
"A number of tail extension services throughout the country may be experiencing a loss of service this morning due to the loss of a Chorus network infrastructure link. Issue is currently under investigation..."

This is causing complete loss of internet for customers effected.

Urgent Spark Wholesale Outage

Date: 23 Oct 15 01:00 - 23 Oct 15 02:00
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

Spark Wholesale have advised us of an urgent outage on our upstream national/international circuits, beginning at 1 AM this Friday the 23rd, for up to 1 hour. This is to replace a failing wavelength router card and clean the fibre terminations.
Xtreme connections may notice a 2-3 minute loss of service as traffic going over our Spark Wholesale connections fails over to our Vodafone Wholesale connections.
No other impact should be noticed unless data is sent to another ISP that depends solely on Spark Wholesale traffic through the failing card for their upstream connectivity, as they will be unavailable for the entire duration of the outage.

Vodafone Packet Loss

Date: 24 Sep 15 10:00 - 24 Sep 15 12:00
Affected services: Cable, CityLink, DSL, Hosted, Wireless

At approximately 10 AM we started experiencing around 40% packet loss on our international Vodafone Wholesale upstream circuit.
To minimise impact we have shut this connection down so all our international traffic is now going through our Spark Wholesale/GGIS circuit.
When Vodafone resolve this we will bring the connection back up.

Update 12:00 Vodafone have advised they had power related issues at Northcote and these have now been resolved.

Border Router Relocation

Date: 10 Feb 15 16:47 - 12 Feb 15 08:00
Affected services:

Tomorrow (Wednesday the 11th) evening after 6:00 P.M. we will be relocating one of our border routers to our new Co-Location facility comms room.
This should not have an impact on any traffic flows as our Vodafone circuits will pick up any traffic that would normally have found a more direct route via the Global Gateway/Spark Wholesale connection.

Going forward this will improve diversity within our core network, as the new facility has 2 new backup diesel generators and 2 UPS’S for complete electrical redundancy.

Kind regards,
The team @ Xtreme

CityLink issues in the upper Willis Street area

Date: 21 Jan 15 12:10 - 22 Jan 15 07:10
Affected services: CityLink

CityLink have advised that a building in upper Willis Street is currently without power and this is impacting on other CityLink connected buildings in the area.

We have no ETA for resolution at this time.

Click through survey in our name

Date: 12 Jan 15 11:47 - 13 Jan 15 11:47
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

We been alerted to a pop up survey purported to be in relation to Xtreme Networks.
If you receive this please do not accept the survey and, if you are happy to (in the strictest of confidence) email abuse@xtreme.net.nz advising the site you were on and the link you clicked to get the pop up survey.

Kind regards and all the best for 2015,

The team @ Xtreme

CityLink Emergency Outage

Date: 18 Aug 14 16:43 - 19 Aug 14 03:00
Affected services: CityLink

Good afternoon,

CityLink have advised of the following urgent outage:
Please be advised that due to operational issues on a core switch an emergency reload is required on Tuesday 19th August. While CityLink is working to minimise disruption, all customers should prepare for a possible loss of service as per the details below.

Between 02:00am – 03:00am there will be a period of up to 10 Minutes where some CityLink circuits will lose connectivity.

Start Date | Time: Tuesday, 19th August 2014 | 02:00am
Finish Date | Time: Tuesday, 19th August 2014 | 03:00am

CityLink have advised that this will not affect all circuits and will not affect any dark fibre circuits.

If you have any issues on Tuesday morning please do not hesitate to call us on 04 474 0770 or 0800 469 873.

Kind regards,
The team at Xtreme.

Name Server Updates

Date: 30 Jun 14 15:28 - 30 Jun 14 15:28
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

Good afternoon,

We now have a new Domain Name Server based in Palmerston North for Geographical diversity.

We will be updating DNS hosting to use this and a Wellington server over the next couple of weeks.
If you have DNS hosting with us you will receive a notification from Domainz advising of the update, you will not need to do anything yourself.

Thank you,

The team at Xtreme.

xDSL/HSNS Outage

Date: 31 Mar 14 19:55 - 31 Mar 14 20:21
Affected services: DSL

The Cisco edge router that our xDSL and HSNS connections come in on automatically shut down for no reason that we can find. Prior to this we were planning an upgrade across our edge routers and will look to bring this upgrade forward. In the mean time we are running continued extensive monitoring on it.

EUBA/HSNS Outage

Date: 18 Feb 14 03:30 - 18 Feb 14 03:50
Affected services: DSL

From approximately 3:30 to 3:50 this morning our EUBA/HSNS handover connection was down while Chorus were performing bearer re-configuration on the circuit.
While we had been advised about the re-configuration of other circuits we were unaware that it would be affecting this circuit.
We apologise to everyone affected by this.

Major email systems fault

Date: 17 Jan 14 06:36 - 20 Jan 14 11:30
Affected services: E-Mail

11:30 AM 20/01/14 Update: After further work over the weekend and this morning we are now confident the issues experienced on Friday have been resolved, we apologise for Friday morning's frustrating incident.

9:45 AM Update: Full mail functionality was restored at approximately 9:30. We will continue investigating the root cause and are planning at this stage to bring the full system back online early (~3:30) Monday morning.

6:40 AM: After running Microsoft's Windows updates early this morning our mail systems stopped accepting inbound emails after a short period of time. We have not yet been able to resolve this on the live/backup systems so we are currently bringing a temporary new server online.

We estimate this will take 1-2 hours (as at 6:40), during which no email services will be available unfortunately.

Please be patient as we work to resolve this for you.

Upgrade to Auckland CityLink Connection

Date: 27 Mar 13 17:02 - 28 Mar 13 07:30
Affected services: CityLink

Tomorrow morning at 7:00 an upgrade will be performed on our CityLink Auckland link to allow additional services to be delivered over it.

The outage will be up to 5 minutes and will only affect our Auckland CityLink customer connections.

CityLink Issues

Date: 10 Dec 12 08:35 - 10 Dec 12 09:07
Affected services: CityLink

CityLink had issues in the Featherston/Johnston Street area due to a faulty UPS. This was resolved around 9:00 AM

Sending mail on port 587 was not working

Date: 16 Nov 12 04:40 - 16 Nov 12 15:46
Affected services: E-Mail

We were recently been made aware that mail could not be sent to us on the "MSA" port 587 since rebooting the servers this morning after installing the windows updates.

This issue has now been resolved.

Spam filtering issues

Date: 12 Oct 12 05:00 - 12 Oct 12 12:15
Affected services: E-Mail

At 5:00 this morning a spam filter update introduced a new set of checks which caused a lot of false positives (rejects of legitimate email). In all cases the sender would have received a notification that the message was not delivered.

Our attention was first drawn to this at 10:00 and we disabled the primary spam filter system at 10:30 as the full extent of the issue became apparent. This resolved the incorrect rejects, but unfortunately allowed more spam through.

We isolated the problem and restored functional spam filtering at 12:15 PM.

We sincerely apologise for any trouble this caused and are investigation options to ensure this does not happen again.

Firewall migration

Date: 10 Aug 12 03:30 - 10 Aug 12 05:30
Affected services:

Early tomorrow morning (between 3:30 and 5:30) we will be upgrading the firewalls that our protect our web, mail and DNS servers.
There will be a brief loss of connectivity to the servers when the new firewalls are patched in, but the process should be largely unnoticeable from the outside world.

Major CityLink Outage

Date: 02 May 12 07:03 - 02 May 12 16:10
Affected services: CityLink

CityLink have advised that they have a fibre down at the intersection of Wakefield Streetand Cambridge Terrace. The fibre appears to have been brought down by a truck.
Engineering staff have been dispatched to site to investigate.
CityLink are unable to provide an ETA for full resolution at this time, but customers and buildings will regain connectivity as individual fibres are repaired.


7:47 Update:
The damaged fibre is being removed and a replacement section installed. Once the replacement section is ready it will be spliced onto the undamaged sections.

CityLink anticipate having the replacement section ready for splicing by 10am. They will then be prioritising the splicing to get the in use fibres repaired first. All services should be restored by 2pm.

CityLink/Chorus Handover Migration

Date: 15 Mar 12 03:30 - 15 Mar 12 09:30
Affected services: CityLink, DSL

On Thursday morning between 3:30 and 5:30 we will be migrating our second CityLink customer connection and our ADSL/EUBA/VDSL/HSNS Chorus handoff to new hardware as part of our IPv6 migration.

We know from past experience that some firewalls (ZyXEL ZyWALL’s in particular) will need to be power cycled in order to pick up our new hardware.

If you or your customers have no connectivity on Thursday morning please try power cycling the firewall in the first instance and if a connection still cannot be made don’t hesitate to call us on either 04 474 0770 or 0800 469 873.

Co-location connection migration

Date: 09 Mar 12 03:30 - 09 Mar 12 09:30
Affected services:

On Friday morning between 3:30 and 5:30 we will be migrating our Co-Location customer connections to new hardware as part of our IPv6 migration.

We know from past experience that some firewalls (ZyXEL ZyWALL’s in particular) will need to be power cycled in order to pick up our new hardware. We will power cycle all ZyWALL firewall's in the datacentre after we have completed the migration.

If you cannot connect to your co-located equipment on Friday morning please don’t hesitate to call us on either 04 474 0770 or 0800 469 873.

CityLink connection migration

Date: 08 Mar 12 03:30 - 08 Mar 12 05:30
Affected services: CityLink

On Thursday morning between 3:30 and 5:30 we will be migrating our main CityLink customer connection to new hardware as part of our IPv6 migration.

We know from past experience that some firewalls (ZyXEL ZyWALL’s in particular) will need to be power cycled in order to pick up our new hardware.

If you or your customers have no connectivity on Thursday morning please try power cycling the firewall in the first instance and if a connection still cannot be made don’t hesitate to call us on either 04 474 0770 or 0800 469 873.

Telecom DSL Services

Date: 15 Aug 11 13:32 - 15 Aug 11 15:00
Affected services: DSL

We have been advised by Telecom Wholesale that they are currently having issues resulting in loss of service for some of our EUBA and HSNS connections, they cannot provide any further details at this time but are working hard to pinpoint and resolve this.

Telecom Wholesale advise this issue was resolved at 14:15

Cable modem network disruption

Date: 11 Jul 11 09:55 - 11 Jul 11 15:00
Affected services: Cable

The recent power cut has taken down the Cable Modem network in the entire Wellington region, we are awaiting an update from the network provider.
This has since been resolved but no further details have been provided.

Core Switch Upgrade and Smartlinx3/Cable Modem Migration

Date: 01 Jul 11 03:30 - 01 Jul 11 05:30
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

On the morning of Friday the 1st of July between the hours of 4:30 AM 5:30 AM we will be upgrading the firmware on one of our core switches. The upgrade should be seamless as the switch has dual supervisor engines, but we will be cold booting the switch after completion to ensure there won't be any surprises in the exceptionally unlikely event of complete loss of power in the future.
The boot process will take up to 5 minutes. This could affect all services should data be going through this particular switch at the time, but we feel this to be a prudent action as we will have a Cisco tech on site should it not reboot successfully.

Also, our Smartlinx3 and Cable Modem migration did not happen on the 17th of June, so this will be happening on the morning of Friday the 1st of July between the hours of 3:30 AM 5:30 AM as well:
You will experience an outage of between 1-5 minutes at the time we perform the migration.
If you have no internet connectivity or the internet seems to be excessively slow on Friday morning, please reboot your firewall in the first instance and if you still have issues don't hesitate to call us on 04 474 0770 or 0800 469 873.

CityLink NOC

Date: 19 Jun 11 20:00 - 20 Jun 11 04:00
Affected services: Cable, CityLink, Wireless

CityLink have advised us of the following NOC event:

Scheduled Outage Status

Scheduled Outage - Scheduled Maintenance Completion of commissioning work on the new core switch at Murphy Street in Wellington

Outage Date

19th June 2011

Maitenance Window

20:00hrs to 04:00hrs

Expected Outage Duration

1x 2 minute to 20 minute outage

Description
CityLink implements a continual improvement programme. Part of this programme requires the completion of it's commissioning of a new core switch housed at the Murphy Street datacentre (formally AT&T house). Customers can expect an outage of between 2 and 20 minutes as a result of this final piece of work.


If any CityLink, Smartlinx3, Araneo or Cable Modem connected customers have any issues on Monday morning please do not hesitate to contact us on 04 474 0770 or 0800 469 873

Smartlinx3/Cable Modem Migration

Date: 16 Jun 11 08:18 - 17 Jun 11 12:00
Affected services: Cable, Wireless

As part of the final stages of our IPv6 implementation path, tomorrow morning (Friday the 17th) between the hours of 3:30 AM 5:30 AM we will be migrating our Smartlinx3 and Cable Modem handoff points to new hardware.
You will experience an outage of between 1-5 minutes at the time we perform the migration.
If you have no internet connectivity or the internet seems to be excessively slow on Friday morning, please reboot your firewall in the first instance and if you still have issues don't hesitate to call us on 04 474 0770 or 0800 469 873.

Cable modem connections down

Date: 04 May 11 07:43 - 04 May 11 08:00
Affected services: Cable

The cable modem network provider has completed work overnight and has left all our cable modem connections off air, we are waiting on an update from them.

Substation and Switchboard upgrades Sunday the 10th April 2011

Date: 08 Apr 11 07:30 - 10 Apr 11 23:59
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

On this coming Sunday at 7:30am the substation in our building at 191 Thorndon Quay, Wellington will be shut down due to the main building switchboard being upgraded, we will be running on our two generators for most of the day and expect no customer impact.
Both of our generators are fully tested every month running on full load and refueled at the end of each test.
We have been aware of this work for 12 months now and have also taken this opportunity to install our own new main switchboard. The installation of both new switchboards will allow us the capacity to draw up to 600 Amps per phase, the option of plugging in a temporary generator if we ever need to and mains diversity up to the data centre on level one.
We will have two electricians onsite for most of the day and as a contingency we have a 3rd generator on standby with our generator service company.

VoIP Issues

Date: 22 Nov 10 16:30 - 23 Nov 10 12:00
Affected services: Cable, CityLink, DSL, Hosted, Wireless

Important information affecting Xtreme VoIP customers only:

This evening at approximately 8:00 we will be making some changes to our VoIP system in order to attempt to overcome issues some customers have been experiencing over the last few days.

Part of these changes will include the renumbering of our VoIP system from the 203.171.32.64/26 subnet where it is currently located to the 203.171.62.0/24 subnet.
We have dropped the TTL's on all our SIP related DNS entries to 5 minutes, so all phones/phone systems using URL's should not have any issues.

The specific IP address changes of interest are:
sip-pbx1.xtreme.net.nz will be changing from 203.171.32.98 to 203.171.62.98
sip-proxy1.xtreme.net.nz will be changing from 203.171.32.104 to 203.171.62.104

You might want to ensure that if you have any VoIP related firewall rules you update them to include the subnet 203.171.62.0/24.

CityLink Issues

Date: 24 Sep 10 09:02 - 24 Sep 10 12:10
Affected services: CityLink

CityLink is currently experiencing some network issues in the Featherston Street and Customhouse Quay areas of Wellington.

12:10 A dead power phase to a key building has been repaired and no further issues should arise.

Cable modem issues

Date: 27 Aug 10 09:14 - 23 Sep 10 16:00
Affected services: Cable

Cable modem connections (primarily in, but not isolated to) the Kilbernie area are experiencing loss of connectivity that may come and go. The cable modem network provider has advised that an update on a single piece of equipment that should not have caused any issues was performed early yesterday morning, no further information has been provided as yet.
Connecting a laptop directly to the cable modem with the connections IP information makes the connection work again, but may not be a permanent fix. Please contact us if you or a customer is having any issues.

23rd September: We believe the issues have been resolved by rolling back an upgrade, but have not received any official report from the cable modem network provider.

Urgent outage

Date: 24 Jun 10 17:35 - 25 Jun 10 12:00
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

We need to do some urgent troubleshooting for a peculiar issue that has arisen since the installation of our first new Cisco core switch.
Once every minute for a second or two, latency through our core router is jumping. We suspect the core router is receiving something from the new switch that is upsetting it and we need to ascertain what is causing this.
At approximately 5:00 tomorrow morning we will be dropping connections from our new switch to our core router for up to 15 minutes in order to try and pinpoint the cause/source of this regular occurrence.

Core switch upgrades

Date: 09 Jun 10 15:58 - 11 Jun 10 12:00
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

This Friday morning between 4:30 and 5:30 we will be patching various border and edge router connections into our first new core switch. This will involve 2 outages of up to 5 minutes per outage. As these breaks are occuring well within our network, customer connections should not need any attention after repatching is completed.

Edge router relocation

Date: 02 Jun 10 04:15 - 04 Jun 10 12:00
Affected services: CityLink, DSL

This Friday morning at approximately 4:30 we will be physically relocating one of our edge routers to make room for new equipment for improved core network redundancy and growth.
This will result in loss of connectivity for approximately 10 minutes on all new EUBA and HSNS connections and CityLink connections with a stub IP address in the 203.171.50.0/24 subnet.
As this is simply a physical relocation all connections should restore automatically, should you experience any connection issues please call us immediately on 0800 469 873 or 04 474 0770 and select option 2.

TelstraClear DoS Attacks

Date: 26 May 10 16:13 - 27 May 10 00:00
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

Over the last week TelstraClear has been the target of DoS attacks coming out of the Asia region which in turn, affects all TelstraClear international traffic.
When we become aware that an attack has commenced we are dropping our TelstraClear international BGP advertisements which forces all international traffic (after BGP advertisement propagation) through our Telecom international circuit.
TelstraClear are keeping us up to date regarding mitigating the effects.

Vodafone/ihug issues

Date: 12 May 10 09:33 - 12 May 10 11:40
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

It appears that Vodafone/ihug have made some routing changes overnight and are currently advertising their routes over WIX but then losing them in their network, we are trying to get an update.

Vodafone/ihug have confirmed that they have issues but currently have no ETA.

Vodafone/ihug ceased advertising their routes on WIX at approximately 11:40 and traffic is now successfully routing over their national circuits.

Power cut and subsequent systems outage.

Date: 16 Sep 09 16:51 - 16 Sep 09 17:40
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

At approximately 16:04 a portion of the Wellington CBD experienced a 3/4 hour power outage.

During this time our UPS's and generator maintained a constant power supply to our core network infrastructure and customer co-located equipment.

Mains power was restored to our premesis at 16:51.

When the mains power was restored our UPS's dropped all power to all equipment briefly as they changed back to mains power supply.

A quick check of our core network infrastructure revealed a core Cisco switch that sits between our core routers (ironically, to provide redundancy to the core routers and border router) was dead and even a complete electrical isolation before repowering was not bringing it up.
At this point we made the decision to reconfigure the primary core router and border router in order to bypass the failed switch and connect the two directly in order to restore connectivity to the outside world. We completed this at 17:40. This restored all services other than unsecure web browsing. We then determined that our port 80 web caching system had not recovered, so we bypassed this system as well. This was completed just prior to 18:00, thus restoring all services.

We also became aware during the network reconfiguration, that our 0800 number and DDI's were not functioning correctly as our VoIP system had not come back up correctly, this system was restored at 17:51.


We are in discussion with our UPS providers regarding why the change back to main supply caused an outage.

We are investigating options to provide complete redundancy from the core to the border, we will post a separate NOC event when we are ready to implement this as there is a good chance that it will require a brief outage.

We need to impliment systems to provice some form of communication with customers should we have another major outage.


We would like to reassure our customers that we strive to cover all bases in our facility. We have two of everything wherever possible. Yesterday morning (as in the morning of the outage) we actually took delivery of a second generater to cope with future growth for power and cooling.

We are pleased that we stayed up for the duration of the power cut, but are bitterly dissappointed that the return to mains power had such disasterous consequences.

Border Router Reboot

Date: 07 Sep 09 16:30 - 07 Sep 09 16:45
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

At 16:30 our border router performed 4 unexpected reboots in a row. We have sent the crashinfo files to our Cisco support vendor for analysis and are awaiting their response.

CityLink Switch Software Upgrades

Date: 07 Sep 09 00:00 - 07 Sep 09 06:00
Affected services: CityLink

CityLink are upgrading the software in their switches and have advised:
"This work will necessitate multiple 5 minute outages on the CityLink network and it is expected that services will be restored within 5 minutes."

If you find you have no connectivity on Monday the 7th please do not hesitate to contact us on either 04 474 0770 or 0800 469 873.

Smartlinx3 Hutt Valley Link

Date: 12 Aug 09 00:00 - 19 Aug 09 12:00
Affected services: Wireless

Smartlinx3 have advised for the 19th of August from midnight through to 5:00 AM: "Our backbone fibre provider FX Networks, is working with Ontrack as part of the Wellington Region Rail Program (WRRP) and is required to complete maintenance work along segments of the FX Networks fibre cable between Wellington and Upper Hutt. They regret that this work does mean that some services to you will be impacted, however please be assured that they are working closely with their contractors to keep any downtime to a minimum.

The following services to you will be affected during the above outage windows:

Outage One
• Internet and tunnel services between Wellington and Lower Hutt
• Internet and tunnel services between Wellington and Upper Hutt

Outage Two

• tunnel services between Lower Hutt and Upper Hutt
• Internet and tunnel services between Wellington and Upper Hutt"

Core Firewall Upgrade

Date: 24 Jun 09 16:00 - 26 Jun 09 06:00
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

On Friday morning between 4:30 and 5:30 we will replacing our core firewalls, this will result in our web, mail and DNS servers being unavilable for a short period of time.
If everything goes smoothly these services should be unavailable for no longer than one minute.
If you have any concerns on Friday morning please don't hesitate to call us on 04 474 0770 or 0800 469 873.

ADSL Outage

Date: 31 Mar 09 12:05 - 31 Mar 09 14:00
Affected services: DSL

Telecom have advised of issues resulting in loss of connectivity for some Wellington and South Island connections. They currently have no ETR.

Power Change Over

Date: 24 Jan 09 13:00 - 25 Jan 09 00:00
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

As a result of a power change over to new UPS's we will have, between 11 PM and midnight, a 20 second outage as our primary core router fails over to our secondary core router, and a 2-3 minute outage as our customer edge routers are restarted.

Widespread Power Outage

Date: 21 Jan 09 08:48 - 21 Jan 09 12:00
Affected services: CityLink, Wireless

There is currently a widespread power outage in the Wellington CBD, surrounding suburbs and parts of the Hutt Valley, this is affecting connectivity across the board.

Araneo Upgrade

Date: 06 Oct 08 15:28 - 06 Oct 08 20:10
Affected services: Wireless

Araneo are expecting a brief outage on their Wellington Wide Area Network this evening, 6th October. Between 8PM and 8:05PM there could be a period of approximately 30 seconds of packet loss as a planned topology change propagates through the network

CityLink cable damage

Date: 14 Aug 08 16:14 - 15 Aug 08 00:00
Affected services: CityLink

Two Citylink overhead fibre-optic cables, one 96-fibre cable and one 24-fibre cable, have been ripped down by a passing bus at the corner of Victoria and Vivian Streets. Citylink engineers are currently on-site assessing the scene and working towards replacement of the damaged sections of cable and restoration of all circuits effected.

Araneo Disruption

Date: 04 Aug 08 06:46 - 05 Aug 08 00:00
Affected services: Wireless

Tonight (4th of August) at 8:00 PM CityLink will be replacing a switch that will cause a break of up to 15 minutes for Araneo Wireless connections fed from Mt Climie and Colonial Knob.

Beijing Olympics Upgrades

Date: 08 Jul 08 17:18 - 11 Jul 08 17:18
Affected services: CityLink

Router upgrades for the Beijing Olympics.

The Beijing Olympics are starting on the 8th and we will all be routing for the Kiwi's.

TVNZ will be streaming this event over the Wellington Internet Exchange (WIX) and to be honest we are not really sure of the increased bandwidth requirements this event will bring.
As both TVNZ and Xtreme Networks peer at the WIX this traffic will not be seen as international.

Each will be Windows Media streams at ~90Kb/s, ~384Kb/s, or 1.5-2Mb/s, and may be some experimental HD Freeview coverage stream at 5-6Mb/s.
Therefore we have decided to bring forward our planned upgrades for our WIX and CityLink Customer Edge connections to 1Gb (1000Mb) to handle the extra traffic we may see.

The WIX upgrade was completed seamlessly today.

Our new CityLink Customer Edge Router is in place and we will be cutting the bulk of our customers over early this Friday the 11th.
The edge router replacement will result in a 5 - 10 second outage for CityLink customers.

We will also be upgrading our main and backup core routers to make sure we have plenty of backbone capacity.
The new core routers are in transit and will be here in the next week or two. We will post another NOC event when we intend to cut these over.
Our core router upgrades will result in 2 x 20 - 30 second outages as the system automatically fails over to the backup router then fails back to the main router.

Our SmartLinx3, Araneo, and Cable Links have plenty of capacity for the number of connections so we will not be upgrading any of these routers or links this month.

CityLink Customer Connection upgrade

Date: 08 Jul 08 17:39 - 11 Jul 08 08:00
Affected services: CityLink

On Thursday the 10th at approximately 5:30 AM we will be transferring customers with IP address starting 203.167.235 and 218.101.3.
On Friday the 11th at approximately 5:30 AM we will be transferring customers with IP addresses starting 203.171.40 and 203.171.41.

This should only result in a 5 - 10 second outage but some firewalls may retain "ARP entries" for a long period of time. If you have no connectivity or your connection appears slow or internittent on Thursday or Friday morning please try turning your firewall off for 5 - 10 seconds and see if this resolves your issues. If you still have concerns please do not hesitate to call us on 04 474 0770 or 0800 469 873

Update on Monday's Power Outage

Date: 28 May 08 08:11 - 28 May 08 08:11
Affected services:

At 11:58 Monday morning the top end of Wellington lost power. Within 30 seconds our generator started and Xtreme itself experienced no downtime. We ran on generator until 12:35 and then cut back to normal mains power. Unfortunately, CityLink’s equipment located at AT&T House did not last through the power cut.
CityLink worked franticly to bring the network up piece by piece and the CityLink network was fully restored at 4PM.

We are meeting with CityLink next week to discuss what they will be doing and what else we can do to ensure future power cuts have less impact on the CityLink network and our customers.

Thank you all for your patience and understanding on Monday.

UPS Upgrade

Date: 13 May 08 15:31 - 19 May 08 06:00
Affected services: CityLink, Wireless

Between the hours of 03:00 and 06:00 on the 19th of May we will be upgrading a UPS in one of our cabinets which feeds our CityLink and Araneo connections. This will result in loss of connectivity for less than 5 minutes while the equipment reboots.

Telecom ADSL

Date: 09 May 08 12:00 - 10 May 08 14:45
Affected services: DSL

Telecom advise they have a congestion issue on some equipmnet in Mayoral Drive. This is causing loss of connectivity for our ADSL customers, they currently have no ETA

Telecom National circuit upgrade

Date: 05 Apr 08 08:30 - 05 Apr 08 08:30
Affected services:

We will be dropping our Telecom national fibre circuit in order to perform an upgrade. During this time all national traffic will continue as normal through our TelstraClear national circuit. Our international circuits will not be affected.

Power Outage in Wellington

Date: 21 Feb 08 11:08 - 21 Feb 08 16:00
Affected services: CityLink

A substation has failed near the top end of Cuba Street affecting CityLink connections in the area.

The power technicians plan to bring the power back in stages as they suspect there will be repais needed on the power cables.

Smartlinx3 in the Hutt Valley

Date: 05 Feb 08 20:20 - 05 Feb 08 22:00
Affected services: Wireless

Smartlinx3 are currently experincing issues in the Hutt Valley resulting in loss of connectivity, they hope to have this resolved by 10:00 PM

Hutt Valley Smartlinx3

Date: 31 Jan 08 07:00 - 31 Jan 08 10:19
Affected services: Wireless

Smartlinx3 are currently resolving an issue that is causing loss of connection for some sites in the Hutt Valley area

CityLink Outage

Date: 25 Jan 08 16:07 - 25 Jan 08 17:05
Affected services: CityLink

CityLink currently have an issue which is resulting in loss of connection in the Willis Street/Victoria Street and Cable Street areas.

Smartlinx3 Planned Outage in Lower Hutt

Date: 12 Dec 07 09:47 - 13 Dec 07 00:00
Affected services: Wireless

Due to an essential network infrastructure upgrade, there will be an outage on the Smartlinx3 network in the Lower Hutt area between 07:00, Saturday 15 December and 17:00, Sunday 16 December. Smartlinx3 are removing an existing mast on top of the HCC building and installing two new ones.

More Power Outages

Date: 11 Dec 07 16:32 - 12 Dec 07 08:00
Affected services: CityLink

Continued sporadic power outages throughout the Wellington CBD are currently impacting Ethernet services on the Citylink network.

Power Loss affecting CityLink

Date: 10 Dec 07 16:15 - 10 Dec 07 17:06
Affected services: CityLink

A massive power event in the Wellington CBD has caused loss off connectivity in the Willis Street/Lambton Quay/Featherston Street/Manners areas.

The CityLink network is now completely restored.

Cable Modem Network

Date: 20 Nov 07 08:05 - 20 Nov 07 13:15
Affected services: Cable

We have received multiple calls advising that cable modem connections are not working, we have logged a fault with TelstraClear and will update this when we have more information.

TelstraClear have resolved issue which was caused by an upgrade the night before.

Email issues

Date: 18 Oct 07 14:33 - 09 Nov 07 17:00
Affected services: E-Mail

Update 09 November: After implementing our new hardware and monitoring the system we are satisfied that the issues are now resolved.

Update 26 October: Our new hardware has arrived, we will be performing a restart on the entire mail system around 17:15 this evening to bring the new hardware online to allow us to begin configuration so that we can implement it early tomorrow morning.

We have recently experienced a large increase in both spam and legitimate email which is pushing out systems to their limits. This has urgently brought forward our upgrade schedule. We will be receiving early next week additional hardware to resolve the issue, until then we have a temprary work around that will unfortunately result in additional spam slipping through from time to time. There will hopefully be no more repeated emails with our work around.

CityLink issues

Date: 06 Oct 07 09:34 - 06 Oct 07 20:09
Affected services: Cable, CityLink, Wireless

CityLink are currently experiencing issues on their network causing connectivity problems for CityLink and Wireless connections.

Border Router

Date: 21 Sep 07 05:30 - 21 Sep 07 05:45
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

We need to replace 2 x ethernet interface cards in one of our border routers.
The current ethernet interfaces are not causing any issues.
This is a recommendation from Cisco.

Router module upgrade

Date: 27 Aug 07 19:39 - 28 Aug 07 05:45
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

At 5:30 am Tuesday the 28th we will be replacing two older modules in our border router with current ones. This will result in an outage of approximately 5 minutes.

Cisco Memory Upgrade

Date: 13 Jun 07 17:27 - 15 Jun 07 03:30
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

At approximately 3:20 on Friday morning we will be upgrading the memory in our Cisco Border router. This will result in appriximately 5 minute outage across all our external connections.

Smartlinx3 UPS Upgrades

Date: 22 May 07 13:04 - 27 May 07 00:00
Affected services: Wireless

The Lower Hutt section of the Smartlinx 3 network will have a few short outages on Sunday 27/05/2007 between 8am and 1pm.
Smartlinx3 will be installing some additional UPS's at a couple of their Lower Hutt POP sites to keep the network running during this period.
The outages will occur while the temporary UPS's are installed and then removed.

Border router migration

Date: 21 May 07 08:48 - 24 May 07 06:00
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

On Thursday the 24th beginning at 4 am we will be migrating to our new Cisco 7206 Border router. Loss of connectivity should be minimal and will occur as the physical connections are moved from the old to the new. There should be no ongoing impact on end user connections.

Loss of connectivity

Date: 20 May 07 08:41 - 21 May 07 00:00
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

A repeat of the issues experienced on the 27th of April have led to the diagnosis of a failing superviser module in one of our Cisco core switches. We replaced it prior to Sunday midnight with a spare off the shelf.

Intermittent connectivity

Date: 27 Apr 07 17:30 - 27 Apr 07 19:30
Affected services: Cable, CityLink, DSL, E-Mail, Hosted, Wireless

One of our Thorndon core switches began to intermittently stop forwarding traffic. We believe we have traced the cause to a trunk port link to one of our Porirua core switches and the issue should now be resolved.

ADSL Connectivity

Date: 18 Apr 07 09:13 - 18 Apr 07 12:00
Affected services: DSL

Telecom have advised of problems on their ADSL network which is affecting email and internet connectivity, they have not given an ETA for resolution at this time.

Thorndon-Porirua Link

Date: 03 Apr 07 09:00 - 03 Apr 07 14:09
Affected services:

Our link from Thorndon to Porirua is currently down due to a fibre cut in the Porirua area. This will only affect customers connected directly to our Porirua Data Centre. We have no eta from TelstraClear for resolution at this time.

Resolved at 14:09

CityLink Wakefield Street Switch

Date: 10 Nov 06 13:00 - 10 Nov 06 16:18
Affected services: CityLink

A power outage at Actrix Building has caused a number of Actrix, NZWireless and other customers in the area (Wakefield St, Blair St, Allen St, Kent Tce, Cambridge Tce) to lose connectivity. It is anticipated that a generator will arrive by 2:30pm and they hope to have all connections restored this afternoon

Mail Delivery

Date: 09 Nov 06 16:00 - 09 Nov 06 17:30
Affected services: E-Mail

Our mail system become severely swamped around 4:00 PM, the system as been restarted, is now operating properly and the backlog cleared.

Power down in The Terrace

Date: 23 Oct 06 10:48 - 23 Oct 06 15:00
Affected services: CityLink, DSL, Wireless

A power transformer under the James Cook Hotel failed this morning causing loss of power to a large portion of the Wellington CBD.

CityLink outage

Date: 04 Oct 06 10:31 - 04 Oct 06 12:00
Affected services: CityLink

CityLink are currently attending to a number of switches which have been affected by power spikes.
ETA Within the hour

Smartlinx3

Date: 15 Sep 06 13:03 - 15 Sep 06 14:02
Affected services: Wireless

Point Howard AP is currently down due to a power outage.