Incident announcement - eClips missing content

Time:

An incident was declared today at 09:15 am

Incident Description:

An incident has occurred which affected eClips content from the first edition dated 2^nd May 2013. Content from some pages is missing from the eClips feed. This issue has mainly affected regional content and a small number of national pages.

Cause:

The cause is unknown at this time & NLA engineers are looking into the issue.

Expected Resolution Time:

Unknown at this time. Our engineers are working to resolve the issue.

Next Update:

Please expect the next update by 10:15am

Update on Thursday, May 2, 2013 at 10:19 by Incident Management Team

Incident update:

NLA engineers continue to look into the cause of the issue & are working on loading missing content.

Next update at 11:30am.

Update on Thursday, May 2, 2013 at 11:30 by Incident Management Team

Update:

NLA engineers have found the root cause of the issue & fixed it. Missing content is currently being loaded & we expect it to be complete by 2pm UK.

Next update at 2pm. A detailed report of the issue will follow.

We apologize for the inconvenience caused.

Update on Thursday, May 2, 2013 at 14:08 by Incident Management Team

Update:

While a majority of the missing content has been loaded, the production facility continues to review & process anything remaining.

Next update by 4pm.

Update on Thursday, May 2, 2013 at 16:17 by Incident Management Team

All content has been processed. This incident is now closed.

eClips Web Incident Notification: Publisher supply failure for "Independent.co.uk"

Time:    An incident was declared today at 9:10a.m

 Incident Description:

A publisher supply failure for Independent.co.uk is currently occurring. We have escalated this to the publisher and are working to resume supply as soon as possible.

Cause:  Publisher supply failure.  

Expected Resolution Time: 

The resolution time is unknown at this time.

 Next Update:

 Please expect an update by 12:00

Update on Friday, April 26, 2013 at 12:03 by eClips Team

Incident Update: The publisher supply has resumed and new article content is now available. We have a coverage gap over the 24/04 and 25/04 and are trying to address this with the publisher.

Cause: Publisher Supply failure cause unknown at this time.

Expected Resolution Time: 

The resolution time is unknown at this time.

 Next Update:

 Please expect an update by 14:00.

Update on Friday, April 26, 2013 at 15:12 by eClips Team

Incident Update: We are waiting on the publisher to determine steps to address the coverage gap.  

Cause: Publisher Supply failure cause unknown at this time.

Expected Resolution Time: 

The resolution time is unknown at this time.

 Next Update:

Please expect an update by 16:30.

Update on Friday, April 26, 2013 at 17:08 by eClips Team

Incident CLOSED

Incident Closure: The content supply for independent.co.uk has resumed. There are some coverage gaps for 24/04 & 25/04 which we are working with the Independent to recover.

Cause: Publisher Supply failure.

Resolution time: 17:08.

NLA eClips Web Service Incident - Announcement

Time:    An incident was declared today at 15:40

Incident Description:

A fault is causing a complete supply failure for the following News International titles:

Thesun.co.uk

Thescottishsun.co.uk

Thetimes.co.uk

No new or updated articles for these titles will be received while this issue is occurring.

Cause:  The cause of the incident is unknown at this time.

Expected Resolution Time:

The resolution time is unknown at this time.

Next Update:

Please expect an update by 16:30

Update on Wednesday, September 12, 2012 at 16:38 by Incident Management Team

Incident Closure

Incident Description: 

Fault causing supply failure of all News International web titles.

Cause:

An external host on which News International is dependent for supply was failing to respond.

Resolution Time:

The incident was resolved at 16:20.

NLA Service Incident - Report

Issue:

Error within the main application that distributes XML files to XML clients.  The application was not working for approximately 2 and a half hours.

Root cause:

Still being investigated and has been escalated to development teams.

Action taken to prevent further incidents:

We have put internal monitors in place to alert the on-call engineer and 9* if this particular issues occurs again.  A temporary solution to fixing the problem is in place until we have a permanent solution.

NLA eClips Service Incident - Report

Problem:

The NLA’s London data centre lost connectivity to the Internet on Friday 20^th March at 18:05. Connectivity was fully restored at 19:39. All NLA services were unavailable during this period.

Cause:

The NLA’s Internet Service Provider detected a problem with one of their upstream peers which affected some downstream customers. NLA was not one of the customers affected by this problem. While the problem was being investigated, the ISP decided to failover all customers (including the NLA) to an alternate peer at 18:00 on Friday. The failover process did not complete as expected and resulted in Internet connectivity failures for some customers - including the NLA.

Solution:

Customers of the NLA’s Internet Service Provider, which were not affected by the original upstream peering issue, were then failed back to the primary peer at 19:20. Normal connectivity was restored throughout the NLA environment by 19:39. The NLA has received assurances from its Internet Service Provider that a review of the actions taken on Friday has been completed and that the procedure for peering changes has been modified to avoid a similarly impacting event in future. Operational steps associated with changes in this procedure will be scheduled for testing during a maintenance window soon.