Incident Announcement - thesun.co.uk content loading into eClips Web [CLOSED]

Time:

An incident was declared today at 16:30

Incident Description:

An incident has occurred which is affecting new content from thesun.co.uk loading into eClips Web. New content from thesun.co.uk has not been populated into the eClips Web service since earlier today.

Cause:

Publisher supply has been disrupted

Expected Resolution Time:

Unknown at this time. We have escalated this to News UK and are awaiting an update.

Next Update:

Please expect the next update by 18:30

Update on Friday, October 25, 2013 at 10:25 by Incident Management Team

Incident Update:

New content from thesun.co.uk is still not being loaded into eClips Web. We have re-engaged the publisher this morning and they are actively trying to resolve as soon as possible.

Expected Resolution Time:

Unknown at this time

Next Update:

Please expect the next update by 12:00 today

Regards,

NLA Service Operations Management

Update on Friday, October 25, 2013 at 14:52 by Incident Management Team

Incident Update:

New articles from thesun.co.uk are now flowing into the eClips Web database, including the backlog of articles that were not loading into the database yesterday (24^th)

Cause:

The cause of the incident is still under investigation at News UK.

Resolution Time:

The issue was declared as fixed by the publisher at 11:00 this morning, and the NLA incident resolved at 12:00.

NLA media access - Incident Announcement

Time:

An incident was declared today at 11:45

Incident Description:

An incident has occurred which is affecting Domain Name resolution. There is intermittent loss of access to our products.

Cause:

This is due to a problem at our external DNS provider, Network Solutions and is outside the NLA’s control.

Expected Resolution Time:

Unknown at this time. We have escalated this to Network Solutions & are awaiting a response. Note that users who have accessed sites in the past 24 hours should still be able to access them now.

Next Update:

Please expect the next update by 13:00

Incident Report - eClips missing content

Problem:

Several pages were missing in the eClips feed for publication date 2^nd May 2013. The problem was initially discovered around 1am but due to there being less than ten missing pages at the time, it was not deemed a wider issue. It was only much later that the impact of the issue was discovered & more pages were found to be missing. An incident was declared at 9:15am & the problem was resolved by 10:30am. Missing content was then reprocessed & loaded into the eClips database with all content completed by 4pm

Cause:

A planned change took place yesterday which enabled processing of new content. Unfortunately, one of the configuration files in the change erroneously enabled a dormant part of NLA infrastructure. Despite passing through a quality checking and approval process, this subtle error went unnoticed. When the dormant system became active, it moved some content out of the production workflow, resulting in severe processing delays. The error was corrected and content was returned to the production workflow.

Solution:

In order to prevent a similar situation and impact in future, the change deployment process has been modified to include specific checks for this type of condition. Additional automated monitoring has been put in place to identify this and similar types of unexpected behaviour. Finally, the dormant parts of the system in question will be removed entirely next month.

Incident announcement - eClips missing content

Time:

An incident was declared today at 09:15 am

Incident Description:

An incident has occurred which affected eClips content from the first edition dated 2^nd May 2013. Content from some pages is missing from the eClips feed. This issue has mainly affected regional content and a small number of national pages.

Cause:

The cause is unknown at this time & NLA engineers are looking into the issue.

Expected Resolution Time:

Unknown at this time. Our engineers are working to resolve the issue.

Next Update:

Please expect the next update by 10:15am

Update on Thursday, May 2, 2013 at 10:19 by Incident Management Team

Incident update:

NLA engineers continue to look into the cause of the issue & are working on loading missing content.

Next update at 11:30am.

Update on Thursday, May 2, 2013 at 11:30 by Incident Management Team

Update:

NLA engineers have found the root cause of the issue & fixed it. Missing content is currently being loaded & we expect it to be complete by 2pm UK.

Next update at 2pm. A detailed report of the issue will follow.

We apologize for the inconvenience caused.

Update on Thursday, May 2, 2013 at 14:08 by Incident Management Team

Update:

While a majority of the missing content has been loaded, the production facility continues to review & process anything remaining.

Next update by 4pm.

Update on Thursday, May 2, 2013 at 16:17 by Incident Management Team

All content has been processed. This incident is now closed.

NLA eClips Web Service Incident - Announcement

Time:    An incident was declared today at 15:40

Incident Description:

A fault is causing a complete supply failure for the following News International titles:

Thesun.co.uk

Thescottishsun.co.uk

Thetimes.co.uk

No new or updated articles for these titles will be received while this issue is occurring.

Cause:  The cause of the incident is unknown at this time.

Expected Resolution Time:

The resolution time is unknown at this time.

Next Update:

Please expect an update by 16:30

Update on Wednesday, September 12, 2012 at 16:38 by Incident Management Team

Incident Closure

Incident Description: 

Fault causing supply failure of all News International web titles.

Cause:

An external host on which News International is dependent for supply was failing to respond.

Resolution Time:

The incident was resolved at 16:20.