Back to Insights

The challenges of quality assurance in the Cloud

'To the Cloud' has been the rallying cry for Office 365, and many businesses are considering making the move. There are compelling arguments that can be made for doing so. For instance, you can be assured that everyone within your organization is on the same version with the same updates and patches. You will always be working on the latest and greatest code with the most up-to-date features. It also reduces the burden on your IT department, or eliminates the need for them for very small businesses.

But, the decision of whether to go with Office 365 rests on two key questions:

  1. How customized will you be making your SharePoint sites?
  2. How much risk are you comfortable with?

If you don't plan to customize your site, if you don't intend to use apps, either ones that you create internally, or ones available commercially on the app marketplace (such as the Important Messages app, to pick a completely random example), then there is very little risk in going to the Cloud. However, if you do want your site customized, then you should consider adjusting your processes to the changes that 365 will introduce.

Anyone who has been testing software for as long as I have can tell you that any modification to the code has the potential of introducing a breaking change. The plan for Office 365 is to do a software update every 90 days. We know that Microsoft tests their new features and bug fixes themselves before they will do the upgrade, but they have no insight into the customizations that their users have made to their sites, and can't account for that.

So, what to do?

Be alert

Keep an eye on the Office Admin panel for upcoming planned maintenance. There is no RSS feed for Planned Maintenance, as yet, but there is one for Service Health, which may be useful. Or put reminders in your calendar for when you should start checking for the upgrade announcement.

Note that the announcements have been notifications that the upgrade will happen sometime in the next couple of weeks. And your site can be upgraded at any time within that window.

Be prepared

Have the tests that you want to run ready for when the update happens. If you plan to run the same tests for each upgrade, consider automating them. The other advantage of automation is that you can run the tests on a daily basis until you know for certain that the update has completed. This way, you can be alerted of any bugs in a timely fashion.

Be responsive

Ensure that your development team is ready to address any bugs that come up as a result of the upgrade. The QA team has to investigate the bugs to see what the impact is on the end-user, and if there are any workarounds that can be used until the bugs can be addressed. The portal manager has to help vet the bugs that are found and prioritize them. The developers need to fix the bugs quickly and get fixes to QA for testing as soon as possible.

Be on the lookout

For those of you using apps that were purchased from the SharePoint App Store, the whole app model is too new to know how Microsoft is going to handle upgrades. For now, it's best to keep an eye on the app store on a regular basis to see if there are any updates to the apps that you've purchased. This is especially key in the weeks following an upgrade, as anyone who has owned a smartphone can tell you that upgrading the OS on your phone can make some of the apps that they've purchased break.

Manage expectations

Make sure that your users are informed that an upgrade is coming, and that there may be some instability after it happens. Provide them with a way to report any bugs or irregularities that they find as they use your site, and use the feedback to make your tests for future upgrades more comprehensive.