Category Archives: Feedback

How feedback is helping improve our digital services

Neil Williams great post about how GDS keep improving reminded me that I wanted to provide an update on what we have been doing with feedback recently. I mentioned feedback a while ago.

Since then we have tried to integrate it more regularly in our work. So we have a weekly retrospective and the first thing we do is cover the feedback that has been generated on the website feedback form.

This has covered a range of issues and usually things we can quickly change. So for example there is an area of content called Successful Commissioning which is quite popular. Over the last few weeks we had a number of comments left saying that some of the frequently used abbreviations were not explained. So we went in and changed these.

Another popular area of content are some FAQs about Taxpayer support for the banks.  This content was created as it was a topic the NAO was regularly asked about. So almost every week we get a comment asking ‘can a particular figure be made clearer or do you have more data on x’? We pass these all onto the authors who have taken the necessary action.

There have also been some useful comments about the difficulty of contacting particular people in the NAO. This is slightly more tricky but clearly worth reviewing especially as the on site search shows users are putting in the names of NAO staff.

So oddly we have found all the feedback quite motivating. Because people care, value the content and want it to work even better. It shows that the effort we put into maintaining these pages is worthwhile and we can fix things that do not work well. It’s a feedback win/win.




Leave a comment

Filed under Feedback

How our website user survey works

I mentioned in a previous post that we were planning to install a user survey on the NAO website. As the survey has been live for few days it is worth saying how it works

We are using SurveyMonkey which is activated for 1 in 20 visitors. We will review the frequency of this depending on the response rate and any user feedback. Current completion rate would mean that there would be about 2,500 responses in a year which should create a nice sample size.

The idea is to have the survey run permanently as it will allow monthly analysis. Also it gets round the old problem of saying lets run a survey in x month and then spending weeks getting it up and running.

We have tried to focus on questions which will provide a consistent benchmark of feedback over time but also helps us check that we are providing the right kind of services.

So we have started off by asking what category of user people are to understand the profile of visitors and why they came to the site.

SM users


SM info


SM find


We also wanted to check if we are providing the right type of content formats.

SM content


Finally what is the level of satisfaction.

SM satis


So we hope that without overburdening our users we have a realistic feel for what they need, in what format and if they are happy or not.

Job done.

Leave a comment

Filed under Feedback, User

Do you really want user feedback? Part 3

In this series of posts about feedback so far I have been talking about some external digital services.

What about internal services? Surely you want some feedback on those as well?

What kind of internal digital services could be relevant?

Well intranets jump to mind. So does your intranet allow colleagues to leave feedback on every page: or perhaps to allow reporting of broken links or factual inaccuracies. Even a simple rating system might help identify poorly performing content.

So that’s all very obvious but what other digital services to you have?

How about your procurement system? Can your users give feedback on how that works? Do you care, have you ever asked them. If you cannot build in an integrated form have you created a user forum?

There is probably a long list of applications such as travel booking, room booking, expenses, time sheets. Can you colleagues give feedback on how they work? Again would a user forum be helpful and appreciated?

Finally what about some of those hybrid systems such as recruitment portals. When you advertise a job and push your eager potential recruits to apply how do you know that they found applying a seamless process?

These are just some ideas and I am sure that you can think of many others.

The point is that we are all users of digital services in different roles whether they be government services, commercial organisations or our own work systems.

Normally we all like things to work easily for us and this will be helped if giving feedback, welcoming it and acting on it are taken as integral to digital services.

Leave a comment

Filed under Feedback, Uncategorized

Do you really want user feedback? Part 2

I wrote last week about the more automated kinds of feedback so here are some thoughts and ideas on direct user feedback.

There are a number of approaches that we have followed so far:

Usability testing

In March we ran two days of user testing with people who use the NAO website. Many people would see this as the gold standard of generating feedback. It is very direct as you can sit in a room and usually watch your users struggling to complete the task you set them. Almost everyone who has ran user testing says the same thing. ‘I did not expect that’ ‘I did not realise that our wording was misleading’ etc. One thing is sure – you never forget what users said if you hear them say it while you are watching.

Accessibility testing

We also ran some accessibility testing in March. As far as possible we tried to set exactly the same tasks. As the site is relatively new we expected to get some comments as indeed we did. Again it is an eye opener when you realise how small details can really damage a users’ experience. Try and get someone to demo how they use a screen reader on your website or one you are familiar with. Often it is a horrifying experience.

Follow up

Of course you will need to take the necessary remedial action.

Feedback forms

Another great idea that is proving really effective is to create a feedback form. We introduced these in November last year and the trick is to keep them simple. So we ask as below and get users to say what user type they are – government, private sector etc:












We have also added a simple form to the publications filter to check that users find what they expect:









We want to add a similar form to the search results page.

What else?

We are also building a slightly bigger user survey that we will trigger to run say the first day of every month. This will cover asking about user types;  what sort of content they came to find, guide, report, data, a survey etc. A quick measure of satisfaction and any other comments.

Once the form is live it should give us a monthly profile of who are users are; what they are looking for and how satisfied they are. I look forward to seeing the results.

So there are some ideas. Have you done the same or got some other tips?


Leave a comment

Filed under Feedback

Do you really want user feedback? Part 1

One of the current mantras for government online services is to say that they are customer focussed.

This is an excellent aspiration and it is becoming a standard protocol to build services around iterative user testing.

However once a service is up and running how do we ensure that we are still meeting our user needs?

This is not a definitive list but a description of some of the things that we have done with our website which might be worth considering.

Let’s break this down into indirect (this post) and direct user feedback (the next post).

You will probably collecting some of this information already. For example if you use an analytics package such as Google you should be monitoring the number of ‘page not found’ events.

Typically this is when some content has been moved or deleted and the person who bookmarked it uses the old link and arrives at a dead end. So every month I (and I do mean myself) go through these records and try and work out what the before and after for these links should be. I send these across to our hosting provider to be uploaded onto our server. If things go to plan, and they are, the number of these ‘page not found’ continually decreases.

Yes this is incredibly tedious but the benefits are that it does really focus on fixing the direct problems that users have. At least it does give sense of satisfaction of making something better. It also makes you more aware of your content what is still of interest – ‘why did we ever delete that page?’ Also any more fundamental problems become evident if the same kind of event keeps recurring.

As an extra touch we have added a small feedback box on the 404 page to ask if the user can say what they were trying to find. Hopefully this shows that we are user focussed and genuinely want to help them find our valuable content.

What else?

Do not forget that your other analytics data will tell you how long people stay on your pages and if they visit more of them or ‘bounce’ off to other sites. Did they come to your site by accident or deliberately. Was your content boring or fascinating?

There are also a number of remote monitoring tools that are useful unobtrusive and do not collect personal data.

Google has a handy ‘in page’ view that will show where users clicked on an individual page. We use software called Clicktale that does something similar though with a higher level of sophistication and range. It works very well for example to see how far users scroll down a page or what they click on. The latter can be very useful on forms to see how far people progress through them and when they give up. Clearly we should then act on these insights.

Of course the other key data is page views and downloads. This is your users voting with their mouse as to what content of yours they find interesting.

Another neat trick, which I have not tried yet, is attempting to measure if there are any pages that never get visited. To do this you need to be able to generate a list of all your pages and match against an export from your analytics. Take a look you might wonder why you ever set up that x page saying how great your organisation is about y?

You probably have other tricks so why not let me know your best ones.

Next week I will talk about the feedback that we have generated direct from users.




Leave a comment

Filed under Feedback, Google analytics

How to create a self-healing website

What is a self-healing website?

I mean a website,  sorry a digital engagement platform, that generates its own information/data/feedback that allows improvements to kept being made. It might not be possible to automate everything but the principles are as follows:

User feedback

We have added a short feedback form to each report landing page and flat web page. This means that we can find out if the content was useful or not – at the moment 75% of people think that it is. It also allows users to categorise themselves into some standard user groups which is handy to check that we are getting the ‘right’ type of visitors.

We are going to add something similar on the publications filter to check that users found what they wanted – yes/no – and if not what were they looking for. This might show up if they were on the wrong site, looking for the wrong title etc.

A similar form will go on the 404 page so we can check what went wrong if a page was not found.

What would be great if we could a similar box to the on site search box (Google custom search) to ask if users found what they wanted?

Automatic feedback

Of course the ‘page not found’ is feedback and we have been doing a lot of work fixing broken links.

I have been looking at Clicktale and it is very powerful particularly where there are clickable events such as a filter or contact form to see where users drop out and do not complete tasks. As it also records mouse clicks and cursor movements we can see how and where content is being read. It might lead to shorter text on landing pages for example.

Sitemorse and similar tools also help pick up broken links onsite and to others sites which we then need to fix.

The Digital Accessibility Forum also have a neat tool that allows users to report issues with Accessibility on a website which can help pick up issues to be fixed.

What’s next?

As we think about this further there are no doubt other touch points where we can easily collect data and feed it back into the improvement cycle. A short form which is completely viewable on the same page really works best.

A next step might be to set up an internal Feedback forum, look at all these metrics once a month, and start to make improvements which of course we should feedback to our users.

Leave a comment

Filed under Feedback