UX @ Red Gate

My 2 cents on sharing UX findings

| 3 Comments
Categories: Agile development, Usability testing, User research

In the past few months I have been a part of a team working on a project called Migrations. We’ve been developing new feature that will work with SQL Compare and SQL Source Control. This feature is meant to make the lives of database developers and database administrators a little better by allowing users to customise their schema scripts.   I was a part of a project team that consisted of a motley crew of software developers, project managers, product managers, technical authors and user experience designers. A design venture such as this warrants some skilful juggling of user requirements, business requirements and functional specifications. We needed to make sure we kept our users happy, as well as the project team happy. It was an extremely interesting design project as both products are quite well known and already have a pretty strong user base. Too many radical design changes and we have an open can of worms! We’ve been doing our part by conducting usability tests  early and continuously throughout the development cycle. This has been immensely useful with all the countless design decisions that had to be made for the Migrations Project. A typical usability session involved testing the product with our target user base, which in this case, were database developers and database administrators. All of the sessions were conducted remotely with users based across the world. Using the wonders of GoToAssist, a desktop sharing tool and a phone, we gave the opportunity to users to evaluate the product by completing a few tasks while the project team observed user behaviour, noting design pitfalls. Once a session was completed, all the test observers would discuss the key issues that were uncovered. This exercise was an extremely useful opportunity for developers to observe use of the product in real time but it also meant that we managed to gather a LOT of feedback. It is a challenge to handle all this data in an Agile development team due to crunched deadlines. As one of the UX representatives in the project team, I needed to make sure design issues were translated into something meaningful and actionable for developers. As an experiment, Richard and I worked on the idea of sharing UX issues on a whiteboard in keeping with the artefacts of ‘Scrum ecology’ and hoping to increase the visibility of usability test findings. What did we do? We decided, quite simply, to track usability issues on a whiteboard. On the completion of the first usability test we conducted, we made a list of the main issues that a user encountered and put this up on the whiteboard. As we conducted additional usability tests, we continued to add to our list and made note of recurring issues. At the end of a series of usability tests, we were left with a whiteboard filled with usability issues and the number of participants who encountered them. The UX representatives in the project team then rated the issues based on severity and came up with design recommendations for each issue. Each design recommendation was discussed with the entire project team in a very simple stand-up meeting around the whiteboard. Software developers had an opportunity to help us assess the feasibility of implementing our design recommendations and including development changes to the overall development plan, The next steps for all the issues were put up on the whiteboard. This was an extremely useful exercise for both the UX team as well as the development team. We played an active role in discussing usability issues keeping in mind, the overall goals and time deadlines of the project.   The impact of increasing visibility of usability issues So, how did using a whiteboard help us? We got rid of processes that take up time and communicated test findings in a clear and highly accessible manner. Usability study findings, sometimes, have a tendency to get lost in 30 page reports. No one *really* wants to read a long report and writing one of these does take time. It really is asking too much of Agile development teams to read though long reports. Besides, a long report is the typical ‘I’ve gone off and done stuff and here is a bunch of things that are not great with your work!’ AKA the ‘here you go’ approach. I don’t think that works! But does that mean you don’t need UX reports? No. UX reports are a good thing to have. You do need to document study findings. You are a good UX practitioner if you do and especially if you are supporting a product. Down the road there will be times when you need to remember why certain design changes were made and having a UX write up is very handy. It is a good idea to write up a short report of study and the key takeaways for documentation but when sharing UX issues with the project team it does make sense to give development teams something actionable, clear and precise We involved the team in deciding how to go forward with our design recommendations. Each recommendation was discussed with the development team during a stand-up meeting conducted especially to discuss UX issues and the next steps. The development team would let us know where they stood on our design recommendation, we made note of their decision in the ‘Actions’ column against the design recommendation and moved on. Done! Short, sweet and simple! We also used a format that was extremely familiar to the development teams. Whiteboards in a Scrum environment are used to help teams keep track of their progress. UX issues and the next steps for the development team were on a whiteboard, it was right there, for everybody to see! Not hidden in report! Does it sound perfect? Surely there has to be something wrong with that! Here are a couple of points on when using a whiteboard may become a problem…
  1. It isn’t scalable. There will be times when there are more UX issues than whiteboard space!
  2. Sometimes a picture speaks louder than words. A whiteboard is not conducive for this, short of killing a few trees and sticking up screens.
  3. A whiteboard filled with issues is difficult to maintain if you are handling different projects at the same time.
  4. Whiteboard space in an Agile environment can be a hard to find commodity
That said, I did find it useful for continuous, fast and furious UX testing and it was a great way to get stakeholders (product management, project management and Development teams) involved. P.S: Thanks for the pictures and the idea Richard! :)  
  • Marine

    Communicate the outcomes of user tests in a timely manner is really important, especially in agile teams. The worse than can happen is a series of tests (might take a week or 2) then crunching a report to aggregate data (add an extra few days) and be ready with it in the next sprint, when everyone have moved on,
    The approach of immediate feedback on a wite board is interesting.

    I think the idea could be taken forward by adding screenshots of problems and solutions: it is maybe easier to visualise what it is you want to do, for the team to contribute.

    I can also think about another “risk”: it might be tempting for the entire team to start acting on issues as soon as they are on the board…. But sometimes we know that it requires 2 or 3 people with the same feedback to recognise the issue fully :)

    Thanks for this blog post.

  • http://www.red-gate.com Tom Randle

    I like the concept of having a public, prioritised UX wishlist for a product.

    What was the step before filling out the board? How did you guys record your observations then choose what to put on the board?

    I’m not sure what I feel about putting something as basic as changing the name of a button on the board? I’d hope something as simple as that could bypass this process…

  • Revathi

    @ Marine: I agree with the risk of having the team wanting to jump at issues but was hoping that this could be delayed by having the number of users on the board.

    @Tom: Before filling up the board, we gathered notes from test observers (usually included a designer, product managers and dev members), my notes that I would write down as soon as a session is completed and going through the recordings (key sections of the video). I make note of the trending issues and put these up on the board.

    I try to put up most issues on the board even minor ones.Im sure whiteboard space will be a problem with products which have lots of UX issues but I think this method worked well with studies that are held on an ongoing basis. In the case of the Migrations project I was testing every two weeks or so just before or during a Dev sprint.It was quite useful in this case. I am keen to see how I can take this forward with long scale projects.