As a team, we\u2019ve been asking ourselves how we can show the success of the programme to our community, and also leave a legacy behind.<\/p>\n
Performance and data are not just for geeks<\/h3>\n
We take data and performance seriously. The University generates large volumes of data, mainly unstructured, and our public website produces millions of interactions with different user groups across a wide range of topics, products and services.\u00a0 Used correctly, this data can give us unique insights into the challenges and opportunities our end-users face, and help us to iterate and improve our services.<\/p>\n
Working for a large and complex organisation brings some challenges in relation to data management and how you track performance, so we\u2019re here to discuss what we\u2019re doing about it as part of OneWeb.<\/p>\n
Starting with OKRs\u2026<\/p>\n
What are OKRs and why are they important?<\/h3>\n
Big and small tech companies often use Objectives and Key Results (OKRs)<\/a> to set goals. Think of these as ways of documenting what is going to be delivered and what the value of those deliverables will be. Using this framework, the OneWeb Programme goals would be set out as follows:<\/p>\n Programme Level Objectives (what we are trying to deliver), for example:\u200b<\/p>\n Key Results (how we will know if we\u2019ve succeeded because success is defined by whether or not the result has value!) might be:\u200b<\/p>\n These programme-level OKRs then cascade down into each project\u2019s OKRs and become more granular.<\/p>\n Performance monitoring is not new to the University\u2019s Digital Team. However, it\u2019s fair to say that, due to the complexity in which we operate, it hasn\u2019t been as effective as it needs to be. . Part of the legacy the OneWeb programme intends to leave behind is a new and improved standard for performance monitoring.<\/p>\n Consistently measured bounce rates and page performance across our new products will certaintly be welcome insights for our Product Managers. But our challenge is to report on how we\u2019re meeting our users\u2019 needs. For example, how do we know if we\u2019ve met a user need such as \u201cDo I want to stay in halls?\u201d? The temptation is to immediately drill down into detail, but we are going to approach this from the top down.<\/p>\n We\u2019ve proposed a reporting hierarchy that would provide users with high level view and allow deeper analysis than before.<\/p>\n At the top, we will report on indicators such as ROI and domain level reporting. This would allow colleagues to drill down into products, such as our education product offering and the digital services that make up our students\u2019 end-to-end journey.<\/p>\n The Undergraduate Journey was mapped in a previous phase of the Programme and identified key user needs at each stage. The various data points, metrics and stats were then mapped to the journey to reveal a conversion funnel. In this way, we hope to provide quantitative analysis that reveals if we are meeting our users\u2019 needs and highlights opportunities for us to deliver a better user experience..<\/p>\n It will take time to build a picture due to the business and users cycles we work with, and we will refine and iterate our approach to performance reporting accordingly.<\/p>\n Of course there\u2019s no substitute for granular, on-page, analytics for product managers who want to know more about how users are interacting with their products (e.g. course pages). The more commonplace dashboards showing bounce rate and average time on page will still be provided at a course level as we\u2019ve done previously. But, where possible, we want to provide information on subject, school and faculty level so that trends can be observed and used to set an internal benchmark.<\/p>\n Another challenge we face, as do lots of other large and complex organisations, is inconsistent data. It\u2019s not always straight-forward to track every interaction, so we\u2019re exploring how we can bridge it where possible.<\/p>\n For now, we use tools such as spreadsheets and dashboards. The emphasis is on providing performance monitoring and insight analytics to the product managers as soon as possible. This means we expect to see some analysis following the Beta launch in phase 3.<\/p>\n Our mantra as a team has always been to collaborate with the rest of the University. We want to empower our stakeholders to put their trust in the performance data, and for our team to have the confidence to make decisions to improve our products. Other off-the-shelf analytics tools can be explored later when the results are shared with a wider audience, with a strong emphasis on transparency. Several organisations have even gone as far as to share their insight analytics with their stakeholders.<\/p>\n We\u2019re keeping a close eye on the UK government<\/a> and Australian government<\/a> sites to see how their product iterates in the open. I hope we got you excited.<\/p>\n We\u2019ll be sharing our progress regularly here. But if you want to know more, please sign up to our Show & Tell and this blog<\/a>, or just drop us an email<\/a> to keep in touch.<\/p>\n","protected":false},"excerpt":{"rendered":" In this post, we\u2019re going to discuss the ways in which we manage the University\u2019s data, and how we\u2019re using it to monitor the performance of the OneWeb programme. The original business case for OneWeb hypothesised benefits such as: improved domain authority link performance to user needs operational cost savings for the University. As a […]<\/p>\n","protected":false},"author":241,"featured_media":945,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"footnotes":""},"categories":[2,26],"tags":[28,27],"class_list":["post-944","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-oneweb","category-performance-monitoring","tag-analytics","tag-performance-monitoring"],"jetpack_featured_media_url":"https:\/\/www.southampton.ac.uk\/blog\/wp-content\/uploads\/sites\/27\/2020\/07\/User-Journey-Map.jpg","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/paLsBb-fe","_links":{"self":[{"href":"https:\/\/www.southampton.ac.uk\/blog\/digitalteam\/wp-json\/wp\/v2\/posts\/944","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.southampton.ac.uk\/blog\/digitalteam\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.southampton.ac.uk\/blog\/digitalteam\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.southampton.ac.uk\/blog\/digitalteam\/wp-json\/wp\/v2\/users\/241"}],"replies":[{"embeddable":true,"href":"https:\/\/www.southampton.ac.uk\/blog\/digitalteam\/wp-json\/wp\/v2\/comments?post=944"}],"version-history":[{"count":26,"href":"https:\/\/www.southampton.ac.uk\/blog\/digitalteam\/wp-json\/wp\/v2\/posts\/944\/revisions"}],"predecessor-version":[{"id":974,"href":"https:\/\/www.southampton.ac.uk\/blog\/digitalteam\/wp-json\/wp\/v2\/posts\/944\/revisions\/974"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.southampton.ac.uk\/blog\/digitalteam\/wp-json\/wp\/v2\/media\/945"}],"wp:attachment":[{"href":"https:\/\/www.southampton.ac.uk\/blog\/digitalteam\/wp-json\/wp\/v2\/media?parent=944"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.southampton.ac.uk\/blog\/digitalteam\/wp-json\/wp\/v2\/categories?post=944"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.southampton.ac.uk\/blog\/digitalteam\/wp-json\/wp\/v2\/tags?post=944"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\n
\n
Project Level Objectives:<\/h5>\n
\n
Key Results (aka Benefits, Outcomes)<\/h5>\n
\n
\n
Tackling our challenges<\/h3>\n
Proposed solutions<\/h3>\n
![\"Performance](\"https:\/\/www.southampton.ac.uk\/blog\/wp-content\/uploads\/sites\/27\/2020\/07\/Performance-Monitoring-Heirarchy.jpg\")
\nImage: the multiple layers of performance monitoring<\/span><\/p>\n![\"Metrics](\"https:\/\/www.southampton.ac.uk\/blog\/wp-content\/uploads\/sites\/27\/2020\/07\/User-Journey-Map.jpg\")
\nImage: snapshot of the UG reporting ideation session<\/span><\/p>\nWe\u2019re not doing it to you, we\u2019re doing it with you<\/h3>\n
Thanks for reading<\/h3>\n