Skip to content

Structure & Navigation in Rendered Notebooks¤

Introduction¤

Hi, I’m _ I‘ll be running this meeting today. This is _, they are here to take notes.

Thank you for taking the time to participate in this study. Before we begin, I’ll give you a brief overview of how this will work. Throughout the hour, I will be reading from a script.

This is completely voluntary. Please let me know at any point if you wish to stop participating.

There’s a few things I want to remind you about before we start.

  • This session will be recorded, and we will be taking notes
  • Recordings and notes will be saved online in a private folder that only our team members can access. Your private info is only available to the people working directly on this project
  • Some anonymous information will be publicly available on our GitHub repo. https://github.com/Iota-School/notebooks-for-all

During this study, we will be exploring a read only webpage that has been exported from a notebook. You can’t edit it like a traditional notebook. I will be giving you a broad task to complete. I will set context for each task such as why you might be doing it and what you hope to achieve. Then I will ask some follow up questions.

As you work through the task, please think out loud. Speak your thoughts as often as you can. Please tell us if something unexpected happens, if something works well, if you think something that could be improved, or if you’re confused. We do not expect all the tasks to be easy to complete with assistive tech and we invite you to complain!

It’s important to know that we are not testing you, we are testing how Jupyter Notebooks behave with assistive tech. There are no correct or incorrect answers. You cannot do anything wrong. We are expecting them to break.

If at any point you have questions, please don’t hesitate to ask. Before we start, do you have any questions?

Participant Introduction¤

First, I want to ask a few things about you and your prior experience with Jupyter notebooks.

  • What is the best way to send you links during this discussion? (in chat, spoken, email?)
  • Would us sending messages in the chat to timestamp certain sections be disruptive for you? (Perhaps if you’re using a screen reader)
  • What operating system and browser are you using today? (We use this info to reproduce errors and support you if needed.)
  • What assistive tech are you using during this session?
  • How have you interacted with notebooks in the past? Tell me about the most common way you interact with a notebook?

Notebook Tasks¤

Now that we know a little bit about you, I’m going to ask you to complete some tasks using an uneditable form of a Jupyter notebook hosted by Space Telescope Science Institute (STScI). Once again, we are asking for honest feedback on the software—please complain.

First, can you please share your screen. When sharing, can you set it so that you either share the sound or share the text with a “speech viewer” ex: In NVDA you can set it to hold the NVDA button (insert or caps) and hit N and then you get options, go down to tools, and select speech viewer.

Task 1¤

Today we want to explore a platform that provides STScI notebooks in a non-editable form with all cell outputs shown. That means all the code has been run, we’re just exploring the document without it changing, like reading a book.

Now I’d like you to open the style guide template notebook. I’ll tell you how to do this part.

  1. Open your web browser of choice
  2. Paste this link: https://eteq.github.io/notebooks-for-all/14jun22_stsci_example_notebook.html https://bit.ly/3pgkFSl

Can you tell me what the title of this notebook is? - Tutorial Title [Y][N]

How easy or difficult was it for you to open the notebook?

If you were to magically make opening this notebook easier, what would you change?

Task 2¤

When you first open a notebook, there’s some information to help you orient yourself.

Can you tell me who the author of this notebook is? - Jessie Blogs, Archive Scientist [Y][N]

What other [descriptive] information would help you orient yourself in the notebook?

Are you satisfied with the organization of this information?

Do you feel that you are missing any information? - What kinds of feedback do you search for to feel confident that you have all the information?

Task 3¤

Next, we’re going to explore the File Information Section.

Show me how you would navigate to File Information. 1. Were they able to navigate to File Information [Y][N] 2. Notes on how they did it:

There is a table in this section. Can you tell me the headers of the first two columns on the table? 1. obsID [Y][N] 2. Obs_collection [Y][N]

How easy or difficult was it for you to navigate to and through the table?

If you were able to magically make accessing this section easier, what would be the same? What would be different?

Task 4¤

There’s more information at the bottom of the notebook, so let’s go there next.

I’d like you to navigate to the final cell of the notebook. 1. Were they able to navigate to the footer and/or citations? [Y][N] 2. Notes on how they did it:

Can you tell me what information the footer (the section at the very bottom) includes? - Citations [Y][N] - Link to top of the page and stsci logo [Y][N]

How easy or difficult was it for you to navigate and read the footer section?

As a reader, what information do you expect to find in the notebook footer?

Task 5¤

Now that you’ve explored the notebook, I’d like you to return to the top of the page

Please return to the top cell. 1. Were they able to navigate back to the top? [Y][N] 2. Notes on how they did it:

Please summarize the general topics this notebook covers for me as if I hadn’t read it.

Do you feel that you are able to access all the information in this notebook? What makes you feel that way?

Follow Up Questions¤

Now that you’ve explored the notebook, I’d like to ask you to reflect on how that experience went and any other feedback you might have.

Please complain – what was frustrating about navigating the Jupyter notebook with your assistive tech?

Did the notebook provide enough (or the correct) information to help you know what to do and what decisions to make?

Did the the notebook give you enough feedback to feel confident that you were completing the tasks successfully?

Are you satisfied with the work-flow of moving through the notebook? (Navigation, number of steps, etc.) If so, what worked well? If not, what do you wish it did instead?

Is there any information, options, or capability that is missing?

Can you see yourself using Jupyter Notebooks in a non-editable form? (Y) (N) Why or why not?

Do you have any other impressions or feedback that you would like to share?

Do you have any questions for me?

Thank you so much for participating!

Interview Debrief¤

After each ethnographic interview you complete, take a few minutes to perform an interview debrief while the session is fresh in your mind. This ensures that key learnings and observations are not lost in the scramble of many interviews or long timelines.

What are our action items based on this feedback?

Any more details on issues we already discovered?

Are there any new questions I should explore in a further script?

What are some key quotes that I heard?