Abstract
Bexar Data Dive, an online data platform, was created to increase accessibility and use of health and social determinants of health data, such as education, economic barriers to healthcare, and hospitalization rates, to decrease racial/ethnic health disparities throughout Bexar County. A model of user-centered design helped us incorporate community input into the platform. We conducted four interviews and five focus groups to gather information on how people use data — specifically beginner and intermediate-level data users from various educational, governmental, and nonprofit organizations. Then, we launched a community survey to assess specific data needs. Lastly, once the alpha version of Bexar Data Dive was ready, we conducted user testing sessions to measure usability, identify bugs, and gather final feedback before launch. Our findings included many recommendations for incorporating user-centered design in health data management. Participants wanted a health data tool that was easy to use, had the indicators they commonly need, and would provide visualizations for presentations, grants, and other projects.
Keywords: user-centered design (UCD) methodology, health data, platform, SDOH, social determinants of health (SDOH), qualitative
Introduction
Bexar Data Dive is a health data platform created by Community Information Now (CINow), a data nonprofit in Bexar County (San Antonio), Texas. The user-centered design (UCD) process helped CINow build a platform that allows users to access the same integrated social determinants of health (SDOH) and health data in multiple ways, depending on how detailed they would like the data. Our goal was for beginner and intermediate data users to use our platform and visualize their communities, create maps, charts, and graphs, and get quick statistics for their projects and work — be it grant writing, social work, community outreach, or other community-centered projects.
Background
Through our work with educational institutions, government agencies, health organizations, community workers, and other nonprofits, CINow has heard the need for more accessible and easy-to-use health and SDOH data. We have also felt this need ourselves, particularly with regard to having data disaggregated by multiple demographics, like age, race/ethnicity, and sex, and nested geographies. Many of the organizations we provide data to regularly check the SDOH conditions of the communities they work in, guide them in disbursing assistance and aid, and assist in completing both small and large projects that benefit those communities. We set out to create Bexar Data Dive to help local people and organizations in Bexar County have an easier way to analyze, visualize, and disseminate data to continue helping their communities decrease racial/ethnic health barriers. We employed a UCD approach to incorporate community feedback in an iterative and ongoing process through every step of creating Bexar Data Dive.
UCD is a design process of continual improvement that takes into consideration the needs of those who would be using the product or service. The UCD process CINow used to create the data platform Bexar Data Dive involved qualitative methods to gather input from data users, to take into account what they would need from a data platform. Qualitative analysis in UCD provides detailed information on how to best serve the target audience and has been used and advocated for within many types of health research.1 Focus groups were used to gather rich qualitative data on how data users would prefer for a data platform to function and what content it would contain. While CINow conducted focus groups and interviews through Zoom, research has found virtual focus groups and interviews generate the same amount of unique ideas as in-person.2 The UCD process has been shown to be beneficial in creating technology-enabled services,3 and this process is novel in applying the process to building a health data platform.
Methods and Materials
CINow developed and deployed a UCD process to understand requirements and features desired by beginner and intermediate-level data users, who were defined as people in grassroots organizations, churches, small nonprofits, small government agencies, and students. The approach and instruments were designed with technical assistance from the UCD-experienced Data Driven Detroit, one of CINow’s peer organizations in the National Neighborhood Indicators Partnership (NNIP). The UCD process included interviews, focus groups, qualitative analysis, user stories for the web developer, a community survey, wireframes and web development, and user testing, all within the span of a year. IRB approval was not needed, as the information we were gathering was solely to inform the data platform’s development. All materials used during the UCD process can be found in Appendices A-C, and are also described below.
Interviews and Focus Groups
To create the interview questions, we researched other community surveys on data needs, such as those put out by three other NNIP partners: SAVI (a program at the Polis Center at Indiana University-Purdue University Indianapolis), DataHaven (a data nonprofit in Connecticut), and MORPC (Mid-Ohio Regional Planning Commission). Some question topics included participants’ organizations, roles, commonly used datasets, problems they were trying to solve using data, preferred features in a data tool, and preferred ways of receiving data training. You can view the interview and focus group guides in Appendix A.
Recruitment for the interviews and focus groups involved assistance from our partners on the project, as well as our prior connections to data users and students at the UTHealth Houston School of Public Health in San Antonio, Texas. Our project partners included The Health Collaborative, C3 Health Information Exchange, COSA Metro Health, and COSA Information Technology Services Department.
We conducted four interviews (two beginner-level data users and two intermediate-level data users). The interviews helped us refine the questions for the focus groups, which we held in January 2022, and the two advanced-user interviews, which occurred in early February 2022.
The focus groups had similar questions to the interviews, with a few minor edits for clarity. We held five focus groups of three to four people each (mixed beginner and intermediate-level data users). All interviews and focus groups were held through Zoom, and CINow used a function on Zoom to save video, audio, and transcriptions of the participants. The interview and focus group guides in Appendix A also show the project information we provided to participants about how their information would be used and stored responsibly.
Qualitative Analysis and User Stories
Qualitative analysis of the interviews and focus groups was performed by synthesizing participants’ answers to the questions. We began with open coding of general themes that emerged, then axial coding to reveal over-arching and sub-categories from the themes, and lastly, selective coding to gather the final themes that represented most of the participants’ thoughts. Word was the only program used, and it was more helpful to write up results in “list” format with bullet points, explanations, and quotes, rather than a traditional qualitative narrative. This made it easier to translate themes into user stories for the web developer.
From the thematic analysis, we created user stories, which are a common part in UCD, to translate the participants’ desires into a standardized format for the web developer. The user stories do not include how the functionality would be accomplished. That part of the process is saved for the web developer, who has training in implementing end-users’ wants into an intuitive design. With these user stories, the contracted web developer, The Johnson Center for Philanthropy at Grand Valley State University, was able to draft the wireframes of Bexar Data Dive.
Community Survey
Next, we created a community survey in Qualtrics to supplement the information we learned from the interviews and focus groups. The answer choices on the community survey were populated from the themes in the qualitative analysis. This allowed us to get input from more people and narrow down what was most important to data users. For example, during the focus groups and interviews, we asked people about the datasets and indicators they used. And when we emailed the community survey, we provided lists of those same indicators and datasets and asked additional people which ones they would like to access. While the community survey was created to be more closed ended than the focus groups and interviews, we still allowed people a way to write in their answers if they had more say. We may release more community surveys in the future to gauge data users’ reviews of Bexar Data Dive. The questions from the community survey are in Appendix B.
User Testing
CINow conducted user testing in Fall 2022, drawing participants from those who joined us for interviews and focus groups previously. This was conducted through Zoom and involved a short demo of the alpha testing version of Bexar Data Dive, engaging users in a group discussion about how they would find data on the platform and gathering input on how to improve functionality before the beta launch of Bexar Data Dive in October 2022. We held three sessions of three to five people each. The user testing guide can be found in Appendix C.
Findings
Interviews, Focus Groups, and Community Survey Results
We expected most of our users to be beginner and intermediate level data users. Based on the interviews and focus groups, many of the users work in the public sector — connected to nonprofits, universities, or public organizations — and are generally acquainted with data, though they do not consider themselves experts. Their main focuses are accessibility and ease-of-use, as their greatest obstacles include data being too difficult to find or too complicated to use.
Difficulties Encountered with Data
The most common difficulties people had with data were related to two obstacles:
- Finding Data — It was difficult for people to find data for a few reasons. For some, it required too much time to find data because they had to comb through so much of it from multiple data sources. Then, by the time they found what they needed, it would be incomplete or not disaggregated by race/ethnicity, age, other demographics and not at the geographic level they needed.
- Using Data — Once people found the data they needed, the second difficulty they had was using the data. This could be because they did not understand the data and would like more training on data literacy. Or, for other participants, this was because they had trouble formatting the data properly (for one user, they had difficulty formatting longitudinal data).
Data Training
The desire for data training aligned with the difficulties encountered with data. Participants specifically wanted training on how to:
- Find data and get community level information;
- Understand and create visuals/infographics, such as bar charts, pie graphs, and tables in multiple programs (including Excel and Google Slides);
- Learn data literacy and data best practices (such as when should databases be updated with data, how to run queries and capture multiple demographics, and how to understand technical data language); and
- Present data.
Tool Design
Participants had great ideas about their dream data tool. They described specific features they would like to experience:
“If we were able to run our own queries and also have it translate into visualizations that would be really helpful and amazing.”
Participant 1
“Something that can be more user-friendly and flexible.” –
Participant 2. They go on to say how they know people who would like to be able to manipulate the data in the platform, but also download it and manipulate it themselves.
There were also users who had a very descriptive idea of how a tool could be useful to them:
“For me, the perfect tool would be like if we were looking at a map of San Antonio and then it goes to the street level. It would be so cool if you could take that map and put whatever it is that you want to look at. You click on it and put a person there, and it pops up with all the information. On the side, there are little tabs or little checkboxes, and you could say ‘I’m looking for poverty level, race/ethnicity’… And then I could do layers and say this is what their income looks like. And then, you could print it out, and in that print-out it’s an infographic. That would be great. Like mind-blown right there.”
Participant 4
Participant 3 agreed and added on to this by saying it would be cool for students to be able to use an “explore your neighborhood, explore your city” type of feature – especially because they have found that students love dropping the little Google Maps guy down to “walk” around their neighborhoods.
Additionally, several participants mentioned wanting a data tool that:
- Is accessible to different types of users (multiple languages, color-blind mode)
- Provides examples of how the data could be used, or how other people had used the data before
- Is similar to other familiar data platforms without being redundant
- Has training on how to use the tool. Most people preferred live training/live virtual training, video tutorials, or written manuals in that order
- Has good documentation on the data (field names, etc.)
- Has the capability to see indicators interact on maps
A very useful recommendation we received from an advanced user (Participant 5) was to focus on a few datasets that were easy to maintain and update, and a few indicators that people usually have difficulty obtaining. This way, the tool is manageable for a small staff while also not being redundant to other tools that offer common indicators.
As for visualizations and outputs, participants wanted a diverse array of options. Some included trend lines (for different numbers of years), charts, bar graphs, pie charts, and heat maps to show density — all disaggregated by demographics, such as race and age, where possible. It was of particular interest for these visualizations to be presented in fact sheets, data sheets, and infographics. Participants wanted a way to easily print out information and disperse it to clients, co-workers, peers, or themselves for reference.
Datasets and Indicators
Some of the most common datasets discussed were:
- Census Bureau’s American Community Survey (ACS)
- Centers for Disease Control and Prevention (CDC)
- County Health Rankings
- City of San Antonio (COSA) data
- San Antonio Metro Health data
The most prevalent indicators needed by participants were:
- Demographics (age, race, sex, income, education, marital status)
- Food insecurity
- Transportation
- Health insurance rates
- Vaccination rates (COVID-19 and other vaccines)
- COVID-19 rates
- Housing
- Business ownership
The community survey launched shortly after the interviews and focus groups, and it received 31 responses. Primarily, it helped us decide the order we would launch indicators, geographies, and functionalities, based on how participants ranked them. Some items were prioritized for phase one of the site’s beta launch, and some items were delayed for a future iteration. High priority items included Spanish translation of the site, demographic indicators, SDOH indicators, census data, and the ability to compare by certain demographics (such as race and age).
User Stories Results
From the qualitative thematic analysis of interviews and focus groups, we created user stories for the web developer. The three parts of a standard user story are:
1) Who wants it?
2) What do they want?
3) Why do they want it?
Below are the most prevalent user stories we derived from the thematic analysis:
As an entry-level user I want the data I retrieve turned into maps and charts so that I don’t have to visualize the data myself.
As an advanced user I want to be able to download the data and metadata so that I can do my own calculations.
As a community health worker I want current COVID-19 vaccination rates by neighborhood so that I know where to target our outreach.
As a data user I want the tool to be in Spanish so that I can access the data in the language I’m most comfortable speaking.
As a grant writer I want quick and easy access to data and statistics so that I don’t have to spend much time looking for the numbers I need.
As a data user I want to be able to click checkboxes of multiple indicators on a map so that I can view multiple indicators (like race/ethnicity and income) at once.
As an instructor who uses data I want an “explore your neighborhood” feature so that students can engage with data on a personal level.
The web developer then took the user stories and planned out how to implement the data users’ wants in an intuitive way. Some functionalities included were the ability to download the data in csv format, trend charts, an interactive map to select geographies, and a page to quickly explore the data in fact sheet form. Some user stories did not translate well to the wireframes process for various reasons. For example, “As a data user, I want to be able to click checkboxes of multiple indicators on a map so that I can view multiple indicators (like race/ethnicity and income) at once” was not incorporated into the wireframes because implementing this type of functionality is difficult and complicated for users to interpret on a map.
User Testing Results
Lastly, user testing provided CINow with feedback on the functionality and ease-of-use of the tool. Participants helped us find bugs in the site, make the functionalities more visible and intuitive, and affirm the usefulness of the platform. Some of the changes implemented as a result of user testing included: Adding an on/off toggle for viewing labels, moving the filter box to be more viewable, and having the selected indicator be more prominent while applying filters.
Discussion and Conclusion
Bexar Data Dive, a health data platform, was created and launched in 2022 through a UCD process which included interviews, focus groups, qualitative analysis, a community survey, user stories, web development, and user testing. This process was valuable in building a data tool that addressed certain needs of users, including availability of data, access to meaningful geographies, and the ability to quickly pull local stats for grant work, policy writing, health assessments, and other data needs.
While we succeeded in gathering community input on how to build a useful data platform, our greatest obstacle was recruitment, particularly of beginner-level data users. Additionally, not all user stories were able to be translated into the tool, due to limitations of the project. Future projects with similar goals would benefit from establishing fresh connections in the community beforehand so that they can easily contact their target audience.
UCD is a beneficial model for creating health data platforms, particularly the way it was used to create Bexar Data Dive. The iterative process of engaging data users and gathering their feedback in a systematic way resulted in a fully realized health data platform. While UCD is typically implemented by user experience (UX) designers and product managers, this methods research is novel in showing it can be beneficial in creating online tools for public health data users. Additionally, our methods have produced specific materials (Appendices A-C) which can be referenced and replicated by others who manage public health data to incorporate UCD in gathering community input. Future expectations include seeing a decrease in local health disparities, as public health data users have easier accessibility of health data to use in grant writing, policy recommendations, patient care, and other concerted efforts to improve community health conditions. We recommend establishing your audience and building connection early in the process, so that outreach for focus groups, interviews, and user testing is more targeted, personable, and beneficial to the research and data users. This will also help with engagement after the data tool is complete, to ask about their experiences since its launch and evaluate efficacy.
Resources
1. McIlvennan, Colleen K, Megan A Morris, Timothy C Guetterman, Daniel D Matlock, and Leslie Curry. 2019. “Qualitative Methodology in Cardiovascular Outcomes Research: A Contemporary Look.” Circulation: Cardiovascular Quality and Outcomes 12 (9): e005828, https://doi.org/10.1161/CIRCOUTCOMES.119.005828
2. Richard, Brendan, Stephen A Sivo, Marissa Orlowski, Robert C Ford, Jamie Murphy, David N Boote, and Eleanor L Witta. 2021. “Qualitative Research via Focus Groups: Will Going Online Affect the Diversity of Your Findings?” Cornell Hospitality Quarterly 62 (1): 32–45, https://doi.org/10.1177/1938965520967769
3. Graham, Andrea K, Jennifer E Wildes, Madhu Reddy, Sean A Munson, C Barr Taylor, and David C Mohr. 2019. “User‐centered Design for Technology‐enabled Services for Eating Disorders.” International Journal of Eating Disorders 52 (10): 1095–1107, https://doi.org/10.1002/eat.23130
Appendices
Appendix A
Interview and Focus Group Guides
CINow Data Use Interview (Zoom)
Location: _Zoom________________
Date: _________________________
Facilitator: ___________________________________
Note taker(s)/Recorder(s): ___________________________________________
Participant: _______________________________________________________
[Make other facilitator a co-host]
[To get the transcribed closed captions: 1) Make sure on Zoom.us>settings>meeting that the options “closed captions” and “save captions” are enabled. 2) During the meeting, enable “live transcription” and disable “live captions” so that you don’t see the subtitles at the bottom of the screen. 3) Make sure you record to cloud. Once the meeting is over, the transcribed captions will be sent to the Zoom cloud.]
Introduction:
Hello, and thank you for speaking with us today. I am ___________________________. I will be facilitating today’s interview. This is _________________________ (facilitator introduces themselves). They will be taking notes.
Before we begin, if we have any connection issues, we can reconnect through email. (provide email information if they don’t already have it). Also, if you would like turn on or off subtitle settings, you can do so at the bottom of the screen by clicking Live Transcript.
To start I would like to tell you about CINow and the Office of Minority Health project. CINow is a nonprofit organization housed at UT Health Science Center Houston School of Public Health in San Antonio. We help the community by opening up access to information that helps assess community conditions, we help people better understand data and how to use it, and help to define results of that information.
The OMH project is to create an accessible online tool that will strengthen local efforts to reduce health disparities through use of local data. This will allow the San Antonio community to have greater capacity to use information to make changes in our community to address health disparities among racial and ethnic minority populations.
The purpose of this interview is to examine if and how you use data. We specifically would like to know how data can help you in your work and what we can do to make data more accessible to you. Findings from this interview will help build a tool that people, like you, can use to find data, and if you would like to track our progress with making this tool, you can view our monthly updates on CINow.info or email us.
Today’s interview will last about thirty minutes to an hour. During the conversation, we want to get your reaction to some questions about your experience with data. We’re here to listen and learn.
Do you have any questions about the information shared so far?
If it’s okay with you, we will start recording the session so that we have the audio transcript. This will only be used internally.
[Start recording session, make sure to click “Record to Cloud” to get the transcription]
Note: Add question to Zoom in the chat as they’re being asked.
Okay, let’s begin.
- What community or organization do you represent?
- What is your role in the community or in your organization?
- What problem(s)/issue(s) are you, your community, or your organization trying to solve?
- What kind of questions do you try to answer?
- How do you use data in your daily work?
- What tools do you use to capture, understand, or analyze data?
- What datasets do you commonly use? Can also ask: What sources do you usually get your data from?
- What difficulties do you encounter when looking for or using data?
- What are your data goals, that is, what information do you want from the data or wish you could get from data?
- What data do you hope to have for current or future work, like your dream dataset?
- If you needed a data tool/platform, what would you imagine an ideal data tool to look like to be useful for you, your role in your organization, and your organization?
- What indicators does your ideal data tool have?
- What output would you like to get from this tool, e.g., trend line, bar charts, tables, maps, etc.?
- What technical support do you think you might need for the data tool you just described?
- Are there any data skills you would like to improve?
- What kind of skills would you like develop or improve?
- What format works best for you to learn a new tool?
- Videos, in-person trainings, manuals?
- Lastly, would you be available for questions or user testing in January, February, April, or July of next year?
- How much advance notice would you like before a session?
That was the last question. Thank you very much for your input today. Are there any last comments that anyone would like to make?
[Stop recording. The audio transcript should get sent to your Zoom Cloud Recordings online]
CINow Data Use Focus Group Guide (Zoom)
Location: Zoom_______________________________
Date: _________________________
Facilitator: ___________________________________
Note taker(s)/Recorder(s): ______________________
Participant: ______________
[Make other facilitator a co-host]
[To get the transcribed closed captions: 1) Make sure on Zoom.us>settings>meeting that the options “closed captions” and “save captions” are enabled. 2) During the meeting, enable “auto live transcription” and disable “live captions” so that you don’t see the subtitles at the bottom of the screen. 3) Make sure you record to cloud. Once the meeting is over, the transcribed captions will be sent to the Zoom cloud.]
Introduction:
Hello, and thank you for speaking with us today. I am ___________________________. I will be facilitating today’s interview. This is _________________________ (facilitator introduces themselves). They will be taking notes.
Before we begin, if we have any connection issues, we can reconnect through email. (Provide email information if they don’t already have it). Also, if you would like turn on or off subtitle settings, you can do so at the bottom of the screen by clicking Live Transcript.
I would like to review the Zoom etiquette for this focus group. Conducting focus groups virtually does occur in many fields of practice. However, it does present unique challenges that in person focus groups do not have. Let’s review some Zoom best practices:
Zoom Guidelines
- It is preferable for you to be on camera.
- You should not be driving while on Zoom. If in a car, the car should be stopped and safely parked.
- If you are not speaking, please make sure to keep your microphone muted so that it does not interfere with everyone’s audio and does not create additional noise distractions during the focus group.
- Please, no screen shots or photos taken during the focus group.
- If choosing to use the Chat feature, be mindful that these messages can be saved automatically and shared.
- If something unexpected happens, turn off your camera and remain muted until you can resolve it.
To start I would like to tell you about CINow and the Office of Minority Health project. CINow is a nonprofit organization housed at UT Health Science Center Houston School of Public Health in San Antonio. We help the community by opening up access to information that helps assess community conditions, we help people better understand data and how to use it, and help to define results of that information.
The OMH project is to create an accessible online tool that will strengthen local efforts to reduce health disparities through use of local data. This will allow the San Antonio community to have greater capacity to use information to make changes in our community to address health disparities among racial and ethnic minority populations.
The purpose of this focus group is to examine if and how you use data. We specifically would like to know how data can help you in your work and what we can do to make data more accessible to you. Findings from this interview will help build a tool that people, like you, can use to find data, and if you would like to track our progress with making this tool, you can view our monthly updates on CINow.info or email us.
Today’s focus group will last about an hour. During the conversation, we want to get your reaction to some questions about your experience with data. We’re here to listen and learn.
Some general guidelines for today’s session are:
- Have a “kitchen table” conversation. You can bounce ideas off of one another, and I will try to make sure I hear from everyone for each question.
- There are no “right answers.” Draw on your own experiences, views and beliefs — you do not need to be an expert.
- It’s okay to have the same or similar answers to different questions, or even if you would like to skip a question altogether.
- If at any point you forget what the question was, you can ask me to repeat it, or check the chat where my colleague is placing the questions.
- Have Fun!!
Do you have any questions about the information shared so far?
If it’s okay with you, we will start recording the session so that we have the audio transcript. This will only be used internally.
[Start recording session, make sure to click “Record to Cloud” to get the transcription]
Note: Add question to Zoom in the chat as they’re being asked.
Okay, let’s begin.
- Ask Individually: What community or organization do you represent?
- What is your role in the community or in your organization?
- Ask Group: What problem(s)/issue(s) are you, your community, or your organization trying to solve?
- What kind of questions do you try to answer?
- How do you usually find information?
- Do you have a preferred way of finding this information?
- What information would be useful to help work towards solutions and goals?
- Ask Individually: How do you use data in your daily work?
- What tools do you use to capture, understand, or analyze data?
- Ask Group: What datasets do you commonly use? Can also ask: What sources do you usually get your data from?
- What levels of geography would be helpful to you in your work? (Example: zip codes, census tracts, neighborhood level, county, state, etc)
- Do you have trouble when using zip code level data?
- Ask Individually: What difficulties do you encounter when looking for or using data?
- Ask Group: What are your data goals, that is, what information do you want from the data or wish you could get from data?
- What data do you hope to have for current or future work, like your dream dataset?
- Ask Group: If you needed a data tool/platform, what would you imagine an ideal data tool to look like to be useful for you, your role in your organization, and your organization?
- **Optional Question**:What indicators does your ideal data tool have?
- What output would you like to get from this tool, e.g., trend line, bar charts, tables, maps, etc.?
- What technical support do you think you might need for the data tool you just described?
- Ask Individually: Are there any data skills you would like to improve?
- What kind of skills would you like develop or improve?
- Ask Group: What format works best for you to learn a new tool?
- Videos, in-person trainings, manuals, live virtual trainings?
- Ask Individually: Lastly, would you be available for user testing in July?
- How much advance notice would you like before a session?
That was the last question. Thank you very much for your input today.
Ask Group: Are there any last comments that anyone would like to make?
[Stop recording. The audio transcript should get sent to your Zoom Cloud Recordings online]
Appendix B
Community Survey (Administered Through Qualtrics)
Contact
- Name (write-in option)
- Email (write-in option)
- What organization do you work for, and what is your role? (write-in option)
Demographics
- Race/Ethnicity (Select all that apply):
- American Indian or Alaska Native
- Asian
- Black or African American
- Hispanic or Latino
- Native Hawaiian or Other Pacific Islander
- Two or More Races
- White
- Other
- Age
- Under 18
- 18-24
- 25-34
- 35-44
- 45-54
- 55-64
- 65+
Familiarity with Data
- Did you know about Community Information Now before we contacted you?
- Yes
- No
- Other (please specify)
- Which, if any, of CINow’s data tools have you used?
- None
- Viz-a-lyzer.
- Somos Neighbors.
- ACS Sidekick.
- Data Explorer.
- COVID-19 Scatterplot.
- Choose the response you most closely identify with:
- I do not use data or know how to find it.
- I’m aware of data and use data occasionally.
- I use data regularly and am aware of a few data sources.
- I am very familiar with data, how to find it, how to analyze it, and how to interpret it.
Data Tool Preferences. The following questions are about what you would prefer from a data tool.
- What indicators/variables would you like to see in a data tool? (select all that apply)
- Basic Demographics (race/ethnicity, age, sex, population counts, etc.)
- Income
- Food Insecurity/Food Deserts
- Transportation
- Housing/Food/Social Assistance
- Crime Rates
- Voter Registration and Turnout
- Births/Prenatal Care/Birthweight
- Educational Attainment
- Child Abuse Prevalence
- Sexual Assault Prevalence
- Uninsured Rates
- Poverty Rates
- Qualified for Chip/Medicaid
- Cancer Incidence Rates
- COVID-19 Rates
- COVID-19 Vaccination Rates
- Disease Prevalence
- Other (please specify)
- What data sources would you like included in the tool? (select all that apply)
- Census/American Community Survey
- U.S. Bureau of Labor Statistics
- Feeding Texas
- Local Area Unemployment Statistics (LAUS)
- Community Health Needs Assessment (CHNA)
- Behavioral Risk Factor Surveillance System (BRFSS)
- Texas Health and Human Services
- CDC Wonder
- Every Texan
- San Antonio Metro Health
- County Health Rankings
- What geographies would you like to see in a data tool? (select all that apply)
- National
- State
- County
- City
- MSAa
- Zip Codes
- Census Tracts
- Neighborhood
- City Council Districts
- County Precincts
- School Districts
- Other (Please specify)
- What visualizations would you like to see in a data tool? (select all that apply)
- Maps
- Data Tables
- Trend Line Graphs
- Comparison Bar Charts (Geography)
- Comparison Bar Charts (Race/Ethnicity)
- Comparison Bar Charts (Age)
- Other (please specify)
- What sort of training would you like for this data tool? (select all that apply)
- None
- Live Workshops
- Virtual Workshops
- Video Tutorials
- Written Manuals
- Other (please specify)
- What do you use data for? (select all that apply)
- Grant Writing
- Program Planning
- Budgeting
- Other (please specify):
- On a scale of 1 to 5, where 1 is “not at all” and 5 is “very much”, how much do you trust the data you have access to?
- 1
- 2
- 3
- 4
- 5
- How do you usually get updates about data?
- Email/Newsletters
- Social Media
- Print
- Other (please specify):
- Is there anything else you would like to add about what you would like to have in a data tool? (write-in option)
Thank you for taking our survey! For data resources or to keep up with our progress creating this data tool, visit us at cinow.info. You can sign up for our newsletter here.
Appendix C
User Testing Guide
Note: PowerPoint presentation has been converted to text below.
Community Information Now: Bexar Data Dive User Testing
Introduction
CINow is a nonprofit housed in the School of Public Health. Our mission is improved lives through democratized data. CINow provides data, tools, analysis, and training to inform decisions and improve Texas communities. We find, collect, link and analyze, and visually display the data that our neighbors need to improve neighborhood and regional conditions. We serve Bexar County and 11 surrounding counties in South-Central Texas.
The Office of Minority Health awarded us a grant to design and implement a data platform which will give users access to health and SDOH data. The long-term goal is to reduce health disparities by providing organizations, nonprofits, government entities, and businesses with an easy way to access health information and use it to inform decisions and policy. We have also created a new geographic measure, which we are calling Statistically Small Areas, to help visualize indicators and health disparities across meaningful regions across Bexar County.
We began this project almost a year ago. Early in the process, we conducted interviews and focus groups to get input from people about what they would want in a data platform. This allowed us to create user stories based on people’s roles and needs. An example of one of our user stories was “As a community health worker I want current COVID-19 vaccination rates by neighborhood so that I know where to target our outreach.” This was something we heard from the interviews and focus groups, along with the desire for maps and charts that can change colors, access to multiple indicators as once (such as insurance coverage rates by race/ethnicity), and an easy-to-use interface. We implemented as many of them as we could, within our restraints, and we will be uploading more data to it soon.
The general purpose of Bexar Data Dive is to offer data users access to multiple indicators and data sources in one place. We have designed it to be useful to beginner, intermediate, and advanced data users – and we hope that we can improve the health outcomes of Bexar County, especially those in marginalized populations.
User Testing
Expectations for the session
- We want your honest feedback (don’t worry, you won’t hurt our feelings)
- Currently, our focus is on the functionality of the platform, not the design or data
Demo
(Provide brief demo of Bexar Data Dive)
Explore Bexar Data Dive
Take 5 minutes to explore the platform on your own: https://dive.cinow.info
(Use this as a chance to note any questions they have as they explore)
Activity 1
https://dive.cinow.info
Please navigate to the My Community tool
Try to answer the following questions on your own:
- In SSA 34, what percentage of the population is between 5-17 years of age?
- How does that compare to Bexar County as a whole?
(Answers: In SSA 34, what percentage of the population is between 5-17 years of age? 19.3%
How does that compare to Bexar County as a whole? 18.5%)
Activity 1 Continued
We need a volunteer to share their screen
Please navigate to the My Community tool
Please walk us through how you tried to answer the questions by narrating your experiences and expectations.
- In SSA 34, what percentage of the population is between 5-17 years of age?
- How does that compare to Bexar County as a whole?
Everyone can chime in with feedback
(Answers: In SSA 34, what percentage of the population is between 5-17 years of age? 19.3%
How does that compare to Bexar County as a whole? 18.5%
Take notes on the following:
What did the user tester do or try to do?
Ex: (Capture method: Searched “hiking” or display pathway Clicked “Activities > Outdoor > Hiking”)
What were they expecting?
Ex: (Have testers narrate their experience and ask them what they were expecting)
What actually happened? Did it meet expectations?
Ex: (Take notes about what actually happened “Clicked on Start button but nothing loaded” or “She searched for “hiking 60601” and relevant results showed up immediately”)
Did the tester successfully complete the task? (first time, second?)
Ex: (Tester first tried to click “Hiking,” but couldn’t find results. Tried using search second time… and thought the results were correct. Never saw the proper result)
How could their experience be improved?
Ex: (Capture ideas and suggestions - it can be process oriented or design… Anything!))
Activity 2
Please navigate to the Explore Data tool
Try to answer the following questions on your own:
- Choose the indicator “18 – 34”
- What percentage of Hispanic or Latino females are ages 18 to 34 in zip code 78244?
- How has that percentage changed between 2015 and 2020?
- How does this compare to Asian females?
(Answers: What percentage of Hispanic or Latino females are ages 18 to 34 in zip code 78244? 27.6%
How has that changed between 2015 and 2020? In 2015, it was 23.8%, so it has increased. Hint: Navigate to Trend Chart, or toggle the year filter.
How does this compare to Asian females? It’s 15.9% for Asian females. Hint: Navigate to Comparison Chart and compare by race -> Asian)
Activity 2 Continued
We need another volunteer to share their screen
Please navigate to the Explore Data tool
Please walk us through how you tried to answer the questions by narrating your experiences and expectations.
- Choose the indicator “18 – 34”
- What percentage of Hispanic or Latino females are ages 18 to 34 in zip code 78244?
- How has that percentage changed between 2015 and 2020?
- How does this compare to Asian females?
Everyone can chime in with feedback
(Answers: What percentage of Hispanic or Latino females are ages 18 to 34 in zip code 78244? 27.6%
How has that changed between 2015 and 2020? In 2015, it was 23.8%, so it has increased. Hint: Navigate to Trend Chart, or toggle the year filter.
How does this compare to Asian females? It’s 15.9% for Asian females. Hint: Navigate to Comparison Chart and compare by race -> Asian
Take notes on the following:
What did the user tester do or try to do?
Ex: (Capture method: Searched “hiking” or display pathway Clicked “Activities > Outdoor > Hiking”)
What were they expecting?
Ex: (Have testers narrate their experience and ask them what they were expecting)
What actually happened? Did it meet expectations?
Ex: (Take notes about what actually happened “Clicked on Start button but nothing loaded” or “She searched for “hiking 60601” and relevant results showed up immediately”)
Did the tester successfully complete the task? (first time, second?)
Ex: (Tester first tried to click “Hiking,” but couldn’t find results. Tried using search second time… and thought the results were correct. Never saw the proper result)
How could their experience be improved?
Ex: (Capture ideas and suggestions - it can be process oriented or design… Anything!))
Activity 3
Please navigate to the Tables & Downloads tool
Try to answer the following questions on your own:
Choose the indicator “Under 5”
In zip code 78109, what percentage of Black or African American males are under 5 years of age, according to 2020 data?
How would you download this table if you wanted to?
What’s the data source for this indicator?
(Answers: In zip code 78109, what percentage of Black or African American males re under 5 years of age, according to 2020 data? 3.6%
How would you download this table if you wanted to? With the download tool in the top right
What’s the data source for this indicator? ACS, 2020)
Activity 3 Continued
We need another volunteer to share their screen
Please navigate to the Tables & Downloads tool
Please walk us through how you tried to answer the questions by narrating your experiences and expectations.
- Choose the indicator “Under 5”
- In zip code 78109, what percentage of Black or African American males are under 5 years of age, according to 2020 data?
- How would you download this table if you wanted to?
- What’s the data source for this indicator?
Everyone can chime in with feedback
(Answers: In zip code 78109, what percentage of Black or African American males are under 5 years of age, according to 2020 data? 3.6%
How would you download this table if you wanted to? With the download button in the top right
What’s the data source for this indicator? ACS, 2020
Take notes on the following:
What did the user tester do or try to do?
Ex: (Capture method: Searched “hiking” or display pathway Clicked “Activities > Outdoor > Hiking”)
What were they expecting?
Ex: (Have testers narrate their experience and ask them what they were expecting)
What actually happened? Did it meet expectations?
Ex: (Take notes about what actually happened “Clicked on Start button but nothing loaded” or “She searched for “hiking 60601” and relevant results showed up immediately”)
Did the tester successfully complete the task? (first time, second?)
Ex: (Tester first tried to click “Hiking,” but couldn’t find results. Tried using search second time… and thought the results were correct. Never saw the proper result)
How could their experience be improved?
Ex: (Capture ideas and suggestions - it can be process oriented or design… Anything!))
Feedback
What were some things you liked about using the platform? (if anything)
What were some things that could use improvement? (if anything)
What were some things that made it very difficult to use the platform? (if anything)
Did everything work the way you expected it to?
(This portion runs similarly to a focus group)
Author Biographies
Danequa Forrest, PhD, (danequa.forrest@uth.tmc.edu) is a research coordinator with Community Information Now, a local data intermediary in San Antonio, Texas. With a PhD in sociology and double minors in applied statistics and African and African-American studies, she uses her mixed methods skillset to analyze and interpret qualitative and quantitative data. Most of her work involves contextualizing data through the lens of health and racial equity.
Jeremy Pyne, MPA, is a project manager with Community Information Now with a background in public administration and Geographic Information Systems. He has two decades of experience managing and supporting community-based research projects including local indicator reports, online dashboards and interactive mapping tools with the focus to support equitable community change.
Laura McKieran, DrPH, is executive director of Community Information Now, a nonprofit local data intermediary, and an Associate Professor of Management, Policy and Community Health at the UTHealth Houston School of Public Health in San Antonio. She has over 25 years of experience connecting data and community via informatics, evaluation and outcome measurement, community assessment, and data-informed planning.
Cristina E. Martinez, PhD(c), MPH, is a research coordinator at Community Information Now with a background in public health and demography. She specializes in applied demographic methods and advanced statistical analysis.