So this is the third workshop in the non-visual data science workshop series, welcome all.
This one is, you know, it has kind of almost like two names, but I've kind of just updated it to a
slightly more descriptive name, which is exploring and representing data, but originally I called it,
which is a more exciting title, talking to your data, which I think is still irrelevant.
I'm able to hear, and in this workshop we will be exploring ways to represent different types
of data, such as, we'll get to it in a minute, but categorical data, different kinds of data,
time data, accounts data, and binary data, all different kinds of data, and we'll be thinking
about different ways to represent those non-visually, you know, traditionally our
most, you know, in data science in general, often people will resort to visualizations
to represent these kinds of data. We'll be trying to, we'll be doing it non-visually, and we'll also
be talking a little bit about how to replace visual, specific visualizations. So we'll kind of
be trying to stay primarily in this non-visual realm, but I will speak sometimes to the practical
elements of like, well, what if my colleagues ask for a bar chart, what if my colleagues ask for a
line chart, something like that. I want to get a few administrative items out of the way. This is
the midpoint in the workshop series. This is the third workshop. This is the last workshop that
will be taught by me, and my colleague Sarah Kane, who's a Marshall Fellow at Cambridge in astronomy,
will be taking over for the next two sessions to show you all sonification, which I'm very
excited about. So we'll be switching gears a little, but we'll still be drawing on a lot of what
we've learned in these first three workshops, and using, you know, IPython and some of these
other tools that we have kind of been using all along. Okay, let me go through there. Since it's
kind of the midpoint, I want to go a little more in depth in some of the administrative
stuff. There's been some questions that have come up a couple of times from people,
so I have a list here. Okay, yeah, so first of all, I'd like to thank those of you,
or those, you know, you've kind of maybe come to know them a little bit, the helpers
who are hanging out in the chat answering questions. So I just want to thank again
Alex, Elizabeth, Sarah, Stephen, and also Paul and Monica, who I think Steve,
Paul, and Monica aren't here today, but, you know, they really help things run smoothly and
keep things humming along in the chat while people have questions. So thank you so much to the helpers.
I'd also like to thank, there's been some participants who have been sort of very
actively contributing, and I'd like to thank Nikhil Vohra, who made some pretty extensive
contributions to the first of the workshop series, the first curriculum for the workshop series,
and he did that using GitHub, so he contributed using GitHub. He, you know, GitHub is basically,
you can think of it as a social media site for programmers, but where people store their
source code, download source code. It's sort of social media meets infrastructure, and he used
features on GitHub to collaborate and contributed to the workshops, and that's something you can do.
You can just sort of do open source contributions to the curriculum that is currently hosted on
GitHub if that interests you. You can also send ideas or updates or anything like that by email.
I will say that, you know, if you're interested in entering this field, it can be really useful
to have a bit of a history of contributions on GitHub, and so this is one way to kind of get
started with that, so you can either create an issue, which is basically like you telling me
that there's something that could be changed, and then I'll change it, or you can even do what's
called a pull request, which is a little bit more complicated, but basically where you write up your
own suggestions for how things should be changed, and I can accept it, pull it in, and then once,
if you do that, you'll actually be a contributor on the curriculum, just like Nikhil is now, so
that's something, an option for you. If you're interested in that, you can drop me a line at
Patrick at IotaSchool.com, just if you wanted a little information on how to get started with
those kinds of contributions, or just have an interesting view. Okay, and then we also had
another contributor, Peter Zigmund, who is following along asynchronously, so probably
not in the room, and Peter has been translating the curriculum to Hungarian, which is really
exciting, and I'm excited to post that when he's completed it, so that's a very cool contribution
as well, and I'd also just like, I think, to thank Chancey here, because I think, Chancey, you've
helped me out with some of the ins and outs of using Zoom, and you just pointed out that I have
a broken link on the curriculum and everything, so I really do appreciate everyone who's
contributed so far. Let's see, one thing people have asked us about a lot is a mailing list
for VI data scientists and people looking to get into data science or learning it,
and I think that's a great idea. We have some, a lot of people signed up for this workshop series,
and I think it's the basis of a good mailing list, you know, so I am looking into setting up a, or
self-hosting a mailing list. If that doesn't go well, I guess I'll use free lists or something, but
that, you know, it's a little bit technically involved, but I'm hoping to set it up this week,
so since a bunch of people have asked for that, I do think it's a good idea, and I think we're
going to go ahead and set that up, so excited for that, and I will send an email to the, to everyone
who registered for this workshop when that is live, and also post on some of the main mailing
lists, like Python Viz and so on. Something else people have asked about is, I think there are
some frustrations with GitHub, I think maybe rightfully so, and some people would find
the raw Markdown files that the curriculum is based on to be more useful. Some people would
also kind of prefer to download the videos directly onto their computers, which I totally get,
so we're going to try to facilitate that all by making, we're going to basically make sure the
home that is a little more user-friendly, we're going to do that toward the end of this workshop
series, or right after the series concludes, and we also, I also want to talk with the folks at
Pandas to see if they want to host the curriculum, or if we should create a new website for it, but
basically there will be a website for it, and there will be, you know, it will make it very
easy for you to download the whole curriculum all at once if you prefer to have it on your computer,
and also the videos to download directly, okay? So, that's not going to be right away, but it'll
be probably shortly after this workshop series concludes in early March. Creating other tutorials,
this is something I wanted to mention. We have a lot of stakeholders, you know, stakeholders is a
stupid word, but people who are very involved in various blind and visually impaired organizations
registered for this, and people who are parts of other major organizations, whether it be
academia, science, the, you know, the blindness community. I'd just like to say, you know, I think this has
been a great partnership, and that NumFocus has allowed us to create this very cool curriculum,
make it open to the public. I just want, you know, I haven't pestered you guys that much about
my own organization, IOTA, but I will say I would love the chance to create more resources like
this for the blind community, to teach the blind community some more of these technical
skills, and to create new curriculum, you know, workshop series like this, recordings.
If you're part of an organization where you think they might be interested in collaborating to
create these kinds of materials, reach out. It can be a, you know, I like, you know, in the best
case scenario, it would be for the public. That's kind of where I like to be, but also, you know,
I do work with organizations to create trainings specifically for your organization or resources
specifically for your organization, too. So, you know, drop me a line if you think your organization would
be interested in working with IOTA, and I'll just say, like, two sentences about IOTA. So, IOTA is
consultancy. I'm the head of principal consultant, and we do various things, including creating
curriculum like this, teaching workshops, but also we consult on what I would say some
tricky intersections between accessibility, writing, you know, like creating a curriculum,
and technology, so programming, okay? So, anything that kind of fits into those things, making web
applications that have some connection to those things, stuff like that. That's where we kind of
live. So, reach out, and if you have any projects in mind, okay? And that's probably the only
that's probably the only I mean, one more little plug at the end of this whole workshop series,
but please, you know, let other people know about this IOTA or the kind of work we do,
if it comes up in conversation, because that's very helpful. It allows us to create things like
this. That's just it. That's everything. Okay. And then I would just finally like to thank
Pandas, Numfocus, and Patrick Hoefler, core developer at Pandas or with Pandas,
to, you know, for taking a chance on this workshop series and creating these materials,
and they're really the reason that, you know, this is all that possible. So, all right. So,
that's more administrative stuff than usual, but we'll, you know, it's the midpoint of this
workshop series. Okay. Let's go ahead and get started with the workshops. We are first I'm
going to share my screen, and then we'll open up our IPython environment, and we will do a little
discussion about different kinds of data. Okay? So, I'm going to go ahead and share.
Okay. I'm opening Anaconda prompt. I'm pressing the Windows button. I'm going to type Anaconda
prompt. I typed A and A, and that was a note, but you may need to type a little more in your case.
Remember, you want Anaconda prompt, not Anaconda navigator. And then, when you open Anaconda prompt,
you have your command line. In Windows, we call this command line environment CMD.
There's also another one that some of you have been using called PowerShell, which will more or
less work with this, but I find that the CMD environment is a little more stable for applications
like IPython, and I'm going to maximize the screen. I'm also going to remove the title bar
by pressing alt enter. Okay? So, now I have a nice just the command line. And then,
um, we need to start IPython. So, let's type IPython. That didn't update. So, that probably
means my review is off. So, let's double check that. Okay. So, it didn't it didn't say the usual
version. Let's just do it again. I quit IPython. I'm typing the word exit to quit.
And I'm running IPython again. So, you can hear how it sounds when you start it.
So, you'll hear and there's a whole bunch of other information, but you'll hear the version.
Just a little review and sort of make sure you're all in the right environment.
Um, so, what are we doing today? So, a big part of data science is taking data in certain
that takes certain forms and then representing it in various ways. Okay? And we'll talk about
the different forms and so on in a second. But I, you know, so, what are the ways you can present
data? So, we'll talk if you follow another data science tutorial, chances are the main way that
they will represent data is through a visualization. And I looked up visual definition of
visualization on Wikipedia, and it was the most general sort of definition of all time. It was
basically like everything in the way they define everything as a visualization, like every picture,
everything is a visualization. But basically, my definition is a visualization would be a it's a
way to represent some kind of data visually. Okay? And the most common visualizations are
things like charts and graphs, though there are other kinds of more complicated visualizations.
And we'll get into some of the, but, you know, the most common charts and graphs would be things
like bar charts, pie charts, line graphs, and scatter plots. Okay? There's a whole bunch of
other kinds. I would say maybe 15 to 25 common visualization types, and then a more, a bunch more
esoteric ones. But, you know, those are the ones you're going to encounter most frequently in the
wild. And that's kind of the ones that people reach for most quickly. Okay? Now, what other
ways are there to represent data? Since, you know, you and I and many people on this call don't,
don't, you know, maybe we use visualizations, but we don't really benefit from them in the same way
that sighted people do. Maybe we have to use them to share with our colleagues or something like
that, but we don't benefit the same way. You can use a sonification. So, we're going to talk about
that next week with Sarah's workshop on an introduction to sonification. Basically, it's a
way to represent data through changes in sound, in qualities of sound. So, a quality of sound would
be something like the pitch. Sarah might have her own definition, but that's my probably not as good
definition. Other ways to represent data might be text descriptions. So, I mean, we sometimes say
alt text in certain contexts, but I would also, the more general term would be a text description.
So, maybe the data is in some ways described in natural language, you know? So, if I say to you,
oh, the data is two columns, and it's times, one column is times, one column is
integers, and the integers are between five and 100, and they represent people's grades. You know,
like that's, you know, that's a way to represent data, right? I'm telling you what the data is,
gives you a general idea of the data. Maybe it's not quite as specific, but, you know.
Then there are other ways to represent data, but those are, I would say, are some of the big ones.
The other one that I really want to mention is tactile graphics, which is a really cool direct
way to experience forms similar to visualizations, but they're really, you access them through
raised surfaces or other kinds of tactile interfaces, such as, you know, dots raised
in a digital, you know, some kind of refreshable braille display, and basically it lets you use
your hands or, if you don't have hands, another one of your, another part of your body to
experience the data manually or through tactician, you know, through feeling.
You know, so, you know, for example, you might feel a line chart. I know,
I'm chanting not to call you out again, but I know you did a bunch of work with tactile
graphics early in the pandemic, and once people are flattening the curve, so,
the, you know, and when you have that available to you, you can feel things like a line chart,
which is pretty cool, and a very direct way to experience something like people usually use with
the visualization. Now, today we're going to do something completely different, really,
and this is, there's not really a word for this. I call it talking to your data, okay, and I would
also say it's an interactive data representation is another, maybe, that would be a more fancy way
to say it. I kind of like talking to your data, okay, or a data conversation, and basically what
that is is we load our data into a manipulatable structure, you know, like our data frame in
Pandas, like a series in Pandas, and then we use programming to query the data, poke the data,
transform the data, explore the data, and represent the data, do all sorts of things with the data,
and clean the data is also another thing that you can do, and do all sorts of stuff with the data,
and it has, there is pluses and minuses. We'll talk at the end about some of the advantages and
disadvantages of this approach. I think, in many ways, it's a very exciting approach, and we're
going to jump in with that now. So, what are we going to be representing in this workshop? So,
let's just really quickly talk about types of data that we'll be working with, okay, and these are not
formal types. Remember, in the first workshop, we talked about Python data types, so those were like
things like a string, a list, an integer, which is a number, a float, which is a number, things like
that, okay. Those are formalized because there's only a certain number of them. They exist in
Python. They have certain formal ways that you interact with them, okay, through programming.
These categories I'm going to explain to you now, that I'm going to say types of data, which sounds
a lot like data types. I'm talking about something different. I'm talking about very general ideas,
okay. So, things like saying, if I say time data, that's, there's nothing formalized about that. It
can take a lot of different forms. It could be anything from a string telling you January 6,
1995, with a comma and everything in it, to a number, which is the number of seconds since 1970,
which is, surprisingly, that is actually something programmers do. They'll represent time as a number
of seconds since a very specific moment in 1970, and that moment is called the epoch,
which is pretty wild. Basically, it's almost like computer history started in a particular second
in 1978. It can't from that point. It's one way to represent time. So, that could be an integer,
and the other one could be a string, and other ones could be, there's other ways to represent it,
more specific data types that are specifically for date and time. However, when we say time data,
we mean data that represent points in time. So, it's a more general concept, okay.
What are the ones we're going to work with today? The main ones are numeric, which you've already
worked with before in this workshop series. So, for example, our prices of the Airbnb listings
from last time were numeric data. What else is there? There's the other really big one
that you'll hear a lot about, and this is as close to a formalized one as you'll get,
is categorical data, okay. Categorical data represents a sorting into various categories,
or kind of like buckets. It will sort each entity in a data set into one of a small set
of categories, okay. And we will talk, we will make that a little less abstract in a minute when
we work with it, okay. There is also time data. I just discussed that as representation of points
in time. And one other useful one to keep an eye out for is descriptive, or if you want to use a
fancy word, nominative data. Basically, that's data that is a little more like a unique value.
So, something like someone's email address in a data set, or Airbnb listings that we have,
or an ID number, something like that. Those things, they tend to be unique, or a little
closer to being unique, and they're usually used to identify things, okay. And for it, it doesn't
usually make sense to order it, and it doesn't usually make sense to do things like get a mean
or a median. Like, if you get a mean or a median of ID numbers, that is literally, it's mean,
right, okay. So, basically, it's just descriptive of an entity, but not useful in the same way,
like number, numeric data, or time data, or something like this, okay. Then I will throw
in two other ones, and these are a little bit different because you will most often use these
as an intermediate step when you work with data. And these, but you will sometimes find them in a
data set directly, and those are, the ones we're going to talk about today are count,
so when you count things up, and then also binary data, so whether something, whether a thing about
the entities is true or false. So, for example, if we were to say in our Airbnb data set,
it is underscore Brooklyn. We could make a new column called is Brooklyn, and that would be to
either say true if the listing was in Brooklyn, and false if it was anywhere else, okay. That
would be binary data, all right. And we'll work with most of these. We might not do so much with
descriptive or nominative data in this workshop, but I wanted you to keep an eye out for it.
But we will work with the other ones pretty directly in this workshop. Hopefully,
we can get to most of this. There's definitely a lot. All right. So,
let's jump right in and say, let's work with categorical data a little bit, because this is one
that I would say it's probably the most, one of the most important to wrap your head around
outside of straight numeric data, which we worked with last week and doing things like the mean and
the median. And so, knowing how to work with categorical data, categorical data is really
critical for data science. And even, especially when you're like starting an analysis or something
like that. So, you may be like, okay, I didn't really get your explanation of categorical data.
Let's actually pull some categorical data from our data set, and then we will talk about what
categorical data it is. Again, when we actually have an example in front of us, so it won't be as
confusing and abstract. We'll actually have an example in front of us, okay.
So, the first thing we'll do is we will pull in our data set. And it's going to be the same as last
week. So, I'll tell you exactly what to type. I will give you instructions for people who participated
last week. I'll give you a little bit of a shortcut, okay. So, you can type, you may be able to do this.
df = pandas And then you can type a little bit of pandas,
P-A-N. "P-A-N." And you can, if you press the up arrow, it may fill in our URL and everything
that we had before. So, we fill in a bit of what you wrote before, and then you can press the up
arrow. So, it did fill it all. I'll go through that. Don't worry if you didn't catch that.
I'm going to press enter. Sorry, I skipped a step. You need to first
import pandas. So,"I-M-P-O-R-T. Space. P-A-N-D-A-S."
P-A-N-D-A-S. Okay. And then we'll run our DF equals. And I will go through it again. So,
in left. Okay. It worked for me. So, what you want to type, if you don't have it stored in
your history, I tried to tell you a way to access this line from your history from last week. If
you don't have it in your history, maybe you're doing this for the first time. Maybe you can't
detect it for some reason. Go ahead and type this. DF space. That's D as in dog, F as in foxtrot.
And then space equals space pandas, P-A-N-D-A-S, dot read underscore CSV. Open parenthesis.
And now you want to pin in a URL as a string. So, we do a double quote. Then do HTTP
colon forward slash forward slash bit.ly, B-I-T, period, L-Y, forward slash NYC
BNB. So, that's Bravo November Bravo, you know?
New NYC for New York City. And then BNB. Bravo November Bravo. B as in boy, N as in Nancy.
And then finish the double quotes, do a right parenthesis, and then run it.
Hopefully, helpers, maybe you can also paste it in so people have the option to copy it.
And it's also present in the written curriculum that we're going to follow along.
Okay. Once you have it loaded in, you should be able to do DF. Just type DF by itself,
and you have your data. That's enough. It's staying the number of rows, and there's a
whole lot of information. It's probably read for like 20 minutes if we, there's a ton of
information. But basically, it's our big data set. We could do, you know, you could do things
like on it like we did last week. We can do len on it to see how many rows. DF.shape to see how
many columns and rows. You could do all the things that we did before. DF.columns to see the columns.
These are things we all did in the last workshop. You could do them, this is the same data as we
used last week in this data frame object. Okay. Now we have our data in front of us.
Let's pull out some categorical data and think about it. Okay. And explore it. So, I'm going to
clear my screen because it helps me out cognitively to clear my screen before I do something new.
So, I did control L, and I heard, you know, in left bracket, five, right bracket. We're entering
the fifth line that we've entered so far. Let's do "DF.neig." And I'm going to let it fill it in
by pressing tab. So, go ahead and press tab. Neighbourhood. Because it was filling in the
rest of the word neighborhood. I say to do this because for some reason this data set, it spells,
which is fine, but some of us are Americans, we don't use the U in neighborhood. Okay.
So, in British English, they say a neighborhood with a U. In American English, we don't do the U.
So, this is a point of confusion. But in this data set, it's spelled the British way. Okay.
So, I recommend using the tab and just letting the computer fill it in. There will be no confusion,
no problems. Okay. I recommend to do that whenever you can. Okay. So, we want to do DF.neighbourhood,
and then I'm going to do a period. So, remember, it's neighborhood with U, dot, dot, head,
"in left bracket, H, E, A, D, head, left paren, right paren." So, I did head, left parenthesis,
right parenthesis. So, it's got DF.neighborhood.head. Okay. And we should get the first five items from
this column. Okay. The neighborhood column in our Airbnb data set. All right. We're getting
"the first five items. Out left bracket, five, right bracket, colon, zero, Kensington, one,
Midtown." Oh, I'm sorry. I meant to do a... This is fine, actually. So, this is our data for
individual neighborhoods. What I actually wanted to use is neighborhood group,
which will tell us something a little more specific about this categorical data. So,
if you want, you can type it again. That's probably the easiest way. You can also press
up and backspace your answer. I'll type it again. So, I'll do DF.neighbourhood. I'm letting it fill
it in. And then I'm just going to go to underscore and tap in and then fill in group. So, it's
neighbourhood underscore group. DF.neighbourhood underscore group. That's what we want. And
essentially, you'll see in a second what this corresponds to if you're familiar with New York
City, the different boroughs in New York City. So, Manhattan, Queens, Staten Island, the Bronx,
and Brooklyn. Okay? So, let's... And that's our column. Let's do head so we only get the first
five. So, it should be DF.neighborhood underscore group dot... Open parenthesis. Okay. So,
open parenthesis, close parenthesis. DF.neighborhood underscore group dot head open parenthesis,
close parenthesis. You'll notice something. And this is what I wanted to show you. And this is
why the neighborhood wouldn't really help us with this. This is the main characteristic
of categorical data is that the values are repeated. Okay? So, this column consists of a
relatively small set of values. So, Manhattan, the Bronx, Queens, Brooklyn, and Staten Island.
And they're repeated over and over again. And when they appear in this column, they tell us that
the item in the row, the entity, which in our case, it represents an Airbnb listing from 2019.
Okay? Each row in this data set is an Airbnb listing from 2019. And when one of these items
appears in this column, it tells us, hey, that listing is from the Bronx. It's from Queens. It's
from Manhattan. Okay? And you'll notice the two here that are appearing most frequently are
Manhattan and Brooklyn. Okay? We can confirm that in a minute. But when you see repeated data like
this, when you see repeated information, and it could be numbers too, but if it's a small number
of items that are repeated, then that gives you an indication that it's categorical data. Okay?
Also, if you think about, hey, this could basically be considered a category. So,
something like neighborhood group, that sounds like a category. And neighborhood sounds like
a category. Okay? Then it's going to be categorical data. It sorts the data, the entities in the data
set into various little buckets. Okay? Now, what do you do with categorical data? How do we
represent it? How do we explore it? So, the first thing you should do with categorical data typically
is to get a set. Okay? So, there's two things you should do right away with categorical data.
To get a set of all the categories and then to count the various items. Okay? These are the two
things we're going to do in order. Okay? So, let's do the line we had before, but now we're going to
put the we're going to put the function set around all of it. Okay? So, I'll type it again. Set.
"Set." Open parenthesis. And then df.neighborhood underscore group.
That's the line in underscore. Group. So, it's neighborhood group, underscore group,
dot. And let's not do the head. It doesn't really matter here. So, we could do set df.neighborhood
group. Set open parenthesis df.neighborhood group. Those are our five items. "Bronx, Brooklyn,
Staten Island, Manhattan, and Queens." Okay? Those are the five items. Those are our categories. And
we use the set function. It basically says give us the unique values from this data set. Okay?
And what it returns here is a list-like object, which is a set object. But it's very similar to
a list. Okay? So, that tells us how many categories we have to work with. The other thing that I
recommend doing with a with categorical data, once you know that it is categorical data,
even if you suspect it's categorical data, is to count it. And I would also say that set is very
good at telling you, is this categorical data? So, if you run the set function on a column,
then you will learn, you will see, oh, is it a small number of categories of repeated values?
If it's a small number of items that come back, then that's a strong indication that
where you're working with is categorical data. But if it's almost as many items as we have
rows in the data set, then it's probably not categorical data. It's probably something
like descriptive or nominative data. If you ran a set on email addresses, it will pretty much
be the same number of items as the full data set. Okay? So, let's do let's now let's do a count.
Okay? And we'll talk about counts. So, let's do df.neighborhood group again.
df.neighborhood group. Remember, I'm making a lot of use of tab here. This is where tab is
going to come in useful. We're going to be writing longer lines in this session.
So, we're going to be doing I'm going to use tab a lot more. Hopefully, you're getting a little
bit used to this environment. I will also say, you know, if I'm moving sometimes a little bit
fast in these workshops, you know, which is kind of the nature of some of these more online
workshops. In-person workshops, they tend to go a lot slower, and there's time to work.
But, you know, we're kind of doing this for the recording a little bit. And you can review
afterward. And there's a raised hand. We'll get to that in one second.
There's a recording, and there's a written curriculum. So, if you're feeling a little
stress at any point in this workshop, just remember that those resources that they're
available to you. And Sarah and I are available. We're just an email away. And there's also the
office hours on Thursday. So, a lot of resources available to you if you're feeling a little
learning stress. We had a raised hand. Is that an intentional raised hand? I mean,
we can, if you want to get on the mic and ask a question, you're welcome.
I hit that button all the time myself, too. So, if you do, you know, if it was a question you
wanted to ask, just feel free to get on the mic. Okay. So, DF.Neighborhood group. And now, so,
we had to DF.Neighborhood underscore group. And now, we're going to use a new method. And we're going to use
this method a lot. Okay. So, try to remember what it's called because we're going to be using it a
lot. Okay. But you'll get a little reinforcement as this goes on. Let's do DF.Neighborhood group.
Value underscore counts. So, it's DF.Neighborhood underscore group. Value underscore counts.
Open parenthesis. Close parenthesis. I kind of hate the name of this method because it's hard
to type. It has an underscore in it. It's kind of long. And I use it a lot. It's one of the methods
I use the most when I'm doing a data analysis. Okay. So, and we're going to use it a lot, too.
So, remember that tab is your friends. In this case, it might not be that useful to you. But,
you know. All right. Let's run it and see what we get. Out left bracket eight right bracket colon
"Manhattan 21,661. Brooklyn 20,104." So, okay. What are we listening to here? We'll zoom in a second.
We're hearing a series which has labels. Remember from last week, what we have the series is
represented with two columns. For us, screen reader users, it's that each line has on the left
first a label. Okay. Which is the index. And on the right, it has a value. In this case,
it's a number. Okay. So, we're listening to a series. The labels are the categories. Okay. So,
Manhattan, Bronx, Brooklyn, et cetera that we got from our set. These are the unique values in this
column. Okay. And then the right most column or the right most item in each line is the value.
Okay. Which in this case, because we're performing a count, we created a count,
it's the count. It's how many times each of these items appears in the dataset. Okay. So,
we heard Manhattan was something like 21,000 something something. Brooklyn was 20,000 something
something. So, a little bit less. Let's listen to one more. So, Queens, there's a bit of a drop,
5,660 something. Bronx is another drop. Okay. So, you're hearing the values. It's pretty
intuitive. It tells us how many items in our dataset. 20,000 some odd are Manhattan. 20,000
some odd are Brooklyn. About 5,000 are the Queens and about something like 1,000 are Bronx. And a
very small number presumably are Staten Island. I guess we could look into that. "Staten Island
373." Only 373. Very few Airbnbs on Staten Island. So, if you want to run an Airbnb,
don't open it on Staten Island. Maybe open it on Manhattan. I'm not in that business.
Don't take my advice. And then at the end, it'll say the D type. Remember, series always tells
you the D type. "Name colon neighborhood line group. dtype colon int." D type int. So, it's
integers. The items in this are integers. Okay. They're counts. Now, okay. Let's talk about this
a little bit. First of all, this is already very useful. You pretty much if you've ever been
familiar with a bar chart, some of us are, you know, you have partial vision. Some of us used
to be sighted or more sighted. Maybe you're familiar with visualizations. You know what a
bar chart looks like. I'll just quickly describe it. A bar chart basically it represents data with
how long different lines are. Okay. So, if a line is longer, it has more items in it. Okay.
Basically, what we have here is basically it's a bar chart. Okay. It tells us how many items there
are. And it's almost it's almost like if we were going to take a bar chart created by a sighted
data scientist and represent it in some format that is accessible to the visually impaired,
we might actually pick a format like this and it actually would be pretty good. You might have
your own preferred format, but this one is actually not terrible. Okay. It's like it's
pretty it has a label. It has a number. Each each pair is on a line. It's actually pretty usable.
Okay. So, basically what you have here is a bar chart. Okay. Now, how is it different from
what what could a sighted person do a little more advantageously with a bar chart that you
couldn't do with this? I would my answer to that is, first of all, not that much. But if I had to
say one thing, it would be that the sighted person might get very intuitively to their mind the
difference between the different items. Okay. So, instead of hearing the or seeing the very specific
numbers, they might be like, oh, okay. They might get an insight like, for example, Brooklyn and
Manhattan are really close together. There's a pretty big drop off between those two and Queens
and then another big drop off between Queens and the Bronx. Okay. Now, to be honest, you can get
that from these numbers pretty easily. Okay. But let's say let's make it even more like a bar chart
and actually and also very similar to a pie chart because a pie chart represents data by
it draws a circle. So, if you make a circle with your hand and then different slices on the circle
represent their proportion of each item in the data set. Okay. So, we have about 20 percent of
the items are Brooklyn. So, 20,000 divided by some 40 something thousand would be something
like 40 something percent. Okay. Just to do some back of the napkin. So, we would take out a slice
of the pie that would be 40 something percent. That is basically how a pie chart works. Data
scientists actually hate pie charts and they look down on people who use pie charts, but in my
experience people like many bosses and stuff that I've had like pie charts. So, but, you know, mostly
things that you could do with a pie chart you could do with a bar chart and probably better.
But so, basically what we're doing here is we're thinking of ways to replace both pie and bar
charts. Okay. But what we're going to do now is even more like a pie chart and it also will give
you a little more of that intuition of the differences in the sizes. Okay. So, let's use
the second technique here. So, you already basically have replaced bar charts, I think.
And bar charts are one of the easiest to replace non-visually in my opinion. But let's make it
even better. Let's make our representation or non-visual representation even more like intuitive
to us. Okay. Or even more useful. So, let's do this. First, let's save our counts to a variable.
Okay. And I'm going to tell you how to do that with some hot keys. Okay. If that's okay. So,
if you prefer to type it all out, that's fine. We'll go over that as well. But let's press the
up button. That loads in the last line we ran. So, that will be the one that where we get our
counts. Okay. Now, press control A. And you won't hear anything. Just press control A to go to the
beginning of the line. And then type let's just call it neighborhood counts. Okay. And you can
spell neighbourhood any way you want. I'm going to spell it the British way, I guess, because that's
closer to the answer. "neighbourhood_counts = " And then I'm going to do a space and equals and a space.
So, what I'm doing is I'm saving the last line to a variable. Now, you could also do a really
quick way to do this is another cool way to do this is you could type neighborhood underscore
counts equals underscore. Because underscore actually represents the last line you put in,
which is pretty cool. And that's one of the contributions Nick Hilvora made when he edited
the first in this workshop series is he added a description of that underscore that Python lets
you do, which is a pretty cool trick. So, you could do that. Or you could do what I did, which
is to press up, move to the beginning of the line with control A, and then type it in. Okay.
Because who wants to type everything over and over again? Certainly not programmers. Programmers
are lazy. In left bracket ten right bracket colon. Okay. So, basically, what we did there is
we saved our previous counts to a variable. So, we did neighborhood underscore counts equals
df.neighborhood underscore group.value underscore counts. Okay. And we didn't want to type it all.
If someone wants to get on the mic for the question, that's fine. I'm hearing raised hands.
But maybe actually just type in the chat if you want to raise your hand because it's
in maybe accidental hand raising, which is something I very frequently do in meetings.
Because I often want to look at the participant list and then I raise my hand by accident.
Okay. So, we now have neighborhood underscore counts. So, that's our counts that we were just
looking at. The five items, each with the label of the borough, you know, Manhattan, Bronx,
you know, the neighborhood group, and then the count, the number of times it occurs in the data
set. So, that's what we have saved to the variable. What we're going to do now is
neighborhood underscore counts. Did it do it? Space. In left bracket. Oh, it filled it all in.
Neighborhood underscore counts. So, I used the tab to fill it in. You can type it all out yourself
too if you prefer. It's neighborhood underscore counts. That's the variable we created. And now,
what we're going to do is it's a new technique. We're going to get the proportions of the data.
Okay. And they're going to be kind of like decimal points, kind of like fractions
that are going to represent how much of the data is each item. Okay. So, we're going to do divided
by. So, it's neighborhood underscore counts divided by. Len for length. "L-E-N. Len. Left
paren. D-F. Right paren." Neighborhood underscore counts.
Then, a space. A forward slash. A space. Remember, forward slashes divide. And then,
len. Open paren. D-F. Close paren. And so, basically, the len D-F tells us the total
number of items in the data set. So, that's 40 something thousand. That's all the items,
all the listings in this data set. And then, so, what this is going to do is going to divide each
item in our counts by the total number in the data set. So, Manhattan is 21,000 something.
It's going to then divide that by 49,000 and something. Okay. And what that will give us
is a decimal that will tell us the proportion of the items in the data set that are that. Okay. And
remember, we said it would be something like 40 something. .44 or .43. It'll be something like
that. Okay. And that will tell us that 44, 43 percent or whatever of the items are Manhattan.
But anyway, let's run it and see what it sounds like. Okay.
"Out left bracket 10 right bracket colon. Manhattan 0.44."
So, you hear that. 0.44. Okay. So, that's about 44 percent of the items in this column are
Manhattan. Okay. So, 44 percent of the items of the Airbnb listings in 2019 were in Manhattan.
"3011. Brooklyn 0.4111." Brooklyn 0.41. Now, you can tell even more intuitively,
these are very close to each other. So, 44 percent are Manhattan. Three percent less,
41 percent, something like that, are Brooklyn. "67. Queens 0.115."
Queens 0.11. That's 11 percent. "Bronx 0.022."
Bronx 0.02. Only 2 percent. .313. "Staten Island 0.00s."
Oh, Staten Island is tiny. "0.00762." So, less than 1 percent. .7 percent,
less than a percent of the items are Staten Island. Sorry, Staten Island.
Okay. And then, so, that's pretty cool, right? So, we're actually getting proportions. So,
now, not only replace the bar chart, we kind of replace the pie chart too, you know,
because now the pie chart tells you the proportions of each item,
and what this does, this tells you the proportions. Okay. And let's do
one more useful thing. Okay. And I would say sometimes you might just want to leave it
there at proportions. And this will tell you very, pretty much tell you the same data that
someone using a bar or pie chart would. And honestly, it's very usable. It's good. Like,
it's pretty straightforward to replace a bar or pie chart non-visually. And even there's actually
some slight advantages to this, because sometimes it can be a little hard to read a pie chart if
there's a lot of items or so on. It's not a problem. That's another thing about bar and
pie charts is that usually sighted people can only put so many items in them, usually
a relatively small number, probably not more than like 20, and usually more like 5 or 10.
And a pie chart really does very poorly with anything more than, in most cases,
5, 10 items. That's about as many as you can put in a pie chart, because just reasons, you know,
it just doesn't look good. It looks bad, and it's hard to read if there's too many items.
So, actually, you don't really lose out that much with this, because already sighted people can only
intuitively grasp a relatively small number of items in a bar or pie chart. And so, we're
actually not at that much of a disadvantage with this type of data. Let's do one more thing,
which I think is pretty cool. It might even kind of be in some ways better than a pie chart
or bar chart. Well, certainly a pie chart, but maybe in some ways even a little better than a
bar chart or show something kind of cool. So, let's do this. We'll do our neighborhood counts.
"in left bracket 11 right bracket colon neighborhood line counts."
So, neighborhood underscore counts. Remember, that's our number of counts of each item that
we created. And then let's do dots. And this one always is hard to remember,
but I think I've got a grasp on it. Dot P-C-T.
That's short for percent. I always want to say it should be P-E-R, but whatever. I didn't get
to name these. So, they didn't ask my input when they created all this stuff for some reason.
P-C-T underscore. "P-C-T.
Line." Line. Remember, line is underscore.
"C-H colon A-N-G-E." Okay. That was hard to understand. But basically, it's
neighborhood underscore counts, our variable, dot P-C-T underscore change. That's for percent change.
And then do an open parenthesis and a close parenthesis.
"Left paren. Right paren." And then let's listen to this.
And we will see what we get. I think this one's pretty cool.
"Out left bracket 11 right bracket colon. Out left bracket 11 right bracket colon.
Manhattan N-A-N." Okay. It said Manhattan N-A-N.
I'll explain that in a minute. Basically, no data.
"Brooklyn minus 0.0718." Brooklyn minus 0.07 something something.
Okay. So, that is actually, what that's telling you is, between Manhattan and Brooklyn,
it decreased by 0.07 percent. Seven percent. Okay. Remember that these decimals are another
way to represent percentages. Okay. That says it decreased by seven percent. So,
from Manhattan to Brooklyn decreased by seven percent. Let's hear the next one.
"880. Queens minus 0.7181." Okay. Queens was minus 0.71. So,
Queens is almost a 70 percent drop from the previous. So, this tells you, what this tells
you is the percentage change from the previous item. So, it only changed seven percent between
Manhattan and Brooklyn. So, those are pretty similar. Then, between Brooklyn and Queens,
it dropped 70 percent. So, that's a big drop. So, this is telling you, now you're getting
exactly what a sighted person would get from the bar chart, that intuition about the change. Okay.
And the groupings. Okay. That you can get, oh, that changed a lot. Okay. Let's just hear the
rest. "66. Bronx minus 0.807." Bronx, down 80 percent. Okay. So,
an even bigger difference from between Brooklyn and Queens. Okay. And that's something that's hard
to get just from the numbers. Okay. And what we're getting is actually even a little more precise,
you know, because you're going to get the exact numbers. You get an intuition
with the sighted visualization, but you won't get the exact numbers. So, actually,
I think this is cool. Let's just hear the Staten Island. "7448. Staten Island minus 0.658."
Staten Island down 65 percent. Okay. So, the biggest drop is between Queens and,
between Queens and the Bronx. Okay. But also, another big drop was between Brooklyn and Queens.
Okay. So, this tells us a very, it's a very intuitive way of telling what you would get in
a bar graph, which is how big are the changes. Okay. That's what you would get with a bar graph,
difference in lines. So, this is a very, let's say a very adequate or actually pretty good
replacement for a bar or pie chart. Okay. Which are, I will say, those are probably
the arguably the most used visualizations would be a bar chart. Maybe you could say a line chart.
That would, I would say it's kind of a toss up between bar and line chart are the most commonly
used visualizations. We've kind of already sort of have a pretty adequate replacement
for the bar chart. Okay. We are going to move on to the next section. I'm going to do a time check.
And then if people maybe have a question. "2 colon 00 PM."
We're exactly at the halfway point, which is exactly what I want to do. Love that. All right.
Do people have any questions or do the helpers want to kind of kick up any questions that were
coming up a lot before we move on to the next section? Okay. We already replaced bar and pie
charts. Admittedly, those are kind of the most easy to replace non-visually. Okay. And we're
going to kind of get into some harder to replace them, but we already have a win under our belts.
Okay. It's been remarkably quiet in the chat. So, I think anyone has, I know. So, if anyone has
questions. You know, I would love, I'm going to do a long wait. I would love if someone were to
ask a question at this time. So, you're doing me a favor. So, if you feel so inclined, just go ahead
and turn on your mic and ask a question. Would sorting the data first, the data
sets first give you a better indication of their differences? I would say one thing is that the
counts are automatically sorted. Okay. Usually from greatest to least. Okay. Now, I will say
we will be using sorting more before we're through here and sorting is really useful, but in this
case, you kind of get the sorting free with the, with the counts. So, it's not really, I don't
think necessary. You can sort the categorical data, but really what you're going to get if you sort it
is something like Manhattan, Manhattan, Manhattan, Manhattan, Manhattan, Manhattan, Manhattan, Manhattan, Manhattan, Manhattan.
It'll sort it into alphabetical order and then you'll just get a lot of repeated values. You'll
be like the Bronx, the Bronx, the Bronx, the Bronx, the Bronx. So, I would say sorting the column
by itself doesn't really tell you that much. But sorting the counts is very useful
and you get that kind of for free. So, usually by default, it gives you from the biggest value to
the smallest value. So, they call that descending order. It gives you descending order. Okay. Great
question. Great question. Thank you for asking it. Did Sarah, was it Sarah or someone was starting
to say something there or? The exact same question. So, nothing further here. Well, thank you. Is that
Prima? Thank you. I really appreciate the question. Okay. So, if people have other questions,
please place them in the chat. You know, I love those long, awkward pauses when I do remote
teaching. So, I really, I live for that. So, I will maybe ask more questions later in the workshop.
Okay. Okay. So, we started out with counting, which is one of the favorite tools in my toolbox.
And I also showed you a couple of other cool tricks, which are getting the
proportions, those little percentages, and the percent drop or the percent change, which is
quite useful. I will say, I'm just going to gesture toward the written tutorial here because we don't
have time to cover all of this. But I anticipated a couple of frequently asked questions at this
point. And those are, one, say I want to work with sighted colleagues and I want to make an actual
visual bar chart or an actual visual pie chart. How do I do that? Okay. Now, why, you know, why,
no, I mean, I'm just thinking, no. I mean, we live in a sighted world. Last time I checked.
And, you know, we work with sighted colleagues and sometimes it's useful to actually create the
visualization. So, I did give instructions for creating a visualization of our counts data
in the written tutorial. So, you can check that out. Another thing is, question is, can you get
actual percentages? And I give instructions for that too. If you don't want the decimals, there's
a way to actually get nice looking percentages. It's a little bit involved, not terribly involved,
so we're not going to cover it here. But that information is there in the thing.
I had another one, FAQ, but let me check.
Max, Patrick, email, I search, creating bar chart, creating percentages.
Oh, you're right. And also, one useful thing is that I won't cover here, but it's in the tutorial
and the written tutorial is you can, a way to skip the step where you divide by len data frame,
there's actually a way to skip that step and to do, just change the value counts a little,
function a little bit. You add normalize equals true into the parentheses. And that will let you
skip that divide step and it will give you those percentage, those proportions directly.
So, check those out in the written tutorial. I'm not going to cover them here,
but those are my guesses and things you would want to do with this categorical data at this point.
And you can always send me email, so I'll add to that.
All right. So, I'm going to now show you maybe, in my personal opinion, the next kind of really
cool tool in the toolbox, in this data exploration and representation toolbox, which is indexing.
Another way I would say this is using binary data.
Okay. So, what we're going to do here is that we're going to look at our full data set. So,
we have a full data set, which of 47,000 rows. Now, let's say we want to answer a question,
not about the full data set, but only about a specific portion of the data. So, for example,
the example we're going to use is, say we only want to analyze rooms that are really expensive,
so over $1,000. Okay. Maybe you're deciding if you want to eat the rich or something like that.
So, we'll look only at the expensive rooms. So, what we can do, and this is what I'm going to
teach you, is you can make a new data set that is a subset of the old data set for which specific
thing is true, for which a specific statement that we make is true. For example, you can say,
okay, I want to make a subset of the data where the price of each listing is over $1,000. Okay.
$1,001 or more. So, we're going to do that now. So, let's clear our screen, clear our minds,
control L. So, we've already entered 11 lines of code. We're on line 12 here.
And now let's do our indexing. And I would say this is probably maybe one of my top most used
techniques when I'm doing this, is creating these subsets of the data. Okay. So, this is a pretty
cool one, and very flexible. So, let's do, we're going to do this in two steps. First, we create
a series that is, it's going to be a series containing binary data. Another way of saying
that is it's a Boolean series. Okay. Boolean, say kind of the same as binary. True or false.
We're going to create a series that only has true or false values. And those are going to
correspond to what we ask for. So, we're going to say, create a Boolean series where it'll be true
if the price for the row is over $1,000. And false if the price of the row is less than $1,000.
So, that's our first step. And we're going to go and save it to a variable write off. Okay. So,
we'll do, I'll call it expensive underscore bools for Booleans. Okay.
I did an underscore there. Expensive underscore bools. That's going
to be our variable name for our Boolean series of true false values. Bools equals space. And
let's do df.price. And this is where we specify what we want. So, I'm going to say greater than
$1,000. I did three zeros there. Zero, zero, zero. So, what we put in is expensive underscore
bools equals, for assigning the variable, df.price greater than sign $1,000. You can leave off the
spaces if you're not feeling it. I put the spaces in because that's what they tell you to do in
programmer school, which I never went to. Okay. So, I'm going to press enter.
"In left bracket, 13 right bracket colon." Okay. So, remember when you assign variables,
you don't get any output. You just hear the input line again. Remember, the way of thinking about
that is that the data that we would normally hear with the screen reader or see, perceive,
depending on how you're using it, goes into the variable. It doesn't come out into our
command line environment. That's one way to think about it. So, let's check out that series.
So, let's do what did I call it? Expensive underscore bools. Expensive underscore bools
dots head. And I just use head a lot because I don't want to hear too much output. Okay. Remember,
head will get you the first five items. So, head is very useful to just get a little bit of the
data so you can kind of get a sense of it. "Out left bracket, 13 right bracket colon.
Zero false. One false. Two false. Three false. Four false." So, that was pretty boring because
they're all false. But most of the listings are not over $1,000. Right? I mean, you know,
thank the universe, you know, because, like, you know, I can't afford $1,000 a year. But we could
do DF. So, now we have our Booleans. But now how do we tell if there's actually any trues in it?
So, the two things you can do with the Boolean series by itself that I recommend are here's a
little trick that I really like. Okay. I haven't really seen this anywhere. I don't know where I
picked this up. Maybe from a colleague or something. But you could do price or expensive underscore
bools. Expensive underscore bools dot mean. So, get the mean. Get the mean of the Boolean series.
And what this will do is it will tell you the percentage there's percent that are true.
So, 0.004. About half a percent of them are over $1,000. That's still a good number.
Okay. So, we did expensive underscore bools dot mean. And that told us the percentage
in a decimal of the items that are true. Okay. That's a cool trick. So, we do know there are
some in there. And now let's find out exactly how many. So, we can also use remember we said
we used our value we had our method value counts. You can also count the true and false values. So,
let's do "EXP line bools." So, expensive underscore bools dot remember using the tab.
That's why it's saying it that way. Dot. Expensive underscore bools dot
value underscore counts. "V-A-L-U-E. Value. Line. C-O-U-N-T-S. S. Left paren. Right paren."
Okay. So, we're using the value counts method on our Boolean series to count the values.
"Out left bracket 15 right bracket colon. False 48,656." That's 48,000 something. That's a lot.
That's not all of them. "True 239." So, we have 239 values in here that are true. That means
there's 239 rows in this data set where the price of the room is greater than $1,000. Okay. Now,
let's do the real useful thing, which is we're going to create a new data frame. We're going
to call it expensive underscore df. Expensive data frame. It's a new data frame. We can't just
keep using df1, df2, df3. Don't do that. Don't do that to yourself. Once you go beyond using one
data frame, you want to give it a name that's kind of somewhat useful. We'll call it expensive
underscore df. Then, what we'll do, what we'll put in it is only the items where the room is,
where only the rows where the price is over $1,000. Okay. So, we do expensive.
Expensive underscore df space equals space.
Let's do df, our original data frame. Now, we open square bracket.
"Df left bracket." Now, we pass in, we give it our bool, boolean series.
"Expensive line bools. Right bracket." So, we did expensive underscore df equals df open square
bracket. Expensive underscore bools, our boolean series variable. Right square bracket. You can
kind of think of this as a fancy, remember we used slicing syntax on our lists in the first
tutorial to pull out specific items from a list. It's very similar to that syntax, but we're
using our boolean series to tell pandas which rows to pull out. We'll go over that again.
I'm going to press enter. We saved a variable so we didn't get anything as output. Let's take a
look at that variable. Let's do length on this variable. Expensive
line. Df. So, I'm putting expensive underscore df inside len.
239 items. We have a new data frame and it's 239 items. You can look at it manually, but now what
we have in here is all the items are expensive. Every item in this new data set had a room that
was over a thousand dollars. So, we can do things like, what would be the mean of the prices
of these rooms? It would probably be more, but let's just try it out. So, we could do...
I did too much. So, I did expensive underscore df dot price dot mean.
So, it's expensive underscore df dot price dot mean, open parenthesis,
close parenthesis. So, if you only take the items that are over a thousand dollars,
then the mean is really large. It's like $2,500, which doesn't tell us that much.
But you could do other things. Maybe I want to know what neighbourhoods are the most expensive.
So, we could do expensive underscore df dot neighborhood.
So, I did expensive underscore df, I filled it in, dot neighbourhood.
You did the math to fill things in. So, it's expensive underscore df dot neighborhood.
And then now let's do value counts, just to count them up. Because we don't want to go through all
that data. We just want to count it up. So, it's expensive underscore df dot
neighborhood. That tells us which neighborhood that each item is in.
Dot value underscore counts, open parenthesis, close parenthesis. So, what essentially this is
telling us is it's going to tell us which neighborhoods have Airbnbs that are listings
that are over a thousand dollars. Okay? "File quote lesson Python."
Expensive df dot neighborhood. Dot price. Oh, no. Dot value counts.
"Right. Riverdale one. East Flatbush one. Sheep's Head Bay one. Flushing one.
So, this is kind of boring. Fort Green one. So, we actually want to sort this.
Cypress Hills one." Oh, you know what it's doing. It's printing. It's because
I'm so zoomed in that it prints out stuff later. So, we actually want to use head as well. So,
I'm going to press up, colon, and do dot head. So, actually we're chaining together a lot of
things. And we'll go over it one more time. Dot h e a d. Let's hear the output and I'll
go over it one more time. "Head. Right paren." So, this prints the first five items. We're
hearing items from the middle, which is not that good. "Colon. Value line counts left paren.
Right paren. Dot head left paren. Right paren. Out left bracket 21 right bracket colon.
Upper west side 23." Upper west side 23. So, there are 23 items or there are 23 listings
that are more expensive than a thousand dollars in the upper west side. That's a pretty seems
like a pretty pricey neighborhood. "Midtown 19. Midtown 19. West village 14." West village 14.
Okay. Okay. So, those are very expensive neighborhoods. Any other ones?
"Upper east side 13. Chelsea 13." Those all sound like very expensive neighborhoods in New York.
Okay. So, what we did there was we have our new data frame that we created that has rooms that
are only over the $1,000 price point, but it has all the other data in it, right? So, it's like a
new data frame, but the only rows in it are items that are over the prices over a certain amount,
but it still has all the other data in it. So, we can pull out the names. We can pull out other
items. We can pull out the neighborhood. We can pull out the neighborhood group from this new data
frame. So, we just then do what we did before, which is pull out some categorical data. So,
we pull out the neighborhood, then we count it, value counts, and then I did head because we had,
you know, it was printing out too many things. Okay. So, we just pulled out the first five items,
and it told us the five most expensive neighborhoods. Okay. So, that was expensive underscore df dot
neighborhood dot value counts, count the data, dot head. Okay. And that tells us the five
neighborhoods that have the most rooms over $1,000 in the data. You can see this is a very
flexible method. Anything you can think about that you could pose as a question about something
greater than, something less than, you can make a new data set out of it. Okay. And then explore
only that data set. Okay. So, it's a very flexible tool in your toolbox. You make subsets of your
data. Okay. I'm going to do a quick little time check because there's one thing, something I would
like to show you with this, but we may not have time for it. I might just have to gesture to it.
222. We can very quickly go through it. I think we might have time for it. So,
I live in New York, as we talked about, maybe briefly before, and I grew up in Queens,
here in Queens, and I live here in Queens. Okay. And I say, actually, I didn't get very far in life
because I grew up in Queens and I live in Queens. But, literally, in terms of distance.
But, you know, so, but I like my neighborhood. It's called Woodside. It's here in Queens.
And I want to know some stuff about, let's say I want to know the average price of an Airbnb in
Woodside. Okay. In my neighborhood. So, we'll use something a little bit, we're using the exact same
technique. We're going to create a Boolean series where if it's, if the neighborhood is Woodside,
it will be true. The row will be true. And if it's not Woodside, it'll be false.
Then we're going to use that Boolean series to create a new data frame. And the data frame will
only be items in Woodside. Okay. Only be items in the neighborhood Woodside. Okay. So, instead of
working with numeric data, instead of using numeric data to create a subset, you know,
greater than, less than, whatever, like we did with the expensive data frame, we're going to
actually use categorical data. So, we'll say the data has to be in the, in terms of the neighborhood
as a categorical data. And then we'll say, okay, the data has to be in Woodside. Okay. So, it has
to be in a specific category in the neighborhood column, which is Woodside. Okay. And you can tell
I'm in Queens because someone's driving by. Hopefully you guys don't hear that. Maybe you do.
Let's do control L to clear our screen. Okay. I guess I'll do the math.
Focus is the worst thing in computers. Okay. So, I'm doing DF, or I'm sorry, I'm doing,
I'm going to call this Woodside underscore bools. I'm using the same technique. I create a Boolean
series. I'm going to call it Woodside bools. I'm going to go a little faster with this because
we already did this. I'm just showing you a slightly different technique. Woodside bools
equals DF.neighbourhood equals equals is equal to,
and then I'll use the string, quote, "W-O-O-D-S-I-D-E." I have to use a capital letter for
Woodside because that's how the data is constituted, I think, quote. Okay. So,
we have our Boolean series. I'm just going to double check it by doing the mean to make sure
there's something in it so that it works. So, Woodside underscore bools dot mean.
So, there is something in it. It wasn't just a zero. So, it's not a lot of the data. It's a
very small percentage of the data, but it's there. And then we're immediately going to
create a new data frame from our Booleans. So, I'll say Woodside underscore DF.
DF. Say a new data frame. If you want, you can just sit back and relax during this part. You
kind of already did this. And then equals, Woodside DF equals, and then DF,
open parenthesis, no, open square bracket, DF, and we pass in our Woodside DF. This is exactly
what we did before. You know, to create the new series, we do DF, open the square bracket,
and we pass in our Booleans. Okay? And then it pulls out all the rows for which a value is true.
Okay? For which, in this case, for which the neighborhood was equal to Woodside.
So, Woodside underscore bools. So, here I have Woodside underscore DF equals DF
square bracket, Woodside underscore bools. Okay? And what this does is it creates a new data set.
It's a subset of our data where only the neighborhoods that are Woodside, or only the
items in the neighborhood column that are equal to Woodside are put into the new data frame. Okay?
That's a terrible way of saying it. It's a subset of the data for which the neighborhood is Woodside.
So, now I have a new variable, Woodside DF. Okay? And let's just quickly, we'll get the mean
price. So, we'll do Woodside underscore DF. This is just like we did in the last workshop.
So, remember, if you kind of remember from the last time, I think if we do it again,
"df dot price dot mean()." The mean price in the full data set was. So, in Woodside, it's only like
$80 or $85, which is pretty cheap. It's relatively cheap. You know, so, you know,
come to Woodside, I guess. And then one other fun thing is you could pull out the, don't follow
along with this, but I'm going to show you the flexibility of this approach. When I was working
with the data, just preparing this tutorial, I noticed that there's some very funny room names
for the cheap rooms in Woodside. So, just for a laugh, I'll pull that out.
So, you don't have to follow along with this if you don't want to. But what I'll do is I'll do
Woodside DF. In fact, I'm going to sort my new data frame on price. So, I'm going to get cheap
and expensive. So, I'm going to do Woodside DF dot sort values. Okay. And then I'm going to do
dot name. So, it's Woodside DF, our variable, dot sort underscore values. And then into that,
I give price. So, I basically say sort the data frame on price from, I think it's from cheapest
to most expensive. And then I did another dot, and I'm going to say I want the names. Okay.
The name column. And then I'll just get the top most, the head.
Don't worry too much about following along with this. I'm just doing this to entertain you because
they're funny. But that is useful. So, basically, I take my new data frame, which is only Woodside,
my neighborhood. Then I'm going to sort by price. Okay. Sort the whole data frame by price.
Then I'm going to pull out the top, the five first results, which I believe will be the
cheapest. ...
...
... So, there's a funny one. It has some emoji, which isn't printing out correctly, I guess.
they're not hearing it and it says wow exclamation mark cozy exclamation mark so
that's one of the rooms and then another one of the rooms was it only has the
word you in it like you like letter u can get to Manhattan and that's
literally the cheapest room and with that it was $28 when I looked at it and
it was you capital letter can get to Manhattan and you know or you can use a
certain number of train lines I thought that was kind of funny and then another
room says just wow cozy and that costs like 30 bucks or something like that so
the ones that are the cheapest are kind of the most eccentric and I thought that
was kind of a little bit entertaining okay and that kind of also shows the
flexibility of combining some of these things so what we did was we created a
data set that was only Woodside then we sorted on the price to get the cheapest
to most expensive and then we pulled out the names from the data frame and then
we got the first five which is the cheap five cheapest rooms you can also do
doctail and get the last which would be room five rooms would be the most
expensive okay so once you're gonna get a little bit used to these methods you
can start mixing and matching and it becomes a very flexible okay so we now
have about half an hour I believe okay "2 colon 32 p.m." exactly okay so what I
want to introduce is one more concept which is correlation and then I may
just kind of gesture toward how you would work with create line charts as
well but we may not have time to fully engage with that but there are materials
in the written curriculum as well so we might just have to do that a little bit
of an abbreviated version of the of creating a line chart it's a lot to take
in anyway so I think we're kind of covering a lot but okay so we we've
learned indexing that's creating subsets of the data I would say if you're gonna
learn two techniques for data for exploring data for representing data
then the two you want to learn or make a count that's incredibly useful to just
do a count of a categorical of categorical data will tell you a lot the
second thing I'll tell you is do that learn to do that indexing learn to use
those boolean series which will allow you to create subsets of the data they
also allow you to create answer specific questions so you can learn oh what our
question well here was what was the proportion of the rooms that are over a
thousand dollars and the answer was it's about half a percent you know we got the
boolean series and then we do dot mean and it tells us okay how many of these
items is is this proposition true and that's a very flexible we can also say
well how many rooms are less than $30 how many rooms are under $100 or we
could say things like how many rooms have the minimum night greater than that
you have to stay greater than a week 30 than seven days anything that's a number
you can use that greater or less than and answer questions about it say what's
the proportion of rooms for which you know the other let me think of other
numeric items that the number of rooms for which the minimum nights you can
stay is less than a week you know and it will tell you the percentage that's very
useful and once you create a subset of the data a new data frame based on that
you can answer other questions like you can say you know of the rooms where you
can only stay for one night what's the mean price and is it greater than or less
than the full data set or greater and less than rooms where you can stay for
more than one these are you can answer very specific questions with these two
tools only two tools in counting and indexing these are I would say if you're
gonna learn two things you can get very far with just those two tools okay very
flexible and you also see it combined well with sorting getting the head and
the head and the tail those are the old and we're really only using a small
number of methods here to do some very flexible stuff okay you learn those
three or four methods count values head tail sort sort values and and then to
get that indexing method that I showed you where you use greater than less
than or equal equal to and then that's a small number of methods that lets you do
a lot okay so let's do something a little more data sciencey and this is a
you know we're kind of coming up on the end this probably the last thing we
fully we managed to fully engage with we'll kind of just put a line charts at
the end and we do have some written curriculum but I think we may not be
able to get to it so let's clear our clear our minds clear our screens
control L "in left bracket 30 right bracket colon" and we're up to 30
we've entered 30 lines of Python we're up to line 30 here and what we're
gonna talk about now is we're gonna try to replace another common visualization
type a little bit but we it's not it's gonna be a little more challenging it's
not quite as easy to replace but it is possible to get kind of a certain
percentage of the way there and that is that what we're gonna learn to do is
we're going to learn to correlate to if we have two variables so we have two
columns each with numeric data then we what we're gonna learn to do is find a
correlation between those two columns okay and what a correlation is it's a
it shows a relationship between the two columns between the two variables okay
and what do we mean by that and you know the fancy way to say this it's a linear
relationship linear relationship basically means you could draw a line to
show the relationship or you can get a number which is would be the slope of a
line and then that's fancy instead the simple way to say it is it say you have
two variables let's just imagine we have ice cream sales and temperature those
are two variables we have the temperature in degrees Fahrenheit let's
say I mean I'm in New York and a Fahrenheit is ridiculous but let's say
we have temperature in degrees Fahrenheit and we have ice cream sales
okay sales of ice cream per day and then at certain temperatures how much do the
sales but how does that affect sales so we could use a correlation and that
would suggest what is the relationship between the two variables the linear
relationship so if one goes up does another one go up down or doesn't
nothing happens that's basically what we're talking about if one variable goes
up then what happens to the other one does it go up down or does nothing
happen that's the direction of the relationship and then also how strong is
the relationship so does it go up a lot does it go up a little does it go down a
lot does it go down a little okay and that is basically what a correlation
okay and what we're specifically going to talk about is called a Pearson
correlation but it's the most common correlation type so let's not worry too
much about that and the way we represent this is in a number from
negative 1 to 1 okay so if the two numbers are if one number goes up the
other one goes up we call that a positive correlation and we that number
is and then the number that correlation number will be positive if if say one
number goes up the other one goes down then the number or correlation number
will be negative okay and then if there's no relationship then the number
will be pretty close to 0 okay the correlation number will be pretty close
to 0 okay so negative 1 means you have a negative relationship that the
variables go in different directions 0 means the variables aren't related
really they there's no association or close to 0 and then positive number
means that there's some kind of relationship if one goes up the other
one goes up and the closer the number is to 1 the stronger the relationship so if
if a number goes up by 1 then the other one goes up by a fixed amount say if one
number goes up by 1 then the other one goes up by 5 and that will be a
relationship of 1 okay it's perfectly correlated okay and so 1 is perfectly
correlated they're basically the same and 1 goes up the other one goes up
straightforwardly linearly and then if negative 1 one goes up the other one will
go down straightforwardly linearly okay so those are that's what we're gonna
calculate and luckily pandas makes this in terms of programming this is pretty
easy the hard part is just understanding what you're doing what the two what the
number means okay basically just what I just explained to you that there's a
direction positive negative or no relationship and that there's a strength
whether it's strongly correlated or weakly correlated and then that number
one number between negative 1 and 1 tells you the story of the correlation
okay so let's correlate two variables in our data set or two columns and two
numeric we only have a few numeric columns let's do number of reviews and
reviews per month those are two columns we have in this data set so reviews per
month is how many reviews come in per month for that listing and the number of
reviews is the total number of reviews okay so let's do DF dot DF number
I'm gonna let it fill it in so number "line of line reviews" so it's DF dot
number underscore of underscore reviews that's our column let's do dot C O R R
dot correlation dot C O R R dot corr dot correlation that's short for
correlation okay so it's DF dot number underscore of underscore reviews dot C
O R R for correlation open parenthesis "C O R R left paren" dot and then now we
pass in the other column so it's gonna be DF dot DF dot and then we're gonna do
reviews per month "R E V I E W review per line month" okay and then close the
parenthesis "right paren" so it's DF dot core sorry DF dot number underscore of
underscore reviews dot core open parenthesis DF dot reviews underscore
per underscore month right paren okay and those this will correlate the two
columns and just give us one number between negative one and one "out left
bracket 30 right bracket colon 0.5 4 9 8 6 7 5" 0.5 4 so that basically means
first of all it's a positive number that means that the when one number goes
up the other one goes up by some amount okay so that means there is a
relationship between these two variables which kind of makes sense that the the
number of reviews you have total should in some way correlate with the amount of
reviews reviews you get per month because it's kind of obvious if you have
a lot of if a if a listing has a lot of reviews they must have some amount you
know of reviews per month it's probably going to be high and in this case we are
a number it's positive so they have a positive relationship that one goes up
the other one goes up by some amount and 0.54 that's a pretty strong relationship
i would call that moderate to strong if it's somewhere around that middle mark
like 0.4 0.5 you'd maybe call that moderately correlated and then if it's
above 0.5 which is a little bit in this case you start calling it a strong
correlation these are all kind of subjective but but you know it's fairly
well agreed upon and then if it was say 0.3 0.2 0.1 that would be weakly
correlated not very strongly correlated okay so in this case it is correlated
um you know you might ask why aren't these numbers perfectly correlated if
you know since you know the number of reviews you have total does depend on
the number of reviews they have per month why are they perfectly correlated
um the answer to that is that there's another variable at play here um uh many
possibly others but the one i can think of is how long has the listing been
okay and if the listing's been around a long time versus a short time then then
that will affect how much the number of reviews has how much effect the number
of reviews has on the um per month has on the total number of reviews okay um
so uh so it's not the only variable that determines the other okay they're not
perfectly correlated they're only somewhat correlated which makes a kind
of intuitive sense i think that these would be correlated in some way
um so we could do this we could try this on one other i'll just run quickly on
one other variable let's try to see if the number of reviews and the price are
correlated and you know i could see there being a maybe i could make a
hypothesis i could say maybe rooms that are reviewed a lot have a higher price
because who knows you know maybe they're popular um that's a guess but let's see
if it's true or not we'll do df dot price dot c o r r for correlation
...open parenthesis
and then let's pass the other one and we'll do df dot number of reviews
"line of line colon reviews right paren" so we have df dot price dot c o r r
for correlation left parenthesis df dot number underscore of underscore reviews right paren
okay let's see if these are correlated remember my hypothesis is maybe they are
kind of correlated because i don't know maybe that if you have a lot of reviews
you're popular maybe you can charge "out left bracket 31 right bracket colon minus 0.047"
okay so i would say it's negative 0.047 okay so i would say that no my hypothesis is totally wrong
and in this case it's first of all it's a negative number so it's if anything it's negatively
correlated if one goes up the other one goes down but really what this is is uncorrelated
it's very close to zero it's a very small number and so basically what these are is they're
unrelated that the number of reviews doesn't seem to really affect the price in terms of this data
set okay which is also another pretty interesting you know factoid or whatever okay so that's
correlation it's um it's uh we're very commonly used like uh you know statistical method in data
science um in general when people want to show correlations you'll often use uh they'll often
use a visualization called a scatterplot and the scatterplot basically puts dots on a say you have
a flat plane so you just imagine just the flat table and then imagine you take a pencil and
put a whole bunch of dots on the table on the on the that surface and that's kind of the
scatterplot and what a scatterplot lets you do if you're sighted or you can otherwise access it
is that it shows you patterns like clusters and that will usually often also show you whether
data is negatively correlated just exactly what we have here negatively correlated uncorrelated or
positively correlated it may also show you useful things like how many outliers you have like if
there's certain values that are way outside of the norm that we can't we haven't done we we
haven't learned a way to do that with this non-visually okay so that's something a scatterplot
can tell you that this won't but what this will tell you is the relationship in the data and that's
about half of what a scatterplot or maybe more than half half to 70 percent of what a scatterplot
will tell you is that relationship between the two very okay so we can get that from just making
this correlation what it doesn't tell you is stuff about outliers or certain other kinds of more
subtle relationships and clustering you can get those it's just quite complicated it's out of
scope in this tutorial so you would want to look at things like gaussian functions which would allow
you to it's a little bit mathematical but basically you can do you can look for outliers
you could also just kind of look for really big numbers in the data set through methods that we've
learned already which might tell you a certain amount of that okay but if you want to get fancy
and kind of more formally look for outliers you can use gaussian functions okay which is
i would say harder than what we've been doing here so but just know that it exists
for non-div for our non-visual use okay um so i would say you know bar pie charts were a strong
win with the non-visual approaches you can really strong pretty pretty well well replace a bar pie
chart with non-visual methods um scatterplots definitely more difficult unfortunately scatterplots
are also maybe you know Chancey is here she does a lot of tactile graphics scatterplots are also
pretty hard are harder to get through tactician through feel on a tactile graphic line charts and
stuff definitely work better i think and other kinds of visualizations um but uh you know
scatterplots are a little tricky um you might get some use out of it but um okay so we only have a
few more minutes i wanted to do time "2:49 p.m" we have 10 minutes i'm just gonna quickly gesture
toward how you could do time because we have a very limited amount of time and i want to kind of
give you a little bit of the philosophy of all of this interaction at the end um but um essentially
what you want to do in time is um and i will point you toward the written tutorial um but
essentially what you want to do is um that there's special a special function in pandas
that will allow you to take a time column and convert it into a form that you can use
so it looks something like this pandas as pandas dot dot um pandas dot two underscore date time
date time and then you can pass in a column and it will make it into a special format that will
allow you to do time-based things with it um so it would be df the only time column on this
data set is called last review when the last review came in "last_review" so here i did
uh i did df uh our pandas dot two date time two underscore date time open paren
and then i passed in our df dot last review column that was a it's time data um i'm going
quickly with this i'm just going to give you a general idea of how to do it i wouldn't follow
along now if you're if you're doing this live um and i'm going to save it to a variable i'm
going to call the variable um i'll just call it date datess so i have now
a column that's date--a lot of the a lot of them are empty but um uh some of them are
not hopefully and then you can then use that you can do year uh date dot dt dot year
"out left bracket 34 right bracket colon" and then we can count that in left bracket 30 dot value
"2018.06" so now we have to we're hearing the output there let's run it again because
"out left bracket 36 right bracket colon 2017.03 205 2016.0 2700" and essentially this is telling
us how many items from each year um there are in the data set okay so this is kind of fairly
useful time data and then you could um just sort this and then get an idea okay over time
how how does something change okay this would be more useful if it was when the the listing was
first posted or something like that but if you look at this and i i encourage you to take a look
at the written curriculum that goes into more detail on how to perform this operation but
basically what we're doing here is we're turning a column into a specific format that allows us to
manipulate it in terms of time then we're using that to pull out the year then we're using our
familiar value counts to count how many of each year we have and then if you sort it you'll get
a sense of how things change over time and you can actually even add in that percent change um
function uh or method that we used at the beginning and you'll get something very similar
to a line chart where you get the idea of the percent went up the percent went you know the
percent is going up by a certain amount per year or down a certain amount per year which is exactly
what a line chart tells you um unfortunately i don't want to keep you guys longer than you
you're you know are two hours um so if you're interested in how to replace these line charts i
i recommend looking at the written curriculum uh and feel free to send me questions and stuff as well
okay um i will say uh yeah so and that percent change it really does actually give you a fairly
once you this is harder to make but once you have it if you if you you know get an account
of all the different years um then uh then get the percent change you do get a sense of
exactly what you would get with a line chart which is how much does something how steep is
the change how radical is the change between these different time periods the different years
okay um this making these uh kinds of visualizations of over of time can get pretty
uh tricky non-visually but once you make them you can get a pretty strong idea of the change
in the data set okay um they're not as easy to make as the bar charts in terms of programming
but but you do get a very strong sense just like you would with a line chart of the change over
time um it's maybe not quite as direct or intuitive as what um sarah will show you uh in the coming
two weeks with sonifications where you'll hear the change with the pitch but it is pretty okay
it's pretty it's it's not terrible um it's it's easier to replace than the scatterplot
um okay i'm clearing my screen in left we're pretty much at the end of the um of this workshop
um i'll just want to give you a little bit of uh i want to point out some of the the things to
keep in mind um and give you kind of the usual uh pep talk for that for the non-visual data analysis
first of all is that um uh is that what was the the basic methodology we have here which is to
create an interactive to to create interactive um objects that we can then manipulate so uh you
know data frames series and then to ask for very specific things from them and to get very specific
results back it really when you get fairly adept at it when you get even a little adept at it or
when you get fairly adept at it it really feels like having a conversation with your data it
really feels like talking to your data and you can get a very strong idea of the relationships
the contents uh and so on uh of the data sets through this method um and i would say possibly
even if you get quite good at it it's it you really honestly you won't be missing the visualizations
that much you can really get a very strong sense of the data set through these methods
um now are there downsides uh obviously uh i would say in some cases um the the main downside
the main downsides i would say are if you're working with sighted colleagues um and you want
to share uh a representation of the data i will say sighted people love charts you know um and you
know they're they're often persuaded by them and so on so you if you want to work with sighted people
it's helpful to actually make the the visualization if you can um you know it's obviously
it's hard as an as a as a blind visual impaired person to make a visualization but if you're
working with sighted people there's advantages um the other thing so if you're working with people
that's one um important important reason that this this is not a great approach um specifically
um the other one is uh uh the other disadvantage is that you do need to kind of know what you're
doing um so i'm not gonna i'm trying to say this in a more diplomatic way but like
um i would say often visualizations can be a little bit of a kludge um that people kind of
just do them out of habit and then they look at the data and gives them some kind of general idea
of relationships and so on um uh and you know a good visualization is a good visualization
but often i feel like sometimes visualizations are a slight crutch for data scientists and
unfortunately if you want to do things this way that's not a visual way you will kind of maybe
need to know what you're doing in terms of the types of data you're working with how you're
transforming them and how you're representing them in a more specific way you know doing things like
the way we divided the um the data to get the proportions or to get the percent change they do
kind of or to get other useful things like the correlation um there's things we didn't even get
into but um there's things like variance standard deviation and so on you really do need to know
what those mean in order to make them useful to you um so you may need to get uh you know get a
think about the math a little bit more or to think about the types of data that you have a little bit
more um so unfortunately there's no real shortcuts for the non-visual stuff but if you
when you get there it's a very powerful way to interact with data um and then i guess i'll leave
you on um uh i mean just i wanted one say one more i'm looking at my notes you might you might
need to actually know what you're doing you might need to actually know what you're doing
you might you might need to act you might disadvantage not as immediate not as good
for collaboration with sided or disadvantage not as immediate so i was good at collaboration
some types of insights are harder to access or take longer we're going to make a we're going to
make tonification for the next two workshops and those are much more direct um ways to experience
the data okay um and they're much more shareable in a lot of ways okay um and i do think sonifications
are going to be et cetera we'll talk more about them but they're going to be much bigger than
they are in the coming years um and i do feel like they will help us out in these areas for when we're
trying to create stuff to share with side of colleagues because what i've found is that while
side of colleagues they maybe won't look want to look at these kind of like you know non-visual
representations we have quite as much they really do often do like a sonification so some hope so
i have high hopes for sonifications in the coming years i think it's good they're going to be bigger
we're going to do more of them in the next two workshops as sarah takes over and i'll just say
i've really enjoyed teaching these three workshops um it's kind of a privilege to teach
workshops specifically for my fellow blind di people um i don't get that opportunity that often
usually i teach sighted um people um and and it's quite nice to to teach you all how i
actually do data science which is this in this non-visual mode so um so i will will end there
except for questions and answers so thank you thank you all okay i'll leave the i will leave
the recording on for questions and answers uh and then we will also then turn off the recording
um if people you know if people want to ask off mic okay so we'll leave the recording on for now
um and uh if anyone wants to get on the mic and ask a question this is the time
or if the helpers have questions from the chat the only thing we've seen recurrent in the chat
is a couple people having trouble um loading the data frame um from the url
yeah um and we think upon upon some examination we believe that for these people it's probably
either a problem with their network so it took too long to load because their internet's too slow
and ipython timed out or maybe they have some sort of uh like proxy enabled so we encourage
people to go look in their network settings um and make sure you don't have some like barrier
like firewall i suppose between you and whatever you're downloading so um but if you're still
having that issue we do encourage you to maybe come to office hours because this is not exactly
an issue with the code so much as something something funky going on with your computer
and one other additional thing i can so in the tutorial when i update the tutorial i will add
a link to download the csv and then you can try to place the csv in your home folder and import
it that way the url is definitely a lot easier if you can access it okay um i would say you know
the most likely if you're having a problem with the url 90 it's a problem with the the way you
type the link um or in the copying and pasting uh i would say that's the most likely reason or
the most common reason when i and and this is also when i do it with excited workshop participants
it's that's almost always the reason and then the next reason is like sarah said network issues um
now if you're on the zoom meeting chances are you know you're on the internet so hopefully you know
you check that box i will say that in the last year or so github is getting more fussy with
what ips they accept requests from and so on but they should accept for most um i am often on my
um you know very suspicious vpn server and you know whatever it is rotterdam or something like
that that a lot of websites don't like and cloudflare doesn't like and github still lets me
look at these pages and import the data sets and all this kind of stuff so i would say triple
quadruple and quintuple check your url remember it's http colon slash slash um bit.ly forward
slash nyc bnb and i would also try both http and https just try both of those okay i did http
because i think it's the most likely to work but you could also try https
um as a possibility okay those are the things i've tried
um okay i would hope someone asked a question just make me feel let me feel happy
i love to hear your you know i love teaching a person and hearing people's voices and
hearing the noise and questions and everything and that's you asking questions on the mic is
the closest thing hey patrick it's marco hey marco how's it going hey it's going all right
thanks again for this whole series here um i put i put this question in the chat but i just
wanted to see if you your take on it uh so you know we talked a lot about the data sets that
we have and um you know kind of understanding it from a text point of view within the terminal
but i could take this airbnb data set you know grab a subset of something that i want to build
as a bar chart or a line chart and then as long as i gain access to those values within a normal
python data type like a dictionary um i could then funnel those values into like something
that's going to write svg code for me to so i can actually create a tactile graphic out of that um
so i was wondering like an elegant way of taking like a subset like if we grab price and the
name i guess of the of you know the listing how to take that and create a dictionary straight from
a subset and i think you know helper posted i guess um pandas has its own two underscore dicts
uh or there's like there's a method that does that for you which is pretty awesome i was just
wondering your thoughts on that well would it help to create an svg file directly like
like an svg file of what side of people would would have would that be helpful or
i mean i would be able to do that because that's that's right the whole thing that i've been working
on is building svgs so you can do that already or do you want me to show you how to do that
no i know how to do that yeah i teach people how to do that but it's just you want what you
want is a representation which would be something like the lengths of the um like can you describe
how the dictionary should look and then i could kind of maybe advise because the only thing is
you know i've you've worked with svgs but in a very lazy way so you know okay yeah i mean like to
let's say to create like a line chart you would just use the polyline svg shape and then
you would the points um you would have to grab the you have to kind of set it set up the axes of
the canvas um take the data that you're getting from the data frame and uh you're kind of piping
that in but then you have to massage the data in a certain way to get it to scale properly
you probably need to we'll need to normalize it um i would say basically what you're talking about
is something kind of fairly um it's it either has something existing for it in which case
you know i probably wind up googling around the same as you and i encourage you actually to
maybe add send me drop me an email and i'll do a quick google or to someone on the chat
on the call no about this so um geopandas does this geopandas has a column for polygons and svgs
um so the geospatial geojson community has already done some of these things
um so if you go yeah if you go and check out geopandas you'll get some of this stuff and um
alternatively um actually yeah no stick with that um that's probably the cheapest way to get svg
um thank you i just want to say this is um the person who just answered there is
tony fast who you know is a contributor to project jupiter and i had the pleasure of working
with tony last year on a project with space telescope science institute to make their
notebook outputs more accessible you can check out that project it's called notebooks for all
and i will say tony is um he's been working on creating a version of jupyter notebooks that is
um uh that is more readily accessible to us um and outputs things in more in more useful ways and so
on so if you're interested in that maybe just send me an email i'll put you in touch with tony i know
he's looking for testers and i heard that tony might be doing some kind of event coming up related
to that as well so i'll let you guys know by email yeah i'm super interested in uh tactile svgs um
especially on like tablets and phones so let's get in touch marco yeah definitely because i know
the one thing is like to get the svg out but also make it human readable so we can
get through the file and add our own things to it which is what i've been doing with like my own
website um yeah yeah we can we can work on that stuff we just have to do some nasty things with
pandas they're outside of the scope of here yeah i would say definitely possible it's just a matter
of how to get from point a to point b in the in the easiest way and and uh and i'll put you guys
does anyone have thanks does anyone else have any uh questions they'd like to get on the mic and ask
and that was a fairly you know advanced question but you could ask a question about anything you
know it doesn't have to be okay i live for those awkward pauses but you have to do them or
otherwise people won't ask but we'll i think then we'll end the recording and i'll stay on for a
minute or two if anyone has any other questions and we're looking forward to seeing you for um
the uh the fourth workshop in the series that um sarah king will be leading on uh
just the same time next tuesday so thank you all