US Mortality - Race and Manner of Death
Preamble¶
import numpy as np # for multi-dimensional containers
import pandas as pd # for DataFrames
import itertools
from chord import Chord
Introduction¶
In previous sections, we visualised co-occurrences of Pokémon type. Whilst it was interesting to look at, the dataset only contained Pokémon from the first six geerations. In this section, we're going to use the TidyTuesday Animal Crossing villagers dataset to visualise the relationship between Species and .
The Dataset¶
The dataset documentation states that we can expect 13 variables per each of the 1017 Pokémon of the first eight generations.
Let's download the mirrored dataset and have a look for ourselves.
data_url = '/Users/shahin/Documents/devel/data/2015_data.csv'
data = pd.read_csv(data_url)
data.head()
data['race_recode_5'].value_counts()
data.columns
capitalise the name, personality, and species of each villager.
#data['manner'] = data['manner_of_death']#.str.capitalize()
#data['race_recode_5'] = data['race_recode_5']#.str.capitalize()
#data['species'] = data['species'].str.capitalize()
It looks good so far, but let's confirm the 13 variables against 1017 samples from the documentation.
data.shape
Perfect, that's exactly what we were expecting.
Data Wrangling¶
We need to do a bit of data wrangling before we can visualise our data. We can see from the columns names that the Pokémon types are split between the columns Type 1
and Type 2
.
pd.DataFrame(data.columns.values.tolist())
So let's select just these two columns and work with a list containing only them as we move forward.
data.fillna(0, inplace=True)
data.iloc[6572].manner_of_death
data.manner_of_death.value_counts()
{'143961': 'Accident',
'44417': 'Suicide',
'18885': 'Homicide',
'4165': 'Pending investigation',
'11054': 'Could not determine',
'2107352': 'Natural',
'388364': 'Not specified'}
import json
with open("/Users/shahin/Documents/devel/data/2015_data.json", "r") as read_file:
codes = json.load(read_file)
codes['manner_of_death']
codes['manner_of_death']['0'] = codes['manner_of_death'].pop('Blank')
remove = ["Natural", "Not specified", "Could not determine", "Pending investigation"]
list(codes['manner_of_death'].values())
left = list(codes['race_recode_5'].values())
pd.DataFrame(left)
right = list(codes['manner_of_death'].values())
pd.DataFrame(right)
right = [x for x in right if x not in remove]
data['manner_of_death'] = data['manner_of_death'].astype('int32')
data['manner_of_death'] = data['manner_of_death'].astype('str')
data.iloc[6572].manner_of_death
left.sort()
right.sort()
left
data = data.replace({"manner_of_death": codes['manner_of_death']})
data['race_recode_5'] = data['race_recode_5'].astype('int32')
data['race_recode_5'] = data['race_recode_5'].astype('str')
data.iloc[6572].race_recode_5
data = data.replace({"race_recode_5": codes['race_recode_5']})
data.iloc[6572].race_recode_5
manner_race = pd.DataFrame(data[['manner_of_death', 'race_recode_5']].values)
manner_race
Now for the names of our types.
Which we can now use to create the matrix.
features= left+right
d = pd.DataFrame(0, index=features, columns=features)
Our chord diagram will need two inputs: the co-occurrence matrix, and a list of names to label the segments.
We can build a co-occurrence matrix with the following approach. We'll start by creating a list with every type pairing in its original and reversed form.
manner_race = list(itertools.chain.from_iterable((i, i[::-1]) for i in manner_race.values))
for x in manner_race:
if(x[0] not in remove and x[1] not in remove):
d.at[x[0], x[1]] += 1
d
for race in left:
d.loc[ race , : ] = ((d.loc[ race , : ] / d.loc[ race , : ].sum()) * 100)
d.loc[ : , race ] = ((d.loc[ : , race ] / d.loc[ : , race ].sum()) * 100)
(d[race].value_counts(normalize=True)*100).astype(int)
d
Chord Diagram¶
Time to visualise the co-occurrence of types using a chord diagram. We are going to use a list of custom colours that represent the types.
colors = ["#2DE1FC","#1883B4","#C5DB66","#90B64D","#DB2B39","#E76926", "#DB9118"]
names = left + right
Finally, we can put it all together.
names[1]= "Asian or PI"
names[0]= "AIAN"
d
Finally, we can put it all together but this time with the details
matrix passed in.
Chord(
d.round(2).values.tolist(),
names,
colors=colors,
credit=True,
wrap_labels=True,
margin=50,
font_size_large=7,
divide=True,
noun="percent",
divide_idx=len(left),
divide_size=.2,
width=850).show()
Chord Diagram with Names¶
It would be nice to show a list of Pokémon names when hovering over co-occurring Pokémon types. To do this, we can make use of the optional details
parameter.
d
Next, we'll create an empty multi-dimensional array with the same shape as our matrix
.
details = np.empty((len(names),len(names)),dtype=object)
details_thumbs = np.empty((len(names),len(names)),dtype=object)
Now we can populate the details
array with lists of Pokémon names in the correct positions.
for count_x, item_x in enumerate(names):
for count_y, item_y in enumerate(names):
details_urls = data[
(data['species'].isin([item_x, item_y])) &
(data['personality'].isin([item_y, item_x]))]['url'].to_list()
details_names = data[
(data['species'].isin([item_x, item_y])) &
(data['personality'].isin([item_y, item_x]))]['name'].to_list()
urls_names = np.column_stack((details_urls, details_names))
if(urls_names.size > 0):
details[count_x][count_y] = details_names
details_thumbs[count_x][count_y] = details_urls
else:
details[count_x][count_y] = []
details_thumbs[count_x][count_y] = []
details=pd.DataFrame(details).values.tolist()
details_thumbs=pd.DataFrame(details_thumbs).values.tolist()
len(right)
Chord(d.values.tolist(), names,credit=True, colors=colors, wrap_labels=False,
margin=40, font_size_large=7,details=details,details_thumbs=details_thumbs,noun="villagers",
details_separator="", divide=True, divide_idx=len(left),divide_size=.2, width=850).show()
np.empty(shape=(6,1)).tolist()
Conclusion¶
In this section, we demonstrated how to conduct some data wrangling on a downloaded dataset to prepare it for a chord diagram. Our chord diagram is interactive, so you can use your mouse or touchscreen to investigate the co-occurrences!