Join the Patreon to get Exclusive Downloads, Direct Support, Early Access, Voting Access, our Books for Free, and so much more!

Data is Beautiful

A practical book on data visualisation that shows you how to create static and interactive visualisations that are engaging and beautiful.

Get the book

StamiStudios.com Everyday Ita Bag - Panels and Colours

Dr. Shahin Rostami

2020-06-13

7 min read

Contents Chat Share Follow Download Source

Preamble
Introduction

The Dataset

Data Wrangling
Chord Diagram
Chord Diagram with Names
Conclusion

Get access to this section and more

Below is a featured selection from this section. You can access this notebook and more by getting the e-book, Data is Beautiful.

Chord Diagram

Preamble¶

In [1]:

import numpy as np                   # for multi-dimensional containers 
import pandas as pd                  # for DataFrames
import itertools
from chord import Chord

Introduction¶

In previous sections, we visualised co-occurrences of Pokémon type. Whilst it was interesting to look at, the dataset only contained Pokémon from the first six geerations. In this section, we're going to use the TidyTuesday Animal Crossing villagers dataset to visualise the relationship between Species and .

The Dataset¶

The dataset documentation states that we can expect 13 variables per each of the 1017 Pokémon of the first eight generations.

Let's download the mirrored dataset and have a look for ourselves.

In [2]:

data_url = 'https://shahinrostami.com/datasets/stami_bags_panel_colour.csv'
data = pd.read_csv(data_url)
data.head()

Out[2]:

	panel	colour
0	Heart	Pink
1	Circle	lilac
2	Sakura	Mint
3	circle	mint
4	Animal	Black

In [3]:

#data['Country.of.Origin'][data['Country.of.Origin'] == 'United States (Hawaii)'] = 'Hawaii'
#data['Country.of.Origin'][data['Country.of.Origin'] == 'Tanzania, United Republic Of'] = 'Tanzania'

In [4]:

#data = data[data.Variety != 'Other']
data = data[data.notna()]

In [5]:

data = data[data['panel'].isin(list(data['panel'].value_counts()[:20].index))]
data = data[data['colour'].isin(list(data['colour'].value_counts()[:11].index))]

In [ ]:

capitalise the name, personality, and species of each villager.

In [6]:

data['panel'] = data['panel'].str.capitalize()
data['colour'] = data['colour'].str.capitalize()

It looks good so far, but let's confirm the 13 variables against 1017 samples from the documentation.

In [7]:

data.shape

Out[7]:

(3040, 2)

In [8]:

d_colours = list(data.colour.value_counts().index)
d_colours.sort()
d_colours

Out[8]:

['Black', 'Blue', 'Green', 'Lilac', 'Mint', 'Navy', 'Pink', 'White', 'Yellow']

Perfect, that's exactly what we were expecting.

Data Wrangling¶

We need to do a bit of data wrangling before we can visualise our data. We can see from the columns names that the Pokémon types are split between the columns Type 1 and Type 2.

So let's select just these two columns and work with a list containing only them as we move forward.

In [9]:

species_personality = pd.DataFrame(data[['colour', 'panel']].values).dropna().astype(str)
species_personality

Out[9]:

	0	1
0	Pink	Heart
1	Mint	Sakura
2	Black	Animal
3	White	Circle
4	White	Star
...	...	...
3035	Blue	Circle
3036	White	Circle
3037	White	Heart
3038	White	Circle
3039	Black	Circle

3040 rows × 2 columns

In [10]:

species_personality = species_personality.dropna()

Now for the names of our types.

In [11]:

#left = np.unique(pd.DataFrame(species_personality)[0]).tolist()
left = list(data['colour'].value_counts().index)[::-1]
#left.sort()

pd.DataFrame(left)

Out[11]:

	0
0	Yellow
1	Green
2	Blue
3	Navy
4	Mint
5	Lilac
6	Pink
7	White
8	Black

In [12]:

#right = np.unique(pd.DataFrame(species_personality)[1]).tolist()
right = list(data['panel'].value_counts().index)
#right.sort()
pd.DataFrame(right)

Out[12]:

	0
0	Circle
1	Star
2	Crescent
3	Moon
4	Bat wings
5	Sakura
6	Heart
7	Dice20
8	Frog
9	Animal
10	Feline-ears
11	Cat
12	Angel-wings
13	Hive
14	Bottle
15	Paw
16	Petals
17	Pixel
18	Citrus

Which we can now use to create the matrix.

In [13]:

features= left+right
d = pd.DataFrame(0, index=features, columns=features)

Our chord diagram will need two inputs: the co-occurrence matrix, and a list of names to label the segments.

We can build a co-occurrence matrix with the following approach. We'll start by creating a list with every type pairing in its original and reversed form.

In [14]:

species_personality = list(itertools.chain.from_iterable((i, i[::-1]) for i in species_personality.values))

In [15]:

for x in species_personality:
    d.at[x[0], x[1]] += 1

In [16]:

d=d/(d.values.sum()/2)*100

In [17]:

Out[17]:

	Yellow	Green	Blue	Navy	Mint	Lilac	Pink	White	Black	Circle	...	Animal	Feline-ears	Cat	Angel-wings	Hive	Bottle	Paw	Petals	Pixel	Citrus
Yellow	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.032895	...	0.000000	0.000000	0.000000	0.000000	0.230263	0.000000	0.032895	0.032895	0.000000	0.855263
Green	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.032895
Blue	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.394737	...	0.296053	0.197368	0.164474	0.164474	0.000000	0.361842	0.098684	0.000000	0.098684	0.000000
Navy	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.625000	...	0.131579	0.098684	0.263158	0.000000	0.164474	0.328947	0.131579	0.065789	0.098684	0.098684
Mint	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.361842	...	0.098684	0.000000	0.131579	0.032895	0.065789	0.098684	0.263158	0.065789	0.197368	0.263158
Lilac	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.723684	...	0.493421	0.296053	0.493421	0.032895	0.098684	0.460526	0.296053	0.526316	0.263158	0.065789
Pink	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.986842	...	0.427632	0.328947	0.361842	0.263158	0.263158	0.032895	0.328947	0.789474	0.394737	0.164474
White	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	2.927632	...	1.348684	0.493421	0.493421	2.861842	0.625000	0.361842	0.263158	0.657895	0.197368	0.361842
Black	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	7.796053	...	1.940789	3.289474	2.138158	0.427632	1.743421	1.315789	0.953947	0.098684	0.953947	0.164474
Circle	0.032895	0.000000	0.394737	0.625000	0.361842	0.723684	0.986842	2.927632	7.796053	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Star	0.098684	0.000000	0.625000	0.427632	0.493421	0.723684	1.118421	1.513158	3.684211	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Crescent	0.098684	0.000000	0.328947	0.657895	0.197368	0.789474	1.743421	0.855263	3.256579	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Moon	0.164474	0.000000	0.690789	0.855263	0.361842	0.921053	0.723684	0.953947	3.059211	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Bat wings	0.000000	0.000000	0.098684	0.230263	0.032895	0.394737	0.065789	0.197368	6.644737	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Sakura	0.000000	0.000000	0.230263	0.065789	0.197368	0.394737	3.026316	0.953947	1.414474	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Heart	0.000000	0.000000	0.164474	0.098684	0.164474	0.592105	1.940789	0.888158	1.743421	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Dice20	0.000000	0.000000	0.164474	0.328947	0.230263	0.493421	0.394737	0.427632	2.993421	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Frog	0.000000	2.565789	0.065789	0.000000	1.907895	0.098684	0.328947	0.032895	0.000000	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Animal	0.000000	0.000000	0.296053	0.131579	0.098684	0.493421	0.427632	1.348684	1.940789	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Feline-ears	0.000000	0.000000	0.197368	0.098684	0.000000	0.296053	0.328947	0.493421	3.289474	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Cat	0.000000	0.000000	0.164474	0.263158	0.131579	0.493421	0.361842	0.493421	2.138158	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Angel-wings	0.000000	0.000000	0.164474	0.000000	0.032895	0.032895	0.263158	2.861842	0.427632	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Hive	0.230263	0.000000	0.000000	0.164474	0.065789	0.098684	0.263158	0.625000	1.743421	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Bottle	0.000000	0.000000	0.361842	0.328947	0.098684	0.460526	0.032895	0.361842	1.315789	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Paw	0.032895	0.000000	0.098684	0.131579	0.263158	0.296053	0.328947	0.263158	0.953947	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Petals	0.032895	0.000000	0.000000	0.065789	0.065789	0.526316	0.789474	0.657895	0.098684	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Pixel	0.000000	0.000000	0.098684	0.098684	0.197368	0.263158	0.394737	0.197368	0.953947	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
Citrus	0.855263	0.032895	0.000000	0.098684	0.263158	0.065789	0.164474	0.361842	0.164474	0.000000	...	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000

28 rows × 28 columns

Chord Diagram¶

Time to visualise the co-occurrence of types using a chord diagram. We are going to use a list of custom colours that represent the types.

In [18]:

colors =["#ffe75f", "#85e063", "#aed6f8", "#1e3c6f", "#affec6", "#d3b0e7", "#fedbe8", "#f5f4e9", "#222222", "#f7a296", "#f48fb1", "#ce93d8", "#a9a3db", "#89cffa", "#80deea", "#80cbc4", "#a5d6a7", "#e6ee9c", "#fff59d", "#ffe082", "#ffcc80", "#f7a296", "#f06292", "#a76fcb", "#7986cb", "#64b5f6", "#4ecaec", "#4db6ac"]

In [19]:

names = left + right

Finally, we can put it all together.

In [20]:

Chord(d.values.round(2).tolist(), names,credit=True, colors=colors, wrap_labels=False,
      margin=40, noun="percent",
        details_separator="", divide=True, divide_idx=len(left),divide_size=.2, width=850).show()

Chord Diagram

Chord Diagram with Names¶

It would be nice to show a list of Pokémon names when hovering over co-occurring Pokémon types. To do this, we can make use of the optional details parameter.

In [ ]:

Next, we'll create an empty multi-dimensional array with the same shape as our matrix.

In [21]:

details = np.empty((len(names),len(names)),dtype=object)
details_thumbs = np.empty((len(names),len(names)),dtype=object)

Now we can populate the details array with lists of Pokémon names in the correct positions.

In [22]:

for count_x, item_x in enumerate(names):
    for count_y, item_y in enumerate(names):
        details_urls = data[
            (data['species'].isin([item_x, item_y])) &
            (data['personality'].isin([item_y, item_x]))]['url'].to_list()
        
        details_names = data[
            (data['species'].isin([item_x, item_y])) &
            (data['personality'].isin([item_y, item_x]))]['name'].to_list()
        
        urls_names = np.column_stack((details_urls, details_names))
        if(urls_names.size > 0):
            details[count_x][count_y] = details_names
            details_thumbs[count_x][count_y] = details_urls

        else:
            details[count_x][count_y] = []
            details_thumbs[count_x][count_y] = []

details=pd.DataFrame(details).values.tolist()
details_thumbs=pd.DataFrame(details_thumbs).values.tolist()

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/miniconda3/envs/analytics/lib/python3.7/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   2645             try:
-> 2646                 return self._engine.get_loc(key)
   2647             except KeyError:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'species'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-22-f54e3a0b2906> in <module>
      2     for count_y, item_y in enumerate(names):
      3         details_urls = data[
----> 4             (data['species'].isin([item_x, item_y])) &
      5             (data['personality'].isin([item_y, item_x]))]['url'].to_list()
      6 

~/miniconda3/envs/analytics/lib/python3.7/site-packages/pandas/core/frame.py in __getitem__(self, key)
   2798             if self.columns.nlevels > 1:
   2799                 return self._getitem_multilevel(key)
-> 2800             indexer = self.columns.get_loc(key)
   2801             if is_integer(indexer):
   2802                 indexer = [indexer]

~/miniconda3/envs/analytics/lib/python3.7/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   2646                 return self._engine.get_loc(key)
   2647             except KeyError:
-> 2648                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   2649         indexer = self.get_indexer([key], method=method, tolerance=tolerance)
   2650         if indexer.ndim > 1 or indexer.size > 1:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'species'

In [ ]:

len(right)

Finally, we can put it all together but this time with the details matrix passed in.

In [ ]:

Chord(d.values.tolist(), names,credit=True, colors=colors, wrap_labels=False,
      margin=40, font_size_large=7,details=details,details_thumbs=details_thumbs,noun="villagers",
        details_separator="", divide=True, divide_idx=len(left),divide_size=.2, width=850).show()

In [ ]:

np.empty(shape=(6,1)).tolist()

Conclusion¶

In this section, we demonstrated how to conduct some data wrangling on a downloaded dataset to prepare it for a chord diagram. Our chord diagram is interactive, so you can use your mouse or touchscreen to investigate the co-occurrences!

Get access to this section and more

You can access this notebook and more by getting the e-book, Data is Beautiful.