Join the Patreon to get Exclusive Downloads, Direct Support, Early Access, Voting Access, our Books for Free, and so much more!

Data is Beautiful

A practical book on data visualisation that shows you how to create static and interactive visualisations that are engaging and beautiful.

Get the book

Arabica Coffee Beans - Origin and Variety

Dr. Shahin Rostami

2020-06-11

7 min read

Contents Chat Share Follow Download Source

Preamble
Introduction

The Dataset

Data Wrangling
Chord Diagram
Chord Diagram with Names
Conclusion

Made with Chord Pro

You can create beautiful interactive visualisations like this one with Chord Pro. Learn how to make beautiful visualisations with the book, Data is Beautiful.

Chord Diagram

Preamble¶

In [1]:

import numpy as np                   # for multi-dimensional containers 
import pandas as pd                  # for DataFrames
import itertools
from chord import Chord

Introduction¶

In previous sections, we visualised co-occurrences of Pokémon type. Whilst it was interesting to look at, the dataset only contained Pokémon from the first six geerations. In this section, we're going to use the TidyTuesday Animal Crossing villagers dataset to visualise the relationship between Species and .

The Dataset¶

The dataset documentation states that we can expect 13 variables per each of the 1017 Pokémon of the first eight generations.

Let's download the mirrored dataset and have a look for ourselves.

In [2]:

data_url = 'https://shahinrostami.com/datasets/arabica_data.csv'
data = pd.read_csv(data_url)
data.head()

Out[2]:

	Unnamed: 0	Species	Owner	Country.of.Origin	Farm.Name	Lot.Number	Mill	ICO.Number	Company	Altitude	...	Color	Category.Two.Defects	Expiration	Certification.Body	Certification.Address	Certification.Contact	unit_of_measurement	altitude_low_meters	altitude_high_meters	altitude_mean_meters
0	1	Arabica	metad plc	Ethiopia	metad plc	NaN	metad plc	2014/2015	metad agricultural developmet plc	1950-2200	...	Green	0	April 3rd, 2016	METAD Agricultural Development plc	309fcf77415a3661ae83e027f7e5f05dad786e44	19fef5a731de2db57d16da10287413f5f99bc2dd	m	1950.0	2200.0	2075.0
1	2	Arabica	metad plc	Ethiopia	metad plc	NaN	metad plc	2014/2015	metad agricultural developmet plc	1950-2200	...	Green	1	April 3rd, 2016	METAD Agricultural Development plc	309fcf77415a3661ae83e027f7e5f05dad786e44	19fef5a731de2db57d16da10287413f5f99bc2dd	m	1950.0	2200.0	2075.0
2	3	Arabica	grounds for health admin	Guatemala	san marcos barrancas "san cristobal cuch	NaN	NaN	NaN	NaN	1600 - 1800 m	...	NaN	0	May 31st, 2011	Specialty Coffee Association	36d0d00a3724338ba7937c52a378d085f2172daa	0878a7d4b9d35ddbf0fe2ce69a2062cceb45a660	m	1600.0	1800.0	1700.0
3	4	Arabica	yidnekachew dabessa	Ethiopia	yidnekachew dabessa coffee plantation	NaN	wolensu	NaN	yidnekachew debessa coffee plantation	1800-2200	...	Green	2	March 25th, 2016	METAD Agricultural Development plc	309fcf77415a3661ae83e027f7e5f05dad786e44	19fef5a731de2db57d16da10287413f5f99bc2dd	m	1800.0	2200.0	2000.0
4	5	Arabica	metad plc	Ethiopia	metad plc	NaN	metad plc	2014/2015	metad agricultural developmet plc	1950-2200	...	Green	2	April 3rd, 2016	METAD Agricultural Development plc	309fcf77415a3661ae83e027f7e5f05dad786e44	19fef5a731de2db57d16da10287413f5f99bc2dd	m	1950.0	2200.0	2075.0

5 rows × 44 columns

In [3]:

data['Country.of.Origin'][data['Country.of.Origin'] == 'United States (Hawaii)'] = 'Hawaii'
data['Country.of.Origin'][data['Country.of.Origin'] == 'Tanzania, United Republic Of'] = 'Tanzania'

/home/shahin/miniconda3/envs/analytics/lib/python3.7/site-packages/ipykernel_launcher.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  """Entry point for launching an IPython kernel.
/home/shahin/miniconda3/envs/analytics/lib/python3.7/site-packages/ipykernel_launcher.py:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

In [4]:

data = data[data.Variety != 'Other']
data = data[data['Variety'].notna()]

In [5]:

data = data[data['Country.of.Origin'].isin(list(data['Country.of.Origin'].value_counts()[:10].index))]
data = data[data['Variety'].isin(list(data['Variety'].value_counts()[:10].index))]

In [ ]:

capitalise the name, personality, and species of each villager.

In [6]:

#data['Publisher'] = data['Publisher'].str.capitalize()
#data['Genre'] = data['Genre'].str.capitalize()

It looks good so far, but let's confirm the 13 variables against 1017 samples from the documentation.

In [7]:

data.shape

Out[7]:

(851, 44)

Perfect, that's exactly what we were expecting.

Data Wrangling¶

We need to do a bit of data wrangling before we can visualise our data. We can see from the columns names that the Pokémon types are split between the columns Type 1 and Type 2.

So let's select just these two columns and work with a list containing only them as we move forward.

In [8]:

species_personality = pd.DataFrame(data[['Country.of.Origin', 'Variety']].values).dropna().astype(str)
species_personality

Out[8]:

	0	1
0	Guatemala	Bourbon
1	Costa Rica	Caturra
2	Brazil	Bourbon
3	Uganda	SL14
4	Honduras	Caturra
...	...	...
846	Honduras	Catuai
847	Honduras	Catuai
848	Mexico	Bourbon
849	Guatemala	Catuai
850	Honduras	Caturra

851 rows × 2 columns

In [9]:

species_personality = species_personality.dropna()

Now for the names of our types.

In [10]:

#left = np.unique(pd.DataFrame(species_personality)[0]).tolist()
left = list(data['Country.of.Origin'].value_counts().index)[::-1]
#left.sort()

pd.DataFrame(left)

Out[10]:

	0
0	Kenya
1	Uganda
2	Hawaii
3	Costa Rica
4	Honduras
5	Taiwan
6	Brazil
7	Colombia
8	Guatemala
9	Mexico

In [11]:

#right = np.unique(pd.DataFrame(species_personality)[1]).tolist()
right = list(data['Variety'].value_counts().index)
#right.sort()
pd.DataFrame(right)

Out[11]:

	0
0	Caturra
1	Bourbon
2	Typica
3	Catuai
4	Hawaiian Kona
5	Yellow Bourbon
6	Mundo Novo
7	SL14
8	SL28
9	Pacas

Which we can now use to create the matrix.

In [12]:

features= left+right
d = pd.DataFrame(0, index=features, columns=features)

Our chord diagram will need two inputs: the co-occurrence matrix, and a list of names to label the segments.

We can build a co-occurrence matrix with the following approach. We'll start by creating a list with every type pairing in its original and reversed form.

In [13]:

species_personality = list(itertools.chain.from_iterable((i, i[::-1]) for i in species_personality.values))

In [14]:

for x in species_personality:
    d.at[x[0], x[1]] += 1

In [ ]:

Chord Diagram¶

Time to visualise the co-occurrence of types using a chord diagram. We are going to use a list of custom colours that represent the types.

In [15]:

colors =[
    "#ff575c","#ff914d","#ffca38","#f2fa00","#94f000","#00fa68","#0087db","#0054f0","#000cf5","#5d00e0",
    
"#6f1d1b","#955939","#bb9457","#7f5e38","#432818","#6e4021","#99582a","#cc9f69","#ffe6a7","#755939"]

In [16]:

names = left + right

Finally, we can put it all together.

In [17]:

Chord(d.values.tolist(), names,credit=True, colors=colors, wrap_labels=False,
      margin=40, font_size_large=7,noun="coffee beans",
        details_separator="", divide=True, divide_idx=len(left),divide_size=.2, width=850).show()

Chord Diagram

Chord Diagram with Names¶

It would be nice to show a list of Pokémon names when hovering over co-occurring Pokémon types. To do this, we can make use of the optional details parameter.

In [ ]:

Next, we'll create an empty multi-dimensional array with the same shape as our matrix.

In [18]:

details = np.empty((len(names),len(names)),dtype=object)
details_thumbs = np.empty((len(names),len(names)),dtype=object)

Now we can populate the details array with lists of Pokémon names in the correct positions.

In [19]:

for count_x, item_x in enumerate(names):
    for count_y, item_y in enumerate(names):
        details_urls = data[
            (data['species'].isin([item_x, item_y])) &
            (data['personality'].isin([item_y, item_x]))]['url'].to_list()
        
        details_names = data[
            (data['species'].isin([item_x, item_y])) &
            (data['personality'].isin([item_y, item_x]))]['name'].to_list()
        
        urls_names = np.column_stack((details_urls, details_names))
        if(urls_names.size > 0):
            details[count_x][count_y] = details_names
            details_thumbs[count_x][count_y] = details_urls

        else:
            details[count_x][count_y] = []
            details_thumbs[count_x][count_y] = []

details=pd.DataFrame(details).values.tolist()
details_thumbs=pd.DataFrame(details_thumbs).values.tolist()

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/miniconda3/envs/analytics/lib/python3.7/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   2645             try:
-> 2646                 return self._engine.get_loc(key)
   2647             except KeyError:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'species'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-19-f54e3a0b2906> in <module>
      2     for count_y, item_y in enumerate(names):
      3         details_urls = data[
----> 4             (data['species'].isin([item_x, item_y])) &
      5             (data['personality'].isin([item_y, item_x]))]['url'].to_list()
      6 

~/miniconda3/envs/analytics/lib/python3.7/site-packages/pandas/core/frame.py in __getitem__(self, key)
   2798             if self.columns.nlevels > 1:
   2799                 return self._getitem_multilevel(key)
-> 2800             indexer = self.columns.get_loc(key)
   2801             if is_integer(indexer):
   2802                 indexer = [indexer]

~/miniconda3/envs/analytics/lib/python3.7/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   2646                 return self._engine.get_loc(key)
   2647             except KeyError:
-> 2648                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   2649         indexer = self.get_indexer([key], method=method, tolerance=tolerance)
   2650         if indexer.ndim > 1 or indexer.size > 1:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item()

KeyError: 'species'

In [ ]:

len(right)

Finally, we can put it all together but this time with the details matrix passed in.

In [ ]:

Chord(d.values.tolist(), names,credit=True, colors=colors, wrap_labels=False,
      margin=40, font_size_large=7,details=details,details_thumbs=details_thumbs,noun="villagers",
        details_separator="", divide=True, divide_idx=len(left),divide_size=.2, width=850).show()

In [ ]:

np.empty(shape=(6,1)).tolist()

Conclusion¶

In this section, we demonstrated how to conduct some data wrangling on a downloaded dataset to prepare it for a chord diagram. Our chord diagram is interactive, so you can use your mouse or touchscreen to investigate the co-occurrences!

Made with Chord Pro

You can create beautiful interactive visualisations like this one with Chord Pro. Learn how to make beautiful visualisations with the book, Data is Beautiful.