Preamble¶

Introduction¶

In a chord diagram (or radial network), entities are arranged radially as segments with their relationships visualised by arcs that connect them. The size of the segments illustrates the numerical proportions, whilst the size of the arc illustrates the significance of the relationships¹.

Chord diagrams are useful when trying to convey relationships between different entities, and they can be beautiful and eye-catching.

Get Chord Pro¶

Click here to get lifetime access to the full-featured chord visualization API, producing beautiful interactive visualizations, e.g. those featured on the front page of Reddit.

• Produce beautiful interactive Chord diagrams.
• Customize colours and font-sizes.
• Access Divided mode, enabling two sides to your diagram.
• Symmetric and Asymmetric modes,
• Add images and text on hover,
• Access finer-customisations including HTML injection.
• Allows commercial use without open source requirement.
• Currently supports Python, JavaScript, and Rust, with many more to come (accepting requests).

The Chord Package¶

With Python in mind, there are many libraries available for creating Chord diagrams, such as Plotly, Bokeh, and a few that are lesser-known. However, I wanted to use the implementation from d3 because it can be customised to be highly interactive and to look beautiful.

I couldn't find anything that ticked all the boxes, so I made a wrapper around d3-chord myself. It took some time to get it working, but I wanted to hide away everything behind a single constructor and method call. The tricky part was enabling multiple chord diagrams on the same page, and then loading resources in a way that would support Jupyter Notebooks.

You can get the package either from PyPi using pip install chord or from the GitHub repository. With your processed data, you should be able to plot something beautiful with just a single line, Chord(data, names).show(). To enable the pro features of the chord package, get Chord Pro.

The Dataset¶

The focus for this section will be the demonstration of the chord package. To keep it simple, we will use synthetic data that illustrates the co-occurrences between movie genres within the same movie.

Chord Diagrams¶

Let's see what the Chord() defaults produce when we invoke the show() method.

Different Colours¶

The defaults are nice, but what if we want different colours? You can pass in almost anything from d3-scale-chromatic, or you could pass in a list of hexadecimal colour codes.

Label Styling¶

We can disable the wrapped labels, and even change the colour.

Opacity¶

We can also change the default opacity of the relationships.

Conclusion¶

In this section, we've introduced the chord diagram and chord package. We used the package and some synthetic data to demonstrate several chord diagram visualisations with different configurations. The chord Python package is available for free using pip install chord.

Preamble¶

Introduction¶

NumPy has many features that Rust's NDArray doesn't have yet, e.g. index arrays and mask index arrays. For example, when we index a one-dimensional array we often use a single integer value to return an element at the corresponding position. That is, given an array example containing ten floating-point value elements

We would access the third element using

However, there is more than one way to index an array! We may wish to index an array with another array to select multiple samples at once, e.g. example[[2, 5, 8]] for the third, sixth, and ninth sample. We may also want to index an array using a boolean mask array, allowing us to select samples based on some criteria, e.g. example[example > 0.5] to select all samples greater than $0.5$.

Currently, NDArray doesn't offer an easy way to index an array with another array or with a mask array, but it can still be achieved with some extra work. What follows is an approach for selecting samples using index and mask arrays.

Loading our Dataset¶

We will continue using the Iris Flower dataset, so we will load it using the darn crate to avoid repetition.

We will only be using our species labels, stored in labels, throughout the rest of this section.

The darn::iris_typed() function returns a tuple of type (Array2::<f32>, Vec<String>, Array1::<String>), where the first element is an array containing our iris flower features, the second element a vector of our feature headers, and the final element is an array containing our iris species labels. As always, let's have a quick look at a few samples from our features.

We'll also check the unique elements in our labels.

To make things easier for ourselves throughout the rest of this section, let's assign the various parts of our dataset to different variables.

Index Array¶

During our analyses, we may encounter the need to select multiple samples from our dataset. For example, we may wish to select the samples at index $0$, $10$, and $20$. Let's output the samples at these indeces for reference.

To return these samples all at once using an array as the index, we can use the ArrayBase::select() function:

Select arbitrary subviews corresponding to indices and and copy them into a new array.

The first parameter for this function is the axis we wish to select along, and the second is an array containing the desired indices.

If we check these against the individually indexed samples above, we can see that it has worked as intended.

Index Mask Arrays¶

We may also want to use a mask to index our array. To work around this missing feature in NDArray, we can build a boolean mask and then use it to generate an index array. We can also use column-wise boolean operations when considering multiple column masks.

Building Boolean Masks¶

We can build a boolean mask of the same shape as our array with true values where some condition is met. For example, in NumPy we could do features > 0.5 to create a mask with true where values are over $0.5$, and false elsewhere.

We can do the same with Rust and ndarray.

Let's take peek at our mask to see how it looks.

We could also build a mask using our labels. For example, we may want to return true where elements are equal to Iris-virginica.

Indexing with Mask Arrays¶

Now to build an index array and a mask array simultaneously.

With this approach, we're iterating through every element in the array we wish to mask, labels. When our criteria is satisfied, i.e. elem == "Iris-virginica", we're pushing the curent index stored in count to a vector named indices. The map transormation itself builds the mask based on the criteria specified.

Let's have a look at the indices returnd from this approach.

Finally, we can use these indices to select all the samples which belong to the Virginica species.

Plotting with Plotly¶

It's always helpful to visualise what we've achieved. Let's plot the petal width and height for all of our samples, and then plot the same for all samples of the Virginica species in a different colour.

Conclusion¶

In this section, we had a look at how to build index and mask index arrays for use with NDArray indexing. We demonstrated this with the Iris Flower dataset, which enabled us to produce a nice visualisation showing the clustering of one of the species for two of the features. In the following sections, we'll have a look at how to use index and mask index arrays to modify values in-place.

Preamble¶

Introduction¶

In this section, we're going to use daily confirmed cases data for COVID-19 in the UK made available at coronavirus.data.gov.uk to create a time series plot. Our goal will be to visualise the number of new cases and cumulative cases over time.

Visualising the Table¶

The first step is to read the CSV data into a pandas.DataFrame and display the first five samples.

Let's filter this data to only include rows where the Area name is England.

This data looks ready to plot. We have our dates in a column named Specimen date, the new daily cases in a column named Daily lab-confirmed cases, and the daily cumulative cases in a column named Cumulative lab-confirmed cases. For this plot, we'll enable a secondary y-axis so that we can present our cumulative cases as a line, and our new cases with bars.

It's an interactive plot, so you can hover over it to get more information.

Conclusion¶

In this section, we went on a rather quick journey. This involved loading in the CSV data directly from a web resource, and then plotting lines and bars to the same plot.

Preamble¶

Introduction¶

In this section, we're going to take a look at some approaches to determining the unique elements in a column of data and their frequency. This is a common task which we may often apply to categorical data, and it can easily give us a good idea of the balance of categorical values across our dataset.

If you're familiar with Python and NumPy, you may have encountered the numpy.unique()function that returns a list of sorted unique elements of an array. In more recent versions of NumPy, the numpy.unique() function also takes a parameter named return_counts, which when set to True will also return the number of times each unique element appears in the array.

Let's see how we can do the same for our ndarray::Array2.

Loading our Dataset¶

We will continue using the Iris Flower dataset, so we need to load it into our raw string array first.

Moving Data to Typed Arrays¶

We need to convert from String to the desired type, and move our data over to the typed arrays.

We will only be using our species labels, stored in labels, throughout the rest of this section.

Unique Elements¶

We can get the unique elements in our array using the unique() function provided by the itertools crate.

Return an iterator adaptor that filters out elements that have already been produced once during the iteration. Duplicates are detected using hash and equality.

We can then iterate through this and output the elements to the output cell.

We can also use the .format() function on .unique() to print out our unique elements.

However, we may want to store these in a vector to use later too. Let's store them in unique_elements.

Count of Unique Elements¶

In this case it's easy to see we have three unique elements, but if the list is too long or we wanted to store the value for later use, we can use the len() function on our vector containing the unique elements.

We could also get the count from our iterator.

Frequency of Unique Elements¶

To find the frequency of a specific string we can use filter() and then count().

We may also want to find the frequency of every unique element and store it in a vector. We'll call this unique_frequency.

We can populate this by iterating through the unique elements and use the strings to find the frequency using the filter() approach above.

Visualise the Frequency of Unique Elements¶

It's useful to visualise the frequency of elements in our array, especially if we're interested in looking at the balance in our dataset. We can do this with a Bar plot using Plotly.

Unique Elements and their Frequency with Hashmaps¶

We can do something similar with a HashMap. First we'll define a variable of type HashMap<String, i32>, where the String part will be the unique element, and the i32 will be the frequency.

We can then populate this by iterating through our original labels array.

Printing out the results, we can see that value_counts now contains all the information that we're after.

We can also get these values directly by key.

However, you will need to map this HashMap to separate vectors if you want to use it for plotting with Plotly.

Conclusion¶

In this section, we've demonstrated a few approaches to identifying the unique elements in an array, counting the number of unique elements, and the frequency of these unique elements. We also visualised the frequency of our unique elements which is useful during exploratory data analysis.

Preamble¶

Introduction¶

In this section, we're going to visualise Novel Coronavirus 2019 time series data for confirmed cases, recovered cases and deaths. We'll be working on two visualisations:

1. A static map visualising our features for the latest time group in the dataset.
2. An interactive and animated map visualising our features over time.

We'll be using the Mapbox service so we'll need to set our access token.

The Dataset¶

We're going to be using the Novel Corona virus - COVID19 dataset with the following description:

The new strain of Coronavirus has had a worldwide effect. It has affected people from different countries. The dataset provides, a time series data tracking the number of people affected by the virus, how many deaths has the virus caused and the number of reported people who have recovered.

Let's download it from their repository and take a peak.

We can see that we have the following features to work with.

• Province/State and Country/Region. These features contain the named location information associated with the sample. We'll use the Province/State field in our on-hover tool-tip and to group our colour grouping. We can see from the first five samples that some of these may be empty, or NaN, so as a workaround we'll copy in the "Country/Region" feature where the data is missing.

• Lat and Long. These features contain the latitude and longitude geographic coordinates associated with the sample. We'll use both of these to determine where on the map we will draw our markers.
• Date. This feature contains the date associated with the sample. We'll use this to build our animation over time.
• Confirmed, Recovered, Deaths. These features contain numerical values for the number of confirmed cases, the number of recovered cases, and the number of deaths, respectively. We'll use the number of confirmed cases to change the size of our markers, and the number of deaths to change the colour.

We'll also add our own feature to estimate the number of active cases. We'll calculate it by subtracting the number of recovered cases and deaths from the confirmed cases. We can use this instead of the confirmed cases for our marker size in the animation.

There's a possibility we will have NaN values in our data. We're not interested in investigating this further or conducting any imputation in this section, we will simply remove any rows that have this issue.

The Latest Information¶

Let's create the first of our two visualisations, this one will present our features for the most recent time point in our data. We need to create a Boolean mask so that we can select only the relevant samples.

We can now use this mask to select a subset of our dataset to produce a Figure object with Plotly Express.

Reading through the parameters, we can see that we've:

• Passed in the masked subset of our dataset;
• Specified the latitude and longitude columns of that DataFrame to position our markers
• Set the size of our markers to be the number of confirmed cases, with a maximum size of 50.
• Set the colour of our markers to the number of deaths, on a continuous scale with the colour palette "Pinkyl";
• Set our on-hover tooltip to be the Province/State value;
• and set our map style to dark with a zoom of 1.

One extra configuration change we'll make is to remove the axis scale that will appear to the right of the figure. This is just a case of preference, you can see what you think of it by removing the line below or setting the value to True instead.

All that's left is to display our visualisation.

You can interact with the visualisation by dragging, zooming, hovering, etc.

The Animated Time Series¶

Let's create the second of our two visualisations, this one will take us on a journey through time, giving us some idea of how the features have changed throughout the duration of the dataset. This time we will be passing in the entire DataFrame instead of the masked one, and we'll use our own active cases feature instead of the number of confirmed cases.

We can see two additional parameters for this plot which are used to specify how the animation frames are generated, and how they are grouped from frame to frame. In addition to removing the axis scale, we'll also make some additional changes to customise our animation and the positioning of some of the control elements.

Now we can display our visualisation.