Behind The Scenes - Coronavirus Web App
As we being the second week of lockdown, and what is undoubtably been a difficult week for us all struggling to juggle becoming teachers, full time employees and family life. I decided to add something else onto the pile and make a small application to track Coronavirus Infections in Scotland.
Like most parents I was always keen to see how many infections were estimated to be in my health board. There was the occasional news feed but it was never easy to find. I then found the official Scottish Government figures that were updated every day, but once I found these I began asking myself other questions:
- What was yesterdays figures?
- What was the change since yesterday?
- What has the growth been like in my region?
- How did that growth compare with other regions in Scotland?
I couldn't find an easy way to answer this and so decided that over the weekend I would write a little React app to help answer these questions.
I have released this app on a special Covid page on my website.
The app was created using:
- Realtime Database (Firebase).
The data comes from the official Scottish Government Covid-19 page. This page is updated everyday at 2PM and displays the data as a simple table. I needed a way to get this information, transform it into a structure I could work with and save it in my database.
The solution was a simple web scraper written in nodejs that could retrieve these figures. Using cheerio.js I was able to load the HTML into memory where I could then parse it to identify the Health Board and the associated number of positive cases. The data is then transformed into a simple JSON structure so I could quickly save to a database and work with it easily. Given the time I had set myself to do this I elected to use Firebase - Realtime Database. The database stores data as JSON, making it easy to write and read from. Once transformed the resulting data tree looks like this:
To automate the app I set up a quick CRON job to execute my script daily to update the figures at 1430. The scraper isn't bullet proof and the structure is subject to change meaning that there might be errors when this is run. So far I have spotted a series of inconsistencies such spelling mistakes, instead of white space and so on.
I managed to get a history of data back to the 7th March 2020.
The second part of the puzzle is display this data and attempting to answer some of my questions. I bootstrapped the application using create-react-app for ease and kept the component hierarchy one level deep. The components were implemented using functional components and I reused App.js as the container for data fetching, Infections.jsx to render the table and a service helper to retrieve the data from Firebase.
With the data loaded into the app I was able to start creating methods to parse the data to sum values for a particular day, compare the previous day from the current day and display these in a table.
I decided to use a sparkline to generate the 7 day history. Sparklines are small graphs with no labels or axis and can fit into small spaces. This gives a good idea of the rate of increase in the infections. The screenshot below you can quickly get an idea that the rate of increase of Lothian has been constant for the past seven days, whilst Shetland has been pretty level.
Before I was ready to release the application, there were a few things that I wanted to add to complete the look:
- Hero Image.
- Useful Information.
- Mobile Ready.
The purpose of the hero image is to show images from around Scotland (after all if we can't get out to enjoy Scotland, let's bring the Scotland to us). Using the unsplash API, I was able to randomly display images that are free to use from around Scotland. To make this more of an impact I made this 100% of the viewport height so that it is all the users sees when they view the site. A call to action button is displayed to direct the users to the data.
A section to the right of the application to remind users of how to stay safe, stay at home, save lives and save the pressure on the NHS. A list of symptoms, useful links and when the data is updated completes the section.
I am fully aware that most people might access this on their mobile phones and largely the site holds up when viewed on a smaller device. The biggest challenge I faced however was how to display the data table. Tables are normally presented horizontally to make it easy for users to scan and view data. However, they do not translate very well on small devices often resulting in scrolling across to view all the data. After a lot of searching I came across a nice pattern that turned each row in the table into effectively a single column table.
I managed it, I wasn't sure I would manage to get it completed in the weekend, but there you go it was possible. I decided not to track deaths, the sad truth is they will continue to go up and it is not in my scope to track the virus mortality rate. However what I did end up with was a learning on how to create a basic scraper, transform that raw data into something that can easily answer the questions I had and let other people get an easy view of the data also.
Please as always, follow the advice we are being given at the moment. It really is for our own good and please remember to check out the app here.