The DRY principle

Don’t Repeat Yourself.

\[ \textrm{RMS}(x_1, \ldots, x_n) = \sqrt{\frac{1}{n} \sum_{i = 1}^n x_i^2} \]

rms1 = np.sqrt(np.mean(
  np.square(error1)
))
rms2 = np.sqrt(np.mean(
  np.square(error2)
))

def rms(x):
    '''Return the root mean square of
    the vector x'''
    return np.sqrt(np.mean(
      np.square(x)
    ))


rms1 = rms(error1)
rms2 = rms(error2)

Follow a style guide

Style guides list rules to follow when laying out your code.

You should start from a publicly available guide.

For R, you can follow the Tidyverse style guide [3].

For Python, the standard is called PEP 8 [4].

You can tweak the guidelines to suit you, but not following some style guide is like writing a paper with inconsistent grammar, or using two different citation styles in the same document (Bertolacci, 2020).

R style example

y <- x*2
if(y > 2) {
  z <- y + 2
}
sqrt_z <- sqrt (z)

y <- x * 2
if (y > 2) {
  z <- y + 2
}
sqrt_z <- sqrt(z)

The R package lintr [5] can check your code automatically:

example.R:1:7: style: Put spaces around all infix operators.
y <- x*2
     ~^~
example.R:2:3: style: Place a space before left parenthesis, except in a function call.
if(y > 2) {
  ^
example.R:5:11: style: Remove spaces before the left parenthesis in a function call.
sqrt_z <- sqrt (z)

Python style example

y = x*2
if y> 2:
   z = y + 2
sqrt_z = sqrt (z)

y = x * 2
if y > 2:
    z = y + 2
sqrt_z = sqrt(z)

The program pycodestyle [6] checks if you are following the rules:

example.py:2:5: E225 missing whitespace around operator
example.py:3:4: E111 indentation is not a multiple of four
example.py:4:14: E211 whitespace before '('

Sometimes you knew what the code does when you wrote it, but a few months later…

loc<-c(137.5, -4.7)
loc2<-c(77.5, 18.1)
dat<-read.csv('locations.csv')
d=2*3389.5*asin(sqrt(
sin((dat$lat*pi/180-loc[2]*pi/180)/2)^2
+cos(dat$lat*pi/180)*cos(loc[2]*pi/180)
*sin((dat$lon*pi/180-loc[1]*pi/180)/2)^2))
d2=2*3389.5*asin(sqrt(
sin((dat$lat*pi/180-loc2[2]*pi/180)/2)^2
+cos(dat$lat*pi/180)*cos(loc2[2]*pi/180)
*sin((dat$lon*pi/180-loc2[1]*pi/180)/2)^2))
#dat$nrst=-1
dat$nrst=0
for(i in 1:nrow(dat)){
if(d[i]<d2[i])
dat$nrst[i]<-1
else
dat$nrst[i]<-2
}

Comments

A rule of thumb: Comments in code should say what your code does, not explain how.

mars_haversine_dist <- function(
  locations,
  origin
) {
  # Take two times the radius times
  # the arcsine of the square root
  # of ...
  ...
}

# Calculate the distance in
# kilometres between two coordinates
# on Mars
mars_haversine_dist <- function(
  locations,
  origin
) {
  ...
}

The code should be clean enough that the how is obvious.

(I break this rule around 10% of the time.)

Long functions

A rule of thumb: functions should fit on one or two screenfuls.

The maximum length of a function is inversely proportional to the complexity and indentation level of that function. So, if you have a conceptually simple function that is just one long (but simple) case-statement, where you have to do lots of small things for a lot of different cases, it’s OK to have a longer function.
- Linus Torvalds [7]

(I break this rule around 10% of the time)

Long files

You should also organise your code into multiple well-named files.

A rule of thumb: code files should not be too long, say no longer than 1000-2000 lines.

(I break this rule less often.)

More clean code tips

• Code for correctness and readability first.
• Shorter isn’t necessarily better—short programs don’t run any faster!
• Tab completion in your editor means you don’t have to type longer function and variable names every time.
• Picking good names for things is hard, but it’s worth the effort.

Discussion on clean code 🗣️

Questions? Comments? Disagreements?

Feel free to add your own clean code tips to Slack!  

Or raise your hand in the Zoom chat. ✋

A tale of two errors…

n <- length(y)
sigma <- 1
log_likelihood <- (
  - n * log(2 * pi) / 2
  - n * log(sigma)
  - sum(y ^ 2 / sigma ^ 2) / 2
)
bic <- log(n) - 2 * log_likelihood
print(bic)

## [1] NA

dat <- data.frame(x = 1 : 10)
dat$y <- sin(df$x)
fit <- lm(y ~ x, data = dat)
print(coef(fit))
plot(dat$x, fitted(dat$y))

## Error in df$x: object of type
##   'closure' is not subsettable

Dead programs tell no lies

Defensive programming can be used when you’re aware you’re making an assumption. The idea is to intentionally throw an error when an assumption found to be false.

Most languages have functions to help you do this.

This way, you don’t have to search for the offending line.

Example: the exponential covariance function,

\[ C(\mathbf{x}, \mathbf{x}') \equiv \exp\left( -\frac{||\mathbf{x} - \mathbf{x}'||}{l} \right) \]

where \(l > 0\).

In R, you can use the stopifnot function:

# Given a vector or matrix x, return the covariance matrix based on the
# exponential covariance function
cov_exponential <- function(x, length_scale) {
  stopifnot(length_scale > 0)
  exp(-fields::rdist(x) / length_scale)
}

cov_exponential(y, -1)

## Error in cov_exponential(y, -1): length_scale > 0 is not TRUE

In Python, you can use the assert keyword:

import numpy as np

def cov_exponential(x, length_scale):
    '''Given a vector or matrix x, return the covariance matrix based on the
    exponential covariance function'''
    assert length_scale > 0, 'length_scale must be positive'
    return np.exp(-np.linalg.norm(x) / length_scale)

cov_exponential(y, -1)

Traceback (most recent call last):
  File "example.py", line 7, in <module>
    cov_exponential(y, -1)
  File "example.py", line 4, in cov_exponential
    assert length_scale > 0, 'length_scale must be positive'
AssertionError: length_scale must be positive

Back to the tale of two errors

stopifnot(all(!is.na(y)))
n <- length(y)
sigma <- 1
log_likelihood <- (
  - n * log(2 * pi) / 2
  - n * log(sigma)
  - sum(y ^ 2 / sigma ^ 2) / 2
)
bic <- log(n) - 2 * log_likelihood
print(bic)

## Error: all(!is.na(y)) is not TRUE

More defensive programming tips

• You won’t always know what assumptions you’re making implicitly immediately.
• But! when your code breaks, add a stopifnot or assert to catch it next time.
• Don’t feel the need to check everything, just the most important parts.
• In R, the assertthat package [9] can help.

Discussion on defensive programming 🗣️

Questions? Comments? Disagreements?

Feel free to add your own defensive programming tips to Slack!  

Or raise your hand in the Zoom chat. ✋

(But note that we will talk about a related topic, automated testing, very soon.)

def add_one(x):
  '''Returns the argument, plus one'''
  return x + 1.1

from add_one import add_one

def test_add_one():
  assert add_one(1) == 2

Running the tests:

======================= test session starts ========================
collected 1 item

test_add_one.py F                                            [100%]
============================= FAILURES =============================
___________________________ test_add_one ___________________________

    def test_add_one():
>     assert add_one(1) == 2
E     assert 2.1 == 2
E      +  where 2.1 = add_one(1)

test_add_one.py:4: AssertionError
===================== short test summary info ======================
FAILED test_add_one.py::test_add_one - assert 2.1 == 2
=================== 1 failed, 0 passed in 0.15s ====================

What makes for a good set of tests?

• Each test tests one thing at a time
• Tests are short and simple; don’t test too many things
• Boundary conditions are tested
• All code lines are explored

def max_one(x):
  '''Return the maximum between the argument and one'''
  if x < 1:
    return 1
  else:
    return x

def test_max_one():
  assert max_one(0) == 1  # First code path
  assert max_one(1) == 1  # Boundary condition
  assert max_one(2) == 2  # Second code path

covariance.py:

import numpy as np

def cov_exponential(x, length_scale):
    '''Given a vector or matrix x, return the covariance matrix based on the
    exponential covariance function'''
    assert length_scale > 0, 'length_scale must be positive'
    return np.exp(-np.linalg.norm(x) / length_scale)

test_covariance.py:

import pytest
import numpy as np

from covariance import cov_exponential

def test_cov_exponential():
    assert cov_exponential([0, 0], 1) == 1.0
    assert cov_exponential([0, 1], 1) == np.exp(-1)
    assert cov_exponential([0, 1], 2) == np.exp(-1 / 2)
    with pytest.raises(AssertionError):
        cov_exponential([0, 0], -1)

Testing in Python and R

The preceding Python examples use the pytest framework [10]. Tests go into files with names starting with “test_”

In R, you can use the testthat package [11]

add_one <- function(x) {
  x + 1.1
}

source('add_one.R')
context('add_one')

test_that('adding one to one makes two', {
  expect_equal(add_one(1), 2)
})

Running it in an R session:

> testthat::test_file('test_add_one.R')

══ Testing test_add_one.R ═════════════════════════════════════════════════
[ FAIL 1 | WARN 0 | SKIP 0 | PASS 0 ]

── Failure (test_add_one.R:8:3): adding one to one makes two ──────────────
add_one(1) not equal to 2.
1/1 mismatches
[1] 2.1 - 2 == 0.1

[ FAIL 1 | WARN 0 | SKIP 0 | PASS 0 ]

Testing an R package

In R you can also test a package. Here are the tests for one of mine, WoodburyMatrix [12]:

> devtools::test('WoodburyMatrix')

Testing WoodburyMatrix
✓ |  OK F W S | Context
✓ |  89       | WoodburyMatrix [0.7 s]
✓ |   1       | lints [3.9 s]
✓ |   9       | normal distribution

══ Results ═══════════════════════════════════════════════════════════════
Duration: 4.8 s

[ FAIL 0 | WARN 0 | SKIP 0 | PASS 99 ]

Here there are 99 separate checks.

A guide to setting this up is in the R packages book [13].

Testing and version control

You may have seen these badges on repositories on GitHub:

Tests are automatically run when changes to the software are published. The code coverage metric counts how many lines of code in the software are tested.

This is beyond what most small research projects need, but it’s nice to have!

How to make the most of automated testing

A few thoughts on automated testing:

• Automated testing is helpful when your code is already organised into functions (or packages).
• If a function is hard to test, it might be too complicated anyway. Break it up into smaller functions.
• If you find a bug, write a test for the bug that fails, THEN fix the bug