3. Error Accumulation

We want to let functions return more than one error. A classic use case is parsing, in which there might be multiple syntax and grammar errors in the input. Callers (and users) would be interested in all of them. Another example is a function that distributes work among threads, during which each thread could encounter different errors. Again, all of the errors can be important to the caller.

A common solution is to create a custom error type that internally stores multiple errors. Or, even more simply, the Err could just be a vector of errors.

But our requirement goes beyond mere multiplicity. In some cases callers might care only that the function succeeds, in which case it would be more efficient to fail on the first error, a.k.a. “fail fast”. We might also sometimes prefer to stream the errors instead of storing them all in memory. For example, consider that our parser might emit thousands of errors on a bad input. It would be more memory-efficient as well as more responsive to print them out as they arrive. In other words, we should be able to accumulate them into the terminal instead of into memory.

Problemo’s solution is the ProblemReceiver trait. If we want to store the errors then we can use the Problems type, which implements the trait by “swallowing” the errors into a vector. If we want to fail on the first error then we can use the FailFast type, which implements the trait by simply returning the error. Finally, we can use NoProblem to ignore all errors.

Problems also supports an optional list of “critical” error types: If it encounters one of these it fails fast instead of swallowing. Also, it is possible to de-dup a Problems (see the example).

If we need custom behavior then we can implement the trait on our own types. It has just one simple function, give(Problem).

Challenges

Although the ProblemReceiver trait is very simple, using it involves awkwardness and requires discipline. We believe it’s worth it for the flexibility and for providing opportunities for optimization.

The first challenge is that this is an inversion-of-control design, meaning that the caller has to provide the ProblemReceiver implementation. Commonly it’s just passed as an extra function argument.

A useful advantage of inversion-of-control is that because the caller owns the receiver it can be reused: A function can pass the same receiver reference along to other functions that it calls, and the caller can likewise call multiple functions with one receiver, finally handling all the accumulated errors at once.

The second challenge is that an error-accumulating function’s Result might actually be Ok even if there are errors. This is because they could have all been swallowed by the receiver. The first consequence is that such a function needs to be able to return something with Ok. This could be a partial result, which can be useful in itself. In our parser example we would be able to show the user what we succeeded in parsing in spite of the errors. In other cases it could be an empty collection or, if even that’s impossible or irrelevant, it could be a None, in which case we would have to make sure to return an Option for Ok. The second consequence is that upon getting an Ok the caller would still need to check for accumulated errors. Problems has a check() function that does just that.

Example:

use problemo::{*, common::*};

/// Error-accumulating functions have special signatures:
///   By our convention we put the receiver as the *last* argument;
///   its type is generic
/// And also special internal implementations:
///   Specifically we have to make sure to give() all errors to the receiver
///   in order to give it an opportunity to swallow them or to fail fast;
///   We provide a few friendly APIs to make this easier
fn read_files<ProblemsT>(
    paths: &[&str],
    problems: &mut ProblemsT,
) -> Result<Vec<String>, Problem>
where
    ProblemsT: ProblemReceiver,
{
    let mut strings = Vec::default();
    for path in paths {
        // give_ok() is like ok() but will give the problem to the receiver;
        // Note that we still use "?" in order to support a fast fail
        if let Some(string) = std::fs::read_to_string(path)
            .via(LowLevelError)
            .give_ok(problems)?
        {
            strings.push(string);
        }
    }

    // If we had swallowed errors then this would be a partial result
    // (i.e. not all files were read)
    Ok(strings)
}

fn main() -> Result<(), Problem> {
    let mut problems = Problems::default();
    let strings = read_files(&["non-existing1.txt", "non-existing2.txt"], &mut problems)?;
    
    // When using Problems the call above will *never* return Err
    // (it swallows all the errors)
    // Thus we thus *must* call check() here if we want to fail
    problems.check()?;

    // By contrast, we can trust that FailFast will *always* return Err on error
    let strings = read_files(&["non-existing3.txt", "non-existing4.txt"], &mut FailFast)?;
    ...
}