Logging

Producer vs. Consumer

In an application, you will typically have many log producers and a single log consumer.

The log producers are where you issue log statements, e.g.: log.info("Everything working!")
These log statements can typically be issued anywhere, whether in a library or in your application’s code itself.
The programming language and logging framework transports the log statements to the log consumer.

The log consumer has to be initialized at the start of the application’s main logic.
It then has to make the received log statements available to one or more external systems, so that they can be viewed by users.

Rust

Topics surrounding the programming language Rust.

macro_rules!

What is it?

macro_rules! is a way of writing macros in Rust. Macros produce code as output, typically derived from code they receive as input.

macro_rules! in particular offers a declarative style for defining these macros.
The output code has fixed placeholders into which input is templated.
And you can select different output templates by matching on the structure of the input.

Here is an example of a macro_rules! definition with two output templates.
One of them has a placeholder $name, which accepts an expression.

macro_rules! hello {
    () => {
        println!("Hello, World!");
    };
    ($name:expr) => {
        println!("Hello, {}!", $name);
    };
}

fn main() {
    hello!();        //equivalent to `println!("Hello, World!");`
    hello!("Alice"); //equivalent to `println!("Hello, Alice!");`
}

macro_rules! are less powerful than the procedural proc_macros, but often simpler to read and write.
Mind that function-like proc_macros also use the exclamation mark syntax!(), so not everything with an exclamation mark is macro_rules!.

What is macro_rules! useful for?

• Reducing boilerplate
  macro_rules! can be used for deduplicating code, similar to functions. But unlike functions, it can be used when the code defines types, modules or similar. Reducing boilerplate is what macro_rules! are most commonly used for.
  Examples: vec![], dbg!(), uuid!(), shadow!()
  The standard library also makes heavy use of macro_rules! for declaring types with small variations, like for example integers.

• Simple domain-specific languages
  You can match on fixed words, symbols or even formats. This way, you can create a small syntax to configure even complex output templates.
  Examples: clap::arg!(), format!()

• Structural validation
  macro_rules! allows you to reject inputs that don’t match a certain format, meaning you can verify the user made no mistake when entering them. However, you cannot use a traditional if-else in macro_rules! (without generating it into the output), meaning you likely want proc_macros for more complex cases.
  Examples: json!(), toml! { }

• Variadic interfaces
  Unlike functions, macro_rules! can accept a variable number of parameters. This is useful for libraries in particular, where it can allow you to build a cleaner interface, especially when you combine it with the points above.
  Examples: vec![], format!()

How to write macro_rules!

For an in-depth explanation, you can read the page in the Rust Book. For simple macros, however, you can often figure out how to write them from looking at examples.

A good explanation for the different kinds of placeholders can be found in The Little Book of Rust Macros.

Tips

• To split up your logic, you can generate code which calls further macros. This can also be used for recursion.

• Macros are evaluated early during compilation.
  As such, you cannot match on the value or type of a variable, unless you generate code which does this at runtime. On the other hand, this early evaluation allows generating type definitions and modules.

• IDEs or text editors with LSP integration can typically ‘expand’ macros, which shows you the output code that will get templated.

• You cannot declare whether your macro is called with parentheses (), brackets [] or braces {}. All of them are allowed. For example, these calls are perfectly legal:
  vec!(1, 2, 3)
vec! { 1, 2, 3 }

• macro_rules! visibility is somewhat unusual:

  • Within the module where the macro is defined, it is only visible after the definition.
  • To use a macro in other modules, you need either pub(crate) use my_macro; below the macro definition or you can annotate it with #[macro_export].
    The #[macro_export] annotation makes it available at the module path crate::my_macro.
  • To use a macro in other crates, you have to annotate it with #[macro_export].

macro_rules! in a library

• When your macro is called from another crate, the crate keyword will refer to that crate (since that’s where the code is templated into). If you need to refer to elements in your own crate, use the $crate variable instead.

• You want to avoid using imports in the generated code, as they will affect the user’s code, too. Instead, reference elements with their absolute module path.
  You should also make it explicit that it is an absolute path by prepending a double-colon, e.g. ::std::path::Path. If you use just std::path::Path instead, a user having a module std in scope would prevent your macro from working.

• If your macro generates items which don’t have to be used, you should mark them as #[allow(unused)].

• If you need to generate a module, which isn’t intended to be accessed by the user, use a name which is unlikely to collide with the user’s modules, like for example __my_library_macro.

• You can expose format!()-like string formatting by calling format_args!().

• You can reject input with an explicit message by generating a compile_error!().

• Documentation comments can be templated with placeholders by using the #[doc] attribute.
  The normal /// syntax does not work, because the $placeholder syntax will be deemed part of the comment text.

• trybuild can be used for testing, whether your macro correctly rejects inputs.

Closures and Function Pointers

Rust knows two kinds of first-class functions:

• Function pointers: Provide just the information which function to call.
• Closures: Additionally allow referencing (“capturing”) variables from outside the scope of where the closure is defined.

Basic syntax example:

#![allow(unused)]
fn main() {
let greeting = "Hello"; //will be captured

let my_closure = |name| {
    format!("{greeting}, {name}!")
};

let hello_world = my_closure("World");
}

Capturing can be thought of as a second point in time, when parameters can be passed into a closure, meaning ownership and borrowing rules still apply.

How the variables are captured (&, &mut, with ownership) changes how the closure or function pointer can be called, which is reflected in the type:

fn (function pointer)

#![allow(unused)]
fn main() {
fn my_function(name: String) -> String { //nothing is captured, only passed in as parameter when invoked
    format!("Hello, {name}")
}
takes_function_pointer(my_function);

fn takes_function_pointer(function_pointer: fn(String) -> String) {
    function_pointer(String::from("World"));
}


// Can also be defined anonymously:
takes_function_pointer(|name| {
    format!("Hello, {name}")
});
}

API-Example: n/a

Fn closure

#![allow(unused)]
fn main() {
let greeting = "Hello"; //captured as &T
let my_closure = |name| {
    format!("{greeting}, {name}")
};
takes_Fn_closure(my_closure);

fn takes_Fn_closure(closure: impl Fn(String) -> String) {
    closure(String::from("World"));
}
}

Cannot move closure outside the scope of captured variables:

#![allow(unused)]
fn main() {
/// Does not compile!

let my_closure = {
    let greeting = "Hello"; //captured as &T
    || {
        println!("{greeting}")
    }
}; // `greeting` goes out of scope here, when `closure` still has a reference on it
}

API-Example: rayon::iter::ParallelIterator::map

FnMut closure

#![allow(unused)]
fn main() {
let mut greeting = String::from("Hello"); //captured as &mut T

takes_FnMut_closure(|| {
    greeting.push_str(", World");
});

fn takes_FnMut_closure(mut closure: impl FnMut()) {
    closure();
    closure(); //can call sequentially, but not in parallel (only one &mut can exist at a time)
}
}

API-Example: Iterator::map

FnOnce closure

#![allow(unused)]
fn main() {
let greeting = String::from("Hello"); //captured as (owned) T
let closure = || {
    std::mem::drop(greeting);
};
takes_FnOnce_closure(closure);


fn takes_FnOnce_closure(closure: impl FnOnce()) {
    closure();
    // closure(); //can only call once
}
}

API-Example: thread::spawn

Overview

ℹ️ There is an inverse relationship between callee freedom and caller freedom.

Tips

• When designing an API, start out by accepting an impl FnOnce. If you need to call the closure more than once, then go for impl FnMut and so on.
  The hierarchy is: fn : Fn : FnMut : FnOnce
  This means FnOnce is a super-trait to all closures and function pointers.

• The type of a closure can be annotated like a normal function:

  #![allow(unused)]
  fn main() {
  let closure = |input: u8| -> String {
      todo!()
  };
  }

  This is typically only necessary when type inference cannot decide a type.

• move can be used force capturing via ownership. This severs lifetimes, making the closure easier to work with when storing in a struct, when executing it in a new thread or when moving to a different scope:

  #![allow(unused)]
  fn main() {
  /// Does compile, thanks to `move`!
  let closure = {
      let greeting = "Hello"; //captured as T
      move || {
          println!("{greeting}")
      }
  }; // `greeting` was moved into the closure, so does not go out of scope
  }

• If you need to pass a value into multiple closures, it can be useful to wrap the value into an Arc<_> (with a Mutex<_> or RwLock<_> inside, if you need mutability). This makes the reference behave like in a garbage-collected language, where it can remain in the closure indefinitely, without handling lifetimes.

Async Closures

async closures can be used to run async code in a closure:

#![allow(unused)]
fn main() {
let mut list = vec![]; //will be captured

let closure = async || {
    list.push(
        std::future::ready(String::from("Hello")).await
    );
};
}

These differ from returning an async block from a normal closure (e.g. || async {}) in that execution of the closure and the async code happens at the same time. This allows accessing captured values within the async code without cloning.
Previously, you would have had to use an Arc<Mutex<_>> to make this specific code example work, since it can both be cloned and allows mutating the value inside:

#![allow(unused)]
fn main() {
use std::sync::{Arc, Mutex};
let list = Arc::new(Mutex::new(vec![])); //will be captured

let closure = || async { //without async closure
    list.lock().unwrap()
        .push(
            std::future::ready(String::from("Hello")).await
        );
};
}

See also:

• Rust Book introduction chapter
• Rust Book advanced chapter
• Discussion on Fn vs. FnMut vs. FnOnce
• How the closure syntax de-sugars