Describe and implement a simple fan-out, fan-in concurrency pattern in Go for parallelizing a CPU-bound task.
Go & Rust interview question for Advanced practice.
Answer
The fan-out, fan-in pattern is a pipeline for parallelizing work. Fan-out is the process of starting multiple goroutines to handle input from a single channel. Fan-in is the process of combining the results from multiple channels into a single channel. This example parallelizes the task of squaring numbers: go package main import ( "fmt" "sync" ) // Generates numbers and sends them to a channel. func generator(nums ...int) <-chan int { out := make(chan int) go func() { for , n := range nums { out <- n } close(out) }() return out } // A worker that reads from an input channel, squares the number, and sends to an output channel. func squarer(in <-chan int) <-chan int { out := make(chan int) go func() { for n := range in { out <- n n } close(out) }() return out } // Fan-in: Merges results from multiple channels into one. func merge(cs ...<-chan int) <-chan int { var wg sync.WaitGroup out := make(chan int) output := func(c <-chan int) { for n := range c { out <- n } wg.Done() } wg.Add(len(cs)) for , c := range cs { go output(c) } // Start a goroutine to close 'out' once all the output goroutines are done. go func() { wg.Wait() close(out) }() return out } func main() { // Stage 1: Generate input numbers. in := generator(2, 3, 4, 5) // Stage 2 (Fan-out): Start two workers to square the numbers. c1 := squarer(in) c2 := squarer(in) // Stage 3 (Fan-in): Merge the results from the workers. for n := range merge(c1, c2) { fmt.Println(n) } }
Explanation
The fan-out, fan-in pattern is a powerful and common concurrency pattern in Go used to parallelize work across multiple CPU cores, significantly speeding up computation.