Goroutines are cheap, but they're not free. The same features that make Go's concurrency model approachable — lightweight goroutines, buffered channels, and the select statement — also create a landscape where subtle mistakes can silently corrupt data, leak resources, or bring your application to a halt. Teams new to Go often embrace goroutines enthusiastically, only to discover that debugging a deadlock at 2 AM is not the rite of passage they hoped for.
This guide is for developers who already know how to start a goroutine and write a channel operation. We're going beyond the basics. We'll walk through the most common concurrency pitfalls we've seen in production Go systems: data races that evade detection, goroutine leaks that slowly consume memory, channel deadlocks that freeze pipelines, and cancellation patterns that leave work stranded. For each pitfall we'll explain why it happens, how to spot it, and — most importantly — how to design your code to avoid it from the start.
By the end, you'll have a mental checklist for reviewing concurrent Go code, a set of reliable patterns for common concurrency problems, and a healthier respect for the go keyword. Let's begin.
1. The Hidden Cost of Lightweight Threads: When Cheap Goroutines Become Expensive Leaks
Goroutines are often described as 'lightweight threads' because they start with only a few kilobytes of stack space and can be created in the hundreds of thousands without exhausting memory. That's true — but it's also dangerously misleading. The real cost of a goroutine isn't just its initial stack; it's the resources it may hold onto for its entire lifetime, and the fact that once started, it runs until it either returns or the program exits.
Consider a simple HTTP server that spawns a goroutine for every incoming request to perform some background work:
func handler(w http.ResponseWriter, r *http.Request) {
go doWork(r.Context(), r.Body)
w.WriteHeader(http.StatusAccepted)
}This looks innocuous. But what happens if doWork blocks on a channel that never receives a value? Or if it enters an infinite loop due to a logic error? That goroutine will never exit. Over time, these 'leaked' goroutines accumulate. Each one holds onto its stack, any heap-allocated data it references, and — crucially — the http.Request.Body which may keep a TCP connection open. The server's memory grows until it hits the OOM killer, or the number of open file descriptors exhausts the OS limit.
How to detect goroutine leaks
The Go runtime provides a built-in tool: runtime.NumGoroutine(). A common practice is to expose the current goroutine count via a debug endpoint or log it periodically. If the count grows monotonically under steady load, you have a leak. More advanced tools like pprof can give you a stack trace of every goroutine, helping you identify which ones are stuck and where they were created.
But detection is reactive. The real fix is to ensure every goroutine you start has a guaranteed exit path. That means:
- Always use a context with a timeout or deadline for goroutines that perform I/O or waiting operations.
- Never start a goroutine without a plan for how it will stop. If you can't articulate the exit condition, don't start it.
- Use errgroups or sync.WaitGroup to track goroutine lifetimes and ensure they complete before the program shuts down.
One team we worked with had a background worker that processed a job queue. The worker ran in a forever loop with a for { select { ... } }. The problem was that when the queue was empty, the goroutine would block on a channel receive with no cancellation path. Over six months, the worker count grew from 10 to over 5,000, and the service started failing with 'too many open files'. The fix was to add a context with a timeout to the channel receive, so the goroutine would wake up periodically, check if it should exit, and then retry.
The lesson: goroutines are not fire-and-forget. They demand lifecycle management just like any other resource.
2. Data Races: The Silent Corruptor That Testing Often Misses
Go's memory model is well-defined, but it doesn't protect you from yourself. A data race occurs when two goroutines access the same variable concurrently, and at least one of the accesses is a write. The result is undefined behavior — your program may read a stale value, see a partially written value, or crash. The worst part? Data races are notoriously hard to reproduce because they depend on specific interleavings of goroutine execution.
Consider this seemingly safe code:
var counter int
func increment() {
counter++
}
// called from multiple goroutines
for i := 0; i < 1000; i++ {
go increment()
}The counter++ operation is not atomic. It compiles to three steps: load, increment, store. If two goroutines execute these steps interleaved, the final value may be less than 1000. This is a classic data race.
Why the race detector is your best friend
Go ships with a built-in race detector that you enable with the -race flag during go test or go run. The detector instruments memory accesses and reports any concurrent unsynchronized access. It's incredibly effective — but only if you run it. Many teams skip the race flag in CI because it slows down tests by 2–20x. That's a mistake. A single data race caught in development saves hours of debugging in production.
However, the race detector has limitations. It only detects races that actually happen during execution. If your test suite doesn't exercise the exact interleaving that triggers the race, the detector stays silent. That's why you need to combine the race detector with deliberate stress testing — run your tests with GOMAXPROCS set to a high value (e.g., 4 or 8) and loop tests multiple times to increase the chance of hitting a race.
Common patterns that hide races
Even with the race detector, some patterns are race-prone:
- Using a map without synchronization — Go maps are not safe for concurrent writes. Even if you only write from one goroutine and read from another, that's a race. Use
sync.Mapor a mutex. - Returning a pointer to a local variable that is then accessed from another goroutine after the function returns — the variable is still on the stack? Actually, Go escapes such variables to the heap, but the access pattern may still race.
- Closing a channel from one goroutine while another is sending — this is a panic, not a race, but it's a related synchronization error.
The safest approach is to adopt a policy: every shared mutable state must be protected by a mutex, a channel, or an atomic operation. No exceptions. If you find yourself thinking 'this variable is only written once during initialization, so it's safe to read concurrently' — think again. Unless you use sync.Once or an atomic store/load, the compiler and CPU can reorder memory accesses in ways that break your assumption.
In one incident, a team had a configuration struct that was loaded at startup and then read by many goroutines. One goroutine updated a field in response to a signal. The update was done without a mutex because 'it's just one field'. The race detector caught it during a stress test — the reading goroutines saw a partially written struct. The fix was a simple sync.RWMutex: writers take a write lock, readers take a read lock. The performance impact was negligible.
3. Channel Misuse: Buffered Channels Are Not a Queue
Channels are Go's primary communication primitive, but they are often misunderstood. A common mistake is to treat a buffered channel as a thread-safe queue, sending values from multiple producers and receiving from multiple consumers without any additional coordination. While this can work in simple cases, it quickly breaks down when you need backpressure, cancellation, or graceful shutdown.
Consider a worker pool pattern:
jobs := make(chan Job, 100)
// producer
for _, job := range jobList {
jobs <- job
}
close(jobs)
// consumers
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for job := range jobs {
process(job)
}
}()
}
wg.Wait()This pattern works until the producer blocks because the buffer is full. If the consumers are slower than the producer, the channel fills up, and the producer hangs. If the producer is a goroutine that also holds a lock needed by a consumer, you have a deadlock. The buffer size of 100 is arbitrary — it doesn't guarantee that the producer won't block; it just delays the problem.
When to use buffered vs unbuffered channels
Unbuffered channels synchronize the sender and receiver: a send blocks until a receive happens. This is useful for signaling and ensures that the producer doesn't get ahead of the consumer. Buffered channels decouple the two, but they introduce a finite buffer that can overflow. The general rule: use unbuffered channels for coordination and signaling; use buffered channels only when you have a clear understanding of the maximum buffer size and a plan for when it's full.
If you need a bounded queue with backpressure, consider using a buffered channel with a select that includes a default case to drop or reject jobs when the buffer is full:
select {
case jobs <- job:
// job queued
default:
// buffer full, handle backpressure (e.g., reject, log, retry later)
}This pattern gives the caller immediate feedback instead of blocking indefinitely. Alternatively, use a library like golang.org/x/sync/errgroup with a bounded goroutine pool that controls concurrency directly.
Closing channels: who is responsible?
Another channel pitfall is the 'who closes the channel' problem. In Go, only the sender should close a channel. If a receiver closes it, the sender will panic when it tries to send. The idiomatic solution is to use a dedicated sender goroutine that closes the channel after all sends are done, or to use a separate 'done' channel for signaling shutdown. The for range loop over a channel is elegant, but it only works if the channel is eventually closed. If the sender never closes (e.g., because it's waiting for an event that never comes), the receiver leaks.
In one project, a team used a channel to stream log entries from multiple producers to a single consumer. The consumer ranged over the channel, but the producers never closed it because they were supposed to run forever. The consumer goroutine leaked because it never exited. The fix was to use a context with cancellation: the consumer selects on both the log channel and a context.Done channel, so it can exit cleanly when the context is cancelled, even if the log channel is never closed.
Remember: channels are great for communicating, but they are not a panacea. Pair them with contexts, select statements, and clear ownership rules to avoid subtle bugs.
4. Deadlocks and Livelocks: When Synchronization Backfires
Deadlocks are the classic concurrency nightmare: two or more goroutines each waiting for the other to release a resource, resulting in a permanent standstill. Go's runtime can detect deadlocks when all goroutines are blocked, but it can't detect 'partial' deadlocks where some goroutines are still running. Livelocks are even trickier — goroutines are actively executing but making no progress because they keep reacting to each other's state.
A classic deadlock example
var mu1, mu2 sync.Mutex
func f1() {
mu1.Lock()
time.Sleep(10 * time.Millisecond) // simulate work
mu2.Lock()
// ...
mu2.Unlock()
mu1.Unlock()
}
func f2() {
mu2.Lock()
time.Sleep(10 * time.Millisecond)
mu1.Lock()
// ...
mu1.Unlock()
mu2.Unlock()
}If f1 and f2 run concurrently, they can deadlock: f1 holds mu1 and waits for mu2, while f2 holds mu2 and waits for mu1. The fix is to always acquire locks in the same order across all goroutines. In this case, both functions should lock mu1 then mu2.
More subtle deadlocks with channels
Channel-based deadlocks are common in pipeline patterns. For example:
func main() {
ch := make(chan int)
ch <- 1 // blocks forever because no receiver exists yet
go func() {
val := <-ch
fmt.Println(val)
}()
}The main goroutine blocks on the send before the receiver is ready. The fix is to start the receiver goroutine before sending, or use a buffered channel. More generally, when building pipelines, ensure that each stage is connected before data starts flowing. Use sync.WaitGroup to coordinate startup order, or use an unbuffered channel with the receiver already started.
Livelocks: the illusion of progress
Livelocks occur when goroutines are not blocked but are constantly changing state in response to each other, preventing any real work. A classic example is two goroutines trying to acquire two resources, but when they fail, they release what they have and retry — leading to an infinite cycle of acquire-release. Livelocks are harder to detect because the goroutines are running and not obviously stuck. The solution is to introduce randomness (backoff) or a deterministic ordering to break the cycle.
One team experienced a livelock in a distributed lock manager: when a goroutine failed to acquire a lock, it would release its current locks and retry immediately. Under high contention, all goroutines kept releasing and retrying, never making progress. The fix was to add exponential backoff with jitter before retrying.
To avoid deadlocks and livelocks, follow these principles:
- Always acquire multiple locks in a consistent order — document the order if it's not obvious.
- Use timeouts — a
selectwith atime.Aftercan prevent indefinite blocking. - Minimize the scope of locks — hold locks for as short a time as possible.
- Consider lock-free alternatives — sometimes a channel-based design or atomic operations can eliminate the need for mutexes.
5. Context Cancellation: The Pattern Everyone Gets Wrong
Go's context package is the standard way to propagate cancellation signals and deadlines across API boundaries and goroutine trees. But it's easy to misuse. The most common mistakes are: not passing a context at all, ignoring the Done() channel, and creating a context tree that doesn't propagate cancellation properly.
Mistake: Starting a goroutine without a context
If you start a goroutine that performs a blocking operation (e.g., a database call, an HTTP request, a channel receive), and you don't provide a context with a timeout or cancellation, that goroutine may block forever. Always accept a context.Context as the first parameter of any function that starts a goroutine or performs a blocking operation. This is a Go convention for a reason.
Mistake: Ignoring ctx.Done()
Even if you pass a context, you must actually check ctx.Done() in your goroutine. A common pattern is to use select to listen on both the work channel and ctx.Done():
for {
select {
case job <- jobs:
process(job)
case <-ctx.Done():
return ctx.Err()
}
}Without the ctx.Done() case, the goroutine will keep trying to receive from jobs even after the context is cancelled. The jobs channel may never be closed, so the goroutine leaks.
Mistake: Creating a derived context without a reason
Every time you call context.WithCancel, context.WithTimeout, or context.WithDeadline, you create a new context that is a child of the parent. If you forget to call the cancel function (the second return value), the parent context's cancellation won't fully propagate because the child context tree is still alive. This is a common source of resource leaks. Always defer the cancel function:
ctx, cancel := context.WithTimeout(parentCtx, 5*time.Second)
defer cancel()Even if you don't intend to cancel early, deferring cancel ensures that the context is cleaned up when the function returns.
Composite scenario: a cancellable worker pool
Let's put it together. Suppose you have a worker pool that processes jobs from a channel. You want to cancel the pool when a signal is received. The correct pattern is:
func RunPool(ctx context.Context, jobs <-chan Job) error {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for {
select {
case job, ok := <-jobs:
if !ok {
return // channel closed
}
process(ctx, job)
case <-ctx.Done():
return // cancelled
}
}
}()
}
wg.Wait()
return ctx.Err()
}Notice that each worker checks both the jobs channel and ctx.Done(). The process function also receives a context so it can respect cancellation. The pool returns the context error if it was cancelled. This pattern is robust and handles graceful shutdown.
Contexts are not optional in concurrent Go code. They are the backbone of cancellation and timeouts. Treat them with the same care as error handling.
6. When Not to Use Goroutines: Over-Concurrency and the Cost of Context Switching
Goroutines are cheap, but they are not free. Starting a goroutine has a cost: stack allocation, scheduling overhead, and context switching. If you create a goroutine for every tiny task, you can actually degrade performance. This is especially true for CPU-bound tasks where the overhead of scheduling outweighs the benefit of parallelism.
The myth of infinite goroutines
It's tempting to spawn a goroutine for each item in a loop because it's easy:
for _, item := range items {
go process(item)
}If items has 100,000 entries, you now have 100,000 goroutines competing for CPU time. The Go scheduler will handle them, but the overhead of managing that many goroutines — especially if process is short-lived — can be significant. You'll see high memory usage and poor cache locality. Worse, if process is I/O-bound, you might be fine, but if it's CPU-bound, you'll get worse throughput than using a bounded worker pool.
When to use a worker pool
A worker pool limits the number of concurrent goroutines to a fixed size. This is appropriate when:
- The task is CPU-bound and you want to match the number of CPU cores.
- The task is I/O-bound but the external resource (e.g., database connections) has a limit.
- You need backpressure to prevent the system from being overwhelmed.
Go's errgroup library (golang.org/x/sync/errgroup) provides a convenient way to run a bounded set of goroutines with error propagation. Alternatively, you can implement a simple pool using a buffered channel as a semaphore:
sem := make(chan struct{}, 10) // limit to 10 concurrent goroutines
for _, item := range items {
sem <- struct{}{} // acquire
go func(item Item) {
defer func() { <-sem }() // release
process(item)
}(item)
}
// wait for all to finish
for i := 0; i < cap(sem); i++ {
sem <- struct{}{}
}When goroutines are the wrong tool
Sometimes, the problem doesn't need concurrency at all. If you have a simple sequential pipeline that processes data in stages, using goroutines and channels adds complexity without benefit. Measure before you optimize. A single-threaded solution that is clear and correct is often better than a concurrent one that is buggy and hard to debug.
Also, avoid goroutines for tasks that are inherently sequential, like reading a file line by line. The overhead of synchronizing access to the file will likely negate any parallelism gains.
As a rule of thumb: use goroutines when you have independent units of work that can execute concurrently, and when the cost of coordination is less than the benefit of parallelism. For everything else, keep it simple.
7. Frequently Asked Questions About Go Concurrency Pitfalls
Q: How do I choose between a mutex and a channel?
Channels are for communicating between goroutines; mutexes are for protecting shared state. If you need to pass data from one goroutine to another, use a channel. If you need to protect a shared variable from concurrent access, use a mutex. There is overlap — you can use a channel to implement a mutex-like pattern (e.g., a channel of size 1 as a token), but it's usually clearer to use the appropriate primitive. When in doubt, start with a mutex for shared state and a channel for signaling.
Q: My race detector doesn't catch anything, but I still see inconsistent behavior. What should I do?
The race detector only catches races that actually happen during execution. To increase coverage, run your tests with -race -count=10 and set GOMAXPROCS to a value higher than 1 (e.g., 4). Also, write stress tests that simulate high concurrency. If you still can't reproduce, consider using the go test -fuzz mode with the race detector enabled — fuzzing can generate inputs that trigger rare interleavings.
Q: Is it safe to use sync.Map instead of a mutex-protected map?
sync.Map is optimized for two specific access patterns: (1) when the key is written once and read many times, and (2) when multiple goroutines read, write, and overwrite disjoint sets of keys. For other patterns, a regular map with a mutex is often faster and simpler. sync.Map has higher overhead for mixed read-write workloads. Benchmark your specific use case.
Q: How do I gracefully shut down a goroutine that is blocked on a network call?
Use a context with a timeout. Most Go network libraries (net/http, database/sql, etc.) accept a context and will abort the operation when the context is cancelled. If you're using a library that doesn't support context, you may need to close the underlying connection from another goroutine (e.g., close the TCP socket) to unblock the call. This is a last resort.
Q: What's the best way to propagate errors from goroutines back to the caller?
The errgroup package is the standard solution. It collects the first error from any goroutine and cancels the rest. Alternatively, you can use a channel to send errors back to a collector goroutine. Avoid using a shared error variable with a mutex — it's error-prone and doesn't compose well.
Q: My goroutine leaks even though I have a context with a timeout. Why?
Make sure you are checking ctx.Done() in your goroutine's main loop. A common mistake is to pass the context to a function but not select on ctx.Done() in the goroutine itself. Also, ensure that the cancel function is called (deferred) in the parent goroutine that created the context. If the parent forgets to call cancel, the child context tree remains alive, preventing garbage collection.
8. Practical Takeaways: Your Concurrency Review Checklist
We've covered a lot of ground. Here's a condensed checklist you can use when reviewing your own concurrent Go code or during code reviews:
- Every goroutine must have a guaranteed exit path. Can it be cancelled? Does it have a timeout? Does it stop when a channel is closed?
- All shared mutable state is protected by a mutex, a channel, or an atomic operation. No exceptions. Run the race detector in CI.
- Channels have clear ownership. Only the sender closes the channel. Use a separate 'done' channel or context for shutdown signaling.
- Locks are acquired in a consistent order to prevent deadlocks. Document the order.
- Contexts are passed as the first parameter and are always checked in blocking operations. Defer the cancel function.
- Limit concurrency with worker pools when tasks are CPU-bound or when external resources have limits.
- Test with the race detector enabled and use stress tests to increase interleaving coverage.
- Measure before optimizing. If concurrency doesn't improve throughput or latency, keep it simple.
Concurrency in Go is a superpower, but with great power comes great responsibility. The patterns we've discussed — goroutine lifecycle management, data race prevention, channel discipline, deadlock avoidance, context cancellation, and bounded concurrency — are not optional. They are the difference between a robust system and one that fails unpredictably under load.
Start by auditing your current codebase for the pitfalls we've covered. Run the race detector on your tests. Add context cancellation to any goroutine that blocks. And remember: the goal is not to use as many goroutines as possible, but to use them wisely. Your future self — and your on-call team — will thank you.
Comments (0)
Please sign in to post a comment.
Don't have an account? Create one
No comments yet. Be the first to comment!