Coordinating Multiple Tasks in C#

Overview

Coordinating multiple tasks in C# means managing several asynchronous or parallel operations so they start, wait, complete, fail, cancel, or produce results in a controlled way. It is a common part of modern .NET development because real applications often need to perform more than one operation at the same time: calling multiple APIs, loading several database resources, processing files, sending notifications, running background jobs, or handling producer-consumer workloads.

In C#, this is usually done with the Task-based Asynchronous Pattern using Task, Task<T>, async, await, Task.WhenAll, Task.WhenAny, Task.WhenEach, Parallel.ForEachAsync, SemaphoreSlim, channels, cancellation tokens, and sometimes TPL Dataflow. The goal is not simply to "make things parallel". The real goal is to improve responsiveness, reduce latency, increase throughput, and keep code reliable under failure, cancellation, and high load.

This topic matters in interviews because it tests whether a developer understands the difference between asynchronous concurrency and CPU parallelism, how exceptions behave across multiple tasks, how cancellation should be propagated, how to avoid unbounded concurrency, and how to choose the correct coordination pattern for production code. Interviewers often ask about this topic because many real performance and reliability bugs come from incorrect task coordination: sequential awaits that should be concurrent, Task.Run overuse, forgotten awaits, fire-and-forget work, shared state races, blocked threads, thread pool starvation, and missing cancellation.

Core Concepts

Task Coordination vs Simple Async/Await

A single asynchronous operation is usually straightforward:

var user = await userService.GetUserAsync(userId, cancellationToken);

Task coordination becomes necessary when multiple operations need to be managed together:

Task<User> userTask = userService.GetUserAsync(userId, cancellationToken);
Task<IReadOnlyList<Order>> ordersTask = orderService.GetOrdersAsync(userId, cancellationToken);
Task<AccountStatus> statusTask = accountService.GetStatusAsync(userId, cancellationToken);

await Task.WhenAll(userTask, ordersTask, statusTask);

User user = await userTask;
IReadOnlyList<Order> orders = await ordersTask;
AccountStatus status = await statusTask;

The important detail is that the tasks are created before they are awaited. This allows the operations to run concurrently when the underlying APIs support asynchronous execution. If each operation is awaited immediately, they run one after another:

// Sequential: each operation starts after the previous one completes.
User user = await userService.GetUserAsync(userId, cancellationToken);
IReadOnlyList<Order> orders = await orderService.GetOrdersAsync(userId, cancellationToken);
AccountStatus status = await accountService.GetStatusAsync(userId, cancellationToken);

The concurrent version can reduce total latency when the operations are independent.

Concurrency vs Parallelism

Concurrency means multiple operations are in progress during the same time period. Parallelism means multiple operations are executing at the same instant, usually on different CPU cores.

In C#:

• async and await are commonly used for I/O-bound concurrency, such as HTTP calls, database calls, file reads, and queue operations.
• Task.Run, Parallel.For, Parallel.ForEach, and CPU-bound work use threads and can run in parallel.
• Task.WhenAll coordinates tasks; it does not create threads by itself.
• Task.WhenAny waits until one task completes; it does not stop the other tasks automatically.
• Parallel.ForEachAsync is useful when processing many items with controlled parallelism, especially when each item uses asynchronous work.

A common interview mistake is saying that Task.WhenAll "runs tasks in parallel". More accurately, the tasks are already started by the time they are passed to Task.WhenAll; Task.WhenAll returns a task that completes when all supplied tasks complete.

Task.WhenAll

Task.WhenAll is used when all operations are required before the next step can continue.

Example: load multiple independent pieces of data for a dashboard:

public async Task<DashboardDto> GetDashboardAsync(Guid userId, CancellationToken cancellationToken)
{
    Task<UserDto> userTask = userApi.GetUserAsync(userId, cancellationToken);
    Task<IReadOnlyList<OrderDto>> ordersTask = orderApi.GetRecentOrdersAsync(userId, cancellationToken);
    Task<IReadOnlyList<NotificationDto>> notificationsTask = notificationApi.GetUnreadAsync(userId, cancellationToken);

    await Task.WhenAll(userTask, ordersTask, notificationsTask);

    return new DashboardDto
    {
        User = await userTask,
        RecentOrders = await ordersTask,
        Notifications = await notificationsTask
    };
}

This pattern is useful when:

• The operations are independent.
• All results are needed.
• Running them concurrently does not violate resource constraints.
• The called services can handle the concurrency.

A practical benefit is latency reduction. If three independent API calls each take around 500 ms, sequential awaits may take around 1500 ms, while concurrent execution may complete closer to the slowest individual operation.

Starting Tasks Correctly

Creating a task is not always the same thing as executing work. Many asynchronous methods start work when the method is called:

Task<Product> productTask = productClient.GetProductAsync(productId, cancellationToken);

But LINQ uses deferred execution, so this code can be misleading:

IEnumerable<Task<Product>> tasks = productIds.Select(id => productClient.GetProductAsync(id, cancellationToken));

Product[] products = await Task.WhenAll(tasks);

This usually works because Task.WhenAll enumerates the sequence, but it can be harder to reason about. A clearer and safer pattern is to materialize the task list immediately:

Task<Product>[] tasks = productIds
    .Select(id => productClient.GetProductAsync(id, cancellationToken))
    .ToArray();

Product[] products = await Task.WhenAll(tasks);

This makes it explicit that the tasks are created before waiting for them.

Task.WhenAll Result Ordering

For Task.WhenAll<TResult>, the result array preserves the same order as the input tasks, not the order in which tasks completed.

Task<string>[] tasks =
[
    GetNameAsync(1, cancellationToken),
    GetNameAsync(2, cancellationToken),
    GetNameAsync(3, cancellationToken)
];

string[] names = await Task.WhenAll(tasks);

// names[0] is the result of GetNameAsync(1), even if that task completed last.

This is useful when the caller needs to keep results aligned with the original input order.

Exception Handling with Task.WhenAll

If one or more tasks fail, the task returned by Task.WhenAll becomes faulted. When awaited, an exception is thrown. A common interview point is that multiple tasks can fail, so production code may need to inspect all inner exceptions.

Task[] tasks =
[
    SendEmailAsync(userA, cancellationToken),
    SendEmailAsync(userB, cancellationToken),
    SendEmailAsync(userC, cancellationToken)
];

Task allTasks = Task.WhenAll(tasks);

try
{
    await allTasks;
}
catch
{
    foreach (Exception exception in allTasks.Exception?.Flatten().InnerExceptions ?? [])
    {
        logger.LogError(exception, "A task failed while sending emails.");
    }

    throw;
}

Important habits:

• Always await the task returned by Task.WhenAll.
• Do not assume only one task can fail.
• Log enough context to know which operation failed.
• Decide whether partial success is acceptable.
• Avoid swallowing exceptions silently.

Cancellation with Multiple Tasks

Cancellation in .NET is cooperative. A CancellationToken is a signal; it does not forcibly kill a task. Each operation must observe the token and stop safely.

public async Task ProcessAsync(IEnumerable<int> ids, CancellationToken cancellationToken)
{
    Task[] tasks = ids
        .Select(id => ProcessItemAsync(id, cancellationToken))
        .ToArray();

    await Task.WhenAll(tasks);
}

If the caller cancels the token, each task should respond by passing the token to lower-level APIs or checking it in long-running loops:

private async Task ProcessItemAsync(int id, CancellationToken cancellationToken)
{
    cancellationToken.ThrowIfCancellationRequested();

    var item = await repository.GetAsync(id, cancellationToken);

    cancellationToken.ThrowIfCancellationRequested();

    await processor.ProcessAsync(item, cancellationToken);
}

Best practice is to propagate the token through the entire call chain instead of creating unrelated tokens in lower layers.

Task.WhenAny

Task.WhenAny is used when the application wants to continue after the first task completes.

Example: use the first successful result from multiple mirrors or providers:

public async Task<string> GetFromFastestProviderAsync(CancellationToken cancellationToken)
{
    using var cts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);

    Task<string>[] tasks =
    [
        providerA.GetValueAsync(cts.Token),
        providerB.GetValueAsync(cts.Token),
        providerC.GetValueAsync(cts.Token)
    ];

    Task<string> completedTask = await Task.WhenAny(tasks);

    cts.Cancel(); // Ask the remaining operations to stop.

    return await completedTask; // Observe success, failure, or cancellation from the completed task.
}

Important details:

• Task.WhenAny returns the completed task, not the result directly.
• The returned wrapper task completes successfully when any task completes, even if the completed task itself failed or was canceled.
• The remaining tasks continue running unless they are canceled or otherwise stopped.
• You still need to await the completed task to observe its result or exception.

Task.WhenEach

Task.WhenEach is useful when many tasks are running and the program wants to process each result as soon as it completes instead of waiting for all tasks first.

Task<Product>[] tasks = productIds
    .Select(id => productClient.GetProductAsync(id, cancellationToken))
    .ToArray();

await foreach (Task<Product> completedTask in Task.WhenEach(tasks))
{
    Product product = await completedTask;
    Console.WriteLine($"Received product: {product.Name}");
}

This is helpful for streaming progress, updating UI, writing partial results, or reducing memory pressure when results can be processed independently. If targeting an older .NET version that does not support Task.WhenEach, the same behavior can be implemented with a loop around Task.WhenAny.

Throttling with SemaphoreSlim

Unbounded concurrency can overload databases, APIs, file systems, message brokers, or the .NET thread pool. SemaphoreSlim is commonly used to limit how many asynchronous operations run at the same time.

public async Task<IReadOnlyList<ProductDto>> GetProductsAsync(
    IEnumerable<int> productIds,
    CancellationToken cancellationToken)
{
    using var semaphore = new SemaphoreSlim(initialCount: 5, maxCount: 5);

    Task<ProductDto>[] tasks = productIds
        .Select(id => GetProductWithThrottleAsync(id, semaphore, cancellationToken))
        .ToArray();

    return await Task.WhenAll(tasks);
}

private async Task<ProductDto> GetProductWithThrottleAsync(
    int productId,
    SemaphoreSlim semaphore,
    CancellationToken cancellationToken)
{
    await semaphore.WaitAsync(cancellationToken);

    try
    {
        return await productClient.GetProductAsync(productId, cancellationToken);
    }
    finally
    {
        semaphore.Release();
    }
}

The finally block is essential. Without it, an exception can prevent the semaphore from being released, which can cause deadlocks or permanent throttling.

Use throttling when:

• Calling a rate-limited external API.
• Running many database queries.
• Processing many files.
• Sending many messages.
• Protecting limited resources.

Parallel.ForEachAsync

Parallel.ForEachAsync is useful for processing many items with a maximum degree of parallelism.

await Parallel.ForEachAsync(
    source: productIds,
    parallelOptions: new ParallelOptions
    {
        MaxDegreeOfParallelism = 8,
        CancellationToken = cancellationToken
    },
    body: async (productId, token) =>
    {
        Product product = await productClient.GetProductAsync(productId, token);
        await searchIndex.UpdateAsync(product, token);
    });

This is often cleaner than manually creating many tasks and wrapping each one with a semaphore. However, it is best suited for cases where the same operation is applied to each item and where the result can be handled inside the loop body or safely collected.

Trade-offs:

• Good for controlled parallel processing of a collection.
• Less convenient when you need a complex result shape or custom per-task orchestration.
• Shared mutable state inside the loop requires synchronization.
• A high MaxDegreeOfParallelism can still overload downstream systems.

Producer-Consumer Coordination with Channels

Channels provide an asynchronous producer-consumer pattern. Producers write work items into a channel; consumers read from the channel and process items.

using System.Threading.Channels;

public async Task ProcessQueueAsync(IEnumerable<Job> jobs, CancellationToken cancellationToken)
{
    var channel = Channel.CreateBounded<Job>(new BoundedChannelOptions(capacity: 100)
    {
        FullMode = BoundedChannelFullMode.Wait
    });

    Task producer = Task.Run(async () =>
    {
        try
        {
            foreach (Job job in jobs)
            {
                await channel.Writer.WriteAsync(job, cancellationToken);
            }
        }
        finally
        {
            channel.Writer.Complete();
        }
    }, cancellationToken);

    Task[] consumers = Enumerable.Range(0, 4)
        .Select(_ => Task.Run(async () =>
        {
            await foreach (Job job in channel.Reader.ReadAllAsync(cancellationToken))
            {
                await ProcessJobAsync(job, cancellationToken);
            }
        }, cancellationToken))
        .ToArray();

    await Task.WhenAll(consumers.Prepend(producer));
}

Channels are useful when:

• Work arrives over time.
• You need backpressure.
• Producers and consumers should be decoupled.
• A bounded queue is safer than creating unlimited tasks.
• Multiple workers process the same type of job.

A bounded channel is often safer than an unbounded channel because it prevents producers from filling memory faster than consumers can process work.

TPL Dataflow

TPL Dataflow is a higher-level library for building asynchronous pipelines. It is useful when work needs to move through multiple stages, such as download, parse, validate, transform, and save.

Example concept:

var options = new ExecutionDataflowBlockOptions
{
    MaxDegreeOfParallelism = 4,
    CancellationToken = cancellationToken
};

var processBlock = new ActionBlock<Order>(
    async order => await ProcessOrderAsync(order, cancellationToken),
    options);

foreach (Order order in orders)
{
    await processBlock.SendAsync(order, cancellationToken);
}

processBlock.Complete();
await processBlock.Completion;

TPL Dataflow can be useful for more advanced pipelines because it supports blocks, linking, bounded capacity, completion propagation, and controlled parallelism. For simpler producer-consumer cases, channels are often enough.

Task.Run and CPU-Bound Work

Task.Run queues work to the thread pool. It is useful for CPU-bound work that should run on a background thread, especially in desktop or UI applications where blocking the UI thread would freeze the application.

Task<Report> reportTask = Task.Run(() => GenerateLargeReport(input), cancellationToken);
Report report = await reportTask;

In ASP.NET Core, Task.Run should be used carefully. Wrapping synchronous blocking work in Task.Run does not make it truly asynchronous; it just moves the blocking work to another thread pool thread. Under load, this can reduce scalability.

Good use cases:

• CPU-heavy calculations.
• Isolating work from a UI thread.
• Running independent CPU-bound tasks in a controlled way.

Poor use cases:

• Wrapping already asynchronous I/O methods.
• Hiding blocking database or HTTP calls.
• Fire-and-forget background work in a web request.
• Creating one task per item for a very large collection without throttling.

Fire-and-Forget Tasks

Fire-and-forget means starting a task without awaiting it.

_ = SendAuditLogAsync(auditEvent, cancellationToken);

This is risky because:

• Exceptions can be lost or become unobserved.
• The request scope may end before the task finishes.
• Scoped dependencies such as DbContext may be disposed.
• Cancellation and shutdown may not be handled correctly.
• There is no reliable completion signal.

In production, prefer a background queue, hosted service, message broker, or durable job processor for work that must continue after the current request.

Shared State and Thread Safety

Coordinating multiple tasks often means multiple operations run at the same time. If they access shared mutable state, the code must be thread-safe.

Unsafe example:

var results = new List<Product>();

await Task.WhenAll(productIds.Select(async id =>
{
    Product product = await productClient.GetProductAsync(id, cancellationToken);
    results.Add(product); // Not safe when multiple tasks write at the same time.
}));

Safer options:

Task<Product>[] tasks = productIds
    .Select(id => productClient.GetProductAsync(id, cancellationToken))
    .ToArray();

Product[] results = await Task.WhenAll(tasks);

Or use a thread-safe collection when shared writes are necessary:

var results = new ConcurrentBag<Product>();

await Parallel.ForEachAsync(productIds, cancellationToken, async (id, token) =>
{
    Product product = await productClient.GetProductAsync(id, token);
    results.Add(product);
});

Best practice is to avoid shared mutable state when possible. Prefer returning results from tasks and combining them after coordination.

Resource Lifetime and DbContext Warning

Many objects are not safe to use concurrently. A common production mistake is running multiple EF Core operations in parallel on the same DbContext instance.

Problematic pattern:

Task<User?> userTask = dbContext.Users.FirstOrDefaultAsync(x => x.Id == userId, cancellationToken);
Task<List<Order>> ordersTask = dbContext.Orders.Where(x => x.UserId == userId).ToListAsync(cancellationToken);

await Task.WhenAll(userTask, ordersTask); // Risky: same DbContext used concurrently.

Better options:

• Use one combined query when possible.
• Run the queries sequentially if they share the same context.
• Use separate context instances when true parallel database queries are justified.
• Consider whether parallel database calls actually improve performance or just increase load.

Timeouts with Multiple Tasks

Task coordination should usually include cancellation or timeout behavior. One modern pattern is to use cancellation tokens with timeout sources:

using var timeoutCts = new CancellationTokenSource(TimeSpan.FromSeconds(5));
using var linkedCts = CancellationTokenSource.CreateLinkedTokenSource(
    timeoutCts.Token,
    cancellationToken);

await Task.WhenAll(tasks.Select(task => RunOperationAsync(task, linkedCts.Token)));

For waiting on a specific task with a timeout, WaitAsync can be useful:

await externalCallTask.WaitAsync(TimeSpan.FromSeconds(3), cancellationToken);

Timeouts should be selected carefully. A timeout that is too short causes false failures; a timeout that is too long can waste resources and delay recovery.

Choosing the Right Coordination Tool

Common Mistakes

Common mistakes include:

• Awaiting each task immediately when the operations could run concurrently.
• Creating thousands of tasks without throttling.
• Using Task.Run to wrap already asynchronous I/O.
• Blocking with .Result, .Wait(), or Task.WaitAll() inside async code.
• Forgetting to await a task.
• Using async void except for event handlers.
• Assuming Task.WhenAny cancels remaining tasks automatically.
• Assuming Task.WhenAll creates new threads.
• Modifying List<T> or other non-thread-safe collections from multiple tasks.
• Running parallel EF Core operations on the same DbContext.
• Forgetting to release SemaphoreSlim in a finally block.
• Swallowing exceptions from background tasks.
• Ignoring cancellation tokens.

Best Practices

Good habits for coordinating multiple tasks:

• Start independent asynchronous operations before awaiting them.
• Use Task.WhenAll for independent operations where all results are required.
• Use Task.WhenAny or Task.WhenEach when completion order matters.
• Use throttling for large collections or limited downstream systems.
• Prefer Parallel.ForEachAsync for simple bounded parallel item processing.
• Prefer channels or queues for producer-consumer workloads.
• Propagate CancellationToken through every async layer.
• Avoid shared mutable state; combine task results after awaiting.
• Use thread-safe collections only when shared writes are truly needed.
• Handle exceptions deliberately, especially when partial failure is possible.
• Avoid blocking async code with synchronous waits.
• Avoid fire-and-forget work unless it is routed through a reliable background processing mechanism.
• Measure performance before and after adding concurrency; more concurrency is not always faster.