IHttpClientFactory and Resilient Outbound HTTP

Overview

IHttpClientFactory is the recommended .NET abstraction for creating and configuring HttpClient instances in applications that use dependency injection. It centralizes outbound HTTP configuration, manages underlying message handler lifetimes, supports named and typed clients, enables outgoing middleware through delegating handlers, integrates logging, and works with modern resilience strategies such as retries, timeouts, circuit breakers, rate limiting, and hedging.

Outbound HTTP calls are common in modern .NET systems. A Web API may call:

• Payment providers.
• Identity providers.
• Internal microservices.
• Third-party APIs.
• Azure services.
• REST APIs.
• Webhooks.
• External reporting services.
• Feature flag services.
• Search services.
• Notification services.

These calls can fail for reasons outside the current application:

• Network interruptions.
• DNS changes.
• Connection resets.
• Server overload.
• Rate limiting.
• Slow dependencies.
• Temporary 5xx responses.
• Timeouts.
• Authentication failures.
• Bad request payloads.
• Downstream deployment restarts.
• Regional outages.

Resilient outbound HTTP means designing these calls so the application can handle expected transient failures without making the system worse. A resilient client should use correct HttpClient lifetime management, sensible timeouts, safe retries, circuit breakers, rate limits, cancellation tokens, observability, and clear error handling.

This topic is important because incorrect HttpClient usage can cause production incidents. Creating and disposing a new HttpClient for every request can exhaust available sockets. Keeping a single HttpClient forever without connection lifetime configuration can fail to react to DNS changes. Retrying every request blindly can duplicate writes, increase downstream load, and make outages worse. Hiding all HTTP failures behind generic exceptions makes production debugging harder.

This topic is important for interviews because it tests practical .NET production experience. Interviewers often ask:

• Why should you not create a new HttpClient manually for every request?
• What problem does IHttpClientFactory solve?
• What is the difference between HttpClient and HttpMessageHandler?
• What are named clients and typed clients?
• Why can typed clients be problematic in singleton services?
• What is handler lifetime?
• How does IHttpClientFactory help with DNS changes?
• What is SocketsHttpHandler.PooledConnectionLifetime?
• How do you add retries, timeouts, and circuit breakers?
• Why are retries dangerous for non-idempotent HTTP methods?
• What is a delegating handler?
• How do you add authentication headers?
• How do you test code that uses HttpClient?
• How do you avoid leaking cookies or scoped data through handlers?
• How do you observe outbound HTTP calls in logs and traces?

A strong answer should explain that IHttpClientFactory is not only about avoiding socket exhaustion. It is also about central configuration, logical clients, handler reuse, outgoing middleware, logging, testability, resilience, and consistent outbound HTTP design.

Core Concepts

The Problem with Outbound HTTP

Outbound HTTP looks simple:

using var client = new HttpClient();

var response = await client.GetAsync("https://api.example.com/products");
response.EnsureSuccessStatusCode();

var json = await response.Content.ReadAsStringAsync();

This code may work in a demo, but it is not a good production pattern when called frequently.

Problems can include:

• Too many socket connections.
• TCP ports stuck in TIME_WAIT.
• DNS changes not respected.
• No central timeout policy.
• No logging or tracing strategy.
• No retry policy.
• No rate limiting.
• No authentication handler.
• No consistent error handling.
• Hard-to-test code.
• Configuration duplicated across services.

Production HTTP calls need a deliberate design.

HttpClient and HttpMessageHandler

HttpClient is the high-level object used by application code to send HTTP requests.

HttpMessageHandler is the lower-level pipeline component that actually sends the request. The default primary handler in modern .NET is usually based on SocketsHttpHandler.

Important relationship:

HttpClient
  -> DelegatingHandler
    -> DelegatingHandler
      -> Primary HttpMessageHandler / SocketsHttpHandler
        -> connection pool
        -> network

The handler owns the underlying connection pool. Reusing handlers is important because creating too many handlers can create too many connection pools and exhaust sockets.

IHttpClientFactory creates new HttpClient objects but reuses underlying handlers for a configured lifetime.

What IHttpClientFactory Does

IHttpClientFactory is a factory abstraction for creating configured HttpClient instances.

Register it:

builder.Services.AddHttpClient();

Use it:

public sealed class ProductApiService
{
    private readonly IHttpClientFactory _httpClientFactory;

    public ProductApiService(IHttpClientFactory httpClientFactory)
    {
        _httpClientFactory = httpClientFactory;
    }

    public async Task<string> GetProductsAsync(CancellationToken cancellationToken)
    {
        var client = _httpClientFactory.CreateClient();

        return await client.GetStringAsync(
            "https://api.example.com/products",
            cancellationToken);
    }
}

Benefits:

• Provides HttpClient through dependency injection.
• Centralizes outbound HTTP configuration.
• Supports named clients.
• Supports typed clients.
• Supports generated clients such as Refit clients.
• Manages HttpMessageHandler lifetime.
• Reuses handlers to avoid socket exhaustion.
• Recycles handlers so DNS changes can be picked up.
• Adds logging for outgoing requests.
• Supports delegating handlers as outgoing middleware.
• Works with resilience handlers.
• Makes outbound HTTP easier to test.

Socket Exhaustion

A common anti-pattern is creating a new HttpClient per request.

Bad:

public async Task<string> GetAsync()
{
    using var client = new HttpClient();

    return await client.GetStringAsync("https://api.example.com/data");
}

HttpClient itself is disposable, but the important part is the underlying handler and connection pool. Creating too many clients and handlers can create too many connections. TCP ports may remain unavailable for a period after closing, which can lead to port exhaustion under load.

Better options:

1. Use short-lived HttpClient instances created by IHttpClientFactory.
2. Use a long-lived/static HttpClient with SocketsHttpHandler.PooledConnectionLifetime.

With IHttpClientFactory:

builder.Services.AddHttpClient("ExternalApi", client =>
{
    client.BaseAddress = new Uri("https://api.example.com/");
});

var client = _httpClientFactory.CreateClient("ExternalApi");

This creates a new HttpClient but reuses the underlying handler while it is valid.

DNS Changes

A different problem happens when a HttpClient or its underlying handler lives too long. The client may keep using existing connections and may not react quickly to DNS changes.

This matters in cloud systems where DNS can change because of:

• Load balancers.
• Blue-green deployments.
• Kubernetes services.
• Azure App Service changes.
• Regional failover.
• Service discovery.
• Container restarts.
• API gateway changes.

IHttpClientFactory helps by recycling handlers after a configured handler lifetime.

Default handler lifetime is commonly two minutes.

Example:

builder.Services.AddHttpClient("ExternalApi", client =>
{
    client.BaseAddress = new Uri("https://api.example.com/");
})
.SetHandlerLifetime(TimeSpan.FromMinutes(5));

When the handler lifetime expires and no clients are using it, the factory can dispose it and create a new handler. New handlers create new connections and can resolve DNS again.

Short-Lived Factory Clients vs Long-Lived Static Clients

There are two valid lifetime strategies in modern .NET.

Strategy 1: short-lived clients from IHttpClientFactory.

var client = _httpClientFactory.CreateClient("ExternalApi");

Use this when:

• You use dependency injection.
• You want named or typed clients.
• You want outgoing middleware.
• You want centralized logging and configuration.
• You want easy resilience registration.
• You want multiple logical external clients.

Strategy 2: long-lived HttpClient with SocketsHttpHandler.PooledConnectionLifetime.

private static readonly HttpClient Client = new(
    new SocketsHttpHandler
    {
        PooledConnectionLifetime = TimeSpan.FromMinutes(5)
    })
{
    BaseAddress = new Uri("https://api.example.com/")
};

Use this when:

• You do not use dependency injection.
• You want a simple static client.
• You understand connection lifetime configuration.
• You do not need named/typed client configuration.
• You do not need factory-based outgoing middleware.

Avoid the worst option: creating a new unmanaged HttpClient and handler for every operation.

Handler Lifetime

IHttpClientFactory caches handlers. Handler lifetime controls how long a handler can be reused.

Example:

builder.Services.AddHttpClient("Payments", client =>
{
    client.BaseAddress = new Uri("https://payments.example.com/");
})
.SetHandlerLifetime(TimeSpan.FromMinutes(5));

Important points:

• CreateClient returns a new HttpClient.
• The handler may be reused.
• The handler owns the connection pool.
• Handler reuse helps avoid socket exhaustion.
• Handler recycling helps pick up DNS changes.
• Disposing a factory-created HttpClient does not immediately dispose the handler.
• Long-lived typed clients can defeat handler recycling benefits.

Choose handler lifetime based on expected DNS and network change frequency. Do not set it randomly without understanding the environment.

SocketsHttpHandler and PooledConnectionLifetime

SocketsHttpHandler.PooledConnectionLifetime limits how long a connection can stay in the pool. When the lifetime expires, the connection is replaced after it completes active work.

Example with IHttpClientFactory:

builder.Services.AddHttpClient("Inventory", client =>
{
    client.BaseAddress = new Uri("https://inventory.example.com/");
})
.UseSocketsHttpHandler((handler, _) =>
{
    handler.PooledConnectionLifetime = TimeSpan.FromMinutes(5);
})
.SetHandlerLifetime(Timeout.InfiniteTimeSpan);

This pattern uses SocketsHttpHandler to handle connection recycling. Since connection lifetime is handled at the socket handler level, factory handler recycling can be disabled.

This is useful when you want precise connection lifetime control while still using IHttpClientFactory for DI, named clients, logging, and handlers.

Basic Client Registration

Basic registration:

builder.Services.AddHttpClient();

Basic usage:

public sealed class WeatherGateway
{
    private readonly IHttpClientFactory _httpClientFactory;

    public WeatherGateway(IHttpClientFactory httpClientFactory)
    {
        _httpClientFactory = httpClientFactory;
    }

    public async Task<string> GetForecastAsync(CancellationToken cancellationToken)
    {
        var client = _httpClientFactory.CreateClient();

        using var response = await client.GetAsync(
            "https://weather.example.com/forecast",
            cancellationToken);

        response.EnsureSuccessStatusCode();

        return await response.Content.ReadAsStringAsync(cancellationToken);
    }
}

This is useful for simple or legacy code, but named or typed clients are usually better for real applications because they centralize configuration per external service.

Named Clients

A named client is configured with a string name.

Registration:

builder.Services.AddHttpClient("CatalogApi", client =>
{
    client.BaseAddress = new Uri("https://catalog.example.com/");
    client.DefaultRequestHeaders.UserAgent.ParseAdd("my-app/1.0");
    client.Timeout = TimeSpan.FromSeconds(20);
});

Usage:

public sealed class CatalogGateway
{
    private readonly IHttpClientFactory _httpClientFactory;

    public CatalogGateway(IHttpClientFactory httpClientFactory)
    {
        _httpClientFactory = httpClientFactory;
    }

    public async Task<ProductDto?> GetProductAsync(
        int productId,
        CancellationToken cancellationToken)
    {
        var client = _httpClientFactory.CreateClient("CatalogApi");

        return await client.GetFromJsonAsync<ProductDto>(
            $"api/products/{productId}",
            cancellationToken);
    }
}

Named clients are useful when:

• The app calls multiple external APIs.
• Each API has different base address, timeout, headers, and handlers.
• A singleton service needs to create clients when needed.
• You want to avoid capturing a typed client in a singleton.
• You want configuration keyed by logical external dependency.

Avoid deriving client names from unbounded user input. Each distinct name may create separate handler state and can lead to resource problems.

Typed Clients

A typed client wraps HttpClient in a strongly typed class.

Registration:

builder.Services.AddHttpClient<CatalogClient>(client =>
{
    client.BaseAddress = new Uri("https://catalog.example.com/");
    client.Timeout = TimeSpan.FromSeconds(20);
});

Typed client:

public sealed class CatalogClient
{
    private readonly HttpClient _httpClient;

    public CatalogClient(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<ProductDto?> GetProductAsync(
        int productId,
        CancellationToken cancellationToken)
    {
        return await _httpClient.GetFromJsonAsync<ProductDto>(
            $"api/products/{productId}",
            cancellationToken);
    }
}

Usage:

public sealed class ProductService
{
    private readonly CatalogClient _catalogClient;

    public ProductService(CatalogClient catalogClient)
    {
        _catalogClient = catalogClient;
    }

    public Task<ProductDto?> GetProductAsync(
        int productId,
        CancellationToken cancellationToken)
    {
        return _catalogClient.GetProductAsync(productId, cancellationToken);
    }
}

Benefits:

• Strong typing.
• Encapsulates API-specific logic.
• Avoids string client names in consuming code.
• Provides IntelliSense.
• Keeps external API calls in one place.
• Easier to mock through an interface if needed.

Typed Client Interface

A typed client can implement an interface.

public interface ICatalogClient
{
    Task<ProductDto?> GetProductAsync(
        int productId,
        CancellationToken cancellationToken);
}

public sealed class CatalogClient : ICatalogClient
{
    private readonly HttpClient _httpClient;

    public CatalogClient(HttpClient httpClient)
    {
        _httpClient = httpClient;
    }

    public async Task<ProductDto?> GetProductAsync(
        int productId,
        CancellationToken cancellationToken)
    {
        return await _httpClient.GetFromJsonAsync<ProductDto>(
            $"api/products/{productId}",
            cancellationToken);
    }
}

Registration:

builder.Services.AddHttpClient<ICatalogClient, CatalogClient>(client =>
{
    client.BaseAddress = new Uri("https://catalog.example.com/");
});

This improves testability because application services can depend on ICatalogClient rather than HttpClient.

Avoid Capturing Typed Clients in Singletons

Typed clients are usually registered as transient. They should be treated as short-lived.

Problem:

builder.Services.AddHttpClient<CatalogClient>();
builder.Services.AddSingleton<ProductCache>();

public sealed class ProductCache
{
    private readonly CatalogClient _catalogClient;

    public ProductCache(CatalogClient catalogClient)
    {
        _catalogClient = catalogClient;
    }
}

If a singleton captures a typed client, the typed client and its HttpClient may live too long. This can prevent the application from getting handler updates and reacting to DNS changes.

Better for singleton services:

public sealed class ProductCache
{
    private readonly IHttpClientFactory _httpClientFactory;

    public ProductCache(IHttpClientFactory httpClientFactory)
    {
        _httpClientFactory = httpClientFactory;
    }

    public async Task RefreshAsync(CancellationToken cancellationToken)
    {
        var client = _httpClientFactory.CreateClient("CatalogApi");

        var products = await client.GetFromJsonAsync<List<ProductDto>>(
            "api/products",
            cancellationToken);
    }
}

Use named clients from singletons, or configure a long-lived client with PooledConnectionLifetime deliberately.

Generated Clients

Generated clients are libraries that generate HTTP client implementations from interfaces or contracts. One common example is Refit.

Example interface:

public interface ICatalogApi
{
    [Get("/api/products/{id}")]
    Task<ProductDto> GetProductAsync(int id);
}

Registration concept:

builder.Services.AddRefitClient<ICatalogApi>()
    .ConfigureHttpClient(client =>
    {
        client.BaseAddress = new Uri("https://catalog.example.com/");
    });

Generated clients are useful when:

• You want declarative REST API definitions.
• You want less manual serialization code.
• You want strongly typed endpoints.
• You want to combine generated clients with IHttpClientFactory.

Generated clients still need the same production concerns:

• Timeouts.
• Resilience.
• Authentication.
• Observability.
• Error handling.
• Testability.
• Safe retries.

Delegating Handlers

A delegating handler is outgoing middleware for HttpClient.

It can run logic before and after the HTTP request.

Example:

public sealed class CorrelationIdHandler : DelegatingHandler
{
    private readonly ICorrelationIdAccessor _correlationIdAccessor;

    public CorrelationIdHandler(ICorrelationIdAccessor correlationIdAccessor)
    {
        _correlationIdAccessor = correlationIdAccessor;
    }

    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        var correlationId = _correlationIdAccessor.CorrelationId;

        if (!string.IsNullOrWhiteSpace(correlationId))
        {
            request.Headers.TryAddWithoutValidation(
                "X-Correlation-Id",
                correlationId);
        }

        return base.SendAsync(request, cancellationToken);
    }
}

Registration:

builder.Services.AddTransient<CorrelationIdHandler>();

builder.Services.AddHttpClient("CatalogApi", client =>
{
    client.BaseAddress = new Uri("https://catalog.example.com/");
})
.AddHttpMessageHandler<CorrelationIdHandler>();

Delegating handlers are useful for:

• Correlation IDs.
• Authentication tokens.
• Custom headers.
• Logging enrichment.
• Request signing.
• Tenant headers.
• Idempotency keys.
• User-agent headers.
• Custom metrics.
• Request/response transformations.

Avoid putting large business workflows in handlers. Handlers should be small, cross-cutting, and focused.

Handler Scope Caveat

IHttpClientFactory creates a separate dependency injection scope for message handlers. This scope is not the same as an ASP.NET Core request scope.

This matters because a handler can live longer than a single incoming request.

Avoid storing request-specific sensitive data inside a long-lived handler.

Risky:

public sealed class BadTokenHandler : DelegatingHandler
{
    private string? _cachedToken;

    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        request.Headers.Authorization =
            new AuthenticationHeaderValue("Bearer", _cachedToken);

        return base.SendAsync(request, cancellationToken);
    }
}

Better:

public sealed class AccessTokenHandler : DelegatingHandler
{
    private readonly IAccessTokenProvider _accessTokenProvider;

    public AccessTokenHandler(IAccessTokenProvider accessTokenProvider)
    {
        _accessTokenProvider = accessTokenProvider;
    }

    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        var token = await _accessTokenProvider
            .GetAccessTokenAsync(cancellationToken);

        request.Headers.Authorization =
            new AuthenticationHeaderValue("Bearer", token);

        return await base.SendAsync(request, cancellationToken);
    }
}

Keep handlers stateless when possible.

Cookies and IHttpClientFactory

IHttpClientFactory pools handlers. If handlers use cookies, the underlying CookieContainer can be shared between clients using the same handler. When a handler expires, cookies stored in that handler can be lost.

This can cause problems:

• Cookies leak between unrelated logical calls.
• User-specific cookies are reused accidentally.
• Cookies disappear when the handler is recycled.
• Stateful cookie-based flows behave unpredictably.

Avoid using IHttpClientFactory for scenarios requiring isolated cookie containers unless you understand and control the handler configuration.

For user-specific cookie sessions, consider:

• A manually managed HttpClientHandler per session.
• A browser automation tool for real browser flows.
• Token-based authentication instead of cookies for service-to-service calls.
• Explicitly disabling cookies when not needed.

Example disabling automatic cookies:

builder.Services.AddHttpClient("NoCookies")
    .ConfigurePrimaryHttpMessageHandler(() =>
    {
        return new HttpClientHandler
        {
            UseCookies = false
        };
    });

Configuring Primary Handler

The primary handler controls low-level HTTP behavior.

Example:

builder.Services.AddHttpClient("InternalApi", client =>
{
    client.BaseAddress = new Uri("https://internal.example.com/");
})
.ConfigurePrimaryHttpMessageHandler(() =>
{
    return new SocketsHttpHandler
    {
        PooledConnectionLifetime = TimeSpan.FromMinutes(5),
        MaxConnectionsPerServer = 50,
        AutomaticDecompression =
            DecompressionMethods.GZip | DecompressionMethods.Deflate
    };
});

Common handler settings:

• Proxy.
• Credentials.
• Client certificates.
• Automatic decompression.
• Redirect behavior.
• Cookie behavior.
• Max connections per server.
• Connection lifetime.
• SSL/TLS options.

Do not configure these globally without considering each external dependency.

Timeouts

Timeouts are essential for resilient outbound HTTP. Without timeouts, requests can hang long enough to exhaust threads, connections, or request capacity.

There are multiple timeout layers:

Example:

builder.Services.AddHttpClient("CatalogApi", client =>
{
    client.BaseAddress = new Uri("https://catalog.example.com/");
    client.Timeout = TimeSpan.FromSeconds(30);
});

With resilience handlers, prefer clear total and attempt timeout design rather than random overlapping timeouts.

Example:

builder.Services.AddHttpClient<CatalogClient>(client =>
{
    client.BaseAddress = new Uri("https://catalog.example.com/");
})
.AddStandardResilienceHandler(options =>
{
    options.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(20);
    options.AttemptTimeout.Timeout = TimeSpan.FromSeconds(5);
});

Timeouts should be based on user experience, downstream SLA, retry budget, and system capacity.

Cancellation Tokens

Always pass cancellation tokens to outbound HTTP operations.

Example:

public async Task<ProductDto?> GetProductAsync(
    int productId,
    CancellationToken cancellationToken)
{
    return await _httpClient.GetFromJsonAsync<ProductDto>(
        $"api/products/{productId}",
        cancellationToken);
}

In ASP.NET Core, the request cancellation token is usually available as a method parameter:

app.MapGet("/api/products/{id:int}", async (
    int id,
    CatalogClient catalogClient,
    CancellationToken cancellationToken) =>
{
    var product = await catalogClient.GetProductAsync(id, cancellationToken);

    return product is null ? Results.NotFound() : Results.Ok(product);
});

Benefits:

• Stops unnecessary work when the client disconnects.
• Frees resources faster.
• Helps downstream cancellation.
• Prevents long-running abandoned requests.
• Works with timeouts and resilience policies.

Cancellation is not a replacement for timeouts. Use both.

Resilient Outbound HTTP

Resilience is the ability to handle temporary failures and degraded dependencies without immediately failing the whole system or making the failure worse.

Common resilience strategies:

Resilience is not just adding retries. It is controlling failure behavior.

Microsoft.Extensions.Http.Resilience

Modern .NET provides Microsoft.Extensions.Http.Resilience for resilient HTTP pipelines.

Install:

dotnet add package Microsoft.Extensions.Http.Resilience

Registration:

builder.Services.AddHttpClient<CatalogClient>(client =>
{
    client.BaseAddress = new Uri("https://catalog.example.com/");
})
.AddStandardResilienceHandler();

The standard resilience handler combines several strategies with sensible defaults:

• Rate limiter.
• Total request timeout.
• Retry.
• Circuit breaker.
• Attempt timeout.

This gives a good starting point for many outbound HTTP clients.

However, do not blindly apply default resilience to every dependency. Review idempotency, downstream capacity, expected latency, and business behavior.

Standard Resilience Handler

Example:

builder.Services.AddHttpClient<InventoryClient>(client =>
{
    client.BaseAddress = new Uri("https://inventory.example.com/");
})
.AddStandardResilienceHandler();

This can handle transient errors such as:

• HTTP 5xx responses.
• HTTP 408 Request Timeout.
• HTTP 429 Too Many Requests.
• HttpRequestException.
• Timeout-related exceptions.

The default pipeline includes retry and circuit breaker behavior. This is useful, but it must be matched with the operation semantics.

For example, retrying a GET request is often safe. Retrying a POST that creates a payment may be dangerous unless the downstream API supports idempotency keys.

Customizing Resilience

Example: disable retries for unsafe HTTP methods.

builder.Services.AddHttpClient<PaymentsClient>(client =>
{
    client.BaseAddress = new Uri("https://payments.example.com/");
})
.AddStandardResilienceHandler(options =>
{
    options.Retry.DisableForUnsafeHttpMethods();

    options.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(30);
    options.AttemptTimeout.Timeout = TimeSpan.FromSeconds(10);
});

This prevents automatic retries for methods such as POST, PUT, PATCH, and DELETE.

Example: disable retries only for selected methods.

builder.Services.AddHttpClient<OrdersClient>(client =>
{
    client.BaseAddress = new Uri("https://orders.example.com/");
})
.AddStandardResilienceHandler(options =>
{
    options.Retry.DisableFor(HttpMethod.Post, HttpMethod.Delete);
});

Always think about whether the operation is safe to retry.

Retry Strategy

Retries are useful for transient failures.

Good retry candidates:

• Temporary network failures.
• HTTP 408.
• HTTP 429 when allowed by service contract.
• HTTP 500, 502, 503, 504.
• Connection reset.
• Temporary DNS/network issue.
• Short downstream restart.

Poor retry candidates:

• HTTP 400 Bad Request.
• HTTP 401 Unauthorized.
• HTTP 403 Forbidden.
• HTTP 404 Not Found in most cases.
• Validation errors.
• Business rule failures.
• Non-idempotent writes without idempotency protection.
• Large uploads.
• Long-running operations.
• Payment creation without idempotency key.

Bad retry behavior:

Retry every failure immediately three times.

Better retry behavior:

Retry transient failures with exponential backoff, jitter, limits, and safe method rules.

Retries increase downstream traffic. During an outage, aggressive retries can amplify failure.

Idempotency and Safe Retries

Idempotency means repeating the same operation produces the same effect.

Generally safe to retry:

• GET
• HEAD
• OPTIONS
• Some PUT operations if designed idempotently.
• Some DELETE operations if designed idempotently.

Potentially unsafe:

• POST
• Some PATCH
• Payment capture.
• Order creation.
• Sending email.
• Creating shipment.
• Creating a ticket.
• Triggering a workflow.

If retrying a write operation, use idempotency keys when supported.

Example:

public async Task<PaymentResponse> CreatePaymentAsync(
    CreatePaymentRequest request,
    CancellationToken cancellationToken)
{
    using var httpRequest = new HttpRequestMessage(
        HttpMethod.Post,
        "api/payments");

    httpRequest.Headers.Add(
        "Idempotency-Key",
        request.IdempotencyKey);

    httpRequest.Content = JsonContent.Create(request);

    using var response = await _httpClient.SendAsync(
        httpRequest,
        cancellationToken);

    response.EnsureSuccessStatusCode();

    return (await response.Content.ReadFromJsonAsync<PaymentResponse>(
        cancellationToken))!;
}

The server must also support idempotency. Adding a header alone does not make the operation safe.

Circuit Breaker

A circuit breaker stops calling a dependency when too many failures occur.

Purpose:

• Prevent hammering a failing service.
• Reduce cascading failures.
• Give the dependency time to recover.
• Fail fast instead of waiting for repeated timeouts.
• Protect the current application from resource exhaustion.

Conceptual states:

Circuit breakers are useful when:

• A dependency is down.
• Calls are timing out repeatedly.
• A service is overloaded.
• Failure rate is high.
• Retrying would make the situation worse.

Circuit breakers should be observable. Teams should know when a circuit opens and which dependency is affected.

Timeout Strategy

Timeouts should be explicit and layered carefully.

Example decision:

User request budget: 2 seconds
Outbound dependency target: 500 ms
Attempt timeout: 300 ms
Max retries: 2
Total timeout: 1 second
Fallback if dependency unavailable

Bad:

HttpClient.Timeout = 100 seconds
Retry 5 times
No total timeout
API request times out after 30 seconds

This can cause the server to keep doing useless work long after the client gave up.

Good timeout design considers:

• User experience.
• API gateway timeout.
• ASP.NET Core request timeout.
• Downstream SLA.
• Retry count.
• Attempt timeout.
• Total timeout.
• Cancellation tokens.
• Queue and thread capacity.

Rate Limiting and Bulkheads

Rate limiting controls how many requests are sent to a dependency.

Bulkhead isolation separates resources so one failing dependency does not consume all capacity.

Example reason:

The app calls Payment API and Recommendation API.
Recommendation API becomes slow.
Without isolation, all outbound connection capacity is consumed by Recommendation calls.
Payment calls also fail.

A rate limiter can limit Recommendation API concurrency so critical Payment API calls still have capacity.

Standard resilience handlers include rate limiting behavior.

For more advanced scenarios, design per-dependency limits based on:

• Dependency SLA.
• Business criticality.
• Downstream rate limits.
• Thread and connection capacity.
• Queue behavior.
• Fallback strategy.

Hedging

Hedging sends an additional request attempt when the original request is slow, often to another endpoint or replica.

Example use cases:

• Read-heavy services.
• Multiple equivalent replicas.
• Regional replicas.
• Search queries.
• Low-latency critical reads.

Hedging can reduce tail latency, but it can also increase traffic.

Avoid hedging for:

• Non-idempotent writes.
• Payment calls.
• Operations with side effects.
• Downstream services already under pressure.
• APIs that cannot handle extra load.

Example registration:

builder.Services.AddHttpClient<SearchClient>(client =>
{
    client.BaseAddress = new Uri("https://search.example.com/");
})
.AddStandardHedgingHandler();

Use hedging carefully and measure its impact.

Fallbacks

A fallback returns an alternative result when the dependency is unavailable.

Examples:

• Return cached product details.
• Return empty recommendations.
• Return a degraded response.
• Queue work for later.
• Show "temporarily unavailable."
• Use a secondary provider.

Fallbacks should be explicit and honest.

Bad fallback:

catch
{
    return new PaymentResult { Success = true };
}

This hides a critical failure.

Good fallback:

catch (ExternalCatalogUnavailableException)
{
    return ProductDetails.Unavailable(productId);
}

Fallbacks are business decisions, not just technical decisions.

Error Handling

Do not blindly call EnsureSuccessStatusCode() everywhere if different status codes need different business handling.

Simple case:

using var response = await _httpClient.GetAsync(
    $"api/products/{productId}",
    cancellationToken);

if (response.StatusCode == HttpStatusCode.NotFound)
{
    return null;
}

response.EnsureSuccessStatusCode();

return await response.Content.ReadFromJsonAsync<ProductDto>(
    cancellationToken);

More structured handling:

using var response = await _httpClient.SendAsync(
    request,
    cancellationToken);

if (response.IsSuccessStatusCode)
{
    return await response.Content.ReadFromJsonAsync<OrderStatusDto>(
        cancellationToken);
}

var body = await response.Content.ReadAsStringAsync(cancellationToken);

throw new ExternalApiException(
    serviceName: "OrdersApi",
    statusCode: response.StatusCode,
    responseBody: body);

Be careful with response body logging. It may contain sensitive data.

Observability

Outbound HTTP should be observable.

Useful telemetry:

• Dependency name.
• URL host.
• HTTP method.
• Route/template when available.
• Status code.
• Duration.
• Timeout vs cancellation vs network failure.
• Retry count.
• Circuit breaker state.
• Rate limiter rejection.
• Correlation ID.
• Trace ID.
• Request ID.
• Error category.
• Sanitized response details.

IHttpClientFactory integrates with logging. Additional observability can come from OpenTelemetry, Application Insights, structured logs, and custom handlers.

Example logging in a typed client:

public sealed class CatalogClient
{
    private readonly HttpClient _httpClient;
    private readonly ILogger<CatalogClient> _logger;

    public CatalogClient(
        HttpClient httpClient,
        ILogger<CatalogClient> logger)
    {
        _httpClient = httpClient;
        _logger = logger;
    }

    public async Task<ProductDto?> GetProductAsync(
        int productId,
        CancellationToken cancellationToken)
    {
        using var response = await _httpClient.GetAsync(
            $"api/products/{productId}",
            cancellationToken);

        if (response.StatusCode == HttpStatusCode.NotFound)
        {
            _logger.LogInformation(
                "Product {ProductId} was not found in Catalog API.",
                productId);

            return null;
        }

        response.EnsureSuccessStatusCode();

        return await response.Content.ReadFromJsonAsync<ProductDto>(
            cancellationToken);
    }
}

Do not log secrets, authorization headers, cookies, or full payloads without careful redaction.

Redacting Sensitive Headers

HTTP logs can accidentally include sensitive data.

Sensitive headers include:

• Authorization
• Cookie
• Set-Cookie
• API keys
• custom token headers
• session headers

Use redaction when configuring clients.

Example:

builder.Services.AddHttpClient("ExternalApi", client =>
{
    client.BaseAddress = new Uri("https://api.example.com/");
})
.RedactLoggedHeaders("Authorization", "Cookie", "X-Api-Key");

Even with redaction, be careful with request and response body logging.

Configuration with Options Pattern

External API settings should usually come from configuration.

Configuration:

{
  "ExternalApis": {
    "Catalog": {
      "BaseAddress": "https://catalog.example.com/",
      "TimeoutSeconds": 20
    }
  }
}

Options class:

public sealed class CatalogApiOptions
{
    public string BaseAddress { get; set; } = string.Empty;
    public int TimeoutSeconds { get; set; } = 20;
}

Registration:

builder.Services.Configure<CatalogApiOptions>(
    builder.Configuration.GetSection("ExternalApis:Catalog"));

builder.Services.AddHttpClient<CatalogClient>((serviceProvider, client) =>
{
    var options = serviceProvider
        .GetRequiredService<IOptions<CatalogApiOptions>>()
        .Value;

    client.BaseAddress = new Uri(options.BaseAddress);
    client.Timeout = TimeSpan.FromSeconds(options.TimeoutSeconds);
});

Benefits:

• Environment-specific configuration.
• No hard-coded URLs.
• Easier local/staging/production setup.
• Easier testing.
• Central validation possible.

Add options validation for production systems.

Authentication and Authorization Headers

Service-to-service HTTP often requires authentication.

Example API key handler:

public sealed class ApiKeyHandler : DelegatingHandler
{
    private readonly IOptions<ExternalApiOptions> _options;

    public ApiKeyHandler(IOptions<ExternalApiOptions> options)
    {
        _options = options;
    }

    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        request.Headers.Add("X-Api-Key", _options.Value.ApiKey);

        return base.SendAsync(request, cancellationToken);
    }
}

Registration:

builder.Services.AddTransient<ApiKeyHandler>();

builder.Services.AddHttpClient("ExternalApi", client =>
{
    client.BaseAddress = new Uri("https://api.example.com/");
})
.AddHttpMessageHandler<ApiKeyHandler>();

For OAuth/JWT tokens, use a token provider and cache tokens safely. Avoid requesting a new token for every outbound request if token reuse is allowed.

Request and Response Disposal

Dispose HttpResponseMessage when using SendAsync, GetAsync, PostAsync, and similar methods that return a response.

Example:

using var response = await _httpClient.SendAsync(
    request,
    cancellationToken);

response.EnsureSuccessStatusCode();

When using convenience methods such as GetFromJsonAsync, disposal is handled internally.

Disposing the response helps release resources associated with the response content stream.

For large responses, consider streaming.

using var response = await _httpClient.GetAsync(
    "api/export",
    HttpCompletionOption.ResponseHeadersRead,
    cancellationToken);

response.EnsureSuccessStatusCode();

await using var stream = await response.Content
    .ReadAsStreamAsync(cancellationToken);

ResponseHeadersRead avoids buffering the whole response before returning.

Testing Outbound HTTP

Do not call real external services in normal unit tests.

Better options:

• Mock the typed client interface.
• Fake HttpMessageHandler.
• Use a local test server.
• Use WireMock.Net or similar fake server.
• Use WebApplicationFactory for in-process test APIs.
• Use contract tests for provider/consumer contracts.
• Use integration tests for real external service only when appropriate.

If application code depends on an interface:

public interface ICatalogClient
{
    Task<ProductDto?> GetProductAsync(
        int productId,
        CancellationToken cancellationToken);
}

Unit test can mock it:

var catalogClient = new Mock<ICatalogClient>();

catalogClient
    .Setup(client => client.GetProductAsync(1, It.IsAny<CancellationToken>()))
    .ReturnsAsync(new ProductDto(1, "Keyboard"));

var service = new ProductService(catalogClient.Object);

This tests application logic without HTTP.

Testing HttpClient with a Fake Handler

For typed client tests, fake the HttpMessageHandler.

public sealed class FakeHttpMessageHandler : HttpMessageHandler
{
    private readonly Func<HttpRequestMessage, HttpResponseMessage> _handler;

    public FakeHttpMessageHandler(
        Func<HttpRequestMessage, HttpResponseMessage> handler)
    {
        _handler = handler;
    }

    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        return Task.FromResult(_handler(request));
    }
}

Test:

[Fact]
public async Task GetProductAsync_WhenApiReturnsProduct_ReturnsProduct()
{
    var handler = new FakeHttpMessageHandler(request =>
    {
        Assert.Equal(HttpMethod.Get, request.Method);
        Assert.Equal("/api/products/1", request.RequestUri?.AbsolutePath);

        return new HttpResponseMessage(HttpStatusCode.OK)
        {
            Content = JsonContent.Create(new ProductDto(1, "Keyboard"))
        };
    });

    var httpClient = new HttpClient(handler)
    {
        BaseAddress = new Uri("https://catalog.example.com")
    };

    var client = new CatalogClient(httpClient);

    var product = await client.GetProductAsync(1, CancellationToken.None);

    Assert.NotNull(product);
    Assert.Equal("Keyboard", product.Name);
}

This avoids real network calls and tests request construction and response handling.

Testing Resilience Behavior

Resilience tests can be tricky because retries and timeouts involve time and multiple attempts.

Example fake handler that fails twice then succeeds:

public sealed class SequenceHandler : HttpMessageHandler
{
    private int _attempt;

    protected override Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        _attempt++;

        if (_attempt <= 2)
        {
            return Task.FromResult(
                new HttpResponseMessage(HttpStatusCode.ServiceUnavailable));
        }

        return Task.FromResult(
            new HttpResponseMessage(HttpStatusCode.OK)
            {
                Content = JsonContent.Create(new { Message = "OK" })
            });
    }
}

For full resilience pipeline tests, consider using a test server or fake HTTP server and keep timeouts short. Avoid slow tests that wait for real long timeouts.

Test most business logic separately. Test only a small number of resilience scenarios.

IHttpClientFactory in Integration Tests

When testing ASP.NET Core apps with WebApplicationFactory, replace outbound HTTP clients with fakes.

Example:

factory.WithWebHostBuilder(builder =>
{
    builder.ConfigureTestServices(services =>
    {
        services.AddHttpClient("CatalogApi")
            .ConfigurePrimaryHttpMessageHandler(() =>
            {
                return new FakeHttpMessageHandler(request =>
                {
                    return new HttpResponseMessage(HttpStatusCode.OK)
                    {
                        Content = JsonContent.Create(
                            new ProductDto(1, "Test Product"))
                    };
                });
            });
    });
});

This lets the API under test use its real code path while the external dependency is controlled.

Common Mistakes

Common mistakes include:

• Creating a new manual HttpClient per request.
• Keeping a static HttpClient forever without PooledConnectionLifetime.
• Capturing a typed client inside a singleton service.
• Retrying non-idempotent operations blindly.
• Retrying all HTTP status codes.
• Using very long default timeouts.
• Not passing cancellation tokens.
• Not disposing HttpResponseMessage.
• Logging authorization headers or cookies.
• Storing request-specific state inside a long-lived handler.
• Using IHttpClientFactory with cookies without understanding handler pooling.
• Creating unbounded named clients.
• Adding multiple resilience handlers accidentally.
• Stacking retries at multiple layers.
• Retrying inside both gateway, service, and client without a retry budget.
• Swallowing all HTTP errors and returning fake success.
• Using EnsureSuccessStatusCode when business-specific handling is needed.
• Not testing error responses.
• Calling real external services in unit tests.
• Using Task.Result or .Wait() on async HTTP calls.
• Forgetting to set BaseAddress for typed clients.
• Hard-coding external URLs in client classes.
• Ignoring 429 rate limit responses.
• Not observing downstream latency and failure rates.

Best Practices

Use IHttpClientFactory for DI-based applications that call external HTTP services.

Prefer named or typed clients over scattered raw HttpClient usage.

Use typed clients to encapsulate each external API.

Use named clients when a singleton service needs to create clients repeatedly.

Configure base address, timeout, headers, and handlers centrally.

Use SocketsHttpHandler.PooledConnectionLifetime when using long-lived clients or when you need explicit DNS refresh behavior.

Set sensible timeouts.

Pass cancellation tokens.

Use resilience policies for transient failures.

Disable or carefully design retries for unsafe HTTP methods.

Use idempotency keys for retried write operations when supported.

Use circuit breakers and rate limiting for unstable dependencies.

Use hedging only for safe read scenarios where extra traffic is acceptable.

Keep delegating handlers stateless and focused.

Avoid leaking scoped or user-specific data through long-lived handlers.

Avoid IHttpClientFactory for cookie-heavy session scenarios unless carefully configured.

Log and trace outbound HTTP calls.

Redact sensitive headers and payloads.

Test typed clients with fake handlers or fake servers.

Replace external clients in integration tests.

Treat outbound HTTP as a production boundary that needs design, observability, and failure handling.