Message Retry Configuration

Some exceptions may be caused by a transient condition, such as a database deadlock, a busy web service, or some similar type of situation that usually clears up on a second attempt. With these exception types, it is often desirable to retry the message delivery to the consumer, allowing the consumer to try the operation again.

public class SubmitOrderConsumer :
    IConsumer<SubmitOrder>
{
    ISessionFactory _sessionFactory;

    public async Task Consume(ConsumeContext<SubmitOrder> context)
    {
        using(var session = _sessionFactory.OpenSession())
        using(var transaction = session.BeginTransaction())
        {
            var customer = session.Get<Customer>(context.Message.CustomerId);

            // continue with order processing

            transaction.Commit();
        }
    }
}

Configure message retry

With this consumer, an ADOException can be thrown, say there is a deadlock or the SQL server is unavailable. In this case, the operation should be retried before moving the message to the error queue. This can be configured on the receive endpoint or the consumer. Shown below is a retry policy which attempts to deliver the message to a consumer five times before throwing the exception back up the pipeline.

services.AddMassTransit(x =>
{
    x.AddConsumer<SubmitOrderConsumer>();

    x.AddConfigureEndpointsCallback((context,name,cfg) =>
    {
        cfg.UseMessageRetry(r => r.Immediate(5));
    });

    x.UsingRabbitMq((context,cfg) =>
    {
        cfg.ConfigureEndpoints(context);
    });
});

The UseMessageRetry method is an extension method that configures a middleware filter, in this case the RetryFilter. There are a variety of retry policies available, which are detailed in the section below.

Note

UseMessageRetry can be called at the bus or the receive endpoint level. When using ConfigureEndpoints, it’s best to call it at the receive endpoint level to ensure the proper filter ordering when combined with other filters, such as the outbox or message redelivery.

To configure retry on a manually configured receive endpoint:

services.AddMassTransit(x =>
{
    x.AddConsumer<SubmitOrderConsumer>();

    x.UsingRabbitMq((context, cfg) =>
    {
        cfg.ReceiveEndpoint("submit-order", e =>
        {
            e.UseMessageRetry(r => r.Immediate(5));

            e.ConfigureConsumer<SubmitOrderConsumer>(context);
        });
    });
});

MassTransit retry filters execute in memory and maintain a lock on the message. As such, they should only be used to handle short, transient error conditions. Setting a retry interval of an hour would fall into the category of bad things. To retry messages after longer waits, look at the next section on redelivering messages. For example, if a consumer with a concurrency limit of 5 and a retry interval of one hour consumes 5 messages that causes retries, the consumer will be effectively stalled for a whole hour as all the concurrent message slots are in use waiting for the retry interval.

Configure a retry policy

YouTube

Learn more about message retry in this video.

When configuring message retry, there are several retry policies available, including:

Policy	Description
None	No retry
Immediate	Retry immediately, up to the retry limit
Interval	Retry after a fixed delay, up to the retry limit
Intervals	Retry after a delay, for each interval specified
Exponential	Retry after an exponentially increasing delay, up to the retry limit
Incremental	Retry after a steadily increasing delay, up to the retry limit

Each policy has configuration settings which specifies the expected behavior.

Configure any exception filters

Sometimes you do not want to always retry, but instead only retry when some specific exception is thrown and fault for all other exceptions. To implement this, you can use an exception filter. Specify exception types using either the Handle or Ignore method. A filter can have either Handle or Ignore statements, combining them has unpredictable effects.

Both methods have two signatures:

1. Generic version Handle<T> and Ignore<T> where T must be derivative of System.Exception. With no filter expression, all exceptions of the specified type will be either handled or ignored. You can also specify a function argument that will filter exceptions further based on other parameters.

2. Non-generic version that needs one or more exception types as parameters. No further filtering is possible if this version is used.

You can use multiple calls to these methods to specify filters for multiple exception types:

e.UseMessageRetry(r =>
{
    r.Handle<ArgumentNullException>();
    r.Ignore(typeof(InvalidOperationException), typeof(InvalidCastException));
    r.Ignore<ArgumentException>(t => t.ParamName == "orderTotal");
});

Configure multiple retry policies

You can also specify multiple retry policies for a single endpoint:

services.AddMassTransit(x =>
{
    x.AddConsumer<SubmitOrderConsumer>();

    x.UsingRabbitMq((context, cfg) =>
    {
        cfg.ReceiveEndpoint("submit-order", e =>
        {
            e.UseMessageRetry(r =>
            {
                r.Immediate(5);
                r.Handle<DataException>(x => x.Message.Contains("SQL"));
            });

            e.ConfigureConsumer<SubmitOrderConsumer>(context, c => c.UseMessageRetry(r =>
            {
                r.Interval(10, TimeSpan.FromMilliseconds(200));
                r.Ignore<ArgumentNullException>();
                r.Ignore<DataException>(x => x.Message.Contains("SQL"));
            }));
        });
    });
});

In the above example, if the consumer throws an ArgumentNullException it won’t be retried (because it would obvious fail again, most likely). If a DataException is thrown matching the filter expression, it wouldn’t be handled by the second retry filter, but would be handled by the first retry filter.