Limit web API response time
When an ASP.NET Core Web API REST service endpoint makes further calls to upstream REST services or interacts with other potentially slow resources, its response time strongly depends on the response times of those dependencies. Of course, you can always set a timeout for individual outgoing calls if you don't want them to take too long. But what if you want to set some time limit for your endpoint to respond instead?
Let's say your endpoint makes two outgoing calls:
[ApiController]
[Route("[controller]")]
public class SubmissionsController() : ControllerBase
{
[HttpPost]
public async Task<IActionResult> Create()
{
var submission = await submissionsService.CreateAsync();
return Ok(submission);
}
}
public class SubmissionsService
{
public async Task<Submission> CreateAsync()
{
var submissionId = Guid.NewGuid();
var phase1CompletedAt = await client.ProcessPhase1Async(submissionId);
var phase2CompletedAt = await client.ProcessPhase2Async(submissionId);
var submission = new Submission(submissionId, phase1CompletedAt, phase2CompletedAt);
repository.Add(submission);
return submission;
}
}
With this simple implementation, the total response time would be the sum of the response times of the calls it makes.
The simplest way to ensure that the endpoint would respond within a fixed time limit, no matter how long the outgoing calls take, would be to wait for the calls to complete only for the given time limit. If they don't complete in time, the method should return anyway:
public async Task<Guid> CreateAsync()
{
var submission = new Submission(Guid.NewGuid(), null, null);
repository.Add(submission);
Task[] tasks = [ProcessAsync(submission), Task.Delay(TimeSpan.FromSeconds(1))];
await Task.WhenAny(tasks);
return submission.Id;
}
private async Task ProcessAsync(Submission submission)
{
var phase1CompletedAt = await client.ProcessPhase1Async(submission.Id);
submission = submission with { Phase1CompletedAt = phase1CompletedAt };
repository.Add(submission);
var phase2CompletedAt = await client.ProcessPhase2Async(submission.Id);
submission = submission with { Phase2CompletedAt = phase2CompletedAt };
repository.Add(submission);
}
As you can see, I moved the outgoing calls into a separate method, so that I could call in parallel with a fixed Task.Delay call. The Task.WhenAny method will complete as soon as one of the provided tasks completes. It's going to be our method only if it completes within the time limit.
Since the processed object is being persisted anyway, I simplified my code by returning its Id only instead. With it, I can then retrieve the full object from the repository in the controller:
[HttpPost]
public async Task<IActionResult> Create()
{
var submissionId = await submissionsService.CreateAsync();
var submission = submissionsRepository.Get(submissionId);
if (!submission!.Phase2CompletedAt.HasValue)
{
return Accepted(submission);
}
return Ok(submission);
}
I use the Phase2CompletedAt as an indicator whether the processing completed in time or not. Depending on that, I return a different status code to inform the client that the processing did not yet complete. This allows it to retrieve the final state from a different endpoint using the Id:
[HttpGet("{id:guid}")]
public IActionResult Get(Guid id)
{
var submission = submissionsRepository.Get(id);
if (submission == null)
{
return NotFound();
}
return Ok(submission);
}
With this approach, the processing continues in the background when it doesn't complete in time, even though the response has already been returned to the client. In many cases, this might be okay or even desired behavior.
But what if you want to interrupt the processing when it didn't complete in time. And what if you only want to do that unless it already performed some side effects which cannot be (easily) reverted. You can achieve that, if you combine the above approach with a cancellation token:
public async Task<Guid> CreateAsync()
{
var submission = new Submission(Guid.NewGuid(), null, null);
var cancellationTokenSource = new CancellationTokenSource();
cancellationTokenSource.CancelAfter(TimeSpan.FromSeconds(1));
Task[] tasks =
[
ProcessAsync(submission, cancellationTokenSource.Token),
Task.Delay(TimeSpan.FromSeconds(1), CancellationToken.None),
];
await Task.WhenAny(tasks);
return submission.Id;
}
private async Task ProcessAsync(Submission submission, CancellationToken cancellationToken)
{
var phase1CompletedAt = await client.ProcessPhase1Async(
submission.Id,
cancellationToken
);
submission = submission with { Phase1CompletedAt = phase1CompletedAt };
repository.Add(submission);
var phase2CompletedAt = await client.ProcessPhase2Async(
submission.Id,
CancellationToken.None
);
submission = submission with { Phase2CompletedAt = phase2CompletedAt };
repository.Add(submission);
}
Notice how I created a CancellationTokenSource which will cancel after a given period of time and passed it to the calls which I want to interrupt if they don't complete in time. In my case, that's the phase one processing. If that one completes before the time limit, I want the phase two to continue even if it's over the limit. That's why I'm passing it a CancellationToken.None instead. I kept the same approach from before to ensure a timely response in any case.
To give the client correct information about what happened to the call, I had to further expand the controller:
[HttpPost]
public async Task<IActionResult> Create()
{
var submissionId = await submissionsService.CreateAsync();
var submission = submissionsRepository.Get(submissionId);
if (submission == null)
{
return StatusCode((int)HttpStatusCode.GatewayTimeout);
}
if (!submission.Phase2CompletedAt.HasValue)
{
return Accepted(submission);
}
return Ok(submission);
}
I'm now additionally checking if the processed object is even in the repository. Since I'm adding it to the repository after phase one completes, it's only going to be there if that happened within the time limit. If not, the repository will return null, because it can't find an object with the given Id. In such a case, the endpoint returns an error indicating that the upstream call took too long and the request therefore failed.
You can find a working sample project in my GitHub repository. Individual commits contain the three different implementations described above:
- the first one without a time limit for the endpoint,
- the second one which always continues execution in the background if it takes too long,
- and the last one which only continues execution if phase one completes in time, otherwise it aborts the operation.
Each implementation also has tests which demonstrate the described behavior using WireMock.Net to simulate different response times of called REST services. Since they depend on timing, they might be flaky on slower machines and when debugger is attached.
Although .NET provides tools for handling long-running operations in different ways, employing those still isn't a trivial task. It makes your code more complicated, and it's easy to get wrong unless you test the implementation thoroughly.
