Let's assume we have a WCF service that has a method to get some data.
For example, it returns a Product object
But to get product information we need to make calls to some external systems (query the database or make a call to another web service etc.). For the sake of simplicity, let's say we have one external web service that we need to call to get product Name, Price and Category:
So to get full product information we need to make three calls to the external service.
The problem with the external service is that it's a bit slow. Here is the implementation:
As you can see, all operations are delayed for 1-2 seconds for illustrative purposes to simulate a busy system.
So how do we call the external service from our ProductService? If we make three calls sequentially, that will be really slow (4.5 seconds). How can we improve this time? Let's make three calls in parallel, then collect the results and return the product back to the client:
Looks good. We use Tasks, a very powerful feature of .NET 4.0. Each Task makes a call to the external service. Then we wait for all tasks to finish and return the data.
GetProduct method call takes about two seconds, life is good. What else to write about here?
The post is called "The Power of Async" but the above code doesn't use any async features. It works on .NET 4.0 and there is not much we can do to improve it (at least without losing its readability).
Now let's see how .NET 4.5 can make our life even better.
Let's create a new service method GetProduct2 by copying the existing method, make it async and change one line of code:
The only difference with the first version of the method is that instead of "Task.WaitAll" we are using "await Task.WhenAll" (and we also marked the method as async). Now it's time to see what both methods look like in action. Let's try and send 100 request to our server at the same time and see how they perform. For that I have created a simple client program calls the service 100 times are waits for all of them to complete.
Please note that even though we do use some async stuff in our client program, it has nothing to do with the server implementation. The same client features are available to us no matter what the service is written in, be it C#, Java or PHP.
Let's now run the client program, open Task Manager and see how many threads our WCF service creates:
In this series of runs I only measured maximum number of threads created in the iisexpress process.
Before running any of the test I always killed the iisexpress processes and started them again to exclude any side effects. While running the client program (it takes about 40 seconds to complete) I was looking at the number of threads in the Task Manager. The maximum value for the GetProduct method (which uses Task.WaitAll) was 121 threads.
Not bad. Now let's modify our client program to call the GetProduct2 (which uses "await Tasks.WhenAll") method:
And the result is... 63 threads. Wow. That's a good improvement for two lines of code.
How is that possible? The short story is WaitAll blocks the current thread while we are waiting for the response from the external service. And this blocking makes ThreadPool create a new thread in order for the product service to continue working. But WhenAll doesn't block the current thread. Instead it asks it to do other work in the meantime (for example process another request).
This is a very short explanation and I will try to elaborate on it in the next post.
Okay, we have dropped the number of threads almost two times by using the new async feature of C# 5. Can we do even better than that? The answer is yes.
While our new code doesn't block threads when waiting for product tasks to complete, we still make calls to the external service in a synchronous manner which, as you may have already guessed, leads to the current thread being blocked.
How can we avoid this blocking? By using the new client API from .NET 4.5 for calling a web service (actually the same as we used in the client program).
Let's add three new methods to call the external service. And this time let's make them async:
Again, these methods don't care about the external service implementation at all. It could be anything conforming to web service standards.
And finally let's add new version of the GetProduct method, GetProduct3 that calls these new async methods:
Here we are using the ContinueWith method that accepts an Action that should be executed once the task completes. And of course it does that in a non-blocking way.
Let's change our client program to use GetProduct3 and run it... 54 threads. That's impresive.
Also, all client calls were completed after 21 seconds, which is twice as fast as the previous two versions.
After that I repeated all the tests for 1000 requests. Here is a summary table with the measures:
N = 100
N = 1000
As we can see, using new async features helped us reduce the number of threads almost twice. And using the new API for calling web services improved the total processing time dramatically (which probably means the average request processing time has improved greatly but to be completely sure I still have to do more detailed analysis).
In the next post I will try to use more advanced tools such as Performance Monitor to see what's happening under the hood. And explain the results by looking at how exactly the new async features work.
The code for this post is available here.
For example, it returns a Product object
public class Product
{
public int Id { get; set; }
public string Name { get; set; }
public long Price { get; set; }
public string Category { get; set; }
}
[ServiceContract]
public interface IProductService
{
[OperationContract]
Product GetProduct(int id);
}
But to get product information we need to make calls to some external systems (query the database or make a call to another web service etc.). For the sake of simplicity, let's say we have one external web service that we need to call to get product Name, Price and Category:
[ServiceContract]
public interface IExternalService
{
[OperationContract]
string GetProductCategory(int id);
[OperationContract]
long GetProductPrice(int id);
[OperationContract]
string GetProductName(int id);
}
So to get full product information we need to make three calls to the external service.
The problem with the external service is that it's a bit slow. Here is the implementation:
public class ExternalService : IExternalService
{
public string GetProductCategory(int id)
{
Thread.Sleep(1000);
return "Category " + id;
}
public long GetProductPrice(int id)
{
Thread.Sleep(2000);
return id * 1000;
}
public string GetProductName(int id)
{
Thread.Sleep(1500);
return "Name " + id;
}
}
As you can see, all operations are delayed for 1-2 seconds for illustrative purposes to simulate a busy system.
So how do we call the external service from our ProductService? If we make three calls sequentially, that will be really slow (4.5 seconds). How can we improve this time? Let's make three calls in parallel, then collect the results and return the product back to the client:
public class ProductService : IProductService
{
public Product GetProduct(int id)
{
var p = new Product();
var nameTask = new Task(() => { p.Name = GetProductName(id); });
var priceTask = new Task(() => { p.Price = GetProductPrice(id); });
var categoryTask = new Task(() => { p.Category = GetProductCategory(id); });
nameTask.Start();
priceTask.Start();
categoryTask.Start();
Task.WaitAll(nameTask, priceTask, categoryTask);
return p;
}
private string GetProductCategory(int id)
{
return new ExternalServiceClient().GetProductCategory(id);
}
private long GetProductPrice(int id)
{
return new ExternalServiceClient().GetProductPrice(id);
}
private string GetProductName(int id)
{
return new ExternalServiceClient().GetProductName(id);
}
}
Looks good. We use Tasks, a very powerful feature of .NET 4.0. Each Task makes a call to the external service. Then we wait for all tasks to finish and return the data.
GetProduct method call takes about two seconds, life is good. What else to write about here?
The post is called "The Power of Async" but the above code doesn't use any async features. It works on .NET 4.0 and there is not much we can do to improve it (at least without losing its readability).
Now let's see how .NET 4.5 can make our life even better.
Let's create a new service method GetProduct2 by copying the existing method, make it async and change one line of code:
public async Task<Product> GetProduct2(int id)
{
var p = new Product { Id = id };
var nameTask = new Task(() => { p.Name = GetProductName(id); });
var priceTask = new Task(() => { p.Price = GetProductPrice(id); });
var categoryTask = new Task(() => { p.Category = GetProductCategory(id); });
nameTask.Start();
priceTask.Start();
categoryTask.Start();
await Task.WhenAll(nameTask, priceTask, categoryTask);
return p;
}
The only difference with the first version of the method is that instead of "Task.WaitAll" we are using "await Task.WhenAll" (and we also marked the method as async). Now it's time to see what both methods look like in action. Let's try and send 100 request to our server at the same time and see how they perform. For that I have created a simple client program calls the service 100 times are waits for all of them to complete.
static void Main(string[] args)
{
var totalCompleted = 0; // number of requests completed
const int N = 100; // number of requests to send
var tasks = new Task<Product>[N];
var sw = new Stopwatch();
sw.Start();
for (int i = 0; i < N; i++)
{
// we are using the -Async version of the method because we want to send all requests at once
var productTask = new ProductServiceClient().GetProductAsync(i);
// when the request is completed (we have the response)
productTask.ContinueWith(_ =>
{
// increment the number of requests completed
Interlocked.Increment(ref totalCompleted);
// output the results
Console.WriteLine("Total: {0}; Completed for: {1}", totalCompleted, _.Result.Name);
});
tasks[i] = productTask;
Console.WriteLine(i);
}
// when all tasks are completed output the elapsed time
Task.WhenAll(tasks).ContinueWith(_ => Console.WriteLine("Completed! {0}", sw.Elapsed));
Console.ReadLine();
}
Please note that even though we do use some async stuff in our client program, it has nothing to do with the server implementation. The same client features are available to us no matter what the service is written in, be it C#, Java or PHP.
Let's now run the client program, open Task Manager and see how many threads our WCF service creates:
In this series of runs I only measured maximum number of threads created in the iisexpress process.
Before running any of the test I always killed the iisexpress processes and started them again to exclude any side effects. While running the client program (it takes about 40 seconds to complete) I was looking at the number of threads in the Task Manager. The maximum value for the GetProduct method (which uses Task.WaitAll) was 121 threads.
Not bad. Now let's modify our client program to call the GetProduct2 (which uses "await Tasks.WhenAll") method:
var productTask = new ProductServiceClient().GetProduct2Async(i);
And the result is... 63 threads. Wow. That's a good improvement for two lines of code.
How is that possible? The short story is WaitAll blocks the current thread while we are waiting for the response from the external service. And this blocking makes ThreadPool create a new thread in order for the product service to continue working. But WhenAll doesn't block the current thread. Instead it asks it to do other work in the meantime (for example process another request).
This is a very short explanation and I will try to elaborate on it in the next post.
Okay, we have dropped the number of threads almost two times by using the new async feature of C# 5. Can we do even better than that? The answer is yes.
While our new code doesn't block threads when waiting for product tasks to complete, we still make calls to the external service in a synchronous manner which, as you may have already guessed, leads to the current thread being blocked.
How can we avoid this blocking? By using the new client API from .NET 4.5 for calling a web service (actually the same as we used in the client program).
Let's add three new methods to call the external service. And this time let's make them async:
private async Task<string> GetProductCategoryAsync(int id)
{
return await new ExternalServiceClient().GetProductCategoryAsync(id);
}
private async Task<long> GetProductPriceAsync(int id)
{
return await new ExternalServiceClient().GetProductPriceAsync(id);
}
private async Task<string> GetProductNameAsync(int id)
{
return await new ExternalServiceClient().GetProductNameAsync(id);
}
Again, these methods don't care about the external service implementation at all. It could be anything conforming to web service standards.
And finally let's add new version of the GetProduct method, GetProduct3 that calls these new async methods:
public async Task<Product> GetProduct3(int id)
{
var p = new Product();
var nameTask = GetProductNameAsync(id).ContinueWith(t => p.Name = t.Result);
var priceTask = GetProductPriceAsync(id).ContinueWith(t => p.Price = t.Result);
var categoryTask = GetProductCategoryAsync(id).ContinueWith(t => p.Category = t.Result);
await Task.WhenAll(nameTask, priceTask, categoryTask);
return p;
}
Here we are using the ContinueWith method that accepts an Action that should be executed once the task completes. And of course it does that in a non-blocking way.
Let's change our client program to use GetProduct3 and run it... 54 threads. That's impresive.
Also, all client calls were completed after 21 seconds, which is twice as fast as the previous two versions.
After that I repeated all the tests for 1000 requests. Here is a summary table with the measures:
N = 100
| Max # of Threads | Total Time | |
| GetProduct (Task.WaitAll) | 121 | 00:43 |
| GetProduct2 (Task.WhenAll) | 63 | 00:42 |
| GetProduct3 (Task.WhenAll + Async service call) | 54 | 00:21 |
N = 1000
| Max # of Threads | Total Time | |
| GetProduct (Task.WaitAll) | 139 | 03:08 |
| GetProduct2 (Task.WhenAll) | 80 | 02:54 |
| GetProduct3 (Task.WhenAll + Async service call) | 53 | 01:27 |
As we can see, using new async features helped us reduce the number of threads almost twice. And using the new API for calling web services improved the total processing time dramatically (which probably means the average request processing time has improved greatly but to be completely sure I still have to do more detailed analysis).
In the next post I will try to use more advanced tools such as Performance Monitor to see what's happening under the hood. And explain the results by looking at how exactly the new async features work.
The code for this post is available here.
