Type issues with data driven tests in MSTest

September 2nd 2022 .NET Unit Testing

Data-driven tests are great for repeating the same unit tests with many different inputs. Here is an example of a test for a method that can convert temperatures between different temperature units:

[DataTestMethod]
[DataRow(Celsius, Celsius, 27, 27)]
[DataRow(Celsius, Fahrenheit, 27, 80.6)]
[DataRow(Celsius, Kelvin, 27, 300.15)]
[DataRow(Fahrenheit, Celsius, 80, 26.67)]
[DataRow(Fahrenheit, Fahrenheit, 80, 80)]
[DataRow(Fahrenheit, Kelvin, 80, 299.82)]
[DataRow(Kelvin, Celsius, 300, 26.85)]
[DataRow(Kelvin, Fahrenheit, 300, 80.33)]
[DataRow(Kelvin, Kelvin, 300, 300)]
public void ConvertsCorrectly(
    TemperatureUnit from,
    TemperatureUnit to,
    double value,
    double expected)
{
    var result = TemperatureConverter.Convert(value, from, to);

    result.Should().BeApproximately(expected, 0.005);
}

To my surprise, a similar test from a project I worked on failed on multiple test cases with the following error message:

Test method DataDrivenTestTyping.MsTest.TemperatureConverterTests.ConvertsCorrectly threw exception: System.ArgumentException: Object of type 'System.Decimal' cannot be converted to type 'System.Double'.

The error was limited to tests with non-integer input values, but the error still did not make much sense because both the literal values in the attributes and the data parameters were of type double.

I could not find a way to get the test framework to pass all numeric values to test parameters with the correct type. Since the DataRowAttribute parameters are of type object, I found a solution that allowed me to explicitly convert the values to the correct type:

[DataTestMethod]
[DataRow(Celsius, Celsius, 27, 27)]
[DataRow(Celsius, Fahrenheit, 27, 80.6)]
[DataRow(Celsius, Kelvin, 27, 300.15)]
[DataRow(Fahrenheit, Celsius, 80, 26.67)]
[DataRow(Fahrenheit, Fahrenheit, 80, 80)]
[DataRow(Fahrenheit, Kelvin, 80, 299.82)]
[DataRow(Kelvin, Celsius, 300, 26.85)]
[DataRow(Kelvin, Fahrenheit, 300, 80.33)]
[DataRow(Kelvin, Kelvin, 300, 300)]
public void ConvertsCorrectly(
    TemperatureUnit from,
    TemperatureUnit to,
    object value,
    object expected)
{
    ConvertsCorrectly(from, to, Convert.ToDouble(value), Convert.ToDouble(expected));
}

public void ConvertsCorrectly(
    TemperatureUnit from,
    TemperatureUnit to,
    double value,
    double expected)
{
    var result = TemperatureConverter.Convert(value, from, to);

    result.Should().BeApproximately(expected, 0.005);
}

I was not happy, but at least it worked.

Then I decided to try upgrading MSTest to the latest version. And it helped: the problem was gone. All test cases worked, even with the original code before my workaround.

With bisection, I determined that the problem was fixed in version 2.2.5. This was the change that fixed the problem.

I have posted a sample project with this problem to my GitHub repository. Various commits show:

  • the original problem with MSTest 2.2.4,
  • the workaround for the problem with MSTest 2.2.4, and
  • the original code that works with MSTest 2.2.5.

I could have solved my problem much sooner if I had updated the library to the latest version before I started doubting the test code that should have worked and looking for workarounds. I'll try to remember that the next time something similar happens to me.

If you're looking for online one-on-one mentorship on a related topic, you can find me on Codementor.
If you need a team of experienced software engineers to help you with a project, contact us at Razum.
Copyright
Creative Commons License