Support URI Patterns parser everywhere (#192)

Author: dnperfors

CHANGELOG.md

@@ -5,12 +5,26 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 ## [0.10] - unplanned
+### Deprecated
+- `ShouldHaveMadeRequestsTo(this TestableHttpMessageHandler, string, bool)` and `ShouldHaveMadeRequestsTo(this TestableHttpMessageHandler, string, bool, int)` have been deprecated. CaseInsensitivity is controlled by the `UriPatternMatchingOptions` that can be set on the `TestableHttpMessageHandler`.
+- `WithRequestUri(this IHttpRequestMessagesCheck, string, bool)` and `WithRequestUri(this IHttpRequestMessagesCheck, string, bool, int)` have been deprecated. CaseInsensitivity is controlled by the `UriPatternMatchingOptions` that can be set on the `TestableHttpMessageHandler`.
+- `WithQueryString` has been deprecated, since `ShouldHaveMadeRequestTo` and `WithRequestUri` now properly support querystrings.
+
 ### Removed
 - `TestableHttpMessageHandler.SimulateTimeout` has been removed, and can be replaced with `RespondWith(Responses.Timeout())`.
 - `TestableHttpMessageHandler.RespondWith(Func<HttpRequestMessage, HttpResponseMessage>)` has been removed, it's functionality is replaced by IResponse.
 - `RespondWith(this TestableHttpMessageHandler, HttpResponseMessage)` has been removed, the response is modified with every call, so it doesn't work reliably and is different from how HttpClientHandler works, which creates a HttpResponseMessage for every request.
 - `HttpResponseMessageBuilder` and `RespondWith(this TestableHttpMessageHandler, HttpResponseMessageBuilder)` has been removed, it's functionality can be replaced with ConfiguredResponse or a custom IResponse.
 
+### Added
+- URI patterns now support query parameters and by default will use the unescaped values, note that the order is still important.
+- URI pattern parsing is extended to be able to parse most URI's.
+
+### Changed
+- Use the same parser for the assertion methods `WithRequestUri` (which is used by `ShouldHaveMadeRequestsTo`) as for the RoutingResponse functionality.
+- `RouteParserException` has been renamed to `UriPatternParserException`.
+- Renamed `RoutingOptions` to `UriPatternMatchingOptions`.
+
 ## [0.9] - 2022-11-25
 ### Deprecated
 - `Responses.NoContent()` has been deprecated, since it doesn't fit well with the rest of the API. Please use `Responses.StatusCode(HttpStatusCode.NoContent)` instead.

README.md

@@ -36,6 +36,46 @@ testHandler.ShouldHaveMadeRequestsTo("https://httpbin.org/*");
 
 More examples can be found in the [IntegrationTests project](test/TestableHttpClient.IntegrationTests)
 
+## URI Patterns
+
+TestableHttpClient supports URI patterns in several places, mainly in:
+- Response routing, where an URI pattern is used for matching the request URI to a response
+- Assertions, where requests can be asserted against an URI pattern.
+
+URI patterns are based on URI's as specified in [RFC 3986](https://www.rfc-editor.org/rfc/rfc3986), but allow the wildcard character `*` to specify optional parts of an URI.
+
+An URI contains several components:
+- The scheme of an URI is optional, but when given it should end with `://`. When not given `*://` is assumed.
+- User Information (`username:password@`) is ignored and is not checked at all.
+- The host is optional and when not given `*` is assumed. Both IP addresses and registered names are supported.
+- The port is optional, but when ':' is provided after host, it should have a value.
+- The path is optional, but should start with a `/`. When `/` is given, it can be followed by a `*` to match it with any path.
+- Query parameters are optional, when given it should start with a `?`.
+- Fragments are ignored, but should start with a `#`.
+
+URI patterns differ from URI's in the following ways:
+- Any character is allowed, for example: `myhost:myport` is a valid URI pattern, but not a valid URI. (and this will never match).
+- No encoding is performed, and patterns are matched against the unescaped values of an URI.
+- Patterns are not normalized, so a path pattern `/test/../example` will not work.
+
+Some examples:
+
+Uri pattern | Matches
+------------|--------
+*|Matches any URL
+\*://\*/\*?\* | Matches any URL
+/get | Matches any URL that uses the path `/get`
+http*://* | Matches any URL that uses the scheme `http` or `https` (or any other scheme that starts with `http`)
+localhost:5000 | Matches any URL that uses localhost for the host and port 5000, no matter what scheme or path is used.
+
+## Controlling the behaviour of pattern matching
+
+TestableHttpClient has several options available that let you control different parts of the library. These options can be found on
+the `TestableHttpMessageHandler.Options` and are passed to the `HttpRepsonseContext` and the `IHttpRequestMessagesCheck`.
+The options include:
+- `JsonSerializerOptions` for controlling the serialization of json content
+- `UriPatternMatchingOptions` for controlling how the URI pattern matching works.
+
 ## Supported .NET versions
 
 TestableHttpClient is build as a netstandard2.0 library, so theoretically it can work on every .NET version that support netstandard2.0.

src/TestableHttpClient/HttpRequestMessageExtensions.cs

@@ -199,33 +199,6 @@ internal static bool HasContentHeader(this HttpRequestMessage httpRequestMessage
         return httpRequestMessage.Content.Headers.HasHeader(headerName, headerValue);
     }
 
-    /// <summary>
-    /// Determines whether the request uri matches a pattern.
-    /// </summary>
-    /// <param name="httpRequestMessage">A <see cref="HttpRequestMessage"/> to check the correct uri on.</param>
-    /// <param name="pattern">A pattern to match with the request uri, supports * as wildcards.</param>
-    /// <returns>true when the request uri matches the pattern; otherwise, false.</returns>
-    internal static bool HasMatchingUri(this HttpRequestMessage httpRequestMessage, string pattern, bool ignoreCase = true)
-    {
-        if (httpRequestMessage == null)
-        {
-            throw new ArgumentNullException(nameof(httpRequestMessage));
-        }
-
-        if (httpRequestMessage.RequestUri == null)
-        {
-            return false;
-        }
-
-        return pattern switch
-        {
-            null => throw new ArgumentNullException(nameof(pattern)),
-            "" => false,
-            "*" => true,
-            _ => StringMatcher.Matches(httpRequestMessage.RequestUri.AbsoluteUri, pattern, ignoreCase),
-        };
-    }
-
     /// <summary>
     /// Determines whether the request has content.
     /// </summary>
@@ -280,6 +253,7 @@ internal static bool HasContent(this HttpRequestMessage httpRequestMessage, stri
     /// <param name="httpRequestMessage">A <see cref="HttpRequestMessage"/> to check the correct request uir querystring on.</param>
     /// <param name="pattern">A pattern to match the request uri querystring, supports * as wildcards.</param>
     /// <returns>true when the request uri querystring matches the pattern; otherwise, false.</returns>
+    [Obsolete("Use WithRequestUri instead, since it now properly supports QueryStrings as well")]
     internal static bool HasQueryString(this HttpRequestMessage httpRequestMessage, string pattern)
     {
         if (httpRequestMessage == null)

src/TestableHttpClient/HttpRequestMessagesCheckExtensions.cs

@@ -8,8 +8,9 @@ public static class HttpRequestMessagesCheckExtensions
     /// <param name="check">The implementation that hold all the request messages.</param>
     /// <param name="pattern">The uri pattern that is expected.</param>
     /// <returns>The <seealso cref="IHttpRequestMessagesCheck"/> for further assertions.</returns>
-    public static IHttpRequestMessagesCheck WithRequestUri(this IHttpRequestMessagesCheck check, string pattern) => WithRequestUri(check, pattern, true, null);
+    public static IHttpRequestMessagesCheck WithRequestUri(this IHttpRequestMessagesCheck check, string pattern) => WithRequestUri(check, pattern, null);
 
+    [Obsolete("Please use an overload without the 'ignoreCase', since ignoring casing is now controlled globally.")]
     public static IHttpRequestMessagesCheck WithRequestUri(this IHttpRequestMessagesCheck check, string pattern, bool ignoreCase) => WithRequestUri(check, pattern, ignoreCase, null);
 
     /// <summary>
@@ -19,9 +20,35 @@ public static class HttpRequestMessagesCheckExtensions
     /// <param name="pattern">The uri pattern that is expected.</param>
     /// <param name="expectedNumberOfRequests">The expected number of requests.</param>
     /// <returns>The <seealso cref="IHttpRequestMessagesCheck"/> for further assertions.</returns>
-    public static IHttpRequestMessagesCheck WithRequestUri(this IHttpRequestMessagesCheck check, string pattern, int expectedNumberOfRequests) => WithRequestUri(check, pattern, true, (int?)expectedNumberOfRequests);
+    public static IHttpRequestMessagesCheck WithRequestUri(this IHttpRequestMessagesCheck check, string pattern, int expectedNumberOfRequests) => WithRequestUri(check, pattern, (int?)expectedNumberOfRequests);
+
+    [Obsolete("Please use an overload without the 'ignoreCase', since ignoring casing is now controlled globally.")]
     public static IHttpRequestMessagesCheck WithRequestUri(this IHttpRequestMessagesCheck check, string pattern, bool ignoreCase, int expectedNumberOfRequests) => WithRequestUri(check, pattern, ignoreCase, (int?)expectedNumberOfRequests);
 
+    private static IHttpRequestMessagesCheck WithRequestUri(this IHttpRequestMessagesCheck check, string pattern, int? expectedNumberOfRequests)
+    {
+        if (check == null)
+        {
+            throw new ArgumentNullException(nameof(check));
+        }
+
+        if (string.IsNullOrEmpty(pattern))
+        {
+            throw new ArgumentNullException(nameof(pattern));
+        }
+
+        var condition = string.Empty;
+        if (pattern != "*")
+        {
+            condition = $"uri pattern '{pattern}'";
+        }
+
+        UriPattern uriPattern = UriPatternParser.Parse(pattern);
+
+        return check.WithFilter(x => x.RequestUri is not null && uriPattern.Matches(x.RequestUri, check.Options.UriPatternMatchingOptions), expectedNumberOfRequests, condition);
+    }
+
+    [Obsolete("Please use an overload without the 'ignoreCase', since ignoring casing is now controlled globally.")]
     private static IHttpRequestMessagesCheck WithRequestUri(this IHttpRequestMessagesCheck check, string pattern, bool ignoreCase, int? expectedNumberOfRequests)
     {
         if (check == null)
@@ -40,7 +67,16 @@ private static IHttpRequestMessagesCheck WithRequestUri(this IHttpRequestMessage
             condition = $"uri pattern '{pattern}'";
         }
 
-        return check.WithFilter(x => x.HasMatchingUri(pattern, ignoreCase), expectedNumberOfRequests, condition);
+        UriPattern uriPattern = UriPatternParser.Parse(pattern);
+        var options = new UriPatternMatchingOptions
+        {
+            HostCaseInsensitive = ignoreCase,
+            PathCaseInsensitive = ignoreCase,
+            SchemeCaseInsensitive = ignoreCase,
+            QueryCaseInsensitive = ignoreCase
+        };
+
+        return check.WithFilter(x => x.RequestUri is not null && uriPattern.Matches(x.RequestUri, options), expectedNumberOfRequests, condition);
     }
 
     /// <summary>
@@ -49,6 +85,7 @@ private static IHttpRequestMessagesCheck WithRequestUri(this IHttpRequestMessage
     /// <param name="check">The implementation that hold all the request messages.</param>
     /// <param name="pattern">The querystring pattern that is expected.</param>
     /// <returns>The <seealso cref="IHttpRequestMessagesCheck"/> for further assertions.</returns>
+    [Obsolete("Use WithRequestUri instead, since it now properly supports QueryStrings as well")]
     public static IHttpRequestMessagesCheck WithQueryString(this IHttpRequestMessagesCheck check, string pattern) => WithQueryString(check, pattern, null);
 
     /// <summary>
@@ -58,8 +95,10 @@ private static IHttpRequestMessagesCheck WithRequestUri(this IHttpRequestMessage
     /// <param name="pattern">The querystring pattern that is expected.</param>
     /// <param name="expectedNumberOfRequests">The expected number of requests.</param>
     /// <returns>The <seealso cref="IHttpRequestMessagesCheck"/> for further assertions.</returns>
+    [Obsolete("Use WithRequestUri instead, since it now properly supports QueryStrings as well")]
     public static IHttpRequestMessagesCheck WithQueryString(this IHttpRequestMessagesCheck check, string pattern, int expectedNumberOfRequests) => WithQueryString(check, pattern, (int?)expectedNumberOfRequests);
 
+    [Obsolete("Use WithRequestUri instead, since it now properly supports QueryStrings as well")]
     private static IHttpRequestMessagesCheck WithQueryString(this IHttpRequestMessagesCheck check, string pattern, int? expectedNumberOfRequests)
     {
         if (check == null)

src/TestableHttpClient/PublicAPI.Shipped.txt

@@ -103,7 +103,7 @@ static TestableHttpClient.HttpRequestMessagesCheckExtensions.WithRequestUri(this
 static TestableHttpClient.HttpRequestMessagesCheckExtensions.WithRequestUri(this TestableHttpClient.IHttpRequestMessagesCheck! check, string! pattern, bool ignoreCase, int expectedNumberOfRequests) -> TestableHttpClient.IHttpRequestMessagesCheck!
 static TestableHttpClient.HttpRequestMessagesCheckExtensions.WithRequestUri(this TestableHttpClient.IHttpRequestMessagesCheck! check, string! pattern, int expectedNumberOfRequests) -> TestableHttpClient.IHttpRequestMessagesCheck!
 
-TestableHttpClient.Utils.RouteParserException
-TestableHttpClient.Utils.RouteParserException.RouteParserException() -> void
-TestableHttpClient.Utils.RouteParserException.RouteParserException(string! message) -> void
-TestableHttpClient.Utils.RouteParserException.RouteParserException(string! message, System.Exception! innerException) -> void
+TestableHttpClient.Utils.UriPatternParserException
+TestableHttpClient.Utils.UriPatternParserException.UriPatternParserException() -> void
+TestableHttpClient.Utils.UriPatternParserException.UriPatternParserException(string! message) -> void
+TestableHttpClient.Utils.UriPatternParserException.UriPatternParserException(string! message, System.Exception! innerException) -> void

src/TestableHttpClient/PublicAPI.Unshipped.txt

@@ -1 +1,7 @@
-
+TestableHttpClient.TestableHttpMessageHandlerOptions.UriPatternMatchingOptions.get -> TestableHttpClient.UriPatternMatchingOptions!
+TestableHttpClient.UriPatternMatchingOptions
+TestableHttpClient.UriPatternMatchingOptions.DefaultQueryFormat.get -> System.UriFormat
+TestableHttpClient.UriPatternMatchingOptions.DefaultQueryFormat.set -> void
+TestableHttpClient.UriPatternMatchingOptions.QueryCaseInsensitive.get -> bool
+TestableHttpClient.UriPatternMatchingOptions.QueryCaseInsensitive.set -> void
+TestableHttpClient.UriPatternMatchingOptions.UriPatternMatchingOptions() -> void

src/TestableHttpClient/Response/RoutingResponse.cs

@@ -10,18 +10,18 @@ public Task ExecuteAsync(HttpResponseContext context, CancellationToken cancella
 
         if (context.HttpRequestMessage.RequestUri is not null)
         {
-            response = GetResponseForRequest(context.HttpRequestMessage.RequestUri, context.Options.RoutingOptions);
+            response = GetResponseForRequest(context.HttpRequestMessage.RequestUri, context.Options.UriPatternMatchingOptions);
         }
 
         return response.ExecuteAsync(context, cancellationToken);
     }
 
-    public Dictionary<RouteDefinition, IResponse> ResponseMap { get; init; } = new();
+    public Dictionary<UriPattern, IResponse> ResponseMap { get; init; } = new();
     public IResponse FallBackResponse { get; internal set; } = StatusCode(NotFound);
 
-    private IResponse GetResponseForRequest(Uri requestUri, RoutingOptions routingOptions)
+    private IResponse GetResponseForRequest(Uri requestUri, UriPatternMatchingOptions uriPatternOptions)
     {
-        var matchingResponse = ResponseMap.FirstOrDefault(x => x.Key.Matches(requestUri, routingOptions));
+        var matchingResponse = ResponseMap.FirstOrDefault(x => x.Key.Matches(requestUri, uriPatternOptions));
 
         return matchingResponse.Value switch
         {

src/TestableHttpClient/Response/SequencedResponse.cs

@@ -1,13 +1,15 @@
-namespace TestableHttpClient.Response;
+using System.Collections.Concurrent;
+
+namespace TestableHttpClient.Response;
 
 internal class SequencedResponse : IResponse
 {
-    private readonly Queue<IResponse> responses;
+    private readonly ConcurrentQueue<IResponse> responses;
     private readonly IResponse _lastResponse;
     public SequencedResponse(IEnumerable<IResponse> responses)
     {
         this.responses = new(responses ?? throw new ArgumentNullException(nameof(responses)));
-        if (this.responses.Count == 0)
+        if (this.responses.IsEmpty)
         {
             throw new ArgumentException("Responses can't be empty.", nameof(responses));
         }
@@ -22,13 +24,6 @@ public Task ExecuteAsync(HttpResponseContext context, CancellationToken cancella
 
     private IResponse GetResponse()
     {
-        if (responses.Any())
-        {
-            return responses.Dequeue();
-        }
-        else
-        {
-            return _lastResponse;
-        }
+        return responses.TryDequeue(out var response) ? response : _lastResponse;
     }
 }

src/TestableHttpClient/RoutingOptions.cs

@@ -3,18 +3,19 @@
 /// <summary>
 /// Options specific for routing.
 /// </summary>
+[Obsolete("Renamed to UriPatternMatchingOptions")]
 public class RoutingOptions
 {
     /// <summary>
-    /// Indeicates whether or not the scheme of a route should be treated as case insensitive. Default: true
+    /// Indicates whether or not the scheme of an URI should be treated as case insensitive. Default: true
     /// </summary>
     public bool SchemeCaseInsensitive { get; set; } = true;
     /// <summary>
-    /// Indeicates whether or not the host of a route should be treated as case insensitive. Default: true
+    /// Indicates whether or not the host of an URI should be treated as case insensitive. Default: true
     /// </summary>
     public bool HostCaseInsensitive { get; set; } = true;
     /// <summary>
-    /// Indeicates whether or not the path of a route should be treated as case insensitive. Default: true
+    /// Indicates whether or not the path of an URI should be treated as case insensitive. Default: true
     /// </summary>
     public bool PathCaseInsensitive { get; set; } = true;
 }