-
Describe the bugI've set up a solution using YARP as a gateway reverse proxy between external requests and the reverse proxy intended to be utilized internally by Service Fabric. When running a 5-node cluster in local debug, I can issue a request to the project housing YARP and see the logging indicating that it's proxying the request to the appropriate endpoint, but it yields a 500. Sending a request via Postman to the same address YARP uses works with a 200. To ReproduceI've created a sample solution SfYarp.zip that demonstrates the issue. You'll need to run it on a machine with the Service Fabric SDK installed, but otherwise just hit F5 in Visual Studio to start it up.
Note that line 6 contains the URL it's intended to proxy to:
When the request is performed manually, it works fine. When performed via YARP, it fails. There are no exceptions observed anywhere, just the 500 internal service error when attempting to use YARP. Further technical detailsUsing .NET Core 3.1 on Windows 10 Pro using VS 16.6.3 |
Beta Was this translation helpful? Give feedback.
Replies: 8 comments
-
|
Are there any more hints of you turn the log level up to Debug? |
Beta Was this translation helpful? Give feedback.
-
|
I'm not in terribly familiar territory here, so bear with me. Finally got some additional information to appear in the other service while debugging it and I see this error:
I recall noting in the Service Fabric documentation that "...unlike HTTP.sys, Kestrel doesn't support port sharing between multiple processes. Each instance of Kestrel must use a unique port." Is it possible that's the issue here? I'm going to try swapping out the use of Kestrel with HTTP.sys and see if that remedies this. |
Beta Was this translation helpful? Give feedback.
-
|
Yep, that did it - simply switching out the use of Kestrel with HTTP.sys did the charm. Here's a working variant of the original solution: Does this mean that Kestrel isn't a supported platform for YARP and that any implementations must use HTTP.sys instead? |
Beta Was this translation helpful? Give feedback.
-
|
There are multiple things at play, let's sort them out one at a time. 1. Port 8603 already in use
@WhitWaldo you must have something else running that already claimed that port. I see that you are (correctly) setting Gateway's InstanceCount to 1 when running locally, so it is not immediately clear who the culprit is. Sugg running 2. 500 status codeSince your localhost communication between Gateway and SF Reverse Proxy is unencrypted, you can easily capture it using a tool like Wireshark. Just compare that to a trace where you hit it with postman. you can also inspect SF Reverse Proxy diagnostics. see docs |
Beta Was this translation helpful? Give feedback.
-
Kestrel should work with YARP, that's what we use by default. Port conflicts are usually a configuration issue. |
Beta Was this translation helpful? Give feedback.
-
|
@davidni
From what I can find, this means that it's listening on all possible IPv4 addresses and the IPv6 loopback for port 8603. Does this count as multiple uses of the port? I thought I'd see if I could just set the Kestrel options to use so it'd specifically just use 127.0.0.1 on 8603 instead of 0.0.0.0 or [::] But this just added another listener explicitly using 127.0.0.1 on top of the other two. Looking at the Wireshark traffic, it just led to the same binding error as posted before:
As a last attempt to get it to quit listening on multiple ports, I updated the I'm not really seeing anything special you have in your samples regarding Kestrel configuration and I'm not really doing anything special here apart from running everything within Service Fabric. Any thoughts on why this isn't working within an SF environment? I'd wait on that SF integration PR, but I don't see anything in there that the Kestrel configuration to demonstrate what's not right here. |
Beta Was this translation helpful? Give feedback.
-
|
Kestrel can be configured via code, config file, or UseUrls. Is it possible you have the same url defined in both code and the config file? |
Beta Was this translation helpful? Give feedback.
-
|
Don't think so. The only sections in my appsettings.json (zipped two sample solutions in the above responses) are Logging, AllowedHosts and ReverseProxy. My WebHostBuilder is equivalent to what the Service Fabric template kicks out + logging: /// <summary>
/// Optional override to create listeners (like tcp, http) for this service instance.
/// </summary>
/// <returns>The collection of listeners.</returns>
protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
return new ServiceInstanceListener[]
{
new ServiceInstanceListener(serviceContext =>
new KestrelCommunicationListener(serviceContext, "ServiceEndpoint", (url, listener) =>
//new HttpSysCommunicationListener(serviceContext, "ServiceEndpoint", (url, listener) =>
{
ServiceEventSource.Current.ServiceMessage(serviceContext, $"Starting Kestrel on {url}");
//ServiceEventSource.Current.ServiceMessage(serviceContext, $"Starting HTTP.sys on {url}");
return new WebHostBuilder()
//.UseHttpSys()
.UseKestrel()
.ConfigureServices(
services => services
.AddSingleton<StatelessServiceContext>(serviceContext))
.ConfigureLogging(logging =>
{
logging.ClearProviders();
logging.AddDebug();
})
.UseContentRoot(Directory.GetCurrentDirectory())
.UseStartup<Startup>()
.UseServiceFabricIntegration(listener, ServiceFabricIntegrationOptions.None)
.UseUrls(url)
.Build();
}))
};
}The input URL just gets set to And that's it - can't find anything else that deals with Kestrel configuration. |
Beta Was this translation helpful? Give feedback.
Kestrel can be configured via code, config file, or UseUrls. Is it possible you have the same url defined in both code and the config file?