Issue with GetSourceAsync() #4279

SilentStorm-1986 · 2022-10-20T06:10:44Z

SilentStorm-1986
Oct 20, 2022

I am using WinForms Cef (latest build). I'm connecting via SSL proxy, which seems to load the page fine, however I can't retrieve the page source with GetSourceAsync() or ViewSource() (empty string and empty txt in notepad respectively).

Here's my code:

Cef.UIThreadTaskFactory.StartNew(Sub()

                                                     Dim newrc = chromBrowser.GetBrowser.GetHost.RequestContext
                                                     Dim dict As New Dictionary(Of String, Object)
                                                     dict("mode") = "fixed_servers"
                                                     //SSL proxy, page loads fine
                                                     dict("server") = proxy & ":" & port
                                                     newrc.SetPreference("proxy", dict, [error])

                                                 End Sub)

chromBrowser.Load(strURL)

//set by EventHandler OnFrameLoadEnd
Do Until boolLoaded = True
    Application.DoEvents()
Loop

query = chromBrowser.GetSourceAsync()
query.Wait()
If query.Result IsNot Nothing Then
   //strHTML = ""
   strHTML = query.Result.ToString
End If

SilentStorm-1986 · 2022-10-20T06:43:16Z

SilentStorm-1986
Oct 20, 2022
Author

In an older build of Cef I have a similar issue. I get only the outer tags: <html><head></head><body></body></html>

I have no issues with any build when I'm using my IP and not a proxy.

1 reply

amaitland Oct 23, 2022
Maintainer

Assuming your proxy works correctly you need to wait for the proxy to be set use SetProxyAsync extension method and await it's result before calling Load. Use LoadUrlAsync instead of Load and await it's result before calling GetSourceAsync would be my recommendation.

http://cefsharp.github.io/api/105.3.x/html/Overload_CefSharp_RequestContextExtensions_SetProxyAsync.htm
CefSharp/CefSharp.Test/OffScreen/OffScreenBrowserBasicFacts.cs

Line 597 in a819fc4

var setProxyResponse = await requestContext.SetProxyAsync("127.0.0.1", 8080);
CefSharp/CefSharp.Test/Wpf/WpfBrowserBasicFacts.cs

Line 49 in f84f398

var response = await browser.LoadUrlAsync("www.google.com");

If you still have problems Check the log file for errors see https://github.com/cefsharp/CefSharp/wiki/Trouble-Shooting#log-file

SilentStorm-1986 · 2022-10-24T05:10:26Z

SilentStorm-1986
Oct 24, 2022
Author

I fixed this the other day by using the following code, before calling GetSourceAsync():

Do Until chromBrowser.CanExecuteJavascriptInMainFrame = True
    Application.DoEvents()
Loop

Now I have a new problem with proxies that will connect through the cefclient.exe test build command line proxy switch, but NOT the RequestContext.SetPreference method, Cef.GetGlobalRequestContext.SetProxy method, RequestContext.Configure().WithProxyServer(proxy, port).Create() method OR settings.CefCommandLineArgs.Add("proxy-server", proxy) method.

I thought the error might have to do with the fixed_servers mode, however using a pac_url doesn't seem to work either.

Honestly I don't need to use these particular proxies, but I am curious as to why they work with the cefclient test build but won't work with the WinForms Cef assembly.

1 reply

amaitland Oct 24, 2022
Maintainer

Likely you've made a mistake somewhere. If you set via command line args you cannot change at runtime.

You can use the xunit test at https://github.com/cefsharp/CefSharp/blob/master/CefSharp.Test/OffScreen/OffScreenBrowserBasicFacts.cs#L562 modify the proxy and see if it runs.

SilentStorm-1986 · 2022-10-24T09:06:20Z

SilentStorm-1986
Oct 24, 2022
Author

I'm quite sure that I've set up the code correctly. All the aforementioned methods were tested independently from each other. I can switch the proxy from one list (the list that I got working the other day) to the other and the first will work and the other will not.

Does cefclient dynamically run through different methods of connecting to the proxy until it has a connection? I set debug log to Verbose and cefclient, WinForms working proxy and WinForms failed proxy all show the exact same output. The proxy seems to connect but the page doesn't load.

The cefclient test build I'm using matches the latest WinForm version.

1 reply

amaitland Oct 24, 2022
Maintainer

Its my understanding proxy functionality is implemented entirely at the Chromium layer. All CEF based applications should behave the same.

SilentStorm-1986 · 2022-10-24T10:01:30Z

SilentStorm-1986
Oct 24, 2022
Author

I have one more piece of information. If I type the proxy address in the address bar in a browser, there seems to be a redirect prior to connection with the one that fails. Here's the server response for the proxy that fails:
This is a proxy server, does not response to non-proxy requests

Here's a typical response from a proxy that works:

Page failed to load.
URL: https://proxy
Error: ERR_SSL_PROTOCOL_ERROR (-107)
ERR_SSL_PROTOCOL_ERROR

0 replies

SilentStorm-1986 · 2022-10-24T10:17:38Z

SilentStorm-1986
Oct 24, 2022
Author

I just solved it. I had to implement a RequestHandler class and leave the username & password as empty strings. Very strange but it worked.

0 replies

Uh oh!

Issue with GetSourceAsync() #4279

Uh oh!

Uh oh!

SilentStorm-1986 Oct 20, 2022

Replies: 5 comments · 3 replies

Uh oh!

Uh oh!

SilentStorm-1986 Oct 20, 2022 Author

Uh oh!

amaitland Oct 23, 2022 Maintainer

Uh oh!

SilentStorm-1986 Oct 24, 2022 Author

Uh oh!

amaitland Oct 24, 2022 Maintainer

Uh oh!

SilentStorm-1986 Oct 24, 2022 Author

Uh oh!

amaitland Oct 24, 2022 Maintainer

Uh oh!

SilentStorm-1986 Oct 24, 2022 Author

Uh oh!

SilentStorm-1986 Oct 24, 2022 Author

SilentStorm-1986
Oct 20, 2022

Replies: 5 comments 3 replies

SilentStorm-1986
Oct 20, 2022
Author

amaitland Oct 23, 2022
Maintainer

SilentStorm-1986
Oct 24, 2022
Author

amaitland Oct 24, 2022
Maintainer

SilentStorm-1986
Oct 24, 2022
Author

amaitland Oct 24, 2022
Maintainer

SilentStorm-1986
Oct 24, 2022
Author

SilentStorm-1986
Oct 24, 2022
Author