Skip to content

Commit 9590447

Browse files
authored
Merge quic-latest into master (#8010)
Squashed commit of the following: commit d232a12 Merge: 837bd0e 2edeae4 Author: Masakazu Kitajo <maskit@apache.org> Date: Tue Jun 29 15:41:34 2021 +0900 Merge branch 'master' into quic-latest * master: reuse multiple times (#7992) Test bad request behavior (#7884) Fix BoringSSL build (#8001) Update TSHttpTxnAborted API to distinguish client/server aborts (#7901) Enforce case for well known methods (#7886) Add null checks for http_load (#7995) commit 837bd0e Author: Masakazu Kitajo <maskit@apache.org> Date: Mon Jun 28 15:11:03 2021 +0900 Fix unit tests for QUICStreamState commit c5bb9e0 Merge: 0a63fa9 202b250 Author: Masakazu Kitajo <maskit@apache.org> Date: Mon Jun 28 10:02:54 2021 +0900 Merge branch 'master' into quic-latest * master: Implement TLSBasicSupport for QUICNetVC (#7959) Reload server session inactivity timeout before placing a session into the pool (#7618) Use OpeSSL EVP API if SHA1 API is unavailable (cache_promote) (#7447) Cleanup: Get rid of HTTP2_SESSION_EVENT_RECV (#7879) Timing and permissions update for regex_revalidate test (#7998) limit m_current_range to max value in RangeTransform (#4843) Allow to TLS handshake to error out on TSVConnReenable (#7994) Cleanup: Get rid of HTTP2_SESSION_EVENT_INIT (#7878) Add hook for loading certificate and key data from plugin (#6609) Doc: Now's Minute invocation error (#7990) Fix typo in configure.ac (#7993) commit 0a63fa9 Merge: 312cf39 bd93f2a Author: Masakazu Kitajo <maskit@apache.org> Date: Fri Jun 25 14:34:55 2021 +0900 Merge branch 'master' into quic-latest * master: Don't rely on SSLNetVC when HttpSM gathers info about SSL (#7961) conf_remap: demote 'Invalid configuration' to warning (#7991) Cleans up the code bit, including milliseconds consistency (#7989) Pass through expect header and handle 100-continue response (#7962) Treat TRACE with body as bad request (#7905) Thread safe Mersenne Twister 64 using c++11 (#7859) ESI plugin documentation updates. (#7970) Add log name configuration and stderr/stdout support. (#7937) Cleanup: Constify MIMEHdr (#7949) Fixed compile error with Linux AIO unit test (#7958) Note YAML parser library bug, and work-around, in documentation. (#7963) Ensure that the content-length value is only digits (#7964) String the url fragment for outgoing requests (#7966) Fix for HTTP/2 frames (#7965) Improve parsing error messages for strategies.yaml. (#7948) fix the scheme of h2 0rtt tests (#7957) Fix double test flakiness due to EOS/TXN_CLOSE race (#7956) Use proxy.config.log.hostname for rotated log filenames (#7943) Fixed memory leak in the QUIC stream manager (#7951) Fixup TS_USE_LINUX_NATIVE_AIO AIO_MODE_NATIVE (#7832) Update GitHub stale action to auto close old PRs (#7952) Revert "Do not invalidate cached resources upon error responses to unsafe methods (#7864)" (#7954) regex_revalidate: add stats for miss/stale counts (#7950) Do not invalidate cached resources upon error responses to unsafe methods (#7864) Add an HTTP/2 304 "Not Modified" AuTest. (#7882) regex_revalidate: optionally retain rule epoch state across restarts (#7939) Fixed memory leak in QUIC ack frame unit test (#7947) cache_promote: Don't promote on uncacheable requests (#7942) Fix dynamic-stack-buffer-overflow of cachekey plugin (#7945) Compilation error fixes for QUIC unit tests (#7944) Adds bytes counting as a trigger to the cache_promote LRU (#7765) Add a JSON schema for strategies.yaml (#7932) Remove second call to TRANSACT_RETURN while handling cache write lock (#7873) Close connection after every bad request for HTTP/1.1 (#7885) Pin Sphinx to 3.x to unblock `make html` (#7940) Add support for Remap rule hit stats (#7936) Remove scrap log object dead code (#7935) Add STL forward iterators to DLL container. (#7934) Add log SQUID code testing to redirect.test.py Au test. (#7870) Fix race condition on server session state (#7921) regex_reval: bug where rule type is always reported as the first (#7928) Remove duplicate entry in overridable txn vars. (#7930) Satisfy ci/jenkins/bin/clang-format.sh (#7929) Add a basic Au test using strategies.yaml, with consistent hashing. (#7911) Add a chunked negative revalidating test. (#7907) Ensure that URL components are valid when alternate eviction is logged (#7924) fix grammar (#7927) AuTest: Enable h2spec generic test cases (#7926) Adjust vc read errors (#7923) Remove bucket search from IntrusiveHashMap::erase (#7848) Ensure TS_VCONN_CLOSE_HOOK hook is called during TS_EVENT_VCONN_CLOSE. (#7913) Update docs languages file to add 9.1.x for en and ja (#7917) * Adds a new peering ring mode to next hop selection strategies. (#7897) Add Au test for strategies.yaml, with consistent hashing, with fallover. (#7914) Make HttpSM server reference a Transaction instead of a Session (#7849) Set accept_options of Http1Transaction in Http1ClientSession::new_connection() (#7894) Reset Http1Transaction before adding vc to keep_alive_queue (#7892) Add dead server policy control and metric. Improve messages. (#7757) Ensure the HTTP protion of the protocol string is upper case (#7904) Fixed spelling mistakes in the docs (#7896) add MISS capability to the regex_revalidate plugin (#7899) docs: fix capitalization of Linux (#7898) Redirect - Make TS to honour the number_of_redirections configuration value (#7867) Clean up producer more regularly (#7386) Fix crash in open_close_h2 (#7586) Cleanup Http2ClientSession SessionHandler (#7876) Enforce HTTP parsing restrictions on HTTP versions supported (#7875) Do not delete the continuation twice (#7862) Cleanup: refer Http2ClientSession::mutex (#7853) Autest - Proxy Verifier Extension, add context template $-base string substitution in the replay file. (#7866) Fixed some spelling mistakes in comments (#7869) Fixed ASAN issues with MMH test (#7868) Cleanup: Move member functions defined inside of class definitions of Http2ConnectionState & Http2ConnectionSettings (#7854) Add URI Signing cdnistd Claim Implementation (#7822) Adds a new --enable-all-asserts configure option (#7858) Unifdef test code for MMH and moved it into its own test file (#7841) Clean up lua plugin doc for overridable configurations (#7844) Save and propagate epoll network error (#7809) Add method to write an IpAddr value to a sockaddr. (#7821) Add proxy.config.http.max_proxy_cycles (#7657) Update NextHop strategies so that unavailable server retry codes (#7837) generator: allow for POST requests (#7635) Fixed double declaration types for log buffer tracking (#7847) Extra braces for clang 5 / ubuntu 16.04 on array initialization (#7842) Conflicts: iocore/net/quic/QUICStreamFactory.cc commit 312cf39 Merge: f90e8dd 5cdc145 Author: Masakazu Kitajo <maskit@apache.org> Date: Mon May 17 10:07:42 2021 +0900 Merge branch 'master' into quic-latest * master: Get rid of code for OpenSSL that has old QUIC API (#7599) Fixed warning in gcc 11 about array not being initalized (#7840) Don't call next next dup on destroyed mime field mloc. (#7833) build_h3_tools: use OpenSSL_1_1_1k+quic (#7836) Address assert on captive_action (#7807) Fix so EOS are delivered to sessions in the pool (#7828) Fix a format specifier for size_t (#7830) Fix stall on sending response for request with trailer header (#7831) Simplification dir_init_done (#7817) Remove unused member from HttpSM (#7835) AuTest: use exteneded help output to determin curl feature support (#7834) Apply fmt compile time argument checking to log functions (#7829) Adds new X-Cache-Info header to the xdebug plugin (#7784) Cleanup: Remove unused members of Http2Stream (#7813) Cleanup: unused functions of Http2ClientSession (#7812) Cancel cross_thread_event on clear_io_events (#7815) Cleanup: Remove a meaningless Http2Stream::do_io_close() call (#7814) Eliminate next dup call using stale mime field mloc is s3_auth plugin. (#7825) NetEvent cleanup - replace #define with constexpr (#7804) fix origin session related crashes (#7808) Update HTTP version info in HostDB on new outbound connection (#7816) Remove a redundant argument (#7811) SSL Cert lookup using PP dest ip when ProxyProtocol is enabled (#7802) Fix MLoc assert caused by s3auth (#7790) Fix cpu utilization problem in session cache (#7719) Fix to cookie_remap.cc tp avoid Intel compiler warning. (#7792) TSHttpTxnCacheDiskPathGet - tighten up the code a bit. (#7806) Doc: tcpinfo plugin table formatting (#7805) fix DNS spike issue for TCP_RETRY mode (#7307) Adds new TS API TSHttpTxnCacheDiskPathGet (#7783) tests: Fixes spelling (#7789) Traffic Dump: Add an HTTP/3 AuTest (#7758) use sendmsg and recvmsg (#7793) HTTP: clean up the http_hdr_describe format error (#7797) Fixes an issue where next hop unit tests crash when run on macOS. (#7787) Apply log throttling to HTTP/2 session error rate messages (#7772) Cleans up uninitialized warning in LogMessage.cc (#7788) Short circuit remap reload when a valid remap file is not specified (#7782) DNS: Clean up argument passing to DNS queries. (#7778) Remove extra verify-callback (#7540) Augment test cases for tls_verify_override test (#7736) Make when_to_revalidate setting available on HTTPS (#7753) Add traffic_server command line option for debugging in Au test. (#7762) Test: Update tls_partial_blind_tunnel to have a nameserver. (#7773) Test: update tls_forward_nonhttp to have a nameserver. (#7774) Test: add nameserver to log-filter test. (#7776) BWF: Add support for std::error_code. (#7777) Test: add nameserver to log-field test. (#7779) Test: add nameserver to regex_remap test. (#7775) Elevate privileges for traffic_manager during SSL cert reload (#7770) Clean up HTTP version processing (#7766) Remove proxy.config.http.down_server.abort_threshold (#7748) Remove undocumented keepalive_internal_vc setting (#7693) doc: header_rewrite random function not inclusive (#7760) Experimental Cache fill plugin (#7470) Remove references to removed options (#7756) Propagate TLS errors (#7714) AuTest extension: check for unrecognized configurations (#7752) Fixes errors in the strategies.yaml documentation. (#7745) Updates to Nexthop strategies to limit the number of simultaneous (#7744) Fixes Issue #7739 - Next hop strategy with bad 'to' URL causes TS crash. (#7749) header_rewrite: Various fixes for MaxMind support (#7746) Remove unused variable is_revalidation_necessary (#7747) Fix simple remapping in regex_remap plugin. (#7718) Adding DNS TTL AuTests. (#7742) Add a chunked disabled test. (#7743) Fix monitor threads in lib records to exit on system shutdown. (#7731) Add overload for memcpy to take a destination buffer and source string_view / TextView (#7732) Test: Add nameserver to TLS tunnel forward test. (#7733) AIO_NOT_IN_PROGRESS should not be 0 (#7734) if transaction status non-success, bypass intercept plugin (#7724) ink_utf8_to_latin1 is not defined, removing declaration (#7737) Fix build on FreeBSD 13 (#7730) Update VSCode CPP Standard (#7723) Updating to use Proxy Verifier 2.2.0 (#7729) header_rewrite: Allow for relative path to geo database files (#7727) Override proxy.config.ssl.client.sni_policy from sni.yaml (#7703) compress.test.py: Reference config file from Test.RunDirectory (#7725) Ran clang-tidy over the code (#7708) Deny unknown transfer encoding values (#7694) Fix doc for http2.no_activity_timeout_in (#7721) Add DynamicStats (#7704) header_rewrite: allow for use of maxminddb as source of geo truth (#7695) Include in parentselectdefs.h in install target (#7713) uri_signing: fix warning which affects ubuntu:20.04 builds (#7717) Increase the maximum slice block size from 32MB to 128MB (#7709) commit f90e8dd Author: Masakazu Kitajo <maskit@apache.org> Date: Tue Jan 12 12:21:51 2021 +0900 Add QUICStreamStateListener commit f66646c Merge: be9837c 9f9594f Author: Masakazu Kitajo <maskit@apache.org> Date: Sat Apr 17 13:57:50 2021 +0900 Merge branch 'master' into quic-latest * master: Fix ALPN support on QUIC connections (#7593) fix mem leak in session cache (#7707) Parent Select Plugin (#7467) Add new TS API function TSUrlRawPortGet. (#7568) Add NixOS support (#7697) Remove support for --enable-remote-cov-commit (#7700) Remove configure-time loopback interface detection (#7702) Add sqpv log field for server protocol (#7680) Call do_io_close instead of HTTP2_SESSION_EVENT_FINI handler (#7594) Fix a bug in tspush that pushes corrupted content to cache (#7696) Automatically marks PRs and issues stale (#7675) New rate_limit plugin for simple resource limitations (#7623) Remove undefined method HttpSM::perform_nca_cache_action (#7692) Remove undefined method HttpSM::setup_client_header_nca (#7691) Scalar; Move "tag" struct to be inside the "ts" namespace to avoid collisions. (#7690) Rollback LAZY_BUF_ALLOC remove in HttpTunnel (#7583) Add class to normalize handling of pending action (#7667) Make HTTP/2 Curl AuTest gold files case insensitive (#7683) Add STL compliant field iteration to MIMEHdr. - rebase. (#7476) Fix use of -mcx16 flag - only use if it compiles cleanly. (#7684) Refine connection failure logging and messages and eliminate suprious connection errors (#7580) Add close header normalize openclose test (#7679) Fix has_consumer_besides_client to deal with no clients (#7685) create a new cache status RWW_HIT (#7670) Updating to AuTest 1.10.0 (#7682) sslheaders AuTest: Skip if plugin does not exist (#7678) Add AuTest for Background Fill (#7613) Do NOT kill tunnel if it has any consumer besides HT_HTTP_CLIENT (#7641) AuTest: address various permissions issues (#7668) Adding TCP Info header support to header rewrite (#7516) Refine Inline.cc carveout for arm64 darwin builds (#7662) Comment why log eviction isn't implemented via a log field. (#7648) Fixing Throttler.h for older clang and gcc compilers (#7651) Update -with-profile and add some profiling documentation (#7601) Use correct default value for verify.server.policy (#7636) Update server_response_body_bytes when background fill worked (#7621) Remove erroneous manager.log mesg with remap include file reload (#7646) Change ROUNDUP from function-like macro to function template. (#7614) Document http.default_buffer_water_mark (#7612) Add proxy.config.cache.log.alternate.eviction (#7629) Fix HttpSessionManager::acquireSession from previous rebase error (#7631) Fix tls_client_versions and tls_hooks18 tests (#7645) Updating documentation for negative_revalidating_lifetime (#7633) Remove reference to client.verify.server from tests and other bits (#7639) Add pooled_server_connections metric (#7627) Expose URL element methods through HTTPHdr (#7628) Add default implementation for allow_half_open (#7630) Add thread yeield to avoid busy waiting in LogObject::_checkout_write(). (#7576) Add proxy.process.http.background_fill_total_count (#7625) statichit: misc. fixes (#7634) Remove unused variables (#7626) Adding negative revalidating AuTests. (#7620) Add failed state to hostdb to better track failing origins (#7291) Use standard isdigit library function (#7619) Typo in output when forcing kqueue for configure (#7617) Implement log throttling (#7279) Increase Proxy Verifier caching delay. (#7616) Set pcre_malloc/free function pointers in core main() only. (#7608) commit be9837c Merge: 99ff68f d4fc16f Author: Masakazu Kitajo <maskit@apache.org> Date: Wed Mar 17 09:38:59 2021 +0900 Merge branch 'master' into quic-latest * master: Fix the connection limit crash while using parents (#7604) Remove inline for detail::cache::CacheData::idAddr (#7592) Remove UnixNetVConnection::startEvent - not actually called. (#7596) Use return values to fix ubuntu release build error (#7591) Fix double destuct on Http2Stream termination (#7600) Add pointer/reference upcast function that is checked in debug builds. (#7582) Call constructors and destructors for H1/2 Session/Transaction via ClassAllocator (#7584) Add gold test for remap config .include directive. (#7589) Change the default value for verify.server.policy (#7587) Build the test library for tls_engine consistently (#7588) Generalize ALPN logic (#7555) Fix the final consumer write size from unchunked to chunked tunnel (#7577) Reactivate accept_no_activity_timeout (#7408) Tidy up session/transaction destruction process (#7571) Remove ProxyTransaction::set_proxy_ssn (#7567) Introduce TLSBasicSupport interface (#7556) Cleanup: Rename IOBufferReader of Http2ClientSession (#7569) Add a check for compress response, if from server and 304, then check cache for headers instead of the 304 response (#7564) Updates the STATUS file with all recent releases (#7566) Make Allocator.h less silly (no creepy "proto" object). (#6241) Cleanup: Remove unused member of Http2ClientSession (#7570) enable origin server session cache by default (#7537) Add tscontdestroy when transaction is closed and pacing rate is reset (#7572) Remove reference to CoreUtils (#7557) Remove unused enums from YamlSNIConfig struct. (#7565) Removes deprecated sni.yaml option: disable_h2 (#7547) This PR updates parent selection to limit the number of simultaneous (#7485) Fix KA header not checking strategy (#7483) Get rid of kruft LogObject copy constructor. (#7553) For TSHttpHdrEffectiveUrlBufGet(), include scheme for request to server URL. (#7545) Adding lower_ support to stats and bonding_slave data points for port status (#7560) Change cookie_remap plugin to allow use of pre-remap URL (and components). (#7519) check verify policy and properties (#7559) Fix parent.config to 504 not 502 on timeout (#7558) use SSL_CTX address as part of the lookup key (#7552) Add ALPN support on TLS Partial Blind Tunnel (#7511) Add server_name option to proxy.config.ssl.client.sni_policy (#7533) Fix a crash on origin session reuse (#7543) Removes the test plugins from the .spec file / RPM (#7551) Convert the inactive_client_timeout test to use Proxy Verifier (#7535) Fix ja3_fingerprint configure syntax (#7550) Fix asserts in multiplexer plugin. (#7532) parse expiration time and reload config at time out (#7281) Fix origin_session_reuse test (#7542) Fix tls_session_reuse test (#7541) Split SSL_CTX initialization logic into small functions (#7434) Remove dependency for SSL stuff from P_Net.h (#7531) Unify all the connect timeouts into one (#7335) Fix lua_states_stats Au test. (#7232) origin session reuse (#7479) Updating to use Proxy Verifier 2.1.0 (#7534) update the session reuse tests (#7529) commit 99ff68f Author: Masakazu Kitajo <maskit@apache.org> Date: Wed Feb 17 11:14:40 2021 +0900 Fix link error commit c4ad0c0 Merge: c40d95a cd33010 Author: Masakazu Kitajo <maskit@apache.org> Date: Wed Feb 17 09:56:25 2021 +0900 Merge branch 'master' into quic-latest * master: Select lua context per thread (#7465) Fix out of bounds access error in jtest (#7526) Disable compiling Inline.cc on macOS (#7389) Makes sure the types are correct, avoiding compiler warnings (#7523) Move has_request_body to ProxyTransaction (#7499) Make the H3 build script work properly on Debian platforms (#7522) slice/handleFirstServerHeader: return sooner on requested range errors (#7486) Add new log field for negotiated ALPN Protocol ID with the client (#7491) Add Outbound PROXY Protocol (v1/v2) Support (#7446) Updates the Dockerfile for debian (#7518) Disable client inactivity timeout while server is processing POST request (#7309) Upgrade Catch.hpp to v2.13.4 (#7464) Move reopen_moved_log_files to log flushing thread (#7450) replace psutil.pid() with psutil.process_iter() for safer execution (#7515) Fix spacing in clang-analyzer.sh script (#7480) Fix out of bounds access error in ats_base64_decode (#7490) Updated to build lastest versions of Fedora and CentOS docker images (#7505) Fix QUIC unit tests build issue on GNU ld (#7496) Fix QUIC unit test failures (#7497) Fixed build issues with Fedora 34 (#7506) Fixing DNS local_ipv* config option (#7507) traffic_dump: AuTests to use Proxy Verifier. (#7502) Disable ja3 plugin when building with boringssl (#7500) Avoid -Warray-bounds on PROXY Protocol Builder (#7488) AuTest: Upgrade to Proxy Verifier 2.0.2 (#7493) fix certs (#7494) Add zlib1g-dev to Debian dependencies in README (#7495) Unit Test - Increase openssl's key size. Place test certs into a common test folder. (#7451) Add basic type aliases for std::chrono types to ink_time.h for future use. (#7482) traffic_ctl - Fix lookup key for run-root option (#7484) update thread config tests (#7370) Perf: Replace casecmp with memcmp in HPACK static table lookup (#6521) Add PROXY Protocol Builder (#7445) Adjust so transfer-encoding header can be treated hop-by-hop (#7473) Convert auxkey form 2 uint32_t to 1 uint64_t. (#7350) Remove the queuing option from proxy.config.http.per_server.connection (#7302) Remove unused function ink_microseconds. (#7481) use std::unordered_map to store sessions (#7405) drop use of BIO_f_base64 and EVP_PKEY_new_mac_key (#7106) Do not write to the cache if the plugin decides not to write to the cache (#7461) API to retrieve NoStore set by plugins (#7439) Update AuTest version update directions for pipenv (#7469) Add command line utility to help convert remap plugin usage to ATS9. (#7426) Cleanup: Get rid of MIMEFieldWrapper from HPACK encoding (#6520) Proxy Verifier: Making use of delay directives for caching tests. (#7468) Cleanup: Add SNIRoutingType (#7453) Updating to Proxy Verifier v2.0.0 (#7454) Adjust to actually try a server address more than once (#7288) Change atoi to atol, causing obvious issues on what needs to be int64's (#7466) Cleans up duplicated TSOutboundConnectionMatchType definition (#7090) Fixing compress expectation for new microserver (#7463) Update to the new MicroServer 1.0.6 release (#7460) CacheRead: clear dir entry if doc is found to be truncated (#7064) Do not provide a stale negative cache (#7422) Generalize SNI support (#6870) Add synchronization between UDPNetProcessor::UDPBind in main Thread and initialize_thread_for_udp_net in ET_UDP Thread (#7407) Fix heap use after free in DNSProcessor::getby() (#3871) Fix comment in include/tscore/Filenames.h. (#7457) Fix Makefile target for creating changelogs (#7455) Change squid log code for self looping (#7443) Enhancements for compress plugin (#7416) Add incoming PROXY Protocol v2 support (#7340) Cleanup: Remove unused members of NextHopProperty (#7436) Small fix to regex_remap PR # 7347. (#7437) PoolableSession (#6828) option to disable compression for range request's response (#7287) Make TSUrlSchemeGet() return scheme implied by URL type when there is no explicit scheme. (#7262) commit c40d95a Merge: 573035c ecd70df Author: Masakazu Kitajo <maskit@apache.org> Date: Wed Jan 20 09:39:34 2021 +0900 Merge branch 'master' into quic-latest * master: Fix a link error on traffi_quic command (#7433) Fix stall on outbound TLS handshake (#7432) Fix the Proxy Verifier AuTest extension to handle cert paths correctly (#7415) Update documentation for TSSslSessionInsert (#7420) Improve zlib detection logic (#7430) Fix parent connect fail segfault (#7429) commit 573035c Merge: 5704095 95b8d57 Author: Masakazu Kitajo <maskit@apache.org> Date: Fri Jan 15 23:24:29 2021 +0900 Merge branch 'master' into quic-latest * master: Doc: Fix typo in negative_revalidating_lifetime (#7427) Change comment handling for long lines in url_sig plugin (#7421) Add unit tests for PROXY Protocol v1 parser (#7332) LGTM: Remove superfluous const qualifier in return type (#7412) Fix issue with unavailable server retry codes (#7410) Remove the warning statement (#7414) default to throttling and subsequently simplify the transfer code (#7257) Improvement to lua plugin (#7413) Make places to bind/unbind SSL object with/from NetVC (#7399) traffic_ctl - plugin msg now require only the tag as mandatory field data field is now optional. (#7364) API - Add new api function TSHttpTxnServerSsnTransactionCount() to retrieve the number of transactions between TS proxy and the origin server from a single session. (#7387) Fix clang compiler complaint about an unused parameter in SNIAction. (#7409) Add compression support to stats_over_http (#7393) Doc: Fix INPUT tag of Doxyfile (#7404) Remove unneeded variables in UnixNetVConnection (#7403) Correctly pass back errno to HttpSM (#7402) Reverting to old negative_caching conditional behavior (#7401) Remove unused MAYBE_ABORT state (#7400) traffic_manager should not retry on disk failure (#7397) Eliminate dangling pointer into stack space. (#7392) This PR aims to address some of the lock contention found and (#7377) Remove a special treatment for SSLNetVC in migrateToCurrentThread() (#7384) Replace ::exit() with _exit() to avoid secondary cleanup cores (#7395) [Doc] Fix build warnings (#7391) Clear call_sm on tunnel reset (#7352) Unused code: HostDBContinuation::removeEvent (#7383) Traffic Dump: Fix stream-id printing after first transaction. (#7311) Add comments to ink_queue.h. (#7376) Cleanup incoming PROXY Protocol v1 (#7331) In CI, only run autopep8 on branches that enforce autopep8 (#7270) Fix FreeBSD 12 link issue in test_libhttp2. (#7367) Adjust flags to ensure tunnel producer is cleaned up (#7336) Cleanup: Remove SSL Wire Trace releated code in UnixNetVConnection (#7368) Use EVP MAC API if available (#7363) Use EVP API instead of MD5_Init/Update/Final (secure_link plugin) (#7355) Use ERR_get_error_all if available (#7354) Use OpeSSL EVP API instead of SHA256_Init/Update/Final (#7342) Cleanup: Get rid of NetVConnection::outstanding() (#7366) Cleanup: Remove unused functions (#7365) Add a post case to the conn_timeout test (#7334) Fix sni ip_allow and host_sni_policy (#7349) AuTest for Split DNS (#7325) Make reloading client certificate configuration more reliable (#7313) Add negative caching tests and fixes. (#7361) ESI: Ensure gzip header is always initialized (#7360) Allow for regex_remap of pristine URL. (#7347) Set thread mutex to the DNSHandler mutex of SplitDNS (#7321) Fix lookup split dns rule with fast path (#7320) Add note to background fetch about include/exclude (#7343) AuTest for incoming PROXY Protocol v1 (#7326) Fix vc close migration race condition (#7337) TLS Session Reuse: Downgrade add_session messages to debug (#7345) TLS Session Reuse: Downgrade noisy log to debug (#7344) Remove the last remnants of the enable_url_expandomatic (#7276) Remove unnecessary cast from ReverseProxy. (#7329) Updates the Dockerfile with more packages (#7323) fixup in HttpSM to only set [TS_MILESTONE_SERVER_CLOSE if TS_MILESTONE_SERVER_CONNECT has been set (#7259) Add option for hybrid global and thread session pools (#6978) Get appropriate locks on SSN_START hook delays (#7295) s3_auth: demote noisy errors around configuration that doesn't affect plugin usability (#7306) Follow the comments in I_Thread.h, add an independent ink_thread_key for EThread. (#6288) Reduce the number of write operation on H2 (#7282) commit 5704095 Merge: 882a79d 0c88b24 Author: Masakazu Kitajo <maskit@apache.org> Date: Wed Oct 28 21:06:11 2020 +0900 Merge branch 'master' into quic-latest * master: Adds a shell script to help build the H3 toolchains (#7299) Remove unfinished h2c support (#7286) Allow disabling SO_MARK and IP_TOS usage (#7292) Enable all h2spec test (#7289) Fix bad HTTP/2 post client causing stuck HttpSM (#7237) Sticky server does not work with H2 client (#7261) 7096: Synchronize Server Session Management and Network I/O (#7278) HostDB: remove unused field in HostDBApplicationInfo, and update remaining types in http_data to fix broken padding. (#7264) Add support for a new (TSMgmtDataTypeGet) mgmt API function to retrieve the record data type (#7221) Fix example in default sni.yaml configuration. (#7277) Fix proxy.process.http.current_client_transactions (#7258) Add AuTest for HTTP/2 Graceful Shutdown (#7271) Fix truncated reponse on HTTP/2 graceful shutdown (#7267) url_sig add 'ignore_expiry = true' option for log replay testing (#7231) Respecting default rolling_enabled in plugins. (#7275) gracefully handle TSReleaseAsserts in statichit and generator plugins (#7269) Removes commented out code from esi plugin (#7273) Allow initial // in request targets. (#7266) Document external log rotation support via SIGUSR2 (#7265) Let Dedicated EThreads use `EThread::schedule` (#7228) HostDB: Fix cache data version checking to use full version, not major version. (#7263) Bugfix: set a default inactivity timeout only if a read or write I/O operation was set (#7226) Treat objects with negative max-age CC directives as stale. (#7260) Remove some usless defines, which just obsfucates code (#7252) Remove useless if for port set assertion. (#7250) Fix test_error_page_selection memory leaks and logic errors (#7248) [multiplexer] option to skip post/put requests (#7233) Incorporates the latest CI build changes (#7251) Add support for server protocol stack API (#7239) Fix for plugins ASAN suppression file (#7249) RolledLogDeleter: do not sort on each candidate consideration. (#7243) Make double Au test more reliable. (#7216) Ensure that ca override does not get lost (#7219) Stop crash on disk failure (#7218) Do not cache Transfer-Encoding header (#7234) clean up body factory tests (#7236) Revert "Create an explicit runroot.yaml for AuTests (#7177)" (#7235) New option to dead server to not retry during dead period (#7142) Increment ssl_error_syscall only if not EOF (#7225) Fix renamed setting in default config (#7224) Log config reload: use new config for initialization (#7215) Introduce proxy-verifier to AuTests (#7211) Follow redirection responses when refreshing stale cache objects. (#7213) Create an explicit runroot.yaml for AuTests (#7177) Support external log rotation tools via SIGUSR2 (#6806) Add support for TS API for Note, Status, Warning, Alert (#7208) If the weight is 0, the SRV record should be selected from the highest priority group (#7206) Cleanup: remove unnecessary memset() within dns_process() (#7209) Docs cleanup (#7210) Strip whitespaces after field-name and before the colon in headers from the origin (#7202) Adds new plugin: statichit (#7173) Add duplicate header field processing when creating outgoing response (#7207) commit 882a79d Merge: 2a9887f bb5c390 Author: Masakazu Kitajo <maskit@apache.org> Date: Fri Sep 18 10:01:14 2020 +0900 Merge branch 'master' into quic-latest * master: Rename ambiguous log variable (#7199) KWF useless member function HttpSM::kill_this_async_hook(). (#7198) Fix the active_timeout test to work without quic enabled (#7197) Remove obsolete cdn_ HttpTransact vars (#7182) Remove unused HttpUpdate mechanism (#7194) Updates the list of supported / linked Docs versions (#7152) Make custom xdebug HTTP header name available to other plugins. (#7193) Update sni outbound policy to allow directly setting the outbound SNI. (#7188) commit 2a9887f Author: Masakazu Kitajo <maskit@apache.org> Date: Wed Sep 16 17:54:01 2020 +0900 Avoid unnecessary QUIC CID randomization commit 42e8898 Author: Masakazu Kitajo <maskit@apache.org> Date: Tue Sep 15 12:41:28 2020 +0900 Simplify interface between H3 and QUIC, and remove memcopy between them commit 112fc71 Merge: ac31ada b090964 Author: Masakazu Kitajo <maskit@apache.org> Date: Tue Sep 15 09:21:25 2020 +0900 Merge branch 'master' into quic-latest * master: Add an autest testcase for HTTP3 (#7063) Fix TSHttpTxnServerPacket* API's to correctly update existing server connections (#7175) Do not lose original inactivity timeout on disable (#7134) Emits log when OCSP fails to connect to server (#7183) autopep8: avoid running on non-tracked files. (#7186) TextView: Add additional constructor tests. (#7189) Remove duplicate code (#7180) TextView: add constructor size values to enable strlen even for null pointers. (#7185) Add virtual destructor to QUICRTTProvider. (#7184) AuTest: Reuse venv if it exists already (#7178) TS_API for Note,Status,Warning,Alert,Fatal (#7181) Traffic Dump: Record HTTP/2 priority. (#7149) leaks in logs (#7172) Additions to enable loading qat_engine (#7150) Removes references to non-existent function handle_conditional_headers (#7162) Fix #7164 Chaning Warning to Debug and creating a stat for inserting duplicates to pending dns (#7166) Fix #7167, make autopep8 failure (#7168) MicroDNS Extension: handle different 'default' types (#7159) Traffic Dump documentation for post_process.py (#7161) Fix memory leaks in multiplexer plugin (#7160) rc: fixes systemd unit file stopping (#7157) Fix lua plugin mem leak problem (#7158) Don't make an error on duplicated RETIRE_CONNECTION frames (#7131) URL::parse fixes for empty paths (#7119) Replace ACTION_RESULT_NONE with nullptr (#7135) Add metric tracking async job pauses (#7153) PluginFactory - Remove unused code that was left from last PluginFactory change(TSPluginDSOReloadEnable) (#7155) Fix stale pointer due to SSL config reload (#7148) slice: check if vio is still valid before calling TSVIODone* on shutdown (#7147) Deprecate cqhv field (#7143) Don't return QUIC frame if the size exceeds maximum frame size (#7121) Check VIO availability before acquiring a lock for it (#7145) Fix #7116, skip the insertion of the same continuation to pending dns (#7117) Allow override of CA certs for cert from client based on SNI server name sent by client. (#7130) Fix typo in cache docs (#7144) remove useless shortopt (#7138) Protect TSActionCancel from null INKContInternal actions (#7128) Check VIO availability before checking whether the VIO has data (#7120) Accept NAT rebinding on a QUIC connection (#7123) Fixes garbled logs when using %<vbn> log tag (#7140) Removes duplicated listing of files in same Makefile target (#7137) Updated gdb mutex script to get process file for Fedora 32 (#7133) SSLConfig mem leak fix (#7125) Replaces "smart" quotes with ASCII equivalents (#7126) Comment out a wrong assertion in QUIC Loss Detection logic (#7129) Add member initialization to the Errata class. (#7132) Cancel active/inactive timeout on closing Http2Stream (#7111) Add modsecurity lua script to example (#7105) Expose remap config file callback (#7073) Make tls_hooks tests more likely to pass (#7122) commit ac31ada Merge: 4d579f4 e904dbc Author: Masakazu Kitajo <maskit@apache.org> Date: Mon Aug 17 09:14:14 2020 +0900 Merge branch 'master' into quic-latest * master: Backing out my update of our jenkin's autest file. (#7118) Don't send image/webp responses from cache to broswers that don't support it (#7104) Updating our autest suite to require Python3.6 (#7113) Squashed commit of the following: (#7110) Supporting out of source builds for AuTests. (#7109) Fixes uninitialized variables found by Xcode (#7100) Add cross references between server session sharing match and upstream connection tracking match. (#7038)
1 parent 658e3cd commit 9590447

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

53 files changed

+2003
-1905
lines changed

iocore/net/quic/Makefile.am

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,8 @@ libquic_a_SOURCES = \
6262
QUICNewRenoCongestionController.cc \
6363
QUICFlowController.cc \
6464
QUICStreamState.cc \
65+
QUICStreamAdapter.cc \
66+
QUICStreamVCAdapter.cc \
6567
QUICStream.cc \
6668
QUICHandshake.cc \
6769
QUICPacketHeaderProtector.cc \
@@ -92,6 +94,7 @@ libquic_a_SOURCES = \
9294
QUICFrameGenerator.cc \
9395
QUICFrameRetransmitter.cc \
9496
QUICAddrVerifyState.cc \
97+
QUICTransferProgressProvider.cc \
9598
QUICBidirectionalStream.cc \
9699
QUICCryptoStream.cc \
97100
QUICUnidirectionalStream.cc \

iocore/net/quic/Mock.h

Lines changed: 85 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@
3636
#include "QUICPinger.h"
3737
#include "QUICPadder.h"
3838
#include "QUICHandshakeProtocol.h"
39+
#include "QUICStreamAdapter.h"
3940

4041
class MockQUICContext;
4142

@@ -717,6 +718,76 @@ class MockQUICLossDetector : public QUICLossDetector
717718
MockQUICCongestionController _cc;
718719
};
719720

721+
class MockQUICStreamAdapter : public QUICStreamAdapter
722+
{
723+
public:
724+
MockQUICStreamAdapter(QUICStream &stream) : QUICStreamAdapter(stream) {}
725+
726+
void
727+
write_to_stream(const uint8_t *buf, size_t len)
728+
{
729+
this->_total_sending_data_len += len;
730+
this->_sending_data_len += len;
731+
}
732+
733+
int64_t
734+
write(QUICOffset offset, const uint8_t *data, uint64_t data_length, bool fin) override
735+
{
736+
this->_total_receiving_data_len += data_length;
737+
this->_receiving_data_len += data_length;
738+
return data_length;
739+
}
740+
bool
741+
is_eos() override
742+
{
743+
return false;
744+
}
745+
uint64_t
746+
unread_len() override
747+
{
748+
return this->_sending_data_len;
749+
}
750+
uint64_t
751+
read_len() override
752+
{
753+
return 0;
754+
}
755+
uint64_t
756+
total_len() override
757+
{
758+
return this->_total_sending_data_len;
759+
}
760+
void
761+
encourge_read() override
762+
{
763+
}
764+
void
765+
encourge_write() override
766+
{
767+
}
768+
void
769+
notify_eos() override
770+
{
771+
}
772+
773+
protected:
774+
Ptr<IOBufferBlock>
775+
_read(size_t len) override
776+
{
777+
this->_sending_data_len -= len;
778+
Ptr<IOBufferBlock> block = make_ptr<IOBufferBlock>(new_IOBufferBlock());
779+
block->alloc(iobuffer_size_to_index(len, BUFFER_SIZE_INDEX_32K));
780+
block->fill(len);
781+
return block;
782+
}
783+
784+
private:
785+
size_t _sending_data_len = 0;
786+
size_t _total_sending_data_len = 0;
787+
size_t _receiving_data_len = 0;
788+
size_t _total_receiving_data_len = 0;
789+
};
790+
720791
class MockQUICApplication : public QUICApplication
721792
{
722793
public:
@@ -726,20 +797,29 @@ class MockQUICApplication : public QUICApplication
726797
main_event_handler(int event, Event *data)
727798
{
728799
if (event == 12345) {
729-
QUICStreamIO *stream_io = static_cast<QUICStreamIO *>(data->cookie);
730-
stream_io->write_reenable();
731800
}
732801
return EVENT_CONT;
733802
}
734803

804+
void
805+
on_new_stream(QUICStream &stream) override
806+
{
807+
auto ite = this->_streams.emplace(stream.id(), stream);
808+
QUICStreamAdapter &adapter = ite.first->second;
809+
stream.set_io_adapter(&adapter);
810+
}
811+
735812
void
736813
send(const uint8_t *data, size_t size, QUICStreamId stream_id)
737814
{
738-
QUICStreamIO *stream_io = this->_find_stream_io(stream_id);
739-
stream_io->write(data, size);
815+
auto ite = this->_streams.find(stream_id);
816+
auto &adapter = ite->second;
817+
adapter.write_to_stream(data, size);
740818

741-
eventProcessor.schedule_imm(this, ET_CALL, 12345, stream_io);
819+
// eventProcessor.schedule_imm(this, ET_CALL, 12345, adapter);
742820
}
821+
822+
std::unordered_map<QUICStreamId, MockQUICStreamAdapter> _streams;
743823
};
744824

745825
class MockQUICPacketR : public QUICPacketR

iocore/net/quic/QUICApplication.cc

Lines changed: 1 addition & 247 deletions
Original file line numberDiff line numberDiff line change
@@ -24,172 +24,6 @@
2424
#include "QUICApplication.h"
2525
#include "QUICStream.h"
2626

27-
static constexpr char tag_stream_io[] = "quic_stream_io";
28-
static constexpr char tag_app[] = "quic_app";
29-
30-
#define QUICStreamIODebug(fmt, ...) \
31-
Debug(tag_stream_io, "[%s] [%" PRIu64 "] " fmt, this->_stream_vc->connection_info()->cids().data(), this->_stream_vc->id(), \
32-
##__VA_ARGS__)
33-
34-
//
35-
// QUICStreamIO
36-
//
37-
QUICStreamIO::QUICStreamIO(QUICApplication *app, QUICStreamVConnection *stream_vc) : _stream_vc(stream_vc)
38-
{
39-
this->_read_buffer = new_MIOBuffer(BUFFER_SIZE_INDEX_8K);
40-
this->_write_buffer = new_MIOBuffer(BUFFER_SIZE_INDEX_8K);
41-
42-
this->_read_buffer_reader = this->_read_buffer->alloc_reader();
43-
this->_write_buffer_reader = this->_write_buffer->alloc_reader();
44-
45-
switch (stream_vc->direction()) {
46-
case QUICStreamDirection::BIDIRECTIONAL:
47-
this->_read_vio = stream_vc->do_io_read(app, INT64_MAX, this->_read_buffer);
48-
this->_write_vio = stream_vc->do_io_write(app, INT64_MAX, this->_write_buffer_reader);
49-
break;
50-
case QUICStreamDirection::SEND:
51-
this->_write_vio = stream_vc->do_io_write(app, INT64_MAX, this->_write_buffer_reader);
52-
break;
53-
case QUICStreamDirection::RECEIVE:
54-
this->_read_vio = stream_vc->do_io_read(app, INT64_MAX, this->_read_buffer);
55-
break;
56-
default:
57-
ink_assert(false);
58-
break;
59-
}
60-
}
61-
62-
QUICStreamIO::~QUICStreamIO()
63-
{
64-
// All readers will be deallocated
65-
free_MIOBuffer(this->_read_buffer);
66-
free_MIOBuffer(this->_write_buffer);
67-
};
68-
69-
uint32_t
70-
QUICStreamIO::stream_id() const
71-
{
72-
return this->_stream_vc->id();
73-
}
74-
75-
bool
76-
QUICStreamIO::is_bidirectional() const
77-
{
78-
return this->_stream_vc->is_bidirectional();
79-
}
80-
81-
int64_t
82-
QUICStreamIO::read(uint8_t *buf, int64_t len)
83-
{
84-
if (is_debug_tag_set(tag_stream_io)) {
85-
if (this->_read_vio->nbytes == INT64_MAX) {
86-
QUICStreamIODebug("nbytes=- ndone=%" PRId64 " read_avail=%" PRId64 " read_len=%" PRId64, this->_read_vio->ndone,
87-
this->_read_buffer_reader->read_avail(), len);
88-
} else {
89-
QUICStreamIODebug("nbytes=%" PRId64 " ndone=%" PRId64 " read_avail=%" PRId64 " read_len=%" PRId64, this->_read_vio->nbytes,
90-
this->_read_vio->ndone, this->_read_buffer_reader->read_avail(), len);
91-
}
92-
}
93-
94-
int64_t nread = this->_read_buffer_reader->read(buf, len);
95-
if (nread > 0) {
96-
this->_read_vio->ndone += nread;
97-
}
98-
99-
this->_stream_vc->on_read();
100-
101-
return nread;
102-
}
103-
104-
int64_t
105-
QUICStreamIO::peek(uint8_t *buf, int64_t len)
106-
{
107-
return this->_read_buffer_reader->memcpy(buf, len) - reinterpret_cast<char *>(buf);
108-
}
109-
110-
void
111-
QUICStreamIO::consume(int64_t len)
112-
{
113-
this->_read_buffer_reader->consume(len);
114-
this->_stream_vc->on_read();
115-
}
116-
117-
bool
118-
QUICStreamIO::is_read_done() const
119-
{
120-
return this->_read_vio->ntodo() == 0;
121-
}
122-
123-
int64_t
124-
QUICStreamIO::write(const uint8_t *buf, int64_t len)
125-
{
126-
SCOPED_MUTEX_LOCK(lock, this->_write_vio->mutex, this_ethread());
127-
128-
int64_t nwritten = this->_write_buffer->write(buf, len);
129-
if (nwritten > 0) {
130-
this->_nwritten += nwritten;
131-
}
132-
133-
return len;
134-
}
135-
136-
int64_t
137-
QUICStreamIO::write(IOBufferReader *r, int64_t len)
138-
{
139-
SCOPED_MUTEX_LOCK(lock, this->_write_vio->mutex, this_ethread());
140-
141-
int64_t bytes_avail = this->_write_buffer->write_avail();
142-
143-
if (bytes_avail > 0) {
144-
if (is_debug_tag_set(tag_stream_io)) {
145-
if (this->_write_vio->nbytes == INT64_MAX) {
146-
QUICStreamIODebug("nbytes=- ndone=%" PRId64 " write_avail=%" PRId64 " write_len=%" PRId64, this->_write_vio->ndone,
147-
bytes_avail, len);
148-
} else {
149-
QUICStreamIODebug("nbytes=%" PRId64 " ndone=%" PRId64 " write_avail=%" PRId64 " write_len=%" PRId64,
150-
this->_write_vio->nbytes, this->_write_vio->ndone, bytes_avail, len);
151-
}
152-
}
153-
154-
int64_t bytes_len = std::min(bytes_avail, len);
155-
int64_t nwritten = this->_write_buffer->write(r, bytes_len);
156-
157-
if (nwritten > 0) {
158-
this->_nwritten += nwritten;
159-
}
160-
161-
return nwritten;
162-
} else {
163-
return 0;
164-
}
165-
}
166-
167-
// TODO: Similar to other "write" apis, but do not copy.
168-
int64_t
169-
QUICStreamIO::write(IOBufferBlock *b)
170-
{
171-
ink_assert(!"not implemented yet");
172-
return 0;
173-
}
174-
175-
void
176-
QUICStreamIO::write_done()
177-
{
178-
this->_write_vio->nbytes = this->_nwritten;
179-
}
180-
181-
void
182-
QUICStreamIO::read_reenable()
183-
{
184-
return this->_read_vio->reenable();
185-
}
186-
187-
void
188-
QUICStreamIO::write_reenable()
189-
{
190-
return this->_write_vio->reenable();
191-
}
192-
19327
//
19428
// QUICApplication
19529
//
@@ -198,84 +32,4 @@ QUICApplication::QUICApplication(QUICConnection *qc) : Continuation(new_ProxyMut
19832
this->_qc = qc;
19933
}
20034

201-
QUICApplication::~QUICApplication()
202-
{
203-
for (auto const &kv : this->_stream_map) {
204-
delete kv.second;
205-
}
206-
}
207-
208-
// @brief Bind stream and application
209-
void
210-
QUICApplication::set_stream(QUICStreamVConnection *stream_vc, QUICStreamIO *stream_io)
211-
{
212-
if (stream_io == nullptr) {
213-
stream_io = new QUICStreamIO(this, stream_vc);
214-
}
215-
this->_stream_map.insert(std::make_pair(stream_vc->id(), stream_io));
216-
}
217-
218-
// @brief Bind stream and application
219-
void
220-
QUICApplication::set_stream(QUICStreamIO *stream_io)
221-
{
222-
this->_stream_map.insert(std::make_pair(stream_io->stream_id(), stream_io));
223-
}
224-
225-
bool
226-
QUICApplication::is_stream_set(QUICStreamVConnection *stream)
227-
{
228-
auto result = this->_stream_map.find(stream->id());
229-
230-
return result != this->_stream_map.end();
231-
}
232-
233-
void
234-
QUICApplication::reenable(QUICStreamVConnection *stream)
235-
{
236-
QUICStreamIO *stream_io = this->_find_stream_io(stream->id());
237-
if (stream_io) {
238-
stream_io->read_reenable();
239-
stream_io->write_reenable();
240-
} else {
241-
Debug(tag_app, "[%s] Unknown Stream id=%" PRIx64, this->_qc->cids().data(), stream->id());
242-
}
243-
244-
return;
245-
}
246-
247-
void
248-
QUICApplication::unset_stream(QUICStreamVConnection *stream)
249-
{
250-
QUICStreamIO *stream_io = this->_find_stream_io(stream->id());
251-
if (stream_io) {
252-
this->_stream_map.erase(stream->id());
253-
}
254-
}
255-
256-
QUICStreamIO *
257-
QUICApplication::_find_stream_io(QUICStreamId id)
258-
{
259-
auto result = this->_stream_map.find(id);
260-
261-
if (result == this->_stream_map.end()) {
262-
return nullptr;
263-
} else {
264-
return result->second;
265-
}
266-
}
267-
268-
QUICStreamIO *
269-
QUICApplication::_find_stream_io(VIO *vio)
270-
{
271-
if (vio == nullptr) {
272-
return nullptr;
273-
}
274-
275-
QUICStream *stream = dynamic_cast<QUICStream *>(vio->vc_server);
276-
if (stream == nullptr) {
277-
return nullptr;
278-
}
279-
280-
return this->_find_stream_io(stream->id());
281-
}
35+
QUICApplication::~QUICApplication() {}

0 commit comments

Comments
 (0)