Skip to content

Commit 111ec38

Browse files
abhinavsinghpre-commit-ci[bot]dependabot[bot]
authored
v2.4.0rc5 (#929)
* Merge pull request #896 from abhinavsingh/missed-scenarios Avoid registering invalid FD with selectors * [ProxyPool] Add support for basic authorization with upstream proxies (#897) * Use `Url` class to parse proxy pool entries * Add support for parsing user:pass from raw url bytes * Add `httpHeaders.PROXY_AUTHORIZATION` headers for upstream proxies * Add support for httpHeaders enum * Send base64 encoded proxy authorization header to upstream proxies * mypy fixes * Document proxy pool authentication support usage info * Add `conn_close` kwarg to packet builder utilities (#898) * Add `conn_close` kwarg to packet builder utilities, passing True will automatically add `Connection: close` header * Add `conn_close` to `HttpRequestRejected` responses * Raise `HttpProtocolException` instead of `ValueError` (#899) * Raise `HttpProtocolException` instead of `ValueError` for clean teardown of the offending connection * Fix circular import errors * Fix tests * fix lint errors * Avoid containerizing until check has passed * Ensure message for every `HttpProtocolException` raised (#900) * Response Packet Utilities (#903) * Add response pkt utility * Unused import * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix tests as some content is now by default gzipped based upon min compression config * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused * Update necessary tests to use `okResponse` utility * Add option to explicitly disable compression Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Introduce `ProgramNamePlugin` plugin (#904) * Add `ProgramNamePlugin` * Update readme * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove `_compat.py` * Add suggestions coming from #659 Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update defaults for `--hostname` and `--local-executor` (#905) * do it * Change defaults * integrate with ci/cd updates * Fix ci * Add dockerfile `SKIP_OPENSSL` option which will allow us to build container without openssl * Skip openssl for latest tag, add another openssl tag for images with openssl support * Push separate openssl image to GHCR for every PR * `Work` can also be `TcpServerConnection`, not just `TcpClientConnection` (#906) * Work can also be TcpServerConnection, not just TcpClientConnection. More over, it can be any generic work type * Add py_class_role and py_obj * Port internal integration tests into public repo * Fix proxy py addr * Use cross-platform compat shasum * Change `--local-executor` flag semantics (#907) * Convert `--local-executor` in an integer flag, defaults to 1 i.e. enabled, use 0 to disable * Consider any value other than 1 as remote mode * Use integer to disable local executor mode in integration tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix mypy errors * Update flags in readme * Check for type * Remove pid file check for now * Update `--local-executor` flag usage Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Clean shutdown on `SIGINT`, `SIGHUP`, `SIGTERM`, `SIGQUIT` (#908) * sys.exit on SIGINT, SIGHUP, SIGTERM * Add todo for pending signal actions * SIGHUP is win only * Remove frametype signature as it causes lint issues and we are not using it anyways * SIGQUIT is not on Win * Fix `HttpWebServerPacFilePlugin` broken routes logic (#915) * Fix `HttpWebServerPacFilePlugin` broken routes logic * lint * Proxy Auto-Configuration (PAC) file should not be compressed (#916) * Move `UpstreamConnectionPool` lifecycle within `Threadless` (#917) * Tie connection pool into Threadless * Pass upstream conn pool reference to work instances * Mark upstream conn pool as optional * spellcheck * Fix unused import * Define work lifecycle events for pool (#918) * Define work lifecycle events for pool * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Use isinstance * Use mocker fixture to pass CI on 3.6 and 3.7 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Hook `UpstreamConnectionPool` lifecycle within `Threadless` (#921) * Hook connection pool lifecycle within threadless * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix test * Fix spell Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * pip prod(deps): bump sphinx from 4.3.1 to 4.3.2 (#902) Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.3.1 to 4.3.2. - [Release notes](https://github.com/sphinx-doc/sphinx/releases) - [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES) - [Commits](sphinx-doc/sphinx@v4.3.1...v4.3.2) --- updated-dependencies: - dependency-name: sphinx dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <[email protected]> * pip prod(deps): bump paramiko from 2.8.1 to 2.9.1 (#923) * Optimize how `HttpProtocolHandler` delegates to the core plugins (#925) * Add `protocols` abstract static method to `HttpProtocolHandlerBase` which defines which HTTP specification is followed by the core plugin * lint * Fix tests * Lint fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TlsParser] Refactored implementation from #748 (#922) * Refactored TlsParser based upon work done in #748 * Add missing `tls_server_hello.data`, thanks to @JerryKwan * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pass `check.py` * Run check.py locally * Fix lint errors * Fix indentation issue * Ignore linkcheck for cloudflare links, GHA is getting a 403 reply, while the link actually works * Fix lint * codespell skip Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * pip prod(deps): bump tox from 3.24.4 to 3.24.5 (#924) Bumps [tox](https://github.com/tox-dev/tox) from 3.24.4 to 3.24.5. - [Release notes](https://github.com/tox-dev/tox/releases) - [Changelog](https://github.com/tox-dev/tox/blob/master/docs/changelog.rst) - [Commits](tox-dev/tox@3.24.4...3.24.5) --- updated-dependencies: - dependency-name: tox dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <[email protected]> * pip prod(deps): bump twine from 3.7.0 to 3.7.1 (#927) Bumps [twine](https://github.com/pypa/twine) from 3.7.0 to 3.7.1. - [Release notes](https://github.com/pypa/twine/releases) - [Changelog](https://github.com/pypa/twine/blob/main/docs/changelog.rst) - [Commits](pypa/twine@3.7.0...3.7.1) --- updated-dependencies: - dependency-name: twine dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Abhinav Singh <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2 parents 2784cfb + 4b915c9 commit 111ec38

File tree

95 files changed

+2639
-886
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

95 files changed

+2639
-886
lines changed

.flake8

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,6 @@ extend-ignore =
148148
WPS420 # FIXME: pointless keyword like `pass`
149149
WPS421 # FIXME: call to `print()`
150150
WPS425 # FIXME: bool non-keyword arg
151-
WPS427 # FIXME: unreachable code
152151
WPS428 # FIXME: pointless statement
153152
WPS430 # FIXME: nested func
154153
WPS431 # FIXME: nested class

.github/workflows/test-library.yml

Lines changed: 82 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -247,8 +247,10 @@ jobs:
247247
path: ${{ steps.pip-cache.outputs.dir }}
248248
key: >-
249249
${{ runner.os }}-pip-${{
250-
steps.calc-cache-key-py.outputs.py-hash-key }}-${{
251-
hashFiles('tox.ini') }}
250+
steps.calc-cache-key-py.outputs.py-hash-key
251+
}}-${{
252+
hashFiles('tox.ini')
253+
}}
252254
restore-keys: |
253255
${{ runner.os }}-pip-${{
254256
steps.calc-cache-key-py.outputs.py-hash-key
@@ -370,8 +372,10 @@ jobs:
370372
path: ${{ steps.pip-cache.outputs.dir }}
371373
key: >-
372374
${{ runner.os }}-pip-${{
373-
steps.calc-cache-key-py.outputs.py-hash-key }}-${{
374-
hashFiles('tox.ini') }}
375+
steps.calc-cache-key-py.outputs.py-hash-key
376+
}}-${{
377+
hashFiles('tox.ini')
378+
}}
375379
restore-keys: |
376380
${{ runner.os }}-pip-${{
377381
steps.calc-cache-key-py.outputs.py-hash-key
@@ -485,12 +489,13 @@ jobs:
485489
path: ${{ steps.pip-cache.outputs.dir }}
486490
key: >-
487491
${{ runner.os }}-pip-${{
488-
steps.calc-cache-key-py.outputs.py-hash-key }}-${{
489-
hashFiles('tox.ini', 'requirements.txt', 'requirements-testing.txt')
492+
steps.calc-cache-key-py.outputs.py-hash-key
493+
}}-${{
494+
hashFiles('tox.ini', 'requirements**.txt')
490495
}}
491496
restore-keys: |
492497
${{ runner.os }}-pip-${{
493-
steps.calc-cache-key-py.outputs.py-hash-key
498+
steps.calc-cache-key-py.outputs.py-hash-key
494499
}}-
495500
${{ runner.os }}-pip-
496501
- name: Install tox
@@ -667,19 +672,41 @@ jobs:
667672
make ca-certificates
668673
python3 -m proxy --version
669674
675+
check: # This job does nothing and is only used for the branch protection
676+
if: always()
677+
678+
needs:
679+
- analyze
680+
- test
681+
- lint
682+
- dashboard
683+
- brew
684+
- developer
685+
686+
runs-on: Ubuntu-latest
687+
688+
steps:
689+
- name: Decide whether the needed jobs succeeded or failed
690+
uses: re-actors/alls-green@release/v1
691+
with:
692+
jobs: ${{ toJSON(needs) }}
693+
670694
docker:
671695
runs-on: Ubuntu-latest
672696
permissions:
673697
packages: write
698+
if: success()
674699
needs:
675-
- build
700+
- check
676701
- pre-setup # transitive, for accessing settings
677702
name: 🐳 containerize
678703
strategy:
679704
fail-fast: false
680705
steps:
681706
- name: Checkout
682707
uses: actions/checkout@v2
708+
with:
709+
ref: ${{ github.event.inputs.release-commitish }}
683710
- name: Download all the dists
684711
uses: actions/download-artifact@v2
685712
with:
@@ -727,7 +754,8 @@ jobs:
727754
$CONTAINER_TAG
728755
--hostname 0.0.0.0
729756
--enable-web-server
730-
--local-executor && ./tests/integration/test_integration.sh 8899
757+
&&
758+
./tests/integration/test_integration.sh 8899
731759
- name: Push to GHCR
732760
run: >-
733761
REGISTRY_URL="ghcr.io/abhinavsingh/proxy.py";
@@ -739,15 +767,19 @@ jobs:
739767
--platform ${{
740768
needs.pre-setup.outputs.container-platforms
741769
}}
770+
--build-arg SKIP_OPENSSL=1
742771
--build-arg PROXYPY_PKG_PATH='dist/${{
743772
needs.pre-setup.outputs.wheel-artifact-name
744773
}}'
745774
-t $CONTAINER_TAG .
746775
- name: Tag latest on GHCR
747776
if: >-
748777
github.event_name == 'push' &&
749-
github.ref == format(
750-
'refs/heads/{0}', github.event.repository.default_branch
778+
(
779+
github.ref == format(
780+
'refs/heads/{0}', github.event.repository.default_branch
781+
) ||
782+
github.ref == 'refs/heads/master'
751783
)
752784
run: >-
753785
REGISTRY_URL="ghcr.io/abhinavsingh/proxy.py";
@@ -757,6 +789,43 @@ jobs:
757789
--platform ${{
758790
needs.pre-setup.outputs.container-platforms
759791
}}
792+
--build-arg SKIP_OPENSSL=1
793+
--build-arg PROXYPY_PKG_PATH='dist/${{
794+
needs.pre-setup.outputs.wheel-artifact-name
795+
}}'
796+
-t $LATEST_TAG .
797+
- name: Push openssl to GHCR
798+
run: >-
799+
REGISTRY_URL="ghcr.io/abhinavsingh/proxy.py";
800+
CONTAINER_TAG=$REGISTRY_URL:${{
801+
needs.pre-setup.outputs.container-version
802+
}}-openssl;
803+
docker buildx build
804+
--push
805+
--platform ${{
806+
needs.pre-setup.outputs.container-platforms
807+
}}
808+
--build-arg PROXYPY_PKG_PATH='dist/${{
809+
needs.pre-setup.outputs.wheel-artifact-name
810+
}}'
811+
-t $CONTAINER_TAG .
812+
- name: Tag openssl on GHCR
813+
if: >-
814+
github.event_name == 'push' &&
815+
(
816+
github.ref == format(
817+
'refs/heads/{0}', github.event.repository.default_branch
818+
) ||
819+
github.ref == 'refs/heads/master'
820+
)
821+
run: >-
822+
REGISTRY_URL="ghcr.io/abhinavsingh/proxy.py";
823+
LATEST_TAG=$REGISTRY_URL:openssl;
824+
docker buildx build
825+
--push
826+
--platform ${{
827+
needs.pre-setup.outputs.container-platforms
828+
}}
760829
--build-arg PROXYPY_PKG_PATH='dist/${{
761830
needs.pre-setup.outputs.wheel-artifact-name
762831
}}'
@@ -766,6 +835,7 @@ jobs:
766835
with:
767836
username: abhinavsingh
768837
password: ${{ secrets.DOCKER_ACCESS_TOKEN }}
838+
# TODO: openssl image is not published on DockerHub
769839
- name: Push to DockerHub
770840
run: >-
771841
REGISTRY_URL="abhinavsingh/proxy.py";
@@ -777,31 +847,12 @@ jobs:
777847
--platform ${{
778848
needs.pre-setup.outputs.container-platforms
779849
}}
850+
--build-arg SKIP_OPENSSL=1
780851
--build-arg PROXYPY_PKG_PATH='dist/${{
781852
needs.pre-setup.outputs.wheel-artifact-name
782853
}}'
783854
-t $CONTAINER_TAG .
784855
785-
check: # This job does nothing and is only used for the branch protection
786-
if: always()
787-
788-
needs:
789-
- analyze
790-
- test
791-
- lint
792-
- docker
793-
- dashboard
794-
- brew
795-
- developer
796-
797-
runs-on: Ubuntu-latest
798-
799-
steps:
800-
- name: Decide whether the needed jobs succeeded or failed
801-
uses: re-actors/alls-green@release/v1
802-
with:
803-
jobs: ${{ toJSON(needs) }}
804-
805856
publish-pypi:
806857
name: Publish 🐍📦 ${{ needs.pre-setup.outputs.git-tag }} to PyPI
807858
needs:

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ proxy.py.iml
2222
*.crt
2323
*.key
2424
*.pem
25+
26+
.venv*
2527
venv*
2628

2729
cover

.pylintrc

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,7 +127,6 @@ disable=raw-checker-failed,
127127
too-many-return-statements,
128128
too-many-statements,
129129
unnecessary-pass,
130-
unreachable,
131130
unused-argument,
132131
useless-return,
133132
useless-super-delegation,

Dockerfile

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
FROM python:3.10-alpine as base
2+
23
LABEL com.abhinavsingh.name="abhinavsingh/proxy.py" \
34
com.abhinavsingh.description="⚡ Fast • 🪶 Lightweight • 0️⃣ Dependency • 🔌 Pluggable • \
45
😈 TLS interception • 🔒 DNS-over-HTTPS • 🔥 Poor Man's VPN • ⏪ Reverse & ⏩ Forward • \
@@ -8,25 +9,27 @@ LABEL com.abhinavsingh.name="abhinavsingh/proxy.py" \
89
com.abhinavsingh.vcs-url="https://github.com/abhinavsingh/proxy.py" \
910
com.abhinavsingh.docker.cmd="docker run -it --rm -p 8899:8899 abhinavsingh/proxy.py" \
1011
org.opencontainers.image.source="https://github.com/abhinavsingh/proxy.py"
12+
1113
ENV PYTHONUNBUFFERED 1
14+
15+
ARG SKIP_OPENSSL
1216
ARG PROXYPY_PKG_PATH
1317

1418
COPY README.md /
1519
COPY $PROXYPY_PKG_PATH /
20+
1621
RUN pip install --upgrade pip && \
1722
pip install \
1823
--no-index \
1924
--find-links file:/// \
2025
proxy.py && \
2126
rm *.whl
2227

23-
# Install openssl to enable TLS interception & HTTPS proxy options within container
24-
# NOTE: You can comment out this line if you don't intend to use those features.
25-
RUN apk update && apk add openssl
28+
# Use `--build-arg SKIP_OPENSSL=1` to disable openssl installation
29+
RUN if [[ -z "$SKIP_OPENSSL" ]]; then apk update && apk add openssl; fi
2630

2731
EXPOSE 8899/tcp
2832
ENTRYPOINT [ "proxy" ]
2933
CMD [ \
3034
"--hostname=0.0.0.0" \
31-
"--local-executor" \
3235
]

Makefile

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ endif
3131
.PHONY: lib-release-test lib-release lib-profile lib-doc
3232
.PHONY: lib-dep lib-flake8 lib-mypy lib-speedscope container-buildx-all-platforms
3333
.PHONY: container container-run container-release container-build container-buildx
34-
.PHONY: devtools dashboard dashboard-clean
34+
.PHONY: devtools dashboard dashboard-clean container-without-openssl
3535

3636
all: lib-test
3737

@@ -143,7 +143,6 @@ lib-profile:
143143
--num-workers 1 \
144144
--enable-web-server \
145145
--plugin proxy.plugin.WebServerPlugin \
146-
--local-executor \
147146
--backlog 65536 \
148147
--open-file-limit 65536 \
149148
--log-file /dev/null
@@ -160,7 +159,6 @@ lib-speedscope:
160159
--num-workers 1 \
161160
--enable-web-server \
162161
--plugin proxy.plugin.WebServerPlugin \
163-
--local-executor \
164162
--backlog 65536 \
165163
--open-file-limit 65536 \
166164
--log-file /dev/null
@@ -175,12 +173,15 @@ dashboard-clean:
175173
if [[ -d dashboard/public ]]; then rm -rf dashboard/public; fi
176174

177175
container: lib-package
178-
$(MAKE) container-build -e PROXYPY_PKG_PATH=$$(ls dist/*.whl)
176+
docker build \
177+
-t $(PROXYPY_CONTAINER_TAG) \
178+
--build-arg PROXYPY_PKG_PATH=$$(ls dist/*.whl) .
179179

180-
container-build:
180+
container-without-openssl: lib-package
181181
docker build \
182182
-t $(PROXYPY_CONTAINER_TAG) \
183-
--build-arg PROXYPY_PKG_PATH=$(PROXYPY_PKG_PATH) .
183+
--build-arg SKIP_OPENSSL=1 \
184+
--build-arg PROXYPY_PKG_PATH=$$(ls dist/*.whl) .
184185

185186
# Usage:
186187
#

0 commit comments

Comments
 (0)