-
Notifications
You must be signed in to change notification settings - Fork 14
Dev/sftp start: Implementing basic SFTP functionality #29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
jubeormk1
wants to merge
399
commits into
mkj:dev/sftp-start
Choose a base branch
from
jubeormk1:dev/sftp-start
base: dev/sftp-start
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 1 commit
Commits
Show all changes
399 commits
Select commit
Hold shift + click to select a range
188f748
[skip ci] WIP: Hack readdir response was not adding header field length
jubeormk1 33104c6
[skip ci] WIP: BUG in sftphandler process keeps responding to the sam…
jubeormk1 1712866
[skip ci] WIP: BUG The EOF needs to be sent once the readdir has sent…
jubeormk1 37ac507
[skip ci] WIP: Added a simplistic last read status approach
jubeormk1 2352fa9
[skip ci] WIP: Missing StatusCode field in Status derived from SSHEnc…
jubeormk1 c792745
[skip ci] Fixed Status Encoding. Shall I extend the sshwire-derive to…
jubeormk1 88ec688
[skip ci] WIP: First ls command successful
jubeormk1 505fcf8
[skip ci] Implementing EOF responsibility in SftpServer and documenting
jubeormk1 09afb82
[skip ci] Added SunsetSftp Feature: Std
jubeormk1 eeccc76
[skip ci] Extracted DirEntriesCollection and added to std helpers
jubeormk1 b27a1e2
[skip ci] Refactor proto.rs Name structure and
jubeormk1 021e4cd
Basic Features Directory read is now complete
jubeormk1 e6b1c6a
Add the necessary structs/types for env var SendEnv (and avoid WARN -…
brainstorm 76dc4f7
[skip ci] Extending SftpSource and Tests
jubeormk1 5796819
[skip ci] SftpOutputPipe checks if already splits
jubeormk1 8000b21
removed TODO, not a good idea
jubeormk1 79d8c9b
[skip ci] intercepting incomplete UnknownPacket in decode request to …
jubeormk1 bdb508a
Handling properly unknown packets
jubeormk1 c735f6a
One more time I missed the Cargo.lock changes
jubeormk1 bca0a67
Adding test stats: For now I get an expected `Operation unsupported`
jubeormk1 f5de051
refactored std helper to expose get_file_attrs
jubeormk1 cb56c6b
completed getting file stats
jubeormk1 9bc53c3
[skip ci] Fixed misrepresentation of SSH_FXP_OPEN packet
jubeormk1 1d904ad
Fixing SftpTrait and implementations to use the proper Open mode (PFl…
jubeormk1 7cce6fb
Address event naming conventions, remove wants_reply field even if li…
brainstorm 8948861
Add small comment on input string sanitisation state to warn C/FFI bi…
brainstorm 490b830
Merge Environment variable passing support
mkj 00657dc
Use #[expect] rather than allow
mkj 64b26c1
Bump minimum version to 1.87
mkj 43faacb
Fix warnings reported by clippy
mkj 5438ff6
Rename Channels::from_handle_mut to by_handle_mut
mkj 83b56e1
Replace some #[allow] with #[expect]
mkj cf1971e
Small documentation fixes
mkj 083ecc7
Print too-long username length
mkj 0706fbb
Remove unnecessary SSHDecode lifetimes
mkj 1d45cfb
Remove unused imports
mkj fedcbdf
sunsetc: escape banner sent by the server
mkj 78cf4a3
Revert raw_command() to be public
mkj 6edfb9f
Fill some unreleased changes
mkj 7fa2965
WIP: Added SFTP Get capability. Chasing unexpected locks
jubeormk1 c210bb6
SFTP Read stuck: Failing to receive a a read request?
jubeormk1 9dd7570
[skip ci] small changes to make visible SftpOutputPipe total bytes co…
jubeormk1 fe54d2b
[skip ci] Removing outdated check in SftpHandler.process
jubeormk1 0318c63
[skip ci] Removed nested enum in Sftp process FSM
jubeormk1 1c77d6b
[skip ci]some tiding up
jubeormk1 76e1712
adding large file 2MB to test_get.sh
jubeormk1 5806711
[skip ci] Bug? Large write buffers cause for Chan_out cause ssh.serve…
jubeormk1 e41dceb
[skip ci] Debugging runner.read_channel and .input
jubeormk1 e09c03c
Fix building with backtrace feature
mkj e2774a9
traffic: Fix accidentally public payload()
mkj e05ffb6
async: Fix discarded channel input data
mkj ecd64f9
Fix building with backtrace feature
mkj d763b4e
sftp: Fix building with sunset backtrace
mkj 748d9e6
async: Fix discarded channel input data
mkj 2869501
Workaround for NoRoom sending window adjustment
mkj 5c53bfa
Start on sftp proto definitions
mkj f9d6904
Add more changes from March 2025
mkj cf031fd
Bumping the sunset version to match the main project
jubeormk1 56d9d9c
cargo fmt
jubeormk1 a0a7eeb
sunset-sftp added to Cargo.lock
jubeormk1 cf480e9
error fix on casting SftpNum to u8
jubeormk1 1969083
fixing error in sftpmessages macro_rules SSHDecode implementation
jubeormk1 c97f6e3
fixing use of sftp proto
jubeormk1 9fb70f7
Adding some extra uses in proto.rs
jubeormk1 e1f449c
making proto StatusCode pub
jubeormk1 5175016
Fixing sftpmessages SftpNum impl issue
jubeormk1 31158f3
removing reduntant use in proto.rs
jubeormk1 b9dfb55
Added variant to proto StatusCode
jubeormk1 6cdd261
cleaning redundant comments
jubeormk1 a333362
Removing some unused lifetimes that caused issues
jubeormk1 91c8c79
Chasing SSHEncode,SSHDecode issues
jubeormk1 b551f5e
Adding empty vanilla DirReply struct as ReadReply
jubeormk1 1641039
Adding Vanilla ChanOut to finish the placeholders for Dir and Read Reply
jubeormk1 4db4a87
Extra fix for SftpNum to avoid moving issues
jubeormk1 a5834a9
WIP: Replacing sunset::Result with proto::Result
jubeormk1 d193e09
commenting out sftpmessages macro rules to focus on the foundational …
jubeormk1 0ea931e
commenting out Result and ane extra SunsetError implementation too
jubeormk1 164c9ee
Removing phantomData for the moment to avoid SSHEncode/Decode issues
jubeormk1 59a2f9d
Implementing SSHEncode/Decode for Attrs
jubeormk1 5c8fac1
Added a response variant "ResponseAttributes" to complete all server …
jubeormk1 dc86dc9
Fixed typo and change other variant string to ssh_fx_other
jubeormk1 5ffc2d2
isolating proto.rs issues. Commenting out sftpserver
jubeormk1 a44f70c
Modifying packets.rs ParseContext for allowing proto.rs StatusCode to…
jubeormk1 f9d637e
proto.rs StatusCode SSHEncode expanded version seems incorrect.
jubeormk1 e41d1be
As per the pull request fails because of the edition, I am downgradin…
jubeormk1 da9b5cc
Clarifying that this implementation will use SFTP version 3
jubeormk1 1af3776
Uncommenting macro_rules sftpmessages partially until problems arose …
jubeormk1 c1b63ae
Implementing SSHEncode and SSHDecode for Name
jubeormk1 0e6bba5
adding macro_rules! sftpmessages closing curly brackets
jubeormk1 85023fe
StatusCode: SSHEncode and SSHDecode
jubeormk1 c3b83ee
moving sftp edition to 2024
jubeormk1 2dd8c7a
WIP: Starting to implement SftpPacket modifying SftpNum
jubeormk1 a3133b9
Adding SSHEncode and SSHDecode for SftpPacket
jubeormk1 371ca77
Added past dependency in Cargo.lock and as use
jubeormk1 14a2ae6
Restricting SSHDecode for SftpPacket lifetimes so they match
jubeormk1 c40280e
cargo check passing and all the code uncommented. However...
jubeormk1 2397aec
Fixing Avoidable warnings:
jubeormk1 40c3a9e
removing outdated TODOs in sftp/src/proto.rs
jubeormk1 9e46db4
Fix from comment @mkj comment on Name
jubeormk1 a805693
reordering dependencies in sftp to match other workspace members
jubeormk1 eecadde
Starting a new sftp-std demo taking the basic std demo as model
jubeormk1 72f5f0e
adding mod sftpserver as I am going to start working on it next
jubeormk1 109d022
Adding PathInfo SFTP Packet as it is requested by sftp clients after …
jubeormk1 60ef17b
adding some Doc comments
jubeormk1 ba28007
exposing sftp entities in lib.rs
jubeormk1 b3aeff6
fixing impl From<SftpNum> for u8
jubeormk1 b92de89
Added log to sftp
jubeormk1 464c9f7
Big changes: Fixed decoding SftpPacket
jubeormk1 2fd52e4
adding from str for Filename
jubeormk1 d4acbd1
changing ranges for is_init, is_request, is_response
jubeormk1 60a331a
fixing decode_request
jubeormk1 5390313
Fixing the exposed items in SFTP library. More changes will come
jubeormk1 463b76b
added sunset-sftp to sftp demo
jubeormk1 e5c02ba
adding log dep to sunset-sftp
jubeormk1 8cab055
WIP: Adding demosftpserver.rs
jubeormk1 02e4f3e
adding the module demosftpserver to the demo. Reordering uses
jubeormk1 ec5420c
Reordering uses to clarify internal and external dependencies
jubeormk1 345a923
tyding up dependencies
jubeormk1 b9dc9e2
adding trait associated function realpath and struct ItemHandle
jubeormk1 c849b25
WIP Added sftphandle.rs
jubeormk1 ec075a0
Adding as_str for Filename
jubeormk1 47ee3f7
Handling the conditions where the SFTP protocol fails, setting things…
jubeormk1 0004394
removing std uses
jubeormk1 068cb19
Reformed sftpserver
jubeormk1 1309a2c
Bug fix in SftpPacket.encode_request
jubeormk1 c73a81e
Working on sftphandle
jubeormk1 a775274
sftp.rs
jubeormk1 a8fb7de
minor fix in decode_request return value
jubeormk1 3299ecd
changes in demosftpserver very unstable at this point
jubeormk1 9b5936c
handling and logging unsupported or uninitialised messages
jubeormk1 d8447dd
fix bug encoding_request->encoding_response
jubeormk1 dd35aba
Adding trait implementation
jubeormk1 81680e6
Simplifying SftpServer trait async and the Handle are out for now
jubeormk1 8ae10e8
Simple random data files generation for big write testing
jubeormk1 1972940
Adding SFTP Packet Header and SSH_FXP_WRITE offset definitions
jubeormk1 569d033
Simplified Main Demo SFTP loop and reduced the SFTP buffers size
jubeormk1 fcfd0aa
WIP: Working on long WRITE packets processing
jubeormk1 190d38b
WIP: Processing long Write Requests
jubeormk1 e0daffa
Adding basic obscuring of handler
jubeormk1 7474cfb
Minimal SFTP Server implementation of write 64kB write pass the test
jubeormk1 6a938c2
Troubleshooting the 65kB limit: There are two write requests
jubeormk1 1f41c86
WIP: Refactoring opaque_file_handle -> obscure_file_handle and adding…
jubeormk1 16384ea
WIP: Fixed DemoSftpServer definition to use a custom implementation
jubeormk1 fe9152b
WIP: Minimal implementation for DemoSftpServer
jubeormk1 43f5820
opaque_file_handle not obscure_file_handle
jubeormk1 76c2789
Letting the `sftp_loop` and `prog_loop` finish DemoServer.run()
jubeormk1 dfa66dd
Tiding up
jubeormk1 8549e01
Bact to SftpHandle: Process failing as soon as possible on short packets
jubeormk1 4c30d04
Added extra docs to SFTP proto decode_request
jubeormk1 9179c23
Broken: Ugly experimentation with multi part consecutive packets
jubeormk1 4d6f6aa
decoding request len in SftpPacket::decode_request
jubeormk1 329575c
Handing consecutive request in a buffer with some problems
jubeormk1 d1bca6f
WIP: Modifying an error condition to RanOut and adding peak_packet_len
jubeormk1 8f2ed5b
WIP: Restructuring SftpHandle. Adding a FSM
jubeormk1 ac0ed59
WIP: handling ok request in FragmentedRequestState::ProcessingClipped…
jubeormk1 b14ffa9
WIP: Removing dead code
jubeormk1 67735c3
WIP: replacing WireErrors and minor renaming
jubeormk1 69e5e22
WIP: Fixing bugs
jubeormk1 855728b
WIP: Successfully received 100MB and 1024MB
jubeormk1 7fe51a5
WIP: Added SftpError::FileServerError(StatusCode) + others
jubeormk1 930391a
WIP: sftphandle->sftphandler fixing sftperror statuscode casting
jubeormk1 d5b3e01
WIP: Removing dead code and adding comments
jubeormk1 6eecbcb
TODO: Use heapless::Vec instead of Vec for Name as it breaks the no_s…
jubeormk1 bed2a08
WIP: fixed bad tag ids
jubeormk1 268086b
WIP: More docs
jubeormk1 94afb1f
WIP: Refactoring SFTP lib and added comments
jubeormk1 fe25c4a
Partial functionality and refactor completed
jubeormk1 47e03e7
SFTP Roadmap added to docs
jubeormk1 5becfb1
[ci skip] Adding no_std, removing unnecessary use of String
jubeormk1 1320d34
[ci skip] WIP: Added OpenDir operation
jubeormk1 2c9bf39
[skip ci] WIP: Added OpenDir and ReadDir requests
jubeormk1 823ba9b
[skip ci] WIP: moving main loop inside process_loop to have stdio in …
jubeormk1 a7aa58c
[skip ci] WIP: commenting out the no_std restriction for now
jubeormk1 2b5d934
[skip ci] WIP: I missed out adding items to Cargo.lock
jubeormk1 5b8e30c
[skip ci] WIP: Exploring the iterator option
jubeormk1 6b48d00
[skip ci] WIP: Refactored constant
jubeormk1 49a769d
[skip ci] WIP: Reworking a wrapper for Sink and ChanOut
jubeormk1 801086c
[skip ci] WIP: Refactoring. Creating a mod for sftphandler
jubeormk1 335cd4c
[skip ci] WIP: Adding Producer and Consumer to avoid mutable referenc…
jubeormk1 8c087eb
[skip ci] WIP: Improved SftpSource readability
jubeormk1 158998a
[skip ci] WIP: Removing excessive lifetimes to build successfully
jubeormk1 af9bd65
[skip ci] WIP: Improved readability in requestholder.rs
jubeormk1 d837a8e
[skip ci] WIP: Fixing Bug where an error was thrown instead of trying…
jubeormk1 97b6f8e
[skip ci] WIP: Refactor sftphandler.rs and main.rs to accommodate the…
jubeormk1 a84c7d1
[skip ci] WIP: write request fails but will be parked for now
jubeormk1 ae5fdc8
[skip ci] WIP: Some issues in DemoSftpServer DirEntriesCollection but…
jubeormk1 8786e12
[skip ci] WIP: FIX read operation
jubeormk1 acee527
[skip ci] WIP: Hack readdir response was not adding header field length
jubeormk1 3ba5410
[skip ci] WIP: BUG in sftphandler process keeps responding to the sam…
jubeormk1 468d117
[skip ci] WIP: BUG The EOF needs to be sent once the readdir has sent…
jubeormk1 90c76ce
[skip ci] WIP: Added a simplistic last read status approach
jubeormk1 4e93507
[skip ci] WIP: Missing StatusCode field in Status derived from SSHEnc…
jubeormk1 1d186ae
[skip ci] Fixed Status Encoding. Shall I extend the sshwire-derive to…
jubeormk1 5a35c7e
[skip ci] WIP: First ls command successful
jubeormk1 3f4edd3
[skip ci] Implementing EOF responsibility in SftpServer and documenting
jubeormk1 a5abc17
[skip ci] Added SunsetSftp Feature: Std
jubeormk1 6b3275b
[skip ci] Extracted DirEntriesCollection and added to std helpers
jubeormk1 503dbaf
[skip ci] Refactor proto.rs Name structure and
jubeormk1 44c21cd
Basic Features Directory read is now complete
jubeormk1 91fc26a
[skip ci] Extending SftpSource and Tests
jubeormk1 9809433
[skip ci] SftpOutputPipe checks if already splits
jubeormk1 add8014
removed TODO, not a good idea
jubeormk1 7bc1a63
[skip ci] intercepting incomplete UnknownPacket in decode request to …
jubeormk1 c1d446a
Handling properly unknown packets
jubeormk1 101aa8f
One more time I missed the Cargo.lock changes
jubeormk1 8c7de5c
Adding test stats: For now I get an expected `Operation unsupported`
jubeormk1 8961f71
refactored std helper to expose get_file_attrs
jubeormk1 53f4bac
completed getting file stats
jubeormk1 c2a83c2
[skip ci] Fixed misrepresentation of SSH_FXP_OPEN packet
jubeormk1 bcf89e1
Fixing SftpTrait and implementations to use the proper Open mode (PFl…
jubeormk1 f45cebb
WIP: Added SFTP Get capability. Chasing unexpected locks
jubeormk1 60cf7e5
SFTP Read stuck: Failing to receive a a read request?
jubeormk1 87ae690
[skip ci] small changes to make visible SftpOutputPipe total bytes co…
jubeormk1 580b71d
[skip ci] Removing outdated check in SftpHandler.process
jubeormk1 057ecd5
[skip ci] Removed nested enum in Sftp process FSM
jubeormk1 c73ab4c
[skip ci]some tiding up
jubeormk1 fdc76ae
adding large file 2MB to test_get.sh
jubeormk1 5bad3fc
[skip ci] Bug? Large write buffers cause for Chan_out cause ssh.serve…
jubeormk1 c28b72a
[skip ci] Debugging runner.read_channel and .input
jubeormk1 a6db092
[skip ci] Removed debug points from runner
jubeormk1 13b1c4a
[skip ci] NoRoom Error mitigated. Found a bug handling RanOut in sftp…
jubeormk1 4669748
Merge branch 'dev/sftp-start' of https://github.com/jubeormk1/sunset …
jubeormk1 1d1b486
[skip ci] RequestHolder function visibility reduced and capacity() added
jubeormk1 10b006b
[skip ci] Removing outdated logs
jubeormk1 6792cec
[skip ci] Adding test_get_single.sh and simplifying test_get.sh
jubeormk1 58ddc56
[skip ci] Refactor packet handling: Packets will have fixed length pa…
jubeormk1 8ae36fc
[skip ci] Adding more unit tests. Noted that it would be nice improvi…
jubeormk1 eeea557
[skip ci] Clearing RequestHolder on reset() and
jubeormk1 cae34f2
[skip ci] caught a condition where an unknown packet would not be flush
jubeormk1 bf4e87e
[skip ci] Checking the errors getting files
jubeormk1 488d3d7
[skip ci] Fixed test to accommodate that sinks don't need to be final…
jubeormk1 5d1c1f9
[skip ci] Adding checks around sftpserver read and readdir
jubeormk1 17991f0
[skip ci] some changes to the testing sh. -o LogLevel=[Error->Debug]
jubeormk1 bd0d53c
[skip ci] extra verbosity strace runs in sftp get tests
jubeormk1 a246e2b
[skip ci] Starting logging stuck communication
jubeormk1 6f6d102
[skip ci] Minor change regarding buffer names
jubeormk1 947cd6e
Merge branch 'matt/sftptesting' into dev/sftp-start
jubeormk1 229c1b8
[skip ci] After finally applying the suggested fix responses stall in…
jubeormk1 7e35720
[skip ci] Recording better logs
jubeormk1 fdfda8a
[skip ci] Exploring logs: Found important issue in OutputChannelHandler
jubeormk1 cb8fbda
[skip ci] Fixing test for Data roundtrip and updating Cargo.lock
jubeormk1 e1148dd
[skip ci] Removed env-logger dependency for sftp
jubeormk1 1760b63
[skip ci] Extracting sftpoutputproducer bytes for analysis
jubeormk1 3d5e1bb
[skip ci] Replacing write_all with write loop with no change
jubeormk1 937fcd3
[skip ci] log_demo_sftp_with_tests: Adding automatic TX/RX data extra…
jubeormk1 d5ba867
Config num_enum for no_std
jubeormk1 ce096d0
Fixing format and orphan use
jubeormk1 9a68d08
Fixing docs format and docs for macros
jubeormk1 3b3b38b
Adding sftp to the CI fixing and passing it
jubeormk1 9d76de8
Changing all SftpServer signatures to async (or impl Future with asyn…
jubeormk1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't think it's related to the current problems, but catching an error after
progress()isn't generally OK. Some operations modify internal state so it should just exit if it fails here.I think the specific changes in 2869501 should be OK though for the window adjust case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My apologies, I did not realise that your fix was not applied