Directory listing encoding problem

[wakproductions]

I'm trying to get a directory listing from an SFTP server and I'm getting this output:

pry(#<Net::SFTP::Session>)> request.response[:names]
=> [#<Net::SFTP::Protocol::V04::Name:0x00007fae6c141a00
  @attributes=
   #<Net::SFTP::Protocol::V04::Attributes:0x00007fae6c141a50
    @attributes={:type=>2, :size=>0, :owner=>"root", :group=>"root", :permissions=>16895, :atime=>1534766503, :createtime=>1534766503, :mtime=>1534766503}>,
  @name=".">,
 #<Net::SFTP::Protocol::V04::Name:0x00007fae6c140fb0 @attributes=#<Net::SFTP::Protocol::V04::Attributes:0x00007fae6c1410c8 @attributes={:type=>2}>, @name="\x00\x00\x00\x02">,
 #<Net::SFTP::Protocol::V04::Name:0x00007fae6c140560
  @attributes=#<Net::SFTP::Protocol::V04::Attributes:0x00007fae6c1406a0 @attributes={:type=>1}>,
  @name=
   "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04root\x00\x00\x00\x04root\x00\x00A\xFF\x00\x00\x00\x00[ \xE6\x89\x00\x00\x00\x00[\"\\F\x00\x00\x00\x00[ \xE6\x89\x00\x00\x00\x04\x00\x00\x00!2018-08-17-15-32-TAFUS-GIDREQ.xml\x00\x00\x02\xBD\x01\x00\x00\x00\x00\x00\x1D\xCF\x15\x00\x00\x00\x04root\x00\x00\x00\x04root\x00\x00\x81\xFF\x00\x00\x00\x00[z\xAD\xF5\x00\x00\x00\x00[z\xAD\xF5\x00\x00\x00\x00[z\xAD\xF5\x00\x00\x00\x00\x00\x00\x00!2018-08-21-17-30-TAFUS-GIDREQ.xml\x00\x00\x02\xBD\x01\x00\x00\x00\x00\x00\x05\xB4p\x00\x00\x00\x04root\x00\x00\x00\x04root\x00\x00\x81\xFF\x00\x00\x00\x00[|L\xA5\x00\x00\x00\x00[|L\xA5\x00\x00\x00\x00[|L\xA5\x00\x00\x00\x00">,
 #<Net::SFTP::Protocol::V04::Name:0x00007fae6c140150 @attributes=#<Net::SFTP::Protocol::V04::Attributes:0x00007fae6c1401a0 @attributes={:type=>1}>, @name=nil>]

The filenames are scrambled with \x00 type of characters. It looks like an encoding issue, but I'm having trouble figuring out how to do a proper conversion. Can anyone help lead me into the right direction to working around this?

[wakproductions]

When I look into the data returned by net-ssh here's what I find:

    28: def parse_name_packet(packet)
    29:   names = []
    30: 
 => 31:   binding.pry
    32:   packet.read_long.times do
    33:     filename = packet.read_string
    34:     attrs    = attribute_factory.from_buffer(packet)
    35:     names   << name_factory.new(filename, attrs)
    36:   end
    37: 
    38:   { :names => names }
    39: end

[1] pry(#<Net::SFTP::Protocol::V05::Base>)> packet
=> #<Net::SFTP::Packet:0x00007ff1864251b0
 @content=
  "h\x00\x00\x00\x01\x00\x00\x00\x04\x00\x00\x00\x01.\x00\x00\x02\xBD\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04root\x00\x00\x00\x04root\x00\x00A\xFF\x00\x00\x00\x00[z\xAD\xA7\x00\x00\x00\x00[z\xAD\xA7\x00\x00\x00\x00[z\xAD\xA7\x00\x00\x00\x04\x00\x00\x00\x02..\x00\x00\x02\xBD\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04root\x00\x00\x00\x04root\x00\x00A\xFF\x00\x00\x00\x00[ \xE6\x89\x00\x00\x00\x00[\"\\F\x00\x00\x00\x00[ \xE6\x89\x00\x00\x00\x04\x00\x00\x00!2018-08-17-15-32-TAFUS-GIDREQ.xml\x00\x00\x02\xBD\x01\x00\x00\x00\x00\x00\x1D\xCF\x15\x00\x00\x00\x04root\x00\x00\x00\x04root\x00\x00\x81\xFF\x00\x00\x00\x00[z\xAD\xF5\x00\x00\x00\x00[z\xAD\xF5\x00\x00\x00\x00[z\xAD\xF5\x00\x00\x00\x00\x00\x00\x00!2018-08-21-17-30-TAFUS-GIDREQ.xml\x00\x00\x02\xBD\x01\x00\x00\x00\x00\x00\x05\xB4p\x00\x00\x00\x04root\x00\x00\x00\x04root\x00\x00\x81\xFF\x00\x00\x00\x00[|L\xA5\x00\x00\x00\x00[|L\xA5\x00\x00\x00\x00[|L\xA5\x00\x00\x00\x00",
 @position=5,
 @type=104>

Could this be some kind of protocol mismatch? Can't tell if this is a problem of net-sftp or net-ssh

[wakproductions]

I've determined that the server is using SFTP protocol version 5, but this gem does not support it properly. It is unable to read some of the file attributes. I was able to get a temporary fix by putting this in my program:

Net::SFTP::Session::HIGHEST_PROTOCOL_VERSION_SUPPORTED = 3

[mfazekas]

@wakproductions thanks for the report, unfortunately i don not have time to fix this, but PR's are more than welcome

[Laykou]

@wakproductions You saved my day! I run into the same issue in the production and was very frustrated what is going on. :)

Thanks!

[Laykou]

I found a workaround by changing the protocol directly and not the constant. This approach works only in a block though

Net::SFTP.start(host, user, sftp_options) do |sftp|
    sftp.instance_variable_set('@protocol', ::Net::SFTP::Protocol.load(sftp, 3))

    puts sftp.dir.entries('/some/folder').map(&:name)

    # The connection is closed after the block
end

[callaars]

You could also do a workaround by simply overriding the Net::SFTP::Session class like this

class Net::SFTP::Session
  HIGHEST_PROTOCOL_VERSION_SUPPORTED = 3
end

That way you don't have to override a constant and you don't need to use it in a block only.

[thbar]

I compared today version v3.0.0.beta1 and v2.1.2 here:

v2.1.2...net-ssh:v3.0.0.beta1#diff-d9042ac2c306d5f092d3ea67d4489942R78

This can now be set via Net::SFTP::Session.new(ssh, version).

This is, though, a lower-level version of the code, so this possibility is not bubbled up to what you'd usually write:

Net::SFTP.start(sftp_host, user, password: password) do |sftp|
  ...
end

Mentioning it, though, in case it helps someone.

[thbar]

FWIW, on JRuby 9.2.5.0, the trick by @Laykou (#78 (comment)) didn't work for some reason.

[thbar]

Net::SFTP v3.0.0 (published a few days ago) now allows (via #107) to do specify the protocol directly via Net::SFTP.start:

Net::SFTP.start(host, user, ssh_options, {version: 3}) do
  # SNIP
end

@wakproductions could you please try out?