Error when serializing RemoteChannel

[iamed2]

julia> serialize(IOBuffer(), RemoteChannel())
ERROR: ArgumentError: elements of IntSet must be between 1 and typemax(Int)
Stacktrace:
 [1] _throw_intset_bounds_err() at ./intset.jl:64
 [2] push! at ./intset.jl:68 [inlined]
 [3] (::Base.Distributed.##133#134{Base.Distributed.RRID,Int64})() at ./distributed/remotecall.jl:249
 [4] lock(::Base.Distributed.##133#134{Base.Distributed.RRID,Int64}, ::Base.Threads.RecursiveTatasLock) at ./lock.jl:101
 [5] add_client at ./distributed/remotecall.jl:247 [inlined]
 [6] send_add_client(::RemoteChannel{Channel{Any}}, ::Int64) at ./distributed/remotecall.jl:262
 [7] serialize(::SerializationState{Base.AbstractIOBuffer{Array{UInt8,1}}}, ::RemoteChannel{Channel{Any}}, ::Bool) at ./distributed/remotecall.jl:281
 [8] serialize(::Base.AbstractIOBuffer{Array{UInt8,1}}, ::RemoteChannel{Channel{Any}}) at ./serialize.jl:629

(This does also occur when there is content in the RemoteChannel)

[ViralBShah]

cc @amitmurthy

[amitmurthy]

A remote reference refers to external in-memory data in the cluster. Serializing/deserializing references to remote nodes increments/decrements reference counts used for distributed gc.

Remote references are only serializable in the context of an active cluster and to IO objects connected to a remote endpoint.

We should throw a better error and document this fact.

[amitmurthy]

The other option is to serialize remote references with zeroed out values when serializing to a non-cluster IO object. Any remote calls using the deserialized zeroed out remote reference will fail.

This would be similar to how pointers are serialized/deserialized.

[iamed2]

In our use case we are actually using this in a cluster context but we want to be able to call serialize directly. Is there a way to do that?

The zeroing out like pointers is probably a good strategy in the general case.

[amitmurthy]

Are you serializing first to an IOBuffer and then sending the buffer over? That will not work as serializing a remote reference involves identifying the remote worker from the IO stream connected to the worker and appropriately sending reference counting messages -

julia/base/distributed/remotecall.jl

Line 280 in 70be8ab

p = worker_id_from_socket(s.io)

Some more information about your exact use case and motivation may help in identifying a solution.

[iamed2]

We can move discussion on our use case here: https://discourse.julialang.org/t/serializing-remotechannels/4249