Different ways to achieve the same thing and documentation is not too clear

[tomdracz]

I'm testing out async-http gem where I'm trying to basically run a method that dispatches multiple HTTP requests concurrently and at the end returns parsed responses.

All good here but I'm confused about different ways in which this can be achieved and the documentation is bit sparse. It says for example To issue multiple requests concurrently, you should use a barrier, e.g. but doesn't elaborate why that is the case.

Quick script showcasing four different ways I can achieve what I want:

require 'async'
require 'async/barrier'
require 'async/http/internet'

URLS = [
  'http://127.0.0.1:8082/delay/1',
  'http://127.0.0.1:8082/delay/2',
  'http://127.0.0.1:8082/delay/3',
  'http://127.0.0.1:8082/delay/4',
  'http://127.0.0.1:8082/delay/5',
  'http://127.0.0.1:8082/delay/6',
]

def through_async_one
  Async(annotation: "Async One Fetcher") do
    internet = Async::HTTP::Internet.new
    barrier = Async::Barrier.new

    results = URLS.map do |url|
      barrier.async do
        internet.get(url).read
      end
    end
    barrier.wait
    results.map(&:wait)
  ensure
    internet&.close
  end.wait
end

def through_async_two
  Async(annotation: "Async Two Fetcher") do |task|
    internet = Async::HTTP::Internet.new
    barrier = Async::Barrier.new
    
    results = URLS.map do |url|
      barrier.async(annotation: "Fetching #{url}") do |task|
        internet.get(url).read
      end
    end
    
    results.map(&:wait)
  ensure
    internet&.close
  end.wait
end

def through_async_three
  results = []
  Async(annotation: "Async Three Fetcher") do |task|
    # Async::Debug.serve
    internet = Async::HTTP::Internet.new
    barrier = Async::Barrier.new
    
    URLS.each do |url|
      barrier.async(annotation: "Fetching #{url}") do |task|
        results << internet.get(url).read
      end
    end
   barrier.wait
  ensure
    internet&.close
  end
  results
end

def through_async_four
  Async(annotation: "Async Four Fetcher") do |task|
    internet = Async::HTTP::Internet.new

    results = URLS.map do |url|
      task.async do
        internet.get(url).read
      end
    end
    results.map(&:wait)
  ensure
    internet&.close
  end.wait
end

through_async_one and through_async_two are pretty much the same apart from the fact that the former is not using barrier.wait call. I'm not sure why it would be needed. In either case I seem to have to map(&:wait) my results and .wait on the Async block to give me array of results.

through_async_three just creates an empty array before the Async block and appends to it in each of the tasks and then return results array at the end outside of the Async block - is this a bad practice?

Finally through_async_four doesn't use the barrier altogether. It still needs to map(&:wait) my results and .wait like the first two but the end result is seemingly the same.

Is there a specific difference between all those approaches? Is there a "better" way to do this?

[tomdracz]

Or maybe in other words - what woukd be recommended way to write a method that:

• takes x number of requests to do
• uses semaphore to limit the number of concurrent requests
• sets a timeout for each request to bail if it takes too long
• returns the request responses (not just the bodies, but full object to indicate status, any errors etc)