Iterators in combination with closures misbehave

[def-]

This is extracted from https://github.com/def-/nim-iterutils . Since it compiles for the first time since Nimrod 0.9.4, I'm not sure what exactly broke it. I believe this to be related to the closure changes and tried the suggested changes, but couldn't fix it:

from sequtils import toSeq

type Iterable*[T] = (iterator: T) | Slice[T]
  ## Everything that can be iterated over, iterators and slices so far.

proc toIter*[T](s: Slice[T]): iterator: T =
  ## Iterate over a slice.
  iterator it: T {.closure.} =
    for x in s.a..s.b:
      yield x
  return it

proc toIter*[T](i: iterator: T): iterator: T =
  ## Nop
  i

iterator map*[T,S](i: Iterable[T], f: proc(x: T): S): S =
  let i = toIter(i)
  for x in i():
    yield f(x)

proc filter*[T](i: Iterable[T], f: proc(x: T): bool): iterator: T =
  ## Iterates through an iterator and yields every item that fulfills the
  ## predicate `f`.
  ##
  ## .. code-block:: nim
  ##   for x in filter(1..11, proc(x): bool = x mod 2 == 0):
  ##     echo x
  let i = toIter(i)
  iterator it: T {.closure.} =
    for x in i():
      if f(x):
        yield x
  result = it

iterator filter*[T](i: Iterable[T], f: proc(x: T): bool): T =
  let i = toIter(i)
  for x in i():
    if f(x):
      yield x

var it = toSeq(filter(2..10, proc(x: int): bool = x mod 2 == 0))
echo it # @[2, 4, 6, 8, 10]
it = toSeq(map(filter(2..10, proc(x: int): bool = x mod 2 == 0), proc(x: int): int = x * 2))
echo it # Expected output: @[4, 8, 12, 16, 20], Actual output: @[]

[Araq]

proc toIter*[T](i: iterator: T): iterator: T =
  ## Nop
  i

This is not a nop. No idea if that's the problem though.

[shaunc]

I think this might be a minimal example of this problem:

iterator t1(): int {.closure.} =
  yield 1

iterator t2(): int {.closure.} =
  for i in t1():
    yield i

for i in t2():
  echo $i

This fails with

1
Traceback (most recent call last)
t2.nim(11)               t2
SIGSEGV: Illegal storage access. (Attempt to read from nil?)

However, if we take out the first {.closure.} it works.

[andreaferretti]

@shaunc The minimal example now works correctly. Not so for the original one, though

[nitely]

I think I've a shorter example:

proc iter1(): (iterator: int) =
  let coll = [0,1,2]
  result = iterator: int {.closure.} =
    for i in coll:
      yield i

proc iter2(it: (iterator: int)): (iterator: int)  =
  result = iterator: int {.closure.} =
    echo finished(it)
    for i in it():
      yield i

echo "start"
let myiter1 = iter1()
let myiter2 = iter2(myiter1)
for i in myiter2():
  echo i
echo "end"
# start
# false
# end

But it works when changing the iter2 (as shown in the manual) to:

proc iter2(it: (iterator: int)): (iterator: int)  =
  result = iterator: int {.closure.} =
    while true:
      let i = it()
      if finished(it): break
      yield i

Doing this instead of for in in @def- snippet also works