recursive symlink significantly slows collection

[pytestbot]

Originally reported by: Brian Kearns (BitBucket: bdkearns, GitHub: bdkearns)

---

placing "ln -s . link" in a collected dir significantly slows down collection. it seems collection doesn't keep track of directories visited to prevent unnecessary recursion?

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/624

[pytestbot]

Original comment by Brian Kearns (BitBucket: bdkearns, GitHub: bdkearns):

---

platform linux2 -- Python 2.7.6 -- py-1.4.26 -- pytest-2.6.4
platform linux -- Python 3.4.0 -- py-1.4.26 -- pytest-2.6.4

[asottile]

I actually found a much more amusing result when attempting to reproduce this:

mkdir t
echo 'def test_one(): pass' > t/test_foo.py
cd t && ln -s . l

and then

$ ./venv/bin/pytest t
========================================= test session starts =========================================
platform linux -- Python 3.6.5, pytest-3.6.4.dev9+g42bbb4fa.d20180707, py-1.5.4, pluggy-0.6.0
rootdir: /tmp/pytest, inifile: tox.ini
collected 41 items                                                                                    

t/test_foo.py .                                                                                 [  2%]
t/l/test_foo.py .                                                                               [  4%]
t/l/l/test_foo.py .                                                                             [  7%]
t/l/l/l/test_foo.py .                                                                           [  9%]
t/l/l/l/l/test_foo.py .                                                                         [ 12%]
t/l/l/l/l/l/test_foo.py .                                                                       [ 14%]
t/l/l/l/l/l/l/test_foo.py .                                                                     [ 17%]
t/l/l/l/l/l/l/l/test_foo.py .                                                                   [ 19%]
t/l/l/l/l/l/l/l/l/test_foo.py .                                                                 [ 21%]
t/l/l/l/l/l/l/l/l/l/test_foo.py .                                                               [ 24%]
t/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                                             [ 26%]
t/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                                           [ 29%]
t/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                                         [ 31%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                                       [ 34%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                                     [ 36%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                                   [ 39%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                                 [ 41%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                               [ 43%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                             [ 46%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                           [ 48%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                         [ 51%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                       [ 53%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                     [ 56%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                   [ 58%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                                 [ 60%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                               [ 63%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                             [ 65%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                           [ 68%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                         [ 70%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                       [ 73%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                     [ 75%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                   [ 78%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .                 [ 80%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .               [ 82%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .             [ 85%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .           [ 87%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .         [ 90%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .       [ 92%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .     [ 95%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py .   [ 97%]
t/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/l/test_foo.py . [100%]

====================================== 41 passed in 0.22 seconds ======================================

I'm amused it stops after recusing 40 levels without erroring -- I suspect it hits ELOOP (Too many levels of symbolic links) and then stops recursing

EDIT: for best effect, I suggest making the terminal wider so you get a nice pyramid

[asottile]

oh right, and of course this becomes a fun exponential problem if there's more than one symlink

[asottile]

apparently attempting to discover 2^41 - 2 tests makes my computer sad :)

[codingkrabbe]

had a similar issue, a recursive symlink was followed until it crashed...
Although strange that you have a different behaviour.

[mdmintz]

Although the pyramid that @asottile showed still happens in pytest 7.0.0, the improvements to test-collection speed make it such that tests are collected significantly faster. Not so much a performance issue anymore as a duplicated-tests issue.

[RonnyPfannschmidt]

closing this one as solved then