period_range fails on python 2.7 (works on python 3.2) - 'str' object has no attribute 'read'

[gorkypl]

I am doing an exemple from http://pandas.pydata.org/pandas-docs/stable/timeseries.html

rng = pandas.period_range('1/1/2011', periods=500, freq='M')

---

DateParseError Traceback (most recent call last)
/home/users/pawel/AGH/mgr/dane/gazowe/ in ()
----> 1 rng = pandas.period_range('1/1/2011', periods=500, freq='M')

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in period_range(start, end, periods, freq)
1064 """
1065 return PeriodIndex(start=start, end=end, periods=periods,
-> 1066 freq=freq)
1067
1068 def _period_rule_to_timestamp_rule(freq, how='end'):

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in new(cls, data, ordinal, freq, start, end, periods, copy, name, year, month, quarter, day, hour, minute, second)
533 fields = [year, month, quarter, day, hour, minute, second]
534 data, freq = cls._generate_range(start, end, periods,
--> 535 freq, fields)
536 else:
537 ordinal, freq = cls._from_arraylike(data, freq)

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in _generate_range(cls, start, end, periods, freq, fields)
551 raise ValueError('Can either instantiate from fields '
552 'or endpoints, but not both')
--> 553 subarr, freq = _get_ordinal_range(start, end, periods, freq)
554 elif field_count > 0:
555 y, mth, q, d, h, minute, s = fields

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in _get_ordinal_range(start, end, periods, freq)
925
926 if start is not None:
--> 927 start = Period(start, freq)
928 if end is not None:
929 end = Period(end, freq)

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in init(self, value, freq, ordinal, year, month, quarter, day, hour, minute, second)
98 value = str(value)
99
--> 100 dt, freq = _get_date_and_freq(value, freq)
101
102 elif isinstance(value, datetime):

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in _get_date_and_freq(value, freq)
382 def _get_date_and_freq(value, freq):
383 value = value.upper()
--> 384 dt, _, reso = parse_time_string(value, freq)
385
386 if freq is None:

/usr/lib64/python2.7/site-packages/pandas/tseries/tools.pyc in parse_time_string(arg, freq)
217 return ret, parsed, reso # datetime, resolution
218 except Exception, e:
--> 219 raise DateParseError(e)
220
221 def _try_parse_monthly(arg):

DateParseError: 'str' object has no attribute 'read'

Works fine with python-3.2.
python-2.7.3, numpy-1.6.2, scipy-0.10.1, pandas 0.8.0

[changhiskhan]

Seems to work on my end:

In [1]: import pandas

In [2]: rng = pandas.period_range('1/1/2011', periods=500, freq='M')

In [3]: rng
Out[3]:
<class 'pandas.tseries.period.PeriodIndex'>
freq: M
[Jan-2011, ..., Aug-2052]
length: 500

I'm on OSX, Python 2.7.3, numpy 1.6.1, pandas master (0.8.1-dev), dateutil v1.5

[gorkypl]

Hmm, any hints where else can I look for the possible reasons?

[changhiskhan]

Are you familiar with pdb? If so maybe you can look at the original error (not the re-raised DateParseError) and post the stack trace. Alternatively, you can also just remove the try/except statement in pandas.tseries.tools.parse_time_string if you installed via the develop command.

That catch-all try/except really shouldn't be there. I'll keep this issue open to remind us to have better error handling for that function.

Please let me know what you find

[gorkypl]

I removed the try/except block and got:

In [1]: import pandas
In [2]: rng = pandas.period_range('1/1/2011', periods=500, freq='M')

---

AttributeError Traceback (most recent call last)
/home/users/pawel/ in ()
----> 1 rng = pandas.period_range('1/1/2011', periods=500, freq='M')

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in period_range(start, end, periods, freq)
1064 """
1065 return PeriodIndex(start=start, end=end, periods=periods,
-> 1066 freq=freq)
1067
1068 def _period_rule_to_timestamp_rule(freq, how='end'):

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in new(cls, data, ordinal, freq, start, end, periods, copy, name, year, month, quarter, day, hour, minute, second)
533 fields = [year, month, quarter, day, hour, minute, second]
534 data, freq = cls._generate_range(start, end, periods,
--> 535 freq, fields)
536 else:
537 ordinal, freq = cls._from_arraylike(data, freq)

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in _generate_range(cls, start, end, periods, freq, fields)
551 raise ValueError('Can either instantiate from fields '
552 'or endpoints, but not both')
--> 553 subarr, freq = _get_ordinal_range(start, end, periods, freq)
554 elif field_count > 0:
555 y, mth, q, d, h, minute, s = fields

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in _get_ordinal_range(start, end, periods, freq)
925
926 if start is not None:
--> 927 start = Period(start, freq)
928 if end is not None:
929 end = Period(end, freq)

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in init(self, value, freq, ordinal, year, month, quarter, day, hour, minute, second)
98 value = str(value)
99
--> 100 dt, freq = _get_date_and_freq(value, freq)
101
102 elif isinstance(value, datetime):

/usr/lib64/python2.7/site-packages/pandas/tseries/period.pyc in _get_date_and_freq(value, freq)
382 def _get_date_and_freq(value, freq):
383 value = value.upper()
--> 384 dt, _, reso = parse_time_string(value, freq)
385
386 if freq is None:

/usr/lib64/python2.7/site-packages/pandas/tseries/tools.py in parse_time_string(arg, freq)
192 yearfirst = print_config.date_yearfirst
193
--> 194 parsed = _dtparser._parse(arg, dayfirst=dayfirst, yearfirst=yearfirst)
195
196 repl = {}

/usr/lib64/python2.7/site-packages/dateutil/parser.pyc in _parse(self, timestr, dayfirst, yearfirst, fuzzy)
354 yearfirst = info.yearfirst
355 res = self._result()
--> 356 l = _timelex.split(timestr)
357 try:
358

/usr/lib64/python2.7/site-packages/dateutil/parser.pyc in split(cls, s)
148
149 def split(cls, s):
--> 150 return list(cls(s))
151 split = classmethod(split)
152

/usr/lib64/python2.7/site-packages/dateutil/parser.pyc in next(self)
145
146 def next(self):
--> 147 return self.next() # Python 2.x support
148
149 def split(cls, s):

/usr/lib64/python2.7/site-packages/dateutil/parser.pyc in next(self)
139
140 def next(self):
--> 141 token = self.get_token()
142 if token is None:
143 raise StopIteration

/usr/lib64/python2.7/site-packages/dateutil/parser.pyc in get_token(self)
70 nextchar = self.charstack.pop(0)
71 else:
---> 72 nextchar = self.instream.read(1)
73 while nextchar == '\x00':
74 nextchar = self.instream.read(1)

AttributeError: 'str' object has no attribute 'read'

[changhiskhan]

What version of dateutil are you running?

It looks like the Exception is saying the dateutil.parser._timelex.instream was a 'str' object. Do you have the source code for your dateutil? Can you check the constructor of dateutil.parser._timelex to see what's happening to the input argument? It's supposed to be converting it into a StringIO object and then calling StringIO.read on it.

[gorkypl]

Indeed, I was using dateutil-2.1. After downgrading to 1.5 everything works fine.

Sorry for misleading you, but I greatly appreciate your help. And maybe some other Gentoo user will benefit from this - I think I'd ask for restricting dateutil version in pandas dependencies.

Not closing because you said you'd use this to remember about the try/except issue, but feel free to do whatever you want. And thank you again :)

[changhiskhan]

No problem, happy to help.

The interesting thing here is that the pandas setup script actually requires dateutil < 2 for python 2.x. Is your default Python 3x? If so then that's probably why you didn't get an exception during the install.

Anyways, good to know your issues are resolved!

[wesm]

I added a check for dateutil 2.1-- dateutil 2.0 would have given you an import error but I guess a new version was released. I don't know how 2.1 got installed; I don't think pandas's setup.py would have done it? but maybe I'm wrong

[gorkypl]

I was installing pandas using a Gentoo ebuild, which pulled the newest dateutil available.
The dateutil 2.1 is indeed available only at http://pypi.python.org/pypi/python-dateutil/ and is not mentioned on the package homepage.

[gorkypl]

I haven't got a clue about pandas internals, but Gentoo developers say that the correct solution would be to make pandas work with dateutil-2.1 both on python 2.x and 3.x
Don't blame me if it is not a trivial task, I am just a messenger :)
https://bugs.gentoo.org/show_bug.cgi?id=424968

[quantumsummers]

This should really be reopened.

[wesm]

Fixed in 31ee064