textwrap.dedent() drops tabs - bug or feature?

[Steven Bethard]

So I've recently been making pretty frequent use of textwrap.dedent () to
allow me to use triple-quoted strings at indented levels of code without
getting the extra spaces prefixed to each line. I discovered today that
not only does textwrap.dedent () strip any leading spaces, but it also
substitutes any internal tabs with spaces. For example::

py> def test():
.... x = ('abcd efgh\n'
.... 'ijkl mnop\n')
.... y = textwrap.dedent ('''\
.... abcd efgh
.... ijkl mnop
.... ''')
.... return x, y
....
py> test()
('abcd\tefgh\ni jkl\tmnop\n', 'abcd efgh\nijkl mnop\n')

Note that even though the tabs are internal, they are still removed by
textwrap.dedent (). The documentation[1] says:

"""
dedent(text)
Remove any whitespace that can be uniformly removed from the left
of every line in text.

This is typically used to make triple-quoted strings line up with
the left edge of screen/whatever, while still presenting it in the
source code in indented form.
"""

So it looks to me like even if this is a "feature" it is undocumented.
I'm planning on filing a bug report, but I wanted to check here first in
case I'm just smoking something.

STeVe

[1] http://docs.python.org/lib/module-textwrap.html

[Peter Hansen]

Re: textwrap.dedent () drops tabs - bug or feature?

Steven Bethard wrote:[color=blue]
> Note that even though the tabs are internal, they are still removed by
> textwrap.dedent (). The documentation[1] says:[/color]
....[color=blue]
> So it looks to me like even if this is a "feature" it is undocumented.
> I'm planning on filing a bug report, but I wanted to check here first in
> case I'm just smoking something.[/color]

While I wouldn't say it's obvious, I believe it is (indirectly?)
documented and deliberate.

Search for this in the docs:
"""
expand_tabs
(default: True) If true, then all tab characters in text will be
expanded to spaces using the expandtabs() method of text.
"""

(This is not to say a specific rewording of the docs to lessen any
confusion in this area wouldn't be welcomed.)

-Peter

[Peter Hansen]

Re: textwrap.dedent () drops tabs - bug or feature?

Steven Bethard wrote:[color=blue]
> Note that even though the tabs are internal, they are still removed by
> textwrap.dedent (). The documentation[1] says:[/color]
....[color=blue]
> So it looks to me like even if this is a "feature" it is undocumented.
> I'm planning on filing a bug report, but I wanted to check here first in
> case I'm just smoking something.[/color]

While I wouldn't say it's obvious, I believe it is (indirectly?)
documented and deliberate.

Search for this in the docs:
"""
expand_tabs
(default: True) If true, then all tab characters in text will be
expanded to spaces using the expandtabs() method of text.
"""

(This is not to say a specific rewording of the docs to lessen any
confusion in this area wouldn't be welcomed.)

-Peter

[Steven Bethard]

Re: textwrap.dedent () drops tabs - bug or feature?

Peter Hansen wrote:[color=blue]
> Steven Bethard wrote:
>[color=green]
>> Note that even though the tabs are internal, they are still removed by
>> textwrap.dedent (). The documentation[1] says:[/color]
>
> ...
>[color=green]
>> So it looks to me like even if this is a "feature" it is undocumented.
>> I'm planning on filing a bug report, but I wanted to check here first
>> in case I'm just smoking something.[/color]
>
> While I wouldn't say it's obvious, I believe it is (indirectly?)
> documented and deliberate.
>
> Search for this in the docs:
> """
> expand_tabs
> (default: True) If true, then all tab characters in text will be
> expanded to spaces using the expandtabs() method of text.
> """[/color]

Thanks for double-checking this for me. I looked at expand_tabs, and
it's part of the definition of the TextWrapper class, which is not
actually used by textwrap.dedent (). So I think the textwrap.dedent ()
expanding-of-tabs behavior is still basically undocumented.

I looked at the source code, and the culprit is the first line of the
function definition:

lines = text.expandtabs ().split('\n')

I filed a bug_ report, but left the Category unassigned so that someone
else can decide whether it's a doc bug or a code bug.

STeVe

... _bug: http://python.org/sf/1361643

[Steven Bethard]

Re: textwrap.dedent () drops tabs - bug or feature?

Peter Hansen wrote:[color=blue]
> Steven Bethard wrote:
>[color=green]
>> Note that even though the tabs are internal, they are still removed by
>> textwrap.dedent (). The documentation[1] says:[/color]
>
> ...
>[color=green]
>> So it looks to me like even if this is a "feature" it is undocumented.
>> I'm planning on filing a bug report, but I wanted to check here first
>> in case I'm just smoking something.[/color]
>
> While I wouldn't say it's obvious, I believe it is (indirectly?)
> documented and deliberate.
>
> Search for this in the docs:
> """
> expand_tabs
> (default: True) If true, then all tab characters in text will be
> expanded to spaces using the expandtabs() method of text.
> """[/color]

Thanks for double-checking this for me. I looked at expand_tabs, and
it's part of the definition of the TextWrapper class, which is not
actually used by textwrap.dedent (). So I think the textwrap.dedent ()
expanding-of-tabs behavior is still basically undocumented.

I looked at the source code, and the culprit is the first line of the
function definition:

lines = text.expandtabs ().split('\n')

I filed a bug_ report, but left the Category unassigned so that someone
else can decide whether it's a doc bug or a code bug.

STeVe

... _bug: http://python.org/sf/1361643

[Peter Hansen]

Re: textwrap.dedent () drops tabs - bug or feature?

Steven Bethard wrote:[color=blue]
> Thanks for double-checking this for me. I looked at expand_tabs, and
> it's part of the definition of the TextWrapper class, which is not
> actually used by textwrap.dedent (). So I think the textwrap.dedent ()
> expanding-of-tabs behavior is still basically undocumented.[/color]

Ah, good point. I saw dedent() in there with the wrap() and fill()
convenience functions which use a TextWrapper internally, but you're
quite right that dedent() is different, and in fact merely uses the
expandtabs() functionality of the standard string class, so this has
nothing to do with the expand_tabs attribute I pointed out.

-Peter

[Peter Hansen]

Re: textwrap.dedent () drops tabs - bug or feature?

Steven Bethard wrote:[color=blue]
> Thanks for double-checking this for me. I looked at expand_tabs, and
> it's part of the definition of the TextWrapper class, which is not
> actually used by textwrap.dedent (). So I think the textwrap.dedent ()
> expanding-of-tabs behavior is still basically undocumented.[/color]

Ah, good point. I saw dedent() in there with the wrap() and fill()
convenience functions which use a TextWrapper internally, but you're
quite right that dedent() is different, and in fact merely uses the
expandtabs() functionality of the standard string class, so this has
nothing to do with the expand_tabs attribute I pointed out.

-Peter