Allowing zero-dimensional subscripts

[Carl Banks]

Re: Allowing zero-dimensional subscripts


Steve Holden wrote:[color=blue]
> Carl Banks wrote:[color=green]
> > Steve Holden wrote:
> >[color=darkred]
> >>Hey, I have an idea, why don't we look at the language reference manual
> >>instead of imagining how we think it might work![/color]
> >
> >
> > I don't know. Sounds risky.
> >
> >
> >[color=darkred]
> >>In section 3.2 we find:
> >>
> >>
> >>"""
> >>Tuples
> >>The items of a tuple are arbitrary Python objects. Tuples of two or more
> >>items are formed by comma-separated lists of expressions. A tuple of one
> >>item (a `singleton') can be formed by affixing a comma to an expression
> >>(an expression by itself does not create a tuple, since parentheses must
> >>be usable for grouping of expressions). An empty tuple can be formed by
> >>an empty pair of parentheses.
> >>"""
> >>
> >>So it seems that your speculation is false. Section 2.6 specifically
> >>defines "[" and "]" as delimiters. Section 5.3.2 defines a subscription
> >>(a term I've not really grown to love, but what the heck) as
> >>
> >>subscriptio n ::= primary "[" expression_list "]"
> >>
> >>and section 5.12, which defines expression_list , explicitly says
> >>
> >>"""An expression list containing at least one comma yields a tuple.""".
> >>
> >>So it would appear that while your change might be very convenient to
> >>allow you to refer to scalar values as zero-dimensional arrays, it
> >>doesn't really fit into Python's conceptual framework. Sorry.[/color]
> >
> >
> > Yes, that would appear to be so. You would have a point... if the
> > documentation were correct. Only it's not.
> >
> > According to the reference manual, the rule for an expression_list is:
> >
> > expression_list ::= expression ( "," expression )* [","]
> >
> > But take the following legal Python subscripted array:
> >
> > a[1:2,...,3:4]
> >[/color]
> But the element inside the brackets there isn't an expression-list,
> it's a slicing (see section 5.3.2).[/color]

Section 5.3.2 says an expression-list is what's inside the brackets
(you quoted this rule yourself). Section 5.12 says an expression-list
consissts of comma-separated expressions. But 1:2 and ... aren't
expressions. I only brought this up to point out that the docs were
not exactly correct, and in particular it swept an important
distinction (for this nit-picky discussion) under the rug.


[snip][color=blue][color=green]
> > trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
> > subscriptlist: subscript (',' subscript)* [',']
> > sliceop: ':' [test]
> > subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop]
> > testlist: test (',' test)* [',']
> >[/color]
> The simplification of the grammar is explicitly documented:[/color]
[snip]

Nice to know.

[color=blue][color=green]
> > Clearly, the grammar rule used for list subscript is different from the
> > one used for list of expressions (for some reason, what an ordinary
> > person would call an expression is called a "test" in the grammar,
> > whereas "expr" is a non-short-circuiting expression).
> >
> > So there's a regular way to create non-empty tuples, and a subscript
> > way.
> >
> > And there's a regular way to create an empty tuple... but not a
> > subscript way.
> >
> > So I'd say this change fits the conceptual framework of the tuple quite
> > well; in fact, it makes subscript tuples more parallel to their regular
> > counterparts.[/color]
>
> Although this debate is beginning to make me sound like one, I am really
> not a language lawyer. However, I should point out that what you are
> describing as a "tuple" should more correctly be described as a
> "slice-list" once you include slices or an ellipsis as elements.
> Slicings are described, as I am fairly sure you know, in section 5.3.3.[/color]

No, I don't think I am. I was careful to distinguish between grammar
rules and tuples. subscriptlist and testlist are grammar rules.
"Evaluation " of both of these guys can create a tuple. The two grammar
rules have exactly the same relationship to the tuple: an testlist with
a comma creates a tuple, a subscriptlist with a comma creates a tuple.

But note neither testlist nor subscriptlist can create an empty tuple.
You need a special case for that--only there's no special special case
for subscripts. (The visual effect, as OP noted, is that you need
parentheses for the empty tuple, but never need them for any other
subscript.)

[color=blue][color=green][color=darkred]
> >>One further point: if you really do conceptualize scalars as
> >>zero-dimensional arrays, where is the value conceptually stored?[/color]
> >
> > Think of it this way: an array with n-dimensions of length 3 would have
> > 3**n total entries. How many entries would a 0-dimensional array have?
> > 3**0 == 1.
> >
> > Numeric has had zero-dimensional arrays for a long time, and has had no
> > problem storing them. Think of the rule for accessing an element of an
> > array: it's a base pointer + sum (indices*stride ) for all indices. Now
> > generalize it down to zero: there are no indices, so the scalar is
> > stored at the base pointer.[/color]
>
> I can see that, and it doesn't seem unreasonable. Fortunately your
> persistence has goaded me into determining the point that *did* seem
> unreasonable to me: you were falsely trying to equate slicings and tuples.[/color]

The OP might have been; I wasn't. But that's fair enough; everyone
should be clear on what's happening behind the scenes.
[color=blue]
> Having said all of which, there probably *is* a case for proposing that
> an empty slicing become syntactically acceptable, so why not write the
> PEP and go for it?[/color]

Well, I'm only +0 on it, so I'll leave it to the OP. I was mainly
concerned with how the language reference was obscuring what I felt
were important distinctions.


Carl Banks

[Carl Banks]

Re: Allowing zero-dimensional subscripts

Fredrik Lundh wrote:[color=blue]
> noam's proposal is to make this work:
>[color=green][color=darkred]
> >>> x[][/color][/color]
> ()
>
> (but should it really result in an empty tuple? wouldn't None be a bit
> more Pythonic?)[/color]

How would you index a 2-D array? With a 2-tuple.
How would you index a 1-D array? With a 1-tuple.
How would you index a 0-D array? ...


Carl Banks

[George Sakkis]

Re: Allowing zero-dimensional subscripts

spam.noam@gmail .com wrote:
[color=blue]
> However, I'm designing another library for
> managing multi-dimensional arrays of data. Its purpose is similiar to
> that of a spreadsheet - analyze data and preserve the relations between
> a source of a calculation and its destination.[/color]

Sounds interesting. Will it be related at all to OLAP or the
Multi-Dimensional eXpressions language
(http://msdn2.microsoft.com/en-us/library/ms145506.aspx) ?

George

[Fredrik Lundh]

Re: Allowing zero-dimensional subscripts

Carl Banks wrote:
[color=blue]
> How would you index a 2-D array? With a 2-tuple.
> How would you index a 1-D array? With a 1-tuple.
> How would you index a 0-D array? ...[/color]

array dimensions don't exist at the Python level. you're confusing
behaviour that a custom class may provide with Python's view of things.

(and None is of course the standard value for "not there")

</F>

[greg]

Re: Allowing zero-dimensional subscripts

Carl Banks wrote:
[color=blue]
> Think of it this way: an array with n-dimensions of length 3 would have
> 3**n total entries. How many entries would a 0-dimensional array have?
> 3**0 == 1.[/color]

Er, hang on a minute. Along which dimension of this
0-dimensional array does it have a length of 3? :-)

--
Greg

[Tim N. van der Leeuw]

Re: Allowing zero-dimensional subscripts


greg wrote:[color=blue]
> Carl Banks wrote:
>[color=green]
> > Think of it this way: an array with n-dimensions of length 3 would have
> > 3**n total entries. How many entries would a 0-dimensional array have?
> > 3**0 == 1.[/color]
>
> Er, hang on a minute. Along which dimension of this
> 0-dimensional array does it have a length of 3? :-)
>
> --
> Greg[/color]

Against all zero of them... ;-)

Cheers,

--Tim

[spam.noam@gmail.com]

Re: Allowing zero-dimensional subscripts

George Sakkis wrote:[color=blue]
> spam.noam@gmail .com wrote:
>[color=green]
> > However, I'm designing another library for
> > managing multi-dimensional arrays of data. Its purpose is similiar to
> > that of a spreadsheet - analyze data and preserve the relations between
> > a source of a calculation and its destination.[/color]
>
> Sounds interesting. Will it be related at all to OLAP or the
> Multi-Dimensional eXpressions language
> (http://msdn2.microsoft.com/en-us/library/ms145506.aspx) ?
>[/color]
Thanks for the reference! I didn't know about any of these. It will
probably be interesting to learn from them. From a brief look at OLAP
in wikipedia, it may have similarities to OLAP. I don't think it will
be related to Microsoft's language, because the language will simply by
Python, hopefully making it very easy to do whatever you like with the
data.

I posted to python-dev a message that (hopefully) better explains my
use for x[]. Here it is - I think that it also gives an idea on how it
will look like.


I'm talking about something similar to a spreadsheet in that it saves
data, calculation results, and the way to produce the results.
However, it is not similar to a spreadsheet in that the data isn't
saved in an infinite two-dimensional array with numerical indices.
Instead, the data is saved in a few "tables", each storing a different
kind of data. The tables may be with any desired number of dimensions,
and are indexed by meaningful indices, instead of by natural numbers.

For example, you may have a table called sales_data. It will store the
sales data in years from set([2003, 2004, 2005]), for car models from
set(['Subaru', 'Toyota', 'Ford']), for cities from set(['Jerusalem',
'Tel Aviv', 'Haifa']). To refer to the sales of Ford in Haifa in 2004,
you will simply write: sales_data[2004, 'Ford', 'Haifa']. If the table
is a source of data (that is, not calculated), you will be able to set
values by writing: sales_data[2004, 'Ford', 'Haifa'] = 1500.

Tables may be computed tables. For example, you may have a table which
holds for each year the total sales in that year, with the income tax
subtracted. It may be defined by a function like this:

lambda year: sum(sales_data[year, model, city] for model in models for
city in cities) / (1 + income_tax_rate )

Now, like in a spreadsheet, the function is kept, so that if you
change the data, the result will be automatically recalculated. So, if
you discovered a mistake in your data, you will be able to write:

sales_data[2004, 'Ford', 'Haifa'] = 2000

and total_sales[2004] will be automatically recalculated.

Now, note that the total_sales table depends also on the
income_tax_rate . This is a variable, just like sales_data. Unlike
sales_data, it's a single value. We should be able to change it, with
the result of all the cells of the total_sales table recalculated. But
how will we do it? We can write

income_tax_rate = 0.18

but it will have a completely different meaning. The way to make the
income_tax_rate changeable is to think of it as a 0-dimensional table.
It makes sense: sales_data depends on 3 parameters (year, model,
city), total_sales depends on 1 parameter (year), and income_tax_rate
depends on 0 parameters. That's the only difference. So, thinking of
it like this, we will simply write:

income_tax_rate[] = 0.18

Now the system can know that the income tax rate has changed, and
recalculate what's needed. We will also have to change the previous
function a tiny bit, to:

lambda year: sum(sales_data[year, model, city] for model in models for
city in cities) / (1 + income_tax_rate[])

But it's fine - it just makes it clearer that income_tax_rate[] is a
part of the model that may change its value.


Have a good day,
Noam

[George Sakkis]

Re: Allowing zero-dimensional subscripts

spam.noam@gmail .com wrote:
[color=blue]
> George Sakkis wrote:[color=green]
> > spam.noam@gmail .com wrote:
> >[color=darkred]
> > > However, I'm designing another library for
> > > managing multi-dimensional arrays of data. Its purpose is similiar to
> > > that of a spreadsheet - analyze data and preserve the relations between
> > > a source of a calculation and its destination.[/color]
> >
> > Sounds interesting. Will it be related at all to OLAP or the
> > Multi-Dimensional eXpressions language
> > (http://msdn2.microsoft.com/en-us/library/ms145506.aspx) ?
> >[/color]
> Thanks for the reference! I didn't know about any of these. It will
> probably be interesting to learn from them. From a brief look at OLAP
> in wikipedia, it may have similarities to OLAP. I don't think it will
> be related to Microsoft's language, because the language will simply by
> Python, hopefully making it very easy to do whatever you like with the
> data.[/color]

Glad it helped, I thought you were already familiar withe these. As for
MDX, I didn't mean you should use it instead of python or implement it
at the syntax level, but whether you consider an API with similar
concepts. Given your description below, I think you should by all means
take a look at MDX focusing on the concepts (datacubes, dimensions,
hierarchies, levels, measures, etc.) and the functions, not its syntax.
Here's a decent manual I found online
http://support.sas.com/documentation...p_mdx_7002.pdf.

Regards,
George