NegatedIndex type

[StefanKarpinski]

Idea spawned by this discussion. The idea is that just as writing v[b], v[!b] where b is a logical indexing vector partitions v into two disjoint sets of elements, writing v[x], x[!x] should also partition v into two disjoint sets of elements when x is a single index or a range of indices. This requires some new NegatedIndex type, together with ! methods for integers and ranges and the appropriate method for indexing arrays with NegatedIndex objects.

[JeffBezanson]

What does it do on integer vectors?

[StefanKarpinski]

I'm basically thinking that the structure looks like this:

type NegatedIndex{T}
    idx::T
end

So you just keep the original object until its time to do the actual indexing, at which point you know what you're "subtracting" the index object from, which lets you compute the actual index set and then do regular indexing with that. So ![2,5] would produce NegatedIndex([2,4]). In v[![2,5]], the indexing operations would compute [1:length(v)]-[2,5] as [1,3,5,...,length(v)] (supposing length(v) ≥ 5) and return [v[1],v[3],v[5],...,v[end]]. Make sense?

[ViralBShah]

We do not define ! on non-booleans. That makes it possible for us to give it a different meaning, at least for Index types. But, ~ in matlab does have a defined behaviour, and that could lead to confusion.

That said, I do like the idea, and here are some thoughts.

ComplementIndex is an alternative name. We could alternatively allow \ to be used as a unary operator, to mirror the set complement operator.

A complement operator may even be useful outside of indexing, like in loop iterations and such.

[ViralBShah]

Looking at the names in http://en.wikipedia.org/wiki/Complement_(set_theory), we could even just have a functional interface, using except, which returns the new Index type.

[StefanKarpinski]

I really think that the ! operator makes sense in this case, although I would be ok with writing except(k) as well. Actually, calling the type Except would make a fair amount of sense.

[ViralBShah]

ExceptIndex would be clearer for the type name.

-viral

On 12-Jul-2012, at 11:40 PM, Stefan Karpinski wrote:

I really think that the ! operator makes sense in this case, although I would be ok with writing except(k) as well. Actually, calling the type Except would make a fair amount of sense.

---

Reply to this email directly or view it on GitHub:
#1032 (comment)

[toivoh]

I think it seems a bit fishy to define v[!integer_array]:
would v[![1,2]] == v[![2,1]] == v[![1,2,1]] == v[3] == 3 for v=[1,2,3]?
The trouble seems to be to negate a sequence (array) rather than a set.
Perhaps better to allow v[int_set] and v[!int_set] for subsetting, (and keep v[!int_index])?

[StefanKarpinski]

It is a little weird, but I was thinking that you resolve, e.g. NegatedIndex([2,1]) when indexed into an indexing slot with range 1:n by iterating 1:n and then "emitting" the index value if and only if !contains([2,1],i). Does that make sense?

[toivoh]

I'm a bit worried about surprises that might come out of this. E g you might expect a[!inds] to always give you a subsequence, but what if someone passed an inds::NegatedIndex argument into your function?

The way interpret it, !index is a set complement operation, with sets represented as strictly increasing sequences.
Perhaps one could refuse to create/use a NegatedIndex from a sequence that is not strictly increasing?
Then common cases like a[!2:5] would work, ! would be its own inverse, and

x, y = a[inds], a[!inds]

would always partition the elements of a.

[johnmyleswhite]

This seems to have fizzled out, but I still really, really want this functionality to be added. It's one of the most convenient tricks in R for describing data resampling. See the jackknife for the simplest example of a statistical method in which this notation helps.

I just discovered that one of the other DataFrame developers hacked together an implementation of this style of indexing for the columns DataFrames simply because it was so unnatural to articulate one algorithm without it.

[punkrockpolly]

If I'm understanding this correctly, the negated index [!x] would strip the member whose position has the same value as the index [x], creating a vector slice with the member removed.

Can someone please explain the following:

"For example, the following creates a vector slice with the third member removed.![2,5] would produce NegatedIndex([2,4]) ...
and return [v[1],v[3],v[5],...,v[end]]."

Why wouldn't ![2,5] produce NegatedIndex([2,5]), and v[5] be removed instead of v[4]?? Is there something I'm misunderstanding?

[StefanKarpinski]

Yes, that's absolutely right. I just made a bunch of mistakes jumbled together in that post :-)