[NEP21] Advanced Indexing Semantics #552

Overview

NumPy's advanced indexing behavior and the proposed (deferred) explicit indexing modes.

NEP 21: Simplified and Explicit Advanced Indexing

Status: Deferred | Full Text

Problem 1: Outer vs Vectorized Ambiguity

For arr[[0, 1], [0, 1]] on 2D array:

x = np.arange(9).reshape(3, 3)

# User might expect (outer):
# [[x[0,0], x[0,1]], [x[1,0], x[1,1]]] = [[0, 1], [3, 4]]

# NumPy does (vectorized):
x[[0, 1], [0, 1]]  # = [x[0,0], x[1,1]] = [0, 4]

Problem 2: Confusing Transpose Rules

arr = np.zeros((X, Y, Z))

arr[:, [0,1], 0].shape      # (X, 2) - intuitive
arr[[0,1], 0, :].shape      # (2, Z) - intuitive
arr[0, :, [0,1]].shape      # (2, Y) NOT (Y, 2) - surprising!

Rule: When array indices are separated by slices, result dimensions go to front.

Problem 3: Library Implementation Difficulty

Dask, h5py, xarray struggle to implement NumPy indexing consistently.

Proposed Solution (Deferred)

arr.oindex[...] - Outer Indexing

Array indices behave like slices, result where indices appear:

arr.oindex[:, [0], [0, 1], :].shape  # (5, 1, 2, 8)

arr.vindex[...] - Vectorized Indexing (Explicit)

Array indices broadcast, result always at front:

arr.vindex[:, [0], [0, 1], :].shape  # (2, 5, 8)

arr.legacy_index[...]

Maintains backward compatibility during deprecation.

Current NumPy Behavior Reference

Workaround for Outer Indexing

rows, cols = [0, 1], [0, 1]
result = arr[np.ix_(rows, cols)]  # Shape (2, 2)

Suggested Implementation for NumSharp

Match Current NumPy Behavior

// Vectorized indexing
var arr = np.arange(9).reshape(3, 3);
var idx = np.array(new[] { 0, 1 });
var result = arr[idx, idx];  // Should return [0, 4]

Handle Mixed Index Transpose

// When array indices separated by slices:
// Result dims go to front
var arr = np.zeros(new Shape(X, Y, Z));
var result = arr[0, Slice.All, np.array(new[] {0, 1})];
// result.shape should be (2, Y) not (Y, 2)

Implement np.ix_

public static NDArray[] ix_(params NDArray[] args) {
    // Create open mesh from sequences
    // For outer indexing workaround
}

Future: oindex/vindex Properties (Optional)

public class NDArray {
    public IndexerProxy oindex => new OuterIndexer(this);
    public IndexerProxy vindex => new VectorizedIndexer(this);
}

Documentation

See docs/neps/NEP21.md