"meta-variable x repeats N times, but y repeats M times" error is confusing

[clarfonthey]

So, I ran into this when writing up a macro for a project. I'm not really sure what a minimal example would be, but here's some slightly-shorter-than-what-I-wrote code: (playground link)

macro_rules! test_cons {
    (
        $ty:ident {
            array: [$($arr:literal),*],
            $(
                $from:ident: $val:expr,
            )*
        }
    ) => {
        $(
            test_arrays! { ($ty::$from)($val) == [$($arr),*] }
        )*
    }
}

macro_rules! test_arrays {
    (($method:path)($val:expr) == [$($arr:literal),*]) => {
        {
            let x = $method($val);
            assert_eq!(x.to_array(), [$($arr),*], "{}({:?}).to_array() != {:?}", stringify!($method), $val, [$($arr),*]);
        }
    }
}

fn do_test() {
    test_cons! {
        MyTy {
            array: [1, 2, 3],
            from_array: [1, 2, 3],
            from_shorter_array: [1, 2],
        }
    }
}

In case it's not obvious what I'm trying to do here: I want to verify that MyTy::from_array([1, 2, 3]).to_array() == [1, 2, 3] and that MyTy::from_shorter_array([1, 2]).to_array() == [1, 2, 3]. This is shorter than the original macro I wrote, but should be enough to demonstrate the error.

The current output is:

error: meta-variable `from` repeats 2 times, but `arr` repeats 3 times
  [--> src/lib.rs:10:10
](https://play.rust-lang.org/#)   |
10 |           $(
   |  __________^
11 | |             test_arrays! { ($ty::$from)($val) == [$($arr),*] }
12 | |         )*
   | |_________^

That's it. No suggestions for what might be wrong, not even a link to an error code. Just saying that what I wrote is wrong. I tried messing around with metavar expressions as specified in #83527 but I don't know what exactly would would help here.

Basically, I have no idea what exactly the circumstances for this error are, or what I'm doing wrong here, but either way, this error is extremely confusing and could be improved to at least explain what the compiler thinks is happening, and what I could do to fix that.

[zirconium-n]

Minimal example

macro_rules! test {
    (
        [$($arr1:literal),*],
        [$($arr2:literal),*],    
    ) => {
        [$($arr1,$arr2),*]
    }
}

fn do_test() {
    test! {
        [1],
        [1, 2],
    }
}

You are zipping meta-variables with different length.

[clarfonthey]

Your example isn't equivalent to mine; while yours nests only at one level, mine nests at two. So, while I unambiguously would understand that to be zipping, what I'm doing is a cartesian product. Here's an example similar to what you wrote:

macro_rules! test {
    (
        [$($arr1:literal),*],
        [$($arr2:literal),*],    
    ) => {
        [$($arr1,$($arr2),*)*]
    }
}

fn do_test() {
    test! {
        [1],
        [1, 2],
    }
}

Which, FWIW, does fail for the same reason. The confusion I get is that this example is interpreted completely fine as a cartesian product, as long as one of the variables isn't repeating:

macro_rules! test {
    (
        [$arr1:literal],
        [$($arr2:literal),*],    
    ) => {
        [$arr1,$($arr2),*]
    }
}

fn do_test() {
    test! {
        [1],
        [1, 2],
    }
}

[zirconium-n]

I see. My bad. I guess what's happening here is that compiler emitted wrong error message.

It really should have been

error: attempted to repeat an expression containing no syntax variables matched as repeating at this depth

See:

macro_rules! test {
    (
        [$($arr1:literal),*],
        [$($arr2:literal),*],    
    ) => {
        [$($($arr2),*)*]   // removed arr1
    }
}

fn do_test() {
    test! {
        [1],
        [1, 2],
    }
}

[clarfonthey]

Yeah, there are definitely two problems here: 1) is the compiler isn't even giving the right error message to begin with, but more importantly 2) the compiler isn't giving any additional context that could help solve the error.

[zirconium-n]

Yeah, there are definitely two problems here: 1) is the compiler isn't even giving the right error message to begin with, but more importantly 2) the compiler isn't giving any additional context that could help solve the error.

1. agree

2. Hmm... I think we need some notation to indicate the inner $ is actually at depth 0. But for your specific case in OP, this would do.

macro_rules! test_cons {
    (
        $ty:ident {
            array: $arr:tt,
            $(
                $from:ident: $val:expr,
            )*
        }
    ) => {
        $(
            test_arrays! { ($ty::$from)($val) == $arr }
        )*
    }
}

[clarfonthey]

I think that labelling scope here would probably be a good start. For example, borrow errors show the exact scopes of the lifetimes to point out conflicts, and here I think pointing out the depth that each variable is repeating (showing the scope of the $()*) would be some good groundwork for this.

[ia0]

2. Hmm... I think we need some notation to indicate the inner $ is actually at depth 0. But for your specific case in OP, this would do.

Are there any plan to get such notation? IMO this implicit zipping feature is not intuitive (although I agree it increases expressiveness) and should not have been the default. My mental model until now (and I wonder if macros didn't use to work like that in the past) was to match each repetition from the output to one repetition from the input with the following algorithm:

• There should always be at least one meta-variable under innermost output repetitions (${ignore(x)} must be explicitly added otherwise).
• If a meta-variable appears multiple times under the same "stack" (i.e. at different depth under the same path of repetitions) then only the outermost counts and the inner occurrences are just duplicated and not repeated.
• All remaining meta-variables under the same output repetition must share the same input repetition (this is where the zipping behavior could be used in case they don't share the same input repetition but those repetition have the same length).

Here's an illustration of what I mean:

macro_rules! ex1 { ($($a:ident)*, $($b:ident)*) => ($($($a)* $b)*); }
// The outer repetition is defined by $b (only occurrence).
// The inner repetition is defined by $a (only occurrence).
ex1!(a b, x y z) => (a b x a b y a b z);
ex1!(, x y) => (x y);
ex1!(a b, ) => ();

macro_rules! ex2 { ($($a:ident)*, $($b:ident)*) => ($(${ignore(a)} $($a $b)*)*); }
// We use ${ignore(a)} to define the outer repetition according to $a.
// The inner repetition is defined by $b since $a is already used higher in the "stack"/path.
ex2!(a b, x y z) => (a x a y a z b x b y b z);
ex2!(, x y) => ();
ex2!(a b, ) => ();

macro_rules! ex3 { ($($a:ident)*, $($b:ident)*) => ($($a $b)*); }
// This should either be rejected because $a and $b don't share the same input repetition.
// Or we could use the zipping behavior if the lengths are the same. That's apparently the current behavior.

macro_rules! ex4 { ($($a:ident)*) => ($($a $($a)*)*); }
// This should be rejected because $a defines the outer repetition.
// As such, there's no meta-variable remaining to define the inner repetition.

macro_rules! ex5 { ($($a:ident)*, $($b:ident)*) => ($($a $($a $b)*)*); }
// This is a mix of ex2 and ex4.
// $a defines the outer repetition.
// $b defines the inner repetition.
ex5!(a b, x y z) => (a a x a y a z b b x b y b z);

Here is a concrete minimal example where the zipping behavior is over-triggering (this case is even more sad because the repetition symbol is different, so just looking at the repetition symbols it's obvious how to assign output repetitions to input repetitions):

macro_rules! copy_doc {
    ($(#[$m:meta])* $(cfg $c:tt $i:item)+) => {
        $($(#[$m])* #[cfg $c] $i)+
    };
}

copy_doc! {
    /// Some doc for foo regardless of feature.
    cfg(feature = "bar")
    pub fn foo(x: Bar);
    cfg(feature = "baz")
    pub fn foo(x: Baz, y: Baz) -> Baz;
}

In conclusion, I believe this zipping behavior should only be a last resort option and in particular should only be an alternative to otherwise rejected macros and not to otherwise valid macros.

EDIT: Actually found a workaround for copy_doc (based on the solution in this thread) by adding 2 more steps (one wrapping the repetition that I want to nest under a token, then nest that token into the repetition that will become the outer repetition, and finally unwrap the inner repetition):

macro_rules! copy_doc {
    ($(#[$m:meta])* $(cfg $c:tt $i:item)+) => {
        copy_doc!(1 { $(#[$m])* } $(cfg $c $i)+);
    };
    (1 $m:tt $(cfg $c:tt $i:item)+) => {
        copy_doc!(2 $($m cfg $c $i)+);
    };
    (2 $({ $(#[$m:meta])* } cfg $c:tt $i:item)+) => {
        $($(#[$m])* #[cfg $c] $i)+
    };
}