For those who don't know what spec does, here's a brief primer: spec is a script that simplifies the generation of a linker script that is passed into program "ld". The linker script lets you specify how to map out all of the C object files in memory and in rom.
There are two address spaces that ld allows you to map file sections to, a Virtual Memory Address (VMA) and a Load Memory Address (LMA). An LMA can be thought of as being the address where the section will be located within the output binary, while a VMA will set where the section will exist when that section is loaded into memory.

So, why do I think `number` shouldn't be part of the spec format? To answer, I want to address the arguments for why `number` should be part of the format, and why these arguments are very poor.
Looking through Roman's comments on the topic, I find two key positions that argue for having `number` in the spec format over simply using `address`.

 * N64 devs reference Segment Addresses by Segment Ids
 * There should exist a separation between static and "vram" addresses, and segment address spaces, because that's what the N64 does.

For the first point, I agree that we should have defines for Segment Ids, not for the entire segment address. That said, I don't understand why that is a strong enough justification for creating a totally new attribute, when defining a macro that generates a full segment address is just as acceptable. Is there a reason we shouldn't be using macros?

For the second point,
 1. There isn't any reason to implement such a separation at the linker level, because at the end of the day we're setting the VMA cursor for both, regardless of the address space.
 2. We aren't going to properly support such a separation anyway. As such, `number` is superfluous at best, and confusing at worst.

To see what I mean let's start by looking at what spec's `address` does. Whenever you add the `address` statement to a segment, the VMA cursor is assigned to the `address` value. From that point on, subsequent segments will start wherever the VMA cursor's value has incremented to.

Now let's compare it to what spec's `number` does. Whenever you add the `number` statement to a segment, the VMA cursor is assigned to the value `(number & 0xFF) << 24`. From that point on, subsequent segments will start wherever the VMA cursor's value has incremented to.

As you can see, `number` is simply syntax sugar for `address`. spec isn't going to differentiate between "object file" address space and overlay address space, but because there's two attributes, users might assume that they are unrelated, or have separate functionality. Consider this example:

beginseg
    name "ovl_Actor_A"
    address 0x80800000
    include "z_Actor_A.o"
endseg

beginseg
    name "object_A"
    number 6
    include "object_A.o"
endseg

beginseg
    name "ovl_Actor_B"
    include "z_Actor_B.o"
endseg

beginseg
    name "object_B"
    number 6
    include "object_B.o"
endseg

Imagine a naive user writes a spec file like this, in an attempt to keep actor and object file definitions next to one another. Say they assume that ovl_Actor_B will follow ovl_Actor_A in memory, because `number` is meant for object files. This spec file will put Actor_B somewhere at address 06xxxxxx, not address 808xxxxx. Furthermore, as long as Actor_B doesn't call any external functions, the game should link without error, and I think that the actor will be relocated correctly as well (haven't really tested this though).

Now assuming that we absolutely must separate addresses at the script level, then my final argument against `number` is that the name `number` itself is terribly generic. What is a number? Why do some segments have "numbers" and most others don't? Why does number affect the address that segments are mapped to?