Secondary Vindexes are additional Vindexes against other columns of a table offering optimizations for WHERE clauses that do not use the Primary Vindex. Secondary Vindexes return a single or a limited set of keyspace IDs which will allow VTGate to only target shards where the relevant data is present. In the absence of a Secondary Vindex, VTGate would have to send the query to all shards (called a scatter query).
It is important to note that Secondary Vindexes are only used for making routing decisions. The underlying database shards will need traditional indexes on those same columns, to allow efficient retrieval from the table on the underlying MySQL instances.