Properties of the space: proximity, directions, losses

What the vector keeps and what is not in it

It helps to stop thinking of a vector as compressed text. It is not an archive: you cannot restore the original string from 768 numbers. A vector is the coordinates of an object on a map built by the model, and exactly one property survives on that map — the relative position of points. The absolute values of the coordinates mean nothing by themselves; only the relations between points carry information.

What is preserved

• Topical proximity. Two texts about delivery end up next to each other even when they are written with different words. This is the main effect everything else is built for.
• Coarse domain structure. Texts about finance, medicine and code form distinguishable regions of the space; clustering over embeddings really does find topics.
• Some directions. The space contains stable axes — a shift from neutral phrasing to emotional phrasing, for instance, or from the general to the specific. The famous 'king minus man plus woman' example works roughly and not on every pair, but the idea itself, that a difference of vectors encodes a relation, holds up on large samples.

What is lost

• Negation. 'A return is possible' and 'a return is not possible' differ by one word and barely differ as vectors. For a policy FAQ this is a disaster.
• Numbers and quantities. The model treats 'a window of 14 days' and 'a window of 40 days' as almost identical: it sees the topic, not the value.
• Order of arguments. 'Transfer from account A to account B' and the reverse transfer often score above 0.95.
• Rare entities. Names, codes and part numbers enter the vector as noise if the model never saw them during training.

pairs where similarity misleads (typical cosine values)
  'service is available' / 'service is unavailable'   ~0.93
  'card payment' / 'crypto payment'                   ~0.78
  'instructions for Windows' / '... for macOS'        ~0.90
  'report for 2023' / 'report for 2024'               ~0.97

The practical conclusion

If a distinction that is critical for your task is carried by a single word or a single digit, vector search will not catch it — whatever model you pick. Such distinctions are moved into metadata and filters: the year of a report becomes a field rather than part of the text; the platform of an instruction becomes a tag; the status of a service becomes a boolean attribute. The embedding stays responsible for the topic, the filter provides the precision. This is a division of labour, not a defect of the technology.

How this is tested

The check takes half an hour. Take 20 pairs from your own domain where you know the answer for certain, compute the similarity and look at the distribution. What matters is not the absolute level but the gap: if pairs that say the same thing in different words average 0.82 while pairs that say different things on one topic average 0.79, no threshold can separate them, and all further work with this model on this data will be a fight against noise. A gap of 0.15 or more means the space fits the task.

Cheat sheet

• A vector preserves relative position, not content — the text cannot be restored.
• Negation, numbers and the direction of a relation are barely expressed in an embedding.
• Move task-critical distinctions into metadata instead of hoping the model handles them.
• Take test pairs from your own domain, not from textbooks.