attention-head based DeepSpeed implementation for llm clustering.

Input

5970-dim embedding

Encoder

37 x MLP with 24 heads

Output

mAP projection