GJjtRddlZddlmZddlmZddlmZddlm Z GddeZ y)N)Optional)Module)QuantizedEmbeddingc leZdZdZdedeffd ZdZdZdZ d de ed e ed e d e fd Z xZ S) EmbeddingavImplements a simple lookup table that maps each input integer to a high-dimensional vector. Typically used to embed discrete tokens for processing by neural networks. Args: num_embeddings (int): How many possible discrete tokens can we embed. Usually called the vocabulary size. dims (int): The dimensionality of the embeddings. num_embeddingsdimsct|tjd|z }tj j ||f||_y)N)shapescale)super__init__mathsqrtmxrandomnormalweight)selfrr r __class__s a/Users/ahmed/devFolder/claude-voice/.venv/lib/python3.12/site-packages/mlx/nn/layers/embedding.pyrzEmbedding.__init__s@  !d(#ii&&nd-C5&Q cl|jjdd|jjdS)Nrz, r )rr )rs r _extra_reprzEmbedding._extra_reprs3++##A&'r$++*;*;A*>)?@@rc |j|S)N)rrxs r__call__zEmbedding.__call__s{{1~rc4||jjzS)z Call the embedding layer as a linear layer. Use this for example when input embedding and output projection weights are tied. )rTrs r as_linearzEmbedding.as_linear"s4;;==  r group_sizebitsmodequantize_inputcL|r tdtj||||S)zPReturn a :obj:`QuantizedEmbedding` layer that approximates this embedding layer.z!Quantized input is not supported.) ValueErrorrfrom_embedding)rr#r$r%r&s r to_quantizedzEmbedding.to_quantized+s* @A A!00z4NNr)NNaffineF)__name__ __module__ __qualname____doc__intrrrr"rstrboolr* __classcell__)rs@rrr st RsR#R A!%)"$ OSM Osm O O  Orr) rtypingrmlx.corecorermlx.nn.layers.basermlx.nn.layers.quantizedrrrrr:s$ %6*O*Or