(HJjX2ddlZddlmZddlmZmZmZmZddlm Z dddddddgfde de de d e d e d e d e d ee dee jge jffdZ d%deee e fdee dee dee dee dee dee fdZee j e j"j$e j"j$de jd e de jfdZee j e j"j$e j"j$ d&de jde d e de jfdZee j e j"j$e j"j$de jde de jfdZee j e j"j$e j"j$de jd e d e d ee de jf dZee j e j"j$e j"j$dZd'd e d!e fd"Zd'd e d!e fd#Zd'd e d!e fd$Zy)(N)partial)CallableDictListOptionaltemptop_pmin_pmin_tokens_to_keeptop_kxtc_probability xtc_thresholdxtc_special_tokensreturnc dk(rdSg dkDrdkr jfddk7r jfddkDr jfddkDr jfd fd }|S) an Make a sampler function for use with ``generate_step``. Args: temp (float): The temperature for sampling, if 0 the argmax is used. Default: ``0``. top_p (float, optional): Nulceus sampling, higher means model considers more less likely words. min_p (float, optional): The minimum value (scaled by the top token's probability) that a token probability must have to be considered. min_tokens_to_keep (int, optional): Minimum number of tokens that cannot be filtered by min_p sampling. top_k (int, optional): The top k tokens ranked by probability to constrain the sampling to. xtc_probability (float, optional): The probability of applying XTC sampling. xtc_threshold (float, optional): The threshold the probs need to reach for being sampled. xtc_special_tokens (list(int), optional): List of special tokens IDs to be excluded from XTC sampling. Returns: Callable[mx.array, mx.array]: A sampler which takes log-probabilities and returns tokens. rc0tj|dS)Naxis)mxargmax)xs ]/Users/ahmed/devFolder/claude-voice/.venv/lib/python3.12/site-packages/mlx_lm/sample_utils.pyzmake_sampler../s12.?ct|SN) apply_top_p)rr s rrzmake_sampler..4 +a*?rrct|Sr ) apply_min_p)rr r s rrzmake_sampler..6s+a@R*Src t|Sr ) apply_xtc)rrrrs rrzmake_sampler..9si?MCUVrct|Sr ) apply_top_k)rrs rrzmake_sampler..<r"rc:D] }||} t|Sr )categorical_sampling)logprobsmethodsampling_methodsr s rsamplerzmake_sampler..sampler?s&&Fh'H'$Hd33r)append) r r r r rrrrr.r-s ```````` @r make_samplerr0 sH qy.. qyUS[ ?@ | ST V  qy ?@4 Nr logit_biasrepetition_penaltyrepetition_context_sizepresence_penaltypresence_context_sizefrequency_penaltyfrequency_context_sizec| g}|rotjt|j tjt|j  fd}|j |t ||ft||ft||fg} | D]'\} } } |  | dk7s|j | | | )|S)a Make logits processors for use with ``generate_step``. Args: repetition_penalty (float, optional): A (sign-aware) multiplicative penalty for repeating tokens. repetition_context_size (int, optional): The number of tokens to consider for repetition penalty. Default: ``20``. presence_penalty (float, optional): An additive penalty to reduce repeating tokens. presence_context_size (int, optional): The number of tokens to consider for the presence penalty. Default: ``20``. frequency_penalty (float, optional): An additive penalty to reduce repeating tokens. The tokens are penalized proportionally to their frequency. frequency_context_size (int, optional): The number of tokens to consider for the frequency penalty. Default: ``20``. logit_bias (dictionary, optional): Additive logit bias. Returns: List[Callable[[mx.array, mx.array], mx.array]]: A list of logits processors. Each processor in the list is a callable which takes an array of tokens and an array of logits and returns the updated logits. cH|jddfjSr )atadd)_logitsindicesvaluess rlogit_bias_processorz4make_logits_processors..logit_bias_processoros!99QZ(,,V4 4rr) rarraylistkeysr?r/make_repetition_penaltymake_presence_penaltymake_frequency_penalty)r1r2r3r4r5r6r7logits_processorsr@repetition_penalties make_penaltypenalty context_sizer>r?s @@rmake_logits_processorsrLHsD((4  123$z00234 5   !56 !"46MN  02GH !24JK 0D+ g|  7a<  $ $\'<%H I0D r)inputsoutputsr+c J|jd}t|tr d|cxkr|ksntd|d|dt j | |dz dd|d f}t j ||t jtd  |jd }|S) z Sample from only the top K tokens ranked by probability. Args: logprobs: A vector of log probabilities. top_k (int): Top k tokens to sample from. rrz(`top_k` has to be an integer in the (0, z] interval, but is .r kthr.Ninfr) shape isinstanceint ValueErrorr argpartitionput_along_axisrAfloatdtype)r+r vocab_sizemask_idxmasked_logprobss rr(r(s#J eS !!e*@j*@6zlCgQ   yeaibA#uv+NH''(BHHeEl]HNNC"O rcd|cxkrdksntd|t|tr|dkrtd|tj|dd}|t j |z}||k}|dkDr>rctj|}tj|d}tj||d}tj|d}tj tj ||tj|jd|jd}tj||d}tj|d|z kD|td S)z Apply top-p (nucleus) sampling to logits. Args: logprobs: A vector of log probabilities. top_p: The cumulative probability threshold for top-p filtering. Returns: token selected based on the top-p criterion. rr)r[r rS) rexpargsorttake_along_axiscumsumrY zeros_likearangerTr[rdrZ)r+r probssorted_indices sorted_probscumulative_probsinverse_indicess rr!r!s FF8 EZZr2N%%e^"ELyyB7'' n% .&&r*.2F2FG  O ))*:ORTU 881u9$ u  rr=c d|cxkrdksntd|d|cxkrdksntd|tj|d}|tj||kD|tjj kD}|rd|d|f<tjtj jdd |kD|tj|tj |S) a} Apply XTC sampling to the logits. Args: logits: The logits from the model's output. xtc_probability (float): Probability of XTC sampling to happen for each token xtc_threshold (float): The threshold the probs need to reach for being sampled. special_tokens_ids (list(int)): List of special tokens IDs to be excluded from XTC sampling. rg?z?`threshold` has to be a float in the [0, 0.5] interval, but is rz?`probability` has to be a float in the [0, 1] interval, but is rF.r )rWrsoftmaxrdrSminrandomuniform)r=rrrrpmasks rr&r&s  %# %Mm_ ]    'C 'MoM^ _   JJvr "E 288EM15"&&AEEG GD(-S$ $% 88 !Q/1 w' rcLtjj|d|z zS)Nr )rrx categorical)r=r s rr*r*s 99 1t8!4 55rrJrKchdkstttfstdfd}|S)ax Make repetition penalty processor. Paper: https://arxiv.org/abs/1909.05858 Args: penalty (float): The repetition penalty factor to be applied. context_size (int): The number of previous tokens to use. Default: ``20``. Returns: Callable[[mx.array, List[int]], mx.array]: The repetition penalty processor. rz*penalty must be a non-negative float, got ct|dkDr8| d}|dd|f}tj|dk|z|z }||dd|f<|SNr)lenrrd)tokensr=selected_logitsrKrJs rrepetition_penalty_processorz=make_repetition_penalty..repetition_penalty_processor,se v;?\MN+F$QY/O hh!#')')O !0F1f9  r)rUrVrZrW)rJrKrs`` rrDrDs9{*WsEl;EgYOPP  ('rcfd}|S)a Make a presence penalty processor. Corresponds to the OpenAI option with the same name. Namely, subtracts ``penalty`` from a logit if the token has occured at least once in the ``context_size`` previous tokens. Args: penalty (float): The presence penalty to be applied. context_size (int): The number of previous tokens to use. Default: ``20``. Returns: Callable[[mx.array, List[int]], mx.array] cRt|dkDr| d}|dd|fxxzcc<|Sr)rrr=rKrJs rpresence_penalty_processorz9make_presence_penalty..presence_penalty_processorLs5 v;?\MN+F 1f9  (  r)rJrKrs`` rrErE;s" &%rcfd}|S)aL Make a frequency penalty processor. Corresponds to the OpenAI option with the same name. Namely, subtracts ``penalty`` from a logit for every time that the token has occured in the ``context_size`` previous tokens. The difference with the presence penalty is that the more often a token occurs the more it will be penalized. Args: penalty (float): The frequency penalty to be applied. context_size (int): The number of previous tokens to use. Default: ``20``. Returns: Callable[[mx.array, List[int]], mx.array] ctt|dkDr(| d}|jdd|fj}|Sr)rr:subtractrs rfrequency_penalty_processorz;make_frequency_penalty..frequency_penalty_processoris> v;?\MN+FYYq&y)227;F rr)rJrKrs`` rrFrFUs( '&r)NNNrNr)r )r)rb functoolsrtypingrrrrmlx.corecorerrZrVrAr0rLcompilerxstater(r$r!r&r*rDrErFrrrrs  11 $&; ; ; ; ;  ;  ;;S ;rxxj"(("#;~.2*.-/(,+-)-,.6c5j)*6 6&c]6uo 6 $C= 6   6%SM6r BIIOORYY__Ehh XXF0 BIIOORYY__E .?hh.? .?.?XX .?F.?b BIIOORYY__E "(( 5 RXX F F BIIOORYY__E! HH!!!S ! XX !F!H BIIOORYY__E6F6(U(#(B&5&&4'E''r