AHjUTddlZddlmZmZmZmZmZmZddlZddl m Z GddZ y)N)AnyDictIteratorListOptionalUnion)UnauthorizedErrorcleZdZddedeefdZdeeeffdZ ddedeeeefd ee d ee d ee d ee d ee dee deede de eee fejffdZ ddedeeeefd ee d ee d ee d ee d ee dee deedeeee ffdZy) ChatClientNbase_urlapi_keyc>|jd|_||_y)a Initialize the ChatClient. Args: base_url (str): The base URL of the LiteLLM proxy server (e.g., "http://localhost:8000") api_key (Optional[str]): API key for authentication. If provided, it will be sent as a Bearer token. /N)rstrip _base_url_api_key)selfr rs Q/root/tools/cai/cai_env/lib/python3.12/site-packages/litellm/proxy/client/chat.py__init__zChatClient.__init__ s"- returncJddi}|jrd|j|d<|S)z Get the headers for API requests, including authorization if api_key is set. Returns: Dict[str, str]: Headers to use for API requests z Content-Typezapplication/jsonzBearer Authorization)r)rheaderss r _get_headerszChatClient._get_headerss0"#56 ==)0'@GO $rmodelmessages temperaturetop_pn max_tokenspresence_penaltyfrequency_penaltyuserreturn_requestc |jd} ||d} ||| d<||| d<||| d<||| d<||| d<||| d <| | | d <tjd | |j| } | r| Stj} |j | j }|j|jS#tjj$r*}|jjd k(r t|d}~wwxYw)a Create a chat completion. Args: model (str): The model to use for completion messages (List[Dict[str, str]]): The messages to generate a completion for temperature (Optional[float]): Sampling temperature between 0 and 2 top_p (Optional[float]): Nucleus sampling parameter between 0 and 1 n (Optional[int]): Number of completions to generate max_tokens (Optional[int]): Maximum number of tokens to generate presence_penalty (Optional[float]): Presence penalty between -2.0 and 2.0 frequency_penalty (Optional[float]): Frequency penalty between -2.0 and 2.0 user (Optional[str]): Unique identifier for the end user return_request (bool): If True, returns the prepared request object instead of executing it Returns: Union[Dict[str, Any], requests.Request]: Either the completion response from the server or a prepared request object if return_request is True Raises: UnauthorizedError: If the request fails with a 401 status code requests.exceptions.RequestException: If the request fails with any other error /chat/completions)rrNrr r!r"r#r$r%POST)rjson)rrequestsRequestrSessionsendprepareraise_for_statusr* exceptions HTTPErrorresponse status_coder )rrrrr r!r"r#r$r%r&urldatarequestsessionr4es r completionszChatClient.completions!s9H 12*/HE  ""-D   !DM =DI  !!+D   ''7D# $  ((9D$ %  DL""638I8I8KRVW N""$ ||GOO$56H  % % '==? """,, zz%%,'**  s>CD "%DD c #K|jd} ||dd} ||| d<||| d<||| d<||| d<||| d <||| d <| | | d <tj} | j| |j | d } | j | j D]\}|s|jd }|jds)|dd}|jdk(ry tj|}|^y#tj$rYuwxYw#tjj$r*}|jj dk(r t#|d}~wwxYww)a Create a streaming chat completion. Args: model (str): The model to use for completion messages (List[Dict[str, str]]): The messages to generate a completion for temperature (Optional[float]): Sampling temperature between 0 and 2 top_p (Optional[float]): Nucleus sampling parameter between 0 and 1 n (Optional[int]): Number of completions to generate max_tokens (Optional[int]): Maximum number of tokens to generate presence_penalty (Optional[float]): Presence penalty between -2.0 and 2.0 frequency_penalty (Optional[float]): Frequency penalty between -2.0 and 2.0 user (Optional[str]): Unique identifier for the end user Yields: Dict[str, Any]: Streaming response chunks from the server Raises: UnauthorizedError: If the request fails with a 401 status code requests.exceptions.RequestException: If the request fails with any other error r(T)rrstreamNrr r!r"r#r$r%)rr*r=zutf-8zdata: z[DONE]r+)rr,r.postrr1 iter_linesdecode startswithstripr*loadsJSONDecodeErrorr2r3r4r5r )rrrrr r!r"r#r$r%r6r7r9r4linedata_strchunkr:s rcompletions_streamzChatClient.completions_streamjsB 12*/HPTU  ""-D   !DM =DI  !!+D   ''7D# $  ((9D$ %  DL""$ ||T..0tD$H  % % '!++- %;;w/Dx0#'8#>>+x7!%$(JJx$8E"'K % $33%$%"",, zz%%,'**  saAE$AD'"D D#E$%D>DE$DDDDE!7%EE!!E$)N)NNNNNNNF)NNNNNNN)__name__ __module__ __qualname__strrrrrrfloatintboolrrr,r-r;rrIrrr r s    x}   d38n  (,!%$(,0-1"$GGtCH~&Ge_ G  G C= GSMG#5/G$E?GsmGG tCH~x/// 0GZ(,!%$(,0-1"OOtCH~&Oe_ O  O C= OSMO#5/O$E?OsmO $sCx. !Orr ) r*typingrrrrrrr,r2r r rQrrrSs  ==)ppr