tnit@dZddlZddlZddlZddlmZddlmZmZm Z m Z ddl Z ddl Z ddl mZmZddlmZddlmZdd lmZdd lmZd d lmZd d lmZmZd dlmZd dl m!Z!m"Z"ej#$ddZ%dZ&dZ'ej#$dde'dz Z'dZ(dZ)dZ*dZ+e"Z,eZ-eGddZ!eGddZ.eGddZ/eGdd Z0eGd!d"Z1d#e2d$ee2efd%e3fd&Z4Gd'd(Z5dS))zK Used to launch the FastAPI web server when worker is running in API mode. N) dataclass)AnyDictOptionalUnion) APIRouterFastAPI)jsonable_encoderRedirectResponse)SyncClientSession) __version__) is_generator)run_jobrun_job_generator) Heartbeat)Job JobsProgressRUNPOD_ENDPOINT_IDzRunpod | Development Worker APIa. The Development Worker API facilitates testing and debugging of your Runpod workers. It offers a sandbox environment for executing code and simulating interactions with your worker, ensuring your applications can seamlessly transition to production on Runpod serverless platform. Use this API for comprehensive testing of request submissions and result retrieval, mimicking the behavior of Runpod's operational environment. --- *Note: This API serves as a local testing tool and will not be utilized once your worker is operational on the Runpod platform.* RUNPOD_PROJECT_IDu ℹ️ | Consider developing with our CLI tool to streamline your worker development process. >_ wget -qO- cli.runpod.net | sudo bash >_ runpodctl project create aC Initiates processing jobs, returning a unique job ID. **Parameters:** - **input** (string): The data to be processed by the worker. This could be a string, JSON object, etc., depending on the worker's requirements. - **webhook** (string, optional): A callback URL for result notification upon completion. If specified, the server will send a POST request to this URL with the job's result once it's available. **Returns:** - **job_id** (string): A unique identifier for the job, used with the `/stream` and `/status` endpoints for monitoring progress and checking job status. a Executes processing jobs synchronously, returning the job's output directly. This endpoint is ideal for tasks where immediate result retrieval is necessary, streamlining the execution process by eliminating the need for subsequent status or result checks. **Parameters:** - **input** (string): The data to be processed by the worker. This should be in a format that the worker can understand (e.g., JSON, text, etc.). - **webhook** (string, optional): A callback URL to which the result will be posted. While direct result retrieval is the primary operation mode for this endpoint, specifying a webhook allows for asynchronous result notification if needed. **Returns:** - **output** (Any): The direct output from the processing job, formatted according to the job's nature and the expected response structure. This could be a JSON object, plain text, or any data structure depending on the processing logic. a Continuously aggregates the output of a processing job, returning the full output once the job is complete. This endpoint is especially useful for jobs where the complete output needs to be accessed at once. It provides a consolidated view of the results post-completion, ensuring that users can retrieve the entire output without the need to poll multiple times or manage partial results. **Parameters:** - **job_id** (string): The unique identifier of the job for which output is being requested. This ID is used to track the job's progress and aggregate its output. **Returns:** - **output** (Any): The aggregated output from the job, returned as a single entity once the job has concluded. The format of the output will depend on the nature of the job and how its results are structured. av Checks the completion status of a processing job and returns its output if the job is complete. This endpoint is invaluable for monitoring the progress of a job and obtaining the output only after the job has fully completed. It simplifies the process of querying job completion and retrieving results, eliminating the need for continuous polling or result aggregation. **Parameters:** - **job_id** (string): The unique identifier for the job being queried. This ID is used to track and assess the status of the job. **Returns:** - **status** (string): The completion status of the job, typically 'complete' or 'in progress'. This status indicates whether the job has finished processing and if the output is ready for retrieval. - **output** (Any, optional): The final output of the job, provided if the job is complete. The format and structure of the output depend on the job's nature and the data processing involved. **Note:** The availability of the `output` field is contingent on the job's completion status. If the job is still in progress, this field may be omitted or contain partial results, depending on the implementation. c@eZdZUdZeed<eeeee e e fed<dS)rzRepresents a job.idinputN) __name__ __module__ __qualname____doc__str__annotations__rdictlistintfloatboola/var/www/html/gpu-tools/venv/lib/python3.11/site-packages/runpod/serverless/modules/rp_fastapi.pyrrhs? GGG tS#ud2 333333r(rczeZdZUdZdZeeed<dZee e e ee e efed<dZeeed<dS)TestJobz>Represents a test job. input can be any type of data. Nrrwebhook)rrrrrrr r!rrr"r#r$r%r&r,r'r(r)r+r+psmB @DE8E$c3t;< =DDD!GXc]!!!!!r(r+cHeZdZUdZeeefed<dZe eed<dS)DefaultRequestzRepresents a test input.rNr,) rrrrrr rr!r,rr'r(r)r.r.{s@"" S>!GXc]!!!!!r(r.cteZdZUdZeed<eed<dZeee e ee e e fed<dZeeed<dS) JobOutputzRepresents the output of a job.rstatusNoutputerror)rrrrr r!r2rrr"r#r$r%r&r3r'r(r)r0r0se)) GGG KKKAEFHU4sC<= >EEEE8C=r(r0cxeZdZUdZeed<dZeed<dZee e e ee e efed<dZeeed<dS) StreamOutputzStream representation of a job.r IN_PROGRESSr1Nstreamr3)rrrrr r!r1r7rrr"r#r$r%r&r3r'r(r)r5r5sl)) GGGFCAEFHU4sC<= >EEEE8C=r(r5urlpayloadreturnc*t5} |||d}| ddddS#tj$r+}t d|d|Yd}~ddddSd}~wwxYw#1swxYwYdS)z Sends a webhook to the provided URL. Args: url (str): The URL to send the webhook to. payload (Dict[str, Any]): The JSON payload to send. Returns: bool: True if the request was successful, False otherwise. )jsontimeoutNTzWEBHOOK | Request to z failed: F)rpostraise_for_statusrequestsRequestExceptionprint)r8r9sessionresponseerrs r) _send_webhookrGs    ||Cgr|BBH  % % ' ' '  (    =#==== > > >555   s4B,A  BB/BBBB B ceZdZdZdeeeffdZddZde fd Z d e d e fd Z d e d e fd Zded efdZded e fdZdS) WorkerAPIzMUsed to launch the FastAPI web server when the worker is running in API mode.configc t||_dddddddddg}ttt t d| |_t}| d d d tr'| dtd|j dg| d|j dgddtdg| d|jdgddtdg| d|jdgddt"dg| d|jdgddt&dg|j|dS)z Initializes the WorkerAPI class. 1. Starts the heartbeat thread. 2. Initializes the FastAPI web server. 3. Sets the handler for processing jobs. z.Synchronously Submit Request & Get Job Resultsz>Endpoints for submitting job requests and getting the results.)name descriptionzSubmit Job Requestsz&Endpoints for submitting job requests.zCheck Job ResultszCEndpoints for checking the status of a job and getting the results./)titlerMversiondocs_url openapi_tagsz/docsc"tdS)NrN)r8r r'r(r)z$WorkerAPI.__init__..s-#666r(F)include_in_schemaz /realtimePOST)methodsz/runTz(Mimics the behavior of the run endpoint.)rWresponse_model_exclude_nonesummaryrMtagsz/runsyncz,Mimics the behavior of the runsync endpoint.z/stream/{job_id}z+Mimics the behavior of the stream endpoint.z/status/{job_id}z+Mimics the behavior of the status endpoint.N) heartbeat start_pingrJr TITLE DESCRIPTIONrunpod_versionrp_appr add_api_router _realtime_sim_runRUN_DESCRIPTION _sim_runsyncRUNSYNC_DESCRIPTION _sim_streamSTREAM_DESCRIPTION _sim_statusSTATUS_DESCRIPTIONinclude_router)selfrJ tags_metadata api_routers r)__init__zWorkerAPI.__init__s  I_   .G   ,d    #"&     [[    66% !       $ $1&1114>F8 %       MH(,>''( !        H(,B+BC !        H(,A*%& !        H(,A*%& !    "":.....r( localhost@rc tj|j|t|t|tjddddS)z, Starts the Uvicorn server. UVICORN_LOG_LEVELinfoF)hostportworkers log_level access_logN)uvicornrunr`r$osenvironget)rlapi_hostapi_portapi_concurrencys r) start_uvicornzWorkerAPI.start_uvicornsZ  KX((jnn%8&AA       r(jobcKt|jt|jd|jd{V}t|jt|S)z Performs model inference on the input data using the provided handler. If handler is not provided, returns an error message. handlerN)job_listaddrrrJ__dict__remover )rlr job_resultss r)rbzWorkerAPI._realtimesl  SV$DK $:CLIIIIIIII  ,,,r( job_requestr:cKdtj}t||j|jdt |ddS)z.Development endpoint to simulate run behavior.test-)rrr,r6)rr1)uuiduuid4rrrr,r )rlrassigned_job_ids r)rczWorkerAPI._sim_run0s`0$*,,00 ! &"*      - P PQQQr(cKdtj}t||j}t |jdrQt |jd|j}dgi}|23d{V}|d|d)6n&t|jd|jd{V}| ddrt|j d|ddS|j r7tjt |j |fd }|t|j d |dd S) z2Development endpoint to simulate runsync behavior.rrrrr2Nr3FAILEDrr1r3Ttargetargsdaemon COMPLETEDrr1r2)rrr+rrrJrrappendrr~r rr, threadingThreadrGstart)rlrrrgenerator_output job_output stream_outputthreads r)rezWorkerAPI._sim_runsync;s0$*,,00 0ABBB  I. / / M0Y1GVV "BJ'7 E E E E E E Em8$++M(,CDDDD(8'7 't{9'=s|LLLLLLLLJ >>'4 ( ( #vJwF LLNNN[>'4 ( ( #Jw$ $ $ $ $ $ N# # # # # # # # r(rI)6rr|rr dataclassesrtypingrrrrrArzfastapirr fastapi.encodersr fastapi.responsesr http_clientrrPrr_ rp_handlerrrp_jobrrrp_pingr worker_staterrr}r~rr]r^rdrfrhrjrr[r+r.r0r5r r&rGrIr'r(r)rsQQ  !!!!!!------------&&&&&&&&------......,,,,,,444444$$$$$$........++++++++Z^^$8$??) :>>%t,,4K    " <>> IKK  4444444 4 """"""" " """"""" "                    sT#s(^0s s s s s s s s s s r(