diff --git a/content/en/llm_observability/sdk.md b/content/en/llm_observability/sdk.md index de3f1d8e9c6..2b48e2789b4 100644 --- a/content/en/llm_observability/sdk.md +++ b/content/en/llm_observability/sdk.md @@ -104,6 +104,12 @@ LLMObs.enable( : optional - _string_
The name of the service used for your application. If not provided, this defaults to the value of `DD_SERVICE`. +### AWS Lambda setup + +Enable LLM Observability by specifying the required environment variables in your [command line setup](#command-line-setup) and following the setup instructions for the [Datadog-Python and Datadog-Extension][14] AWS Lambda layers. + +**Note**: Using the `Datadog-Python` and `Datadog-Extension` layers automatically turns on all LLM Observability integrations, and force flushes spans at the end of the Lambda function. + #### Application naming guidelines Your application name (the value of `DD_LLMOBS_ML_APP`) must be a lowercase Unicode string. It may contain the characters listed below: @@ -564,7 +570,7 @@ def separate_task(workflow_span): return {{< /code-block >}} -### Flushing in serverless environments +#### Force flushing in serverless environments `LLMObs.flush()` is a blocking function that submits all buffered LLM Observability data to the Datadog backend. This can be useful in serverless environments to prevent an application from exiting until all LLM Observability traces are submitted. @@ -656,3 +662,4 @@ def server_process_request(request): [11]: https://docs.datadoghq.com/tracing/trace_collection/compatibility/python/#integrations [12]: https://docs.datadoghq.com/tracing/trace_collection/compatibility/python/#library-compatibility [13]: /llm_observability/auto_instrumentation/ +[14]: https://docs.datadoghq.com/serverless/aws_lambda/installation/python/?tab=custom#installation diff --git a/content/en/llm_observability/trace_an_llm_application.md b/content/en/llm_observability/trace_an_llm_application.md index 10b716cb1a8..3615fc8e7f9 100644 --- a/content/en/llm_observability/trace_an_llm_application.md +++ b/content/en/llm_observability/trace_an_llm_application.md @@ -22,7 +22,7 @@ If you're new to LLM Observability traces, read the [Core Concepts][3] before pr ## Instrument your LLM application -
This guide uses the LLM Observability SDK for Python. If your application is not written in Python, you can complete the steps below with API requests instead of SDK function calls.
+
This guide uses the LLM Observability SDK for Python. If your application is running in a serverless environment, follow the serverless setup instructions. If your application is not written in Python, you can complete the steps below with API requests instead of SDK function calls.
Datadog provides [auto-instrumentation][4] to capture LLM calls for specific LLM provider libraries. However, manually instrumenting your LLM application using the Python SDK can unlock even more of Datadog's LLM Observability features. @@ -185,4 +185,4 @@ Depending on the complexity of your LLM application, you can also: [14]: /llm_observability/sdk/#tracking-user-sessions [15]: /llm_observability/sdk/#tracing-multiple-applications [16]: /llm_observability/submit_evaluations -[17]: /llm_observability/core_concepts/#spans \ No newline at end of file +[17]: /llm_observability/core_concepts/#spans