assistant ========= .. py:module:: assistant .. autoapi-nested-parse:: viur-assistant AI-based assistance module plugin for ViUR Submodules ---------- .. toctree:: :maxdepth: 1 /viur/assistant/bones/index /viur/assistant/config/index /viur/assistant/modules/index /viur/assistant/skeletons/index /viur/assistant/version/index Attributes ---------- .. autoapisummary:: assistant.BONE_ACTION_KEY assistant.CONFIG Classes ------- .. autoapisummary:: assistant.BoneAction assistant.ImageBone assistant.ImageBoneRelSkel assistant.Assistant Package Contents ---------------- .. py:data:: BONE_ACTION_KEY :type: Final[str] :value: 'actions' The key of the parameter .. py:class:: BoneAction .. py:attribute:: TRANSLATE :type: Final[str] :value: 'translate' Translate text .. py:attribute:: DESCRIBE_IMAGE :type: Final[str] :value: 'describe_image' Describe image with an alt text .. py:class:: ImageBone(*, using = ImageBoneRelSkel, validMimeTypes = ('image/*', ), enable_describe_image = True, **kwargs) Bases: :py:obj:`viur.core.bones.FileBone` A specialized ``FileBone`` for image files. This bone type extends ``FileBone`` by: - Restricting accepted MIME types to images (by default). - Has a using skel with an alt ``StringBone``. - Optionally enabling the *Describe Image* bone action, which allows AI to generate an alt-text or caption for the uploaded image via an admin-triggerable action. Initialize an ImageBone, a file-based bone specialized for handling image uploads. Optionally adds a bone action that allows AI-based image description (alt-text generation) to be triggered from within the admin interface. :param using: The relational skeleton class used for additional data of this image. Defaults to ``ImageBoneRelSkel``, including the alt ``StringBone``. :param validMimeTypes: A list of accepted MIME types. Defaults to only allow image types (``("image/*",)``). :param enable_describe_image: If ``True``, the bone will include the ``DESCRIBE_IMAGE`` bone action, allowing AI-assisted image description via the vi-admin UI. :param kwargs: Additional keyword arguments passed to the base ``FileBone``. .. py:attribute:: type .. py:class:: ImageBoneRelSkel Bases: :py:obj:`viur.core.skeleton.RelSkel` RelSkel designed to use as using skeleton in a ``ImageBone``. .. py:attribute:: alt .. py:data:: CONFIG :type: Final[AssistantConfig] The viur-assistent config instance. .. py:class:: Assistant Bases: :py:obj:`viur.core.prototypes.Singleton` Provides LLM-powered utilities within a ViUR module context. This class includes various AI-assisted features such as script generation, image description, and language-aware prompting. It is designed to be used in the context of a ViUR admin and integrates with services like OpenAI and Anthropic. Responsibilities: - Generating backend scripts from user instructions and data models. - Producing accessible image descriptions (e.g., for alt attributes). - Translate text. - Managing prompt and API settings. Integrated Services: - OpenAI (e.g., GPT) for image description generation and translations. - Anthropic Claude for structured script generation with reasoning capabilities. Configuration can be made in - this singleton skel itself. - The backend configuration `CONFIG`. .. note:: The methods in this module assumes a properly configured environment with API keys and AI model settings. However, you only need to configure what is actually used. Error handling is included for common failure modes such as misconfiguration or service unavailability. .. py:attribute:: kindName :type: Final[str] :value: 'viur-assistant' .. py:method:: generate_script(*, prompt, modules_to_include = None, enable_caching = False, max_thinking_tokens = 0) Generates a script based on a user prompt and optional module structures using a language model. This method builds a structured prompt for the LLM Anthropic Claude and optionally includes application-specific module metadata to enrich the generation context. Additional configuration such as caching behavior and token budgeting for "thinking steps" can be provided. :param prompt: The main user instruction or query that guides the script generation. :param modules_to_include: Optional list of module names whose structures should be included as part of the model context. These are injected into the LLM prompt as JSON. :param enable_caching: If set to True, instructs the system to use ephemeral caching for the scriptor documentation prompt section. :param max_thinking_tokens: If greater than 0, enables the model's "thinking" feature with a token budget for intermediate reasoning or planning steps. :return: A JSON-encoded string of the model's response, typically containing the generated script. :raises InternalServerError: - If configuration (`skel`) is missing. - If the LLM request fails due to connection or model errors. .. note:: - Requires a valid `anthropic_model` configuration in the current context. - The actual parsing of the generated code (e.g., extracting specific script content) is currently marked as a TODO and has to be discussed. .. py:method:: get_viur_structures(modules_to_include) Collect and return ViUR module structures for a given list of module names. For each named module, its structure is extracted if it is of type ``List`` or ``Tree``. ``Tree`` structures will return separate entries for ``node`` and ``leaf`` skeletons. :param modules_to_include: List of ViUR module names to retrieve structures for. :return: A dictionary mapping module names to their respective structure definitions. For ``Tree`` modules, nested keys ``"node"`` and ``"leaf"`` are returned. :raises ValueError: If a module exists but is not a supported type (i.e., not ``List`` or ``Tree``). .. note:: Modules that are missing or not found are silently skipped. .. py:method:: translate(*, text, language, characteristic = None) Translate a given text into a target language, optionally using a specific style. This method sends the input text to OpenAI with instructions to translate it into the requested language, optionally applying predefined translation characteristics such as simplification. :param text: The source text to translate. :param language: The target language code (e.g. ``"de"``, ``"en"``, ``"de-x-simple"``). :param characteristic: Optional translation style (e.g. ``"simplified"``, ``"formal"``, etc.) as defined in ``CONFIG.translate_language_characteristics``. :return: Translated text as a plain string. HTML tags from the original text are preserved. :raises InternalServerError: If configuration is missing. .. note:: - The translation style is determined by merging base rules (`*`) and the selected characteristic. - The returned translation contains only the translated text, with no explanation or additional formatting. .. py:method:: describe_image(filekey, prompt = '', context = '', language = None) Generate an HTML ``alt`` attribute description for a given image using OpenAi. This method reads an image via its filekey, resizes it to a configured pixel target, and sends it along with optional prompt and context data to an OpenAI model. The model returns a plain-text alternative description for accessibility purposes, formatted as an HTML ``alt`` text in the specified language. :param filekey: Key identifying the image file in the ViUR file module. (Key of the :class:`file skeleton `). :param prompt: Optional user-defined hint or instruction for how the image should be interpreted. :param context: Optional additional background information to support a better description. :param language: Target language code for the generated description (e.g., ``"en"``, ``"de-x-simple"``). Falls back to the current session language if not specified. :return: A plain-text string suitable for use in an HTML ``alt`` attribute (no quotes or labels). :raises InternalServerError: If required configuration is missing. :raises NotFound: If the referenced image file could not be loaded. .. note:: - The image is resized and converted to JPEG before being sent to the model. - This function uses a preconfigured OpenAI model, pixel count, and JPEG quality settings. .. py:method:: _get_resized_image_bytes(image, target_pixel_count, jpeg_quality = 50) Resize an image to approximately match a target total pixel count and return it as a JPEG byte stream. The image is scaled proportionally to meet the target pixel count (width × height), while preserving the original aspect ratio. If the resized dimensions would exceed the original image size, no upscaling is performed. The result is encoded as a JPEG. :param image: Input image, provided as a file-like object, byte string, file path, or raw bytes. :param target_pixel_count: Desired total number of pixels for the resized image. Used to compute the new dimensions. :param jpeg_quality: JPEG compression quality (0 to 100). Higher values yield better image quality at the cost of file size. Default is 50. :return: The resized and JPEG-compressed image as a byte stream. :raises ValueError: If `jpeg_quality` is not in the 0–100 range or the image input is invalid. .. note:: - Images in PNG, SVG, or WEBP format are automatically converted to RGB JPEG. - This function is intended for preprocessing images before passing them to AI models, balancing detail and data size. .. py:method:: openai_create_completion(*, model, messages, **kwargs) Creates a model response in a new chat conversation. Uses OpenAI API and structured JSON format, see https://platform.openai.com/docs/guides/structured-outputs?api-mode=responses#json-mode . :param model: Model ID used to generate the response, like gpt-4o or o3. :param messages: A list of messages comprising the conversation. :param kwargs: Additional arguments passing to the client. :return: The response in plain text on success. :raises errors.HTTPException: If an API error occurs. .. py:method:: render_text(text) Render the give text as usual for the current renderer. If the current renderer is the JSON renderer, JSON string is returned and the content-type header is also set to JSON. :param text: The text to render.