HeatWave GenAI Features

Key features

In-database large language models

In-database large language models (LLMs) greatly simplify the development of GenAI applications. You can quickly benefit from generative AI; you don’t need to select an external LLM; and you don’t have to consider integration complexity and costs or the availability of an external LLM in various data centers.

Build generative AI apps for a wide range of use cases across clouds

  • You can, for example, use the built-in LLMs to help generate or summarize content and search data to perform retrieval-augmented generation (RAG) with HeatWave Vector Store.
  • You can also combine generative AI with other built-in HeatWave capabilities, such as machine learning, to help deliver more value to customers, reduce costs, and get more accurate results faster.
  • You can use the built-in LLMs in all Oracle Cloud Infrastructure (OCI) regions, OCI Dedicated Region, Oracle Alloy, Amazon Web Services (AWS), and Microsoft Azure and obtain consistent results with predictable performance across deployments.
  • Oracle HeatWave GenAI is integrated with the OCI Generative AI service and with Amazon Bedrock for accessing pretrained foundation models from Cohere and Meta.
  • In-database LLMs and HeatWave Chat help developers deliver apps that are preconfigured for contextual conversations in natural language. There’s no need to subscribe to external LLMs or provision GPUs.
  • LLM inference batch processing helps developers improve application throughput by executing multiple requests simultaneously.
  • The LLMs can take advantage of HeatWave Vector Store to expand their knowledge using proprietary data instead of relying on fine-tuning.

Help reduce costs and risks

  • There are no additional costs to use the in-database LLMs.
  • You can reduce infrastructure costs by eliminating the need to provision GPUs. Additionally, system resources are optimized (optimal configuration of thread count, batch size, and segment size) to further help reduce costs.
  • Native LLM execution within HeatWave helps minimize the risks associated with data movement.

HeatWave Vector Store

HeatWave Vector Store lets you combine the power of LLMs with your proprietary data to help get more accurate and contextually relevant answers than using models trained only on public data. The vector store ingests documents in a variety of formats, including PDF, and stores them as embeddings generated via an embedding model. For a given user query, the vector store helps identify the most similar documents by performing a similarity search against the stored embeddings and the embedded query. These documents are used to augment the prompt given to the LLM so that it provides a more contextual answer for your business.

No AI expertise is required

  • HeatWave Vector Store lets you use generative AI with your business documents without AI expertise and without moving data to a separate vector database.
  • Developers can create a vector store for enterprise unstructured content with a single SQL command.
  • The generation of embeddings in the vector store processes multiple input files in parallel across multiple threads on all cluster nodes. As a result, creating the vector store and ingesting unstructured data in various formats, such as PDF, DOCX, HTML, TXT, or PPTX, is very fast and scales with the cluster size.
  • The pipeline to discover and ingest proprietary documents in the vector store is automated, including transforming users’ unstructured text data and generating embeddings, making it very easy for developers and analysts without AI expertise to leverage the vector store.
  • Multilingual support lets you load documents in 27 languages into HeatWave Vector Store to perform similarity searches and ask questions in various languages.
  • Native JavaScript support lets developers use JavaScript with the VECTOR data type and invoke HeatWave GenAI capabilities from a JavaScript program, for example, to easily build chatbots accessing enterprise data.
  • Optical character recognition support helps you conduct similarity searches by leveraging HeatWave Vector Store to convert scanned content saved as images into text data that can be analyzed, for example, to detect plagiarism.

Costs and risks can be reduced

  • The vector store resides in object storage, making it very cost-effective and highly scalable, even with large data sets. You can easily share the vector store with different applications.
  • Data transformation is completed inside HeatWave, which helps reduce security risks by eliminating data movement and helps reduce costs by eliminating the need for client resources.

Flexible and fast vector processing

Vector processing accelerates with the in-memory and scale-out architecture of HeatWave. HeatWave supports a new native VECTOR data type, letting you use standard SQL to create, process, and manage vector data.

  • You can combine vectors with other SQL operators. For example, you can run analytic queries that join several tables with different documents and perform similarity searches across all documents.
  • In-memory representation and a scale-out architecture mean that vector processing is parallelized across up to 512 HeatWave cluster nodes and executed at memory bandwidth—extremely fast and without any accuracy loss.

HeatWave Chat

A new HeatWave Chat interface lets you have contextual conversations augmented by proprietary documents in the vector store, using natural language.

  • You can interact with unstructured data stored in MySQL Database and in object storage using natural language. The context of the questions is preserved to enable a human-like conversation with follow-up questions. HeatWave maintains a context with the history of questions asked, citations of the source documents, and the prompt to the LLM. This facilitates a contextual conversation and lets you verify the source of answers generated by the LLM. This context is maintained in HeatWave and is available for all applications using HeatWave.
  • The integrated Lakehouse Navigator lets you see data available in MySQL Database and object storage. You can then easily load selected data in HeatWave Vector Store and instruct the LLM to retrieve information from that specific source. As a result, you can reduce costs by searching through a smaller data set while increasing both speed and accuracy.
  • You can search across the entire database or restrict the search to a folder.
  • Several LLMs can be selected via HeatWave Chat, either built-in or accessible with the OCI Generative AI service.