HeatWave GenAI FAQ
General
What is HeatWave GenAI?
Oracle HeatWave GenAI provides integrated, automated, and secure generative AI with in-database large language models (LLMs); an automated, in-database vector store; scale-out vector processing; and the ability to have contextual conversations in natural language—letting you take advantage of generative AI without AI expertise, data movement, or additional cost.
What are the key features of HeatWave GenAI?
Key features of HeatWave GenAI include
- In-database LLMs
- Integration with OCI Generative AI and Amazon Bedrock
- An in-database vector store
- Automated generation of embeddings
- Scale-out vector processing
- HeatWave Chat
For more information on these features, please read the HeatWave GenAI technical brief (PDF).
Which LLMs are available as in-database models in HeatWave?
Quantized versions of the following in-database LLMs are currently available in HeatWave:
- llama3-8b-instruct-v1
- mistral-7b-instruct-v1
How is HeatWave Vector Store automated?
You can create a vector store for enterprise unstructured content with a single SQL command. All the steps to create a vector store and vector embeddings are automated and executed inside the database, including discovering the documents in object storage, parsing them, generating embeddings in a highly parallel and optimized way, and inserting them into the vector store, making HeatWave Vector Store efficient and easy to use.
Does HeatWave require GPUs to run in-database LLMs?
HeatWave runs on commodity hardware. In-database LLMs don’t run on GPUs; they run on CPUs. As a result, you can reduce costs and don’t need to worry about the availability of LLMs in various data centers.
How does HeatWave GenAI help improve data security?
When using in-database LLMs and an in-database vector store, data doesn’t leave the database, helping increase data security.
Can I run HeatWave GenAI on AWS?
Yes, HeatWave GenAI is available natively on AWS, along with other HeatWave capabilities, including HeatWave MySQL, HeatWave Lakehouse, HeatWave AutoML, and HeatWave Autopilot.
Are multiple languages supported for generating embeddings?
Yes, embeddings can be generated for text data in 27 languages.
Which languages are supported for issuing prompts?
Prompts can be issued in English. Prompts issued in other languages, such as Spanish and German, can be translated to English.
Can I use vector search without the HeatWave cluster?
No, vector search is performed within the HeatWave cluster.
Which shapes should be used for HeatWave GenAI?
HeatWave runs on a MySQL node. We recommend a MySQL node with a MySQL.32 shape, plus HeatWave nodes using the HeatWave.512GB shape for a production environment. For development/testing, a smaller MySQL shape can be used. You can review supported MySQL shapes here. For HeatWave GenAI, the HeatWave.32GB shape isn’t supported.
What type of documents are supported for vector store creation?
PDF, text, PowerPoint, Word, and HTML are the supported formats.
What is the cost of HeatWave GenAI?
There is no additional cost beyond the cost of the HeatWave cluster for using HeatWave GenAI. You can invoke in-database LLMs and embedding models provided with HeatWave GenAI at no additional charge. You can also invoke external LLMs available via OCI Generative AI on OCI and Amazon Bedrock on AWS and will then be charged for those services.
Is my data used for training LLMs?
No, LLMs are pretrained models. Your data isn’t used for training LLMs.
Does HeatWave GenAI support fine-tuning?
No, in-database LLMs provided with HeatWave can’t be fine-tuned.
Can I bring my own LLMs or embedding models?
No, you can’t bring your own LLMs or embedding models. However, you can invoke the external LLMs or embedding models available via OCI Generative AI when running HeatWave GenAI on OCI and via Amazon Bedrock when running HeatWave GenAI on AWS.
What is the expected quality of results since the in-database LLMs are quantized?
Based on our testing, results are comparable to non-quantized LLMs for use cases that rely on HeatWave Vector Store. You can easily test the models to evaluate the performance and quality of results.
How often do I need to create embeddings?
You need to generate embeddings only once and they will be stored in HeatWave Vector Store. Changes to unstructured documents in object storage will automatically trigger updates to associated vector embeddings.
Can documents containing images be imported into HeatWave Vector Store?
Yes, optical character recognition support allows HeatWave Vector Store to convert scanned content saved as images into text data that can be analyzed, for example, to conduct similarity searches.