Vectors Generation
The generation of vectors needs to be used in conjunction with stores, which means that you need to configure stores before you can understand vectors.
Vectors are actually the result of embedding, which is the process of converting various types of data, such as text and images, into dense vector representations. This step is essential to facilitate efficient data processing and analysis within Casibase. With embedding, questions in chat and knowledge files in storage will be converted into vectors that will be used in the next step of knowledge search.
1. Automatic Embedding
Casibase automatically generates vectors when you upload files to a store. After configuring a store with storage, model, and embedding providers, simply upload your files through the file tree interface. The system immediately queues them for embedding and processes them in the background.
Each file progresses through states you can monitor in the Files page: Pending, Processing, Finished, or Error. The system calculates token counts during embedding and displays them alongside each file. If embedding fails for any file, the error details appear in the file list so you can quickly identify and fix issues like missing provider configurations or unsupported file formats.
When you delete files from storage, their vectors are automatically removed from the database. This keeps your knowledge base synchronized - you won't get outdated responses from files that no longer exist.
2. Refresh Vectors
The Refresh Vectors action is set as a button on each store data under the stores menu. After configuring stores with storage providers, navigate to the Stores page to view the file tree for the storage providers.
By clicking on the Refresh Vectors button for a particular store, it will generate the corresponding vectors for all the files in the file tree for that store by embedding them. The following figure shows the page and the operation.
When you refresh vectors, the system automatically removes all existing vectors for that store before generating new ones. This ensures your vector database stays synchronized with your current files - if you've deleted files from storage, their vectors will be removed as well. The refresh always creates a clean, up-to-date vector set based on your current file tree.
Re-embedding Individual Files
For targeted updates, use the "Refresh Vectors" button in the Files page next to any specific file. This regenerates vectors for just that file while leaving others untouched - useful when you've changed embedding settings or need to fix errors on specific documents without reprocessing your entire knowledge base.
3. View vectors
After that, we can view the specific vectors generated by that store in the vector menu.
We can see that the files in the stores from the previous step of refreshing vectors have been converted into vectors to display here.
The edit page of my vectors shows specific information such as the name of the store, the name of the embedding model, the name of the file in which the embedding was performed, the file size, the dimension, the vectors data, and so on.