SEMANTIC About Semantic Search
About Semantic Search
What is Semantic Search?
The term 'semantic' in semantic search refers to 'concept' or 'meaning'.
So, what is semantic search?
Semantic search is an information retrieval method where the search engine understands the 'meaning' of both the query and the target objects, much like a human would. This enables types of searches that were difficult to achieve with traditional keyword-based search methods.
- Text document search using text queries
- Similar image search using an image as the search source
- Image, video, and audio data search using text queries
- Search for documents in other languages using text queries in your own language
Moreover, what if the response speed of search results could be achieved without compromising on the speed of traditional keyword searches?
Semantic search can be considered a revolutionary technology that changes conventional wisdom.
Semantic search may be referred to by the following names, but they all represent the same technology. Here, we will use the term 'semantic search' for consistency.
・Vector Search
・Dense Vector Search
・Neural Search
Comparison with traditional keyword search
Comparison between Semantic Search and traditional Keyword Search:
In traditional keyword search, users need to input keywords present in the document to perform a search.
- A certain level of domain knowledge (specialized knowledge for each industry) is required to conduct a search.
- Even with domain knowledge, since there are multiple keywords with the same meaning, maintenance of a thesaurus is essential.
In Semantic Search, it is not necessary for the query to match keywords exactly.
- Even without domain knowledge, users can search by expressing their desired information in their own way (using their own level of expression).
- Searches can be conducted even if there is a time gap between the document collection period and the query period.
Traditional keyword search required the following techniques and maintenance to improve search performance, which are unnecessary in Semantic Search, leading to reduction in effort and cost:
- Various types of normalization such as character normalization.
- Differentiation and combination of morphological analysis and character N-gram.
- Countermeasures for keyword spelling variations and synonym definitions.
- Weighting of fields, etc.
Examples of Semantic Search Application:
By leveraging the characteristics of semantic search, it can be applied to the following business applications:
Searching FAQs and manuals
In searching FAQs and manuals, while the documents being searched are highly specialized, the searchers (such as product users, typically external individuals) are not necessarily experts. This often leads to vocabulary and knowledge gaps between the two parties.
Unlike traditional keyword searches where a query must match the keywords exactly, semantic search can retrieve results even if the keywords do not match exactly but have similar meanings. This enables users to resolve issues on their own and reduces inquiries to call centers/help desks.
Internal enterprise search, knowledge sharing, search at call centers/help desks
In scenes of internal enterprise search and knowledge sharing, the documents being searched are highly specialized, thus possessing strong 'meaning (semantic)'. Semantic search, which allows searching for items with similar meanings even without knowing specialized terminology, is well-suited for such scenarios.
Similar Product Search on E-commerce/Online Shopping Sites
By using semantic search on EC sites/online shopping platforms, it's possible to present other products similar to those specified during the search, based on text and image similarities. This can lead to increased opportunities for purchases.
Similar design and trademark searches, among others, involve searches beyond just text
Semantic search can be executed even when the search target or query is not text. Therefore, it is possible to build an application that searches for similarities to designs or trademarks that are about to be registered with the patent office, based on illustration image files on hand, and displays results in order of the 'meaning' conveyed by the images.
Inquiry based learning and research and development
Meaning-based associative search can be easily implemented with educational resources (dictionaries, encyclopedias, textbooks) and highly specialized papers. It is suitable for exploratory learning and discovery-oriented searches such as research and development (R&D).
Multimodal search
Semantic search allows searching based on the 'meaning' of both the query and the search target document, enabling seamless searching across different types of objects. For example, it's possible to search for similar images by specifying an image, search for images using text, or search for an English manual using Japanese.
Comparison with ChatGPT
ChatGPT is a generative AI chat tool that engages in real-time conversations with humans through text prompts and commands.
On the other hand, KandaSearch's search engine, which supports semantic search, is a tool for finding desired items from large amounts of data such as text, images, videos, and audio. The two serve entirely different purposes.
ChatGPT excels at inference without a correct answer. Therefore, users need to be cautious about whether ChatGPT's output is correct. In contrast, KandaSearch's semantic search retrieves results that are close to the 'meaning' of the specified query at the time of search, from a reliable set of documents. Therefore, there is no concern about 'hallucinations', making it suitable for business applications that involve direct interaction with external stakeholders such as customers.
Are you hesitating to introduce semantic search engines just because generative AI chats like ChatGPT are making waves on the internet? The video below explains clearly the situations where semantic search engines should be introduced instead of generative AI chats. Be sure to check it out.
AI Chat vs. Semantic Search Engines
Related Information
We also provide various information and data related to semantic search on other pages. (Including the above introduction)
- Frequently Asked Questions (FAQ)
- Document
- Semantic Search Demo Site
- YouTube