How to boost your search results with filter scoring

Headquartered in New York and Paris, Hugging Face is an open-source provider of machine learning technologies, allowing users to train, deploy, and share AI models using Hugging Face open-source libraries and the Hub.

**By partnering with Meilisearch Cloud, Hugging Face actively fulfills its visionary commitment to advancing the democratization of AI.**

![](https://unable-actionable-car.media.strapiapp.com/ezgif_com_video_to_gif_1_e697b647e1.gif)

> _“Today, Meilisearch is used by Hugging Face to power the discoverability of 300,000+ AI models, datasets, and demos. It’s really important for AI democratization because if the knowledge is out there, but you can’t access it - what’s the point?” -_ Mishig Davaadorj, software engineer at Hugging Face

Challenge
---------

On its platform, Hugging Face enables users to upload their AI models, datasets, and demos. With a vibrant community of users constantly engaging with the [Hugging Face Hub](https://hugging-face.cn) - a platform for sharing and searching for Machine Learning artifacts - the question of discoverability takes center stage.

The Hugging Face Hub hosts over 220,000 AI [models](https://hugging-face.cn/models) catering to a variety of [machine-learning tasks](https://hugging-face.cn/tasks), all neatly stored in repositories. These model repositories are designed to make the exploration and utilization of models as seamless as possible. Each AI model in the Hugging Face repository is accompanied by a model card, a project file containing valuable metadata, which plays a crucial role in enhancing discoverability, reproducibility, and sharing. Model cards also provide important information about AI models’ biases and limitations, model descriptions, and training guidelines, and serve as comprehensive guides for users seeking models on the Hub or uploading their own.

Before the introduction of Meilsearch, Hugging Face relied on a simple filtering and keyword search solution. However, it became evident that there was a growing demand for a more flexible and typo-tolerant full-text search solution. This solution needed to ensure higher default relevancy for every search query and leverage the additional attributes and metadata stored within model cards.

Why Hugging Face chose Meilisearch Cloud
----------------------------------------

Before integrating Meilisearch into its ML model repository, Hugging Face had already been utilizing Meilisearch's free-of-charge open-source solution as a search engine for its [open-source libraries](https://hugging-face.cn/docs)’ documentation, including transformers & diffusers. This documentation comprised approximately 500 pages and had been in use for over a year. Because the team had already gained positive experiences from implementing and operating Meilisearch, when the need for enhanced discoverability in model card searches arose, there was no requirement for additional testing or proof of concept.  

**1\. Customizability of the ranking rules**

During the evaluation process, the Hugging Face team also considered Mongo Atlas Search. However, they weren't satisfied with the customizations available for ranking. Several factors were taken into account in order to enhance model card discoverability, such as project names, descriptions, and the number of likes or downloads on each card. Meilisearch has demonstrated greater flexibility and adaptability in accommodating these search criteria.

**2\. Transitioning to Meilisearh Cloud for its ease of use**

Having achieved successful implementation of Meilisearch in its documentation, the Hugging Face team smoothly transitioned to incorporating Meilisearch Cloud into their model cards repository, benefiting from out-of-the-box relevancy.

**3\. Quality of support and infrastructure outsourcing**

The Hugging Face team chose Meilisearch Cloud, in part due to its dedicated support capabilities. Given that achieving highly relevant search results within the repositories is of utmost importance to Hugging Face, the decision to opt for the Cloud version was further influenced by entrusting the infrastructure to the expertise of Meilisearch Cloud specialists. Outsourcing the infrastructure allows the Hugging Face team to enhance their development speed.

Implementation
--------------

With Meilisearch already implemented for Hugging Face documentation, the expansion of the search solution to other use cases was smooth, once the ranking and internal rules were in place.

Today, the Meilisearch engine powers the discovery of 220,000 model cards, 38,000 datasets, and 60,000 demos in the Hugging Face repository. The keyword filtering mechanism has already been implemented on the Hugging Face homepage for users who are already familiar with specific model names.

However, with the implementation of Meilisearch, users are given the option to perform full-text searches as well. This search functionality operates not only on the model names and IDs but also includes the entire content of model cards.

The additional button to “Try full-text search” was implemented in order to minimize changes in the search behavior, where users are free to choose the search experience that most fits their preference.

![](https://unable-actionable-car.media.strapiapp.com/ezgif_com_video_to_gif_bce9ae142e.gif)

Vision
------

With the ongoing evolution of AI, Hugging Face anticipates a shift in search behavior and the methods people use to find the projects they need. Hugging Face is looking forward to incorporating Semantic Search into its documentation and model card searches. Currently, the platform is actively testing Meilisearch [VectorDB](/blog/langchain-semantic-search-tutorial/ as one of the potential solutions.

Hugging Face also anticipates gaining more detailed usage insights, encompassing information about users' daily search patterns and the level of activity certain model card projects receive over time.

Hugging Face facilitates AI accessibility with Meilisearch

In this era of information overload, the efficiency of documentation is determined not just by its quality but also by how easily users can find what they're searching for. Enhancing your documentation with advanced search capabilities can significantly elevate user experience.

Meilisearch Cloud's **Documentation crawler** offers just that, bridging the gap between vast content and user-friendly searchability.

What is a crawler?
------------------

A web crawler, sometimes known as a 'spider' or 'bot', is a digital tool that roams the vast expanse of the internet, indexing and cataloging content for search engines. This ensures that the most relevant pieces of information are presented to users during their searches.

What is Meilisearch’s Documentation crawler?
--------------------------------------------

Meilisearch's Documentation crawler is not just a traditional crawler; it also functions as an efficient scraper. While it extracts vital data from web pages, it simultaneously honors the inherent structure of the documentation, ensuring precise and structured indexing.

This system allows users to integrate [search functionality](https://meilisearch.com.cn/docs/learn/what_is_meilisearch/overview#cloud-release&utm_source=blog&utm_medium=crawler-release#features) into their documentation, ensuring that their audiences can swiftly locate the information they need.

How to use the documentation crawler?
-------------------------------------

Using the Documentation Crawler is a breeze; all it requires is a Meilisearch Cloud project and your existing documentation's URL.

### Setting up your project

First of all, you need a running project in [Meilisearch Cloud](https://meilisearch.com.cn/cloud#cloud-release&utm_source=blog&utm_medium=crawler-release). Once you have created a project, navigate to your project's index list and create a new index.

![Step 1 of Create Index form: Enter the index name.](https://unable-actionable-car.media.strapiapp.com/create_an_index_57ca6f70f6.png)

### Crawling your content

When prompted to import your documents, choose 'Documentation Crawler.'

![Step 2 of Create Index form: Import documents via crawler or file upload.](https://unable-actionable-car.media.strapiapp.com/select_docs_crawler_e7c7fcfa52.png)

Then, input or paste your documentation's URL and click on ‘Next’.

![Step 2 of Create Index form: Import documents using a crawler by pasting the website URL](https://unable-actionable-car.media.strapiapp.com/paste_url_2f1c23072e.png)

Click on 'Run' to start crawling your documentation content and push it into a Meilisearch instance. This process may take several minutes.

![Setup interface for the Crawler with a 'Run' button.](https://unable-actionable-car.media.strapiapp.com/click_on_run_eeecb9b7eb.png)

### Post-crawling verification

Once the status updates to 'ready', you can return to your project's overview and access the 'search preview'. Click on the project’s URL or select the 'Preview search' option from the top menu.

![Project overview](https://unable-actionable-car.media.strapiapp.com/go_to_search_preview_255bd45a28.png)

Perform a few searches to ensure that your documentation has been indexed correctly. This step is especially useful for understanding the schema of your documents.

![Search preview displaying results for the query 'releva' with highlighted matches.](https://lh4.googleusercontent.com/BJ9jNV_RA4q42ONIw54vQXyCSEu41Q_6xKY5W6sMxK02XqufBV8iedNRSZZl6q2NtqKaZEjpOVDLEC7TFQmrj7nci5-FT87gFs8UnqfDa0M1PRrm-NLUD6odr2a4m1bbSIGgwlqxtyIqmEYQ3sI4YIc)

Adding a search bar to your documentation
-----------------------------------------

Once you've successfully indexed your documentation using Meilisearch's Documentation Crawler, the next step to enhancing user experience is integrating a search bar into your documentation.

Return to your project and navigate to your index settings. From there, select the 'tools' tab and then choose 'crawler'. This will reveal the crawler settings and associated resources.

![Setup interface for the Crawler with the 'UI starter kit: Doscsearch' option.](https://lh3.googleusercontent.com/woHfyr-bTDS7fbDTRnpq-GR61IwVq6fvfmCGoRDkiVORf7x6wSjSsXUgXRnmepr7YQ_E3HZHRBW3FScZSE1vS8GIJwvGs6GfUCIh0eJckapWsJViZyaZ-Bqa6Vdj_2EpXgHG4WLQTgUUN9ubYfXCQcA)

The [UI starter kit](https://github.com/tauri-apps/meilisearch-docsearch#cloud-release&utm_source=blog&utm_medium=crawler-release) helps you add a functional search bar to your documentation with just a few lines of code.

You are now fully equipped to add a seamless search experience to your documentation!

\_\_\_\_\_

Stay in the loop by subscribing to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&utm_campaign=cloud-release&utm_source=blog&utm_medium=crawler-release). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/#cloud-release&utm_source=blog&utm_medium=crawler-release) and come participate in our [Product Discussions](https://github.com/meilisearch/product/discussions/#cloud-release&utm_source=blog&utm_medium=crawler-release).

For anything else, join our developer community on [Discord](https://discord.gg/meilisearch#cloud-release&utm_source=blog&utm_medium=crawler-release).

Introducing Documentation Crawler — Meilisearch Cloud

This feature enhances your dashboard experience by offering a comprehensive view of Meilisearch's [asynchronous operations](https://meilisearch.com.cn/docs/learn/async/asynchronous_operations#cloud-release&utm_source=blog&utm_medium=tasks-view-release).

**Core features**
-----------------

*   **Detailed task list**: the Tasks View showcases a list of your tasks, detailing their UID, status, type, index UID, duration, and enqueued date. Moreover, you can sort the list based on these parameters.
*   **JSON panel**: alongside the list, there's a panel displaying the selected task's JSON, which can easily be copied.
*   **Filtering**: you can sift through tasks based on their status, type, and date.
*   **Refresh option**: whenever a task is enqueued, the `Show new tasks` button lets you decide when to view the latest task without disrupting your current view.
*   **Customization**: customize the view further by toggling the columns' visibility according to your preferences.

![Tasks view screenshot](https://storage.noticeable.io/projects/OQ6S6bocohVRzziV6Tlc/publications/BLDwc6IdPRkf5ZsdEOoa/01h821n7ed9mje0m5y9dagk9rt-image.png)

**Compatibility notes**
-----------------------

*   **Version requirements**: this feature is exclusively available for projects that run on Meilisearch v1.0.0 or higher.
*   **Latest version perks**: for those using the [Meilisearch v1.3.0](/blog/v1-3-release/#cloud-release&utm_source=blog&utm_medium=tasks-view-release) or above, the Tasks View also displays the count of fetched tasks.

We hope this addition enhances your project management efficiency. Enjoy exploring!

Introducing Tasks View — Meilisearch Cloud

At Meilisearch, we have long maintained our commitment to simplifying and streamlining the search process by prioritizing intuitive, user-friendly, and human-like search experiences.

In today's search technology landscape, _[keyword search](https://meilisearch.com.cn/docs/learn/what_is_meilisearch/overview#vector-search&utm_source=blog&utm_medium=launch#features)_ continues to play a pivotal role, swiftly locating content via specific keywords. Leveraging advanced algorithms and robust indexing, it ensures users find relevant information across vast data volumes.

However, traditional keyword approaches have inherent limitations in understanding user intent and at times can be overly predictable and simplistic. This is why we are introducing Meilisearch's _[vector search](https://meilisearch.com.cn/docs/learn/experimental/vector_search#vector-search&utm_source=blog&utm_medium=launch),_ which aims to challenge and enhance the familiar keyword search paradigm.

Vector search for AI-enhanced future
------------------------------------

The market-wide introduction of _vector search_ marks the arrival of a new era for search technology, with countless versatile applications. In _e-commerce_, it offers product recommendations based on similarities and multi-modal search capabilities such as image, audio, and video search. The fusion of vector search with conversational AI technologies opens doors for intuitive Q&A applications. Adding user data like geolocation and search history further expands its potential.

New search technology enhances relevancy alongside traditional keywords with semantic understanding going beyond simple keyword matching. It is capable of comprehending intent, returning results like **\[gloves\]** from a **\[warm clothes\]** query. These applications rely on _vector embeddings_—a type of data representation with semantic information that enables AI to gain understanding and maintain long-term memory. Stored in _vector databases_, embeddings represent different data dimensions, essential for understanding patterns and relationships. AI-powered vector database then leverages this information when matching search queries to answers.

Democratizing search
--------------------

In a data-rich world, effective content discovery is not a luxury - it’s crucial. Meilisearch's vision balances ambition with accessibility, aiming to make cutting-edge search technology inclusive. Key Meilisearch partners, such as [Hugging Face](/blog/hugging-face-democratizes-ai/ are already actively testing Meilisearch’s new vector database and storage capabilities.

> **Quentin de Quelen, Meilisearch’s CEO and Co-founder, said:** _“Meilisearch's hybrid search will democratize advanced search technology, focusing on simplicity and open-source customizability.”_

Building the future of search with vector search
------------------------------------------------

This release brings semantic and hybrid search capabilities to enhance applications for image, video, and audio search, product recommendations for e-commerce, precise personalization, and intuitive Q&A interactions.

Here are a few examples of potential applications made possible with the introduction of vector search:

*   **Semantic understanding:** Enable your search engine to act as a human-like assistant, delivering natural, context-aware results in a more conversational manner
*   **Multiformat capabilities:** Cover images, audio, and video, ensuring versatility.
*   **Personalization**: Matches preferences for tailored results, with a deeper understanding of user behavior and preferences,
*   **Precise Recommendations:** Enhance the discovery experience by suggesting similar items that align with users' genuine interests.
*   **Improved accuracy:** Improve relevancy and search accuracy through a deep understanding of each search query's meaning

Search evolution culminates in a hybrid search
----------------------------------------------

Meilisearch's journey leads to _hybrid search_, which combines traditional full-text efficiency with the innovation of semantic search and the adaptability of vector database. The hybrid approach enables search solutions to cater to a broad spectrum of use cases driven by customer-generated data sources. By integrating models from AI solution providers like OpenAI, Cohere AI, or Hugging Face, Meilisearch empowers users to calibrate _vector embeddings_ and infuse their business logic into the search engine.

Getting started
---------------

<Callout>
  For more guidance on how to get started, explore our [step-by-step tutorial on building semantic search with LangChain](/blog/langchain-semantic-search-tutorial/#vector-search&utm_source=blog&utm_medium=launch)
</Callout>
```json
from langchain.vectorstores import Meilisearch
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.document_loaders import JSONLoader

# Load documents
loader = JSONLoader(
    file_path="./movies.json",
    jq_schema=".[] | {id: .id, title: .title, overview: .overview}",
    text_content=False,
)
documents = loader.load()

# Index documents
embeddings = OpenAIEmbeddings()
vector_store = Meilisearch.from_documents(documents=documents, embedding=embeddings)

# Search
query = "superhero fighting villains in a city corrupted by crime"
results = vector_store.similarity_search(
    query=query,
    k=3,
)

```

If you want to use Meilisearch vector search capabilities without LangChain, choose your favorite language and check out the [relevant SDK documentation here](https://meilisearch.com.cn/docs/learn/what_is_meilisearch/sdks#vector-search&utm_source=blog&utm_medium=launch).

Want the quality of open-source without the headache of managing your servers and service? [Sign up for a free trial](https://meilisearch.com.cn/cloud#vector-search&utm_source=blog&utm_medium=launch) with Meilisearch Cloud today.

Introducing Meilisearch's vector search and vector database to navigate the future of search

[HitPay](https://hitpayapp.com/) is a one-stop payment fintech platform from Southeast Asia on a mission to enable small and medium businesses to accept payments easily.

**HitPay chose [Meilisearch Cloud](https://meilisearch.com.cn/cloud) to power its customers' search securely and reliably, all while maintaining a lean engineering team and staying focused on its core offering.**

<Callout>
  ___"Using Meilisearch's open-source version for proof of concept, a single developer was able to quickly implement the complete flow, including UI and backend changes. This validated our use case and confirmed that Meilisearch was the right solution for our requirements. After that, transitioning to the Cloud was effortless."__ - Nitin Muthyala, Co-Founder and CTO at HitPay_
</Callout>

Challenge
---------

As an small and medium business-focused company, HitPay strives to simplify the learning curve for small business owners who lack time to learn and adapt complex payment tools.

Merchants utilizing HitPay often have sizable catalogs and a broad customer base, requiring efficient search functionality for e-commerce and in-store purchases. With search functionality not being within HitPay's core suite of solutions, it was decided to integrate a third-party tool. While searching for a suitable solution, the HitPay team came across Meilisearch.

Why Hit Pay chose Meilisearch
-----------------------------

### 1\. Easy to integrate

Historically, HitPay has been able to maintain a lean engineering team by actively seeking out nimble and adaptive tools to streamline its developers' work. When searching for a search solution to integrate into their existing back-end system built on PHP with the Laravel framework, HitPay prioritized simplicity and compatibility. They discovered that Meilisearch offered a driver for [Laravel Scout](https://laravel.net.cn/docs/master/scout) that perfectly matched their requirements. Additionally, HitPay’s team was well-acquainted with the open-source space, and adopting an open-source search solution seemed like a logical initial step.

Nitin Muthyala, Co-Founder and CTO of HitPay, emphasized: _'We always begin with the open-source version of a solution, which has proven successful for us thus far. After testing Meilisearch's open-source solution on the internal use case, we realized how powerful it is.”_

### 2\. Developer experience

During the initial evaluation, Meilisearch was integrated into HitPay's internal dashboard, which contains a database of all its customers. The dashboard serves HitPay’s own account managers who need to search through their accounts by business owner names, business names, and affiliated emails. This presented challenges with long and repetitive names that included variations in spelling. With the efforts of a single developer, supported by helpful documentation and Meilisearch’s [out-of-the-box relevancy](https://meilisearch.com.cn/docs/learn/core_concepts/relevancy), the pilot project didn’t require many tweaks and was successfully completed within a couple of days.

Implementation
--------------

After successfully testing the open-source version of Meilisearch, HitPay made the decision to switch to the Meilisearch Cloud. A highly relevant and reliable search solution was crucial for HitPay's customers as it directly impacted their sales. To avoid the additional burden of constantly monitoring the production environment, HitPay transitioned to the Cloud version. This decision provided peace of mind, knowing that the business-critical search activity would be in safe hands at all times.

Nitin Muthyala, Co-Founder and CTO of HitPay, explained: _"Search is a fundamental piece in our production process, and HitPay couldn't afford for it to ever go down. So we opted for Meilisearch Cloud."_

During the transition, other competitors were briefly considered but eliminated early on. Meilisearch Cloud stood out as the ideal choice due to its ease of integration and the ability to check all the boxes right from the start.

Since the implementation of Meilisearch, HitPay's customers have especially appreciated the ability to search for amounts within an order, typo recognition, and the **increase in search API response speed by at least 50%** for free-text searches.

![](https://unable-actionable-car.media.strapiapp.com/Transactions_Search_gif_d6e3b9c704.jpg)

Transactions search

Results
-------

Today, Meilisearch is a valuable asset for HitPay’s customers and is integrated into two of HitPay's commerce products. The first is an **[online eCommerce shop](https://www.hitpayapp.com/online-store)**, where businesses can create their own storefronts. The second is **[HitPay's POS](https://www.hitpayapp.com/pos-software)** (point of sale), the product that allows businesses to search, catalog and **sync products across online and offline sales channels, and manage unlimited locations** for brick-and-mortar stores, where sales assistants need to **locate specific products** quickly to ensure the overall customer experience.

![](https://unable-actionable-car.media.strapiapp.com/Products_Search_gif_9639f112fe.jpg)

Meilisearch in HitPay's Point of Sale

To always keep the search relevant, HitPay offers a feature where merchants can pin or favorite specific products, which are then prioritized in search results. For customers who opt-in, HitPay sets up a workflow to automatically update the index with the most recent SKUs (Stock-keeping units) or favorited products, ensuring highly relevant search results that are always up-to-date.

Vision
------

In the future, HitPay aims to provide customizable user-facing analytics that caters to their customer's specific needs. This is particularly important for small and medium businesses, whose analytics requirements vary due to seasonal business patterns. HitPay envisions offering customization options for running analytics on any software, allowing customers to identify their most sought-after products and analyze unsuccessful searches. This aligns with Meilisearch’s product vision and HitPay's mission of powering product search for online shops and simplifying catalog searches for sales assistants in physical stores.

HitPay streamlines customers’ search capabilities while staying laser-focused on its core offering

Hey there, here's your monthly recap of everything Meilisearch. Let's dive in!

**🚀 Releasing Meilisearch v1.3**
---------------------------------

[![What's new? v1.3](https://unable-actionable-car.media.strapiapp.com/75_454d2f4f6e.png)](/blog/v1-3-release/)

We released v1.3 which introduces **vector search**, **ranking score details**, **search for facet values**, and **dynamic searchable fields**.

[Read the release blog](/blog/v1-3-release/)*🔢 Vector search is coming to Meilisearch**
---------------------------------------------

[![Empowering search through AI: vector storage is coming to Meilisearch](https://unable-actionable-car.media.strapiapp.com/73_1_1_65f8c54f7d.png)](/blog/vector-search-announcement/)

Vector search is our first step towards **semantic search**. It is an important step toward our long-term goal of providing **hybrid search**: combining the benefits of full-text and semantic search.

[Read the announcement](/blog/vector-search-announcement/)*🤝 Qogita customer story**
----------------------------

[![Qogita success story: simplifying B2B trade with Meilisearch Cloud](https://unable-actionable-car.media.strapiapp.com/78_1_1_ea6d441573.png)](/blog/qogita-simplifies-b2b-trade/)

With **Meilisearch Cloud**, Qogita can ensure high seller exposure and increased relevancy, resulting in an excellent buyer-seller interaction.

[Read the success story](/blog/qogita-simplifies-b2b-trade/)*📰 In the news**
------------------

  
**🦜🔗** [Meilisearch LangChain integration guide](https://python.langchain.ac.cn/docs/integrations/vectorstores/meilisearch)

Learn how to integrate the Meilisearch vector store in LangChain.

🏀 [From ranking to scoring](/blog/from-ranking-to-scoring/)ead about our engineers journey to add relevancy scores to search results.

🎮 [Meilisearch SaaS demo](https://saas.meilisearch.com/)

We’ve released a new demo feature using **multi-search**, **multi-tenancy**, **filtering**, and **sorting** using Laravel.

⚡️ [Blazing fast async data loading in Meilisearch using Python](https://thedataquarry.com/posts/meilisearch-async/)

Read this community article how to use Meilisearch with Pydantic.

🍿 [Meilisearch, the open-source search engine with a UX focus](https://www.youtube.com/watch?v=ncnBRrF-e2g)

Watch this talk explaining how to use Meilisearch (in French, subtitles available.)

**🤗 Made with Meilisearch**
----------------------------

Our featured project of the month is [pbvault.com](https://www.pbvault.com/). Paintball Vault is a marketplace where you can easily buy, sell, and trade both your used and new paintball gear.

The main navigation search bar and all the product grids throughout the site and app use Meilisearch to make browsing intuitive and easy.

[![Screenshot of pbvault.com](https://unable-actionable-car.media.strapiapp.com/www_pbvault_com_3cda884404.png)](pbvault.com)

Buy, sell, and trade paintball gear on pbvault.com

_We’re always looking to promote new projects from the community. Post yours in the_ [_#built-with-meilisearch_](https://discord.meilisearch.com/) _Discord channel!_

**🏡 Life at Meili**
--------------------

Remote work is the future of work: find out how we aim to improve inclusivity and work-life balance through remote-working.

[![A forest](https://unable-actionable-car.media.strapiapp.com/remote_work_is_the_future_of_work_cover_2fb7ead2fa.jpg)](/blog/remote-work-is-the-future-of-work/)

[Read the blog post](/blog/remote-work-is-the-future-of-work/)*🦸‍♀️ We’re hiring!**
-----------------------

See our open positions:

*   [Head of Finance](https://jobs.lever.co/meili/14970d91-fb2f-4130-b568-97872eb62b31)
*   [Founding Account Executive](https://jobs.lever.co/meili/92e2ebdc-3023-4fbe-8dd2-c3cfc9ecd7bf)

* * *

Get these monthly updates straight to your inbox by subscribing to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e).

For more things Meilisearch, join our developers community on [Discord](https://discord.gg/meilisearch/#blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/#blog.meilisearch.com) and participating in [product discussions](https://github.com/meilisearch/product/discussions#blog.meilisearch.com).

Until next time.

Meilisearch August updates

In this guide, we'll walk you through implementing promoted search results with Meilisearch. Our goal is to search results prioritizing specific documents when certain keywords match a user's query. These boosted documents should be returned at the top of the search results.

<Callout>
  This guide explains to implement promoted documents on the backend. For a frontend-first implementation, see [implementing promoted search results with React InstantSearch](/blog/promoted-search-results-with-react-instantsearch/)
</Callout>

Overview
--------

Here's a simplified breakdown of how to implement document boosting using a second index for “pinned documents” and the [multi-search feature](https://meilisearch.com.cn/docs/reference/api/multi_search#perform-a-multi-search):

1.  **Create Indexes:** Set up two indexes: one for regular search and one for boosted results. The boosted index will have a special attribute, `keywords`, to trigger the boosting.
2.  **Populate the 'games' Index:** Populate a `games` index with your dataset using the provided JSON file. This index will serve as the source for our boosted documents.
3.  **Configure the 'pinned\_games' Index:** Configure the `pinned_games` index to display attributes without revealing keywords. Adjust the searchable and displayed attributes accordingly.
4.  **Boost Documents:** Identify documents you want to boost and assign relevant keywords to them. For instance, you can assign the keywords `fps` and `shooter` to the game `Counter-Strike`.
5.  **Implement the Multi-Search:** Utilize Meilisearch's multi-search feature to perform a search query across both the regular and boosted indexes. This way, boosted documents matching keywords will appear first.
6.  **Display Results:** Present the search results in a user-friendly format, highlighting boosted documents with a visual indicator.

Implementation
--------------

### Installation

Before diving in, make sure you have Meilisearch up and running. If you haven't installed it yet, follow these steps:

1.  Launch a Meilisearch instance — you can run [Meilisearch in local](https://meilisearch.com.cn/docs/learn/getting_started/installation) or via [Meilisearch Cloud](https://meilisearch.com.cn/cloud).
2.  Ensure you have your favorite [language SDK](https://meilisearch.com.cn/docs/learn/what_is_meilisearch/sdks) (or framework integration) installed.

<Callout>
  This guide uses the [Python SDK](https://python-sdk.meilisearch.com/), but it works the same with any other Meilisearch integration. 🎉
</Callout>

### Initializing indexes

For our example, we'll work with a dataset of Steam games. You can adapt this process to your own data:

1.  Download the `steam-games.json` and `settings.json` files for our [Steam games dataset](https://meilisearch.notion.site/Steam-6e6141755a854909b31ba8c8e9e74853)
2.  Load the dataset in your Meilisearch instance by [adding documents](https://meilisearch.com.cn/docs/reference/api/documents#add-or-replace-documents) from the `steam-games.json` file.

#### `games` index

```python
import meilisearch
import json
from typing import Callable

client = meilisearch.Client(url="https://:7700")
games = client.index("games")

# helper to wait for Meilisearch tasks
def wait_with_progress(client: meilisearch.Client, task_uid: int):
    while True:
        try:
            client.wait_for_task(task_uid, timeout_in_ms=1000)
            break
        except meilisearch.errors.MeilisearchTimeoutError:
            print(".", end="")
    task = client.get_task(task_uid)
    print(f" {task.status}")
    if task.error is not None:
        print(f"{task.error}")
            
print("Adding settings...", end="")
with open("settings.json") as settings_file:
    settings = json.load(settings_file)
    task = games.update_settings(settings)
    wait_with_progress(client, task.task_uid)


with open("steam-games.json") as documents_file:
    documents = json.load(documents_file)
    task = games.add_documents_json(documents)
    print("Adding documents...", end="")
    wait_with_progress(client, task.task_uid)
```

#### `pinned_games` index

This index will contain the promoted documents. The settings of the `pinned_games` index are the same as the `games` index, with the following differences:

*   the only `searchableAttributes` is the `keywords` attribute containing the words that trigger pinning that document.
*   the `displayedAttributes` are all the attributes of the documents, except for `keywords` (we don't want to show the keywords to end-users)

```python
pinned = client.index("pinned_games")

print("Adding settings...", end="")
with open("settings.json") as settings_file:
    settings = json.load(settings_file)
    settings["searchableAttributes"] = ["keywords"]
    # all but "keywords"
    settings["displayedAttributes"] = ["name", "description", "id", "price", "image", "releaseDate", "recommendationCount", "platforms", "players", "genres", "misc"]
    task = pinned.update_settings(settings)
    # see `wait_with_progress` implementation in previous code sample
    wait_with_progress(client, task.task_uid) 
```

### Updating the promoted documents index

We'll now populate the index with documents from the `games` index that we want to promote.

As an example, let's say we want to pin the game `"Counter-Strike"` to the `"fps"` and `"first", "person", "shooter"` keywords.

```python
counter_strike = games.get_document(document_id=10)
counter_strike.keywords = ["fps", "first", "person", "shooter"]

print("Adding pinned document...", end="")
task = pinned.add_documents(dict(counter_strike))
wait_with_progress(client, task.task_uid)
```

### Customizing search results

Now, let’s create a function to return the search results __with__ the pinned documents.

```python
from copy import deepcopy
from typing import Any, Dict, List
from dataclasses import dataclass

@dataclass
class SearchResults:
    pinned: List[Dict[str, Any]]
    regular: List[Dict[str, Any]]

def search_with_pinned(client: meilisearch.Client, query: Dict[str, Any]) -> SearchResults:
    pinned_query = deepcopy(query)
    pinned_query["indexUid"] = "pinned_games"
    regular_query = deepcopy(query)
    regular_query["indexUid"] = "games"
    results = client.multi_search([pinned_query, regular_query])
    # fetch the limit that was passed to each query so that we can respect that value when getting the results from each source
    limit = results["results"][0]["limit"]
    # fetch as many results from the pinned source as possible
    pinned_results = results["results"][0]["hits"]
    # only fetch results from the regular source up to limit
    regular_results = results["results"][1]["hits"][:(limit-len(pinned_results))]
    return SearchResults(pinned=pinned_results, regular=regular_results)
```

We can use this function to retrieve our search results with promoted documents:

```python
results = search_with_pinned(client, {"q": "first person shoot", "attributesToRetrieve": ["name"]})
```

The `results` object should look like:

```json
SearchResults(pinned=[{'name': 'Counter-Strike'}], regular=[{'name': 'Rogue Shooter: The FPS Roguelike'}, {'name': 'Rocket Shooter'}, {'name': 'Masked Shooters 2'}, {'name': 'Alpha Decay'}, {'name': 'Red Trigger'}, {'name': 'RAGE'}, {'name': 'BRINK'}, {'name': 'Voice of Pripyat'}, {'name': 'HAWKEN'}, {'name': 'Ziggurat'}, {'name': 'Dirty Bomb'}, {'name': 'Gunscape'}, {'name': 'Descent: Underground'}, {'name': 'Putrefaction'}, {'name': 'Killing Room'}, {'name': 'Hard Reset Redux'}, {'name': 'Bunny Hop League'}, {'name': 'Kimulator : Fight for your destiny'}, {'name': 'Intrude'}])
```

Tada 🎉 You now have a search results object that contains two arrays: the promoted results and the “regular” ones.

<Callout>
  Got stuck? Don’t hesitate to ask for help in our [Discord community](https://discord.meilisearch.com/).
</Callout>

Going further
-------------

This tutorial explored one approach for implementing promoted results. An alternative technique would be implementing documents pinning in the frontend; take a look at our [React implementation guide](/blog/promoted-search-results-with-react-instantsearch/)his different approach has the benefits of being [compatible with InstantSearch](https://github.com/meilisearch/meilisearch-js-plugins/tree/main/packages/instant-meilisearch).

Both techniques can achieve similar results. We also plan to integrate [promoted documents](https://portal.productboard.com/meilisearch/1-meilisearch/c/98-promoting-documents?utm_medium=social&utm_source=portal_share) in the Meilisearch engine. Give your feedback on the previous link to help us prioritize it.

For more things Meilisearch, you can subscribe to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e). You can learn more about our product by checking out the [roadmap](https://roadmap.meilisearch.com/) and participating in our [product discussions](https://github.com/meilisearch/product/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/).

Cheers!

How to implement document boosting with a second index

## What is bucket sort?

Bucket sort is a sorting algorithm that distributes elements from an array into several buckets. Once divided, each bucket is sorted individually, either using a different sorting algorithm or by applying the bucket sort method itself.

## What is recursive bucket sorting?

**Recursive bucket sort** is a specific application of the bucket sort algorithm, where the bucket sort algorithm is used to sort the individual buckets. It is named "recursive" because the same method is used repeatedly to sort sub-buckets until the entire array is sorted.

## What is bucket sorting used for?

Bucket sort is used for efficiently organizing data. By distributing the data into different buckets, it makes the later stages of sorting more efficient. **It also allows to use different sorting algorithms within individual buckets**. This flexibility allows for potential optimizations and makes bucket sort adaptable to various scenarios and data distributions.

## What is the best and worst-case of bucket sort?

Bucket sort's performance varies greatly, with the best and worst cases depending on factors such as data distribution, size, and the choice of sorting algorithm used within individual buckets.

### The best case of bucket sort

The best case for bucket sort happens when the data is spread evenly across the buckets. This uniform distribution minimizes the complexity of sorting each bucket, leading to an overall time complexity of *O(n+k)*, where *n* represents the number of elements and *k* refers to the number of buckets.

### The worst case of bucket sort

The worst case for bucket sort arises when all the elements end up in a single bucket, losing the advantage of distributing them across multiple buckets. This scenario leads to a time complexity determined by the sorting algorithm used for that single bucket. It can result in a worst-case time complexity of *O(n2)*, depending on the algorithm. 

## How does a bucket sort algorithm work?

To better illustrate how it works, let's take the example of [Meilisearch](https://meilisearch.com.cn/). Meilisearch is a  search engine that uses bucket sort to sort search results on a set of consecutive rules called [ranking rules](https://meilisearch.com.cn/docs/learn/core_concepts/relevancy#built-in-rules)**.** 

For example, the [\`words\` ranking rule](https://meilisearch.com.cn/docs/learn/core_concepts/relevancy#1-words) sorts documents based on the number of words from the query found in the document.

Given the query “Badman dark knight returns”, the \`words\` rule will sort the returned documents into 4 buckets. These range from documents that contain all the words (possibly with a typo) to those that only contain the word “Badman”.

If \`words\` is the last ranking rule, or if the buckets only contain one document, then Meilisearch returns the buckets from the “best” (matching all words) to the “worst” (matching only one word). Documents matching 0 words are never returned as they have 0 relevancy to the search query. 

![buckets of documents after applying the `words` ranking rule on the query `Badman dark knight returns`](https://unable-actionable-car.media.strapiapp.com/image2_dd7457e68e.png)
_Diagram illustrating bucket sorting using the `words` ranking rule._

Sorting documents using only a single ranking rule would force Meilisearch to either have a very complicated ranking rule or to perform a simplistic ranking. Meilisearch uses multiple ranking rules sequentially. 

If a bucket from the first ranking rule contains multiple documents, the second ranking rule is used on that bucket to “break the tie” between documents. This technique could qualify as “recursive bucket sorting”. The sorting finishes when all the “innermost” buckets contain a single document, or after applying the last ranking rule.

## How does a recursive bucket sort algorithm work?

To illustrate recursive bucket sorting, let’s imagine that we now added the [\`typo\` ranking rule](https://meilisearch.com.cn/docs/learn/core_concepts/relevancy#2-typo) after the [\`words\` ranking rule](https://meilisearch.com.cn/docs/learn/core_concepts/relevancy#1-words) from the previous example. The \`typo\` ranking rule differentiates between documents that match the query directly and documents that would match if we were correcting one or two typos in the query. The former are better ranked than the latter.

Continuing with our sample query “Badman dark knight returns”, the \`typo\` ranking rule helps us further differentiate between documents from the last bucket. Note that it has no effect on the three other buckets, because they only contain documents such that the query has the typo “Badman \-\> Batman”. 

![buckets of documents after applying the `words` and `typo` ranking rules on the query `Badman dark knight returns`](https://unable-actionable-car.media.strapiapp.com/image1_04ad01e3b3.png)
_Diagram illustrating the application of recursive bucket sorting using the `words` ranking rule followed by the `typo` ranking rule._

By applying recursive bucket sorting to our query on the [sample movies.json dataset](https://www.notion.so/meilisearch/Movies-Demo-97ea014ca2034e64b8497425b972aedb#blog.meilisearch.com), Meilisearch returns the ranking below. For simplicity, we have configured the dataset so that only \`title\` is a searchable attribute, which makes the results easier to grasp. 

| Rank | Query words | Typo correction | Resulting documents |
| --- | --- | --- | --- |
| 1	| "Badman dark knight returns"     	| "badman" → "batman" | Batman: The Dark Knight Returns, Part 1 <br/> Batman: The Dark Knight Returns, Part 2 |
| 2	| "Badman dark knight"             	| "badman" → "batman" | Batman Unmasked: The Psychology of the Dark Knight\<br/\>Legends of the Dark Knight: The History of Batman |
| 3	| "Badman"                         	| No typo         	| Angel and the Badman                                  	|
| 4	| "Badman"                         	| "badman" → "batman" | Batman: Year One <br/>Batman: Under the Red Hood       	|

<Callout intent="tip">

💡See bucket sort-powered relevancy in action in this [Meilisearch demo](https://where2watch.meilisearch.com/).

</Callout>

If you want to try it yourself, you can create an account on [Meilisearch Cloud](https://meilisearch.com.cn/cloud) and easily add and set up the movies dataset used to illustrate bucket sort. You can start searching in just a few minutes\! Alternatively, you can use our open-source version and [run Meilisearch locally](https://meilisearch.com.cn/docs/learn/getting_started/quick_start#blog.meilisearch.com). You just need to follow the instructions of our [quick start guide](https://meilisearch.com.cn/docs/learn/getting_started/quick_start). 

<Callout intent="tip">

🧪Experiment with bucket sorting by [changing the order](https://meilisearch.com.cn/docs/reference/api/settings#update-ranking-rules) of the [built-in ranking rules](https://meilisearch.com.cn/docs/learn/core_concepts/relevancy#built-in-rules) and adding your [custom](https://meilisearch.com.cn/docs/learn/core_concepts/relevancy#custom-rules) ones.

</Callout>

## Conclusion

Bucket sort is a powerful and flexible algorithm, but its workings can be complex to understand without a concrete example. We hope this article has helped you gain a better understanding of what bucket sort is and how it functions.

[Meilisearch](https://meilisearch.com.cn/) is an open-source search engine that not only provides state-of-the-art experiences for end users but also a simple and intuitive developer experience.

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/#blog.meilisearch.com) or subscribe to the [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/#blog.meilisearch.com) and participating in [product discussions](https://github.com/meilisearch/product/discussions#blog.meilisearch.com).


Meilisearch 最新消息和公司动态

如何通过筛选器评分提升搜索结果

Hugging Face 利用 Meilisearch 促进 AI 可访问性

推出文档爬虫——Meilisearch Cloud

推出任务视图——Meilisearch Cloud

推出 Meilisearch 的向量搜索和向量数据库，引领搜索的未来

HitPay 优化了客户的搜索能力，同时保持专注于其核心业务。

Meilisearch 8月更新

如何使用第二个索引实现文档提升

桶排序的实用指南