_This blog post was originally published on kero's [personal blog](https://blog.kerollmops.com/meilisearch-indexes-embeddings-7x-faster-with-binary-quantization)._

<Callout intent="note">
This is one blog post in a series:
- [Spotify-Inspired: Elevating Meilisearch with Hybrid Search and Rust](https://blog.kerollmops.com/spotify-inspired-elevating-meilisearch-with-hybrid-search-and-rust)
- [Multithreading and Memory-Mapping: Refining ANN Performance with Arroy](https://blog.kerollmops.com/multithreading-and-memory-mapping-refining-ann-performance-with-arroy-in-rust)
- [Meilisearch Expands Search Power with Arroy's Filtered Disk ANN](https://blog.kerollmops.com/meilisearch-expands-search-power-with-arroy-s-filtered-disk-ann)
- [How Meilisearch Updates a Millions Vector Embeddings Database in Under a Minute](https://blog.kerollmops.com/how-meilisearch-updates-a-millions-vector-embeddings-database-in-under-a-minute)
- Meilisearch Indexes Embeddings 7x Faster with Binary Quantization.
</Callout>

For the past year at Meilisearch, [we’ve been working hard on hybrid search](https://github.com/meilisearch/meilisearch/issues/3838), which mixes keyword search with semantic search using [Arroy: our vector store, based on a Spotify technology](https://blog.kerollmops.com/spotify-inspired-elevating-meilisearch-with-hybrid-search-and-rust). A vector store is a data structure that efficiently stores embeddings (vectors) for fast and relevant retrieval, called Approximate Nearest Neighbors (ANN) search.

When our customers started to use arroy heavily, some of them started reaching the machine limits. For example, for a customer with a 768 dimensions model, we found out that we couldn’t index more than 15M of embeddings on a machine with 64GiB of RAM.

### Binary Quantization to the Rescue

The concept behind this elaborate term is that we are going to *quantize* the embeddings. Quantizing a number involves spreading it into a defined number of values; for *binary* quantization, two values can be represented on a single bit, e.g., 0.2561 becomes 1, and -0.568 becomes -1.

This implies that we could convert a 32-bit float number into a 1-bit number, which would divide the disk and RAM usage by 32. That would mean we could index up to 480M embeddings with 64GiB of RAM, instead of being limited to 15M embeddings.

An issue we faced is that bits cannot represent -1 and 1 but only 0 and 1. We had to use some tricks to make it work and fake 0s into -1s in our computations. For example, one of the functions that we had to optimize a lot was the function in charge of converting a binary quantized vector to 32-bit float vectors.

We do not publicly expose raw quantized embeddings and utilize this function to provide real 32-bit float embeddings to the users. However, the primary reason for converting 1-bit quantized embeddings to 32-bit floats is to ensure high precision when [generating hyperplanes to group embeddings together](https://blog.kerollmops.com/spotify-inspired-elevating-meilisearch-with-hybrid-search-and-rust#spotify-s-hyperplane-trees-for-efficient-anns) and maintaining relevant ANNs.

![Profiling backtrace showing slow 1-bit to 32-bit float conversion](https://unable-actionable-car.media.strapiapp.com/68747470733a2f2f692e696d6775722e636f6d2f497165713649562e706e67_596adc5fd5.png)

_Converting a quantized embedding back to 32-bit floating points takes up to 24.21s on a 1m53s run._

Below is the SIMD neon-specific version of this function. It processes raw bytes corresponding to a 1-bit quantized embedding, converting them into a properly aligned 32-bit float embedding. Using masks, the function separates the low and high bits, processing them in batches of four in 128-bit registers, which are then transformed into four 32-bit floats. These values are collectively written to the output vector.

```rust
unsafe fn to_vec_neon(bytes: &[u8]) -> Vec<f32> {
    let mut output: Vec<f32> = vec![0.0; bytes.len() / u32::BITS];
    let output_ptr = output.as_mut_ptr();
    let low_mask = [0b_0000_0001, 0b_0000_0010, 0b_0000_0100, 0b_0000_1000];
    let high_mask = [0b_0001_0000, 0b_0010_0000, 0b_0100_0000, 0b_1000_0000];
    let ones = unsafe { vld1q_dup_f32(&1.0) };
    let minus = unsafe { vld1q_dup_f32(&-1.0) };

    for (current_byte, base) in bytes.iter().enumerate() {
        unsafe {
            let base = *base as u32;
            let base = vld1q_dup_u32(&base);
            for (i, mask) in [low_mask, high_mask].iter().enumerate() {
                let mask = vld1q_u32(mask.as_ptr());
                let mask = vandq_u32(base, mask);
                // 0xffffffff if equal to zero and 0x00000000 otherwise
                let mask = vceqzq_u32(mask);
                let lane = vbslq_f32(mask, minus, ones);
                let offset = output_ptr.add(current_byte * 8 + i * 4);
                vst1q_f32(offset, lane);
            }
        }
    }

    output
}
```

_<a href="https://x.com/lemire/status/1835700368648323261">Daniel Lemire doesn't seem to have found a better algorithm</a>_ 🤭

### The Results

We chatted about the theoretical benefits of this approach earlier, but how does it work out in real life? Here are different tables summarizing the results for you.

<details>
  <summary>Open to see the original benchmarks</summary>
  <img src="https://unable-actionable-car.media.strapiapp.com/68747470733a2f2f692e696d6775722e636f6d2f67667765626b592e706e67_a96a995c7c.png"  alt="Original arroy benchmark results"/>
</details>

#### Embeddings with 1024 Dimensions

When examining small embeddings, we can observe a significant impact on relevancy due to binary quantization. The primary advantages of this approach lie in faster insertion times and reduced storage requirements. It is not advisable to enable this feature for small embeddings, as they already occupy minimal disk space. Storing over four million 1024-dimensional embeddings requires just 8 GiB of disk space.

| Version                 | Recall @1 | Recall @20 | Recall @100 | Indexing time | Search time | On disk size |
|-------------------------|-----------|------------|-------------|---------------|-------------|--------------|
| Cosine                  | **1.00**  | **0.72**   | **0.84**    | 491s          | **23s**     | 8 GiB        |
| Binary Quantized Cosine | 0.87      | 0.50       | 0.52        | **147s**      | 27s         | **850 MiB**  |

#### Embeddings with 1536 Dimensions

Binary quantization proves advantageous for managing larger embeddings. It significantly reduces search times, utilizing just 15% of the original disk space. Indexing about 500,000 1536-dimensional embeddings is achieved in a quarter of the time. Despite this, there remains a noticeable impact on relevancy...

| Version                 | Recall @1 | Recall @20 | Recall @100 | Indexing time | Search time | On disk size |
|-------------------------|-----------|------------|-------------|---------------|-------------|--------------|
| Cosine                  | **1.00**  | **0.82**   | **0.90**    | 1219s         | 50s         | 13 GiB       |
| Binary Quantized Cosine | **1.00**  | 0.70       | 0.70        | **235s**      | **47s**     | **2 GiB**    |

#### Embeddings with 3072 Dimensions

The impact of binary quantization on large embeddings is evident. While relevance may be slightly affected, the reduction in disk usage, search time, and indexing time is significant. Being able to store the same number of embeddings with just 15% of the space and index 6 times faster is a major achievement! It takes half as much disk space to store a million 3072-dimensional binary quantized embeddings than four million 1024-dimensional embeddings.

| Version                 | Recall @1 | Recall @20 | Recall @100 | Indexing time | Search time | On disk size |
|-------------------------|-----------|------------|-------------|---------------|-------------|--------------|
| Cosine                  | **1.00**  | **0.86**   | **0.92**    | 4000s         | 150s        | 26 GiB       |
| Binary Quantized Cosine | **1.00**  | 0.77       | 0.77        | **560s**      | **100s**    | **4 GiB**    |

### Conclusion

The impact of binary quantization on recall was expected to decrease, given the reduced information in the algorithm. However, for datasets with over 1500 dimensions, the impact on recall is relatively minor compared to the significant benefits in disk space and indexing speed.

In practice, we observed that space utilization decreased not by 32 times but by just 15% of the original space, a notable achievement. This is primarily due to arroy storing not only user vectors but also internal nodes for efficient searching. Split nodes (hyperplanes) are retained as full 32-bit floats to maintain precision.

- Indexing speed improves nearly 5 times with 1536 dimensions and about 7 times with 3072 dimensions.
- Disk size shrinks to around 6.5 times smaller.
- Search time shows slight improvements for embeddings starting at 1536 dimensions.

We recommend using Meilisearch/arroy's binary quantization feature for OpenAI users with large embeddings. In an upcoming article, we will compare Meilisearch/arroy with competitors, focusing on our Filtered DiskANN feature. Stay tuned!

Meilisearch indexes embeddings 7x faster with binary quantization

How to add AI-powered search to a React app

Meilisearch is too slow

This guide will walk you through setting up a Next.js storefront for your Shopify commerce. We will use the open-source [Enterprise Ecommerce](https://github.com/Blazity/enterprise-commerce) starter developed by [Blazity](https://blazity.com/) which comes with a CLI to get started in a snap.

Overview
--------

The diagram below gives an overview of the architecture of our application.

![Architecture of an application developed with Enterprise Ecommerce start](https://unable-actionable-car.media.strapiapp.com/blazity_commerce_architecture_0a30272f59.png)

The application stack is composed of 4 key components:

*   **Data sources** — Shopify admin, personalization tools, etc.
*   **Search engine** — Meilisearch
*   **Full-stack app** — Next.js application
*   **Third-party integrations** — Analytics, CRM, etc.

In this architecture, the Meilisearch database aggregates data from multiple sources to make it accessible by the Next.js application.

Our final application will look like this: [live preview](https://commerce.blazity.com/) ✨

Prerequisites
-------------

This guide requires:

*   [Node.js](https://node.org.cn/en/download/package-manager) ≥ 20.x
*   [Yarn](https://yarn.npmjs.net.cn/getting-started/install) ≥ 4.x
*   [Meilisearch](https://meilisearch.com.cn/cloud#blazity&utm_source=blog&utm_medium=shopify-nextjs-guide) ≥ 1.8
*   A [Shopify](https://www.shopify.com/) store

Setup
-----

This setup phase provides you with the necessary credentials for your application to interact with Meilisearch and your Shopify store.

These credentials match an environment variable that will be needed during the installation phase. For clarity, this guide will indicate with a "✅" emoji each of these variables.

<Callout>
  Thanks to Blazity for the instructions. If you’re running into issues, please consult the [Blazity documentation](https://docs.commerce.blazity.com/).
</Callout>

### Creating a Shopify store

If you don’t have one already, go to [Shopify](https://www.shopify.com/) and create a new store. Shopify associates a unique ID with each store named the _store domain_.

To find your Shopify store domain, go to _Settings_ > _Domains_ . Your store domain looks like this: `your-domain.myshopify.com`.

✅ `Shopify store domain`

### Installing Shopify Headless

In the Shopify Marketplace, find the [Headless application](https://apps.shopify.com/headless) and install it. After installation, click on _Create storefront_.

Then, we will update the API access. In the _Manage API access_ section, click on _Manage_ next to _Storefront API_.

Edit the Storefront API permissions to add the following:

*   `unauthenticated_read_product_inventory`
*   `unauthenticated_read_customer_tags`

After saving the permissions, you can copy the **private access token**.

✅ `Shopify app Storefront API Token`

### Creating a Shopify App

Now, we will create a Shopify App for our store. Shopify Apps allow you to build custom integrations for your store. This app will allow us to get API keys to read and manage data in our Shopify store.

To create a Shopify app:

*   Navigate to your store’s Shopify admin dashboard
*   Open the store’s _Settings_ and navigate to the _Apps and sales channels_ tab
*   Click on _Develop apps_
*   If requested, follow the steps to allow custom app development for your store
*   Create an app and enter a name, e.g., “Next.js App”

After creating your app, navigate to the API credentials tab to find your Shopify app’s API key (not the API secret key!)

✅ `Shopify app API key`

### Configuring the Shopify Admin API key

Shopify apps work with a set of two keys:

*   An **admin API key** — to manage the Shopify admin, e.g., to edit your product catalog
*   A **storefront API key** — to read the product catalog, create carts, etc.

We can configure the access scopes for these API keys in your Shopify app’s _Configuration_ tab. In this section, we will enable the required scopes for our Admin API key.

To configure the scopes for the Admin API key, follow these steps:

*   In the _Admin API integration_ section, click on _Configure_
*   Enable the following scopes:
    *   `write_product_listings`
    *   `read_product_listings`
    *   `read_products`
    *   `write_products`
*   Under _Webhook subscriptions_, select the 2024-01 API version
*   Save :)

<Callout>
  The API version might have changed since this guide was published. We recommend checking out the [Blazity documentation](https://docs.commerce.blazity.com/providers#3-private-admin-app).
</Callout>

With our tokens API scopes defined, we can return to the _API credentials_ tab and install our app. Click _Install app_ to generate your API access tokens.

After installing your Shopify app, you will have access to your Admin API access token. For security reasons, this token will only be revealed to you _once_—**make sure to save it**.

✅ `Shopify app Admin API token`

### Creating your Meilisearch database

Meilisearch is an open-source search engine that provides full-text, vector, and hybrid search capabilities. Our Shopify products data will be synced with Meilisearch, enabling the front-end app to get fast, relevant search results for end users.

If you don’t have one, create a [Meilisearch Cloud](https://meilisearch.com.cn/cloud#blazity&utm_source=blog&utm_medium=shopify-nextjs-guide) account. If necessary, you can first learn how to [get started with Meilisearch Cloud](https://meilisearch.com.cn/docs/learn/getting_started/cloud_quick_start#blazity&utm_source=blog&utm_medium=shopify-nextjs-guide).

<Callout>
  Prefer self-hosting? Learn how to [install Meilisearch locally](https://meilisearch.com.cn/docs/learn/getting_started/installation#blazity&utm_source=blog&utm_medium=shopify-nextjs-guide).
</Callout>

First, create a new _Project_, e.g., Shopify store. This will create a new Meilisearch database and should take a few seconds to complete.

Then, you can navigate to your project’s Settings page to find your _Database URL_ (your host) and _Default Admin API Key_.

✅ `Meilisearch host URL`

✅ `Meilisearch API token`

<Callout>
  We recommend enabling [analytics & monitoring](https://meilisearch.com.cn/docs/learn/analytics/analytics#blazity&utm_source=blog&utm_medium=shopify-nextjs-guide) at this step, so you don’t need to change host later.
</Callout>

### Configuring your Meilisearch

After creating your database, we will create the indexes. You can choose any names for them, but you’ll have to configure your environment variables accordingly. Let’s create two indexes:

*   An index for our products, e.g., `products`
*   An index for our product categories, e.g., `categories`

✅ `Meilisearch Products Index Name`

✅ `Meilisearch Categories Index Name`

With our indexes created, we can now configure them. Meilisearch defaults will work well for most settings. For each index, we will only configure the filterable attributes and sortable attributes.

**Product index** — Filterable attributes

```json
[
	"collections",
	"collections.handle",
	"collections.id",
	"flatOptions",
	"handle",
	"minPrice",
	"tags",
	"variants.availableForSale",
	"vendor",
	"hierarchicalCategories",
	"hierarchicalCategories.lvl0",
	"hierarchicalCategories.lvl1",
	"hierarchicalCategories.lvl2",
	"avgRating"
]
```

**Product index** — Sortable attributes

```json
[
	"minPrice",
	"updatedAtTimestamp",
	"avgRating"
]
```

**Categories index** — Filterable attributes

```json
[
	"handle",
	"id"
]
```

You can leave all other attribute settings to their default value.

Installation
------------

With all credentials in hand, we’re ready to launch the installation wizard. Let’s create our project using Blazity’s `commerce` CLI.

```bash
# with NPM
npm create commerce

# with Yarn
yarn create commerce
```

The first prompt will ask you to select the services to enable. For this guide, we’ll only enable the defaults: Shopify and Meilisearch.

Then, the CLI will ask for a series of credentials. If you followed the Setup section, you should already have all the necessary credentials. After providing all the credentials, the CLI will scaffold your project in a new folder.

After creation, navigate to your project directory. Your `apps/web/.env.local` should look like this:

```bash
# apps/web/.env.local

SHOPIFY_STOREFRONT_ACCESS_TOKEN=shpat_a-very-long-token
SHOPIFY_ADMIN_ACCESS_TOKEN=shpat_another-very-long-token
SHOPIFY_STORE_DOMAIN=your-domain.myshopify.com
SHOPIFY_APP_API_SECRET_KEY=a-32-characters-long-api-key
SHOPIFY_HIERARCHICAL_NAV_HANDLE=hierarchical-categories
MEILISEARCH_PRODUCTS_INDEX=products
MEILISEARCH_CATEGORIES_INDEX=categories
MEILISEARCH_ADMIN_KEY=your-meilisearch-admin-api-key
MEILISEARCH_HOST=https://edge.meilisearch.com
LIVE_URL=https://your-domain.myshopify.com
FLAGS_SECRET=randomly-generated-secret
CRON_SECRET=another-randomly-generated-secret
```

Syncing Meilisearch and Shopify
-------------------------------

Blazity provides a CLI for synchronizing the data between your Shopify and Meilisearch. In this section, we will see how to achieve that using `commerce-cli`.

To sync your Shopify data with your Meilisearch, navigate to your project folder and run:

```bash
npx commerce-cli@latest sync
```

This command will pull all the necessary data from your Shopify store and sync it into your Meilisearch database.

<Callout>
  Additionally, the [`commerce-cli` package](https://docs.commerce.blazity.com/features/cli) can assist you in configuring webhooks to keep your Meilisearch in sync at all times.
</Callout>

Starting your app
-----------------

After initializing your application and syncing the data between Shopify and Meilisearch, you are ready to start your application.

To start your application in development mode, run:

```bash
# with NPM
npm run dev

# with Yarn
yarn dev
```

And that’s it! You can visit your application at `https://:3000` 🎉

You can also consult a live example here: [https://commerce.blazity.com/](https://commerce.blazity.com/#blog.meilisearch.com)

Going further
-------------

This guide doesn’t cover all available features in the Blazity template. For more advanced configurations, you can learn how to set up hierarchical categories, implement highly scalable redirects, and more on [Blazity documentation](https://docs.commerce.blazity.com/introduction).

Additionally, here are resources to help you integrate your new Shopify store with Meilisearch and Next.js:

*   [Meilisearch documentation](https://meilisearch.com.cn/docs#blazity&utm_source=blog&utm_medium=shopify-nextjs-guide)
*   [Next.js documentation](https://nextjs.net.cn/docs)
*   [Integrate Meilisearch Cloud with Vercel](https://meilisearch.com.cn/docs/guides/integrations/vercel#blazity&utm_source=blog&utm_medium=shopify-nextjs-guide)

* * *

Stay in the loop of everything Meilisearch by subscribing to our [monthly newsletter](https://dub.sh/meili-newsletter#blazity&utm_source=blog&utm_medium=shopify-nextjs-guide). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/#blazity&utm_source=blog&utm_medium=shopify-nextjs-guide) and participate in our [product discussions](https://github.com/orgs/meilisearch/discussions).

For anything else, join our developers community on [Discord](https://dub.sh/meili-discord/#blazity&utm_source=blog&utm_medium=shopify-nextjs-guide).

Build your Next.js Shopify storefront with Blazity

<Callout>
  Meilisearch 1.8 is available on [Meilisearch Cloud](https://cloud.meilisearch.com/#release-v1-8&utm_source=blog) too—upgrade now!
</Callout>

New: negative search
--------------------

Meilisearch 1.8 allows you to exclude specific terms from your search results using the `-` operator. 

For example, to find documents that contain the word "room" but not "escape", place a `-` before "escape”:

```Bash
curl \
  -X POST 'https://edge.meilisearch.com/indexes/places/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "-escape room" }'

```

Experimental: Improved hybrid search
------------------------------------

<Callout>
  AI search is an experimental feature. Give feedback in this [GitHub discussion](https://github.com/orgs/meilisearch/discussions/677).
</Callout>

### New Ollama embedder

Meilisearch 1.8 introduces support for Ollama models. [Configure the embedder](https://meilisearch.com.cn/docs/learn/experimental/vector_search#release-v1-8&utm_source=blog#generate-vector-embeddings) in your index settings:

```JSON
"default": {
  "source": "ollama", // 👈 Use the ollama source
  "url": "https://:11434/api/embeddings",
  // ☝️ optional, fetched from MEILI_OLLAMA_URL environment variable if missing
  "apiKey": "<your-api-key>",
  // ☝️ optional
  "model": "nomic-embed-text",
  "documentTemplate": "{{doc.field_name}}" // like for openAI and huggingFace sources
}

```

### New REST Embedder

Starting with Meilisearch 1.8, you can use any embedder accessible via a REST API. Here’s how to configure an OpenAI embedder with the `rest` source:

```JSON
{
  "source": "rest",
  "apiKey": "<your-openai-api-key>",
  "dimensions": 1536,
  "url": "https://api.openai.com/v1/embeddings",
  "query": {
    "model": "text-embedding-ada-002"
  },
  "inputField": ["input"],
  "inputType": "textArray",
  "pathToEmbeddings": ["data"],
  "embeddingObject": ["embedding"]
}

```

For more details on the API and available parameters, please consult the documentation on [REST embedders](https://meilisearch.com.cn/docs/learn/experimental/vector_search#generate-auto-embeddings#release-v1-8&utm_source=blog).

### New embedder setting: distribution shift

<Callout>
  This is an advanced API for custom models. OpenAI and Hugging Face models don't require this setting.
</Callout>

Depending on the embedding model, [vector embeddings](/blog/what-are-vector-embeddings/#release-v1-8&utm_source=blog) are not always uniformly distributed within the [vector space](/blog/what-are-vector-embeddings/#release-v1-8&utm_source=blog) When vectors are overly clustered, semantic search scores can be very closely ranked. As a result of semantic scores being packed, hybrid results might have sub-optimal relevancy.

To address this issue, you can now shift the distribution of results with an affine transformation. This transformation normalizes semantic scores to prevent them from clustering too high or too tightly, making them more comparable to the [keyword scores](https://meilisearch.com.cn/docs/reference/api/search#ranking-score).

```JSON
"default": {
  "source": "huggingFace",
  "model": "MODEL_NAME",
  "distribution": {  // describes the natural distribution of results
    "mean": 0.7, // mean value
    "sigma": 0.3 // standard deviation
  }
}

```

*   The **mean** value represents the central point around which you observe semantic search scores to be densely packed. In the provided code sample, the setting indicates that semantic scores from this model tend to cluster around a mean of 0.7.
*   The **sigma** value determines the spread of ranking scores from the mean (standard deviation). The sigma provided in the code sample above means that before shifting the distribution, the results tend to have an average distance of 0.3 to the mean.

This allows for a more effective comparison of `_rankingScore` between semantic and keyword search results, leading to improved results ranking.

### Breaking: trimmed API response

To speed up the search response and reduce bandwidth usage, Meilisearch v1.8 no longer returns the following fields in the search responses:

*   the `vector` field, which returned the query vector 
*   the `value` field included in the [`vectorSort`](https://meilisearch.com.cn/docs/reference/api/search#release-v1-8&utm_source=blog#vectorsorttarget_vector) object, which is returned when you add \`”showRankingScoreDetails”: true”\` to a search query
*   the `_semanticScore` field; its value is the same as `_rankingScore`, which you should use instead. Set [`”showRankingScore”: true”`](https://meilisearch.com.cn/docs/reference/api/search#blog&utm_campaign=release-v1-8#ranking-score) in the search parameters

Search robustness improvements
------------------------------

### New search cut-off setting

To enhance stability and performance, Meilisearch automatically terminates any search request that exceeds 1500 milliseconds. 

To adjust this timeout threshold, you can customize the `searchCutoffMs` setting:

```bash
curl \
  -X PATCH 'https://edge.meilisearch.com/indexes/movies/settings' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "searchCutoffMs": 150
  }'

```

### Experimental: concurrent search requests limit

This release introduces a limit on concurrent search requests to prevent Meilisearch from using excessive amounts of RAM and crashing. The default queue limit is set to 1000 requests. 

To modify this limit, restart your self-hosted instance with the `--experimental-search-queue-size` option:

```bash
./meilisearch --experimental-search-queue-size 100
```

👉 This limit does not impact the search performance. It only controls the number of search requests that can be queued simultaneously to enhance system security.

<Callout>
  Give feedback regarding concurrent search requests limit in this [GitHub discussion](https://github.com/orgs/meilisearch/discussions/729).
</Callout>

Contributors shout-out
----------------------

We are really grateful for all the community members who participated in this release. We would like to thank: [@availhang](https://github.com/availhang), [@hack3ric](https://github.com/hack3ric), [@jakobklemm](https://github.com/jakobklemm), [@mohsen-alizadeh](https://github.com/mohsen-alizadeh), [@redistay](https://github.com/redistay), [@rohankmr414](https://github.com/rohankmr414), [@shuangcui](https://github.com/shuangcui), [@Toromyx](https://github.com/Toromyx), [@6543](https://github.com/6543) for their help with Meilisearch, and [@Gusted](https://github.com/Gusted), [@mosuka](https://github.com/mosuka), [@6543](https://github.com/6543), for their work on [Charabia](https://github.com/meilisearch/charabia).

Special thanks to our SDKs maintainers, thanks to whom Meilisearch is available across so many languages. 🫶

* * *

And that’s a wrap for v1.8! This release post highlights the most significant updates. For an exhaustive listing, read the [changelog on Github](https://github.com/meilisearch/meilisearch/releases/tag/v1.8.0).

Stay in the loop of everything Meilisearch by subscribing to our [newsletter](https://dub.sh/meili-newsletter). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/#release-v1-8&utm_source=blog) and come participate in our [Product Discussions](https://github.com/meilisearch/product/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/#release-v1-8&utm_source=blog).

Meilisearch 1.8

We're excited to unveil Meilisearch v1.7. In this article, we’ll review the most impactful changes. For an exhaustive listing, check out the [changelog](https://github.com/meilisearch/meilisearch/releases/tag/v1.7.0) on Github.

<Callout>
  Meilisearch 1.7 is available on [Meilisearch Cloud](https://cloud.meilisearch.com/#release-v1-7&utm_source=blog) too—upgrade now!
</Callout>

Improved hybrid & vector search
-------------------------------

<Callout>
  AI search is an experimental feature, read the docs to [enable AI search](https://meilisearch.com.cn/docs/learn/experimental/vector_search#release-v1-7&utm_source=blog#activate-vector-search).
</Callout>

### New OpenAI embedding models

Starting with Meilisearch 1.7, you can use OpenAI’s third-generation embedding models:

*   `text-embedding-3-small`
*   `text-embedding-3-large`

Compared to the previous `text-embedding-ada-002` model, these new models offer improvements allowing to balance search results relevancy and cost efficiency.

The `small` model is more cost-effective, generating smaller vector embeddings that require less storage. The `large` focuses on delivering higher precision in search results, making it ideal to optimize relevancy.

### Configuring OpenAI model dimensions

When using the new `text-embedding-3-small` and `text-embedding-3-large` OpenAI models, you can now configure **dimensions** for your model. When not provided, the model's defaults dimensions are used.

You can do so when updating your settings:

```json
{
  // other settings...

  "embedders": {
    "small": {
      "source": "openAi",
      "model": "text-embedding-3-large",
      "dimensions": 512 // must be >0, must be <= 3072 for "text-embedding-3-large"
    },
  }
}

```

Higher dimensions typically offer more complex embeddings which can lead to more accurate search results. However, increased dimensions are more costly to compute and require more storage.

<Callout>
  You cannot customize dimensions when using OpenAI `text-embedding-ada-002`.
</Callout>

### Use AI Search on Meilisearch Cloud

Meilisearch hybrid search and vector search are coming to Meilisearch Cloud. We're rolling out invites every day. Join the beta. 👇

[Get early access](https://share-eu1.hsforms.com/1RN-rGYhHQPu5BNivfknlwAfg3aq#release-v1-7&utm_source=blog)

GPU support for computing Hugging Face embeddings
-------------------------------------------------

Meilisearch 1.7 comes with the ability to enable GPU support for [self-hosted Meilisearch](https://meilisearch.com.cn/docs/learn/getting_started/installation#release-v1-7&utm_source=blog) for computing Hugging Face embeddings. This can significantly speed up the embedding generation process.

Find detailed instructions in our dedicated guide on [Using GPU to compute Hugging Face embeddings](https://meilisearch.com.cn/docs/learn/cookbooks/computing_hugging_face_embeddings_gpu#release-v1-7&utm_source=blog).

Display ranking scores details at search
----------------------------------------

Meilisearch 1.7 stabilizes the `showRankingScoreDetails` search parameter that was introduced in [Meilisearch 1.3](/blog/v1-3-release/#release-v1-7&utm_source=blog This parameter adds a `_rankingScoreDetails` field to each document in search results, enabling further inspection of the search results.

You can include the parameter when making searches:

```sh
curl \
  -X POST 'https://edge.meilisearch.com/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{
    "q": "batman",
    "showRankingScoreDetails": true
  }'

```

Learn more about the [`_rankingScoreDetails` object](https://meilisearch.com.cn/docs/reference/api/search#ranking-score-details#release-v1-7&utm_source=blog) and how to use it with your preferred SDK in the API reference.

Contributors shout-out
----------------------

We want to give a massive thank you to all contributors that made this release possible: [@capJavert](https://github.com/capJavert), [@codesmith-emmy](https://github.com/codesmith-emmy), [@Elliot67](https://github.com/Elliot67) and [@Gosti](https://github.com/Gosti) for their PRs in Meilisearch, and [@agourlay](https://github.com/agourlay), [@choznerol](https://github.com/choznerol), [@ngdbao](https://github.com/ngdbao), [@timvisee](https://github.com/timvisee), and [@xshadowlegendx](https://github.com/xshadowlegendx) for their work on [charabia](https://github.com/meilisearch/charabia).

Special thanks to our SDKs maintainers, thanks to whom Meilisearch is available across so many languages. 🫶

* * *

And that’s a wrap for v1.7! This release post highlights the most significant updates. For an exhaustive listing, read the [changelog on Github](https://github.com/meilisearch/meilisearch/releases/tag/v1.7.0).

Stay in the loop of everything Meilisearch by subscribing to our [newsletter](https://dub.sh/meili-newsletter). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/#release-v1-7&utm_source=blog) and come participate in our [Product Discussions](https://github.com/meilisearch/product/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/#release-v1-7&utm_source=blog).

Meilisearch 1.7

In this guide, we will see how to use the search functionality in Laravel 10. We'll start with by introducing the benefits of full-text search. Then, we'll walk you through setting up full-text search in your Laravel application.

Why use full-text search?
-------------------------

In traditional SQL or NoSQL databases, queries find results exactly matching given criteria. Conversely, full-text search queries can match some _or_ all of a text query with the database’s content. So essentially, full-text search can provide results even in case of partial matches.

When building user-facing search interfaces, full-text search is empowering for users. Tolerance to typos, prefix search, and synonyms help them get results more quickly. It improves discoverability when users do not know _what_ they're looking for.

<Callout>
  See all [full-text search features](https://meilisearch.com.cn/docs/learn/what_is_meilisearch/overview#features#laravel&utm_source=blog&utm_medium=full-text-search-guide) of Meilisearch search engine.
</Callout>

How to use search functionality in Laravel 10?
----------------------------------------------

### Installing Laravel Scout

Laravel comes with out-of-the-box full-text search capabilities via Laravel Scout.

To enable it, navigate to your Laravel application directory and install Scout via the [Composer](https://getcomposer.org/) package manager:

```sh
composer require laravel/scout
```

After installing Scout, you should publish the Scout configuration file. You can do this by running the following artisan command:

```sh
php artisan vendor:publish --provider="Laravel\Scout\ScoutServiceProvider"
```

This command should create a new configuration file in your application directory: `config/scout.php`.

### Configuring the Laravel Scout driver

Let’s configure Laravel Scout to use the Meilisearch driver. Meilisearch is an open-source search engine built in Rust. This will allow to get the best full-text search performance. Indeed, the database driver comes with [limitations inherent to SQL databases](/blog/postgres-full-text-search-limitations/#laravel&utm_source=blog&utm_medium=full-text-search-guide

First, install the dependencies required to use Scout with Meilisearch via Composer:

```sh
composer require meilisearch/meilisearch-php http-interop/http-factory-guzzle
```

Then, update the environment variables in your `.env` file:

```sh
SCOUT_DRIVER=meilisearch
# Use the host below if you're running Meilisearch via Laravel Sail
MEILISEARCH_HOST=http://meilisearch:7700
MEILISEARCH_KEY=masterKey
```

Laravel’s official Docker development environment, [Laravel Sail](https://laravel.net.cn/docs/master/sail), comes with a Meilisearch service out-of-the-box. Please note that when running Meilisearch via Sail, Meilisearch’s host is `http://meilisearch:7700`.

<Callout>
  This comes from Docker [Bridge network driver](https://docs.dockerd.com.cn/network/drivers/bridge/#differences-between-user-defined-bridges-and-the-default-bridge) behavior.
</Callout>

For production use cases, we recommend using a managed Meilisearch via [Meilisearch Cloud](https://meilisearch.com.cn/cloud#laravel&utm_source=blog&utm_medium=full-text-search-guide). On Meilisearch Cloud, you can find your host URL in your project settings.

<Callout>
  Learn [how to get started with Meilisearch Cloud](https://meilisearch.com.cn/docs/learn/getting_started/cloud_quick_start#laravel&utm_source=blog&utm_medium=full-text-search-guide).
</Callout>

### Making Eloquent models searchable

With Scout installed and configured, just add the `Laravel\Scout\Searchable` trait to your Eloquent models to make them searchable. This trait will use Laravel’s model observers to keep the data in your model in sync with Meilisearch.

Here’s an example model:

```php
<?php

namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Laravel\Scout\Searchable;

class Contact extends Model
{
	use Searchable;
}
```

You can use the `toSearchableArray` method to configure which fields to store in Meilisearch. This notably enables storing a model and its relationships’ data in the same document.

The example below shows how to store a model's relationships data in Meilisearch:

```php
<?php

namespace App\Models;

use App\Models\Company;
use Laravel\Scout\Searchable;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Relations\BelongsTo;

class Contact extends Model
{
    use Searchable;

    public function company(): BelongsTo
    {
        return $this->belongsTo(Company::class);
    }

    public function toSearchableArray(): array
    {
	    // All model attributes are made searchable
        $array = $this->toArray();

		// Then we add some additional fields
        $array['organization_id'] = $this->company->organization->id;
        $array['company_name'] = $this->company->name;
        $array['company_url'] = $this->company->url;

        return $array;
    }
}

```

### Configuring filterable and sortable attributes

Meilisearch allows you to perform advanced [filtering](https://meilisearch.com.cn/docs/learn/fine_tuning_results/filtering#laravel&utm_source=blog&utm_medium=full-text-search-guide) and [sorting](https://meilisearch.com.cn/docs/learn/fine_tuning_results/sorting#laravel&utm_source=blog&utm_medium=full-text-search-guide) on your search results. Choose which attributes are filterable and sortable via your Meilisearch index settings.

Configure your Meilisearch index settings via the `config/scout.php` file:

```php
<?php

use App\Models\Contact;

return [
	// additional configuration...
	
    'meilisearch' => [
        'host' => env('MEILISEARCH_HOST', 'https://:7700'),
        'key' => env('MEILISEARCH_KEY'),
        'index-settings' => [
            Contact::class => [
                'filterableAttributes' => ['organization_id'],
                'sortableAttributes' => ['name', 'company_name']
            ],
        ],
    ],
```

The example above updates Meilisearch index settings for the Contact model:

*   it makes the `organization_id` field filterable
*   it makes the `name` and `company_name` fields sortable

Update your Meilisearch index settings by running the following Artisan command:

```sh
php artisan scout:sync-index-settings
```

Laravel full-text search example
--------------------------------

We built a demo application to give you a feel of what full-text search looks like in a Laravel application. This demo showcases an app-wide search in a CRM (Customer Relationship Management) application.

[![CRM demo application build with Laravel](https://unable-actionable-car.media.strapiapp.com/saas_demo_screenshot_e72232c181.png)](https://saas.meilisearch.com/#laravel&utm_source=blog&utm_medium=full-text-search-guide)

[Laravel SaaS search demo](https://saas.meilisearch.com/#laravel&utm_source=blog&utm_medium=full-text-search-guide)

This demo application uses the following search features:

*   [Multi-search](https://meilisearch.com.cn/docs/reference/api/multi_search#laravel&utm_source=blog&utm_medium=full-text-search-guide) (search across multiple indexes)
*   [Multi-tenancy](https://meilisearch.com.cn/docs/learn/security/tenant_tokens#laravel&utm_source=blog&utm_medium=full-text-search-guide)
*   [Filtering](https://meilisearch.com.cn/docs/learn/fine_tuning_results/filtering#laravel&utm_source=blog&utm_medium=full-text-search-guide)
*   [Sorting](https://meilisearch.com.cn/docs/learn/fine_tuning_results/sorting#laravel&utm_source=blog&utm_medium=full-text-search-guide)

The code is open-sourced on Github. 🎉

👉 Check out the repository: [https://github.com/meilisearch/saas-demo](https://github.com/meilisearch/saas-demo)

* * *

We hope this guide helped to understand the importance of full-text search and how to implement it with Laravel. For more information, read the [Laravel Scout](https://laravel.net.cn/docs/10.x/scout) and [Meilisearch](https://meilisearch.com.cn/docs#laravel&utm_source=blog&utm_medium=full-text-search-guide) docs.

[Meilisearch](https://meilisearch.com.cn/#blog.meilisearch.com) is an open-source search engine with intuitive developer experience to build user-facing search. You can [self-host](https://meilisearch.com.cn/docs/learn/getting_started/quick_start#laravel&utm_source=blog&utm_medium=full-text-search-guide) it or get a premium experience with [Meilisearch Cloud](https://meilisearch.com.cn/cloud#laravel&utm_source=blog&utm_medium=full-text-search-guide).

For more things Meilisearch, you can join the community on [Discord](https://discord.gg/meilisearch/#blog.meilisearch.com) or subscribe to the [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). You can learn more about the product by checking out the [roadmap](https://roadmap.meilisearch.com/#blog.meilisearch.com) and participating in [product discussions](https://github.com/meilisearch/product/discussions#blog.meilisearch.com).

A complete guide to full-text search with Laravel

[Meilisearch Cloud](https://meilisearch.com.cn/cloud#teams-release&utm_source=blog) introduces the all-new Teams feature. Now you can host and manage your projects in collaboration with your teammates.

Keep reading to learn how to use our Teams feature.

**Creating a team**
-------------------

From your main dashboard, click on the dropdown menu next to the Meilisearch logo on the top navbar, select _Create a team_. By clicking it and assigning a name to your team, your collaborative space is instantly set up.

![Create a team](https://unable-actionable-car.media.strapiapp.com/teams_1_cb16300cee.png)

**Inviting team members**
-------------------------

To invite your colleagues, navigate to the team settings, click on _Invite new member_, and enter their email address.

![Invite new member](https://unable-actionable-car.media.strapiapp.com/team_2_03149ccabf.png)

When you invite someone to a team, they receive an invitation link via email, and upon acceptance, they can start collaborating immediately. If they already have an account, a notification pops up on their dashboard too.

Collaborating
-------------

Every team member can create their own projects, ensuring everyone has a say in the collaboration.

Managing a team
---------------

For team owners have extensive control. They can modify project plans and update billing information.

From the _Team settings_ section, team owners can rename their team, manage user invitations and removals, and even transfer ownership.

![Team settings](https://unable-actionable-car.media.strapiapp.com/teams_4_468f5be5d0.png)

We hope this new release will improve collaboration with your team! Now start sending those invites 😜

[Open Meilisearch Cloud](https://cloud.meilisearch.com/#teams-release&utm_source=blog)

Learn how to [get started with Meilisearch Cloud](https://meilisearch.com.cn/docs/learn/getting_started/cloud_quick_start#teams-release&utm_source=blog).

* * *

Stay in the loop by subscribing to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&utm_campaign=teams-release&utm_source=blog). To learn more about Meilisearch's future and help shape it, take a look at our [roadmap](https://roadmap.meilisearch.com/#teams-release&utm_source=blog) and come participate in our [Product Discussions](https://github.com/meilisearch/product/discussions/#teams-release&utm_source=blog).

For anything else, join our developer community on [Discord](https://discord.meilisearch.com/#teams-release&utm_source=blog).

Introducing Teams — Meilisearch Cloud

This guide breaks down the concept of nested hierarchical facets and offers practical tips for implementation with [Meilisearch](https://meilisearch.com.cn/).

* * *

Table of contents
-----------------

[What is faceted search?](#what-is-faceted-search?)  
[Nested hierarchical facets](#nested-hierarchical-facets)  
[Data structuring for hierarchical faceted search](#data-structuring-for-hierarchical-faceted-search)  
[Creating a hierarchical faceted search experience with Meilisearch](#creating-a-hierarchical-faceted-search-experience-with-meilisearch)  
[Using InstantSearch and instant-meilisearch](#using-instantsearch-and-instant-meilisearch)

* * *

What is faceted search?
-----------------------

Faceted search lets users refine search results via broad categories or facets. It's an intuitive way to sift through content, enabling users to find results matching their exact requirements.

Take ecommerce websites as a case in point. Users, when searching for products, encounter both a result list and a range of facets to help refinement. These facets can typically be found on the sidebar.

Example from our [ecommerce demo](https://ecommerce.meilisearch.com/):

![](https://lh6.googleusercontent.com/vYLVdlN0R66skuGADtqg1mO_yYVuNs-kaNUKE0FRAS55tGHj5_08OEZ4-0rLMoyx9NpBm5z-AIV6ySCJThl1jjoh5GtDFFzAH1B-Lzk38N9ksnGMFD7zzMCn0zL2zb-3fiYFQPBUlAQwi9S5BKXC6n0)

Nested hierarchical facets
--------------------------

Nested facets provide a multi-level structure that helps users navigate through data by applying successive filters, often displayed in a hierarchical manner.

**Nested hierarchical facets** are particularly useful for navigating and searching large datasets or collections. Users can navigate up and down through various levels, thereby improving discoverability.

Picture an online book shopping experience. Users can:

1.  Begin with the overarching "Books" category
2.  Dive deeper by selecting a "Genre"
3.  Focus on a sub-genre
4.  And further refine by attributes like "Author", "Publication Date", or "Rating"

The foundation for such a search lies in the structuring of data.

### Data structuring for hierarchical faceted search

When implementing a hierarchical faceted search, data documents need a hierarchical categorization too—broad categories leading to finer ones.

Take a product dataset as an example:

```JSON
{
    "id": 1,
    "name": "Seagate BarraCuda 1 To (ST1000DM014)",
    "hierarchicalCategories": {
        "lvl0": "Electronics",
        "lvl1": "Electronics > Computer and Tablets",
        "lvl2": "Electronics > Computer and Tablets > Computer Parts",
        "lvl3": "Electronics > Computer and Tablets > Computer Parts > Storage"
    } 
}

```

Each product is linked to hierarchical categories, flowing from broad to specific, to establish a clear pathway through the data. This organized categorization is crucial in implementing faceted search, steering users from general categories like "Electronics" to more precise ones, such as "Storage" within "Computer Parts"

### Creating a hierarchical faceted search experience with Meilisearch

In Meilisearch, facets are a specific use-case of [filters](https://meilisearch.com.cn/docs/learn/fine_tuning_results/filtering). Whether something is a filter or a facet depends above all on UX and UI design. To be able to refine results based on facets, you need to add the attributes you want to use as facets to the [filterableAttributes list](https://meilisearch.com.cn/docs/reference/api/settings#filterable-attributes). From our product example, you should set  `hierarchicalCategories` as filterable.

Hierarchical facets typically use the [`AND` logical operator](https://meilisearch.com.cn/docs/learn/fine_tuning_results/filtering#and), as each level of the hierarchy acts conjunctively with the others to narrow down the search progressively.  

Example:

*   Category: Electronics
    *   Subcategory: Computers
        *   Brand: Apple

The results would show items that are in the Electronics category AND the Computers subcategory AND of the Apple brand.

When creating a faceted search interface, it is helpful to display the distribution of results across the different facets. Showing users the number of results in different categories as they refine their search can enhance their search experience.

Meilisearch supports this functionality through the [`facets` search parameter](https://meilisearch.com.cn/docs/reference/api/search#facets). When you add the `facets` parameter to your search query, Meilisearch returns a [facetDistribution](https://meilisearch.com.cn/docs/reference/api/search#blog.meilisearch.com#facetdistribution) object. This object provides the number of matching documents distributed among the values of the given facet.

```JSON
  "facetDistribution":{
    "categories":{
      "Electronics":20,
      "Electronics > Computer and Tablets":7,
      …
      "Electronics > Computer and Tablets > Computer Parts > Storage":3
    }
```

Leveraging the facets search parameter, you can present a neatly organized and intuitive search interface, allowing users to filter through hierarchical categories effectively. Meilisearch hands you the keys to build your own faceted search UI, yet for a quick and easy setup, employing InstantSearch with instant-meilisearch is a highly efficient alternative.

### Using InstantSearch and instant-meilisearch

[InstantSearch](https://github.com/algolia/instantsearch#blog.meilisearch.com) is an open-source front-end library to build search UIs. [Instant-meilisearch](https://github.com/meilisearch/meilisearch-js-plugins/tree/main/packages/instant-meilisearch#blog.meilisearch.com) is the go-to search client for integrating InstantSearch with Meilisearch.

In this guide, we’ll be using React code samples, you can find more examples in InstantSearch’s documentation

To use InstantSearch with instant-meilisearch, you need to:

#### 1\. Import the required modules

```JavaScript
import React from 'react';
import { InstantSearch, SearchBox, InfiniteHits, HierarchicalMenu } from 'react-instantsearch';
import { instantMeiliSearch } from '@meilisearch/instant-meilisearch';
```

####   
2\. Establish a Meilisearch client with your Meilisearch host and search API key

```JavaScript
const searchClient = instantMeiliSearch( 'https://ms-7053a8dd7c09-72.lon.meilisearch.io', 'meilisearchApiKey' )
```

#### 3\. Set up instantsearch with your Meilisearch index name and the search client  

```Javascript
function App() {
  return (
    <InstantSearch indexName="products" searchClient={searchClient}>
      {/* Widgets */}
    </InstantSearch>
  );
}

```

To have a better understanding of how InstantSearch and instant-meilisearch work together, you can refer to our [dedicated blog post](/blog/instant-meilisearch/

In this guide we’ll use InstantSearch’s `HierarchicalMenu` widget to display categories in a hierarchical fashion.

This widget has a required prop called `attributes`. It takes an array of string values. Each string identifies a specific attribute from your data structure, intended to be used in generating the hierarchical menu.

```JavaScript
<HierarchicalMenu
    attributes={[
        "hierarchicalCategories.lvl0",
        "hierarchicalCategories.lvl1",
        "hierarchicalCategories.lvl2",
        "hierarchicalCategories.lvl3"
    ]}
/>
```

With this widget you can create a menu akin to the example below:

![Screenshot of a hierarchical faceted search interface under 'Categories'. 'Electronics' is the main category with 2874 items, under which 'Image & Sound' is a subcategory with 1047 items. Further nested, 'Television' is highlighted with 689 items, which breaks down into an additional level showing 'TV' with 689 items.](https://lh5.googleusercontent.com/sv4j33IY1r0fpIqWnFRZpT_ntj1ccjqPKnzQuI40BvzH2Ko26pTfsYDCbal8llJsoc2y3RuQACarskaoOJrtMJOrV0Tr6bf8yLwVFeb-s_tEudtr7UIdXpufpoR7SWMGjRHhc40fpMJ-Teb2-Gwe5dE)

* * *

This guide equips you with the fundamental understanding and steps needed to implement a hierarchical faceted search using Meilisearch and InstantSearch, enhancing user navigation through layered, intuitive categorization.

For more things Meilisearch, you can subscribe to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e&ref=blog.meilisearch.com). You can learn more about our product by checking out the [roadmap](https://roadmap.meilisearch.com/#blog.meilisearch.com) and participating in our [product discussions](https://github.com/meilisearch/product/discussions#blog.meilisearch.com).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/#blog.meilisearch.com).

Guide to hierarchical faceted search

  
In this guide, we’ll delve into the concept of faceting in [Meilisearch](https://meilisearch.com.cn/) and how to use it to display facet names while filtering by IDs.

**What is faceting?**
---------------------

[Faceting](https://meilisearch.com.cn/docs/learn/fine_tuning_results/faceted_search) is a technique used in search engines to classify search results into multiple categories or “facets”. These facets can be anything from categories, tags, price ranges, to even colors. This makes it easier for the user to navigate and filter through the results, providing a more refined and efficient search experience.

**Why use the ID as a facet filter?**
-------------------------------------

For most applications and users, filtering by facet names, such as the genre of movies, is often sufficient and intuitive. Facet names are human-readable and provide a clear idea of what the filter does. However, in some use cases, it is preferable to filter by ID for a couple of key reasons:

*   **Less prone to errors**: IDs are usually simpler and standardized, making them less susceptible to typos or inconsistencies that might be present in facet names.
*   **Unique identifiers**: In some databases, facet names might be repeated with slightly different characteristics, whereas IDs are always unique. This is especially useful in cases where you have items with similar or identical names but different properties.

By using IDs for filtering but displaying the corresponding facet names to the users, you can achieve the best of both worlds: efficiency and usability. This way, you are leveraging the strengths of both IDs and names, making your application robust and user-friendly.

**The ID-to-name challenge**
----------------------------

In Meilisearch, facets are a specialized use-case of filters. Essentially, you can use any attribute added to the [`filterableAttributes`](https://meilisearch.com.cn/docs/reference/api/settings#filterable-attributes) list as a facet. When you add a facet parameter to your search query, Meilisearch will return a `[facetDistribution](https://meilisearch.com.cn/docs/reference/api/search#facetdistribution)` object. This object provides the number of matching documents distributed among the values of the given facet.

```JSON
 "facetDistribution":{
    "genres":{
      "Classics":6,
      "Comedy":1,
      "Coming-of-Age":1,
      "Fantasy":2,
      "Fiction":8,
      …
    }
  }
```

However, if the field you’ve added to the filterableAttributes list is an ID, the `facetDistribution` object will return these IDs.

```JSON
 "facetDistribution":{
    "genres":{
      "5":6,
      "6":1,
      "8":1,
      "13":2,
      "16":8,
      …
    }
  }
```

While IDs are great for back-end operations, they are not necessarily user-friendly or meaningful on the front-end. This is why you might want to map these IDs back to their corresponding facet names when displaying them in your user interface.

**ID-to-name mapping in the frontend**
--------------------------------------

Suppose you have a dataset of movies following the structure below:

```JSON
{
    "id": 5,
    "title": "Four Rooms",
    "overview": "It's Ted the Bellhop's first night on the job....",
    "genres": [
        {
            "id": 7,
            "name": "Crime"
        },
        {
            "id": 6,
            "name": "Comedy"
        },
    ],
    "release_date": 818467200,
}
```

You want a faceted search based on the movies genres. So you add genres.id to the [filterable attributes](https://meilisearch.com.cn/docs/reference/api/settings#filterable-attributes) list. Of course, in the UI you want to display genres.name, so that the user will see “Crime” on the UI instead of “7”.

Let’s dive into how you can accomplish this with InstantSearch and instant-meilisearch.

### **Using InstantSearch and instant-meilisearch**

[InstantSearch](https://github.com/algolia/instantsearch) is an open-source front-end library to build search UIs. [Instant-meilisearch](https://github.com/meilisearch/meilisearch-js-plugins/tree/main/packages/instant-meilisearch) is the go-to search client for integrating InstantSearch with Meilisearch.

To use instant-meilisearch with InstantSearch, you need to:  

**1\. Import the required modules, including instantMeiliSearch**

```JavaScript
import { instantMeiliSearch } from '@meilisearch/instant-meilisearch'
import instantsearch from 'instantsearch.js'
import { searchBox, infiniteHits,refinementList } from 'instantsearch.js/es/widgets'
```

  
**2\. Establish a Meilisearch client with your Meilisearch host and search API key**

```JavaScript
const searchClient = instantMeiliSearch(
  'https://ms-7053a8dd7c09-72.lon.meilisearch.io',
  'meilisearchApiKey'
)

```

**3\. Set up instantsearch with your Meilisearch index name and the search client**

```JavaScript
const searchIndex = instantsearch({
  indexName: 'movies',
  searchClient
})

```

In this guide, we’ll use InstantSearch’s `refinementList` widget to map IDs to user-friendly names.

This widget comes with an optional parameter called `transformItems`. This function receives the `items` (or facets) and allows you to transform them before they are displayed on the UI. It also includes the full results data in its parameters.

Each item—or facet—includes the following properties:

*   `count`: the number of occurrences of the facet in the result set
*   `value`: the value used for refining (in our case it would be genres.id)
*   `label`: the label to display
*   `highlighted`: the highlighted label. This value is displayed in the default template

As you can see `highlighted` is the label used by default in the `refinementlList` widget.

With this information, we can use the `transformItems` function to display the `genres.name` instead of `genres.id`.

```JavaScript
transformItems(items, { results }) {
    // The 'results' parameter contains the full results data
    return items.map(item => {
      // Initialize genreName with the existing highlighted label
      let genreName = item.highlighted;
      
      // Loop through results.hits to find a matching genre
      for (let hit of results.hits) {
        const matchedGenre = hit.genres.find(genre => genre.id.toString() === item.value);
        
        // Update genreName if a match is found
        if (matchedGenre) {
          genreName = matchedGenre.name;
          break;
        }
      }
      
      // Return the updated item with the new tagName as the highlighted value
      return {
        ...item,
        highlighted: genreName
      };
    });
  }
```

With this configuration, you’ll efficiently map IDs to more user-friendly names, enhancing the search experience for your users.

The full code should look like this:

```JavaScript
import { instantMeiliSearch } from '@meilisearch/instant-meilisearch'
import instantsearch from 'instantsearch.js'
import { searchBox, infiniteHits,refinementList } from 'instantsearch.js/es/widgets'

const searchClient = instantMeiliSearch(
  'https://ms-7053a8dd7c09-72.lon.meilisearch.io',
  'meilisearchApiKey'
)
  
const searchIndex = instantsearch({
  indexName: 'movies',
  searchClient
})

const searchBox = instantsearch.widgets.searchBox({
  // ...
});

const hits = instantsearch.widgets.hits({
  // ...
});

const refinementList = instantsearch.widgets.refinementList({
  container: '#facets',
  attribute: 'genres.id',
  transformItems(items, { results }) {
    return items.map(item => {
      let genreName = item.highlighted; 
      for (let hit of results.hits) {
        const matchedGenre = hit.genres.find(genre => genre.id.toString() === item.value);
        if (matchedGenre) {
          genreName = matchedGenre.name;
          break;
        }
      }
      return {
        ...item,
        highlighted: genreName
      };
    });
  }
})

searchIndex.addWidgets([searchBox, infiniteHits, refinementList]);

searchIndex.start()

```

This example shows how to implement ID-to-name mapping with [vanilla JavaScript](https://www.algolia.com/doc/api-reference/widgets/refinement-list/js/#widget-param-transformitems), but you can achieve similar results with your preferred frontend framework. Checkout the respective documentation for [React](https://www.algolia.com/doc/api-reference/widgets/refinement-list/react/#widget-param-transformitems) and [Vue](https://www.algolia.com/doc/api-reference/widgets/refinement-list/vue/#widget-param-transform-items).

For more things Meilisearch, you can subscribe to our [newsletter](https://meilisearch.us2.list-manage.com/subscribe?u=27870f7b71c908a8b359599fb&id=79582d828e). You can learn more about our product by checking out the [roadmap](https://roadmap.meilisearch.com/) and participating in our [product discussions](https://github.com/meilisearch/product/discussions).

For anything else, join our developers community on [Discord](https://discord.meilisearch.com/).

Meilisearch 最新消息和公司动态

Meilisearch 使用二值量化将嵌入索引速度提高7倍

如何向 React 应用添加 AI 驱动的搜索

Meilisearch 太慢了

使用 Blazity 构建您的 Next.js Shopify 店面

Meilisearch 1.8

Meilisearch 1.7

Laravel 全文搜索完整指南

推出团队功能 — Meilisearch Cloud

分层分面搜索指南

使用 InstantSearch 通过 ID 优化分面