From accfa8c4007646fbdad4e6203d12c03a1d0704d9 Mon Sep 17 00:00:00 2001 From: Tom Aarsen Date: Fri, 11 Apr 2025 09:49:04 +0200 Subject: [PATCH 1/5] Add ONNX & OpenVINO support for Cross Encoder (reranker) models --- docs/cross_encoder/usage/efficiency.rst | 602 ++++++++++++++++++ docs/cross_encoder/usage/usage.rst | 3 +- docs/img/ce_backends_benchmark_cpu.png | Bin 0 -> 58081 bytes docs/img/ce_backends_benchmark_gpu.png | Bin 0 -> 55245 bytes .../sentence_transformer/usage/efficiency.rst | 10 +- docs/sentence_transformer/usage/usage.rst | 2 +- sentence_transformers/backend.py | 173 +++-- .../cross_encoder/CrossEncoder.py | 249 +++++++- 8 files changed, 981 insertions(+), 58 deletions(-) create mode 100644 docs/cross_encoder/usage/efficiency.rst create mode 100644 docs/img/ce_backends_benchmark_cpu.png create mode 100644 docs/img/ce_backends_benchmark_gpu.png diff --git a/docs/cross_encoder/usage/efficiency.rst b/docs/cross_encoder/usage/efficiency.rst new file mode 100644 index 000000000..7e13de733 --- /dev/null +++ b/docs/cross_encoder/usage/efficiency.rst @@ -0,0 +1,602 @@ + +Speeding up Inference +===================== + +Sentence Transformers supports 3 backends for performing inference with Cross Encoder models, each with its own optimizations for speeding up inference: + + +.. raw:: html + +
+ +
PyTorch
+ The default backend for Cross Encoders. + + +
ONNX
+ Flexible and efficient model accelerator. + + +
OpenVINO
+ Optimization of models, mainly for Intel Hardware. + + +
Benchmarks
+ Benchmarks for the different backends. + + +
User Interface
+ GUI to export, optimize, and quantize models. + +
+
+ +PyTorch +------- + +The PyTorch backend is the default backend for Cross Encoders. If you don't specify a device, it will use the strongest available option across "cuda", "mps", and "cpu". Its default usage looks like this: + +.. code-block:: python + + from sentence_transformers import CrossEncoder + + model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2") + + query = "Which planet is known as the Red Planet?" + passages = [ + "Venus is often called Earth's twin because of its similar size and proximity.", + "Mars, known for its reddish appearance, is often referred to as the Red Planet.", + "Jupiter, the largest planet in our solar system, has a prominent red spot.", + "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." + ] + + scores = model.predict([(query, passage) for passage in passages]) + print(scores) + +If you're using a GPU, then you can use the following options to speed up your inference: + +.. tab:: float16 (fp16) + + Float32 (fp32, full precision) is the default floating-point format in ``torch``, whereas float16 (fp16, half precision) is a reduced-precision floating-point format that can speed up inference on GPUs at a minimal loss of model accuracy. To use it, you can specify the ``torch_dtype`` during initialization or call :meth:`model.half() ` on the initialized model: + + .. code-block:: python + + from sentence_transformers import CrossEncoder + + model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2", model_kwargs={"torch_dtype": "float16"}) + # or: model.half() + + query = "Which planet is known as the Red Planet?" + passages = [ + "Venus is often called Earth's twin because of its similar size and proximity.", + "Mars, known for its reddish appearance, is often referred to as the Red Planet.", + "Jupiter, the largest planet in our solar system, has a prominent red spot.", + "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." + ] + + scores = model.predict([(query, passage) for passage in passages]) + print(scores) + +.. tab:: bfloat16 (bf16) + + Bfloat16 (bf16) is similar to fp16, but preserves more of the original accuracy of fp32. To use it, you can specify the ``torch_dtype`` during initialization or call :meth:`model.bfloat16() ` on the initialized model: + + .. code-block:: python + + from sentence_transformers import CrossEncoder + + model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2", model_kwargs={"torch_dtype": "bfloat16"}) + # or: model.bfloat16() + + query = "Which planet is known as the Red Planet?" + passages = [ + "Venus is often called Earth's twin because of its similar size and proximity.", + "Mars, known for its reddish appearance, is often referred to as the Red Planet.", + "Jupiter, the largest planet in our solar system, has a prominent red spot.", + "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." + ] + + scores = model.predict([(query, passage) for passage in passages]) + print(scores) + +ONNX +---- + +.. include:: backend_export_sidebar.rst + +ONNX can be used to speed up inference by converting the model to ONNX format and using ONNX Runtime to run the model. To use the ONNX backend, you must install Sentence Transformers with the ``onnx`` or ``onnx-gpu`` extra for CPU or GPU acceleration, respectively: + +.. code-block:: bash + + pip install sentence-transformers[onnx-gpu] + # or + pip install sentence-transformers[onnx] + +To convert a model to ONNX format, you can use the following code: + +.. code-block:: python + + from sentence_transformers import CrossEncoder + + model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2", backend="onnx") + + query = "Which planet is known as the Red Planet?" + passages = [ + "Venus is often called Earth's twin because of its similar size and proximity.", + "Mars, known for its reddish appearance, is often referred to as the Red Planet.", + "Jupiter, the largest planet in our solar system, has a prominent red spot.", + "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." + ] + + scores = model.predict([(query, passage) for passage in passages]) + print(scores) + +If the model path or repository already contains a model in ONNX format, Sentence Transformers will automatically use it. Otherwise, it will convert the model to the ONNX format. + +.. note:: + + If you wish to use the ONNX model outside of Sentence Transformers, you might need to apply your chosen activation function (e.g. Sigmoid) to get identical results as the Cross Encoder in Sentence Transformers. + +All keyword arguments passed via ``model_kwargs`` will be passed on to :meth:`ORTModelForSequenceClassification.from_pretrained `. Some notable arguments include: + +* ``provider``: ONNX Runtime provider to use for loading the model, e.g. ``"CPUExecutionProvider"`` . See https://onnxruntime.ai/docs/execution-providers/ for possible providers. If not specified, the strongest provider (E.g. ``"CUDAExecutionProvider"``) will be used. +* ``file_name``: The name of the ONNX file to load. If not specified, will default to ``"model.onnx"`` or otherwise ``"onnx/model.onnx"``. This argument is useful for specifying optimized or quantized models. +* ``export``: A boolean flag specifying whether the model will be exported. If not provided, ``export`` will be set to ``True`` if the model repository or directory does not already contain an ONNX model. + +.. tip:: + + It's heavily recommended to save the exported model to prevent having to re-export it every time you run your code. You can do this by calling :meth:`model.save_pretrained() ` if your model was local: + + .. code-block:: python + + model = CrossEncoder("path/to/my/model", backend="onnx") + model.save_pretrained("path/to/my/model") + + or with :meth:`model.push_to_hub() ` if your model was from the Hugging Face Hub: + + .. code-block:: python + + model = CrossEncoder("Alibaba-NLP/gte-reranker-modernbert-base", backend="onnx") + model.push_to_hub("Alibaba-NLP/gte-reranker-modernbert-base", create_pr=True) + +Optimizing ONNX Models +^^^^^^^^^^^^^^^^^^^^^^ + +.. include:: backend_export_sidebar.rst + +ONNX models can be optimized using Optimum, allowing for speedups on CPUs and GPUs alike. To do this, you can use the :func:`~sentence_transformers.backend.export_optimized_onnx_model` function, which saves the optimized in a directory or model repository that you specify. It expects: + +- ``model``: a Sentence Transformer or Cross Encoder model loaded with the ONNX backend. +- ``optimization_config``: ``"O1"``, ``"O2"``, ``"O3"``, or ``"O4"`` representing optimization levels from :class:`~optimum.onnxruntime.AutoOptimizationConfig`, or an :class:`~optimum.onnxruntime.OptimizationConfig` instance. +- ``model_name_or_path``: a path to save the optimized model file, or the repository name if you want to push it to the Hugging Face Hub. +- ``push_to_hub``: (Optional) a boolean to push the optimized model to the Hugging Face Hub. +- ``create_pr``: (Optional) a boolean to create a pull request when pushing to the Hugging Face Hub. Useful when you don't have write access to the repository. +- ``file_suffix``: (Optional) a string to append to the model name when saving it. If not specified, the optimization level name string will be used, or just ``"optimized"`` if the optimization config was not just a string optimization level. + +See this example for exporting a model with :doc:`optimization level 3 ` (basic and extended general optimizations, transformers-specific fusions, fast Gelu approximation): + +.. tab:: Hugging Face Hub Model + + Only optimize once:: + + from sentence_transformers import CrossEncoder, export_optimized_onnx_model + + model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2", backend="onnx") + export_optimized_onnx_model( + model, + "O3", + "cross-encoder/ms-marco-MiniLM-L6-v2", + push_to_hub=True, + create_pr=True, + ) + + Before the pull request gets merged:: + + from sentence_transformers import CrossEncoder + + pull_request_nr = 2 # TODO: Update this to the number of your pull request + model = CrossEncoder( + "cross-encoder/ms-marco-MiniLM-L6-v2", + backend="onnx", + model_kwargs={"file_name": "onnx/model_O3.onnx"}, + revision=f"refs/pr/{pull_request_nr}" + ) + + Once the pull request gets merged:: + + from sentence_transformers import CrossEncoder + + model = CrossEncoder( + "cross-encoder/ms-marco-MiniLM-L6-v2", + backend="onnx", + model_kwargs={"file_name": "onnx/model_O3.onnx"}, + ) + +.. tab:: Local Model + + Only optimize once:: + + from sentence_transformers import CrossEncoder, export_optimized_onnx_model + + model = CrossEncoder("path/to/my/mpnet-legal-finetuned", backend="onnx") + export_optimized_onnx_model(model, "O3", "path/to/my/mpnet-legal-finetuned") + + After optimizing:: + + from sentence_transformers import CrossEncoder + + model = CrossEncoder( + "path/to/my/mpnet-legal-finetuned", + backend="onnx", + model_kwargs={"file_name": "onnx/model_O3.onnx"}, + ) + +Quantizing ONNX Models +^^^^^^^^^^^^^^^^^^^^^^ + +.. include:: backend_export_sidebar.rst + +ONNX models can be quantized to int8 precision using Optimum, allowing for faster inference on CPUs. To do this, you can use the :func:`~sentence_transformers.backend.export_dynamic_quantized_onnx_model` function, which saves the quantized in a directory or model repository that you specify. Dynamic quantization, unlike static quantization, does not require a calibration dataset. It expects: + +- ``model``: a Sentence Transformer or Cross Encoder model loaded with the ONNX backend. +- ``quantization_config``: ``"arm64"``, ``"avx2"``, ``"avx512"``, or ``"avx512_vnni"`` representing quantization configurations from :class:`~optimum.onnxruntime.AutoQuantizationConfig`, or an :class:`~optimum.onnxruntime.QuantizationConfig` instance. +- ``model_name_or_path``: a path to save the quantized model file, or the repository name if you want to push it to the Hugging Face Hub. +- ``push_to_hub``: (Optional) a boolean to push the quantized model to the Hugging Face Hub. +- ``create_pr``: (Optional) a boolean to create a pull request when pushing to the Hugging Face Hub. Useful when you don't have write access to the repository. +- ``file_suffix``: (Optional) a string to append to the model name when saving it. If not specified, ``"qint8_quantized"`` will be used. + +On my CPU, each of the default quantization configurations (``"arm64"``, ``"avx2"``, ``"avx512"``, ``"avx512_vnni"``) resulted in roughly equivalent speedups. + +See this example for quantizing a model to ``int8`` with :doc:`avx512_vnni `: + +.. tab:: Hugging Face Hub Model + + Only quantize once:: + + from sentence_transformers import CrossEncoder, export_dynamic_quantized_onnx_model + + model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2", backend="onnx") + export_dynamic_quantized_onnx_model( + model, + "avx512_vnni", + "sentence-transformers/cross-encoder/ms-marco-MiniLM-L6-v2", + push_to_hub=True, + create_pr=True, + ) + + Before the pull request gets merged:: + + from sentence_transformers import CrossEncoder + + pull_request_nr = 2 # TODO: Update this to the number of your pull request + model = CrossEncoder( + "cross-encoder/ms-marco-MiniLM-L6-v2", + backend="onnx", + model_kwargs={"file_name": "onnx/model_qint8_avx512_vnni.onnx"}, + revision=f"refs/pr/{pull_request_nr}", + ) + + Once the pull request gets merged:: + + from sentence_transformers import CrossEncoder + + model = CrossEncoder( + "cross-encoder/ms-marco-MiniLM-L6-v2", + backend="onnx", + model_kwargs={"file_name": "onnx/model_qint8_avx512_vnni.onnx"}, + ) + +.. tab:: Local Model + + Only quantize once:: + + from sentence_transformers import CrossEncoder, export_dynamic_quantized_onnx_model + + model = CrossEncoder("path/to/my/mpnet-legal-finetuned", backend="onnx") + export_dynamic_quantized_onnx_model(model, "O3", "path/to/my/mpnet-legal-finetuned") + + After quantizing:: + + from sentence_transformers import CrossEncoder + + model = CrossEncoder( + "path/to/my/mpnet-legal-finetuned", + backend="onnx", + model_kwargs={"file_name": "onnx/model_qint8_avx512_vnni.onnx"}, + ) + +OpenVINO +-------- + +.. include:: backend_export_sidebar.rst + +OpenVINO allows for accelerated inference on CPUs by exporting the model to the OpenVINO format. To use the OpenVINO backend, you must install Sentence Transformers with the ``openvino`` extra: + +.. code-block:: bash + + pip install sentence-transformers[openvino] + +To convert a model to OpenVINO format, you can use the following code: + +.. code-block:: python + + from sentence_transformers import CrossEncoder + + model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2", backend="openvino") + + query = "Which planet is known as the Red Planet?" + passages = [ + "Venus is often called Earth's twin because of its similar size and proximity.", + "Mars, known for its reddish appearance, is often referred to as the Red Planet.", + "Jupiter, the largest planet in our solar system, has a prominent red spot.", + "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." + ] + + scores = model.predict([(query, passage) for passage in passages]) + print(scores) + +If the model path or repository already contains a model in OpenVINO format, Sentence Transformers will automatically use it. Otherwise, it will convert the model to the OpenVINO format. + +.. note:: + + If you wish to use the OpenVINO model outside of Sentence Transformers, you might need to apply your chosen activation function (e.g. Sigmoid) to get identical results as the Cross Encoder in Sentence Transformers. + +.. raw:: html + + All keyword arguments passed via model_kwargs will be passed on to OVBaseModel.from_pretrained(). Some notable arguments include: + +* ``file_name``: The name of the ONNX file to load. If not specified, will default to ``"openvino_model.xml"`` or otherwise ``"openvino/openvino_model.xml"``. This argument is useful for specifying optimized or quantized models. +* ``export``: A boolean flag specifying whether the model will be exported. If not provided, ``export`` will be set to ``True`` if the model repository or directory does not already contain an OpenVINO model. + +.. tip:: + + It's heavily recommended to save the exported model to prevent having to re-export it every time you run your code. You can do this by calling :meth:`model.save_pretrained() ` if your model was local: + + .. code-block:: python + + model = CrossEncoder("path/to/my/model", backend="openvino") + model.save_pretrained("path/to/my/model") + + or with :meth:`model.push_to_hub() ` if your model was from the Hugging Face Hub: + + .. code-block:: python + + model = CrossEncoder("Alibaba-NLP/gte-reranker-modernbert-base", backend="openvino") + model.push_to_hub("Alibaba-NLP/gte-reranker-modernbert-base", create_pr=True) + +Quantizing OpenVINO Models +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. include:: backend_export_sidebar.rst + +OpenVINO models can be quantized to int8 precision using Optimum Intel to speed up inference. +To do this, you can use the :func:`~sentence_transformers.backend.export_static_quantized_openvino_model` function, +which saves the quantized model in a directory or model repository that you specify. +Post-Training Static Quantization expects: + +- ``model``: a Sentence Transformer or Cross Encoder model loaded with the OpenVINO backend. +- ``quantization_config``: (Optional) The quantization configuration. This parameter accepts either: + ``None`` for the default 8-bit quantization, a dictionary representing quantization configurations, or + an :class:`~optimum.intel.OVQuantizationConfig` instance. +- ``model_name_or_path``: a path to save the quantized model file, or the repository name if you want to push it to the Hugging Face Hub. +- ``dataset_name``: (Optional) The name of the dataset to load for calibration. If not specified, defaults to ``sst2`` subset from the ``glue`` dataset. +- ``dataset_config_name``: (Optional) The specific configuration of the dataset to load. +- ``dataset_split``: (Optional) The split of the dataset to load (e.g., 'train', 'test'). +- ``column_name``: (Optional) The column name in the dataset to use for calibration. +- ``push_to_hub``: (Optional) a boolean to push the quantized model to the Hugging Face Hub. +- ``create_pr``: (Optional) a boolean to create a pull request when pushing to the Hugging Face Hub. Useful when you don't have write access to the repository. +- ``file_suffix``: (Optional) a string to append to the model name when saving it. If not specified, ``"qint8_quantized"`` will be used. + +See this example for quantizing a model to ``int8`` with `static quantization `_: + +.. tab:: Hugging Face Hub Model + + Only quantize once:: + + from sentence_transformers import CrossEncoder, export_static_quantized_openvino_model + + model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2", backend="openvino") + export_static_quantized_openvino_model( + model, + quantization_config=None, + model_name_or_path="cross-encoder/ms-marco-MiniLM-L6-v2", + push_to_hub=True, + create_pr=True, + ) + + Before the pull request gets merged:: + + from sentence_transformers import CrossEncoder + + pull_request_nr = 2 # TODO: Update this to the number of your pull request + model = CrossEncoder( + "cross-encoder/ms-marco-MiniLM-L6-v2", + backend="openvino", + model_kwargs={"file_name": "openvino/openvino_model_qint8_quantized.xml"}, + revision=f"refs/pr/{pull_request_nr}" + ) + + Once the pull request gets merged:: + + from sentence_transformers import CrossEncoder + + model = CrossEncoder( + "cross-encoder/ms-marco-MiniLM-L6-v2", + backend="openvino", + model_kwargs={"file_name": "openvino/openvino_model_qint8_quantized.xml"}, + ) + +.. tab:: Local Model + + Only quantize once:: + + from sentence_transformers import CrossEncoder, export_static_quantized_openvino_model + from optimum.intel import OVQuantizationConfig + + model = CrossEncoder("path/to/my/mpnet-legal-finetuned", backend="openvino") + quantization_config = OVQuantizationConfig() + export_static_quantized_openvino_model(model, quantization_config, "path/to/my/mpnet-legal-finetuned") + + After quantizing:: + + from sentence_transformers import CrossEncoder + + model = CrossEncoder( + "path/to/my/mpnet-legal-finetuned", + backend="openvino", + model_kwargs={"file_name": "openvino/openvino_model_qint8_quantized.xml"}, + ) + +Benchmarks +---------- + +The following images show the benchmark results for the different backends on GPUs and CPUs. The results are averaged across 4 models of various sizes, 3 datasets, and numerous batch sizes. + +.. raw:: html + +
+ Expand the benchmark details + +
+ Speedup ratio: +
    +
  • + Hardware: RTX 3090 GPU, i7-17300K CPU +
    • +
  • + Datasets: 2000 samples for GPU tests, 1000 samples for CPU tests. +
      +
    • + sentence-transformers/stsb: ``sentence1`` and ``sentence2`` columns as pairs, with 38.94 ± 13.97 and 38.96 ± 14.05 characters on average, respectively. +
      • +
    • + sentence-transformers/natural-questions: ``query`` and ``answer`` columns as pairs, with 46.99 ± 10.98 and 619.63 ± 345.30 characters on average, respectively. +
      • +
    • + stanfordnlp/imdb: Two variants used from the ``text`` column: first 100 characters (100.00 ± 0.00 characters) and each sample repeated 4 times (16804.25 ± 10178.26 characters). +
      • +
    +
    • +
  • + Models: + +
    • +
+ Performance ratio: The same models and hardware was used. We compare the performance against the performance of PyTorch with fp32, i.e. the default backend and precision. +
    +
  • + Evaluation: +
      +
    • + Information Retrieval: NDCG@10 based on cosine similarity on the MS MARCO and NQ subsets from the NanoBEIR collection of datasets, computed via the CrossEncoderNanoBEIREvaluator. +
      • +
    +
    • +
+ +
    +
  • + Backends: +
      +
    • + torch-fp32: PyTorch with float32 precision (default). +
      • +
    • + torch-fp16: PyTorch with float16 precision, via model_kwargs={"torch_dtype": "float16"}. +
      • +
    • + torch-bf16: PyTorch with bfloat16 precision, via model_kwargs={"torch_dtype": "bfloat16"}. +
      • +
    • + onnx: ONNX with float32 precision, via backend="onnx". +
      • +
    • + onnx-O1: ONNX with float32 precision and O1 optimization, via export_optimized_onnx_model(..., "O1", ...) and backend="onnx". +
      • +
    • + onnx-O2: ONNX with float32 precision and O2 optimization, via export_optimized_onnx_model(..., "O2", ...) and backend="onnx". +
      • +
    • + onnx-O3: ONNX with float32 precision and O3 optimization, via export_optimized_onnx_model(..., "O3", ...) and backend="onnx". +
      • +
    • + onnx-O4: ONNX with float16 precision and O4 optimization, via export_optimized_onnx_model(..., "O4", ...) and backend="onnx". +
      • +
    • + onnx-qint8: ONNX quantized to int8 with "avx512_vnni", via export_dynamic_quantized_onnx_model(..., "avx512_vnni", ...) and backend="onnx". The different quantization configurations resulted in roughly equivalent speedups. +
      • +
    • + openvino: OpenVINO, via backend="openvino". +
      • +
    • + openvino-qint8: OpenVINO quantized to int8 via export_static_quantized_openvino_model(..., OVQuantizationConfig(), ...) and backend="openvino". +
      • +
    +
    • +
+ + Note that the aggressive averaging across models, datasets, and batch sizes prevents some more intricate patterns from being visible. For example, ONNX seems to perform stronger at low batch sizes. However, ONNX and OpenVINO can even perform slightly worse than PyTorch, so we recommend testing the different backends with your specific model and data to find the best one for your use case. + +
+
+ + +.. image:: ../../img/ce_backends_benchmark_gpu.png + :alt: Benchmark for GPUs + :width: 45% + +.. image:: ../../img/ce_backends_benchmark_cpu.png + :alt: Benchmark for CPUs + :width: 45% + +Recommendations +^^^^^^^^^^^^^^^ + +Based on the benchmarks, this flowchart should help you decide which backend to use for your model: + +.. mermaid:: + + %%{init: { + "theme": "neutral", + "flowchart": { + "curve": "bumpY" + } + }}%% + graph TD + A("What is your hardware?") -->|GPU| B("Are you using a small
batch size?") + A -->|CPU| C("Are you open to
quantization?") + B -->|yes| D[onnx-O4] + B -->|no| F[float16] + C -->|yes| G[openvino-qint8] + C -->|no| H("Do you have an Intel CPU?") + H -->|yes| I[openvino] + H -->|no| J[onnx] + click D "#optimizing-onnx-models" + click F "#pytorch" + click G "#quantizing-openvino-models" + click I "#openvino" + click J "#onnx" + +.. note:: + + Your milage may vary, and you should always test the different backends with your specific model and data to find the best one for your use case. + +User Interface +^^^^^^^^^^^^^^ + +This Hugging Face Space provides a user interface for exporting, optimizing, and quantizing models for either ONNX or OpenVINO: + +- `sentence-transformers/backend-export `_ diff --git a/docs/cross_encoder/usage/usage.rst b/docs/cross_encoder/usage/usage.rst index e1522037e..bc19f583c 100644 --- a/docs/cross_encoder/usage/usage.rst +++ b/docs/cross_encoder/usage/usage.rst @@ -73,4 +73,5 @@ Once you have `installed <../../installation.html>`_ Sentence Transformers, you :caption: Tasks Cross-Encoder vs Bi-Encoder <../../../examples/cross_encoder/applications/README> - ../../../examples/sentence_transformer/applications/retrieve_rerank/README \ No newline at end of file + ../../../examples/sentence_transformer/applications/retrieve_rerank/README + efficiency \ No newline at end of file diff --git a/docs/img/ce_backends_benchmark_cpu.png b/docs/img/ce_backends_benchmark_cpu.png new file mode 100644 index 0000000000000000000000000000000000000000..7ef1cee8bea3efebffdcbcbf14ea2cdc70b5b09f GIT binary patch literal 58081 zcmcG$bySpp_cl6&C<-Da4bq(w(nxoA3qyA|2r4Bp4Bd^=f^_>uVnFE-DW$tR&u6~R z^E>CPb^bc%JuhpqbeNetKKH)&zV>xp`;JmqmBW5a`WOO%U@OQ=YeFEXZ^7^IBMk7D zqi(kh@QSF^9K2g@>%?o9R#wZsv!MZ+c#@x{(-NS4H?Ft zh=vi(Jo!+`{HE5{?Jv|)z2Sqfzhn*E&?B^2R}w6nF4!BX@YdE=B4$-1Mdsw2>1!`W zr7?ovV|W33+d?B*S#*8wLz(AarG>IpAJTvycVg8PQxxR?uD%dM@NjW)1&J}(d#@iFqp&ZeVYd_(7$&Mr$h1n7Yfj1x+CLI; z7QB9kA)m+)n^C4)8SS+_)w3nKVqL7Fs;YW$cBU)PWYX&0v)vLN9$p~x@wxBW4he&D z+FS9v3n>bZXfi>u@V=p$8IsNMPkM{=f}BfRA@|q0VxMaW)pEsT22md<0<{#z#nsi)NWN4WuN|KM`JU6JPMu+BXef(%!0i8+^-at|N%-tMyOGR1TA+RZ+~1 z@lQC4sT`GIu$MMN=Djf_TWi64bAqQq(Mk}e*U&@;m6)|gr`hMZ!B@=2CMIv3=&l(S zBdCHeQ0na`qZ`lL121c@G>R~vKJESX?&0BX<6?ncwdvI**nDlehY@O*MxktJXSq(f z_2wisSsw(pq#P`xq!Df0rB$qGbuw2d7thvU+F-ue5!4iL&XK}tR#l5*$n^{l4{!T? zAdwLVT>rw5xjH*70%lEgd`8v4eFzs5ll+5EzYlpuhYk-H;ws!%2UyI}^r1JFOpdvC;_UB-CqxG`Ho4SZTN9tPOX2>iXMWw`*g9)UQ%cdOR%CS+ zRXXLmrG3A}#QctmMgx@Al&84m;>hhu5Lt*zTA7b)Q-6QgJJ|af=v|_W*I3FW8*ta# zj?yGCYhmo|?b!#FYnO(x8#NHK9}kP)Z#0?=u``c}hun!QDK02oJCj_UPMhga@TsY( z)rUaEZ@??r!Ttt?KQ-pxP=iKmbsuuH2)%`~r|hR@`Dh0H8RYDY(Y#mP`WbYDV6^WX zxJ&kstc|+6yW5}3A!OD4?3L`5;-MM*x&PdVc0GY$@G;^&Y1Q#;un3~ImP62E?K@bJ zpBy&&57pz^bt_m&d@jj%FB`LG_mA77ehSiTCy4mD*SpXKu!antZf#e0%KG{?8Op|G z|GmOjs(D{P4Kl!>?I_!>B>os%0gdRvdx-jT#XHMpX}x@_jGHaA zaTKGyR;)QfBj%*(%@aNzTt_3$q~)A>X1?0!p;Qn&=(9QddwINW5AV|N|5;+u6I~X( z2BVgmh?VDwcP}((b}t?vCWU9F?VN+Xm%o)nrShvJ;Ix-X!OQySG}}V#%R% z5Gv#@3vJ)Vij^iij5~wd?~GGU6Ff}9bT3sOVZsV%XBlUid#d1GNkJ^zoFuom*XOiu zOW(J?La$fklC(Erx2&(D1FO)Z@g(HrP-z7j+&lU^QuZ4hBib5ws07!GU+D%*AP(WT zY<;Uv+iZ(&GxzSn16L6rK9ugmyv1Zc8GN}|Lv#T z;4-Cjp5i@FlC}=1LT>Us+}+1^=Ub<3n@L_;e;%Ic4C(X_+?rrMZ34?KI~a5}V`G08 ziA#HO{JYVmdN6~}A;~M%O1YDTdp>8wxFgUYm0sz&37k&BGnqwa?7@Qvn@f^Yug_*c zI`fLB6dj`w3)I*9YOTT-GS}p0PvU3XwhK_ptoU$^#OA^fPfiOI<;ayx@>{ZE6tBXHttodL)QTEFJE8$#Jh`j_gZHQU!(q*k67{Z@rYnQ~t&vZt59& z!NnDF?m1b(GfGOzGL(c7oiwxtiFWV%jFS7hnE>Q=02LVo~Yqr+V>k(uyE!g+_3 z`Re8L+tM_{Mvu|OnN$6 zE7}c2VIgXL49Ux~AO;yt_d*yAL-T9-nwFD~&l810@jMmXn$a0G`{cJg?FdJTZFRyn z_dBjtRYrGNF@N)EViDojx)jc)x@)Cm*4tTKY%I(kggX=a%D1a}9MeDIX){pLC?WWm z!7s+6r}p#YYu%ErpBiRx(Oq`d%kcFoX2JXSXYaa`+SHWgVax)TshnnI#}WGz@FYTs9@BLEHXe3y77#=)?Q{jUgpm%CwWwQD(mtefJ^5WcSJskb5 zrWb7kSV0}EA3^&o9zSmXM3y5L@Qge`V&=r3VZG#UvnoL@F+{TsMiznojz{4comU8* z662|dg<>s-q%((MkE@np5Rri{DMo)Rf_%J-nxUXIy@NfIhzNd~c$J8E69v*#Sw%eh zTR1LUxeS`W5>AD$`B$W6?`2&OOERKx5N_-B;HOpQl9A^=g-I`W%VqfKLwY6JYi*Wu zr4cr-R%k1IVTn_Sln9$aEjp&kR=z@t56DwxxG~UidMWl(R%|liUy1)Y&zdRS-_TCv2_~kv z8{+0Sn|%*2`W9YSI}v=Th@pg9<#YOwfkYJXO7Y;Nfh`i<(49J^BJK^pBuiT++-!DD z_QgmoNr^?R=vx2PKu5yko!=@FVZ;a~d6@t^1-9)1os|$O_)lCsuJ1Nf4w4Q!cFtys z+X{B4=_4j@44%--?Cw?fuGh?6msI8usG zVqo^Ta0+-#*&A7XR0%EapN~i>g?Gtu;fU0!<&s&t&@yF!>a@N!@W#PTKkmsz|Gmz% z?t+>c3mK;DC!rgx#Z@+YjEpU2zJDdRlyOMU6X6A69x8hJvAH5jm%!5ShW7LU_BNZs zr%|{LOgG3-+-hw8{oFy(qlu)c_|)fI3KZmvlb3aaa-Z!QbZYSZ~qGqYY zqk;EAp^-K`^eeeAf!zRW-bHT`bg76~*RVLl`dR&O!ZMwV@0h=~kXu~! zEz5dd;KQN^mASdO!DOqwE@@FI9^dC8soaOYLy7qWAJRK;Eht-BG=!)wEqYpGQzApIdK2~5xhQinY74HvD-A!R3AoUZ+j9b2_#ZB+VdeM^(EidX{Dp}avdpe1P0-Z~?<{E`Y zN;o%MNUWsDXc%h1b9bj_|k!RCJvkHRl*-X;9uy zG{R~Ap3`?bC?pijHLZT7Z>2VImxr+4QY$j>RRv$J5)nd`(4KCV$!%lEZbbw|-XT^n zSY-+|9XSLHN)ze7eR@L`Swk-qFeVxmqfFf=(J~8LW}PpTx5UPrTlH9-gB^d7j>6lT zOTbD-XBg{x9rt&lRBQR@f-@V{k7ueRDT13XQ6W&vMb+pY$}-zY0ivwjK$USj}LX z))%B4yHKo>;@%AZ$Q1pmk%}P#2`N8w@ExrVep1R2Y2MP6Y^m9pZ*ANII9YaZp!94t z)oklIyJ1m_x~rF$jtM!j_j-Q9^gykcm{`mADEw_6Xn$S{^y2n>vCxx`+j~df z>@>Era$_I)=Gp43OXuY3k8~}k!U`kMM{A!`<>%+Ou;`S1zMO4v3fR83-9F@)t~S$~ zukG(hI{%r(oLKdyJG`ay?uu1*-c5R1CmiNgp<(EpxCvKEpcO{1zQ<(T>r|cXmX~q??ar;!3@pbt(;Y8Ubs; zbY1K`-?HUV9Dpa7Lm}kJwME&V)wrvsQEA9m&uP}?n@>5M{!%U^2XU@4H8bbuw>d`d zwbx$*Td#9gFPN3xJd;^a8^Q#Qt_gsYp2NA~)8Zk^z1XJ|sk`u9CBTnV`JZm>i*WJ* zQe%8y%5Qf)H>4^qHUF0<8m;&KLS@JL5yj60+K8Bf? z^-L{t5^sHR~qoMjq!WJ^SU2)W@oKo+qr?O&x)kV+Gr#i}yKr*N+YXDzSb!eW(4>%=1-5+10!)E;8QJ(iU3pE8~D0`#WebcN+m1Y9Oo z+H-57lxgtxYP;H6Txo6axRzYQ(Us{{G(HKVPI$Wi?NxmXvx0e(%c2f$nvz!AX~jKiRLd#z3RPl$#05yY=*FmUM!O|I!tp61d{wS zFzUHIg>4pcTTeQ`CKFA-?H`Lp%OL&?Dq=xmW7ylCa=g@^Kx-cZ{SQd*PnVI#nckdCR*cCR3sW{#HCdihe8IV>3ly^{OV@$k?jz#-r3^!_F^ zY4Lmwn3~C7xuC-Yz}fL~VV{bCpa`R(QD)!J2XTOSl?NB(qd?OQ>%_!o$TRB<)^KLw zxepRMmr9gSBh#9~Ma0u0?&LAloNwTVCN&l^F?~LO#A?1WwaehOE9ebOJ~uZv&%V3b z<|Mp9)EL!5qe$^*4zAfPlLpf!!=YK`-TP6-WKc(IOmv%}tjpKax#UkizGXJ;|Fg4n zb-FEk%xTh6#1Rfhz<=bS$Kugtu<9jOQrpa)b`RQ0>=3-Y+>b}o9g zoLA@j)kF9*{LOy{xVhxY`1^wokB&6=4ZfJa{*f&#pqP+Ljx%G1gdMXlX7Qdq%gagi zXpv8#(Qoy3$~sRbV07?j$X>%E`t3glwD&($hbi2qf63S(DuW-N;GH5g4$2P+u1-M=H z!9o5a#pK7}TId}sUtV68Zaq5^g;`RsODy-sT58`k12*hdXwasAn{L@!{@N5Ylq!|+ zY>eAeksj1fY5+0kBp=LT-TiF<$@1J~Da+i|QR%%-+P=9tpF z?ouN>CQ{*88bqed14)_R29jBIT&?9%MVnvuW2-o?ENzSyzH)R{$fvwKt zccsZ#B6y&iI=39FnoV^-pTYWJRbn~^2~;tEde?KZOpmf4Nhe5+HcQ>lVd8- z>+pO*z=c*!h$m*wgEO1t0jXIM^w<-3DVDSXLNjLJUP%qk^JUSW?58S9Yg;83q{9Am z1YLJ{%+r|%9IYrMUE;|rS}PylTzU{j%5yQ!SRx9rC`I%iu()HrN5^ts21IbH5Yfsp z8XDT9kYYU=rnl8KJuLChCqON+C^3)_VQ$es;{K60AO`AB!s8rP?az5}jp*DZlwyIA zfCAkllk2~OThi}ye#^G|!V%i&n29ck2PJj>l%)hx$ZKWMfHP&PCdAcXfYKl9v^S|< zX`%rQLaN}J>0CpHvIJR1xY$`zyi%wXYt^i-R|6A045CTv9XpsYtykyZRE$S=OY!1& z>L?k3;aNm@#0VeF;I~;Keww#`M*7_nR|itq%TW&jL-qzxA6ves8>3R}w0$aJ+@+E< zi`_Su$K}V`V%{YX3RPQl+eq?a{HNqI%IwgqV!0xmhyBrRa? zo0Q_g700Kiled5fZSmY3+XOWEq@~Kw3_dSF)K0Yr-x`W@($GliYg>7<`hV>IRw#fU`!`pkR-S<#ZoD-~O3kEjn=T1hI zj3T8Zri48}z;hYO8FaRIZIiaARw$kA%y`v1OivAyaa+8a)H7nIme%9&)y0 zPKjJ~)VU zSGJH`5N_E7Fl>TT577NgeLQq@(y2g}m}krlEXWn`*)y)6yEt4f3{3HknW``-Ot}OE zcx(>H5Y0lpK*D(WTMk44ui@U;VKJ>^t3`VupS_B~H15JZL?WZ?VDQ!ErxuVQwuXnZ zg?-AV*ZLE>cLZ6%?Qvgf1;UKD(9%H&ES4gY9xAk2{ndZRa{>Z7J2f@6O6!yAm7n|E zswm*E@OSU#gN()9uaiueZ^W`$@TClqQBO4Cc(Gcp>G{c~ z)@|GPC$;ij2XJOdK`u@E(&Hs6UO72AF1ImY^~<%2Sby_bqi~{L`|K|g z4%{VNvg%dw8gjJkrd%GivB?pT0Ttp^0*x{vTwWs+nnBmfDAQ_)TMXr-E2rr!qGzQ} zf016qQwK#~nXa`fe}mwX*Z4%uoG~FkPvX~J_cbn3o zIhT>dlDRD9M?CYc?_k5=C0LhSrdf`fSm<8e>ibRik{J=d*ubi~g{02agSS|lL-uNGd63%AsBItpxF}<# z;z%<^emXR!X*xtCK0O_h=0i$%^RN{Lrc)C-Cb*CwX{i*g0A49RZK9xNpyjV;lF`P> z3L3El-Tm0=sIuKlI+u=GW3ge)nN^$EQd?w-Ty!$A;iq@+(L??*#!dPQR9m;Tsaq%I zk5R??+VWq>$x3orL#qh%lwgkX&K!6ncA;M;Y?90+4T}{=KhOCV*_wu#?!Hn?Xd{?W z8cio>WadeEYZ~ofDH)8FHsuxLEI9r``fNA-rOiqFY6;WnCDwLlI^5~_>@3b5ErmvA z*RMN>t_K#ra*M4nuYu`Z6#Z6tG`l-|!JE>yZ_d-Gkd{3dn=G^+hn;yZwM)JX&$g(^ zELx}dVw5XQz?f(zq}BxsEpLO(xJms~jk%`1f?w%ozi-x0er-d&`K zBiAlfzNAlTNC`8BE-&>eO_P^%61AwfSyz0Fe77mHBiGwmo)2XlL))b_F3oxQIyhog zu3UQ+=Aqe9sM$AhU`VRPEm|GPiv?j;rIRY0B!hRo-ny4Rcrl-8Yohn4^a=e8o`>=~ z0*`e3<{5`%oOG-iY&_7QXeV{J)WhztN*WwtDHV3rc&k3vG6|>ZUbBtW`gQxf=Q)~$ zzsqlIx#+W%66)B;MYZ*?6jyEps*GG1LGz`exy7%hF5@TPe!()ov^AnvX;!cjj?hcd z)ct_n6olk@h0Ll*8eS;SK8%IVK*a~JMH{eRV=3fXx+kV6$EIA4k-qE~OI*iRcrH@{ z#9jq2qeDP$m#0*W$xFy5WX`!-!OxQ`@^~M?$g1rixiE0vmd_Ec=?*5$(AlEePN+H7 z*SBzz@toHl0UH%OWsKj`eIiSvLM6Up@J7Ra6PVh}BkxSRbr+ z!Q%y{Ga-0f3b==@wQt>xi~Zk2KhV#zV!PDcmIevXjl@*%hm+%6V$CpgQ|XrUb-NnT zJz+}BT0eTHP=#|eAGLQ|7$7bvu#)y&?p#5WhGsnxeM2+Q`HGgiY9%nC*LidoC!KS+ z?=<3pAjcF+kuSBt6U1Z1awE)<6%6WQI(?YXs}c(#wiI-klFpXm(;PbiJU75TV-l~V z)=xYb+gev}P-#ZUm&bz`fl2=MUj8Nv(r9ZuCkTbDa5uYQgP5X-J3aaq^QYoD+7UprS;p}HhVLJ zgH~n^kKB$uy8fOKNy24qiY0oeiRo<(D=00Wi{cdh(yjgFM*l5y;y&?5W?*NX7eRQR zCh96+_DS1y8>CVaXztUX^((KG=3m*JvVZGL34%Lz+;G8DzO8M=^{%#3_o<*7F}qLo zyYb#q%9kirv4^h&uzcH`9g<@CSPJ|C`<z5iuy~oH*(VpBS+vn85sJE1k5|{DM}!IAg?+3=FL(DyETC)trqGCk$0V;&bArO) z^{41*fwp_<`f33gk)|>&k;4qeCf5`U-4~t=@5O|VCODg`uOC1l2QB})QLgr`aPiPV54F7I8lK?lSP*F!J`k-d2 zqVs_JOTHf*vg-iyTYu%z3{gVe`vtSGDa~YvHG&2dX?7628wgISDB^vUAR*JR4=5iU zli&RvFiTbv5@w-^TD{sk%|?|5vn_W;0WNqb+`qg8h=7H0%>?ZOf^6$~JNS0y)VY;j zlSdqydg@!efW%AXUa&^&DLhv|t-C(oFW)nvbIs+p3>TYaT#w9Rbtk89N@Xh);oP>n zSe7~a2u-2iNCvg`Y(?$^AX$df$?o-dx@-4W0%2DXnma7(5$N_urenzoS@H0ExQF)E zgyLaM^NU-jl_tb?(TYS|W<{%Gq8){qU@n~zXU|w|41s`>54XFD-}Jd;nMeP*0jlff zQzW5S#d;`s=25O&sg3&uI#H}dE;wA5AW?L$COAwtPHBYaI(P^R;_$*Ko+X{pI}%Ox zc%8!-U9VqH!VvMvtI?wB8U{o@6M*I?i@Ye@fxl$(XPD=9!YelmlM>~0Pi!)NZIiz5 zE272Q5dddyjg_dB|4Zg&tb)$AjFo89ddXxz-)orqJ`)^`$Hl=L6X2up)G5;*^;xDi zspWN;QuexkVKUdh4OoTJ*^Z!qBxl+pvWPUpdi&(9locRmYUCu|GMMzp6A_P9hVQ(* zG@f}FZt}ft*F#E5N@p5x%qq5Rp|4L)z;i>raL-pY>s5XW4EQ`g{^sALk%}7x8q~>P zKX)0B6a6_9;B@vgfKJ<_6!ASUo$r#L<|MsrHUAc-VanO3+b|0wPj%_M)$KHKS5+Y5 zbdMwP)}k@l@(l?p(^aCE_=bpG=Bh+kVFjb90ZL_7+wB5f2G?JOS7`k$z(o))@4%g*%BG z^3yuNn=yfue1qc1j{)I90KKuuG zLk><(v`M7FCko!@jc-09CVu4N;_^zcr$ZtYB`>sGueyw4b8RggU>(bojnRIfLe~ur z9)oDQ%G*Zz@QMykPi5AAq~jVzdE!ap-CeFzwFO?{0inMKDfycF>@SFB3Hv-{)+~zD zz!+IvGnBS*YkxA#e*OH}GYBAz+I$aDnYBt{ zPPfxhZuHvyPftRKfcMI3FhvIRcr}5S?w)&dGdm8*J{PJOCJt9N9HA>@F@1JAklHkAzs)-@8Qqk|K%{t?N=4$A#1DM{( zB|@~WkRZ^H{%Q{p0QcVqw1-dg?8Zx6y}f0Dql3NO9(+~$Fxm@p?{&UsLbxLubVY!4 zU4iz{bz_A3Q~>Ew6nz7^s4iAYHS{M$ZdB~ia}@mlEmlT?VIj|rw?I~rNe2y{^@%aC zXuw$YI08Xm`RK_Lt!m(l<0;dtehOIirsig9z%~%`I}%GuNuBPCJ4~@@R~eI(YJJkx z0{y=0N$o;R7O-mp4@Xl|Qy!rDti@^R=+GB~ui0y@e?HdJ(}NDCU;}>U zw#!IyUn#EMJEq>l@<$rC>|lk#SJ|He#tbbALuiLbt4b$ZlTSczAUjs1pphqm0zQKl z_q}I~RRRj!P8+m8G6jZhO#xjIlaoj`0i0!rKunV{O1+&|sjli005 zRM7|Rblv`9t(B5U;KehKwI3Rq+2> z>)zuKb3uf?ce;S_Qd_;rYVbKQ$Kfw5EL>bH$Nzl<9H{gV#4Bh&>~HokAb!xm?a~WO zkd@BP@WVs}+LRASQN;os6s6THoxVxB!KLLZOO881&-Da~l zaB9}MEK*LE>1f4wUMxNNktL{=T=Gg0(obVf^CZ!+H&ejnhZrM~P;)wvY50CmBh8+K zxDEnBG;&vQX|&%gN;QkM-@Y(76!AOygLJLAy1Ir{G17}5{wUairPOjg-#{*Q+wtC}xfeLwTWrs^1wb3TITS$^`@+t`$l1hU07lHrO%neDcNQ zj7R$*O=4Tu3KX9=kjx*UQ9q|8jof8iTP*Yr*ij?E+od~y0rzz{Z5C2WBUT%vlF2Vq zW6{H9@jdcYXCF8U?RN1}G$t>V;z(aW?|;@TeoX%0ztj$Ww_Mpm7lM+NviFC$w-VrE=)>e8=O$6CChaaa!Ti z{A*UGWZM%8%9q^n+E4DFE6-#Nx><%($f{f@)O(o}LMh}aZ-r7^2B@q6Z4)iIEAF2V z7!ykWPi2;z&fQ+dPN7u`T6PkZ%ZrQ8_p3x@+Wx6m%w|ES$sMR)VR`DP5(pTujnbhs zhWB4}(7MnP-WP97f8o%-ySdbn)riKX|Ek)c!s7+PS?5%;c-6&GUgxb#$EB=Q1R1Yw z-Wr1M1hj5jEdvl(q}W9)XNzNa3g#%PMCt6wxSDur0e9)P^h*CCt`?}3-+@V4TaOB^ zK%m(vHzEq3Fw4dfn0RcdUE#6Ll9Vhw0uIcrw?^dmV#10 zu-}!MHGyQHd0bx*4+Kt208q-T3V|@5Q9W5qQ|m*-VXPi23D%*cOi_MKR-WrcQU3+t z0VTjK$PKhi5J%EF#Tyf)_#Z=dBaIow?{7Smb3_Dns{xm%j=jB-1v`En{kC}W0uYTl zIw*4A{yhYZRDiE5VM{E{TnaYDz?3)5Nh@RNew=uv99|^p>A0nUJ5vT=$1(_*tG=F( zM0)pwKC=Bd`ISbE1)#>TNbPNImR>!$0mL7B!%z}eQE1jgdR!$)!evy`coG9Rn9Wm) zrHUY$|1#li*12G{Hv&(k&iZ#yk#7HaLE}%fnWuk6l^Ew#yho}L__x_=5q06^nz;ZM zGU5OKZa)7{$tO~(hJd;-8i6dmq@<))!TS1XSBpLL_WDTxy3#2oJv#f3?OhNZ^R7fS zo3wlynNa~c3~hgd>&lCS`!sjKw^s)LQ@&olwI&z!Cjc&L=>!@Xwssb9^aN@}uk?8Q zOL{tATK`ZFL6&4i?k{baGHsM z>;l|0a>#9!6GQG;fz-&P{TVje2IS6fJ|F$KmkctwVz;0-PnF*1v(Eo?H%l@MH;?t-`;4{aUOS3l6J)>sKJeclS3R zuk8o~Tsaw}5uk0fP_DbL?;&7IB|y)^-X0AOTw1qh?c>|)lg)SJg6>_@(?pD_S?c3} z(1wWyc%PCZFT;!h#=rU8bK@B(c7?g?o10PGmc4nv4gB0h0J%60&|?w_P~g!qFl5v3 zK>-H_-0w8o$jw84I*LIYe|@3u0ELDeu;O55Lj{hzv(VmZ{|Jewq`jX*RC7cO_)eJ2 zIz;B?<|gQ5I-A{B|0z;PWNY_FenPizZ5vVoq%}~=djQw(20Q?KoFR8NEdL7bG{RUA zKBW4K3@O~1H;`B6U67lDZXEd(w)CrOC%|ajf13i9(#Zbjzn~Cbpf9}-58t|HKpwg< z4#a0~&hE?w`NI9JZWq!DI*Vhjt`75$17C^UVS^ZdF-Hk|_pb0qsvqbj>iUI{`!RT# z$%gtIMDkT!EpT!Ih+78f0&9vo~_VKAmrlW(9(0~7w-Mdas14B z@#{y(ec$%24XRw9ZbuMtn*IT5sC))5Gr!aCqL%Okx6u2$YZs+zO7O9S$(l>7KcMsd z;=OB_#G<1(iIzlzbl-P(ccX#X2U(=y<3tQ;s?GqYx0sR$G!VeEDhFUUp6?TI2uuwn zH_KF<`}ma4qmMfhCi_ZUp1%F6wB+lzxNxfZPsU6jhPv#{$knw{wop9_-gBTi^+w?{ zTHJ^M_}81pZRvV*;YKd(tz6FmY~b8Ox#GFz+o~WWpqTtr5U?Obz;UP>Ld0%}%w6Bx z1J1cDd$FJZF!t_`f7YO#s5Ubrq)_S%6Tdyd2WTTsD&i?UXd%GK1^M!uq0cqCn1NPV z;#Sn+`=h{|Ux|06i~K&2Q851dWc4 zgJn0;k<$Nr#u4`bB-0UKJ<-|c09{}iSsG}%@&Q_Rb#wa${1iGE`$6Csi6;+f!R~4w z@{>WHlJP6o|Gr(zbkXh`22!>Z5RMKz?{9H{nK}OSTEVUAb@?YI;^N|`jjErt zC?u5@gTevxVv%3}?@_DrQ$Q3oD$#>U?FeX!Ug@t5-WzV4Bez_82sr>@lVbV+`6?6o z6~!F1h3xH*!8uFwBdm@7ZMih&{>RG0Z-lN4-k}QTtw_q(Rrvr6`}bDp99keqbz`{^ zLfpsEsQ>m%J zHL_5k(>?&>4ZQ%_fU5X}53vYJnG~3#_)os%SKB%Q7v=ANtMJ#mT(7T}Sb69HNOTQ4 z&XeP$WE!hwBb`Ae$lEZ;SV3UQTpkZ_Y>$|wx z9px+67aiAo!XK;1kk^8*_LAlg=<%?DtSFVjW-vhqRAMwx3q*o%h`~KM!5Ogh9rH;Q zGxEb;UoIhEK&<`G@>?9E`~mX=Z2w*U9+2jdWia8L6(l<=YvK1aY)X(9`~i1H0n!9) zTujS^IU;^|PoA@N0fiEbULmjA)W8~A7jnEPeY(xhg8-x;B#2)h%4VS;)$?Wm;Jr6w2?DYP&Hlt~AXI$XJ zs&F}Et$bhnrXNW3|5pu0>*BE*_>4gfW)G3+ZlVp~xxYXgm#T##{%@a>pblLiG7EgO zIr;BmVDg(iR7V1AFztoc*52-Redcug%ng+C`jax`i6z7C-gS3N#+F1za@`&uHEtdo z=60X;uDyH<3JX~{4pkgLs(*nYXTs-(Z5g7L^$HiPSHlSl2$clF$VWKcGZbG7gYIss zE;ZH&t41FTaYVgwaTG&)Wi>$XBZKb=l7TQbzWZpSl+04x5*fv8$7kH)X;&nlfQlS{ zynSfe5hxfsm4RG5ckn%?!cYQEQ;~r{B%*onpywh8*<5pW-e^7u1;&R9kOB z7-)&K2sejvD1rpAU#NnBm*V{ zl)chu=fwyp0dP1(1YMuCdkL1>F#Y?v-UV!rFCpFJB0(GFlr}#yaH#4j-=h{g$X#b7 z227iGQ;Gkb=@yn>&&H?nA&m+v-rmT?!3>V-T#x-H{T%K^!lua(SXd~3euJJl~l z-nbCO!dSr0`~^u3SkU(tJ8{4d9e_=``vCHBR+x2CB2j(!ATnl76_cmPVuj+gL&=L+ zew~SPZ$}Js*$z-be~{L*cf~%5s4;i6;oagweK0uDb7;cKbsaIx?Rw) z=XCm8b6$mHJ)It=80oa0jaIlQ@!~)i2|+h92gn1=o9g6-0I#c_7YZLh96bOrSl)=? ziEtR!JyJ?x(+EPj0{%1RSFc`$PGy0(Uec2GeHFKF8U>rKhOvi`d>xcGPy6JnQ|Nc8 zG}ZM}N$w>6>pJHF)@OW0@}G6mKZnLs0ne3NHZ%-wkk>1z$nrXWhz-(HkVb0I=(@t^p(( zg2lZ>k@!Qi$=AO!r(IPtuuM9P-p3dmrJL~oxs>uP;(0ozS)>>*@$SbwqiAq%Eh31! z%>-o<$GkvV)pp(o&z-R9vzaVUM5^lE_c6rn!Jjjkq{_+PCD9R1VvwZgt-lXk!uJ;1 z%X$2`t*R|=L9EQQo}_zN9!#`9diuDlG{~XG))?pH(od@H^`F+A%WEHSnXpXcZlZWM zgS0jFqc_nKcvNIp*eTPQ+!eCWcQJnc^DBoiL8u)kBluAK7hWbOYRae(^oLouDrGVK z0jOT;ogKT6t99tve^{`hw}juyr&N3AnGOddaB`pEZOk9nzrCrQ}SD>oR89>Q+DM+FuuwgX_M87vv4!L2Ywe?tOEA zcgv)n_jU`*&=AF;r*rJtdvI0BKXTBZzRACjov>mu>KJQsw{IM6Q)2X?4(R9?diVEl z{v(_xIz1iXSy|bW*Y=d+O;M9yJHCAiVxV?d&}7+WDuy@wvxX+IS+)wM!R&Vy0HIj! z7TOm>@-C54t-~h-+3-T=i#j_TY|fWAyJyQd%vY9!m~<+O{Y^>j`$beHd*|H_FMv0l z?*UFwwfS3bxw_Zw)DT6L;kDRl>xXV5qUDXVVkXV}{Eiv~RcBym zAiaH_aW?-&+dc1Grx&HLp2~;HFPB1Z|LN&@;Pkt$=HF61`tWS*p8I+1_S1VCoCMzV zhizYpq~ob{H;s10LIR1RvhD_bR+-eF?>_w58~#DuV*Z9o`isBykd28fmGRwWu7jAt7 z;WX=r0W!iLz^`t(AqltK;LD?)$|FW(M1>DuwTo^~+3bWf9Y^aq7nE%H79IYxj#eUf z9oRmMQI1d_ji$2vXR%@QVV(&AO0{vbX6$qT=wt}k4YgH}4R*HFpa)o7X{ZfJ7qFGS z*%-;cxVzquT~lH;_uETU6RY`!gk=Vq!2z)^_1p+?pNBpS_L*30JQIJLsC2WE#Gq@9 zHiYvH6iFOP5s6ms-GU#fLZL_h?bjhpv_Ac|FG@gKhyXIN5i}b_OJqg*Pa>2MP>P8~ ze7}^T=~o#G^6i89eEk}W2Y_|`(0PA1Yij`V8I2sWy#cnX$$LPc>m@d?*WpV3z$QUL zg%w+Xg6fJ*8B`0r{)YYFMtz@;vVIKFBV(ks2k^=OG5tTL9{xx6{O{Xe$cRCQgie0OO(WtSDW@VW zop!j`tJaN}`Oqz<8DVmb5Bg`!L&-5Y%$3By5Qv@~ zhe$lqVV+uIYx~Il1qBl^B)T%k#o9|+ekpuqFm zjeQ?0m(h)rnE^}dKnx*$^!TylXo1XVsTL!U_OZ^Uh;)GRQDK@09+q(?BO?u6`aN^| ztHYfqusU0|>!r9SNy8%<7Ty@|%k6cXrTu2w!-GA-_u=TYUj9_t71Jn_nZb5-bD-iL zp0_15xjL2`aya#pBQqitp>>1iNGM2(@fHKvY(JHxJ;ub;NMv~NLgIFRef?9ILTqLx zndT?e*e_qWK&$m?;2N+V8n?hGix0>xQqfcv9Tvh!#I@aiHQ;?QMMOox^$!JIqpaT| z$oAdf*&pBr?*>@@3uxL4r(2qu0Hru*OZGl7G0|`x#C!MRqWHiyGMbRh?!FzVHiJ?M z7MOwE5?%FBS)3jmRJwI?4yufWhFS$#wdaZjMT1_3Sd?>?dDh{{EcM*{d;*V^qE{TE z*}DHZYLJt#^6)-r>X%Qqr@s(_JHq$!sQ9iXkf?4yfxa5IWI7b69WVTjR_y0jTAk&q zD1bovGQfE0n3Q_ki3yNn$T_K5JKtEsS{$fZns6ufdH#;s%<$S-J~5!x@9OdoV~eXri8 z@ktoiIJ`BJ&l$sy-&>RzBPe~Q<<15Mi`^z0bQ<~&ehprl?_v!zH(Mj>w*eTn-jmz= zVvVERK^DG#4p@wsA?EJm*%MM+6Y@9yRlU#Ay3V=$Pi$jI`B(;!@Tc_MXQYrlz%(G^ z07P$%6e>aYqN@;KvPKR{jd~mj`P~A;FOocrIT0aWJ=Ihv_OpA*8Ji`NFVwWiD>n1~ zc(z0GWy1^0r42N+X14Tp-qSm|?2F@dG9OnKqFd0Nf9?ca9#&GgPNF3rV2!-ktQNtT z%KXM9z|)1w={#S&e+%k?t_Kjp67%FL!%|)yx?wsmO>6{HSF(PC$1HFdHyteE^SzlY zibeoc82B=2s)F{Hmr36+fuv+R+fey9`bYbIS16i}M1$Y4HJI<8*=Pn0VgY#G0C*|^ z2C#$R(x(AxA@d5PpBW8jD4;mBUq@<}3%V`6AN2sw0FVS4k%eb07q%_vmfwO$8vF%} z)2~LC+))pD#iTCK<5dJdOLHzqmst!t&%O!(3Z;hg4|c;k2{8NS21vDfPOMcx`T_Zd z|0+Qv&(n#oCrNFnzZqU_<#xSV+#@Q~05}d&6jpbnDd$W=@ zP}A2KYFWDG*CfW-Ht^vvqxK9+fJr%+G(qhHXbd>@4^yuKgm`ym!5+rcNvhmnmf z9{18y*@OC=N^O(bWLf2AF+6V%4IgraHqbJVbaCr3{`&6S#)|gp?6-9JLL5~z_D47h zVAI4$tIfodbu2MP{9nHPKWKZ)pt!p3Zxgp5f#4b}I0OR0-66P3kO09UxNC4HSn%NP z?iSn$?gR@K+--K}dH(M^_11it`7n3YUDcJ)=|lJFbJkw#`d#~11HP$=Sms^PsurBp z(Dj}92>vPz+9JAWVDw^Mem1SB^ z*K3UVqxi$52J4bLEL}gfce*}0wC_Ws_UvMaY8H$faMYQa&!w5eB+9#rr43AOV3nj! zVq@lznoe7fyteyZk!W#RKaRL9-OCtItE5$6PR1NqI&!v}jUW4%`;~BGoeUXA!Koh~ zwA90&Og)G0!!@s(ie7YOy6(HYxPL#Qb_rwIKA^&MRqN^CAtGZ4=Wmrncb>olv z!|5>I7x^?et+RNcglNilRo#v)COX@?abCCeUZ1Q-%1Yq;*a*0B>cYNp4BjMqmBa6` zwn>l`Qqr(CBJAQg<&!Uy9LAh|G)}?g~q^o+?<&kOJ*xqn@{>=s|V*8`PL-^4wTG` zt$1@b&rYV-9^FtAzVS`^Em!)tlc0fX2YeoCd;rkF1r5Dej&D#3C9uX(Yq)E)=egyI z6nF@}Tk{l-m`E=kD^^XSV75>GWY|3h_FtD5F&nW9`;Aw*iYQE*U*f=I2>t|hc4y~c zOYBdMI|}#gKVPU`!a^F8phr1FmCKRx!0+j5ERzv($Qeh6vxrteoBZL|?t=H2VIW+` z&y@qyX{ro?*0Lw>E7~)}@wIb_jcEPnlGFy%6~;9F2Mp4cmh$|ssd%6Ihl*&`+7i5z zE#y3Y&*tUKS6dO2u66hs@oKDh3cdhC|42uYIC|?h)7Jbi!6GXl&%o(h zShF7hy?gJZwB~R`;Jx7es^1ItqIEqP)G~AYQfnML4B)cc!*PhBWy_L<} z6IoWDb4>D&nj-a`mjm=WiLgiGVN_CfC6h~8kHf==gRysIB*x=$`3BE-^o*>(59ZD{ zI$ToaqnhXXw7SnW>_4;BY3GdkuDv{2#uz7ip8FU*eXJw+?7?^0f(Cj=xuf6FAWf36 zOP>9t!IXjk3%ORCr|bFJEmPOe17V+6Byt=t6ZdZ^m!4Hg#4u8-5y!@uEi5=sTs{!r zV4@@x5tZ1Ohrq!K5#e6E(SSMm4{fS&BSz9U+5ARLx~c9+EfmlH-Fm$n}0=boK+4ist8qM7e@1|CG7OO5r;g#cda>34x)6pcylRQ`S7W}9aSZLc!R0*p$h#6EVt_c zyu~)_-|=#r_F9W}*H^e%VR4q*kG2Ff1!DWBU8%9ds>4U2-ZRx%M`Wh!WW-@{?a((i zJ!&I1A49sx798KG@KkZ6r+rMHG5*V4Z~O+#c~O7Bp1wfx(+H!IIB!*Mq*!;t_g@Rx zbo)ZvP3ibKsp;aRL+{$hDuzghg})0=c6@XBYm%A%`&R$JI`Qi@ z`$G|(z4{o*(b=4{ephlG&FR)YWk7HcrQB65?OYW_b>g5+?ed-)&Gdb{J8nQN_pco7 zHSGb#PUoNk3VY+$t3@&dU-VqhdewQ?uEyT^*7avDEIT*|m2n1bm9GITFfr(b=E4?Ztou zP^S}WBB%`&xu>qb;v(8s0Ijd61zKt|17B;3s&}G;Wi!JE3AdgpJL6by5K-lNwd+S+ z(qxhDo5RL4)M14?L(vATyK8(!{#2AdGPk*6$)$9xc5_TKDB`ZhC3e+|+ z&FLPLo@#0NJ`468wFdfeM7|S`X+i~i&Wc=WOOof|x!FYvFZRnGa$P=8o-dXcQsWaxb0f5&Vf zR)R%vdb3T0y5_3JK}eUl0GsL_Kh<#ZXnyXr?0rRe(2}4U{Vw0J=QtBr?BKRQi6)aK z?DN@|`OZ*^JqkzH^L?~sWifOGYUp1%4Bqq;zo_hG0vM@lx{n{cF1Z{|8EIFuvEM(} zLT8R2AwA!-d@&mnm?pclOY`q&gsUbcP|3Df8!)(f=&SjhWOx3S)c#=TN>=JH1;438 z9IU==1bZ_-;{MqA5gk+E>^ETItlORVY%O-MaI~JxWSv$YI56dUR42&i`2It)@ub}Q zHS@xl*AvT@^62F^v)2i~^?E4uMSBjmmE*ELi@$qq41-8jAf%hh_cb0hZnYZ>ia9 zEHqu|x+SQweix;CEg(mxg0S+lHy&xBP@TcoY>D&m_e|)=-N}`=!I$$q?v$-(CPI@6 zPbHs)#f;Iw(pIUpvGtLn{<&2e-e-zdERH>|rTE3g{;kiYT3{{0d!|*dFMl)co}Gj$ zi}g*976iwM_eScC%lUIjGnQQ=BsEC0$0qI$QND(hqKEqq{>I9#%XgHscbpV%ZM=rT zG{lrj;O3C2rH_1YdZ9h)jAZ^$X}XWj*K&zH_^Ut>?Ip@|xD_q=PJNo6P=_yV1;bP> zS20FqM%NmPZHsT2(fh2`5Qzaj0n_*qy*oaFb&rR7Zyzas;k8!{O1XPJ$*kOo9{Gnt zqkd!?`yH1%xk@3^c(w1dx@^Cg^oz$Is16>UM_2GmXnrZwzW!0b63b|PhS44+cIq|y z?RcwKZZ~^GA#U`QHJrN34cixV?KV`V_I>D9tatpg}>w2b1cv{G(n+8^x3%4lt%;J)tI&iq%e9 znyDo%%q8b<_0tOlX{kKu#5?kq5>+^7$}FbLhQs+d&M9xi-=dGM;3eY>B#_E^TW9Hav@XL8db=1f`}e5h}t zsD@h}ztJ=91tfd_3aJ0mKJU7{KcO>UUgVKYKAeAN(z38ky-BrD|C>p6?;xI9?3(T@ z!|NI~?8=R)@E3dLGLOVD0~&d@C8|=rs5l3~kQxVuyY(}Q9nO-Fx>V)W+5y?`v-)tU z*e=-gM3=d7AMf3wJMu|m-C^2HIzcuYl4ifznx&q>USxS@X+;-J(b7M4_zMhM5i_HS z1B=r9^WdV_MJXy1?tCj|2LvC7>I1_sM{bRqEtKdZUlw0iM3&~&5Y)}KI{j(3D5JZa z7I3tb8w`-vpbg3mxs#h93J464pks^IUm|DG&k<@9kvO-mx{t>oulFl;T{=sd*5b?g zDDN+u42++e$0Yij5ZV>3mOh5xPKjYm@jj6(=7BY2v10^8HC?%tQ0!n$>!`;Ip2MlQ-m^aZHNJ1*W*d3gp zsd>_>-$0`S5yFqrews=df*< z#sjoiiG$?vv?P0VE#>Iy<7QKoFYdC88UK{|^N1=leycLrY8(z?w&DxVEn3cTtopR* zB@+a*T1+19SR-<^0hV{gNVQ-}f9D1=<)9q?inw|7E*Gf|l0Ij21)+FVJ4+Dw*f>6p z1{0s(EywLWpu4`Yy=}s=qy!TGR%`7f5N10X(D0F9{+k8zI{S7YH*x}?@8awu;22#8 zox(o=<{QG6c`0LeAkg=`8cr)Hh1TLZW%g0Pg!WHaX1`^klL5GL(SQqAUEVB=ie(wW zi7mXBc^Mr}5SsZgQT?o^`$6cF+u+5DfV-^P+eCpYc}9MJ+0^k8{n8z6OZ7dqj~{(= z>vn<{qq-x*yPo|7uO91V>tqDl?x@QDw&oJL;eWeqJ{jl6$(8AW5zSvIFcV*#@v!M;JV=UNuKYDi$7hA68MqQ6U z=FG7&!Lo8K`=f+|!=+}=-yFmC^GE@V%;egZL-3}%H%zX#H`KbiA7kFg5>AuM3OPxB zQ!L{!Y9XO(Q^`7=S5TFlD0t$D+S*h8G5aGr`$C0QOW5sW7s9hs-4E%IzbXT#z+Wk8 zGLj1&NKOX>w7;wXdh!RnV6g=LtVEcD2}xc^1BiQZIqk=zK$+-J8O%trIsjxwbykTm zNz=hg-?IHmW|L>nY_0a(E0X%Z_Y~&^?HZA}A&W41a952Ob{??I%^F&g62f1(e*-S0 z#veiO?S-l7F74&tEjvdv%1@%&b2fn12gFKPeN;MdODCs*4#H=(C00Pqd*Lc#GwW(v z{{jk4Xfa+DycyloWi$JVD$LIO!JBKCiKW_&8t(D~>x;Y)Aa5_0MJKCHIj0WA^ zBgmo6JS+1Nga3BNsj3kn7N*iP$8-wedBDf2YR~|`vf=`xxf}x^7=yt`nvkP0CR;Ii z>lnBFWjOP}JY-{T6R_nUtEWXjRvKPWkpjCQLcc1D_nE zd!L_pwHs{8nzdpD9!d>)b2|jh(;3nDa|MNq?&r#MI4=o4ZFC3D0~e=7FQahzhpFFM zgn{f<^NMTQ-v=fkgOX}!fMqJjaR&*f$2^_nR@A%13NbftQ~6H$)5-xE7_e5uST06^ zCcFW_u`SpntHam`zQsyXQ130aOGPz@7qL8Ep;;(&{J77fFLyTEz zdD)8!nf>sBnN^(C*N9^xBIR1fvdGZ?3}+TF5#&L00P?9Eh9vHVA6a6tL#1KeC)S5L z@rXqooM%oT8I1J~oVN1rx=4JA(>@F}wSxZrZSe3>;ZR@HcoQ<=(aPb1lk29|MNGqp zUa|CEgW=NASf}b*jcBk6#zi9BSDPo0PJ3mQ5ORp}?@%bTSdjxh1U~Y<4GK?gC;RZu zT+H|x@((RwnvWPVzxZsK%!*Jgh8zy**lCe?ARC94;`p#60~Rl3-)nVrR1VGC8zh`$ zD2QcA1?Xx*YzmH_EvUlN4Jt;R!zLW>|1Bn23YIB0zRPjm@r zseypI{6Azi?P?|ZI>?)!fZT`?82kRaG#9Fs;G&cAnO$+P(Kc^%{sDgJ{byfa-vFJ- z_4(`5%dXxp_oDu*wu-j_xW6_Bj&9x_I`P5)^^KF*lLCO8KYGuVdFy;?BsQ_D8D8*xti$Y;M8=&77;NYp1#$ zZ|RGMp1b+s`Y-<;*;Ad@lXWA>Y~Yfi*P)5de;yG)0(|F|3rJh~Z0MgMQgg3+=bVTPK4$&)R`EFq z;7%(5QV2p}4uI*423O@QQ2u=9vZL_#_g5_eR}_Gdg5ZJBf4_C8WCFzA-roQH-a(}3 z^9*#|iQX7;;nmh?4L!}^*rVI5z%K$GS!qz1BY)aUfWT|n7(h!28FUl>JG1}eZu@^T z^!y)u6mquze}8tPraSzhw3rwiprE25LoopAChC!Z1FBZxNk<2xGhp}+ic>i}cSD8R zjD_<7&cH-HH1PNKFWcMO2hzA5O7;7p-jYi1DDdKgi5=w0)zIKA_|9od3aNEa3D`8< zod%*#maA<6fkwZV+e=#23!Y)#+m7`Bpw{f~VAknhQc}_!a^5U_8oHZkB!%5iZ)ybx zVfO!jPCMu_w!uK?h$@b4G9S&Ff28>T#4Jjgp_Xe*F$g&<;6UUQl63*YbGKbf=cC0y zENYp~nHd~NcmPEDav0xPFS0{2hc-4gv^w>SQN-MzE<8ciGrdb74=lpbbJqW8VFYzN zCV3Rw&eK;{hdGAb=0EK=y0C)vd-m`Tr~fUQD0p(Ee@Kxt(Ju{}X}k}9Cb`|$m(FudC*P=eAo z9?VzeJaK?zMklB}Ay~kpM^j4+B`|!Qwu6y5pj?0Mv-wsz0ol@YvD&a%2AnsWK8U~~ zfO$2ib{~jr@y{ksGU~PBgV?D!5Kq4aLqLH4?U!YeU}5=#&{7WMOtg}9b;Z9bO$}^% zwv1j7tnZ3aQ+au}!*Q5AV*+I~JU))0r>7@zzEuTh)YaAH`t-mJLJgzL4j^#xxH&Fx zj|vzBH7AVmH14mEViJ_EnspXxwVB}7`fuC`T^|NsMIb)m9m3{hn5V2WNDlY=8yZ=w+>55cUiPvZ6e|)F)tDUU@>oO6~qoAcLt3h%%EHf z_So2%1zW?&637qA8rlKbt{JS5xf3IVTwI6})xq5us27|(8@7`I6R>0l+}`Sa#>kSB>F#DKqR12*@Nj zy*u0y_pbts!D|5Q70+J+ou6jomn?U^yZuxiCwh>V3IL~{A-Io#qiC|&E{XlL2zZ_x zu>A(DNQxBc;E;?6CbqrVuJB)H@I&>mX|YTxqur2aBEzGZ z#k%Lg{nh@AtlW-NX9Fn9xJsEIA6bfNv}e`lD;L*49`LIRFS4bR$liWAW46+|V(9bM zLLP0$j|AvxPsV~L2`E4)vgb9~%UkweHQMbS$5N}mepzYTJ>6#wc9l;1s^kEugN z@cM@ z*QHNXvatwl{iQwMLfcpm&s(LowgNO*^w+~NW&hWE0ES|45@Ukl!R;Cw3EX(Pac?9u zs}Ag}cI^Y&-6C6iOL`a_Z2spr`l7q?ja>Hcr@Q{pxW+8c73VM5|9!vt0f*RsZQD&5 znPfp?b^PBqSWDoDf&DML49;UaGIAeThjUNz*8}Kubg6ixCKR=wc$817<4X*n0|E{_ zxcWKMjsE0S>hlA90}EMyl3-aE3&U~N@qDD~(+%@`77|2{zJK5oakzZJ^ZD|I96izZ z0y7*cI|mKQ0QRH1S7XGQ|LbQIvc>j0e0MQ6(%XJasw7wAb)VB*8;t+=gB* zROaI5?s_iDRO%lG^VXQ3GbC3L|M@iH(*o(&H`+?}Z7UevP7h-%CeQJrY{dIYZw>zE z?Ad$g+@q7pqE!YzH2sA7-zUcU)e_WntyW<|!eFs}U;qx`mMTq0{-rMnZj5v}`I=qO z;${WEwpBNyu7lj}1PD_BPY5NbL^Q!#fmgoRLORraL(BU6bgi$#{#A~DUj$DxZQdM%E7mK4UjUfRKMc6tT%+azyMqrp-}&W zSE+U4z5*nX7)X~Xwz)f{@i?IXR!$T^e3gJZd1M3ejvilVSdjBZ0!6Dxn-6e~Leig) zY~b4051LJtAhiVohD2atB?%4|%a{n<`v%ZN9^r0yk8p5_P5=H>?}!H;VtgxAy}+^Kx9aAo`5Ztt@62IAJ1JX#kfGlp5jM zZ-F9DK)jt1DW)*49eVp<3Nuy&43`h=!e-@Sj6(3?oY`O0U5o%rqHG)vMIcEepgyQr^=wd(qx& zT%Diqj;me5irVXL-F#mPw&&Y!lxV=%_KZ32UKVh4aghXO($T-7MK)l+sUWlKth5e$ z($uBk#aogUf%xl$Yt9A?6EEX7@M-6Hha~fMIPs+4$&0&q|N9xH^X=O|W43PkZaxmX z4W-egNloWE^`iAO&F4B-Uk@?B9|9w6G$D7?;&sRzk{%_5MO&g@GIBjU+9QPi*_N{t zGbgP1K2K2oP_IY=PCAjIE`<#HMi=dsM)p3Oi=oc*&mU zoo{T1CQYJnD@@-1%aLpd{uuNJReUM{f;yB`4)zq=M;ruoL1Zxk61P5ZJeM(7?nFgC ze2FolyMF@InZ)=^g+3YRNp}HcZf2KoD2=Er<1$@y~^dV6EW!k@leE7 zRImUWy75I?tAri{E(mc^3+*j8)!eNPV&EMiM&*Od+r9@qp7SrKZ`RY^R7@Z%Ec24$ zUg~NF$TBcyS zw0G>RYK&2H7XS@y7^p+>`5_z8K*T1Ho;h$56KwaYN<6zG4>GbjBvJb#;8`te z&Pw%I11s-hDDiekX7#Bsr{Udx~Q1tajVeh3XJzjTV>q_rnL{jIb0U>;aUvAy#QSeO&w=j-}INWNymy%qo$ zapfF&!nj0cS7+b%a=G1;ctwIqa`n^9`h{&ZJp~_elEQJ{&>Di__6RvhRM@8p_BiTn zx<`gEN4;%x?p3Eq-y2`!iXqBsjr26!-g<>KINT@olR?-ZO`TbZqExSq5TKZ$&dEU8 zP?-I(2Q^a{ZksAGHAfc}q{9M$_@*#CSpwaE`jXf4rjq_H;ri`Lvb{r6b0DZRBkd3r z+coXUXWFe!)1gOhG-x#2R-aGi_}}@uk3hBa!qN&C$7>I`gSCCzT*#hIL=D?)zFU=s z#2nJC9i1Tdwxn;pKKSYx3cQX^?qaIRUK-dQQ2J52uxpzp7?7N!ZFqRLbnL{3#RLAO z1_*_w1+U*9B7QdqG%@+w%2iLFip<|$Wlqga$n;#8#Jf`SS z%u@=&z68E#4DgLo9UnIR0Jb;6*>m2XMg}?@#miTy;putu(!gP>RIEx@=M6T*Q!TPB zQ>%O!N^-}nPVHgE2HCFxkdZkZ_HPjJ`$w_7F z65O4F+m^mOVLE^G3OYd|q2G~;R0tE-rvmK!>Fz`9XrsGxriJTkr^zri3#&afd;Gt2 z%31vwK3&ISCLmyh4gKs4yBW*FU%kr$%aX}g=>iT~h9nx|-JOlm9@gFyRnV^anON)X zsnde(kmCtD@Y|D1K-wWIXIsKRm9EfV3v1zVvJzCH>2yaxqJl*bfcbslcS)<+95pS~v#+%V$~rHz)lDD_Nf8~4p{__FR%&&I19Ar&j#wuM zE|ORMpaB%%p@>}kjTGPBgaQp|mem{8RL`xi@V<{%=s_H*nhTx>wqQxs?66H$y+v2< zlr@@xcOFCZFZSzX?FrSH+UZE$MM{z zmgHrP0N7JyW#O|$tBD@Dh@p}G!sGKuPwUPL<-Z7zqhqqH^<25_$@ zg?v4kta^tj6uXE*XM45H8{wXFOtiRm_0JO^ym!-Kujt31e_%IRJSw%7+$5CtY*Y+w z&K+?o{oaMW$+OcNd2D^@{_KPaFg1}z`2znQ>Z34icMfaxcgbQG%=&WkQAZ{D>$?JK zla1%sH?3=`50^`?4|U&>G$R$kYLXryjMraS@I-8@Aa?TVK%=HKBTBDCw)7x-F4Z5_oY89#c?6}vibWpbqu<5^HO)OFLF<0yCPg@dXo-*Yc#M3e>jofL-2S> z_<(?k$Q$Z2h3Kh_T&3qCR04c_Vj&&>+WS>3>UbhNjFKPA>^HC2 zDukD8S#5kz-_~5c(F4)bzZxT#u(Th`1{{n(+UGj;?_}gn)=IgnCOg9Yy878x;9z-j zG5hLa!3jFwhaK}B(TBe5D~Hn)`wP(<&=r?ij95@rFOs?q`p#Y`0GdZ&lT&*s>A1X~ zHNnr9w4e6(a(FEWJaAc{f1q+0@hnHjxM}-%8N0FQ1%8@X95!P==L#z8@Xr@-`*Kra z&*k1vw%teabK(A&v$8ht@W$yy*n0-QjMV#_rRLQEEe?$O2keFo#Ly6|?vf;yO6RdJ zBg6|R2Y3%5**Jv7sMF~sG;hm&n3g--R;Nn-C=Sh8hl_7E8}06!H-eS21sp}(1JgNf z^l>N(SzR!{M}>MPTjCGL1&TOhs5}Oo$OtJQ;?6KBYIdO*Sb2{P_V-!Yi?NI~|A$y;qaMSOId}{dV ztMkd2B?;D4eaWUx71!j|LZlMzll3CRKdEv2h*8ccB$d%5v zJ@~5bmsql(1-8Ox%c{+Nl{8{i=%y)ZQ2D^YC7c&Ig*T;AQn`I88X}$}aOeAvS*@k@3>QB{(y!UC6dt zXb+kAlBQ|*0X_h*K=l{JvD6=m5KHFn$z&`1Mq@gUI|a{`+HH7sd756CPu=75XL5xS z5jx@9R9!k4gX0uOzRM77504C&*gSu_;^q430q+aE3p3Gk2cQ5CQl>oMxULZage>=m zB@e`2w!F?#g%@4wO>bPpM=K6~uO_$5pLIkIDAO|xxfBs=p;}RvmeHoJEWD&n8^4q5 z&5XM9i4Tp z%2_^jutGZfK2NOr#IMmr2HXPzCD&WmWN+XD3i}E8bd_F< z+EY2#N231CE5oaN+?A91HiBbl+pTz=dIzCI&!z{0NjB@(f9E?CE;itP%d7gUBzIUx zn;MdhM@ov&v4j`7pJp=f4P%VCUd7!I0n{f|DI`ds%Z4kLLy&g;_B&QEnja*=P z4>pYn0%iXukb&KR>_028x7^$yg&|H=JKpq_6eG<)JAI#Ypj*+MD3MW*C{B7IFF)%w zMRwzbVS!c@+{$rY^7K?6d5g$RzpWnmCPl#}5>QD-Zqed@8cT^v)A_MYB!kP6%VZU$ z@VhCYqH|Aq;P`qK!N}zO)2ZO$%G(zeUtuCL9AtO==+RoT^-^s$6ZT-Mc=hqSA@Div<4PqB*s=Jivk5!SpNVuu8@)dKk z2b2ci_f6LZXIrUQjaq$V=5q~o4j)_pxdJtyz&m&~{@nUj=5y7tyiM6?vl}Joh+%my z8&aJmlxC56ya$H@)r->g}^+;K}uR zM@*3`E?@IjFrfb=wEFn#oba9(Rg~)K>Oa>ruOy~RR?urrRJ%%C!rFG1>+E}YKc&hf z>Tv|ygSLL{qx+k)q*8PCwOV0LiWrV7GY@lc@EpA z7@R`8llo?K-W*SlaDUre%X}UH=<2CU%fB4QMVa97X__TQ-sEww2gU6Yi|&_LCj7P3 z=H;w!z;BH)5gf%IsuxC&&CS%Jf4i?9De3gsd;$Lgk&{lZQA$K~N|Jd6fK$Ds9SSV6 zfrCT%^oMpi`;Gf3HKUCNws{;4tNY)RRD&agkB;HFr@S+*j-XHYE{Yj)%o9jFRK&f+s9 z#@jA@!a8=NYVUo&9=?UL`jsE~1Zn^O29tqCkWvCB4u8<2>FbwN}7&3%*rA;W=l^>TYGM%_miF<&&J>m5s6CU@U=A)w@)f^zYJ-`X_pm)*a z^`IO2Z29?M3RET2gS(+!KKjBPJF3W!OFx(1+&T5JpLm6kqVXVvX!RDQ-@yn?YOHUyB=#MXSzQ+eba~oI8kosuTwvokY?0Zm1c`5DrVU;gRF^?1 z5_(EqvnVg?fDQo{FE1q!E`bGs29s}3cQ-ziHYW1PK8{qu8=X3jsrOsuiFh3C<*yc$ zhka4yXd_=b%PmH!cRVH~*Q;MC##>jEC0j4l{NQZ7WycR}i`%^EM7dc@B%*yf_x@m( zLU#FL@vqC;thBENB-@6$;0OiTM+;yaFTs>{UxIO^6SNyKzyjPaJp2XNggz8}?)__v z%B@`X_weYeo`%zyj)8|o*7=%$PV4#$`D?x7*A)g@0Tc-VEi0W{-q?u#-FJ&EDm?xE zqrOiMyf?+v{eMfy$@rIbPk0z%UMyjEg}66O6pJA^>O1Q>!E@xfdAqt*`bfw&dfjr_ z=<$65FL4E963Y>7t=d6;f2s$)4p)-^T7x%?|!oM#I&CNz+aS^ z8uo|#D!4W1H`xyXqK&K+_KYrfZ(;QL`qDBNH+BL=i@D|je*u<8uRZ+i|2!OtXYy_RS`XwO`*-Ro}_(= z)9tVqnv+FbrCOm1vPSn6si%L=L~r6pV6J`M$yQ{*cU|9*r0Mt&Cb+8yGnuiKQ``LQ z2zn(Oy}q8-HlY3<5#>F-|KYYhMJ(EMe{I3+$klHXDXz2p=m+_&4+l{iS$!XQoyyj~ z_61UY#kl*DsBn*hiMes}c?Z+gWX$?vxu->=%2;U#vz7 zfv71IaCdpZd+xo|mOx-t1x@GcNBl6va%Op?w*;{BiUv~Owfk_lijwu^HVijhA1O3P z9b^-*LFutDq%^DJubvD;im=`t^scdTFU?|d*>p}w^PPvqs1-&Z=Wz6X6TjV_(o3HB z!2q7y1)L;~FTmtcq5n8%*jyB|E>wp)8GN?t;$eQP1AI&zfQpjgK#A=DQRwn#!FmvQ z4g$&4uF+96FzxS+rJOX)%k8bg7Tg8Z|0&TeoW?Wso0?$NAD(YUZWXBO0QSUO1MN?Us;kg&i|Qs7@Gu~aTf!E;=-QKy zrUcrn0bqPj4yFeXn86~%9w4e-CkuJ9co5Y#5JMmWJ=4q?03?(bh5=BswD#H?Ik`wdPro#Xp?eYc^i4`Po1YT(#DlA!mMMvwj-o;NlKm-a+TtQw-(@*qI?+sh`h~t~?NQ=x zec>LSpm0HW=L2gx=wTt?B@I~VYt9pKaX0<&T&Xu(-j6~9egcn=UlHA`cFDq{c{v0R;0q7Vxg7e3hi=qEr1mGWe8>LMH! zYHU_`AkuqCT^kA0j!KYCRx+vjPv$g562;*lcNnmyjq`Ip`uuleh2X;z2?o0;ibni} zG;(w<@)V0}Ym3)belblF2rs$fjRjbGlxG_}c7L&xjTl~HDK6@r9xrVdx9;D*nxwxv zTvRERO|5O3+L+ewS8DUIJ|ynIX|mi>@w?5S<9&&X74(})23TB4x4s4R63U-3l!nWM#sSL zIbcfx#9I-g5x{8y9f^iR3_Nnj7GhG8b3jeyUluD=;hyoQVR-QvT*W0c|0R#Ofpbjb2sL0S3;K3@q z`;V>NwJ6XN1gz<$0Wmgb!f0G++J(;5umOX!Nc_XApY(kb6l4( z++tpYtPeyy!GgqiMesI@=A)q6;H|#wnMKpo`qlXyuC6HC?pQ)xqqB2Ip2k<=5H)yT zJ67{4DzNQ@&dhV%#vi~3<%LC~^{d8z`ZQQ5%-oCsQHo zF25!aJwC6I2Lxj*YEkLl^EW)9T6n4eSiskl@;m6M$jXY{V#FO!r}vKIZ-~UwK1^F zA%0R5@0=?55kdWMvLTvtd&bq=47Q2Q>j67`Ln=)^uRB$avzTUoZu^HM@^84^-zMY! zn&)Qdf!XL}u^K~;$%jFQY5{OvP+>|Z>AKR(!z3*KS{LWNQy;k!Fm*K4#a;D6o|!j{ z7{?wwtZDZPk(4+foBg|AK3sS%=VeTnfF_=T+wVXAIjajpPE{0lBXNpk$!8(J!ppKN zMVe+D>@+3VNpxR9?z197Uqg5KoQv;qeDQJM6J?~2m=%Odq6^XUJyxhpzD9*E*5qO5*8~l|o6bbEQaHMyS zNIBC%X*TfX3)Wi#=esl>2Mf68rT#3AlIb@3tAN1Gp{wu#P1;w*mnC_GUrV1q*X&mlCKF8`@2@={tx)l_%(`kL5L7ux;C03wYi4I`qbsv0WSKx@n372x zZk@-={4>XyYrTl{I&YpmK1Y{!Y`bg-d@HE?n>a#YH$_#*J<&Km@~v#R5L1VuyC?SN zV^4<0e|zh0R{rfiY#MkEPerF8AN1!&qV&|N3sHSgI7{en?6{!ENq^678%dP;)@(Du zN3-`it%YJ~x9TE8b~%Kp2dfaGx|D>cn_StX`IzhlzyccGoNN?t1j9K`Rn^SC74!el z)9j&q!QchG+GrqT_XZ?`F(gG2v#+L9WUh%#43~NQEzG-+79drs{0~)h{~z{ zSo8h=HbWo!9hVm}a)k&lpz&yuSa^pFF>?Wd3sYP1*6ZKf`>h~nGjm#vy+QbGl8=f3 zPe*x~=wob!c}@gqI2EI(uStY`9mPQ=az>`q67iUG*`}W1J`A#RDGY?viE9cnFJV`M}8)i9R5u((W;3+wH?h}QuTj~UAKtc ztf*wYZ!c_0AFN-hj}iONdeP{(s|^Z6S~ue5JI%7O5vkblaZZ|Uix@=B31Wf7&ol2{ zVId90C^-l+LO?8BL24G$b>QJ=3dCD5c*K0l+(Rs{q(i%5F3W{bQWWG~bf6Yr2!tc;z?`F`TatKyh9B(GsI6*MS$5s`kc!E;EF(;(3vp6{&xd@%2$`ZY})U_>!O#fEXnRuK}Z^@+^B*0d}c?c=FXAd8*M>bg&ii zp1*aF3hn2H#tMjURZL3y*`82vsiF}J*4_H_#?F4&ANaWHjCno23WEJf^aqT|$;1f` zl(K_2WI2miRzjBllHI|S65PWQT7=~O#Xv>}YOCiw zC*^_x6s^~pG$#|p>YgG9wtSt*8(#04U13Se8O$VaO0k+LmtKmk`{2xBoMQZBOuW)k z;5lg_p@U;qdTRk6#mMJtpH2@cu>Bn`7>&SeXy1powf-};PLgJzXBHyxtg}E zO~=2WHtZJx_}-gu~S-N9AuK)ya%c}C84EKhVDFmxx{B%>Fj;KA3I(8 zzleLwuqwMQYIIXl3W}n1igXA_mkN^7-AbcKcZv!k-3_9&lz?=KN{1VyQyQeZzq$2& z-}CEy=jZwJcs;t~-uJ!MTw~5T<{0_0wwu9P);JBvAIUyB?~B@#CqF8DxW6r^Qluu8 zyz%Ea*;z|{-*n2nHuqz4 zwfD(dCWoe*rNFr*KlYB^MHu$2^^b$^FLZe?HR5QR{2YCtQgM0DTsvmX^E6IHWgq6^ zpAZ%%Vo)8njIJe>5NGMeH^=K;dav6!IxK1y2ntJdth&7~in%`r3wH+=)-GO3!2JE~mUffTn0~PSV7+xJY5l=U1Gqi5^;(ldN)E%i+~f03Fn8BCpbwsF6Kaz|z;piY>g@QzoJz^aTarHOyo zXP((-+=~(_=`RaIkTk@58|g=tTd(^YLm64-A3@D31pNWjtE5Ku>0p$0SipB5IU&Qk@NQX_P4 z;<3rj`@Su_D_zUF=UCXhXScth=H_!ky8VmU&wtN`U+w1U9SUD6Uq5gDwAuR-I4ML7 z55mf2e`c24vR>GGU0Aa@ot6K|>-P)48Xw}e2#OtX&ga)nDCMu&5>=XlKq1~>7uB7_ z=r%?tp-h>zH@e*?8JfWjafXi_Xn>gl!!S?4LSeYUTNnnkL1`A}<0AMbU!$gexMgg& zMqyoZv>OY(-w{z8#{S#RW$!AP-Z)UQ=C?-Siqx;XwiekE+PtT^r}yqv*?xxgR8oog zzG{pOlnc!!f zx+`qP^4E=@=k>BlGoYng^(~%=kkIoPo|F_~%ua8`2If9fE>H<$Jc+fP=l_LH$JIs>gi zh|DbRkZGhdnVN-S=nz`*lQn20jTs6pi0uv8m+fS6WQSa)6X$sCL}1hHV&oa7_G)`I zT>4^VWE0h#B-_1I-Ot$a&HM-oK@7<_h_RUcN1=Jrn`ZNDO9l*|I?jnH>ZvsxvmRMi z5eOHFeAl`cQcGQ?QH-8G+qN%Giiono@&i+VM@hUDTdTv7XWo3JZL~uCX?6bWZ92^S zE~hnEP~=wo?jn7YLcJx@>a0w5kl9^mm3pPe@b4PlS!W)L6tQ{7q@zW8X&t%X2-5=r z>LO+P2rdoPThR7frf&4O9#(>r>f=^Lm#CWS@v`E*)Yj$P=-%tEE*ktDW(l7>>hvE* zmfiRd1uI~n<0Vq3u<<-yz|(S8FWvXI>*}jQe3d#ZKk?H|43um`6Ot`E>2`1es zQ_6kUk6fY`s!(<_YLGXJMbqaujm@`B`fPk>fYN;Ymkj9kG>wkP{|#Im1G!98hF%&4E3xA{YNz`pJ2MrSxcEM8e6qW%5Z%{#4~jFVjp3Q zuX^WkL511W-lApW;brT^$DfYbV(EBA$dh)38_R#B}1 znPN%oC8#B9)vdp?T7#08(e$mSfWtuQrxLXI&c+2eQOPy>#aIrNh3r~@Mr!bCdc21d zhE4A9u1+8MxEEQ-*U5H!FJr7z+s-RDF5JcM@OFnEWA!&{x@ETe-4ykeirii}`5r?% z#yMrzK3bh^nh}E3B6HvY2aHG8W z@{P?qW664bOAR`V8cyMGROel8=X7(sWZk3mE2Eamp`kFb4I*L5VC4P>x0{@Xv)o1y+jYnz8;pFKB8!x4Of)!nVJ2%9NEh%=(5Kg ztb^^P;`5buoJ56quxNqN`|7sPvB#iQz2p0#1ZQfe)V9wfE_iRAW;yg!@(;)^toY|w zk0dNU{QU36JWmQ`2KlGbn4auMSh5RNXNlU~pM}r!sU?o?$Mb&XDH8K}(<^8GEqN2i ztA2rKfvP6>v8degDyD8(Jnc+dvRLV!b5HrY8?p5%M#VEMYF-Ws!4CmnI(2ft>ZQ9n z%Lk0VDXh&B8HsnRMt>a`X!7+nE=-UKq6#R~gxWiRq7Dy9=xR+@|K61misCxb>A0^~ zWTQqEKL+`H;`reE<#~RtI-8IOwc5eB<{3iFm-ro-TA1;0z^8;=Ss(4%eY3is#tZi9 zovDwoDs1H4li9cLd>9UncSwZt?-Q73wEUG%jGB`GA10JeE{IQ%2s)U0d0}4Dv46Gz zG=mjt`TdQbv4Q#b$}p9XNk=Q|oh2IhV_ljbQ-|MOZ$PuuJ8qPag$V65+TwRL(?zL( zEa=p^gRsKbtL!vkB%kJ;#eUPk0v6Zl)cd%%lSXx!R5v*dL!iBp0i&BN)6XVq?!h== z5S`S!X_%l`f=Y>(*X3(Ew6IU=pp#|3C@=0;d&jol0*(B9;Krarja#ILuORY~OZ^TP zmn5*|t}sikJFVyUyXyVSwe#RD-IcpKk^wwe(ACedn`!$=4>HU}rG6h^*3J5L#BdU7 zU|jLX|C&b99}M`QPmD6W!4oh$La`mdHRn+$IAnc*iI}#ed#q7FQvF}&2n-T`#b;bL zW9)?SPJAduhwjx}7bGjZw^P`{KMj;EzO$0aJ9<1uQ?44sI*hXG*1Wj5=a;GV%I>_W zSH~ZDe2dCt5qJY$GoiI-$A*7!HJKhl`WkqVY4vE%q$539!B_|X5mEcamSE^mT8 zMg2vM-&%aSE-*)*>(-efz^Ww#3jC;HM3$>#rDpgjZz6v-t)XE9u6TC6>~|3sT?rZf zjS!qZz|bpbG;kwx!gWVnhfAJlms>`5NGx>56QUeu;CG_LPryP$UtDtwe7QD3YNKgv zOpl!tZ4!#f5zK0LzI649UxYvz{!>o@%$7mYJlIc=-`whdUv6r}sa`MN@}a*V_~z_lDzPuyZv?Pk-IT| zktl)c6DMfP zBV&+H$#=BI-35OzlvzuGLG4YAe96hznF$^+mW0x=*#u{{{6FcKNkj^KFUGam%TW5~ zpZ8G3X{fp-84N{4T)k(7j*kBHOQGa`lc6rpBGhq6@!d@N!&mKhnw$s7P9yVMET2Wy z))?!wK6Rrc6U0mr;ZtX9w81R>(AyDw$mA<@I#}vwoha(EdgV{UxxzimNFRb#1@QoC zexu4D`_Q0*$91RA$$!}b8WQ9ai|?RY_G!Mny|tC` z$C#XHcwyq>SK@lJk#`w%6^<^a2_m1`x4KtB$IB)p+%iD#=^|tug(Kc;f(`j;?4>}; zJo{wPQqifj#k+LHAx9}7sC)ZKbL0s?QM%+98m&Me3w>g)011;1{`cDJNTXU|42=7P zj5!DwWEltNg*8-7C~S|cURh&4h>K*&!$*Jk6?%;IL0#5&jw;(KWwLMN zH{)N^;X>5;UNm&%!kn|rFAVOOh(bTGRLOi7l69VbDi=ee{4PAJZMY<%3Y2y78`8mt zFF&1$J(gkYP%P8OAE+)|%y_Aw;WcR?9!&0X zbIW{IpgLA8031=MD|$Wh4SExZIeml?7MXt%-h0a)w7#vp2!&+6LcBhd&=9W?l|`Ge z&uItD$K-i*oo0G{-X09FCXaRVW*JGvmT!V0O8N5<&&=x_tE~HXgzVXq37g4N99ikwVUEW0lgt+EWH8i(f)o+^)Kx zra=iHPeO^z7zY;#+C(|%{6|K*hjM)apx1onDOUn_0hDEef$sMNa^3tkZBM0kRXOpO zNIC)V&^vx%d^x1Onus;EUgw=*;j;grC$*xP|bs(=QkrSbf3zNiKmx6YE4Jz`BRd57;M~Wl|f}^XQ}*t|IS#F=gDCtPh5X z?`bf2Xa+l;*WZHZh4YDtxa8g+feg_!5X!(LDe72%A5BK`Lr>)oZ*pYdR@22jpeG0L zt_P{G7Bfawow}Y8;^DSbQC12_QbcA!y8NR(mzs#H{Vh$v%h(9bn$NV~>wgj-t&e`` zT(XDZDh-QnTl(+K3-4ZEk-e$>=td+=L$WCec@^=TF=f)aSh`_yXF0SaaePo7m8PjT4{~yg<_A_tHv?)UdfNe5Tb;NT(Q(( zXxrd-Xd@UD0nlO-n|$}Sjn4GXlxV+8KAmwMhTHUksP&DA+v|}nqL;+*B#`Rss7pj< z`G+Tc;Hh#UxOXiQ3%v=VyLz!9B@B)Rxm59iy`ff~v_`>+`8}vbNsB7Z%)bl}KOMT} zwHa(2^W5Nv|1%j$Qz^9fz(K2V-3UYZf%THXq=292imBzE9{I0ndJ$dgd=0yA zdCCXD`rpb;X`SbJ@yyBT7ieO)*<&=pIP&jL#SN+t3uD22u|0}3d`ue{L?~4I0h9jh ztdOjPB9A-EA2b?4-jc%o8L@<ZrG~e%?mx+w_-_b>i?ve z>w_@DZupV!BX4gvD5k!q*N^-pCuvHFhN9!l&f366TR_ztPHZsFOxn+FD0c|`)39iD zl1y`l>&Ys@CQ*Yk_<+vD!^N8#$a}Xyidr#?v z6$CtdRTshW^JeBTn$zPrNm0dA!Fz_-B@H8iPHXm8tse^m;>D?|5tvyI9?CVMp$wbc zZ?!$~`dRSyR$MtJ`mMq4+ocK!rXnz~2Q0UuhjEKZSt9gm6ac`(V;s1H(PgSHP-7V^ zdj;9YXd25R|=fcy{Y^w=*NrtmlH$jGksd~ z_IoNMG%2sod1Y#cBS1+SFj3Ix>MorY3=&_*e5khD_G3*k{#f=rKv`7U;$h= zJD+ZTr2=Ap|DJR-NFA<(TZfcHaB1wahF>vbkDhqCx{#~N;CvsbY3Ey?_-Al}`)J!6 zr5dETvy}JHms^HKt%O;;R7iuOANlbAw*_O-9(<4UO``|r=S^KCb~i%qvn`Che(c%+ zr)yxrT?OK>>hr{BpdH^3Ts(7`nlOEc zOY3Q|esLfY|7UXI&m^sufg>1SDL3V6-U<&K=oe-p6eXAfShY=b!rSAqE0b_cpka8rdt%DKLbl zupxaV_(ZMKVI&3pfeZ!V3-R|4Y>6|*=Xa0`va8GS`;NIDy_L)+h9QSKbO><75d2kS zkPO9~+uMt>G<6g8o?0&!{jMSqGz~6qCgOzrj<|2ri*qUox$lX2YYX8^Jrd8Lk`MnG z(%AcRW#89fnGv#Tw);hoXFPk5E1X)&Y1{70d*M?hN_c3G z8DC;zW5Z~2h_4x>S}!*{Af+CvP4U;HoUEtM6`d;DuFIxqQL*`>wru|8rflI-)3 zyYJ+oOq@tO?mO?xc)RISDfP{7v*FpKN`!>Kne;<1vWTo9&%Uv9IeymVI-_v1bXdQC_Lg{m##iyDx{wd^(wcX=I+>e zBvxZLO)>8yN;BwP#l>q$h1kXnClovM1Hn-0e4|FO?i9gjWM=a=Cjt?{MRs#1$Ava< zIr$Qn8T)kcxqy_$b~e4Zty1;bVJuEZ-009mpcQEycTe;D$fJo$`=OKKR;mApRCs+>6ffM!H?(h2heLm4X=ZNvtnFAu)MnZjVq+wl7kcr zC-F=w{^0&g`3XD9f@QHiIuN8<=NgQ>_hK&_i~%l2P9|H8Sp%!~YbyZ)forlPe8kM< zh?jrRLY05lDksK2@n=Eq+5YXA*8xMU(}O1EM27UjOA|=-!jok60-uPv)0Z`d9xFGn zRREV}0Z#eBf=NkDOG~RNYzO<1=roj4IBwGC#74;Hw5sz164WtrKxCi~1_W-?%sU6U z`^#BcGR3kTP5Ec&hi{&2H0D47rTzFD%1#VaLsfv!m)bF4*VHPS$OjT*oD)6zGy`9Yf z!B?3lRs?;ZS;ufyA&!l-JK=82{FyIMtZ~Iw2l5^Qy?vm;+q1gs9N{@Zt zf1h}`^sVOO4azsCKCuik7i-|rmuu0R2ISm~c#o8}Hkr|SPySy8I{nN|bE}ZQ2lL>} zW*A9}bk&-5`oCF{66%tieZ#qyB9L zyL1@EmNh>D;fH6vpD$(DV7S{_61!{ivAx{>RH(+~IKP$<|2OUZ2Gx7ElPRF)#(CsA zIXOw>Cs+ffevYC&Flcn!sCQ~=%1Ycw+~rR~=hV#1ll6NrLsvStZ~wm1ZR=@HVk?Ba zxJf}Ox`m`SUSmME&xOaN*wPqEb}ve#88S}UVmj=%6mHuaRbM})32->K_P(TnpyTwN z#~0(-t&b=C^roWI^q&Zj=ifb<)X;ocbc->Gb5-8fMbmnBaHG*IE?0L}+0UA`CA<<& zt7^VaLr5)5Kzgy=$r+U+U^^+&jy7ItOAQKV8gZXyaoIvFc*kzAYKFYK#mN!=z1|;W zQ0$JzMNC6P?n78kGPR0UOC;=Uck%HRpfq%k;Cp;3UqRsOP3vv%}w5oGAwtB=~Piq%>cVuA(wM+ zF)9wn>skmL1j3lup8YVQP?BII-%InA;`zKsIYAd=^Mc(@33G27_y^{B>r_GcEfrW` zr{m$xEv+EZ5cfns8APVq_MoaDzUL}P$OUROyP(Ud3l%K2BT}lg1^oxo6qeg2t@E(2 ziIhQnaK1X68iA0|3d`!(-;!OYZ)q~lK-hr|Hu6&i$P%`ryqVLyOTPoT%y8+mKu(x^ zja_>joGl%uV8V_=I{0Ot++jU6bWIRMbWwmOPG5}R4=^vy8kK;0etXe&z9V)Q!YdNy zcy8N9bbjX&GhAnw%UU~hArKJs>x$=0pCf^zReQpM+_SxpbBXqTfS6^Ejyw=I*Q@wS|fR10Eh;)YqPhAX@OPRjhXx+3ge!r$A(b#*18& zofS!|hU^`M1e!PR!H$WhK3YWK6e?03j>8@ppbtrNe-U-MxNux=`Tikt78`*;)W}94 zHsPe?Zl=4#k@uCy(?}g$%S|2MFq#SEt7hR*3OdZ?g}2^kR#FdoR*KiFyMT^_w@r_3 zgD9}?mj^M8)@X7ov-9gMXgrvH<6|yczonQnep^Cb92JlH0&C&2wHv7uJ^_3oC((F= zObSwy%!=n&21&l6pxjf$37AK{l za|A)*u+lOHx9ZQD?|IXDFXxhx?kbl`BoK#tLq=*Ksg{MqeM@J7Vf=6r#?qON+~NK2 z#To~E)JyG2M>r^}WdgOR-uEphTXn`CcaUO(e`D}ty`a?@oTiGn+_ip-X(Zdy(wYG z{WMw8bdU6MK_tWImLEAr6IAf}=h~yQUgL4LR)FG>j^lw9>vMRupT{KSP>dJX&xhok z#0Uo!bB?;_YB0N$h~)(mv9AUA;57&?Jw)5;`Zifz7{C=K7M2yWaD4dd8>M@JkIP-Y zZLEu3mI(n2!{r22rqnq3*{l)Zz)QDBU)a`57=F41uiYiK-{Y95cYuQvE$AAe!ty8G zV8g}fqB!SH1pM>+_m#)LoDXOv;GRKp;gx8FdQ1on{QGds`<*KWoPLApfg%;%((bkWwPTaPy39ism;@^cZx;O!5dIKpR#P7){vMX)hPZfBj&X5&x{|R% znjeXp8Rqxdq)L(N?!s3Qx*3-*!>hGhQiCg*Ek;Bf1f}!zO-0-vjL4Z*PNj)5F-}x_ zwT4RLaH6Bpzzap9ZCS)?JeMo<8(h~oM1y=DxKg4w2*=w-b86zLTng}m)i9li&a&`P zT)HUx>z8-P<-!Z)naW@kW@NYs*FqZ;amSeVVy^@!opv2v0zT6%``uc8Uv}$Hr>7Gt z9jd(K8f&_0cV3+iy5dTShh_h{FSrz{#99kzOpq3V3AfK;dwVCFO!S%H8tX(k8_bHl zOOuq2b}hXq#JjW*`0wY-fBWL;WISRBC`m+0zmc(}F|RB;!|&Y@bBT+}_U5+%bd70F zEco_br=wx1qzNw}KnPZe2#3|v#~tciJnlyxKjYe0t)46%-EhB%{Cl+?h`+y%Pw`OR z`m}y>lw}+idme*`DM;}Mj>4Em%0cM6 zcOO99Ng3QU=g+|=?mfge;R!||p`=TfNhE-UK?}1))f-wy-ckWihBsQ`u?Z+-bV^Ml zPX-Efq7~Cbj^xI0VL9S8mbc&D=v_Du#UaJpmtAKc>Ni)CC)^Nmn!VZHM|A$CNr% z?LIXgKbC8}y|)!LW*ac|{syU9)d|=AUkE#3U<_nGCI^5>jCJXfoCGMwb0^(33Fthz z07l_JrL9rES`GmgCZ;k7s?2Z00Mgn4?v*Q9@B@^g^^j$;Revz~v2xO*bF!?h-WTL| zVvVCVwCddKlSDoFV1VIe9EU-Yr48&#u20*AtSU9*%T`hw-^;7?aHOQA)VevXg(nt6 zD{?V%@zfrp2Ze-Wf@rE5h&TN?fe8Rp0T;kd9KMSH#<+J^-v-)MJ|B77Xb7_L$1pE} zf=}u&O*h);+c@6Sd`B2q#ULamo7Q><%1qp+UA-sG*wo*S#_LALt6$x_iUy%@vb`>^ zC;rOP*2h-v-OeG(d2lrGzkekg4*}xbcaA(j7FqG+=on@$^v?+h`yFVssvXSQ*PuR5 z$!=JuP1TwWTy~-qVixtc;CDG+0lhgD(0`kAAvk~Ddws{VMF zT`|e;LvtcU_fdfp{Ua#A$=fZitvv?1VAPDsx?nlVU)B;@r@SZ_#F_#MELv?oKfp_RNC*;kS6G89^V*!*W^)q|7<%MK;az7 zH0Az+o83PnYWpJr4p4)C1lTl|st(4phN+X&11->m#cY+8l~Yy*mj3I%Rr%V2C^4W?334VoTi1UO_X}M-T9vhXmmv z%(umkRKwI1cR^Jd$G`zR+tMfbo{u`ar2)e?mpf{E& zd3o`Q5BLZ>c!KpoE>zNaL2Q;(altQQ^^$|_)0TiM6T9LNClJPwh!W@wIPI-yFn4ra z=MpHH1vG+7_CGW-ucrv02+T1fh`9900KD+-pV-xl6D$bL7c*_U8{Gwq2w@;0qA>JO+q1H#%(FqUXaU&C4F3e42^a?ax$5RP`Wo55_-0 z-+o6|hq{)fE_Fda)LS-J{)SLQ2?^%k|G)X_Z@6rFdKCdE3!!}(T>}0<1m8!&OoZ`W z98}0duw6ok7HPWc6F;`46LRl#Wwhd|$uTeJq2|Uges^Ik2Egxv|c+9V%PAeG&*>+Pp zdU`9EyBf$*qCOD=(cgTXvOBOoy5~1UphXwpLwR&`^b@8be|v>N$}7W3Pq-tL1S0I? zKboO`v}XzaBt$oU?A1Z^Y4FzbnXGmUG)%+}Z{JNTI}4{fKk&79)O+G8DNGbtw1zTi ztqb4mg&_XSy(qbXy>}!G`@m?%(-XbQk+I#ezCvAC+Udx%F8_kDh3oeLBA`7*+&4|{ zDH!nzt!<2y<)~CV*NJcJFSk;o=i=%Eq4n3-C@F0q82s4W+2MhyOM^-A+qb`M?(W7c zI`D$;i3x1DA1I-AxOftf(&T~TW>#bno>cp1)cyjY=;4w`@(QR;yM5P_J*4K21j#Ms zisvItyu2|WQD-|f^+}6!w91~|28=BRH}va0Gy+1j3bbiOJw(boMFT`kPXWklA1r!3 z03Ge>i~du9#_BM=cf6SHJ9jtz7ser22}7ECDGTxz&E5|>Tl{Y+1Ra#!+^Uc*6PAJ1 z?Pa!cZ;+QEA1-Ey1GePT7cx`c2gl!dX@qqnbf+quE zF#aM0dhYUK#tIPnvY=HP4dji}E|Zy&R<xXahcS#86 zu(>bXg|%iE4!+*2Jh-nt7v$nO9z(s@aYbi#HB8jM$c~EUfJ)5gGy#ICBsj0Ubxu#@ zv4C1%VlHqNdPU+u!=b$wq&vTTVuzK4FgLBClK2tL1L@y=C}C!RMGkWKvk5aD-2-#1 zG8ht)f6osGMs-iiSCDzgdHmKrk?0znYtv(@^A^9wB^@ar56m7&XLiH!V@)v>b)$qCY$`HD&YyJkuS?HWo6B zim-rpYb6&+_Xg7*Lg9iP8{ygZ1}e;Qcv2m?=#{irBY-ARBvRDlhLgO?{)3`J^UCNG z4$Hu!CFITW7X3Z~=foF8sF;PQ)6MQgLxm|nEqpW@)Sv&?uywl>3P|nQ_y&0INRrBD z-66!8fy<3p&ser%YEo`1`tfhV*Go&5s>H=1sWMyx7{*YUI03sk7gDoQiLc=BYc>TOoUC&cXJEO7P(0?IbGKr=OC4_!2F_p! zkS1oKA9@!U$5s@g9s|o2Ea<}!T*Qq3bsq(foTbTj}Bi4>J5eKAT=KX&u)|N?}!}) z!R4AsJvU-zdD-glGyY}P?YECf@=~$$`ySEKe^`F&_7A=3AtXexCAS|!JeU0Vs zhH4TSF{>soMD6+SFHmCYX8u=_A-N_4bLe|Ow0S}y<~5Asip;gV6mJ6k0i%b32rgNo zyk9S@Yw9YF4>mZ9)1jG74$}XRpgISb(?LcLWYf(+-o6{SRhds*#sh%1(J>E-lH~WE z4aRUj`U4uA+gQs00{{KA4v>uf43q8JCqSWKK?@q$Ix2yOS?vW5u!6-$!#0BjIwo@+ zv7KDP2BTZ0kQhlSD@TBGkHEUbk{Mm4qm0^z43RYTXKFAC96&chr0h*5ilO?Ev3QRQmrmHtyT-1f# zTg0dy6`+>Ff~$m1j07YW5uHcd^U-_FHbW!NitACZjG>HG`8{H7`13;!M=`Z7{cNc9 zuteHs%L`L%ty+Oy#^?PVvDTp^o%ycBXdoTmi3bXSEJ*8qz6c!#X%1#6FPXa3l+R|> zJK^AVTo#3~a`wS<2;U?q%rnkN>|sT51(XLJ$^^8QVB{D{-&C(M9(|pR4DQk**N8{E z{z$EBF_G_e14BKLIXB={o$LW=Okh62{)Gv6Xk0E!7Ie(sO@Qq>nIP&}F(+^aW<@`M zHWJ!+ek^b33ag9{bzi@UFjl#yjBUfvdgqV$BasnMZhzteB>U7|^=&(^(Q>QMt=U#` z2$y_NO=36-1->s|z-7{>38@@wZaO#!c9bc?{vJ{?_ZO zX(V4?W{iA!KB5{GJ_`?@2^!|9G6?ctBBXjQ z;&VEz+^qk2bc_3T%jx{dwdOy-s*=gb$N(&mE+C)Zl_r{O-jgDmU*)n!28Bl2o_);)*&k*>0-3=- z>Go=Su~&)ZXBH*G{4Y$TxrU+J!(WSHdQtp-RouT`uF9q>4^fG`22GDtaghm3q9 z@$#P`!ww+?+#R2tTV7s{`O_h)GT#{=KS>x1YVduqVJsn@a7-yww@P544i^yAl_k8v zs$D_@^;DVa+ot6YQ+Lb((bw9?Ko2GzAnegi`sDM2YtaP(~R;NqGVQluNKk2oG!lc~2Q+ zB4YGqAMUxJ!r?9~tA6<1g{CX55P|>1hXZJAj=pk12@R0KZ?_mAq9DgpU*iPLehe3j z(g_7UdHP1!R($Gk3lkk{#SvH2Q znr|!To4kwESw@D%zd`iTLC@!Q4m0!#98!ZXZ;Vwc;Z2qT-GT*9Aa1Fc5lE>l0pFp> z8w`PJZf_%lBOJmKS@j2!f(etoRUF`42coGWAmki4r@kM}96&;vBxZ-qq72JcN>@bT zet4F$wNDoe09j4hv|bu~MIjM9ZOfa2ZHbQ%c3zp=y>bF8bvxiw6r}3%;aQE1;*diK z4<;`I>Tl~he6?zI-S&`=SZo}?94=Ihcj{mRsnhbVH z*|NgF*RslR?&H_>32|m!3JTOpohBxLT+vigN^0a_XC7z_9X53oaU&U&h(X{nD4KkG zdo5ta!5Xy#322zY)8Wyn{yNdt{s2aY$Rok1)Uwr$Hiw_=or9G24fF5Ox+t-Goq(=ujhPx9IY*EyS%LLSRPa|AWu@W4 za{o~w*GB-<3jIadXQvnGqMj~^q3uVVcZ2ux)eDW`9E*YWv_L$m>1kS)+?I#W)I6>V zc$v`0dVheMf{*t8`z9Xt=@U&Lzv-N{?7wU+qrSzaEdeNzBhMdScW1N!I!7)#I5m<% zL@$K-5Uy%R`iryka#;Ftfs7t@gfvF3QYRwp*U7}Zs&-=>)3QH}bmi}w5)47?fVKP7 zO*=TA?wugM4TxN;YD#4Z`H4JBjQ9V&eXnM^UGUU~cVRRpAJD(jykn{-$C1Dhi94UK zvGIGak>H@l-9+yS->!BSih+E^__Qt&SSAgyOfE0}+~gLhIQHce8UHG3&hiUwuE?{e z9XWGFqcY`2dCQrT;LxmYv4)hsX+k{7Gw9=BF0yp7V0*h)ogiqow1iRGHJ!HEkQvg& zqH1rb(~6ZxztP74Z&&CA3v&L1WOT--CS_HmT0U?(9O_*62Qe1zb-A}ib~<{4M3vL} zxFA&JwXb?0*v$bbMDeu9UmJ-U735rk2o|ePZpf_|o4-5GV?3L1ku!Sxs^PE|sE$1=4=Pf7+rxc1i{ArV&r_-Q7qShg}*FYLYdODoEwB|bf1 zXQfLn-sLtXrr*t35?;|zJhU6W!9GbW8u<~{;H@+NN@vfR=_kQK|Gd7IJ9lT&Go72%I`%R(3$^$+qiwf`3RA`V5h zOgzH?rXv0iaB?F-yh9E`^|XevJGZ<(h@8uzdc?rKMuJgJ9-QRJAG%+xt55oxpsiS);#Sl4Q~O@3Tf z#nKvby?jn$w2;`G!kLMp?91d`E8=gJH{L3cR#aVZ5!pzNCVfx2p3_+k)|d`CxGX54 z?159Kdx=tCGK?tg-d$PzxurDb$$sJeVyc0?SiCHqvF6L)?SNQpx2qTFiJiH{+gK&I zA?d9Dk)4f=ZBRi%aeQ$kIiLMmD;hhe_xCJ1w?XMESAYCL;(~$+fNTY3=RL z+o6hmnG!aGj>9>~Js7`!^QN88_uOmbV8kMwG@k@w!W<^7s&=={>6iB4J4MgL)P{qD z!}Z|d!z&t)l7Hy0E()ghIc$cq1usl{gn}VW+iFJzkn)hqQUD!8U{ib_?L7g%Q;S@> zS9jLla>6dSNk#QSQd*ik-vi~F1>#djCHtG35#6n=t>DD^X$vG=9ulKqC@jSJ@^D6) zO-alb$t(~kRnPQ?vM5mr*P{@UC%Iva522>OJLofMu^*%=R6*ash5PNRy5T~{HNx%@ zx%+nQIELu*3=l?xd1zCfQRb`mUR~qjH0W{uHI{L_HD-rqmUi_Ql+juADhmFLHF(z+ zq@Mkhre;=?sIL9b#b$4_U?Ys{9S!zeHlI5fw&;JU`<@`+x&_s#Ea>3>ghH5^5ftW^ z%6i43v)0v_Q_dy-b10~tb*lsA?b>#zBMRtUZ>i1<#e-jggpFQL3A-@fFblRrhi(%j zP>Pj11vbBi2Mrb{AIDDo%%tQ*FY{v$sjctNl3!eJDG@2sYkL?hGS>vWY<5vUmxOqZ z#hR5hlrT}ICVgYWQx3ECVjf})iL}H*kML~8HMeU^Xqa|9hS3fXl~iwQnptA|?JY$O zCBT_ghqDWDcu3$Fn!Ud%CXwD-&v@j3QI4!93(&mNUgNseHA1^{w@jf*qQ6T6Y0A;1vkq&Q{p7LpPK# zXXLOz=r;1fegQBbrYS0glLE{qG8X zykz6V=6Cs{4Iwbjdtw2g#ZM?%p}H=aWQOj{ATQj}n{WjVE6xXN4Hj-sfLRg4X(Z-K zeM!V*gq1i0mS|^nk-YFQEI?-PNg3pINzwmS|FS|in`Jr+mAl-DcbsdRaTku)b`r55p1-XdLfidCws1cUJZ!wCF)tQk?sU(fZ9!OLiSr`|VhWEg%2>lBXq51y!l=bL%2MEfQOpO{$|Y#`FhLM#tz0G$ z^t;(&uy@+c<9mLpcmmlS^FvmV)oVnz8lS)v#MnG}BmgeQpYy0VA~b_=0bDasVWW4v zkZijIQ@D|_j&6cq;FdG-q!>W|apXrh^+tJm;1VjU_q{mdUsl+-Ed4eMR^sP?3RHqW z&nM#rHJK=hLZSbz{J<+YSK0Kdlmisp^?+4^S;aDw@ht792=orO_x56sL%<3lV!Rtx zUla?Zx>ArgL^!7Pi@D8q>bs?2=uKo`1M;dSFEY7VR)FsNY8G=I8eG+D!v*iIkDov# z=;(gY&9_=+_Nd(8-(EvGZOkR3x@&h7h67aL6J=O63nC=suuidpR(Wc^R)>I<2rXn( zb}s=pO9BBRdKIuh+^KjGZmuu{<(BqeP*RR~S{dfn7Tj13#Kj?-R&$N8tVQBfw;*yFj5fPj#=H_u^JQ`L9PbTWN=?A5mjTCH>q2n(7 zN)=*|VvCx2rnNS$$}jW~GHDUaW@=-o{$^o1x>j9l(9; zYDG#+!aP?O!5SCgf3A1!Hvr-=NYmHL)5n@Szfp>~MW_P_PVMN!x9j5i-7%mcXbhD> z=oa%Zqj%?B3R|1*tm(ykgs{AyveTroSA8tvOFv&D4NbuhK&k=I#1`a<>1Lp;n9YGY zI%Y{N$H|RZuV(-T@%sBjTH)ztYEs=ITbQ3F>E)$_3h8dHeIg05!(%~`MgPHy-|hJ< z%J2OO?sgwF)X4?^f_oPMRm>yfWNtILf+)OcvGGsMxa)vC)BFsE}*8y$mt3lp>v&7Ew;3D6t@ELOH46+$>54 zN?4&9u3=g@(>h`L`WpXFoZYAUF+3!O={TK{R#H-;jV29!05*jp0DS)zVIX^MKW(EElxtC$_4dI*{Dbkk@(lJP zRK!pYkB90mv(dVKte1m6ouV{UlL7)ut2XCh)5!wh3j6ZBH{q_yxXW|R?`5P%7}IdR z#@^!_0fNtbXi#*a#!HukU^wwO*50x<#xTCIBS?}v%7VhtZAu_BKJsxOFjp|A4p71& zL9a-FR@ZlT^2TX*KC;_IP{P3F5a?~MphNgxhgL}at_RdCBj6zZPZ+3~i}#206+HoS zF>b2_uM-l((&~F~T23Big&D=^|LW~byi-X~1Hq5**{Vkfe}s1T^NM4udGS>O+QE z{xm40q(G2fRL@2r2D1Lf_~&Tffe5C*5Z?eaQ(id!RMCbmW@*9C%(Ipavm2|pTmNf- z<6Q{robj_?ZB(tKq7NkqAr&Ogwt19V_;wMxUYT6W05GgzG;>`{5+Q?tY^bLEmHl+< zODxX23oMQ5#6TH{{*qv_Z<;?1Aen#;F`Vgom6~`Z6$5#eFBCpn$-Y z{=49oVvIUgZ|?jtYmJpxFNYe<{MaVc|c z_~*l@`oC-~JgdIF)gdzj^VY}2a7M&GnFhjYM^eA$D@Zg2_+JeTgajQV3ff{I3fxin zBNeuz+91~g0**X|7(N@_5w}p|^E@P3KdhPRg1)cKZpzA6QTg0{`=gqct0mtz@l!vg z@9>@hx39ASdV%e1yoO_utLkWe$N#QKG3`rcYGVB7V0fYymOv-r1-J|7nM9*VUnh?& zdg~=zN);wr?<9n+sJb>%&ZX{Z-jtY-XZ&{iTG*!15>G0y2uyMW7wT;#>k2*FPl*n`A<65(Y1}@ zYXD0jGgRmyDM@e;>*V6Gt`|3kAJu=Z8bIPBXa_o0XBq&OjnzRGK|`+GxEZ}23#GVh z1+!*Er+ET*8BtXjNdOu1j6?6q1nl1TLR)gJ5?=Xu2=q|-sb~?mG5Rj9iE%$Vvb!EJ?;QtPMj|-U?2Cr}++I9hbgq2O_JN|2CpT zDp5Of-|fv!E&0(S)J_Dw-8-#aAS=dW;iB80zzO}*K&ODeEqS)eOgM(CGgJC*8F}5; zeEsh)8R0BLjUM3P;LxysRkefmma!l_kA?yEa+WUiOYr}l?a+xSIBteyQ@1IlmBg_$ zxQ7c9#qvVsrD+1Tv$8Z$ExwcbUnamDs`c-e)j3Kb?lW$P{K4{<3*g7lIqbFF&Zyb| zj7hrE8r)TIBpcuvphTB|k19!l>P#Q>h^(L?JGZ*}Y|02oc6hkBv=dKma={CU$y?o0{EWi%B1jzLOVx*y`q5~T4CqFh`NWW1IVve7Hl zRR6ue&6yT;5cNjYmWkQ4mG@Hx?ESg_A&d7g&K}kC?-tDi+$>WI&Lkan6{r>1P)J{N>n*Y6)`5R2ien1AMcy_3(!`S-ZDHaFjX!F$Vo+o7%} z-o2va#cui{>SvY9b_M3V!+WGP_3eBK9kyc!grWL^K#OPU-|#qjH5 z4<0hQ<_SF zT&@IM+^9}?NfylHMiT#*sPcDh|6ge>e&nDCoBSh;GI^=D(~J{E3tQ^NM=bcmhNB69 z^F;2w=@=3YoZuhS0(MOkftL^wG^od4)r2;MN`c`{q_W+-fbMM7NeR$DFwrFth+AG> zUQCV)U8Pu9SSpaTndRl83QTKI9Kj-nnX`94E@5K!fsrS$Pdk?HY8GhUN8v!!BY@%o zlYoFg&*BO66_|k)MD)BAnt!O`Co?3Ja~m5<)ZV+-Ky|p==K`LaDs(hKg4ZyH!$3%| z2l{Bjq6?_LASA!rr-09)X+4U0ZiJ4|Ky5R(0Kh^&q4JJeYTOiEZl4d+eU5{G=kfqW z0)yxK({;+*;vyARqg<0#jw0nK^x!aEU|BmK9fG8B6v7d>4k)q&#O9kC#fGVJacr`v zsR=mQ`k~8{2N6}V5e#tH~;_ zJ>X-_Re<{_5=#8bQ0Xd}x`2%m1GC@I@2uJd`wwF$r}FXM01O2J3U^91f;&K%kVo+@ zfPT`kMj_)+{2SsxE;O!i55+Gb@?}@nAwY0_1Q_)bBp^BC!KG3bqGsH*732*u(B_8_ zmjPArPO=Z<58Hy&QAbPd<1+dvL^_xDeTP**r60`qkpCv(?J1^v!(Iw){Z@* z4N}2%CvxmODA+}IqTssU&zq;8;lOS^xxhf|`qLW2vT?JF4fJxK6%p(CIcppm=mFN! zkKlGk&ls!|&(ThT$&8ArY3A9%+39K85e3wNqYUeee2%_biDSp0cxp6A`?C4n5jS^t z!858}ZJ<^FUG_e(P@5d5r#o6&UQvm-NkG34iQjfV{R-h1q2H5&JtxnAagY+;{;&c? z{UdD^>8yh5D@_kfyv442yX9M*xES6o4z_4bx3`I&mquW)fU>w19dY&^!gZfo&6X4Pl!{?X*5D6}i#n?i_}qmBIjgeER7L$ZTZ781N4dsvctfFV@7~ zKr-C_+6P3fOrSm}!t~7Xj?=f98B=VX9iBtGE&gP4?s5;cQx^1* zxSv%4rxitKZ6AY^{1c|9ZT;R&Xr3$lpQ-f}%#?W7S|e9u&*!!C_@rtnP$>tj;JhdN smu3i%2bb8u!UVhycmNi~Ib8cx&&h0jbef^?aRwmpboFyt=akR{0L^Z>Z~y=R literal 0 HcmV?d00001 diff --git a/docs/img/ce_backends_benchmark_gpu.png b/docs/img/ce_backends_benchmark_gpu.png new file mode 100644 index 0000000000000000000000000000000000000000..0e401099f999484d2c368263a9383191d7aca8e3 GIT binary patch literal 55245 zcmcG$cT|(n*EJYKL6jm*K$=REBE1tp6hx&8(xi7mdJP>!x)A9dR0O2=66rN`M5Klu zigW_fB*5JGoA+C@zIkWtjm8xRPD!O~Ox>L99Np|}%XaB$6!0YH@&3E_gq7z(%$myA$D+EGjiu=X;CYx^qflNp~mzR45%h;N| zmh#Gyo?wAcSND4Z_>sEGBcChxlAN4Up8V#`YYNx!$?+7Z_H;h-#`3N|{ccWkyLPQ5 zi)>h2NUCA0z5SP~@kCH&s*ttnpv0~(#-GzpG@ax=_@Us}@bSOzJ?=UG!N7vC9iAR^RxXo z*0m128$5TPhLJLUl=kru60!MtwdMRsKQ5^)Jy9fAIhEq{U`jg~$&5bRZ4pBnIFP{M=nADoE^3#xA^T<}D4)Za)NZ3WnXPYE$W5BPgzZpvHE$=B%hPA(9f_3XvW)OBhDADM zPphm3)$5&1iD~&EUI(jP73RotkL}kV=*39YWY4Sfbw=JAB3!zz+Ac0G!WKhl21s^? zWiLA~FOFs1HhytF?>B3mt+G~|v=e2J@udZeD3K=S7=v(K`(oJO(OcO}NXFC|%c8h9 zrepj$;PRrkQW=kcjOoMc8r#&n7N}bL6n@ip2*mDWMTc*;0~^HR)>N(%P5zqCpWi`Y zOy1Yq+S>L+dHm4Fi-ThG^8HB<h00w|jluWhOs9$sMg7Z%v_v z)PHJJ1CvRB?sS>=o-(Nj7;LZRx>FlKFINY^h_IWn;4a z|Mn)x=(arq@feq^_1(<|`uBIv5fa?L!b)GFfXCJ(iWJh(-mdIq%cwSi&S2;#wTQkW zB8Zm`woVYX6v4r7*+`v`Y=>)|xm}Gak)1`^9poF}aejSu3Xra}N0u zCq0c}h=^lTN;|zl&1K<`Rw_ruEPJ+9)sx6gBWT(8-e(7aylPxJX%xO4N-^5}fD?s| z(GqOakW%b@`B<|+gKc>rbu@uMGRcMihl!TZM52~4kG;*aV2&s_G8izM*1$7@5G=vU^i&zvSo4PhsXVX+VOMTPAqe>8Uc%~hBy zYVLO(nrYpwYb#s@Tan~tw~cc}^hDC3Sfs~Og`lPviN29W=fD5nA z&dwAs*wzU{1=SD9Zd)Wz-3zurmwuR>W#WyZA};M>*yyQOP=akhjjf&1oo4WVx10_MtenX&Bq&``6UkN2#aCnt3&doWgeljiWx1_+ltdotAC z7OHNzfRU=m9VTSG(3buH|F+S4M;)_T4w=SYDsJt8tL4QvREK_sDJAo!Mg$39tVTb{ z;g8B3tRx$Yirr6w?ms=~=W9u!c%s-C;oQ=4H2s$SlgUEIai#Z2VWowWY7`BhXzmB% zW*Cc^2t6ub|3K7nj)CLTs!Hvv0u3v@m^Qv(L8fdZbHL11B4U^>^$^#{Vn1Zb~(k+CIVYKL+m1WO3@R>0rn2lyG_SgeoS*F z2x+MC91KnJ^?5u2$f-hWq;Q&w658NoQTyqXW5ICjQWT%EN&nrkS$ITx#T8+KjF`!< z@7B&(;LvwHJftmsBMV*AW9(;EhZ@Zc zLwSixgj&!umrz;6#mLCWs+2`09D_KX^BG*jgqa-g%#Jks`A}M5{4klQy0fO|8xrl0 z;0I#`n#hbxtV1-NQ1=!Y6Eko6?#evY&VjO`+ITr>Qr=;tJ>;f+fkv*NW|Q}!9bfaw z7yGt%*J^uudc+U*U!kj=(NE7@6=N7Ga#hmD$7|l|oVkKvsRG4e*y950m47Vz=f{&; z9=*!RQo{x-CQxyXYZEv%r!TMvYWb(^$|)u?Yh^7Lg77+rSz9LA)8%;818MAD$LcA_ z!_(QDpe~d=i@onWo_SF3uSL-yjhSljEGuzZY_Aoz`8jz<)b7pAn>VXwZ zXFZ%%Q=(rtH2y{5<^eWEzpCBqf^|gO;U7yF>otg3CBB{BjLFL-{;~Pp$H{i9^Cs`Xt#w-f=D&3oTJNvz6R20;T zxn`qk%l_MS$aAZDe<)vl45&eMvX`Jjr>``D2pbhKE;YKeA9TFKMsJ4Z;UjzO9J{kj)g ztuFnOr-5vNJgFz(`WO)9>Bi;QuC3Q9j}JG-gvR70FTT5WbD+$)TL^187{`3! z2Lj}*bEUE`f=yI=j6&(WFX`c?Fte2G58Pf{9;^-uT^>v{u@tdlJZ9{CmO8dgp5F^t zuVNuNW(*jqs_ts1i2wBIlY9*kb~W8mc^#ruQLmg-PRuCz%Z|lIwDMPEu94jv$88R6 zxJ!1W_i)9Fn90u^0^41xnUa=2EzAE#YFXS)dbT%He$cfNh%x} ze>QgLm2{$~+w{gX|BeH*Qf#lJF~w^_?P2GD^7pB_-UAd?Nz39M_wN=7$L7ZH;F;UR zp-++L-R{*xE$OCOB5=kcV=vXc8IuBXr7sK;4lGKZ63i2**&{XrrDRw$hQWE}Dbv|N zs!%$qyhY+3Cc{HVg(*9lUM#KForl_dtRN&e5j*s{U+so}emxUUkSmk}#<+mU$cRJx zxN@(BH@J>s3}~LK;;MB!JvHhL#;X01)zQ{e-kqdvD_VxVZU@t`>;_We#)|SFQAhbk zlS;|(cq#2*d!xXqp#fV;%6Z?dS>(!hZSIQcb&pxFz?)sYwi!8!@U%w{jhlU~vMXLh zirvr_z!0?Ay00oIDW^Xo$Smfyc4`-WYBy>r^2mCOD|lVU*l$c3v&~JgeOtSOjo{ec z_bg7#a<15rz5`ijhot{*zR>28xszpmYZ+E$rNv&PU?|0mc$>faS5MoWJGpluY?k0c_xG-#FvyKHH9`fPXGy`}^L z#n?R!@WVR8cdgRYZRSc$3Ybx9f+sn*AHGh}U?vFUAvb%@rSuurp++n!4(Ia=tx=mw zfi`PU6qS5x_?fqSd%hCQ?fhZa3wadGle;KmiS{QLv_o}mTScVRoIf{Q9n|e#adxHo zUG+y)07j~hjA7HqVp`)^YsssIO!ElMestBQ*je`-gl-=+10b%PEyL(}2lcrk;TTjX zkxUcav>^)H+A_DTtpzm@9Pz(gJN12RqAmej>5Y+IDdu|Y*%HoJ(Lvy&_qxSb?_F2m zrqyYy?iVD5B7quF;yeByOCc$_k}R${T)ps1OG@IgkY_hlB z4Q0Dw_hvY&(}1Sv-sQH54XJ?1vmu7J7Kepyb$nAkaZNJ2Zd}AkF(B7>CO|5MAAUBE z*M-)>hRTtG_0*8Ked$WeKISCgj#pSVKrs#2Klmkr(f1CSqDW=dBX;hF9+ zx0Lp$d}iF!X}`JL5iomCR68Z{Zj)vnPK8`)ND9Ot^q*G`>{12ucp}GVC4R5(4rYog zI@EOPbTZc9CFpmH^6b1&DK4a)*kG=uakH!H{5ry;Wl|=?twuMhvKNZvQrFRD;^Q_k zqi0}ubnmjgUqq9YNVprXCgYqICb2(bSgvQ;Q+@w7foI5 zLYD$*t9$gq{h?g7Oc*jk3$`WQ$2!zoW9&t`FV)Iaov)T%>&W7WjbR&sD$Ycow7Iei zbbP}wSQJRN(8Qd@Y(GC`Ecd=$Wyl)vP2B8m&w0H1*{h4|%LwG%5oejB-*r!0pL6XH z?sz(grK{`KLkW<*oD$7qo=XgiaHSU;>R$JEzFh^RLa9jCvWt}1ZldESeiA9a$c<6@ z@;j5{nmK!4ey50r6b~F}6J5tw5{Xq2m5 zju<`&ZB4(0Po-V5Uj;Kae@&J|TM-TCUKRCO&rKdUa$+n=ASwRoF)=A4fI^ejMcLTs znss&W6VrOduS7?W`OVxWfY>>9aV?^{7Hf2Kd+5bv({t&Sd)pvMr?>Rtyo+Q`7Vs%8 zs_+Rl4|Q!!CS|r9mO0QQNm@C@qH8Ki_La20lBlqp@tNbDw)VygytARH^U+_nFjjGAoOS!Io z0N7zH%X@1yyYsYTc{_SOs-MxGFj!siT&Gr`B*vAM*EdUqp1w>JxZk;M+I<$bh{-I@G zg1{!d@tu@T2e_2-**>7DlP?k;nAXN_dH?{N_J{f9%a^s?(e&wSG<-%G3ogZn^{K&x zWNFnrGsy-HL07KXF-mzc~jGYzo{Fa3~KVTu%2oDWV}s8Au^;u9j093S_F_#twPl8 zt|>nI>1u)O{y(hHtR|mhj~8Uis*lg&(?soUk5&NK97H=dpLY9aVa~eUJv@zermBz` zQqbGJgD-+9=0yd!ZzdyM@2h1&YdNxD?H=nt+0HhBz9Bg8>_Ah;=X11GYc-fQ`=_yb zAVr{~ne22+WHghfaz)@L<}B3$YW zm6I*(TQu5xwx6{HEQN;L?~T&ZWw2b(R#Iq;Zn1|0rRa7I9u+w`IT8O2hyq~lS3X>M z1Rlmhazy`61&QtdrX=BoJbxqQvGsez-6YPK-3h?&LdA(EOL<4ncb3PLUk_A7I9him zHXzQG|89|p|0J3SBN|ay9#V%qi0FMNL3Z zTITGlLn_w=iOo^dWhr#(yk)!yi>X&-K(KvuN+5}uT$J20_N5J!+~Ffdbl*EZ$+6s? zuJQEsTRCzTW+v6@eajMtLVpuyhz@yj(}k5>5nC$7msBQ z-4raO_M*NgPOd^G&#x;HOU&_xedN+Dzw3*K6a?F9ov|k2tM$nXi$P*Qu%(+008t^~ z+p*Jifs02bw^{}ABvwU+qYgJG(icg(q?}=jr1yGE(I*`#mj{vvn^E=;fa&5%>_;tjuA0s?Sy#D}xfmdo>D?0%D> zsi~=zx{tA2mAf?&GZJFC9=Lf@MtmW2abiwPC)l|zl0>_IiaMQXhz`mU$+Ek`O0 z%hQHVLGWNQC0wn(AU&EdFU|&!`+frHNb^dk3Hs-P*2EJ6}Jb6pUPBW6{ zo*TAwkMK#Vpyj}WCk9MNt=kC7SJ!yzqVcQZH~CP~K1V6^bV`-Ff<2#N<-Ro7u26Gp zr?kzw0=#XSM(M{A4h2lQ{Id$9CGcS{X%gV|sT)#88Ke1Xjs(?f`?FdTV9##?Cc_W%k`)15?ZE?+j}w)aO8ILN<=D;$ zN|l1tw|j~>X2|wz-2eG+*PNW3L6sc&2#O;vizN1^^77&jU3xbw(h= z*Eom0{ArRO>fc^F=Xt(dffiW4K-V5%oApND)y;YLU#$T0TfsR(_Tr?&msA#hR`w)} zbQ0m3KYFJ7w~In~N0y)ro1Wc!kfOrKVA$+CH85=7lP1FLeYjzndW#HlAVM%tG9R(m-0WMP0D zUwZB?jC>^oE^0R!I98;C{v5_6NaN>ETGUM|&i`g1Z5c*n66yZz}lPID9UL^~V-_x!Dwg)?W)K9M#El4hVi#HqNe7PM zTnX3JyA`}fjc>?i>MsmL@x_o79;F>PX((nC5cdHY)4jIVxthY1%>m{YGx;aR`S6P(UcR>rdoPekd+J39ZHLg7$yOCJ3PK=>u8EKKDgpWYs_@ z3H|h!oSaOV-RT;jBA5=QiCA55YUXw7Jq4QgptWw6-;thv;j03>WIp3sS)H%X$n`;cOJB@bcR&AYDxlw17TE z{u(07*B@Lu;Gf&qB4}gFu0tfPpJ(J^P`MtSe*lMb^9cbjl zxGGZw_s2N?T?}L1Isk19)}e|I;o;>ggXt3<^M1!u+3CVIajCn2K1LqsjSHlKKS^B! zMAOCl90(X<$P*v}tDgPt<;ZqR+XWX99d@064dTYSKbbFk8%Q&%2QvSqZVeAx_9vwt z-{v!#*_Ke+Z9D*%0^B}AvbI!nyjaivU~SklEg2bl)<2S~lx*j*V}+e$d}N?7%5quib@Px3J^*yK3=EO?hE3>%JBZA$d*ennTTU`%RiweG*MPu{Fp z^A82KJGsmW{uGz05MR(b*>BR$`mQxUV<@&8zWWkVn26n_!B&}dtqYgE?cBU zCofKL5VP5Sj`Ty*Lb}4F)l;+M#=aB(neh{wq-tl}gx2cXlNiEF(8saJVEa=8e2xip z+d-V{IPH%rY4)PkDV{*-!IGoc9bM$CyVp3P&FV@i<2Q=$G4&2PU6Q*7u0Ade#mp~> zS7lEYg2i_&sfB&K4rUf)(9Twgt{QAdGZYmMAPKDIMA2go;X(MQOMaM%Hzl|GwTg#S zN=h@%BpNe`8GBa91$eFyP9u%}<`ru$#Xq4T?($IX1K)z#6aTrHsjLX;9R|mwaTl-8 zF=chRn}b}<8q7Ldoo2Jo7)BAe(}@Ks@`&|t@58REqX)~vrrlfQk+HH;NBUhUN&b%u zt}`-ofw~%P6T05r5y*F91_X?~Y!%2YhpHj`B-QR?^T6C^5z=JEpO6CIlx{FlCRJ9j zL{OKPzsIRir2ApSaJplT0AYkj&kLT~D73RR*((BUj9gBQ+DVf$jT?gCH--_V-?a%* zvY1MWTVo3_aM9O8sh7MKfoH{!d#2cH7#>F)K6%g_^AwsSc2?2Z`cXD~wnFmTgKNpG ziB}#W9*&cQY56qTlxyPA>r;->u;x)ckRjH99ax$yO(I$i$=bdysHF5;E%ezMiaxR? zBGO}fg3S$(+RXZ)vb49@kry8vl4~~uZ~V+r!xElzUm-FjU0BJsIF7GK2>+PQ*t1iE zO4YIDTr^KDI|j$={#aN(@i0rIcdXB-H%Pis`2^zFSc-?u>@;s5;^A40w?(Gw@I>Ao4@5T#Z4W)V)Ks7+=ndhc1xK_D_?E%R-Ju12}{1ec}Rk5<=|c@d8} zXX3wxMc#-@TKV%B0{fpD*zbSnz@?@$E?hq?VZ`J^urj^RdArfl?b*higg;wETPT`f49( zpFxktmsLWWZqJg8c}UsN)!Pwhl@^FiCdJv`)l*nbap-6d3=u<|ytu`)jj9v7JP;QL zt!4PYrM8+*(o@8aPSNtn3mWrnvYSOJB@}!U34)$?ii)PjKgjYM=%L!|*Sr=b$%^Xf z81$qgOD;hxhcf@v;FxY$+MgFSj7Zv;>GNu})lV{;=OS>lyF|u`6?y;QDsX`&5hqL( z>!JS`IS{UyCI**Kk;hf zE&eeh08;|oaa3G*2^MW!5AxVd3yDO(SV_qxl~;U?qLEK}Rn8hI#z7#6>s6^F_ln9C zJD80*;yMF9bk%xV1$9xmjEzM6x+{axE#+Rp=F@}(H;TEeNcn4n%<1JtUDh^>MVqLL zznJPvI^Zj6I(I_gxf)v;trUz7QYa>Yu50m^mDTUG=7|7#isTfqOc$BknjD9(KY|^Y zi+mo#3x>;HVqN1S!^B=DrB#qn)D3>WS4s6zju7P|plhYiHJ9Ke z>>f^Nb!d$e zb&y552$B#heVTL_qgFIs5U?KO9yZMip5fRsX&bK9sF@)~-KbPSYpO{n{;b-ZdY@gVRrBqL!z- zlh-Km8MKvLnY2p;4T;M*#mU!nXZ}NxKx1FEWE%Ew2Q&#>^ z0=TMGrA8P76!zn*p2=%C(AZEns2-SXPFCbRXN*s=6g+&AE#qegC_uF(Xrq6%0{R2| z`AWMmrI>zbVpi~FgA%^EUY$ecL(M#8oH~XYh4HiX0}1Z!Pg(vhlKErMK$}kD(WBt7 zzdFj#3V>jh^6NOzVd?gYK8r3eOSuwkBT_$nf|wzU;WVT8Ymk>hD`K1QT(9GphU--i zU~K&PZqRfwM^R3T8#KIjwb7h{FauGCnK#*C$ZvoKuGWAqXo?pg*3JN=DncQIUED%nq4Jmk8&s;KW8JWY^f)yO{5_Mfwt~*0V@imv{s%6jJzH`TW)NhTK zgm|LDsL8t)=Si_Ka-bRQcQVF^3wHH^j_HoaC1z%+*`$x0NiEL-I43ggNApx1Mq9=6 z#E5R(s08;u=!-1bUh#}wdox{aGw9juxjU!2Mm7<&$vA%`VdkC0B<-D2{ltL%a`kj4 zg&?#5R_|aCz?+QgS#u1@M38CfhyaB#iPuo_Kmxkj;IZw};7~fxz4_PWWN%U4Z+w5H ze-P12^%!b>;wAEST)5x!{DL)Mu8X>*15k_aQ-sjD88UDmH7c2sA}@DooD<(&!X&R# z6HyhlueD~T2*?U~a zO7z9vhlDI1&j-}W>eM+HzAAZ5P06kt?wJZ!%Br%nyL$;(aC%}{2sOSwb0(~P46bw4 z(weGk@gU=ULtwAMJahtvZiNx26kNEY+%4=K_2u@L~a)k?Jr;AX}}c z+_WR(3i9K}TU5LTlMKYQ(p2Oa(Z*6gK$m}y=9BHO4sHUMmh+bbAq-HHUIIzVqA!7{ zuH6y8708e#z!t*(;0*YEMv8R!ighdQ&)wbAzDh`#-kUgfDlgxODi?h}@o-MqEtA4A z=Rj!Pp{6%=o*>ZX=T|wPUJT9yZLAfLYkr3rapeqgfuTlhqVwtiUH;mMZi@_9k51-O zOFk3NNx6=Sj!+hDYCh0vTSwsZag-GC6QLZCiJ#YFp{ zRUQI5dMRIj5MKZroTZVh_n&W?$?G`z?(H>K*n`!#1H_HVh}{>Y!ncf?txu?mM;1*63F=7wK0*Am3)^)WzCOso<;*bMpUTA zZi-@aqU?C#cGL;-m=6k_8Nj*Gu3o$L83+sxvMl%SUny(czY$5r`3|&Us2;wiif32( z-mkyW0wXT=b7ZXYocEis$pXb9WsOpDb-WY(M5oNC>yuotdbx?*SEXc9(5FDYC!$=~ z#Jcd9kQ9&m$L;1a4od)Y?F+zDin0oWn&eKp{KXwGr5)}27$98%cd?jj)Q@*x7C@4rfWpsNYgn}JIZ z8(l!A7I5EqMF~zs{Vg?_Q`Y^pVK`7+37cO4*DH-_!6TreI{uxeRD%YA@`w(&Jj+Jl zRdmARL)eIkPjEh;GgpxMU0=`r^HXUnCGiNfUH}KXCri4)YlzD)yC7<(L!v=|cwHCT|g- zI)>z_q^BX_(nqv7BaW@QuLR^3b$K}4EaHiU#aGk>Yy?e`51L7s=1ZVi`oIP&#ykBps^*M6!RQ;Qf~N;DQqD_P z?L_I59_ohz)|GmP$~X_A-Wi1jdQne`K#rgHLJ;B7K*!A=AeU8`v|SN)oSRt}De|rZ zp0vCsW)x@FT5htDpcXAQk>`Hh<+;*zr3>wn*CC*Jmh!U$WUHmM>;O@%2$wat?P+SD z+;oAA=;%d<)17UOk)nZi_Q0V9e8XP?4( zh%@J8&AW+#ir7_+=L^iRdqBck0;~PuK*H{LAC!PJTQh&(qFg$(lqP3A82M;Ejhb%zAXd?UK{dYiBPwsGl6R0N%veNy_kb=iw95{t+MJ0 z<&1o`MS$UA_Kyq{7__|RYKs8O_^8FV?WaW~P zcuY!Ey1AWERFugwz%zT~rRPlaNTFhl?TT5U-YTR&ar2nQnRB(qMD_8@k6mj$B; zoNiHjAWqYN9y3U|q%;OPHXSA*$0dH0YcimobBw`dFW&g83k6yP4sUY5)gT~cxNe-( zZk3tof&xWM+aT6|sD2FOX0#u( z(m5#G>A!~~!o&IF(X9jEt@r1K_>>@qA@AR(TvmayxZn#iwLzzbZTwfox&m{jAhLo%QdtCvtqYKc!84GK;b*q27G+->nh(>s_;} zno`~q^EtBPG5k&iR!nJM5TyTdfO(etlXqi^a{#If&%b*5wq_SNm{ojaRPHbwj-KfVTf64K8Sxf1hrRKU%5^1-wqu!UAUx8FbTkZ@*2T$kDvZ1vO6 zk3;x9cMO0py?wL@hwzpT8VhfL{zlsaqegLX8bSxt#nRV*kK{biE>7{nT}&R!12L#D zy9P?TpT%(2?1cd^&hG)MeA)y>RBmXE?bzcy<o%vQ*N@1tBIS?M~#@S%L-2^8zskpfDiOuQsc za8K$T>p1BCj5K&Ujs|@N;5P+yY6@P=RoiGb?$Uvt_H?a%6paxK51iT#Tv^x{D>4Tl z0@Do6>y^)+KdZ*$;;-(Y5V8k+X9A4U;0@M=rV@b9LV(*>7;n1d1`(0KrIhx=hSybA zB(H1j*8doQls0fcq$PqwKQa|nIHHEThEc0tl~osDw2|j$$L7OX(yJ0l-2#wCE#P&g z6|@K^Vg(Ks+Ff5@G<%u_^~%UXo69g~gZyOr`^V=aBoE%JTeyvy8!Ec;NC`+%0|^Y1 zw?A&5;9!Z=h*j*rd*#ExAUzbqwRQa;?*(hxq)9RR{dpOeB>m>cT^#n&MYyotxXqKY z0664o7XX?6K7|jrifT|MNG0;2&-&{C4nhwmufOa00=PV7+&r^P-bhy~Wl(7mQNUYU zPAHH~1I`_WgNY0RTJWn!yQq?*Oc}rLHflWvf!b0s9>1O?^Jbnl{1=bu`M>g*l#yrt zSd60GOg%a1U0YTb|7VH!!14f~=AKUY@$~HEwV5BtQF&{Q0FTD<8Oxjr1Aue}Rley8 znk#2$9H;pdi^r^q#C4agl$YL1#^4tD51JC(>`4*DIOE z9N(jR=H%=k@ievNGi4(5C#bnq3CCg>uUdmUJjm1W{y4cIG$l!N7*1it>$^o|OZ3~_QW zTGR@d(t!HEm07aZY6X<_gP-pl=v02ON7bf?+Nbqi6ZhJ;P%F#AUBnn%WN$=~x#rR^ z=P3z*L05sPOK!0}Bo++x!AsL|?`hd9uyOuJ!3U?@BuG(jzNutRnOD~uaRSCw?P6VWKy5=gulWLb zni~6W6W&5#z|!W!XH1HR9AYLWz-cg@YDIxI0H=xr*m{bxs!DzTgMqH|<-fi6_!n;- zZ?ClCJj|`b?W*@WH%w%PFH7dya4#*P@fN_WjC9it+#|dW@FhT9f_YQp()E>49k(n0 zBBVj_|DlkS{l2=iu{Bl2PIfnuQ;l5G1cZv?{}p(JSFFnGa2yLv zcwOMKK|XZKuG<;IQhN$uOZHzrzr7$_UPp2kuCv z&NQuxht~5s7(Yg>Dxo;;H+K~k2DY3m-DD7Vf`Cy+QA$8tslZ^MxaST7j+tX23*4w%lIfx(j9n{o0?+;w%}2)Pl&gwtl$EO=dvv#QB;|Aa%6n!sl$5SWnRJ#Ir^;Vt#>}{(OCUmg0IKoaw$G&K ze^s19h!dD*aK5ABI8}KK_%$*W3tztlBY1!n&mIm_gB>PHfs}*P<($<3EOjpdOab;y ze8AHiiZ-i&OGHzI2du@C8mQwR)Icy{z<3(lx-`K0BAk{yxPciwL+}YIk#x25IARZA zhM5twlm`zOud#YJxVPuz3)W2cyBjAM_Dk9N4=Q+W#0)WN@n_8T-6sNw`Qz7PXY2w< zeN)UrZNeb2+yn!<-9W%mj*+`bSt@fpRRKhr@`2TQDf}u)Kh2{@Db9){79~f}MrPh?6G}kY63u z%HH8J;rsy%QV$M_{BXaA*L@=G5qSlKUo;D^x-ZMalgHH2EsojYV&+=BN(TmdW36(W z=m7Be#(wDM37Bi;^rjaVdG(15f#QFlsW_BI(O}QxqS^1hk!)O4dZB^VZe-d1i6dp6 zwFI#Go(6!n2Rn5?5H=hr9=R^fv%N^*@2JA1gK*{q$ZbxI&WmGQ`P68xcATFpm`+CT zH~-xz`u#IEDr{WC7q3WCZ+fNFI{^(H1%@2-tN4At=GOT-HIRxPK-{W9TGYJqJ8~Gn zyMC5URvB#t*Z4wJ0>$Rnj2pWsj^;s{(XYi&Cvs*Ui4Q8s$Ie{BSE|l*o+S5q0SUaC zgadOC*D)EuORV=Z_i2NR1V%1duqSr0Wp{Sc@^T@nC1&Y&uG8|sM}lLW!Qhe17cBO+ zV>D66xS|E56S`Zgf^}KI(zqvH$SUcl((L!qY zc(?$yhiS4LDKK-$sAmWYlHKkMWcD!AzO=QYgRlU>0d` zI+X}KYULRmxk=OR6Gb+)rssDec!z2@GVJZD7(8nA8GJ`%-*h&%e-(q^j{Wv$hl|)j z<7$U~ay=&NN%D_BqlYKvCMwkr;_ zM5ph_24p_&JNfbz#a}z(Q%`^8p2~~hr78XrYihUjM(V|v72z>8pEbJ- zPPT&ff$p;JDZ@iSf-J?aN_Wl*BDpc8bQX?>4)hM$>q%&NS*3p4;<{?cNM;}GO%bcQ zmmvgKsLYo-Bl1|CfE_L`nLjL?8!#S8&z&u>8nF$ucW0yTt~*?Nwshu^`90Fwe=2%n zI!QjgQUU}d10VJ~4ZihOrx=-z-~rEk0NySS^!+gN1GuC@)lw_sEzD}%p;6)zVq~xX zv$>e8AwjNdWR2f-)5gyGv&=hb1FF74gQQhNWns{%x9=|H{IXWaF0SblO3D}syqb%E z;~#YnGZ4Z6pUE%8JcX)h3-v7~T&v#(TV>NHrly8b__afxa@I(1#^8O#5 z(;f!bqukQg_G=zM4@Y?qy!WbZs{b8K%zHhCiZ*iVsowSZtRV1;_al*g-%9%$*{jyC zH5^V{tTO`#8#DNM-|GU-AAd`brTnhl8|*2WB)SV+6=f>3Z3h!hg2GpPH?nU|EZXq2 zt-!e_IJKO(gb+T_@za;5ykI9~&;Mxp9**h2RuXkGTvEr1^+vyj15ny9uQXl}DJ0Nj zaW=*snAt&ZW|dv28z9{IC$2{RDe}wM8vWjI*X9YrQEQ8FriWGTv47`#!lf-AdR8TGjfKC_RCp`lyC_-9 zM5b``r{JZyk0kNgyivyWh}Hxqu}Y2dCg$KNz6)ri=!)(&hM;>@9yuPdJU!WLC4O1( zr1N0gYeG4yBRp28Jk%D`DX%ykuZEUko5}7?LjsaLlp;mxC-3&+T}cO^0> zy@}KP4d7KiYAjoiX(gLysoCx%MtWX^><)D7oJuW0vzy+K{z>^cjBT3J) z$8wwBff}$d8`HCmwi2T6{g5{t|N_VP?a+v zg14Q>!j}*dJ|9I4ZH4(V8Fyz7j0t1N?O&wWj*z<^qvfB~Pc^GjyP&sYa<}ZSB6(%s zS*lGuU3C+4Xq!L&_MU6EL^NoHGJ2`Dqe1KC;>#Yr`(k}{W$zIzt014U2kU=oLq~7J z&CTsn=8p9fZ2!(5Sq>!j4#d}(h#sWip&dJ{7Z)A9SrjWWdTVzmOz`faE~O8DX{)?0)hGVzZ~%2;g~sd7Nh#6(KArx{8!TM z!Wo|_b%v0xk~=rO4p(D9ij0k5ihKWS0++h@tU3T`l} zeXYjWwQ`f1_HRdzRCr)MyiBRT2=8(5nQw~~{qX-KedD6>1E@TIdRX8o5qJxr=^G1V z7`3vv@Nm|OWS)wAyHH`dH^L5#a~tY6PAk7EG!L}pZD)@$#q_NVhL3(G&vTVe3mC=L zAH`Y1Un6`-Egz{u~@>}^YTy2#=Q4A{o?QL zfb!+w(|V^9XkSf9;1UVi(dR+dqPuJXP#!B_7_)e?kGVl^U__dd7!WhZ<2XwZOwfU; zD(B0iOcJ4>&(zq0m7Y-hsRMg7jIE8TO`pz0PMH>`>@dTxW^b?BdseZ146?o83D>y z>=Q3u0wz~s(My4&vd=z&L~XlUCK^AcloZe+3Q)jSY4Vn$VSk!YCADH%JAQm^no(ob z-}B_TBK2A~-8~BxJ8_D4eIU2D$?kRwsYEWUXm@3L_xjK0RIan73tHY7RXQ3DPF6PI z^kf=EzR#%taD#Elq>7K4@RKq?$+)o;I{Mqls^q-pqu+9v? z-e>XMr^$~;z9={^bw(Rg0$J5_Cc>!lRhh16v0rHx(x>-tL(kjT>k`hr7vjsmEHh96 z1Z}CK_gu3EPTnrvFpv(Qwl`W|_#=~avGOpROd<+7h}U~eRiep|x^E%cKi>eOiE)&U zpUH1MX1z<$E5vH5g=mt?+7;Q5zbuVa}5@h3wc-e7#`2LjMyRHa(Ph!=(C z`Tx!;$U+YDcGC}$H+7%`eT z=@fLfr>=D$yh9mYz~TPL38%tgOVbrZyMLwNJas`r>W7y=$l2e^`VRq!00(irM8UJn zX^Z}130YmV4W-6cb#$LcfJ7!NNS~}qKEwvibOpl3oc0@*RXuVx&_?8H^K4s z0*HyBG0_xO!0jxTreeiJAk9C)dK7oS183sORTqfz1cOM*C?G%y0%^-)XPBj)sl=U3 z^F@Iy|24Aog!r{JzL^(X-D}q*-&-fVIiLO@ta?Ns6P@IDw%x1v#pjWNJ;-u>qr}kI+w!}?+`nojP zS^gfp*M2TjAfXx95Sy<&H=A@SGdMN*!zKkeOWvWzF{3fgMsgV+RLjuM4~%&vq$y5F@6aIDMjjBNuoBMx->KI^7T25?V6_?n!@V zUZ@MKe=6GUDf{0PAzIi_8SfF)rpT_a7<7I;5Bd`S%rkeEsVO|CjTN_%2zc=mzTlWV zI1TOJwU?~nU+%KkqZHv7DUEsgWaQEs9VHdj`!bd~msu=Nq0Zw(tWWyTLOj1Tc(Row zZ$vlgDh`MbFL%$dCu(Mgd=k0viQ2J>`Nw5fKYNEz*%aS+de#Ui8q_b;~MNNmumAemxrQeQ|Hpm3m6Bz44wy@1bSn_1VHph123s16C+zlm-z@pn|ofUK> z!J`xW>F$x8zIWbxnG4hU#TemA()9w_e60#@3}ZdnQ{U+hu6~mbBOYbo_ieYbNig3_ z0$e3NKGN*TBiZ&z^q*Inw6iv7$hQS!87YYIq&aTs68jV`xCvE2z!h+ zVq!_cfSip*887jo)gb<9F8-YJSIDCY3G-Jx`U}efH_G2}HN;h}aT{FQ{+pfOL;-=V zp2b%C-1`f55S5Y!UhrKWXJV5cuWrjFQaflZvmC8+t45>rzqILf>48H*m;U4yn`%i$ zr6m`gm4oFH+UGC;AZM?t;<_d#SBz+TVP;BO%_MwNsHe zRdU|T!|)`$u~M-!lH7_cJ-qsPzY-^EtzRn<7`;*fnTI*Mah{(zZA3pPbz&Q1dV|^r zHg1YLm=*HD(@A1nn%`Z!t!?tp-dDDu-6#%4t820r^~eP-m(ZKWNBTVMr;a2$${fb$ zR(|M_f1hh$w(t^Pb1f+wVtcnbkm8FTMf$XQ%zuE-Q`bCdpD|&@K=T+vZE_d@#oifTaj^+)rkQ?#~gv49n13-oq8 zK)Zx+07Dpm_<*u>+kG<+pR3OF(|(M)!^o1*G+qp{MdR|V;SBj^)a-{vj^NplN&vSg z+(~epuLpk9$d*5j78b6Pf1%Q~qB6Kr!Cu#YLoVx6*6p0IF51Mmu3O}TFVLh7A1W~|BI z9Y(daipiX4FJ^zCU#nLOT7Oo==vGEJTCUG8yCve!ejanrGJmj?8uQNO+LdV(ZRVTw zCy7N)@=IbXG@)xj?bgChy9*9Kg$&ClP)%{c^j;cTxtsRwC&;C%TjCfyHywM9Y}wr1 zAtY~PqIJO5bmUKNIb(Y8Q;pfVzO|Ie)>7T7fSbqv zAv}6-7GGU(WAc*+n?t6L;ZwCn3dnX|S!2$Bp6*Z;`>21=S>htQ`ysKb?H0xI=n_Z7 z#UEuK|1y_`NT^Sg}A}+;xQqkzEjtUe3 zBzXkS32B;Y0OVTm^uZPpI?;od{vVaEj- z!KpHPgEw_9hAql@obglH>wA8I|?KU8Ks8asdYxb2m+=E~&pCowfA z{AXHP!6i>AzV}7*svue_)3aab7KGsobB3F#3k}dRnTV;kr;2te-K}1|8ww}kA$jor zOr7;RLI=bO!|=0T0;vQB4&H(kL^ws83c91sz%j&caORrN~IBZ#PWcrFSKE5j=@4=%9Sd`sK3Q8hK0Y%%m%t zSgXS4Dk;C7w%3#B-^C9i*Q-4+;_DJ{kTh1ioTgnfkL} zGv7OMl9!cpm<{bsN-vs@XFtB8bnmkX@voMT^%7GUQDndIpsw-7M5R3 zFmi9P1;KjfiL6FjCEDm(yddp^mCn!g%y|vNyUH@JDCV6xoQ6F3O!Si- z*~)q5=gBN7&)jQoh#1z3?53cRNefXiXYbfXoE=?kMe^g1^K<8eXE1!CZc&Ju4ry{z z_kap&&ta{+mSJ1vYw@i2U-7Y;XEiIFpZZf!=z_V5Dn7VYg_v5KnVTQ{*6SA>Eoe*~ zFw7a6TJn(LdIdbX%x3CDHNo_+0S=a#g3jec;KP{V)zs@uu4`#ay}wbmRQIKXSx_bl zd@Mda<9e9j&%v!}ddZZ(|JKvI1cvKpk@VFZLDv-Oiuai}DF;jLI=j{rA?knOfGXXd zZF{I^IAhXDjhcFuSK4!2^sSSJ|J?V4-LmQg7IVXu!BjV?gIo4iCgCQ|j2#sb>Za3V zvo?wE`qBDXuY_yHsHkiWpZtlrhHq38`l>;>WkkMdps#Y`b&0HXaZvE6a9M9QEq88z zc}G0U6>4IW#PzlIuWd$3ZQ`<(H-5d_JWD;5iF#M%^p2<1w~>KGl3m8td3D!c)rf>7 z|1!>u>#j`LYyr5a%&jf3j2ZQBJq0S^2`GA^Rl%4=J7fGE{S(WxH;K}Xmxw~z_B$xm ztLVI=RpKU`4&aMG|5BrUr8AJY>$u*YFlzmm&3TyB!-cJNrgSQf+wbnw_nh|*_Su!c zYgikm1TAS8o-`w8S3*Xmf{(cq{ zyE>OeX+P@41%GRf)0nJt2Z_TaPB3yuwOv`_&QGUjI2|Yl$n!pI&f>zc>t$(6o}KR& zWljuZzRQ%hp969Q+M8i)aDD-i9q!h)&} z_6_0qkrwaz?DAg3u>EkF-1$*)V39VG10u_AfMx?Z^IN+p>BvmAn^O2o z2aCNoKl3OvBqR#kGM$SVx^P<~_H9SZZK5|nq~|^))^N~1QlhULz`@Ly-sL$qS=9=j zuaxGEy|=6?jQDf2Z&TRock}qEbYAICg(z$7R?U{SQR#GY1q~5Byx(%~^pboZE3^hH zN)r$Mpy!ZyJ^m}>Ol3h|i{eALcKapSU|OapXXZ5U4%Tz#R5q%&H=K<3-txFeUY0eZ zSs54n#Hy=8+f`;2Jlbc`sAem(*+8&yZ!BbCnQAS}?G!Z5P!RHL9Dd!49o|FPml$uI z->p$BQcihonC_&jXRgxZ6o8HjT%QH6;llx)>;BK6#HQLb3;x~X>&vPW<#zmRc1QWX zKdRMD2j=7t=;G=;8cC$u;8VNc)V0X#|H*Ve!*`?J!ag>CKtV?4`YO8n@cTwKEy>~o z{C76?;o6*vvPzPdF}<3V;jbnA^Q+x0l9XKMMJ{j)J8rve22c5Hjm&nfUVDhZJf~(! zQ+N;JO3tzKjXqj|xb=y<4D+Tn-w9MK`ax&EDd5nHIokGK8+8^T8yiZ0KDWNgwT^JH zX1H@s!TBv;KCzj~rG+z*01CuZAUJT^t95}#@4MgkxS{I*_9piR#CqfzCbMT~+_}=m zan_z1x2llt;gn#GPmjh}4mo`i^`40A_J0Lp8Fx3P@ydM+O=05%_qL9&e^Kte7N=hB z5o#~8$Z%1V3};wT^MpZ1C@!nlo0z|vS0;Eht0@i3q7=tl9x_TRG?g%#$wd`Q%)5lS z6s6!7+Huw`{a&4FkPaS=`|uO`O-juGQb@=7No*!U{n0SVqCngbBlh=SHJvK`38U5C zp#A9@JpB0^r;CdZv?q5EuenTBU8ib4ik>I5&rT#>$$LIu>U1iT9-4KYM-M~FyCLd2 zqiJAfSpTOB5E4g?Hn8d|sd z+f%|$TP7xy0O0dI_ei`UoV`qLyzNxT>yg9O*v?nm%l_Gqg94*F6Gp}(9)_j3WG_Bz zX-5*ryOrDIhAK`uE})GH9FP|JU2#-y2x?;lW^mw&K;?!&)n_NJKBMVgbyEk1hFlP9 z%X-JnvSh}rmv)d+qg?j6om9r}(8I4JG=sQA1BOzi^vWCtsuACIU0!{<4}2()lOs|l zHyD^6)-$Qi-9+{|@Z082`Qt>rOsaN0!chv=GTE8smed?RHI3G6`2G4lh_Fqy3v^Gc zd}w}D5pu;4x=h61&zM|lyA}W%S`z%eqxT+h`MS7vaX%TS`(2%5Y`MBcJmpZ_cjVWA zFsPHM^}rH0|FhKT{|YSq-@wuTY>fK<_siXBaoi8ajP9zI+y$^Y5Og(d03;wm0_+r) zUV1+d4+r=PgFq7GG3;r8*abK2{mrIpsIYhnP&HTx!*M1Ix{Kn4?7PHhfvCyLD+Up< zV`{b>cNUu$0Iv5N;A3~S>O`Q?)dNH&y}{I%$U5I2?!W_<%0b&jWj6MsVzA){1|0Ma ze>jmOgNk^tf^E8>%Lj&K`cR|R`3A6Yj{ze6S!rvu4r#MSEVli~*f?Y8ANY3zn@d7q zM1XCc51&OML0g%sO9yx?&2IZYAZ!vEnm_>gu-#%v_dZZTu;u5cEe-Nm>|)~B0LSqj z2#LN>fbku+sK$e-mjT}J1}k3&+*@Rw(P)9rrOTHKCSozi=Nh=cPxbq4SB^RUF*z$K8_^9X)u6u1XmftL< zs_woky2GJW_Zi#ik?OfRQ(x~}j{WBDIAJqiSoaVMH+oj|9< zLoAi!-XfxY*bzR@t5Lpu|26A)2c*XwHWj23`C#GKgfe2Wa z_Uf%V8cPJuZ!#M}LUwe5)L23Hk>8tVZ&YRUNYrUZ&xXR59 zi?H%QSlYEG$?*LM zURQ6r%2@aOA3*xL75o#y^=Fs~JIcAmX!(|?e{RK451Q>cC5;+K5~!zX!vnunYn-JZ_Z-lBv)?%VE_2_>n|}-bQAz_!tYS)#2oZF|KG@+ znD8tfi$l)0z9y1KW_)TNyGRW5vAzfpk z6&M`McgB-kEDXBua|q4NhWvxH`_1TXgIjT==DYc(3Oy}mARSCWort7lPmQ^I7U6up3q3H3srriHBLMb!utsE+gU&f25i}jJtxS%5- zI}F`uqb2W2;W{!za6r|b@mLWP09^|QqOOTdKEW!w0!DrDj^aSJH%UiglSBw(ja^VGRV?qiZ}KlSmbsyjM8I6gggk}T&GI;L=v z)Wh{%Dh^t>UwE7hrS?{qqYx3<*h67oF4!w&6Njn5D)NENkiG)xC>}LFHS`=byoc+PvaSgp87a1;z?=T>ZXy zD$MhVl?kY!MmbHqfIpSEO$Ra|*jam%#NjY7!->}bQ!^M~B?LC}^U$tqJalcAZ_Vw# zj7h>i@fYtx26@{6b?#_9>K1E>TU~Ebd4E!Q5d2AIVkMzSztg6BYrk8w0UPm!DGUW* z;ieA?isdQiCr%u0JGCGCCp?;u)52b)%Gl?4p-CCwi5Xa=6G%tM12V9)_e|fb_?LOd zc^*P%PA9(G!-HM7+CEcxL9y2?u#+u-DkGu#&Dl{3eiaOcfYls;$7XmcZ;zYi&%uhE zp<2oY>8KD9r=94L5IRv4=L>4sxDqU(djCDfOXBIFIMx&MNA<@B=YM}w^=}GT{L+=N z@ob4rx_UD_tUb1%|7)&F^ewr34{?7;V|9kHghDbLyO9~n#)dZBy9|rx zx+3m%B#Dc|&cfL#@mzy`%WVGZ8Z7Kjp~5~4C3fDU(~k-9yRic*O4F4k zCGG@68him9lV@=Me1UITE)YK*f2|1&TnPfQ3jM?K+jY-hoiF^)ake9Ar z<>M#PNN<_=gOaXNGq*{Ty@mZi{!eGZuet; z0$0>mNSoNiJ7MntYzX=NP}(z~KMSzs@FUI5%}9v4+?EW!32{_LNGyoK^~?OO{GZeg zP3m47`z&0{+_-AG*a0ZCLUxZ}qDldSCtsPcIaaM+zDKSHBWMZv7|()X4)kEm(^!)rY)wA3J!KOz==hYk9!O z_uKrv0X#55*%aB-Ya9pOi~2XO7o_gORuNKeMMI_#W`hwWQOBnc9gV+G}X;Txj z;BFV)2lqwY%^D?D$^{OND^$4ZBH1i(7+RBf1$)HY2u(S#tkSP7EP{08lps zYTNo3R@z!x1kkq%tlTQ6DLB+0K_q#|bW+wLzuK+ifBvO9yVb9(ZXa3}nidu;=yx1< zD%#44oy4|ltjdYu!VG~Je#Mk8yLoxrlpXv4vm*lkOzy65omtL=QJ*D8RJVhJyRBP==4PLz#l>=#y4|=_&(r3q<6d+- z`i}+ubdV|j*^1CMLW42>FgcGTvHrP0Vp6K91Zw2c2zq@NSNlya*!AFs**OP$(o3E5 zzo-LI8cII1=%L56E=QX4Ya(;BQ3_Bhfh|zzA=UaP@A;XU7jkZdbWYdq1M1B8!ync3 zUyaD&DTt6cI0tPloH}ey`ooD~ZvW=mB>BR0MM-6psOQKaTDX-ZdC3s%V8b#xSDUYu zp@&(C`p8uH5b|FF{Mk?d5NG17)X(GR(87knwt>3JYaHILp9P*j>(~WWD{GE4Vv%he z0M|XB+DJHpPyzf`Y}uqOiIszyBJxZ-4*FcXQL8Z>C5 zIjS+@KVj@9=?)Nu0Q8)Z91cT#{GXLz1fcp5;1PaKN>7@vVQWgyM=-6Qp>{)tOhwW} z$^fiw#Lg~CA93W^j-P!tIA!C+fs9v$WgA}O(w-_zFai945u0~`U7^>IX-z?KrSHCm zJ&`LNRKk$5Z{M6O(1HIBgbvRL#dQ2p$^ePw8%`MGaeRtp?rrtlu^VI38=97*X07rz znv6FiV5W?y^B&g-ziDO7dn?*M{}`m~?wkjkQ)^Jt#8Q3;hr~3)7Lr5aLsB;&%6a|r{V0lPtx*x|`zN-@S(8U$^`=mDq_h@y?KJnV=JPq|6|su-eKzlE z*4sRC9N2D6r?dd=I=?cEv0MIS=jhOaVi-wl&v;lnKzce3Af?>RXo6nfin zp2iF-d9sUJkt9uPO%G)=!}9HU4W}!i9BF*XgdCHE zx!?D&B|b<6v6;&fUoB=oCs6KmTc`)tgnDH-2JcZ8uhXxK+TNIu(`=;34o}gSpsw!krZ;M5>l!`d>s_t(id?lRk{3FScNCgXPvvM z-D;cG{*lEr*>q{rSj0jUtizU}Y^;Xj;Q0y%@8^=Kyra>;?R7D?8;S(_P2ITxDQ@2C zjur=Wzi5juXxZocy*i#Ee#%%bsW=HocA~fEC_S8Irh@Iy)cM|P9V8Mplk$AHE$QLJjrt=I7w{@h-FX7g=$`=CSNd1u};i91}ZIE0UW z*cdcKWrw67YCo!!tcV2YJ8Yst`Rv(hhB6@7&hV$DXl^UESn&~Rjdl^g1+|bSa=oX$l7DYmR=RFeWX+S4gr>g1q4WTi z-qC#8-SuEa{?jK(;o3+h^J^D)IX=p7yn=*1*|uUMmDj-hF`q!;h5&gT53#o?`Qutl zkTh?B_*-j!nsEXuTY~_F)}WG1nHz^?V`~3I6z5yXa4vj{LQME}uF1|$xx7k|8H?`- zV}-Ei@+*Au%}NjDl?_RKJND@{^$T@uB6~tlWW8@0A=Uq2?B&dk-0!Cy#(i${icL|e zy}oVWsnmMkVIX~D);1B-JY;%nJ`DTYS=Z!dPw66lrm1tJuVBL`?bNCKwI5w%r#6dm zXA_@Q7c8QWUxeMJ90(mS=BW_{^37wgw_A5L+Iv(KaWPh8=sH!Akq76_GMfcc6*&o# zhybqO2)}6jc64u);Qb~B-PvpdX7Y%5~aTLGVRNVF_Jh*BFY(f=SKZbaOt74&(RI>}`f;WAM2_QR2lP`H`5 zMGoRO>K(}qx}`nmR$0ZL=1nQv208PFR;G&9#U){ilfENIa_8WYQ)!F6L746Cg&dIb z-JbC?00vYfW-xlCU){guAWT_WZgz$~|EiW^p7}G>){i9lZ>l<2yLxH7rPnrEEY4Et z?81{9ce#R{=;4Q62GS1)MGbeoHUG$@QhLgA9qv%TEK~nd@L$1<5{YGJ0sbu@H zc(EzGXj)MMaI&Q&IlsA11Yn_9)XQ6Mt*h@18*J^*67mbN`&*9MgRv(&<19XwEXnX{Ae|G_NzZ+sK;SsJ<*hY^P*83cI1I~`*su|w;Qz0UUAIukOor~!L)VXrUj z68IsBdJ)ET0mt%r8ceDEv2=A^GuaiitevHntEeSISB$n3S%-5<*jQzDXhpiNMXFp_ zm-I&W%7bIcXDz$f%coeG!D@zI4LLF=bvOQgt^!?ykCQGu?2rsi$suHc;ua+UGYi-QSLPb7e}C8RL2lUqwWQsC~51IaW7shLa?pa-Olkm)6n4cw^z51Xe?!I?)!%^ zX_!d=QNqo>s`(`U{lw2KY1x{wXPpxrMx?pLHJ*AB0Ge&)_3_U2byR!)dGeMJ>Efr@!}xvnaLHfkwm6 zF_)jUK3kDRZxI>fk**auMu&-ylc~K!F*B}?BOE?+qiHFl$kQ1tGImtNp-0y;SASJ| zHv~G64_X;861dtpCmE*iTE@*BjdzqJ+Q+{v&kTyCy}$G-<{bL( zjTYa8?k45J5#uVToiMmT`GzSx6(#{pv>9RtJijU8Qg+5{<#HTTm20``mT1q#zBtCOo;sBkTOOFFN~Fvm%soSXokij zQwq{{giV}U{QW^3DR2`9gx*XokDM$ReyIiewW;}Sday7nnnTp2BX2SZD#wDJO`J%<{k3cvlJ0FgViKeP7i@f{7V1P=n z8y%{Ak^6o7nP>RoIK7*lrQpMR5qK_rKFJOn=ZrNt($ke+O3Ww2um6KY^Q{Zh_(WTq z&jkkOh?|W!-^f+Fd3^6igJ%S!e%+Yxf5-&YN_~i1^+Xb%n)Q!~i62|;r;D>xUL{Ll zTYAnSmoMQ#4FUUYUXXiNRuwO)5a2KcGkKZB7&-p`Iqe9G!OXK0){uf${V+O>< z)SW5U1MMv%L>zNG&zADZu=fH}FzTJf=Y`d54!TM|2r5j_^Mt5(bar17Mhur+%o(Z{ zTdq5~=X%4XJ@5Fh9M2-{(04Auos%skJpi+o_jQS0#o~9>srI@V@8gDLhyFmon{FwT zkt|qkC;Dt5|J|DPZB=?!y0ZK{H!?qk@}Vt1Hg#%!FL<{2V%I`v(uePA`>rn{9-Ig4 zC1_2a+c7QEl285`k_B$l(bb=qa!yx@6gX!gWXO;ovht@Z`W_=#y?#B%7KDN~Z03tt zsifkVrt0&J9wZRKtL)lTt7_j1H&6|~XY|C{vG+0QUSaM+tfi7p?+D`iD$o5n)3URw z;m>woi}g9o&`zYWq7ww&Vyd6s^mfOZt3iB3LQ*1S9FJGDHfz3(ALO% zjYp^caa?Iq??lEj3Y=;!2G|`DNa*%r(0l~iz~9Xi zDOSDTGbT%^>s*fR**O*muFgJrU!2XNc>k%!O6RSB?jm??%P)?+{u+>3!a-lH zzmr+y>;9VBn+Wqn@&S(MZa3Y~bED0L;ejG)Sq0vwH7YPO4Oi~0=t@7Thz2&?m$crl z4LAdO=)FIa`&jzz=T(vkeXA>{Ygm40H#v7MyXk2j<63>@8-bfQrP^+Wi5cCCCK@^G zb^BxbM9%x$c0H{)fg?+MabEJBJFrm~uvrx%Fp~2PyFJG| zfEtjR%ZL^ab3%8tT0w`#ZM18ve|^YYhuKiF`GA?74%1*~xHUbQhQIWDTdQG}BL9T+Og7&~q~> ziBnRCzSo-a(sXtDLk2xvWXoM}JJlR{-qvN?2FYLep?#rhrEqB2_ z6yaY@r==SumI%fl?budGaDw$fQD**G?T!`#BZ>lbvX*Z=Sy33X0&OYnP|fZGg48)| zYn?(v6#rHb=}eM+&X+48=nB#qyL>+rZ_KI9P^4b6<9vF;pSRo;s2d6m>eZA!N}T0h zoLL9jGO68>bm508nuOV~r=TWw?d=m$vT)7;cI==Ns=sjr%Sj2Pg?Le@Kq>>g=P zRb%^Z)e|R~k2BoJQN2V&SC14W-wL}ea7jySanf$_n7ow>_Pk-LYznIR)k}U}hEG*> zk*B*dL@&{ppq`8;;(KRT#|Vu0hZ`tDG$n?NFs8DMb-mejLI2$>ODnN}G&q#Hk&x(Q zxziTLIsm@DyanbajA6bb5Mlu3tZCucMl|E;n##*pZ5RFOt3D?6c3!7G(Nc`yr#XA% zB!Zfn$w1DT=u1UWSMvPe?oX<{?%zFN%wl0!W%WiVLmwx*}T`Bdqg|J**E z0nP9U)aCRRsj!WH_daE?jC@;E`(5v{cJh}C9SA6>y=0mSQn3-Y|ok8;fU5F=6 ze1bb%HC&Xgcm3>(8SX8qNnkF#rm8I$9bp9fIT(GF=qS}c*$KJ7n6rQJ6KhxVU0^v3 z#P~%-Zi1n}l^`;zRN#m_tIFj0q6S@R5LuW00+~56J5bWz(9^!>+O{3*K$-7r1hn@U`|F7YTlrDwH+bxmwsc{?Y1=`fNq&QoX1?Z{YUr+T zdTZKr_cIZ*zlc|TcROctyKLoG~$ul-1iBE-9%vi zhV^nM6qChZ^H|PT0Npr6a)eAYd3E;}@1plA&SlR=NQf!$kcJMswdwrTm zbxG#%Et;VaK&LURF6Nc~aRrm%&38NxRRhSd zM{J1Z`dE-UN8g@b7U3sdOV7ccA+Wo==Wv{omNSwybFH)8gMz&^+ftNT3VT)#Pv}Ls0Tl$pRzhGw>CpMBRp&wAp z0$1?`O25Lc`j}&|nZ_iRJ9k=2M)Wf2MnXNO!SxOLm1+4_vS8c^ES@8n z%HDLyxsJjvitFCOo>i?y7pthc zGLS>PG9MMRuHxYR2#9-GEr^WX&?>tqykgP5zT)0AyTs|}JbdS}K#u;F(D~kyEwgBF zpy%ZmlE)2?TGg3s<_3{c8RFmMMQDp!=VIh&;8QQu4;rp|+4^_-C<`Q)W_e&9 zq@S#ODi9jgh$zr|MHzYhLO{;yn}wvnk&R}a%9E8Qd!CSdf$;`fIeBo?M!C;Nh=^zatp(SFHMIac-gy?)wnfc@G5RK@tTSUFGb>JB_Wc%<%Crx%r{w^)NG<`#Mt&X9{aLX{gcve#M9Htq}*kad-3{~^0} zVseAuMX;D?_AjUMiQU3~+kC%$P!R;*I%mn3)%PV?o-|pnzo|%9 zemZY$JdDBDA{L~w8TtLJMgT{@iX2KCG)3D7xF+Xbo6ie#Xa+VptjQgj;dJy}EB3*B z$~j*YJ8*|1sp8Xv``wIuY!|Y{w0tyn%Tl_14^P)CiZGh38~HEurrzyM(?ejH{_laQ zabJJEk$D@6-<@Ye5Ix`!q6EEgFV4BU7?izNZTf&h7RDyNz`5uF`yuX^QMvW44k*jh zF?=m?I}Ek!Sb$vAcQ`!Cm-h#eNw8mD42s}aRMP38-eLUhT1p_3Xf56pSiXD6W`U5k zsnwml{tJ;qf(*$GkO(;eqW> zcy*oTyaDbk(1nwj`_uW|BXKhAAxDUX#%b;rii8P=HvQc4f;~&1jc={%LbqnW#|6AN z-GM(&Vl+4>E;q`>yQ76uDn0pbj<(#)1_^tvy*)klp;+Mnw?KuBcvlGPNyIa*fqMp)f-<>wE+`JVd^7)G%qs6g*O(3Q0PD=`lsCssjxpMN0BK7 zt>e=B?l0*%8ke}j?u$=&tjQN%y86n0A*y+x&Xoh(B*Cdf+guy5tL>cMb!m{8EWag< zQGSl!}j6U!w{3 znaw?aQM5MpddN9Nf@I@_BL$Db)hCTv-dlf#{G%>-K#QwkG%xvO+dJ39%Fnb$d*mw#j7X^QYOK3y;k<%T}3 zF<;+o^;==@Deg}DmokHp@t{&-?2Zol(6j(OlDkYDc3bko+f7wkUi!?1p&!4#N<#-H zqF^r{5RfCF*}jOLA|B(l<3O7C5=szN7bwZ;!|0Hf8bZwJa1o(i>X;mK>;f_z?NuW8jlj zGx{ig!E^r9V(sc!>PynyGk|)r9B-<0rcW({LrOp#lSj^ca8S^*I)d{*4J9ttXUrcQ z-Wd_Gc(xY7B`&LSz19Zx=G64@`;V`okwD?cR`&Fk`UUI(d}nLY*h^JFO||E*axdlt z91+LlO3SE*Q=T%(-f;fFW0$S(8pdKJ5$~g3nbAZ;yff!W-yge5us^%QzM29BLD<;8 z-)Gbog@|Ha&SDZnXKPY+1zNPz9(jR@d#hASKIiev_{k5GsR-+c6cyQ{tjU;*6#%et%TLyW>bQ-(ZwA91*28bQXIxV(9|XY%S?LIE8LMS?X=u==0vbkDM+ z6gUB0CG5VEEQimgcm9s9a(-PhqWmunyFB-$|LxdaUOc))av29B7R=+KSk>Hj|0dO^ zr#IdVpSl%?TkRIUmUh~!H!+XI>?hRG!Y1tti*3Q`ok5rJ51*^^Z=znRNIcZl_3T-7 zt=H#Ke9iWDQ@vMue&*|v8O*yP1RN+3i0ALv{%Zm2@5e5nhIK$s>c5`^g1+6UHC;+N zk$1$zY*~GxSo`9ev{wet#gM}|UsyFqH}f6)XhciFALVkw;W<-VQe#tgR<9vHd#zVz zIFgzDN6?;zkDi!!Aw8-;ZXn$G_g&W%mR!G?iyqkFHGoGNV6X)W*rshDAY;@$pmtL> zIOJL=&TRk7yiDD6HrOtbjn|ELepb>uE*bOv^I-Y$;@!X1q8^#B*is7Uu>g6|wutL$ z+xYk)YbfpNVCsYXEm0bubMFS5v_!=8SioHWd@+0JDY)NaniyhmP=gbi-FW1Wd#(JQ zyR*d9sQ+_vHQO@EgscbDt$u`YE#Iu>nmS^J$~lc0Yz~iXznYprk`3D~Rh)EonYVi6 zal~m>b5hN4A(^a9$fI;le#SP#r?ubTahI+j6q{JToJ$dIrRD8Xw!_%)h#?9@2=rx4 z`%N9?S|?v5F-OFVNA-L6V?+oLH$H3<uSE}ALSpcR9;?ODRPRRB&R6P1&7d$RkzxL;49xZgTT|RW5>?O?Q1=F&i@dDy2-+mCA7Gn2* zZ;3K6|MrPeBbUr&Fgatvy@r#gT1o(dsf4ldpCBZz>F*Yp8XpKp{abhMUV<(YA(;OX z;c9x10k@GpOuc%Zw-~DcRVc~xu)XlPS*KP?`=TgS%DhcT(|-H9v&@8#l7Y>9@6{jD zHwwi4g8iBx^=@r*TJI#FfQs`imS-i)v)22?i6kzMxM&4rv@xWMVKk`^mdl6si$CL~ z_okgzXxCvpI<_kp@U^d&=FcQh(uoh8dNZ!P9~6xG5a3WnN$xKr`Sdj_;yH9Q)~(K| zZ=mbBELQ}Nsz3_Sc@<(QZA8rMAy$MbSdHjI#oH#HUEY0;34jJ@ShF3U{yVSgKkJ^4 ze>5!i!|W`CO)k_Y%`9|WsW}dCh^s^1A*d_6NefMmp(Dz(AsQ>(MH(+rLf?d@wQMdC z%Z{7a(4Ay0Eks5elK-zqDrRD%5LUF6;o-SNVr{H+iKcw;!Ez zfas=D57A{D1vl{G=hXEQi4*_sC0Ly0V-V* zl1d9oN(mAIixg>)?i5&bbH{Y=ea`uvbDsO$bMHU*FSqWs=A3KJZ;o$_cf8|$N61N9 zAH8}WSV3y$LwSwL=%zusoL6cgHz;|5&Oz@B3uX{aDxBjwC4OGS?~;$?n%glo(bORW zLRYfVUwWwqSb2E~3DQ_R*zh1*xm|@7>?wA+xOz!Vq&oZ9N{dxQfQmpe^qn2Lp*^wL z&smK+Sb_+*cL|vp7OYlSlV&__lnO}L% zY#1DCS5K`izA)WW)=q1m7N;}V{TjN$@ya;e{&B}3_vUBanz@ZD73RBlhzc9?76l${Eyd)eO7Hxl(viZvL*>g{cETgOEPGD){g6&a>FzWMMe^(TvYsYEA@98u7f~};*@%fxmIK95>s>QNu<@A*RFBI zEGj8_WBwT}Rue-G_n9tX2>R*3Mm5^o{}Zu#A~lSY?qwI_3V zPdn-oVR&;))R*O?{nWrewP=71G8ZsObDEW`o^EVg5_+FaTG!(N>aYe`Q|{tJ%Tu54 zHP8w%Glb}05S%)({!hildgt{wd&kBEM|hg&4f&^R^u_G(jI^TfS*qfzdws+II%{<= z=HhB;DIW3IY`jVK>(|NSz~?pXklvXbIdf8Y)v9%-ZlUzu#C%o;Igr$zS|MU|RL&P| zUK8AtTI+lMVoi-rOg8{@Anxqsxlcgos6UdudvnfUej zPZb*EWcT_UCCdmeTlsi$@SN!^5J=lAZ7H>XrjVhVk|!6#QDwGcQVvI-#=`$$ODE~} zr`o%+Qp-=AXK~7Py6JMU)etF|;{l5h2YAVT0J2U1BF`Jjq!1fMNliyP*$>^m+U}g| zc&2KMxc)yzU*L_f#tQgSQlyzbYio+yZv3=4-(WlaWBRMX%@|XSbMoRkulz zWb}m9A~e^|jQdx4>vdL!?OqK~+<1zEOxXd^{2Y+O8(?}uVZ3=OTm93!xJ;1lgpcEX zJ303FW4hUa3jIDn!u+77Z38Sbroj2579<;9fi(L>?<0NwYBLsuG4P`6Ar)~BGx^;DW`)G;CTEn>?iR}Xo zpz40ZiX8mziB>BL-9`BLddvtfzv-BnisAm30-X_$uiK{0?U)H5ZZ=S6hWgr`XnF17 z4Ijqj`#bi$lC_dRe8Pw~+i#axkNm=UKzjVaXf70)@OsQ%Kubqg02+5HH%;_wqgdLf zU}m=d>dW%zp9SZTw%Ya&7JNmbcdw*=foaIE1&1QT=BIMkrw(+kfJS#*$2(BMiki0~ z39l*g941r=;CcK*WqxOa$aGDEXyEBHK(AB&!Wj8LjN$h}Zxl|#IXrX^0n-iGwC_~l zEIqm0={gm9ib`8B{3}?R6#s_QgNMO+Vq(G-d?bGNRxh88&;a*3!CI|9HY;-mXG~Y} z-$keUTjrr7nh^Cbm_!hAT8wf_WC#O7K4J}HCbwDUsgA{)Uo=|~ofy@T4!$fjfJ=)( zzQf)*#7g^Yu6_)Sa7sO{gqvUN1NO(sxOG`3nzahMZQDLd?ZbWt zOjeS9ojLg%J;AwTj{LM|QfJvvQK)mM^Q7#rp)qn5@i0WDc)B5fcNuGdQH0@=A$di3 zz*1oTq5!=yNlHG`uxv5o=Jy8tH*(wtN4bQA28y?qMytMdi6O5PJlDu zb(n}N?lw>wgQ4ibxq|5SjtPs9WH24+)gGm~eEA;bfy+OeaeE_d#_t|VL*Q;cIn_HVJx7y`gh| zgFwS;w-DVYUr)#2OL=4fU3R-97-7Rk`jVBKUGi`ANdJtDw@#Df!eHcrPF;kE`jRpX zPTx;Kr$myc6?~$RU)=rI&JQc%eTM+_u75Xw6iU!fl4?pv-CI?ew9OuCyc>Wa48s8_ zT3U&HONyVr1Ml6xt$XJAVBoZb+^eKC=x$=>_hsu?XIo$QzT%3zEKYfA>_aRCQQ3iu zkB8;R9Z^D5Vd_hR-fV-d+}=*w0u6qTm0{t?sJK?6KkXP?f)Ihp0q0P)YK1ZT`JXg~ zV5z)$(>T5wBtU=7J;5b#9(b(v4rX1EY4q=ZH)IUKi8~0Nj~YCfz#o!=;Hm&$at@_0 zZA3!d5B-KDKM_0op=vjj%5gky{u379;B~TGCtmF;QupSiTC4H*o2uM?+B%s(=!@r* z&6|Sr0M)d~w0ycPg7=|Z zZydE>*(WL_Z8G=oukf5CgvvW~Ad5VA-;=_*eJF&Jdep{-hjm{e3ephiKJD$Q2{fKR>ft1Q7KV^fbCQb7wYTsKNm3v@#ae?%eTpIQWtB z@5mv39j8ByRYtMtzafgA8nk-~rdP&M1kM?clSF~N zOru+}nK42=bl3@Ex{MbegA&GfYa^f+O}w9=(vvS&`_m|h2ua4#pptmX6a*Or!yGNt zJkoFc)AKjK`vGZFB@);6$i?D8zBx@dRoH-t5FogaHkIEUyQMVw> z9hdE8%}D=-%O3LquW>3s?m3RIW=t>1=yT*AE3riUxqg>HUik#=FXl16F%7c809g$v~g2_G-0nlytGbvF~Ct zqLxAZ-<}=$oW&Q`$Wu}#tFe}`^x{rUFZNNvv-6(8Vd(BEEBsMX*t zrrJ&>khsksZJAq>1iPoN}6TmjZ#0tm6F%Wy#y0P==h z`Rb>!5V-w+i}poS`XBBQdNvP8uhy=1*C6f{`d(Xa@lY?GKS$;Pi7xT%YJYXMLO`t7 zukR7WeYx5Jp25LDcTSdCe7sKgHhcKWi206)L$o+=o<4ncGh$%Yw zJ)(=BLeg!`-EnSul~ddhxVlKv9*}*DZAf3~88P`tcJaObhlE_qhidH9rQ}Q{R8p3? z1VCJcbde*B!ABkExk#59;zEvCLSxAO3lJ64ZLq{BTRLAQ9tMzA&+qo7)%xd!I~E^M|u5jzzN}#b(=q8 zBa^Ze4kq;%Pi3TlY$=SB5X%X2y`rX04*1{{NGEXf4DO?qdvWSBmzg6NP==WoXAhx9 zA&0T^Xm19Xw4!a!klwio?I-+bugbGuj|ZLs^NiMJ&r|5z3^RzQ@_By1lM~>0ArTHB zI(X|MlKw2aKoz^3*KmPKSIiaVLAp}k@xn&}q>56Yca>nydFpQA9ZMioeg`5@q`^bv zEG}(ZVN1+I71&h!g-kzwiw7nr#RfRwcF_K$hgLX5&Gsy6;BN`iJx^2gN|j8XNevZ) z?jfA!ayi4l^mINPW&JSSvKOHo^2FJ=cjW(cOD@HF?98?z%`qq3au|k9QHZU5rvpsa+gCY+rq=j>@kGc^Ww3F}O+mBn-E`gYJWin@ z3zkl`_bnJ=3>fn{xkkdO`nns>5EAod%njsjmeK=fILlq1Kq=4P=d>?kt%%5jQwAcw zVY1HgJ~}ppw@iM-H+1JrQe!=KbmV)-Z{ZKd#B?^JmCZ%!$T6DdJGi0^dh1D;H9m2( zI%fqD2Fu4Q*Gf1ISs`(T8^$T&*U)!dfx!8C7>h&SGO))?#h`Y`0kc(=Y|y|YZ+rYx z80UjwHa0L_1WGIipM(7x{onWOn#Xap%_F>iPQN&euM1=#&{CI%uJMBc6_P0Hhooj* zxOA83<+%_R1YyUm9ZFF62!&edF*3QVqU`mrYEZF?jF>k80d|Ty)_ETwk zJ7OZD7mxJ4NI=X}3Lwt5N}F$cjFtlNUR1N(hF#|9Y!DKW)pLQHg0V(WrS?}#DRVu? zWj4c7O?rHp&F6QW`a9soNXJ=w|B8y|mz%k`1<}(_H$ygds+y7P=0GR;rtp(L1 zMCKm6J?|o#9&O?vs|+H`BoLZOTB12Dr0Ev`bwvzgL9!{%4Fm^nA%rG`q@%z!gJ|G@ z`czW{Ltyt!-thyFCrDhsiU<;{2HF@FLjN`{8xpE~p9LzM07xeQi3^&pvwMMyPyQI2 z@9A|!@|?G;2^<$GO^#t)mcWzOxnc2Mm!6y{PHU+*uoGuY5XoY>!FUdu_;v~L;@Qmcn5j3=)O)T~`W z*C^qmPh4x*Z=79Z~?F6ekz%m6BB?eYPu&8rXLF&LV1Lk()ww+-p zy)Kat?Bt_Z)mm^|K>DN>q<$2(ZpdHYK$Lkwjo2Si9RVM_432TQk*3EkmS|U4swp5R zkLcP^^cuq6m!Zln#-5G;j~G`fRKxIo&iZhjJRdplOnZ~I1>Qk%24D6k9Zd!Y>IReDSWU1SEKbZgrqBU)o#WvXLRoO0ad)DYL~w-2IZZD|dZfh*q!F zou4f~DRp65yD>6yQWF|R@aBQGx;e`S`hd+hUb2Fc=~|0sR(Bh;h|`I`^eFZ!A3Lkb zQOjGYeWk4@l@C(&r?`$0GfpJu}x7Wea>GfDi1!61|%GT3oS?s<^>Pd=Kq`sx*7 z-b=+4r%efN{>I&yUn_j!xHGHUbw9coWzRMY?T<7z0gh3VJjd2lTRd{>w>iq`0mQMh zc7Dyx&3&4+py(BGCscct+=U;FMyuG`=6hZP`&oKtuY+Toe3Mk^V8b?j2?<|Ed52DO zD#0c^R`6Jl4w?b|sqA9Db#jU0V|e_{! zLPP6R`D6Z2yh`wSq@4Q_T&}@a;R?^EM}_aFc8S)=VrTXzruqIa_skm5^4SL^Vrehf zWl_TA>uYOazdWJ|ITP;KS?K(_-5MImWWQ#tv@c5myZmvhwXwI&a70u5Egb=Wrb76& zzIq8G2}W*0qn{fY!a0*%@yB82-H<{+-pMKnr*3MNo%iXUK+Ln|+;su?bWPDD*|L(b z!ywm^arjs@YGLvXk%XRg@K<>bT^MaKH?x`|KtA@6FR|yj&Mix_ke_kg-3o^TMJA!c zCAoX&;eVpeg<=H-1w|>dCG29hv-lmmKexayLyem{A5|wWI4kp1b*&O~)eb>Dg~@DVtc`UG&_xNa@Y0bAZOumxAKRXDs!oI71wUszywW&Unq&e5g`gNM!xiPtUS$ig%sZsK3^R48oYO5N7aQirXA?-{BvWp7vj+;PesG=Hjcs_XWxvqF{ zC|{=8<(1o82bI=EdG>*q&bco!xbKc*l}|e}BpS+MyKc@u1l^K{{ztKB#ZR+nA zWYfcj6OO;p6$+O)uhkxX)Au5^XqpA)QMPT|hw0^61(8gv^b<#Ih3rz5KaT;S32!zt zB|muj^aFmH=k6K}C=Ld~I1&dxHnJ!SySJOQ(;D>y<324UvYcc*IuJ~;u#??v3~TH$ zmc3VzY#-{}F)Ulr6oIn_zMLsQ5R9m;k1xMG zgNv!5DVyuym^yZCqyCy0#jacX*-dBK+q?SEai>wYt3idqceqys5+*n;NqRQ|apcT}D$Q5Y&05O{R8t+UwJan=ZvPm&?_GNw)!M5Wq8}_D;q=T2qlz~UQZuGa zVKl^~q>QrEWWL{AF|_UAbGtN30|%*5IcKG>qJ?_2qs>X60gHd(%d-iA(ClNHMU+@H zQ~^IBS<3>IHG3b(Z-}X=6k+^>mWjzLL-`c)f;%+-zD*$J6Sza7iS1^cgh*UOt6y0eZ<&CufiE6qFZL(u}Q6Gre?~aR>!K9tgTDVGl__Z zbi+_p(ze&1z(MDXb=6-I6n|(Q^`CGg+wZLC1_TAAfu5Ph4radw4ueXOz#ag3O8f)x zl0V9HlSBT^b{fYP0+;5FBJ#KKxEfVmx3sXl&ZiSUZ9q zRRCyfDT2vQ6y$?$BjCs)7FmYe+v58u=mScC99Rl49h4tGPN6phR#%3w!|Z_9A$XBU z0jj_P?)1Je=_d(lV-Z7e=u$C8R@Q(~kRU&I@nTGL1F$mU^}M#!ELv$xkG=M0Vj>jQ zr&!fSr?4t^?7Mez@QcD$9-U`mVk)nOTd5Gq zy|RBl@T)BdLYYmz-HE1d~6PQ|W^9f$t6 z1VTc>+|Y&oJnAZl6sjO!W$)~q0mP<+ms;D|*(s5T zBhShiZ#)u_SbWQ=s}&(aMn<;y>r0L!wL|Os2Eg4G0LD&-)=zU;9VDFHVB9yo)NSkJ z#Gs_4^zU~-Bmc2v3%q~2Wwy0&kz8OLGqTE}niUq*Ew8AU0YtYhXhkwZg~>zp6RCCj z)N7!f^#F3aY)}U5@@r-9IC6oxJa~C?DOs_m|0IO1uD8yw^vxUqRWXRfkUM>L&ax#6#4ebDPmqt+~NAYwEpr!62ajk(4a%19t)H=)!O34LTRf4 zlTgg`^mmNs?3buqmI^`3P4rGXpgULQ$mDRl$?24rr)L)Ibc37X$RbRU#Rknw2ZT>U zY&45Uo-knd#H1A`7`mA8eWIVc(KFfr_tEKA>tOrKe|L3r)1>~sbu9FLX}G+ohiJO~ zGeo5IE2IhB_8;~pUQu^Ju9XEY*XtDvCgjLch)w*?;2VTwVjZx^Emr`7)jlU79`Neb zt77`~XbS7o(g(N^Nv2?}3-u^>=KKHLGkJ6DZhXBI?@oVgv)Bzuww^RnYpU`0Niap2TX*NqEFz^@&o^u86oz{hX^~ z;3`xy_9K&dprGBkxc|Z3^J8+dao}pBc-;9kH}CJQN#4 z>DL9e6Su$j_xA&I(zICxfP%&-n}8)KVna24)#y7Z8QIRAJ@pI6uPB=!nl<9Vc?WXglh&QOqEJm^yp)`r97{7dH@EOO443bqH{S7udJ}6_bf&!}J|^z! zq=N$FswyUgXG`1EKI_B@*o<0s7dGy6mz@3eeF{~+wRU`TP;5EK;|^^dUQuI6G*ruM zpTK?yXFz(KdyBt+AKfWHR;Cc`D+??`QQ=fTNmPO3at;sgJv6o>A|fM=)VVEsKfa|0 zx7v^>_z9HM)JnCrwd+ohqebx9m)dDnh7w75ujrIpN{M^z^Kobu%NSpDAX@=uL>6oS zsyXT$A2kXld4s6nL=xX|@&PpB`E2ta%3t8nzora@0(fC{7Ab&TXWJ-R;sx{Q?R2g% zdbzN>9IfwoL+jI7n+j(82B&XefkOM+xn3+6BZrZOP2FD~CI00pI0+Ba|0&*fO@xQZ zC6FJuYeMsXkW+o4xI%hI68mh+FBq)LK*RFGD2$!2(Lchtt&L-4`#&;5Hs$uDTiT1c zA4yXeF`8}hf}x-Fs}+r7A|h@>LKuCT3UqKe4*V}tb8fx3B`P{Uc5+;rOOQ+O`P5l= z&9Ba%IzfXp6}HJnc(JWQ7#J8R;GXlVj!#ZL-CP+NlDU8X%LMs(&sk^0`++WMdq zk~VxuKEAi9iixj|d^N!*=dQ_8%Za{W`t$mrAlPuyK_+7Vt;D>y1(xZJwvWKIZv+wb z8D|}x1X#_QbfAJP9Zq|9+QRP~Nf3-8QW&SZd&b#0I|~4qU>vIp&3OW9YU(NIfC)e% zWa79oJb3o(S#DBJ9aDC#;wI?VM0G;HMT$%Y#lP{pbgadXRaA5sWaRxPW@j0hs_#Pa zvOsh;5Ks;Z==AUbgCYn#q6EGO(ds2t9 z2xO6AeW1&;2cugTU@Hz$Rfa4M(9=tRi+0I#Rnc2WUc4%x(D(XBI2~?WC&<5VHBAlQ zH|QI16cTrL;Rw-YL{ENGnB zS4%v@F7);Q(nWKwYUnDo7*wH)=RTL{l6z40J_|(e?G*tJi-^vJmT<~<;%@9j_$!v(N^THs`2g1?DlM-#eL z3qWo#R6j&J!3Ayt*{n78$tRLAi{3$Txsc@@}+(n^!=n)r(>t4An3VXqp zHx#v&jX{{$dyuPs&jmj- zz?#u++mA#MBYgh!Ch0j&KDyjvMq!trOeDk}lN|NeSKHONDbb)y%Z5N&KE{2)t^XzQ5N@ zqyc+m%mN1RzAK2lY@6lbE`}$&0SYbej@>zxp{l1iXg=-QV5#|`ve8- zhcsBlLAkVzNq01=aHObQ?9YCAN0iksWHJ`p}~kJR>QJ&Mit{pM_BR3o1Vjr}q(Pc8v^-Z$L`kP4)Qv?b z(N0fK{}a|W2OhPHX0A?ogjR_qGEyEnHSAYlH6*am#r+l13z-;NG#Zn}6}k?J&fyfW zUjzda|8916_VmJn+0k?huUz>{Imm>X+hF6K@`wWLNnVz*s6+jcu-My<4%qCP`p3r~ zo6Ssh-6B2Y)vorg$|c+aAA|_fLN|O`Q3DW0iHds#a6ZpFK^x9eHxzrv9Z|r~5iRUg zn7Q*m=PDgzL>kXp7Ryg2H!HUd^t;lPxgnjIB$FkK+!bTL-L`AV7ub5UD3cfGn8=Dt z6b71M&-4#w+G$I=HT3he7K(prYKqQ)`gkkP=b^H4ghIr4Z5w1JF;j=xuW^r(#Jt|M zh2Z{jXf-akpOGN}>6`ag_|2R4-)`em3WYhZ46}SHGkyuD(f>tC;!*Z?2I{2@D;kws z$o>6X(9Cj??Ahp7m%JYGFXN<&D};(7d~!|@;ipzsIHztN9v#hyk?l|S^04>*1eyA5 zX!t0z4=%%|AOnKFr$B~YKTfk~nBn_r1>X2zwBP6Q*nCYgzI&dtA))7UH_kCZ1G8!K z;u>NF`wK^QYgbiV9Om7zJMn$$ChUBpalks`vp_PU6=42Lji>9XxOZi-;rd4k5M}A$ zE}sD?&`{v;)y%_4$K6=XlU?q5la*apOA>u;EqL)3$b}WdkiY~-?asSd3_9UoiEwZU zuH3mPft^M1%@?CAL_wTQdJAHnBB=_}ezIn+p#j`ucP zFqf#Pj`_5azqloAqXB;(mc#f52yT{(;(Rpi{2)Mtvsc22cb)N>Z!iQlO&$VDqCDDK zwoA|qRExGO93KKyBix`;=dC7$Q|05a0{_J!|hul-QEc=6&C z=;ZsaaO2%E5KVDepKf+M+_fw3C^CXZMMj&wxaeRyY`%rX8d{z$4!qTI=)$!^1t73Q z+p#bEk?ExXoA;5iOAZPXmu%iomL!1|=Monek7H>u$nGX1RQH1=c%qBw@WU1u@dNgn zfSk$=E=Smb47V<}a^31c0s63-1@laX&N}n&`qh5-=OvHa{s}SA^};h`-FFbEHgJ=01h;E(KSUl0RjTA2p%7$ac?}8C0bMk;%oJ& zwx0QRNNZvjZt;hf5?HF{L38_+mw-Hleu zQ6CiT3j)QHX^SfNEz8D}Y;HW9n(zf%uL8klzw=8uV$=1xNMM3&BJ#bY=fOi)&y)*4 zPv^3%)VWd`D@K*rj2$;kL5{*mQs2#=O=G=+wN^G1qMXwz|x1%C(ovy8&(hm zU!I$W{aFTA1|?~x^Th|ek@}PtL~Y~cQNT$LZ3~LwZds-kFlfYI94MfRqnPA%@%*S_ z7a^}DB+eiHN$u=JTuZWewcSD&L-|OFl{WmQtNtxe*qVs$3_Bos8FbWX;**n;mqozr z^ocPiC#N)I3&A}-s!L+WgHVnATw7n~X}f@l3svkV)4Mzc`3l8$GC*S?aftzQOolH> zC4z0+S(D<>Mdf9}$tDE|$3eB>MY|g*ZQg+?OTX>Ou@~&oRSFv?dvnRnxn*VidBQE? z-LMOGQu`dzaK!5vy1+iC3Tiz(y!9sEb{#9e@d5aUj6InTm3(SH25?Z0Yz^w@JH{E4 zdvV|acE`UWKMshL!rABWQlt?9s0!N&Y@}uaO)#bQc)^&(?QI~fylM+E2z>LVnVb0l z$niYR=;d@~Ao4P7l^<^;pETv(p&D;1N}}GWo1KSEBg@-MR>W=waxLZMa!6C0naZ>B z>t34AnP>qJ9FFl7C4PNE%AJ(&2rfM4U1@Lb2PO36oGbF>uQO07>wZ_MhS|5It7cktdPSj#@o9?@*tcBmIV|Ll-lmU zS_*7|)dmHvDVPV*e)k@I1d3otJy82PFp!r@)b()|BLhRjJml6oO$iAJh7L|n{O+6c zPll^J`nii>81o7&)uKW4sS>ww0knnlYo%)+_4HLkl(se^nF?Vi=YfM}cGHsuVd2cg z>`z#qm)AVQIabrE8aD!jgKWs8nO&kgar0vbwQg^1*GG8UQ0`7w((DzaLPgD=y3Y2`Tq8Aws<35SL^0p z0z!!}<2A6uGTa&Rt$t=fVgCr<<*?;@q z)6=s!ZZDn`QjwX-$=fiv+pOw-@P+;x#TV(fy#!x?A2mFS??4)6a_Yb`=RZ0 z7w#N_X(Pcx5*lf#z$=&O_bzzaAcH%P-6!+SbI)=G>^cmGJf3AbtWV39KgRg6a~zAJ zIS57NJK-tsO{yII4Q81^{Jt+&ZF^K5qvtM5sFSo?=l9;pyz?%0Ryaf%;~i~Z_Bve> zNWMNme$K3!MFN5EpDQYknO^zVa;&)m15g=_ZVEh!;b||3bf9-o+kUg3xK%U;BR13~Ja-5a(jU7D09*d?thg_Q^9GIp;8O@&8;>0<8K^kTNqWB`H{2TPwHPLG*PJTk?~4;FaUj9wid}2I284p!AK9I|jqQT}4_9gGh+~x5Gk#s+@ra z+}d~^+W^JiU~m(v#<(XZkg47muX%)v6z^Z4A1g5Bph}SzjHJE)T+#IH(YS1&ku4x! z1mANydl#2XNdL6iQZ~4pfP}*$EF9;3h(RxV2d{hC6o1Hpn5u^X z`t5koh-!PLea)4l3kFQ&syG{sSb*EbH9{tSa>$Ep&^DeQ_4QRlkr;UkrlyNQU4Rt~ zYmmcaK$C^u2w%0zH#Fw^TjziLqtVIo0k9Mn++gUH?oz2_2GLsp-hbPulr=o zr&3w0e7X7E_P>z;@b*{0Cf4! zZz)0=D@UZ1*cHC~YNggA3Za-F=A`B`A#D8oYMhOt7v7wZ-_U#h5nc@C(VE`gw{3uT ziirIc!T0y@PW*5#)zrJP|3I)FXXt0Xz1s177gZPvLCf6yhrf@z?q9GASiRajtAG3Fj1cHBZ&iU%HQBN&=8g?qLVayEQj{2?st(2ZcG5)={TmW;GJ?x4p?<9a9J_DuQDNg{{EsK&Z=$uxDsI|1eg9D(A_W|BW2T-LG5C%hnx#D;P zDq>}3mgy@{2;ITl0Q$q$(4$A^U&Aw~EZj#w7vu!f;w&e=U8 zB_(Av2CFo^OC&G6$wCqvC5rdg&A)lPcS{b!sx0J~{0*&#YqVqVkkY{stw~IeJRY5A zJE5ezcpXck@2}VMW*P946sCo}56gxx;^Rkkaf=pdBOu~i0Qlu(WWr`wi(N%=R=d+L z5TXok{dJNG=w5U$w|A%+Ww1|qkrN)%CsPY5?(z-m>zuyTUl*=ML4CGak z=bVZjg0HC{JyNJY8Ii!C0EOl>+anQb7$Ix#5jk+8o;^|D z4%g+oAl?AP@>3d?F}+9SJJ1<5`0?Whq|r>8qaE-jIQrrs$ab;A!M4>+ul?d-P`FHM zk5z=<_ZMX#`Z>}i{NGUrn$95gb+o%dWk3Ufy~%q^*uDk;dZWS=VFRImSZE1s#{_mN z9{}c-y)95EO>DgDDBo)dYe>da0!WS~TQPK929HXB)PaLsu!t+aFJyQvq?D8jNJHbx zZ7$skdbIf7G?6p@&Y+%by&(2B%Wg`t*ItuzWz;m9S znPF&VJOB9(N^uyqzRg&iS*U7A|z$)=On!eDQoK1Uq3BMdse@pGb1i^Kq5 z%Ik@9bB767563o~$is7yL1Cee>mV^Pg@vyAQI%!kf^ntkAsliZa{EHgWytryo*b|+C>E2`@I!nV$WGXb>I{|7lgP-U)QFq$B zFE_4z2Vjt7D95FMy0A_O#$;a@-GIcK2BT|%IyySjLR=mVvTP{Sce(2*lyf7>z7EY{ z!VbF{P%F$pe-$R?xCW1?^fZpakTmjTFot$xn5U^Da*#d@5CIvo4PZFXz0rnF&&&V} z!kR&11yfeyn$$ipzqY=!lR4&na9@k6{PbGw3*2>~^NQTUcGFu5!g2T)FJ?k>J|Z+U zlydG)gf_{eTn$Mu9BZySxq;%2*BLxIgF;b0gtTn|-nHlds4YPTNk+q8U2FzBM%oZAs`b;+6MA?jm?G870^_vL41%0&(tJ5e;udn;d^!dO zKSClRqp6i1C<`^UXXRoz?-vvl@XUsOH7sX~MfaxjL!PDutZdJjRdEOy!Bn?yeqUPC zsD&O+CxU}<;EzD#Sx!#w1Avr2Vg6;B-&OQFQ{?c|T_J!8(sIh+lU?a;2B*6jb5SXR z^uDla)?VUS3s3!q*CP7PG9-U7&1At>1vB8{s5z7L#iv)F4g^}$oXaOD&F~g I`||GPU| B(Is your text usually smaller than 500 characters?) - A -->|CPU| C(Is a 0.4% accuracy loss acceptable?) + A(What is your hardware?) -->|GPU| B(Is your text usually smaller
than 500 characters?) + A -->|CPU| C(Is a 0.4% accuracy loss
acceptable?) B -->|yes| D[onnx-O4] B -->|no| F[float16] C -->|yes| G[openvino-qint8] diff --git a/docs/sentence_transformer/usage/usage.rst b/docs/sentence_transformer/usage/usage.rst index 28349e1e6..384ec52b0 100644 --- a/docs/sentence_transformer/usage/usage.rst +++ b/docs/sentence_transformer/usage/usage.rst @@ -56,6 +56,6 @@ Once you have `installed <../../installation.html>`_ Sentence Transformers, you ../../../examples/sentence_transformer/applications/parallel-sentence-mining/README ../../../examples/sentence_transformer/applications/image-search/README ../../../examples/sentence_transformer/applications/embedding-quantization/README - efficiency custom_models + efficiency diff --git a/sentence_transformers/backend.py b/sentence_transformers/backend.py index 9c60e4613..1825e167f 100644 --- a/sentence_transformers/backend.py +++ b/sentence_transformers/backend.py @@ -13,7 +13,7 @@ logger = logging.getLogger(__name__) if TYPE_CHECKING: - from sentence_transformers.SentenceTransformer import SentenceTransformer + from sentence_transformers import CrossEncoder, SentenceTransformer try: from optimum.intel import OVQuantizationConfig @@ -26,7 +26,7 @@ def export_optimized_onnx_model( - model: SentenceTransformer, + model: SentenceTransformer | CrossEncoder, optimization_config: OptimizationConfig | Literal["O1", "O2", "O3", "O4"], model_name_or_path: str, push_to_hub: bool = False, @@ -34,7 +34,7 @@ def export_optimized_onnx_model( file_suffix: str | None = None, ) -> None: """ - Export an optimized ONNX model from a SentenceTransformer model. + Export an optimized ONNX model from a SentenceTransformer or CrossEncoder model. The O1-O4 optimization levels are defined by Optimum and are documented here: https://huggingface.co/docs/optimum/main/en/onnxruntime/usage_guides/optimization @@ -46,10 +46,14 @@ def export_optimized_onnx_model( - O3: same as O2 with GELU approximation. - O4: same as O3 with mixed precision (fp16, GPU-only) - See https://sbert.net/docs/sentence_transformer/usage/efficiency.html for more information & benchmarks. + See the following pages for more information & benchmarks: + + - `Sentence Transformer > Usage > Speeding up Inference `_ + - `Cross Encoder > Usage > Speeding up Inference `_ Args: - model (SentenceTransformer): The SentenceTransformer model to be optimized. Must be loaded with `backend="onnx"`. + model (SentenceTransformer | CrossEncoder): The SentenceTransformer or CrossEncoder model to be optimized. + Must be loaded with `backend="onnx"`. optimization_config (OptimizationConfig | Literal["O1", "O2", "O3", "O4"]): The optimization configuration or level. model_name_or_path (str): The path or Hugging Face Hub repository name where the optimized model will be saved. push_to_hub (bool, optional): Whether to push the optimized model to the Hugging Face Hub. Defaults to False. @@ -58,17 +62,17 @@ def export_optimized_onnx_model( Raises: ImportError: If the required packages `optimum` and `onnxruntime` are not installed. - ValueError: If the provided model is not a valid SentenceTransformer model loaded with `backend="onnx"`. + ValueError: If the provided model is not a valid SentenceTransformer or CrossEncoder model loaded with `backend="onnx"`. ValueError: If the provided optimization_config is not valid. Returns: None """ - from sentence_transformers import SentenceTransformer + from sentence_transformers import CrossEncoder, SentenceTransformer from sentence_transformers.models.Transformer import Transformer try: - from optimum.onnxruntime import ORTModelForFeatureExtraction, ORTOptimizer + from optimum.onnxruntime import ORTModelForFeatureExtraction, ORTModelForSequenceClassification, ORTOptimizer from optimum.onnxruntime.configuration import AutoOptimizationConfig except ImportError: raise ImportError( @@ -77,17 +81,22 @@ def export_optimized_onnx_model( "or `pip install optimum[onnxruntime-gpu]`" ) - if ( - not isinstance(model, SentenceTransformer) - or not len(model) - or not isinstance(model[0], Transformer) - or not isinstance(model[0].auto_model, ORTModelForFeatureExtraction) - ): + viable_st_model = ( + isinstance(model, SentenceTransformer) + and len(model) + and isinstance(model[0], Transformer) + and isinstance(model[0].auto_model, ORTModelForFeatureExtraction) + ) + viable_ce_model = isinstance(model, CrossEncoder) and isinstance(model.model, ORTModelForSequenceClassification) + if not (viable_st_model or viable_ce_model): raise ValueError( - 'The model must be a Transformer-based SentenceTransformer model loaded with `backend="onnx"`.' + 'The model must be a Transformer-based SentenceTransformer or CrossEncoder model loaded with `backend="onnx"`.' ) - ort_model: ORTModelForFeatureExtraction = model[0].auto_model + if viable_st_model: + ort_model: ORTModelForFeatureExtraction = model[0].auto_model + else: + ort_model: ORTModelForSequenceClassification = model.model optimizer = ORTOptimizer.from_pretrained(ort_model) if isinstance(optimization_config, str): @@ -111,11 +120,12 @@ def export_optimized_onnx_model( create_pr=create_pr, file_suffix=file_suffix, backend="onnx", + model=model, ) def export_dynamic_quantized_onnx_model( - model: SentenceTransformer, + model: SentenceTransformer | CrossEncoder, quantization_config: QuantizationConfig | Literal["arm64", "avx2", "avx512", "avx512_vnni"], model_name_or_path: str, push_to_hub: bool = False, @@ -123,16 +133,20 @@ def export_dynamic_quantized_onnx_model( file_suffix: str | None = None, ) -> None: """ - Export a quantized ONNX model from a SentenceTransformer model. + Export a quantized ONNX model from a SentenceTransformer or CrossEncoder model. This function applies dynamic quantization, i.e. without a calibration dataset. Each of the default quantization configurations quantize the model to int8, allowing for faster inference on CPUs, but are likely slower on GPUs. - See https://sbert.net/docs/sentence_transformer/usage/efficiency.html for more information & benchmarks. + See the following pages for more information & benchmarks: + + - `Sentence Transformer > Usage > Speeding up Inference `_ + - `Cross Encoder > Usage > Speeding up Inference `_ Args: - model (SentenceTransformer): The SentenceTransformer model to be quantized. Must be loaded with `backend="onnx"`. + model (SentenceTransformer | CrossEncoder): The SentenceTransformer or CrossEncoder model to be quantized. + Must be loaded with `backend="onnx"`. quantization_config (QuantizationConfig): The quantization configuration. model_name_or_path (str): The path or Hugging Face Hub repository name where the quantized model will be saved. push_to_hub (bool, optional): Whether to push the quantized model to the Hugging Face Hub. Defaults to False. @@ -141,17 +155,17 @@ def export_dynamic_quantized_onnx_model( Raises: ImportError: If the required packages `optimum` and `onnxruntime` are not installed. - ValueError: If the provided model is not a valid SentenceTransformer model loaded with `backend="onnx"`. + ValueError: If the provided model is not a valid SentenceTransformer or CrossEncoder model loaded with `backend="onnx"`. ValueError: If the provided quantization_config is not valid. Returns: None """ - from sentence_transformers import SentenceTransformer + from sentence_transformers import CrossEncoder, SentenceTransformer from sentence_transformers.models.Transformer import Transformer try: - from optimum.onnxruntime import ORTModelForFeatureExtraction, ORTQuantizer + from optimum.onnxruntime import ORTModelForFeatureExtraction, ORTModelForSequenceClassification, ORTQuantizer from optimum.onnxruntime.configuration import AutoQuantizationConfig except ImportError: raise ImportError( @@ -160,17 +174,22 @@ def export_dynamic_quantized_onnx_model( "or `pip install optimum[onnxruntime-gpu]`" ) - if ( - not isinstance(model, SentenceTransformer) - or not len(model) - or not isinstance(model[0], Transformer) - or not isinstance(model[0].auto_model, ORTModelForFeatureExtraction) - ): + viable_st_model = ( + isinstance(model, SentenceTransformer) + and len(model) + and isinstance(model[0], Transformer) + and isinstance(model[0].auto_model, ORTModelForFeatureExtraction) + ) + viable_ce_model = isinstance(model, CrossEncoder) and isinstance(model.model, ORTModelForSequenceClassification) + if not (viable_st_model or viable_ce_model): raise ValueError( - 'The model must be a Transformer-based SentenceTransformer model loaded with `backend="onnx"`.' + 'The model must be a Transformer-based SentenceTransformer or CrossEncoder model loaded with `backend="onnx"`.' ) - ort_model: ORTModelForFeatureExtraction = model[0].auto_model + if viable_st_model: + ort_model: ORTModelForFeatureExtraction = model[0].auto_model + else: + ort_model: ORTModelForSequenceClassification = model.model quantizer = ORTQuantizer.from_pretrained(ort_model) if isinstance(quantization_config, str): @@ -195,11 +214,12 @@ def export_dynamic_quantized_onnx_model( create_pr=create_pr, file_suffix=file_suffix, backend="onnx", + model=model, ) def export_static_quantized_openvino_model( - model: SentenceTransformer, + model: SentenceTransformer | CrossEncoder, quantization_config: OVQuantizationConfig | dict | None, model_name_or_path: str, dataset_name: str | None = None, @@ -211,16 +231,20 @@ def export_static_quantized_openvino_model( file_suffix: str = "qint8_quantized", ) -> None: """ - Export a quantized OpenVINO model from a SentenceTransformer model. + Export a quantized OpenVINO model from a SentenceTransformer or CrossEncoder model. This function applies Post-Training Static Quantization (PTQ) using a calibration dataset, which calibrates quantization constants without requiring model retraining. Each default quantization configuration converts the model to int8 precision, enabling faster inference while maintaining accuracy. - See https://sbert.net/docs/sentence_transformer/usage/efficiency.html for more information & benchmarks. + See the following pages for more information & benchmarks: + + - `Sentence Transformer > Usage > Speeding up Inference `_ + - `Cross Encoder > Usage > Speeding up Inference `_ Args: - model (SentenceTransformer): The SentenceTransformer model to be quantized. Must be loaded with `backend="openvino"`. + model (SentenceTransformer | CrossEncoder): The SentenceTransformer or CrossEncoder model to be quantized. + Must be loaded with `backend="openvino"`. quantization_config (OVQuantizationConfig | dict | None): The quantization configuration. If None, default values are used. model_name_or_path (str): The path or Hugging Face Hub repository name where the quantized model will be saved. dataset_name(str, optional): The name of the dataset to load for calibration. @@ -234,17 +258,23 @@ def export_static_quantized_openvino_model( Raises: ImportError: If the required packages `optimum` and `openvino` are not installed. - ValueError: If the provided model is not a valid SentenceTransformer model loaded with `backend="openvino"`. + ValueError: If the provided model is not a valid SentenceTransformer or CrossEncoder model loaded with `backend="openvino"`. ValueError: If the provided quantization_config is not valid. Returns: None """ - from sentence_transformers import SentenceTransformer + from sentence_transformers import CrossEncoder, SentenceTransformer from sentence_transformers.models.Transformer import Transformer try: - from optimum.intel import OVConfig, OVModelForFeatureExtraction, OVQuantizationConfig, OVQuantizer + from optimum.intel import ( + OVConfig, + OVModelForFeatureExtraction, + OVModelForSequenceClassification, + OVQuantizationConfig, + OVQuantizer, + ) except ImportError: raise ImportError( "Please install datasets, optimum-intel and openvino to use this function. " @@ -255,20 +285,25 @@ def export_static_quantized_openvino_model( "Please install datasets to use this function. You can install it with pip: `pip install datasets`" ) - if ( - not isinstance(model, SentenceTransformer) - or not len(model) - or not isinstance(model[0], Transformer) - or not isinstance(model[0].auto_model, OVModelForFeatureExtraction) - ): + viable_st_model = ( + isinstance(model, SentenceTransformer) + and len(model) + and isinstance(model[0], Transformer) + and isinstance(model[0].auto_model, OVModelForFeatureExtraction) + ) + viable_ce_model = isinstance(model, CrossEncoder) and isinstance(model.model, OVModelForSequenceClassification) + if not (viable_st_model or viable_ce_model): raise ValueError( - 'The model must be a Transformer-based SentenceTransformer model loaded with `backend="openvino"`.' + 'The model must be a Transformer-based SentenceTransformer or CrossEncoder model loaded with `backend="openvino"`.' ) if quantization_config is None: quantization_config = OVQuantizationConfig() - ov_model: OVModelForFeatureExtraction = model[0].auto_model + if viable_st_model: + ov_model: OVModelForFeatureExtraction = model[0].auto_model + else: + ov_model: OVModelForSequenceClassification = model.model ov_config = OVConfig(quantization_config=quantization_config) quantizer = OVQuantizer.from_pretrained(ov_model) @@ -306,6 +341,7 @@ def preprocess_function(examples): create_pr=create_pr, file_suffix=file_suffix, backend="openvino", + model=model, ) @@ -318,7 +354,10 @@ def save_or_push_to_hub_model( create_pr: bool = False, file_suffix: str | None = None, backend: str = "onnx", + model: SentenceTransformer | CrossEncoder | None = None, ): + from sentence_transformers import CrossEncoder, SentenceTransformer + if backend == "onnx": file_name = f"model_{file_suffix}.onnx" elif backend == "openvino": @@ -349,7 +388,8 @@ def save_or_push_to_hub_model( commit_description = "" if create_pr: opt_config_string = repr(config).replace("(", "(\n\t").replace(", ", ",\n\t").replace(")", "\n)") - commit_description = f"""\ + if model is None or isinstance(model, SentenceTransformer): + commit_description = f"""\ Hello! *This pull request has been automatically generated from the [`{export_function_name}`](https://sbert.net/docs/package_reference/util.html#sentence_transformers.backend.{export_function_name}) function from the Sentence Transformers library.* @@ -380,8 +420,47 @@ def save_or_push_to_hub_model( similarities = model.similarity(embeddings, embeddings) print(similarities) ``` +""" + elif isinstance(model, CrossEncoder): + commit_description = f"""\ +Hello! + +*This pull request has been automatically generated from the [`{export_function_name}`](https://sbert.net/docs/package_reference/util.html#sentence_transformers.backend.{export_function_name}) function from the Sentence Transformers library.* + +## Config +```python +{opt_config_string} +``` + +## Tip: +Consider testing this pull request before merging by loading the model from this PR with the `revision` argument: +```python +from sentence_transformers import CrossEncoder + +# TODO: Fill in the PR number +pr_number = 2 +model = CrossEncoder( + "{model_name_or_path}", + revision=f"refs/pr/{{pr_number}}", + backend="{backend}", + model_kwargs={{"file_name": "{file_name}"}}, +) + +# Verify that everything works as expected +query = "Which planet is known as the Red Planet?" +passages = [ + "Venus is often called Earth's twin because of its similar size and proximity.", + "Mars, known for its reddish appearance, is often referred to as the Red Planet.", + "Jupiter, the largest planet in our solar system, has a prominent red spot.", + "Saturn, famous for its rings, is sometimes mistaken for the Red Planet." +] + +scores = model.predict([(query, passage) for passage in passages]) +print(scores) +``` """ + breakpoint() huggingface_hub.upload_folder( folder_path=save_dir, path_in_repo=backend, diff --git a/sentence_transformers/cross_encoder/CrossEncoder.py b/sentence_transformers/cross_encoder/CrossEncoder.py index 1101e082e..05fdf5872 100644 --- a/sentence_transformers/cross_encoder/CrossEncoder.py +++ b/sentence_transformers/cross_encoder/CrossEncoder.py @@ -1,11 +1,15 @@ from __future__ import annotations +import json import logging import os import tempfile import traceback -from typing import Callable, Literal, overload +from fnmatch import fnmatch +from pathlib import Path +from typing import Any, Callable, Literal, overload +import huggingface_hub import numpy as np import torch from huggingface_hub import HfApi @@ -34,6 +38,14 @@ logger = logging.getLogger(__name__) +def _save_pretrained_wrapper(_save_pretrained_fn: Callable, subfolder: str) -> Callable[..., None]: + def wrapper(save_directory: str | Path, **kwargs) -> None: + os.makedirs(Path(save_directory) / subfolder, exist_ok=True) + return _save_pretrained_fn(Path(save_directory) / subfolder, **kwargs) + + return wrapper + + class CrossEncoder(nn.Module, PushToHubMixin, FitMixin): """ A CrossEncoder takes exactly two sentences / texts as input and either predicts @@ -99,6 +111,9 @@ class CrossEncoder(nn.Module, PushToHubMixin, FitMixin): model_card_data (:class:`~sentence_transformers.model_card.SentenceTransformerModelCardData`, optional): A model card data object that contains information about the model. This is used to generate a model card when saving the model. If not set, a default model card data object is created. + backend (str): The backend to use for inference. Can be one of "torch" (default), "onnx", or "openvino". + See https://sbert.net/docs/cross_encoder/usage/efficiency.html for benchmarking information + on the different backends. """ @cross_encoder_init_args_decorator @@ -118,6 +133,7 @@ def __init__( tokenizer_kwargs: dict = None, config_kwargs: dict = None, model_card_data: CrossEncoderModelCardData | None = None, + backend: Literal["torch", "onnx", "openvino"] = "torch", ) -> None: super().__init__() if tokenizer_kwargs is None: @@ -129,6 +145,7 @@ def __init__( self.model_card_data = model_card_data or CrossEncoderModelCardData() self.trust_remote_code = trust_remote_code self._model_card_text = None + self.backend = backend config: PretrainedConfig = AutoConfig.from_pretrained( model_name_or_path, @@ -157,9 +174,10 @@ def __init__( if num_labels is not None: config.num_labels = num_labels - self.model: PreTrainedModel = AutoModelForSequenceClassification.from_pretrained( + self._load_model( model_name_or_path, config=config, + backend=backend, cache_dir=cache_folder, trust_remote_code=trust_remote_code, revision=revision, @@ -167,9 +185,9 @@ def __init__( token=token, **model_kwargs, ) + if "model_max_length" not in tokenizer_kwargs and max_length is not None: tokenizer_kwargs["model_max_length"] = max_length - self.tokenizer = AutoTokenizer.from_pretrained( model_name_or_path, cache_dir=cache_folder, @@ -209,6 +227,229 @@ def __init__( self.model_card_data.register_model(self) self.model_card_data.set_base_model(model_name_or_path, revision=revision) + def _load_model( + self, + model_name_or_path: str, + config: PretrainedConfig, + backend: str, + **model_kwargs, + ) -> None: + if backend == "torch": + self.model: PreTrainedModel = AutoModelForSequenceClassification.from_pretrained( + model_name_or_path, + config=config, + **model_kwargs, + ) + elif backend == "onnx": + self._load_onnx_model(model_name_or_path, config, **model_kwargs) + elif backend == "openvino": + self._load_openvino_model(model_name_or_path, config, **model_kwargs) + else: + raise ValueError(f"Unsupported backend '{backend}'. `backend` should be `torch`, `onnx`, or `openvino`.") + + def _load_openvino_model(self, model_name_or_path: str, config: PretrainedConfig, **model_kwargs) -> None: + try: + from optimum.intel import OVModelForSequenceClassification + from optimum.intel.openvino import OV_XML_FILE_NAME + except ModuleNotFoundError: + raise Exception( + "Using the OpenVINO backend requires installing Optimum and OpenVINO. " + "You can install them with pip: `pip install optimum[openvino]`." + ) + + load_path = Path(model_name_or_path) + is_local = load_path.exists() + backend_name = "OpenVINO" + target_file_glob = "openvino*.xml" + + # Determine whether the model should be exported or whether we can load it directly + export, model_kwargs = self._backend_should_export( + load_path, is_local, model_kwargs, OV_XML_FILE_NAME, target_file_glob, backend_name + ) + + # If we're exporting, then there's no need for a file_name to load the model from + if export: + model_kwargs.pop("file_name", None) + + # ov_config can be either a dictionary, or point to a json file with an OpenVINO config + if "ov_config" in model_kwargs: + ov_config = model_kwargs["ov_config"] + if not isinstance(ov_config, dict): + if not Path(ov_config).exists(): + raise ValueError( + "ov_config should be a dictionary or a path to a .json file containing an OpenVINO config" + ) + with open(ov_config, encoding="utf-8") as f: + model_kwargs["ov_config"] = json.load(f) + else: + model_kwargs["ov_config"] = {} + + # Either load an exported model, or export the model to OpenVINO + self.model: OVModelForSequenceClassification = OVModelForSequenceClassification.from_pretrained( + model_name_or_path, + config=config, + export=export, + **model_kwargs, + ) + # Wrap the save_pretrained method to save the model in the correct subfolder + self.model._save_pretrained = _save_pretrained_wrapper(self.model._save_pretrained, self.backend) + + # Warn the user to save the model if they haven't already + if export: + self._backend_warn_to_save(model_name_or_path, is_local, backend_name) + + def _load_onnx_model(self, model_name_or_path: str, config: PretrainedConfig, **model_kwargs) -> None: + try: + import onnxruntime as ort + from optimum.onnxruntime import ONNX_WEIGHTS_NAME, ORTModelForSequenceClassification + except ModuleNotFoundError: + raise Exception( + "Using the ONNX backend requires installing Optimum and ONNX Runtime. " + "You can install them with pip: `pip install optimum[onnxruntime]` " + "or `pip install optimum[onnxruntime-gpu]`" + ) + + # Default to the highest priority available provider if not specified + # E.g. Tensorrt > CUDA > CPU + model_kwargs["provider"] = model_kwargs.pop("provider", ort.get_available_providers()[0]) + + load_path = Path(model_name_or_path) + is_local = load_path.exists() + backend_name = "ONNX" + target_file_glob = "*.onnx" + + # Determine whether the model should be exported or whether we can load it directly + export, model_kwargs = self._backend_should_export( + load_path, is_local, model_kwargs, ONNX_WEIGHTS_NAME, target_file_glob, backend_name + ) + + # If we're exporting, then there's no need for a file_name to load the model from + if export: + model_kwargs.pop("file_name", None) + + # Either load an exported model, or export the model to ONNX + self.model: ORTModelForSequenceClassification = ORTModelForSequenceClassification.from_pretrained( + model_name_or_path, + config=config, + export=export, + **model_kwargs, + ) + # Wrap the save_pretrained method to save the model in the correct subfolder + self.model._save_pretrained = _save_pretrained_wrapper(self.model._save_pretrained, self.backend) + + # Warn the user to save the model if they haven't already + if export: + self._backend_warn_to_save(model_name_or_path, is_local, backend_name) + + def _backend_should_export( + self, + load_path: Path, + is_local: bool, + model_kwargs: dict[str, Any], + target_file_name: str, + target_file_glob: str, + backend_name: str, + ) -> tuple[bool, dict[str, Any]]: + """ + Determines whether the model should be exported to the backend, or if it can be loaded directly. + Also update the `file_name` and `subfolder` model_args if necessary. + + These are the cases: + + 1. If export is set in model_args, just return export + 2. If `/` exists; set export to False + 3. If `/` exists; set export to False and set subfolder to the backend (e.g. "onnx") + 4. If `` contains a folder, add those folders to the subfolder and set the file_name to the last part + + We will warn if: + + 1. The expected file does not exist in the model directory given the optional file_name and subfolder. + If there are valid files for this backend, but they're don't align with file_name, then we give a useful warning. + 2. Multiple files are found in the model directory that match the target file name and the user did not + specify the desired file name via `model_kwargs={"file_name": ""}` + + Args: + load_path: The model repository or directory, as a Path instance + is_local: Whether the model is local or remote, i.e. whether load_path is a local directory + model_args: The model_args dictionary. Notable keys are "export", "file_name", and "subfolder" + target_file_name: The expected file name in the model directory, e.g. "model.onnx" or "openvino_model.xml" + target_file_glob: The glob pattern to match the target file name, e.g. "*.onnx" or "openvino*.xml" + backend_name: The human-readable name of the backend for use in warnings, e.g. "ONNX" or "OpenVINO" + + Returns: + Tuple[bool, dict[str, Any]]: A tuple of the export boolean and the updated model_args dictionary. + """ + + export = model_kwargs.pop("export", None) + if export: + return export, model_kwargs + + file_name = model_kwargs.get("file_name", target_file_name) + subfolder = model_kwargs.get("subfolder", None) + primary_full_path = Path(subfolder, file_name).as_posix() if subfolder else Path(file_name).as_posix() + secondary_full_path = ( + Path(subfolder, self.backend, file_name).as_posix() + if subfolder + else Path(self.backend, file_name).as_posix() + ) + glob_pattern = f"{subfolder}/**/{target_file_glob}" if subfolder else f"**/{target_file_glob}" + + # Get the list of files in the model directory that match the target file name + if is_local: + model_file_names = [path.relative_to(load_path).as_posix() for path in load_path.glob(glob_pattern)] + else: + all_files = huggingface_hub.list_repo_files( + load_path.as_posix(), + repo_type="model", + revision=model_kwargs.get("revision", None), + token=model_kwargs.get("token", None), + ) + model_file_names = [fname for fname in all_files if fnmatch(fname, glob_pattern)] + + # First check if the expected file exists in the root of the model directory + # If it doesn't, check if it exists in the backend subfolder. + # If it does, set the subfolder to include the backend + model_found = primary_full_path in model_file_names + if not model_found and "subfolder" not in model_kwargs: + model_found = secondary_full_path in model_file_names + if model_found: + if len(model_file_names) > 1 and "file_name" not in model_kwargs: + logger.warning( + f"Multiple {backend_name} files found in {load_path.as_posix()!r}: {model_file_names}, defaulting to {secondary_full_path!r}. " + f'Please specify the desired file name via `model_kwargs={{"file_name": ""}}`.' + ) + model_kwargs["subfolder"] = self.backend + model_kwargs["file_name"] = file_name + if export is None: + export = not model_found + + # If the file_name contains subfolders, set it as the subfolder instead + file_name_parts = Path(file_name).parts + if len(file_name_parts) > 1: + model_kwargs["file_name"] = file_name_parts[-1] + model_kwargs["subfolder"] = Path(model_kwargs.get("subfolder", ""), *file_name_parts[:-1]).as_posix() + + if export: + logger.warning( + f"No {file_name!r} found in {load_path.as_posix()!r}. Exporting the model to {backend_name}." + ) + + if model_file_names: + logger.warning( + f"If you intended to load one of the {model_file_names} {backend_name} files, " + f'please specify the desired file name via `model_kwargs={{"file_name": "{model_file_names[0]}"}}`.' + ) + + return export, model_kwargs + + def _backend_warn_to_save(self, model_name_or_path: str, is_local: str, backend_name: str) -> None: + to_log = f"Saving the exported {backend_name} model is heavily recommended to avoid having to export it again." + if is_local: + to_log += f" Do so with `model.save_pretrained({model_name_or_path!r})`." + else: + to_log += f" Do so with `model.push_to_hub({model_name_or_path!r}, create_pr=True)`." + logger.warning(to_log) + def set_activation_fn(self, activation_fn: Callable | None, set_default: bool = True) -> None: if activation_fn is not None: self.activation_fn = activation_fn @@ -396,7 +637,7 @@ def predict( self.set_activation_fn(activation_fn, set_default=False) pred_scores = [] - self.model.eval() + self.eval() for start_index in trange(0, len(sentences), batch_size, desc="Batches", disable=not show_progress_bar): batch = sentences[start_index : start_index + batch_size] features = self.tokenizer( From cc1cbe9f5cdb43fec5b835e3dd2fc04367be43ab Mon Sep 17 00:00:00 2001 From: Tom Aarsen Date: Fri, 11 Apr 2025 09:58:06 +0200 Subject: [PATCH 2/5] Remove accidental leftover breakpoint --- sentence_transformers/backend.py | 1 - 1 file changed, 1 deletion(-) diff --git a/sentence_transformers/backend.py b/sentence_transformers/backend.py index 1825e167f..a9b3bf2a4 100644 --- a/sentence_transformers/backend.py +++ b/sentence_transformers/backend.py @@ -460,7 +460,6 @@ def save_or_push_to_hub_model( ``` """ - breakpoint() huggingface_hub.upload_folder( folder_path=save_dir, path_in_repo=backend, From 95c5a79a36697dceaa51d14a518b652987952491 Mon Sep 17 00:00:00 2001 From: Tom Aarsen Date: Tue, 15 Apr 2025 14:15:14 +0200 Subject: [PATCH 3/5] Improve typing for tokenizer --- sentence_transformers/cross_encoder/CrossEncoder.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sentence_transformers/cross_encoder/CrossEncoder.py b/sentence_transformers/cross_encoder/CrossEncoder.py index 05fdf5872..5c761b2f3 100644 --- a/sentence_transformers/cross_encoder/CrossEncoder.py +++ b/sentence_transformers/cross_encoder/CrossEncoder.py @@ -22,6 +22,7 @@ AutoTokenizer, PretrainedConfig, PreTrainedModel, + PreTrainedTokenizer, ) from transformers.utils import PushToHubMixin from typing_extensions import deprecated @@ -188,7 +189,7 @@ def __init__( if "model_max_length" not in tokenizer_kwargs and max_length is not None: tokenizer_kwargs["model_max_length"] = max_length - self.tokenizer = AutoTokenizer.from_pretrained( + self.tokenizer: PreTrainedTokenizer = AutoTokenizer.from_pretrained( model_name_or_path, cache_dir=cache_folder, trust_remote_code=trust_remote_code, From 2027bc965b078bdd18c1d0fc574aa808cabda7fd Mon Sep 17 00:00:00 2001 From: Tom Aarsen Date: Tue, 15 Apr 2025 14:15:33 +0200 Subject: [PATCH 4/5] Improve docs for save_pretrained --- sentence_transformers/cross_encoder/CrossEncoder.py | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/sentence_transformers/cross_encoder/CrossEncoder.py b/sentence_transformers/cross_encoder/CrossEncoder.py index 5c761b2f3..071dcfffa 100644 --- a/sentence_transformers/cross_encoder/CrossEncoder.py +++ b/sentence_transformers/cross_encoder/CrossEncoder.py @@ -776,7 +776,16 @@ def save(self, path: str, *, safe_serialization: bool = True, **kwargs) -> None: def save_pretrained(self, path: str, *, safe_serialization: bool = True, **kwargs) -> None: """ - Saves the model and tokenizer to path; identical to `save` + Save the model and tokenizer to the specified path. + + Args: + path (str): Directory where the model should be saved + safe_serialization (bool, optional): Whether to save using `safetensors` or the traditional + PyTorch way. Defaults to True. + **kwargs: Additional arguments passed to the underlying save methods of the model and tokenizer. + + Returns: + None """ return self.save(path, safe_serialization=safe_serialization, **kwargs) From 556b8d3ce15dd94a58688a63baf6987f99db3dac Mon Sep 17 00:00:00 2001 From: Tom Aarsen Date: Tue, 15 Apr 2025 14:15:47 +0200 Subject: [PATCH 5/5] Apply minor improvements/fixes to efficiency docs --- docs/cross_encoder/usage/efficiency.rst | 18 +++++++++--------- docs/sentence_transformer/usage/efficiency.rst | 10 +++++----- 2 files changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/cross_encoder/usage/efficiency.rst b/docs/cross_encoder/usage/efficiency.rst index 7e13de733..edda1b768 100644 --- a/docs/cross_encoder/usage/efficiency.rst +++ b/docs/cross_encoder/usage/efficiency.rst @@ -137,7 +137,7 @@ If the model path or repository already contains a model in ONNX format, Sentenc If you wish to use the ONNX model outside of Sentence Transformers, you might need to apply your chosen activation function (e.g. Sigmoid) to get identical results as the Cross Encoder in Sentence Transformers. -All keyword arguments passed via ``model_kwargs`` will be passed on to :meth:`ORTModelForSequenceClassification.from_pretrained `. Some notable arguments include: +All keyword arguments passed via ``model_kwargs`` will be passed on to :meth:`ORTModelForSequenceClassification.from_pretrained `. Some notable arguments include: * ``provider``: ONNX Runtime provider to use for loading the model, e.g. ``"CPUExecutionProvider"`` . See https://onnxruntime.ai/docs/execution-providers/ for possible providers. If not specified, the strongest provider (E.g. ``"CUDAExecutionProvider"``) will be used. * ``file_name``: The name of the ONNX file to load. If not specified, will default to ``"model.onnx"`` or otherwise ``"onnx/model.onnx"``. This argument is useful for specifying optimized or quantized models. @@ -164,7 +164,7 @@ Optimizing ONNX Models .. include:: backend_export_sidebar.rst -ONNX models can be optimized using Optimum, allowing for speedups on CPUs and GPUs alike. To do this, you can use the :func:`~sentence_transformers.backend.export_optimized_onnx_model` function, which saves the optimized in a directory or model repository that you specify. It expects: +ONNX models can be optimized using `Optimum `_, allowing for speedups on CPUs and GPUs alike. To do this, you can use the :func:`~sentence_transformers.backend.export_optimized_onnx_model` function, which saves the optimized in a directory or model repository that you specify. It expects: - ``model``: a Sentence Transformer or Cross Encoder model loaded with the ONNX backend. - ``optimization_config``: ``"O1"``, ``"O2"``, ``"O3"``, or ``"O4"`` representing optimization levels from :class:`~optimum.onnxruntime.AutoOptimizationConfig`, or an :class:`~optimum.onnxruntime.OptimizationConfig` instance. @@ -236,7 +236,7 @@ Quantizing ONNX Models .. include:: backend_export_sidebar.rst -ONNX models can be quantized to int8 precision using Optimum, allowing for faster inference on CPUs. To do this, you can use the :func:`~sentence_transformers.backend.export_dynamic_quantized_onnx_model` function, which saves the quantized in a directory or model repository that you specify. Dynamic quantization, unlike static quantization, does not require a calibration dataset. It expects: +ONNX models can be quantized to int8 precision using `Optimum `_, allowing for faster inference on CPUs. To do this, you can use the :func:`~sentence_transformers.backend.export_dynamic_quantized_onnx_model` function, which saves the quantized in a directory or model repository that you specify. Dynamic quantization, unlike static quantization, does not require a calibration dataset. It expects: - ``model``: a Sentence Transformer or Cross Encoder model loaded with the ONNX backend. - ``quantization_config``: ``"arm64"``, ``"avx2"``, ``"avx512"``, or ``"avx512_vnni"`` representing quantization configurations from :class:`~optimum.onnxruntime.AutoQuantizationConfig`, or an :class:`~optimum.onnxruntime.QuantizationConfig` instance. @@ -369,15 +369,15 @@ Quantizing OpenVINO Models .. include:: backend_export_sidebar.rst -OpenVINO models can be quantized to int8 precision using Optimum Intel to speed up inference. +OpenVINO models can be quantized to int8 precision using `Optimum Intel `_ to speed up inference. To do this, you can use the :func:`~sentence_transformers.backend.export_static_quantized_openvino_model` function, which saves the quantized model in a directory or model repository that you specify. Post-Training Static Quantization expects: - ``model``: a Sentence Transformer or Cross Encoder model loaded with the OpenVINO backend. - ``quantization_config``: (Optional) The quantization configuration. This parameter accepts either: - ``None`` for the default 8-bit quantization, a dictionary representing quantization configurations, or - an :class:`~optimum.intel.OVQuantizationConfig` instance. + ``None`` for the default 8-bit quantization, a dictionary representing quantization configurations, or + an :class:`~optimum.intel.OVQuantizationConfig` instance. - ``model_name_or_path``: a path to save the quantized model file, or the repository name if you want to push it to the Hugging Face Hub. - ``dataset_name``: (Optional) The name of the dataset to load for calibration. If not specified, defaults to ``sst2`` subset from the ``glue`` dataset. - ``dataset_config_name``: (Optional) The specific configuration of the dataset to load. @@ -467,13 +467,13 @@ The following images show the benchmark results for the different backends on GP Datasets: 2000 samples for GPU tests, 1000 samples for CPU tests.
  • - sentence-transformers/stsb: ``sentence1`` and ``sentence2`` columns as pairs, with 38.94 ± 13.97 and 38.96 ± 14.05 characters on average, respectively. + sentence-transformers/stsb: sentence1 and sentence2 columns as pairs, with 38.94 ± 13.97 and 38.96 ± 14.05 characters on average, respectively.
  • - sentence-transformers/natural-questions: ``query`` and ``answer`` columns as pairs, with 46.99 ± 10.98 and 619.63 ± 345.30 characters on average, respectively. + sentence-transformers/natural-questions: query and answer columns as pairs, with 46.99 ± 10.98 and 619.63 ± 345.30 characters on average, respectively.
  • - stanfordnlp/imdb: Two variants used from the ``text`` column: first 100 characters (100.00 ± 0.00 characters) and each sample repeated 4 times (16804.25 ± 10178.26 characters). + stanfordnlp/imdb: Two variants used from the text column: first 100 characters (100.00 ± 0.00 characters) and each sample repeated 4 times (16804.25 ± 10178.26 characters).
diff --git a/docs/sentence_transformer/usage/efficiency.rst b/docs/sentence_transformer/usage/efficiency.rst index bdaf881dc..59be87388 100644 --- a/docs/sentence_transformer/usage/efficiency.rst +++ b/docs/sentence_transformer/usage/efficiency.rst @@ -132,7 +132,7 @@ Optimizing ONNX Models .. include:: backend_export_sidebar.rst -ONNX models can be optimized using Optimum, allowing for speedups on CPUs and GPUs alike. To do this, you can use the :func:`~sentence_transformers.backend.export_optimized_onnx_model` function, which saves the optimized in a directory or model repository that you specify. It expects: +ONNX models can be optimized using `Optimum `_, allowing for speedups on CPUs and GPUs alike. To do this, you can use the :func:`~sentence_transformers.backend.export_optimized_onnx_model` function, which saves the optimized in a directory or model repository that you specify. It expects: - ``model``: a Sentence Transformer or Cross Encoder model loaded with the ONNX backend. - ``optimization_config``: ``"O1"``, ``"O2"``, ``"O3"``, or ``"O4"`` representing optimization levels from :class:`~optimum.onnxruntime.AutoOptimizationConfig`, or an :class:`~optimum.onnxruntime.OptimizationConfig` instance. @@ -204,7 +204,7 @@ Quantizing ONNX Models .. include:: backend_export_sidebar.rst -ONNX models can be quantized to int8 precision using Optimum, allowing for faster inference on CPUs. To do this, you can use the :func:`~sentence_transformers.backend.export_dynamic_quantized_onnx_model` function, which saves the quantized in a directory or model repository that you specify. Dynamic quantization, unlike static quantization, does not require a calibration dataset. It expects: +ONNX models can be quantized to int8 precision using `Optimum `_, allowing for faster inference on CPUs. To do this, you can use the :func:`~sentence_transformers.backend.export_dynamic_quantized_onnx_model` function, which saves the quantized in a directory or model repository that you specify. Dynamic quantization, unlike static quantization, does not require a calibration dataset. It expects: - ``model``: a Sentence Transformer or Cross Encoder model loaded with the ONNX backend. - ``quantization_config``: ``"arm64"``, ``"avx2"``, ``"avx512"``, or ``"avx512_vnni"`` representing quantization configurations from :class:`~optimum.onnxruntime.AutoQuantizationConfig`, or an :class:`~optimum.onnxruntime.QuantizationConfig` instance. @@ -329,15 +329,15 @@ Quantizing OpenVINO Models .. include:: backend_export_sidebar.rst -OpenVINO models can be quantized to int8 precision using Optimum Intel to speed up inference. +OpenVINO models can be quantized to int8 precision using `Optimum Intel `_ to speed up inference. To do this, you can use the :func:`~sentence_transformers.backend.export_static_quantized_openvino_model` function, which saves the quantized model in a directory or model repository that you specify. Post-Training Static Quantization expects: - ``model``: a Sentence Transformer or Cross Encoder model loaded with the OpenVINO backend. - ``quantization_config``: (Optional) The quantization configuration. This parameter accepts either: - ``None`` for the default 8-bit quantization, a dictionary representing quantization configurations, or - an :class:`~optimum.intel.OVQuantizationConfig` instance. + ``None`` for the default 8-bit quantization, a dictionary representing quantization configurations, or + an :class:`~optimum.intel.OVQuantizationConfig` instance. - ``model_name_or_path``: a path to save the quantized model file, or the repository name if you want to push it to the Hugging Face Hub. - ``dataset_name``: (Optional) The name of the dataset to load for calibration. If not specified, defaults to ``sst2`` subset from the ``glue`` dataset. - ``dataset_config_name``: (Optional) The specific configuration of the dataset to load.