How to create your own Question and Answering API(Flask+Docker +BERT) using haystack framework — Part I

Note from author :

In this article we will learn how to create your own Question and Answering(QA) API using python, flask and haystack framework with docker. The haystack framework will provide the complete QA features which is highly scalable and customizable. In this article Medium Rules text will be used as target document and fine tuning the model as well.

Basic Knowledge Required: Elasticsearch & Docker

This article contain the working code which can be directly build using docker.

Content of the article

1. DocumentStores set up — Installation of Elasticsearch
2. API Code Explanation
3. Build the Flask API in docker
4. Load data from the API
5. Demonstration of the API
6. Fine tuning a model
7. Code tuning (Trained/Pre-Trained)
8. Summarize Result

1. DocumentStores set up

Haystack rise queries to the document which is available at DocumentStore. There are various DocumentStore included in haystack which are ElasticsearchDocumentStore, SQLDocumentStore, and InMemoryDocumentStore. In this article I am going to use Elasticsearch which is recommended. It comes up with preloaded features like full-text queries, BM25 retrieval, and vector storage for text embeddings.

haystack- component

Run the below command for the installation of Elasticsearch

verify the installation by browsing the below url. If install successfully it will display as in below:

Elasticsearch Server started

2. API Code Explanation

Haystack framework has main 3 basic component DocumentStore, Retriever and Reader which we have to select based on our requirements.

DocumentStore : As recommended ElasticsearchDocumentStore will be used in this article. It comes preloaded with features like full-text queries, BM25 retrieval, and vector storage for text embeddings. Documents should be chunked into smaller units (e.g. paragraphs) before indexing to make the results returned by the Retriever more granular and accurate.

Retrievers: The answer need to be display based on similarities on the embedded. So DensePassageRetriever is the powerful alternative to score similarity of texts.

Readers: Farm Reader going to be used in this articles if you wish Transformer readers can be use. For better accuracy “distilbert-base-uncased-distilled-squad” model going to be used by the readers

The app structure as shown below:

Code Explanation for main.py

In this article we are going to use ElasticSearch document . The application configuration are declared as in below:

Lets implement endpoints for uploading the PDF document. The pdf document will be uploaded in the ElasticSearch with the provided index name.

Lets implement endpoints for querying and will response back with relevant (n)answers from the ElasticSearch document. The index need to provided during the search query.

requirement.txt

Flask
gunicorn
futures
farm-haystack

Docker File

The QNA API will be available @port 8777.

4. Build the Flask API in docker

Lets run the below command to build the docker image:

docker build -t qna:v1 .

Build and run the flask API in docker container using the below command:

docker run — name qna_app -d -p 8777:8777 xxxxxxxxx

Note : xxxxxxxxx is the image id

Confirm the docker container using the below command:

docker ps

Its will show all the process as in below:

Now the QNA API is successfully running @ http://localhost:8777

4. Load data from the API

Lets prepare pdf document from Medium Rules which suppose to be upload in elastic search document. Once you prepare the pdf document lets upload using the API as in below snapshot:

verify the upload document as in below snapshot:

5. Demonstration of the API

Lets ask a question using the qna_pretrain endpoints. Currently we have used only pre train model “distilbert-base-uncased-distilled-squad” which is provided good accuracy. In the next article I will demonstrate on how to annotate and improve the model. For querying a question use the API as in below snapshot:

How to create your own Question and Answering API(Flask+Docker +BERT) using haystack framework — Part II

The complete source code is available here

References

1. https://github.com/deepset-ai/haystack
2. https://github.com/deepset-ai/haystack/tree/master/annotation_tool
3. https://github.com/deepset-ai/haystack/blob/master/tutorials/Tutorial2_Finetune_a_model_on_your_data.ipynb

If this post was helpful, please click the clap 👏 button below a few times to show your support! ⬇⬇