Importing external package in custom query module

I want to use an external package in a custom query module. The module function should be called like this:

CALL rasa_nlu.run(“Some text input here”) YIELD intents, entities

This requires me to import rasa in /usr/lib/memgraph/query_mod/rasa_nlu.py
But when I import rasa, the function becomes unavailable. My boss implemented this on another memgraph database, but he is currently on vacation, so I hope someone here can help me.

1 Like

Hello @tordahl! :grinning_face_with_smiling_eyes:

Did you maybe receive a specific error message associated with this problem?
It would also help us if you included any relevant log entries, which can be found in /var/log/memgraph.

I would also like to check if the path you provided /usr/lib/memgraph/query_mod/rasa_nlu.py is indeed the correct location for query modules. By default, Memgraph searches for query modules in
the directory /usr/lib/memgraph/query_modules and another directory needs to be specified with the --query-modules-directory flag either when starting Memgraph or in the configuration file /etc/memgraph/memgraph.conf. You can find out more about configuration options in the reference guide.

I will take you through my problem step by step. I’m on the newest version of Ubuntu and also using the newest version of Memgraph and Memgraph lab.

First I have the following code in /usr/lib/memgraph/query_mod/rasa_nlu.py

import mgp

@mgp.read_proc
def run(text: str) → mgp.Record(intents=list, entities=list):

intent_list = []
entity_list = []
message = str(text).strip()

intent_list.append('test')
entity_list.append('test')

return mgp.Record(intents=intent_list, entities=entity_list)

Calling this with

CALL rasa_nlu.run(‘This is a test’) YIELD * RETURN *;

works as expected.

Now, if I add the following line to the python script:

from rasa.nlu.model import Interpreter

I get the following error message in Memgraph lab when executing the same call as above:

Error: Query failed: There is no procedure named ‘rasa_nlu.run’.

Have you checked /var/log/memgraph/?
Is there an error message there?

Yes, apologies. The following error message was in the log:

[2021-07-30 00:15:44.710] [memgraph_log] [error] Unable to load module “/usr/lib/memgraph/query_modules/rasa_nlu.py”; Traceback (most recent call last):
File “/usr/lib/memgraph/query_modules/rasa_nlu.py”, line 2, in
from rasa.nlu.model import Interpreter
ModuleNotFoundError: No module named ‘rasa’

However, the import works otherwise, but not when I run it with Memgraph.

Interesting, I managed to reproduce your problem with another python module.
Short term, I solved it by restarting Memgraph.

Currently, it seems Memgraph loads python modules when it is run, so you need to restart it whenever you add a new module. That’s my hypothesis, at least.

Can you try restarting Memgraph. Does the problem persist?

Also, can you tell me the output of python3 --version and python3 -c 'import rasa'?

Thank for your reply. Yes, I have tried to restart Memgraph and the problem persists.

My Python version is Python 3.8.10. Also, python3 -c ‘import rasa’ does not ouput anything, so I guess the import works as expected here.

It seems to me that Python modules run through Memgraph don’t get imports from the ordinary sys.path directories. I am not sure how to proceed with this.

I have a few ideas; using importlib.machinery.SourceFileLoader to hot load the module manually, or updating the python path.

Apart from that, the issue could be somewhere in the rasa module. It has a million dependencies, it’s not easy to find what could go wrong within it.

Can we get on a Google meet call? Would any of the following slots work for you?

2.8. Monday 12 to 13 CET
3.8. Tuesday 12 to 13 CET
4.8. Wednesday 12 to 13 CET

I get the same error when trying to import numpy, so may be you can check if you can reproduce the error that way.

Perhaps on wednesday. I’ll get back to you on that.

In the mean time try these:

  1. instead of

from rasa.nlu.model import Interpreter
try
from rasa_nlu.model import Interpreter

  1. Try importing the module manually specifying the path: python - Using newly installed modules without restarting an interactive session - Stack Overflow

  2. try manually changing the $PYTHONPATH environment variable. Maybe you’re installing packages in /home/user/.local/lib/python3.8/... and the memgraph user is trying to find them in /usr/local/lib/python3.7/...

  3. try installing packages with pip globally

  4. try reproducing the issue in a clean docker container (I can’t reproduce it, especially not with numpy, that works out of the box)