Running a Python app using Poetry or Pipenv

etienneh

Hiya, I'm a bit new to uWSGI and running a python app with it.
My desired result:

Use a version of python I control
Install predictable dependencies using poetry or pipenv
Not have to worry about constantly monitoring/restarting my application (i.e. starting a screen session, running the process manually, instead relying on the crontab entry auto-generated by opalstack)

My attempted steps:

Install pyenv via basic github checkout ✅
Install pipenv, install dependencies into a virtualenv location (e.g. /home/<opalstackuser>/.local/share/virtualenvs/<appname>-oaU6hgWW) ✅
Configure uWSGI via:
-- set virtualenv line virtualenv = /home/<opalstackuser>/.local/share/virtualenvs/<appname>-oaU6hgWW/
-- set module based on my flask app (I have main.py which contains the if __name__ == 'main.py, which I gather is unused) to <appname>.main:app
-- set touch-reload as per instructions here: https://help.opalstack.com/article/60/pythonuwsgi-applications
Run ./stop && ./start

What I see in my ~/logs/<opalstack_appname>/uwsgi.log is:

uWSGI http bound on 127.0.0.1:<port> fd 4
uwsgi socket 0 bound to TCP address 127.0.0.1:43212 (port auto-assigned) fd 3
Python version: 3.6.8 (default, Apr  2 2020, 13:34:55)  [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
PEP 405 virtualenv detected: /home/<opalstackuser>/.local/share/virtualenvs/<appname>-oaU6hgWW/
Set PythonHome to /home/<opalstackuser>/.local/share/virtualenvs/<appname>-oaU6hgWW/
Python main interpreter initialized at 0xfd03c0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 250128 bytes (244 KB) for 4 cores
*** Operational MODE: preforking+threaded ***
Traceback (most recent call last):
  File "./<appname>/main.py", line 2, in <module>
    from <dependency> import <RequiredClass>
ModuleNotFoundError: No module named '<dependency>'
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 29587)
spawned uWSGI worker 1 (pid: 29588, cores: 2)
spawned uWSGI worker 2 (pid: 29589, cores: 2)
spawned uWSGI http 1 (pid: 29590)

Looking at this, it doesn't look like it's using my user's python is being used (not surprising, I haven't configured it and don't know where to), nor that it's using the correct dependencies? e.g. I can do ls /home/<opalstackuser>/.local/share/virtualenvs/<appname>-oaU6hgWW/lib/python3.8/site-packages/ and see <dependency> as a folder in there.

Any tips on how to configure things from here?
I suppose I could just DIY an app listening on <port> but would prefer to stay close to the default Opalstack app setup.

etienneh

Update (took a break, came back to it):

I've now done the following:

Stopped the running application
Added uwsgi to my pipenv dependencies
Updated the start and stop scripts to reference the /bin/uwsgi within my virtualenv
restarted the application

It now looks like my version of python is being used. Just getting 404 on my main route now, so progress?

sean

I was about to mention that the uwsgi binary is built in the app's preconfigured virtualenv so there might be a problem with pointing at some other virtualenv. Another possible fix may have been to your environment's library directory via a pythonpath directive in the uwsgi config.

Regarding the 404, are you running the uwsgi app from the root of the site, or from a sub-URI?

etienneh

sean I'm not running it from the root path, so my site route is <mydomain>/somePath/

sean

etienneh ok, please try the steps shown in our docs at: Serving a Django app on a sub-path

The article was written for Django but is probably applicable to most uwsgi apps.

Let me know how that goes, thanks!

etienneh

Alright so in the end, it worked! Thanks for the pointer sean

Summarizing this for anyone else.
Full steps:

Install pyenv via basic github checkout
Ensure uwsgi is a pipenv/poetry dependency
Install pipenv, install dependencies into a virtualenv location (defaulted to /home/<opalstackuser>/.local/share/virtualenvs/<appname>-oaU6hgWW)
run the stop script
Configure uWSGI via editing uwsgi.ini --> change the virtualenv line virtualenv = /home/<opalstackuser>/apps/funkplanet/env/ to /home/<opalstackuser>/.local/share/virtualenvs/<appname>-oaU6hgWW
Adjust the start and stop scripts to refer to /home/<opalstackuser>/.local/share/virtualenvs/<appname>-oaU6hgWW/bin/uwsgi instead of the original path to /bin/wsgi
Start again via start

The two further errors I encountered were:

I was using python-dotenv to load my .env environment values. Because the uwsgi process started in the parent directory, I had to move my .env file next to uwsgi.ini instead of within the module
I was serving the flask app from a path other than http(s)://domain.com/ instead https://domain.com/<somepath> so I had to further adjust my uwsgi.ini file to instead change:
wsgi-file = /home/<opalstackuser>/apps/<opalstack_appname>/<modulename>/wsgi.py
to
mount = /<somepath>=/home/<opalstackuser>/apps/<opalstack_appname>/<modulename>/file_containing_flask_app.py

and also add at the bottom:
callable = app
manage-script-name = true

I followed these instructions around mount points:
https://uwsgi-docs.readthedocs.io/en/latest/Snippets.html#multiple-flask-apps-in-different-mountpoints