I’m new to Python and come from the write-everything-yourself world of PHP (at least this is how I always approached it).
I’m using Flask, WTForms, Jinja2, and I’ve just discovered Flask-Login which I want to use. My question is about the reliability of using thirdparty libraries for core functionality in a project that is planned to be around for several years.
I’ve installed these libraries (via pip) into a virtualenv environment. What happens if these libraries stop being distributed? Should I back up these libraries (are they eggs)? Can I store these libraries in my project itself, instead of relying on pip to install them in a virtualenv? And should I store these separately?
I’m worried that I’ll rely on a library for core functionality, and then one day I’ll download an incompatible version through pip, or the author or maintainer will stop distributing it and it’ll no longer be available.
How can I protect against this, and ensure that any thirdparty libraries that I use in my projects will always be available as they are now?
I’ve been deploying python web applications for years now, and this has rarely been a problem.
Note that what you describe and fear could happen to any open source project, including to PHP, Apache, or even Linux. What happens if any OSS project goes AWOL? If there has been any number of people relying on it, backups of distributions can always be found somewhere on the world wide internet.
Also note that if you use a reliable, reproduceable distribution method, where you pin dependency versions to one version only, you should never run into the accidental ‘incompatible version’ scenario you describe. pip has the freeze command to give you a re-usable list of versions, for example.
Using a tool like buildout gives you another avenue to build a fully re-playable deployment configuration, and not only can you pin your library versions with it, you can instruct buildout to never install a python package unless you configured an explicit version for that package (allow-picked-versions = false). Buildout caches downloaded package distribution files, giving me easy access to those backups I mentioned earlier.
Last, but not least, you create such a backup cache automatically with a sufficiently smart, local PyPI mirror such as collective.eggproxy. Both PyPI and buildout can be instructed to use your mirror instead of the central PyPI repository, and it’ll build up a cache of packages as you go along, fully automatic.
3