In my Django app I had a bit code that would check if a template with some file name exists:

from django.template.loader import get_template


def has_help_page(name):
    """Return whether a help page for the given name exists."""
    try:
        get_template(f"help/{name.lower()}.html")
    except TemplateDoesNotExist:
        return False
    return True

The name argument is derived from a Django model’s verbose name, and some models have verbose names that contain non-ascii characters (f.ex. “Broschüre”).

When running locally or in a Docker container (via mod_wsgi), this works absolutely fine. But when I let Apache serve the app from a Debian machine (Apache/2.4.56 (Debian)), and when I request a page that would call the above function with a name that contains an Umlaut I get an UnicodeEncodeError:

Traceback (most recent call last):
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/contrib/auth/mixins.py", line 109, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/views/generic/base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/views/generic/list.py", line 174, in get
    context = self.get_context_data()
  File "/srv/archiv/MIZDB/dbentry/site/views/base.py", line 1127, in get_context_data
    ctx = super().get_context_data(**kwargs)
  File "/srv/archiv/MIZDB/dbentry/search/mixins.py", line 62, in get_context_data
    ctx = super().get_context_data(**kwargs)  # type: ignore[misc]
  File "/srv/archiv/MIZDB/dbentry/site/views/base.py", line 783, in get_context_data
    ctx = super().get_context_data(**kwargs)
  File "/srv/archiv/MIZDB/dbentry/site/views/base.py", line 126, in get_context_data
    if has_help_page(self.opts.verbose_name.lower()):
  File "/srv/archiv/MIZDB/dbentry/site/views/help.py", line 15, in has_help_page
    get_template(f"help/{name.lower()}.html")
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/template/loader.py", line 15, in get_template
    return engine.get_template(template_name)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/template/backends/django.py", line 33, in get_template
    return Template(self.engine.get_template(template_name), self)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/template/engine.py", line 175, in get_template
    template, origin = self.find_template(template_name)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/template/engine.py", line 157, in find_template
    template = loader.get_template(name, skip=skip)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/template/loaders/cached.py", line 57, in get_template
    template = super().get_template(template_name, skip)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/template/loaders/base.py", line 23, in get_template
    contents = self.get_contents(origin)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/template/loaders/cached.py", line 26, in get_contents
    return origin.loader.get_contents(origin)
  File "/srv/archiv/MIZDB/.venv/lib/python3.9/site-packages/django/template/loaders/filesystem.py", line 22, in get_contents
    with open(origin.name, encoding=self.engine.file_charset) as fp:

Exception Type: UnicodeEncodeError at /brochure/
Exception Value: 'ascii' codec can't encode character 'xfc' in position 39: ordinal not in range(128)

Where origin.name is:

'/srv/archiv/MIZDB/templates/help/broschüre.html'

Note that it crashes when trying to open the file, not when attempting to read it. The file, as such, also doesn’t exist; the name of the real, existing template does not contain non-ascii characters – it’s just the function making a guess with a probable template name based on a model.

I have since patched the function and got it all working again, but I am unsure what the real issue was here.

My questions are:

1. Why is this only an issue when serving the app via Apache directly?
2. Is there some kind of setting that I am missing? (I tried setting env vars and the default charset to no avail)
3. Why is Apache trying to use “ascii” when the default and filesystem encoding is “utf-8”?

Apache site conf:

<Macro VHost $VENV_ROOT $PROJECT_ROOT>
        <VirtualHost *:80>
                ServerName localhost

                WSGIScriptAlias /miz $PROJECT_ROOT/MIZDB/wsgi.py
                WSGIDaemonProcess mizdb python-home=$VENV_ROOT python-path=$PROJECT_ROOT
                WSGIProcessGroup mizdb

                Alias /static $PROJECT_ROOT/static
                <Directory $PROJECT_ROOT/static>
                        Require all granted
                </Directory>

                <Directory $PROJECT_ROOT/MIZDB>
                        <Files wsgi.py>
                                Require all granted
                        </Files>
                </Directory>

        </VirtualHost>
</Macro>

USE VHost /srv/archiv/MIZDB/.venv /srv/archiv/MIZDB
UndefMacro VHost